git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/3] index-pack: no blobs during outgoing link check

From
Junio C Hamano <gitster@pobox.com>
Date
Dec 3, 2024, 22:16 UTC
Message-ID
<xmqq1pyocszr.fsf@gitster.g>
In-Reply-To
<Z06ejDgTnC6gWXgx@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 14 quoted lines
>> The benefit of doing this is as above (fetch speedup), but the drawback
>> is that if the packfile to be indexed references a local blob directly
>> (that is, not through a local tree), that local blob is in danger of
>> being garbage collected. Such a situation may arise if we push local
>> commits, including one with a change to a blob in the root tree,
>> and then the server incorporates them into its main branch through a
>> "rebase" or "squash" merge strategy, and then we fetch the new main
>> branch from the server.
>
> Okay, so we know that we are basically doing the wrong thing with the
> optimization, but by skipping blobs we can get a significant speedup and
> the failure mode is that we will re-fetch the object in a later step.
> And because we think the situation is rare it shouldn't be a huge issue
> in practice.

That is how I read it, but the description may want to make the pros and cons more explicit. One of the reasons why the users choose to use lazy clone is to avoid grabbing large blobs until they need them, so I am not sure how they feel about having to discard and then fetch again after creating a potentially large blob. As long as it does not happen repeatedly for the same blob, it probably is OK.

Show 8 quoted lines
> Without the context of the commit message this code snippet likely would
> not make any sense to a reader. The "correct" logic would be to record
> all objects, regardless of whether they are an object ID or not. But we
> explicitly choose not to as a tradeoff between performance and
> correctness.
>
> All to say that we should have a comment here that explains what is
> going on.
Thanks for reviewing and commenting.
Previous: Jonathan TanNext: Jonathan Tan
Message 7 of 29 in “Performance improvements for repacking non-promisor objects”
  1. 0/3 Performance improvements for repacking non-promisor objectsJonathan Tan, Dec 2, 2024
  2. 1/3 index-pack: dedup first during outgoing link checkJonathan Tan, Dec 2, 2024
  3. Josh SteadmonDec 2, 2024
  4. 2/3 index-pack: no blobs during outgoing link checkJonathan Tan, Dec 2, 2024
  5. Patrick SteinhardtDec 3, 2024
  6. Jonathan TanDec 3, 2024
  7. Junio C HamanoDec 3, 2024
  8. 3/3 index-pack: commit tree during outgoing link checkJonathan Tan, Dec 2, 2024
  9. Junio C HamanoDec 3, 2024
  10. Jonathan TanDec 3, 2024
  11. Junio C HamanoDec 4, 2024
  12. Jonathan TanDec 9, 2024
  13. Junio C HamanoDec 9, 2024
  14. Josh SteadmonDec 2, 2024
  15. Junio C HamanoDec 3, 2024
  16. Junio C HamanoDec 3, 2024
  17. Junio C HamanoDec 3, 2024
  18. Junio C HamanoDec 3, 2024
  19. 0/3 Performance improvements for repacking non-promisor objectsJonathan Tan, Dec 3, 2024
  20. 1/3 index-pack --promisor: dedup before checking linksJonathan Tan, Dec 3, 2024
  21. 2/3 index-pack --promisor: don't check blobsJonathan Tan, Dec 3, 2024
  22. 3/3 index-pack --promisor: also check commits' treesJonathan Tan, Dec 3, 2024
  23. 0/3 Performance improvements for repacking non-promisor objectsJonathan Tan, Dec 3, 2024
  24. 1/3 index-pack --promisor: dedup before checking linksJonathan Tan, Dec 3, 2024
  25. Junio C HamanoDec 4, 2024
  26. 2/3 index-pack --promisor: don't check blobsJonathan Tan, Dec 3, 2024
  27. 3/3 index-pack --promisor: also check commits' treesJonathan Tan, Dec 3, 2024
  28. Junio C HamanoDec 4, 2024
  29. 4/3 index-pack: work around false positive use of uninitialized variableJunio C Hamano, Dec 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.