git/list[1] front-page[2] threads[3] people[4] search[5] about
wed 2026-10-07 16:50 UTC

Re: [PATCH 2/4] pack-objects: ensure tree/tag closure with '--stdin-packs=follow'

From
Taylor Blau <ttaylorr@openai.com>
Date
Oct 2, 2026, 00:51 UTC
Message-ID
<ar8AJLYZVb6sCIO-@com-79390>
In-Reply-To
<CABPp-BHE662t9aaNcZ4DZ+2AU_C7jR7_VyHZwt2Tm8JSPE3JZw@mail.gmail.com>
On Thu, Oct 01, 2026 at 04:22:11PM -0700, Elijah Newren wrote:
Show 10 quoted lines
> With the oid_array and parent-first pack order, the blobs are visited
> as sub/a and sub/b, so sub/* -delta applies. With the oidset, the
> subtree is visited first and the blobs are seen as a and b, so one is
> delta-compressed. When the root is processed later, the subtree is
> already marked SEEN and is not revisited with the sub/ prefix.
>
> The oid_array does not manufacture parent-before-child ordering if the
> input pack itself has the subtree first; this path information is
> explicitly best-effort. But it preserves a useful order when one
> exists, whereas an oidset discards it.

Sure, though as Peff and I discussed elsewhere in the thread, there are also situations where you can produce a sub-optimal pack even with oid_array. That's because the namehash you get for a given tree object depends on the path you took to get there.

So you can certainly come up with examples where the ordering of tree objects in an array of extra roots produces a lesser-quality delta selection than the same objects permuted into some different order.

The other thing to keep in mind is that, while there are clearly trade-offs as we have discussed here, the oidset ensures that we don't allocate memory wastefully when the same object is listed multiple times as an extra root.

The other other thing to keep in mind is that the size of this set is almost always going to be puny compared to the size of the overall pack. These objects are merely meant to pull in the (likely) few objects that need refreshed out of the cruft pack in order to ensure reachability closure.

So I think it's clear that this is a trade-off, and neither decision (oidset vs oid_array) is absolutely perfect for all cases. But on balance I think that the trade-offs push us towards oidset much more than they do towards oid_array.

Thanks, Taylor

Previous: Elijah NewrenNext: Jeff King
Message 29 of 41 in “repack: various corner cases for cruft-less MIDXs”
  1. 0/4 repack: various corner cases for cruft-less MIDXsTaylor Blau, Sep 30, 2026
  2. 1/4 pack-objects: introduce `stdin_packs_context` structTaylor Blau, Sep 30, 2026
  3. 2/4 pack-objects: ensure tree/tag closure with '--stdin-packs=follow'Taylor Blau, Sep 30, 2026
  4. 3/4 repack: retain cruft packs in MIDXs after incremental repacksTaylor Blau, Sep 30, 2026
  5. 4/4 repack: retain cruft packs in MIDXs containing kept packsTaylor Blau, Sep 30, 2026
  6. Junio C HamanoSep 30, 2026
  7. Junio C HamanoSep 30, 2026
  8. Derrick StoleeSep 30, 2026
  9. Jeff KingSep 30, 2026
  10. Jeff KingSep 30, 2026
  11. Jeff KingSep 30, 2026
  12. Jeff KingSep 30, 2026
  13. Taylor BlauOct 1, 2026
  14. Taylor BlauOct 1, 2026
  15. Taylor BlauOct 1, 2026
  16. Taylor BlauOct 1, 2026
  17. Taylor BlauOct 1, 2026
  18. Taylor BlauOct 1, 2026
  19. 0/8 repack: various corner cases for cruft-less MIDXsTaylor Blau, Oct 1, 2026
  20. 1/8 pack-objects: introduce `stdin_packs_context` structTaylor Blau, Oct 1, 2026
  21. 2/8 pack-objects: ensure tree/tag closure with '--stdin-packs=follow'Taylor Blau, Oct 1, 2026
  22. 3/8 repack: retain cruft packs in MIDXs after incremental repacksTaylor Blau, Oct 1, 2026
  23. 4/8 repack: use a sorted list for explicitly kept packsTaylor Blau, Oct 1, 2026
  24. 5/8 repack: follow kept packs when omitting cruft from the MIDXTaylor Blau, Oct 1, 2026
  25. 6/8 repack: track the preferred pack explicitly in MIDX write stepsTaylor Blau, Oct 1, 2026
  26. 7/8 repack: defer allocating the append plan's write stepTaylor Blau, Oct 1, 2026
  27. 8/8 repack: include required packs in incremental MIDX writesTaylor Blau, Oct 1, 2026
  28. Elijah NewrenOct 1, 2026
  29. Taylor BlauOct 2, 2026
  30. Jeff KingOct 2, 2026
  31. Jeff KingOct 2, 2026
  32. Jeff KingOct 2, 2026
  33. Jeff KingOct 2, 2026
  34. Jeff KingOct 2, 2026
  35. Jeff KingOct 2, 2026
  36. Taylor BlauOct 3, 2026
  37. Taylor BlauOct 3, 2026
  38. Taylor BlauOct 3, 2026
  39. Taylor BlauOct 3, 2026
  40. Jeff KingOct 3, 2026
  41. Jeff KingOct 3, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.