git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 4/6] pack-objects: generate cruft packs at most one object over threshold

From
Taylor Blau <me@ttaylorr.com>
Date
Mar 12, 2025, 19:33 UTC
Message-ID
<Z9HhnpYYbe/FNUWT@nand.local>
In-Reply-To
<CABPp-BH0rbieCV4Z11pHOX-mwrtEO-FPNdywV0P5HxXnusdRKQ@mail.gmail.com>
On Wed, Mar 12, 2025 at 12:13:10PM -0700, Elijah Newren wrote:
Show 29 quoted lines
> > > > But in current Git we will keep repacking
> > > > the two together, only to generate the same two packs we started with
> > > > forever.
> > >
> > > Yes.  That is because the logic that decides these packs need to be
> > > broken and recombined is flawed.  Maybe it does not have sufficient
> > > information to decide that it is no use to attempt combining them,
> > > in which case leaving some more info to help the later invocation of
> > > repack to tell that it would be useless to attempt combining these
> > > packs when you do the initial repack would help, which was what I
> > > suggested.  You've thought about the issue much longer than I did,
> > > and would be able to come up with better ideas.
> >
> > I think in the short term I came up with a worse idea than you would
> > have ;-).
> >
> > Probably there is a way to improve this niche case as described above,
> > but I think the solution space is probably complicated enough that given
> > how narrow of a case it is that it's not worth introducing that much
> > complexity.
>
> Would it make sense to break the assumption that --max-cruft-size ==
> --max-pack-size and perhaps rename the former?  I think the problem is
> that the two imply different things (one is a minimum, the other a
> maximum), and thus really should be different values.  E.g.
> --combine-cruft-below-size that is set to e.g. half of
> --max-pack-size, and then you can continue combining cruft packs
> together until they do go above the cruft threshold, while avoiding
> actually exceeding the pack size threshold?

We could, though alternatively I think leaving the behavior as is presented in v3 is equally OK.

We'll never make a pack that is as large or larger than the given --max-cruft-size, but because repack tries to aggregate smaller packs together first, it's unlikely that we would ever repeatedly repack the larger ones making them effectively "frozen", which is the goal.

Thanks, Taylor

Previous: Elijah NewrenNext: Junio C Hamano
Message 35 of 54 in “pack-objects: freshen objects with multi-cruft packs”
  1. 0/2 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Feb 27, 2025
  2. 1/2 builtin/repack.c: simplify cruft pack aggregationTaylor Blau, Feb 27, 2025
  3. Elijah NewrenFeb 27, 2025
  4. Taylor BlauFeb 27, 2025
  5. Patrick SteinhardtFeb 28, 2025
  6. Elijah NewrenMar 4, 2025
  7. Junio C HamanoMar 5, 2025
  8. Taylor BlauMar 5, 2025
  9. 2/2 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Feb 27, 2025
  10. Elijah NewrenFeb 27, 2025
  11. Taylor BlauFeb 27, 2025
  12. Elijah NewrenFeb 27, 2025
  13. Taylor BlauFeb 27, 2025
  14. 0/2 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Mar 4, 2025
  15. 1/2 builtin/repack.c: simplify cruft pack aggregationTaylor Blau, Mar 4, 2025
  16. 2/2 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 4, 2025
  17. Elijah NewrenMar 4, 2025
  18. Taylor BlauMar 5, 2025
  19. Taylor BlauMar 5, 2025
  20. 0/1 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Mar 5, 2025
  21. 1/1 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 5, 2025
  22. Patrick SteinhardtMar 6, 2025
  23. Taylor BlauMar 13, 2025
  24. Patrick SteinhardtMar 6, 2025
  25. 0/6 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Mar 11, 2025
  26. 1/6 t/t5329-pack-objects-cruft.sh: evict 'repack'-related testsTaylor Blau, Mar 11, 2025
  27. 2/6 t7704-repack-cruft.sh: consolidate `write_blob()`Taylor Blau, Mar 11, 2025
  28. 3/6 t/lib-cruft.sh: extract some cruft-related helpersTaylor Blau, Mar 11, 2025
  29. 4/6 pack-objects: generate cruft packs at most one object over thresholdTaylor Blau, Mar 11, 2025
  30. Junio C HamanoMar 11, 2025
  31. Taylor BlauMar 12, 2025
  32. Junio C HamanoMar 12, 2025
  33. Taylor BlauMar 12, 2025
  34. Elijah NewrenMar 12, 2025
  35. Taylor BlauMar 12, 2025
  36. Junio C HamanoMar 12, 2025
  37. Elijah NewrenMar 12, 2025
  38. Junio C HamanoMar 13, 2025
  39. Elijah NewrenMar 13, 2025
  40. Junio C HamanoMar 13, 2025
  41. 5/6 builtin/repack.c: simplify cruft pack aggregationTaylor Blau, Mar 11, 2025
  42. 6/6 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 11, 2025
  43. Junio C HamanoMar 11, 2025
  44. Taylor BlauMar 12, 2025
  45. Junio C HamanoMar 12, 2025
  46. Taylor BlauMar 12, 2025
  47. Junio C HamanoMar 12, 2025
  48. Taylor BlauMar 12, 2025
  49. Junio C HamanoMar 13, 2025
  50. Taylor BlauMar 13, 2025
  51. Jeff KingMar 13, 2025
  52. Junio C HamanoMar 13, 2025
  53. builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 13, 2025
  54. Junio C HamanoMar 13, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.