git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/2] pack-objects: freshen objects with multi-cruft packs

From
Elijah Newren <newren@gmail.com>
Date
Feb 27, 2025, 19:28 UTC
Message-ID
<CABPp-BEK3T28iqDFxbMHa7GVrythYs+ze7wCZ5Q-LDjOmquiVQ@mail.gmail.com>
In-Reply-To
<cover.1740680964.git.me@ttaylorr.com>
On Thu, Feb 27, 2025 at 10:29 AM Taylor Blau <me@ttaylorr.com> wrote:
Show 27 quoted lines
>
> This short series contains a fix for a bug I noticed while rolling out
> multi-cruft packs (via 'git repack --max-cruft-size') within GitHub's
> infrastructure.
>
> The series is structured as follows:
>
>  - The first patch simplifies how 'repack' aggregates cruft packs
>    together when their size is below the '--max-cruft-size' or
>    '--max-pack-size' threshold. This simplification changes behavior
>    slightly, but not in a meaningful way. It occurred to me while
>    writing the second patch.
>
>  - The second patch describes and fixes the main bug. The gist here is
>    that objects which are (a) unreachable, (b) exist in a cruft pack
>    being retained, and (c) were freshened to have a more recent mtime
>    than any existing cruft copy are unable to be freshened.
>
> The fix pursued in the second patch changes the rules around when we
> want to retain an object via builtin/pack-objects.c::want_found_object()
> when at least one cruft pack will survive the repack.
>
> Previously the rule was to discard any object which appears in any
> surviving pack, regardless of mtime. The rule now is to only discard an
> object if it appears in either (a) a non-cruft pack which will survive
> the repack, or (b) a cruft pack whose mtime for that object is older
> than the one we are trying to pack.

I think in (b) you got the meaning reversed, and instead mean s/older than/at least as new as/ ?

Show 18 quoted lines
> I think that this is the right behavior, but admittedly putting this
> series together hurt my brain trying to think through all of the cases.
> I'm fairly confident in the testing here as I remember it being fairly
> exhaustive of all interesting cases. But I'd appreciate a sanity check
> from others that they too are convinced this is the right approach.
>
> Thanks in advance for your review!
>
> Taylor Blau (2):
>   builtin/repack.c: simplify cruft pack aggregation
>   builtin/pack-objects.c: freshen objects from existing cruft packs
>
>  builtin/pack-objects.c  | 118 ++++++++++++++++++++++++++++++++++------
>  builtin/repack.c        |  38 +------------
>  packfile.c              |   3 +-
>  packfile.h              |   2 +
>  t/t7704-repack-cruft.sh | 106 ++++++++++++++++++++++--------------
>  5 files changed, 171 insertions(+), 96 deletions(-)

Code changes look good to me, but I had some wording suggestions in a few places for commit messages and comments. (Sorry for missing some of those in my preliminary review before you sent this series to the list.)

Previous: Taylor BlauNext: Taylor Blau
Message 12 of 54 in “pack-objects: freshen objects with multi-cruft packs”
  1. 0/2 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Feb 27, 2025
  2. 1/2 builtin/repack.c: simplify cruft pack aggregationTaylor Blau, Feb 27, 2025
  3. Elijah NewrenFeb 27, 2025
  4. Taylor BlauFeb 27, 2025
  5. Patrick SteinhardtFeb 28, 2025
  6. Elijah NewrenMar 4, 2025
  7. Junio C HamanoMar 5, 2025
  8. Taylor BlauMar 5, 2025
  9. 2/2 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Feb 27, 2025
  10. Elijah NewrenFeb 27, 2025
  11. Taylor BlauFeb 27, 2025
  12. Elijah NewrenFeb 27, 2025
  13. Taylor BlauFeb 27, 2025
  14. 0/2 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Mar 4, 2025
  15. 1/2 builtin/repack.c: simplify cruft pack aggregationTaylor Blau, Mar 4, 2025
  16. 2/2 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 4, 2025
  17. Elijah NewrenMar 4, 2025
  18. Taylor BlauMar 5, 2025
  19. Taylor BlauMar 5, 2025
  20. 0/1 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Mar 5, 2025
  21. 1/1 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 5, 2025
  22. Patrick SteinhardtMar 6, 2025
  23. Taylor BlauMar 13, 2025
  24. Patrick SteinhardtMar 6, 2025
  25. 0/6 pack-objects: freshen objects with multi-cruft packsTaylor Blau, Mar 11, 2025
  26. 1/6 t/t5329-pack-objects-cruft.sh: evict 'repack'-related testsTaylor Blau, Mar 11, 2025
  27. 2/6 t7704-repack-cruft.sh: consolidate `write_blob()`Taylor Blau, Mar 11, 2025
  28. 3/6 t/lib-cruft.sh: extract some cruft-related helpersTaylor Blau, Mar 11, 2025
  29. 4/6 pack-objects: generate cruft packs at most one object over thresholdTaylor Blau, Mar 11, 2025
  30. Junio C HamanoMar 11, 2025
  31. Taylor BlauMar 12, 2025
  32. Junio C HamanoMar 12, 2025
  33. Taylor BlauMar 12, 2025
  34. Elijah NewrenMar 12, 2025
  35. Taylor BlauMar 12, 2025
  36. Junio C HamanoMar 12, 2025
  37. Elijah NewrenMar 12, 2025
  38. Junio C HamanoMar 13, 2025
  39. Elijah NewrenMar 13, 2025
  40. Junio C HamanoMar 13, 2025
  41. 5/6 builtin/repack.c: simplify cruft pack aggregationTaylor Blau, Mar 11, 2025
  42. 6/6 builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 11, 2025
  43. Junio C HamanoMar 11, 2025
  44. Taylor BlauMar 12, 2025
  45. Junio C HamanoMar 12, 2025
  46. Taylor BlauMar 12, 2025
  47. Junio C HamanoMar 12, 2025
  48. Taylor BlauMar 12, 2025
  49. Junio C HamanoMar 13, 2025
  50. Taylor BlauMar 13, 2025
  51. Jeff KingMar 13, 2025
  52. Junio C HamanoMar 13, 2025
  53. builtin/pack-objects.c: freshen objects from existing cruft packsTaylor Blau, Mar 13, 2025
  54. Junio C HamanoMar 13, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.