git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] builtin/pack-objects.c: introduce `pack.extraCruftTips`

From
Jeff King <peff@peff.net>
Date
May 5, 2023, 21:23 UTC
Message-ID
<20230505212322.GA3321533@coredump.intra.peff.net>
In-Reply-To
<ZFLQfMtIGcZHNERE@nand.local>
On Wed, May 03, 2023 at 05:22:04PM -0400, Taylor Blau wrote:
Show 28 quoted lines
> > OK, I understand the use case you're trying to support, and your
> > approach mostly makes sense. But there are two things I was surprised by
> > in the implementation:
> >
> >   1. Does this need to be tied to cruft packs? The same logic would
> >      apply to "repack -A" which turns objects loose (of course you
> >      probably don't want to do that in the long term for performance
> >      reasons, but it's conceptually the same thing; see below).
> >
> >   2. Why is there a separate walk distinct from the one that rescues
> >      recent-but-unreachable objects?
> 
> > Conceptually it seems to me that the simplest and most flexible way to
> > think of this new feature is: pretend these objects are recent enough to
> > be kept in the grace period, even though their mtimes do not qualify".
> >
> > And then everything else would just fall out naturally. Am I missing
> > something?
> 
> I originally shied away from it, thinking that I wouldn't want to do an
> expensive walk with all of the recent objects during a non-pruning
> repack.
>
> The two code paths are quite different in practice. In the pruning case,
> we add only new objects from the kept packs and then start our traversal
> there. In the non-pruning case, we just do
> `add_objects_in_unpacked_packs()` which is really just a call to
> `for_each_packed_object()`.

Right, that's what I'd expect. I think you are describing a cruft-pack world here (because you say "kept packs"), but the traditional "repack -k" versus "repack -A" is similar (if we are not doing something special with recent objects, then there is no need to figure out what they reach; we can just add them all).

Show 10 quoted lines
> So it gets tricky when you have a pack.extraCruftTips program and want
> to invoke it in a non-pruning case. You could do something like:
> 
>   - call enumerate_and_traverse_cruft_objects() *always*, either because
>     we were doing a pruning GC, or calling it after
>     `enumerate_cruft_objects()` (in the non-pruning case)
> 
>   - ensure that enumerate_and_traverse_cruft_objects() is a noop when
>     (a) cruft_expiration is set to zero, and (b) there are no
>     pack.extraCruftTips programs specified

I'm not sure why you'd need to traverse, though. If we are in "-k" mode, we are keeping everything anyway (so I don't even see the point of asking the helper about extra tips). And all of those objects that are not reachable from the regular traversal are by definition "cruft" and go into the cruft pack.

Maybe I don't understand what you mean by "non-pruning" here.
-Peff
Previous: Taylor BlauNext: Taylor Blau
Message 16 of 31 in “builtin/pack-objects.c: introduce `pack.extraCruftTips`”
  1. builtin/pack-objects.c: introduce `pack.extraCruftTips`Taylor Blau, Apr 20, 2023
  2. Junio C HamanoApr 20, 2023
  3. Taylor BlauApr 20, 2023
  4. Junio C HamanoApr 20, 2023
  5. Taylor BlauApr 20, 2023
  6. Chris TorekApr 21, 2023
  7. Taylor BlauApr 21, 2023
  8. Derrick StoleeApr 25, 2023
  9. Taylor BlauApr 25, 2023
  10. Derrick StoleeApr 26, 2023
  11. Taylor BlauMay 3, 2023
  12. builtin/pack-objects.c: introduce `pack.extraCruftTips`Taylor Blau, May 3, 2023
  13. Derrick StoleeMay 3, 2023
  14. Jeff KingMay 3, 2023
  15. Taylor BlauMay 3, 2023
  16. Jeff KingMay 5, 2023
  17. Taylor BlauMay 6, 2023
  18. Taylor BlauMay 6, 2023
  19. Taylor BlauMay 3, 2023
  20. Jeff KingMay 5, 2023
  21. Jeff KingMay 5, 2023
  22. Taylor BlauMay 6, 2023
  23. Taylor BlauMay 6, 2023
  24. Jeff KingMay 6, 2023
  25. builtin/pack-objects.c: introduce `pack.extraCruftTips`Taylor Blau, May 3, 2023
  26. Junio C HamanoMay 3, 2023
  27. Junio C HamanoMay 3, 2023
  28. Taylor BlauMay 3, 2023
  29. Taylor BlauMay 3, 2023
  30. Jeff KingMay 5, 2023
  31. Jeff KingMay 5, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.