Re: [PATCH] builtin/pack-objects.c: introduce `pack.extraCruftTips`

From: Taylor Blau <me@ttaylorr.com>
Date: Apr 20, 2023, 20:48 UTC
Message-ID: <ZEGlJx4ibYSp6qmD@nand.local>
In-Reply-To: <xmqq1qkez5yg.fsf@gitster.g>

On Thu, Apr 20, 2023 at 12:52:55PM -0700, Junio C Hamano wrote:

Show 15 quoted lines

> Taylor Blau <me@ttaylorr.com> writes:
>
> >> But it makes me wonder if it would make the life of end-users simpler
> >> if we reserve a special ref hierarchy, say "refs/crufts/*", than
> >> having to write a program for doing something like this.
> >
> > Ideally, yes. But I think there are certain instances where there are
> > far too many (disconnected) objects that creating a reference for each
> > part of the unreachable object graph that we want to keep is infeasible.
> >
> > Another way to think about pack.extraCruftTips is that the program
> > invocation is acting like the refs/crufts hierarchy would if it existed,
> > but without actually having to write all of those references down.
>
> [...] Is there a less hand-wavy use case you have in mind?

Sure. The use-case I have in mind directly is keeping certain entries from GitHub's `audit_log` file (see a description from Peff in [1]) while excluding others.

We use the audit_log to track every single reference change, like the reflog but with the reference name prepended to each entry and some optional metadata attached to the end of each entry.

Our goal is to be able to prune the test-merge objects that GitHub creates without (usually) pruning any objects that was pushed by a user. E.g., even if a user force-pushes their branch from A to B (where B is not strictly ahead of A) we want to keep the objects from A around, even though a reference is no longer pointing at it.

This already works with reflogs, which are considered as reachable objects when generating a cruft pack (that is, they go in the "big" pack with the rest of the reachable objects. This is extending that mechanism to work with GitHub's custom format, but doing so in a way that is not tied to that format whatsoever.

The hope is that others may find it useful in other special circumstances like above.

Thanks, Taylor

[1]: https://lore.kernel.org/git/20150624094919.GC5436@peff.net/