Re: [PATCH v2 0/4] [RFC] repack: add --filter=

From: Taylor Blau <me@ttaylorr.com>
Date: Feb 26, 2022, 17:29 UTC
Message-ID: <YhpjbQeFaMNVnyP9@nand.local>
In-Reply-To: <36CA51FE-8B7F-4D08-A91D-95D8F76606C9@gmail.com>

On Sat, Feb 26, 2022 at 11:01:46AM -0500, John Cai wrote:

Show 13 quoted lines

> let me try to summarize (perhaps over simplify) the main concern folks
> have with this feature, so please correct me if I'm wrong!
>
> As a user, if I apply a filter that ends up deleting objects that it
> turns out do not exist anywhere else, then I have irrecoverably
> corrupted my repository.
>
> Before git allows me to delete objects from my repository, it should
> be pretty certain that I have path to recover those objects if I need
> to.
>
> Is that correct? It seems to me that, put another way, we don't want
> to give users too much rope to hang themselves.

I wrote about my concerns in some more detail in [1], but the thing I was most unclear on was how your demo script[2] was supposed to work.

Namely, I wasn't sure if you had intended to use two separate filters to "re-filter" a repository, one to filter objects to be uploaded to a content store, and another to filter objects to be expunged from the repository. I have major concerns with that approach, namely that if each of the filters is not exactly the inverse of the other, then we will either upload too few objects, or delete too many.

My other concern was around what guarantees we currently provide for a promisor remote. My understanding is that we expect an object which was received from the promisor remote to always be fetch-able later on. If that's the case, then I don't mind the idea of refiltering a repository, provided that you only need to specify a filter once.

So the suggestion about splitting a repository into two packs was a potential way to mediate the "two filter" problem, since the two packs you get exactly correspond to the set of objects that match the filter, and the set of objects that _don't_ match the filter.

In either case, I tried to use the patches in [1] and was able to corrupt my local repository (even when fetching from a remote that held onto the objects I had pruned locally).

Thanks, Taylor

[1]: https://lore.kernel.org/git/YhUeUCIetu%2FaOu6k@nand.local/ [2]: https://gitlab.com/chriscool/partial-clone-demo/-/blob/master/http-promisor/server_demo.txt#L47-52