Partial Clone garbage collection?

I've been experimenting with the new Partial Clone feature, attempting to use it to filter out the otherwise full history of the large binary resources in our repos. It works really well on the initial clone. But once you start jumping around in history a lot, the repo will grow out of proportion again as promised pack files are fetched.

Are there any plans to add a --filter parameter to git gc as well, that would be able to prune past history of objects and convert them back into pack promises? Or am I wrong in assuming that this could ever act as a native replacement for LFS? Without this, a repo would only continue to grow ad infinitum, resulting in the same issues as before unless you actively chose to delete your entire clone and re-clone it from upstream once in a while.

This operation will change the hash, and hence the history. I tried to draw attention to a specific trick:

https://public-inbox.org/git/25196441572039199@iva5-58d151f416d2.qloud-c.yandex.net/T/#t

but he was not supported.

On Wed, Oct 30, 2019 at 09:17:02PM +0300, без имени wrote:

I don't think this is quite the same thing. Simon is just talking about the partial-clone system. The files are still mentioned in history, but the client does not store the matching blobs themselves. Instead, it can fetch them on-demand from a remote repository.

-Peff

On Wed, Oct 30, 2019 at 06:08:18PM +0100, Simon Holmberg wrote:

I don't recall seeing anybody actively working on this, but I think it would be a good idea. You'd probably want to be able to specify it in your config somehow, so that subsequent repacks pruned as necessary without you having to remember to do it each time.

You could naively just drop everything that matches the filter, and then re-fetch it as needed. But for efficiency, you may want to keep some other objects:

  - objects mentioned directly in the index, or the tree of HEAD; you'd
    end up re-fetching these next time you "git checkout"

  - perhaps objects fetched recently are more worth keeping (e.g., ones
    with an mtime less than a day or two). I don't know if that helps,
    though. What you really care about is how recently they were
    accessed (assuming there's some locality there), not written. A
    frequently-accessed object may have been fetched immediately after
    you cloned, giving it an old mtime.

    Since we can get any of the objects again if we want and we're just
    optimizing, this is really just a cache-expiration problem. But it
    may be hard to implement any of the stock algorithms without having
    logs of which objects were accessed.

-Peff

I agree with this.

We should also retain the promisor objects that are not referenced by any other promisor object, regardless of whether it matches the filter. (If not, the resulting repository will not pass fsck.)

The place to make this change is most likely in repack_promisor_objects() in builtin/repack.c. Currently, it just repacks all known promisor objects into one - modifying it to only repack the ones we want (and adding a CLI option etc.) should be sufficient.