Re: Preserve/Prune Old Pack Files

From: Jeff King <peff@peff.net>
Date: Jan 9, 2017, 10:55 UTC
Message-ID: <20170109105545.gchfklcuzwhlfbtf@sigill.intra.peff.net>
In-Reply-To: <20170109070119.lite2o7k3t2wuvtt@glandium.org>

On Mon, Jan 09, 2017 at 04:01:19PM +0900, Mike Hommey wrote:

Show 14 quoted lines

> > That's _way_ more complicated than your problem, and as I said, I do not
> > have a finished solution. But it seems like they touch on a similar
> > concept (a post-delete holding area for objects). So I thought I'd
> > mention it in case if spurs any brilliance.
> 
> Something that is kind-of in the same family of problems is the
> "loosening" or objects on repacks, before they can be pruned.
> 
> When you have a large repository and do large rewrite operations
> (extreme case, a filter-branch on a multi-hundred-thousands commits),
> and you gc for the first time, git will possibly create a *lot* of loose
> objects, each of which will consume an inode and a file system block. In
> the extreme case, you can end up with git gc filling up multiple extra
> gigabytes on your disk.

I think we're getting pretty far off-field here. :)

Yes, this can be a problem. The repack is smart enough not to write out objects which would just get pruned immediately, but since the grace period is 2 weeks, that can include a lot of objects (especially with history rewriting as you note). It would be possible to write those loose objects to a "cruft" pack, but there are some management issues around the cruft pack. You do not want to keep repacking them into a new cruft pack at each repack, since then they would never expire. So you need some way of marking the pack as cruft, letting it age out, and then deleting it after the grace period expires.

I don't think it would be _that_ hard, but AFAIK nobody has ever made patches.

-Peff