Re: Repacking a repository uses up all available disk space

From: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Date: Jun 12, 2016, 21:54 UTC
Message-ID: <20160612215436.GB4584@gmail.com>
In-Reply-To: <20160612213804.GA5428@sigill.intra.peff.net>

On Sun, Jun 12, 2016 at 05:38:04PM -0400, Jeff King wrote:

Show 26 quoted lines

> > - When attempting to repack, creates millions of files and eventually
> >   eats up all available disk space
> 
> That means these objects fall into the unreachable category. Git will
> prune unreachable loose objects after a grace period based on the
> filesystem mtime of the objects; the default is 2 weeks.
> 
> For unreachable packed objects, their mtime is jumbled in with the rest
> of the objects in the packfile.  So Git's strategy is to "eject" such
> objects from the packfiles into individual loose objects, and let them
> "age out" of the grace period individually.
> 
> Generally this works just fine, but there are corner cases where you
> might have a very large number of such objects, and the loose storage is
> much more expensive than the packed (e.g., because each object is stored
> individually, not as a delta).
> 
> It sounds like this is the case you're running into.
> 
> The solution is to lower the grace period time, with something like:
> 
>   git gc --prune=5.minutes.ago
> 
> or even:
> 
>   git gc --prune=now

You are correct, this solves the problem, however I'm curious. The usual maintenance for these repositories is a regular run of:

- git fsck --full
- git repack -Adl -b --pack-kept-objects
- git pack-refs --all
- git prune

The reason it's split into repack + prune instead of just gc is because we use alternates to save on disk space and try not to prune repos that are used as alternates by other repos in order to avoid potential corruption.

Am I not doing something that needs to be doing in order to avoid the same problem?

Thanks for your help.

Regards,

-- 
Konstantin Ryabitsev
Linux Foundation Collab Projects
Montréal, Québec