Re: git repack vs git gc --aggressive

From: Jeff King <peff@peff.net>
Date: Aug 7, 2012, 18:44 UTC
Message-ID: <20120807184405.GA440@sigill.intra.peff.net>
In-Reply-To: <87zk66r28y.fsf@bitburger.home.felix>

On Tue, Aug 07, 2012 at 08:22:21PM +0200, Felix Natter wrote:

Show 5 quoted lines

> I read this:
>   http://metalinguist.wordpress.com/2007/12/06/the-woes-of-git-gc-aggressive-and-how-git-deltas-work/
> where
>   git repack -a -d --depth=250 --window=250
> is mentioned as a (recommended) alternative to git gc --aggressive.

Note how old that post is. In fact, on the very same day it was posted, the discussion on the mailing list resulted in this commit:

  commit 1c192f3442414a6ce83f9a524806fc26a0861d2d
  Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
  Date:   Thu Dec 6 12:03:38 2007 +0000

      gc --aggressive: make it really aggressive

      The default was not to change the window or depth at all.  As suggested
      by Jon Smirl, Linus Torvalds and others, default to

          --window=250 --depth=250

So the packing parameters are the same these days for either method. Note that "git gc --aggressive" will also use "-f" to recompute all deltas. This is more expensive, but gives git more flexibility if the old deltas were sub-optimal (typically, this is the case if the existing pack was generated by fast-import, which favors speed of import versus coming up with an optimal storage pattern).

Show 5 quoted lines

> So my questions are:
> 
> 1. is the above repack command (with --depth=500) safe? Of course I want
>    to be absolutely sure that our repo will be consistent.
>    Do I need another command ("git gc", "git prune") as well?

Yes, it's safe. Changing the depth parameter can never lose data. However, it's probably not a good idea for two reasons:

  1. It probably does nothing. You're not likely to hit a 500-depth
     delta chain (the point of the "250" in --aggressive is that it is
     already ridiculously high).

  2. Even if you did come up with a 500-depth delta chain, it may not be
     a good tradeoff. You might save a little bit of space, but keep in
     mind that to generate the object data, it means that git will have
     to follow a chain of 500 deltas to regenerate the object.

Of course, every workload is different. One can develop pathological cases where --depth=500 saves a lot of space. But it's unlikely that it is the case for a normal repository. You can always try both and see the result.

In fact, I'd also test how just "git gc" behaves versus "git gc --aggressive" for your repo. The former is much less expensive to run. You really shouldn't need to be running "--aggressive" all the time, so if you are looking at doing a nightly repack or similar, just "git gc" is probably fine.

-Peff