Re: Why does send-pack call pack-objects for all remote refs?

From: Jeff King <peff@peff.net>
Date: Dec 14, 2015, 22:37 UTC
Message-ID: <20151214223717.GA20167@sigill.intra.peff.net>
In-Reply-To: <20151214223155.GA6594@google.com>

On Mon, Dec 14, 2015 at 02:31:55PM -0800, Jonathan Nieder wrote:

Show 20 quoted lines

> > I suspect there's room for improvement in the way we select commits to
> > store bitmaps for (so that the average walk is smaller). But it's rather
> > tricky; there's not a single constant to change to make it work better.
> 
> Git gc and JGit GC differ here.  JGit partitions the commits being
> packed by branch and then runs a selection algorithm on each part.
> Git runs a selection once on a list of all commits.
> 
> Some effects:
> - JGit selects more bitmaps, so the gc takes longer and the resulting
>   bitmap file is larger (bad)
> - JGit is more likely to have bitmaps for the commits involved in
>   pushes and fetches (good)
> 
> The commit selection code, for reference:
> 
> https://eclipse.googlesource.com/jgit/jgit/+/86af34e1/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/pack/PackWriterBitmapPreparer.java#151
> https://kernel.googlesource.com/pub/scm/git/git/+/ed1c9977/pack-bitmap-write.c#383
> 
> Thoughts?

My thought is it would be great if somebody wanted to work on this. :)

My understanding is that JGit's approach has some problems, too. Terry's message doesn't seem to have made it to the list, but you can see in the quoted bits he mentions some OOM problems during the bitmap write:

  http://article.gmane.org/gmane.comp.version-control.git/281476

That may not be a big deal to work around. I really just haven't looked at it at all. Vicent did the original bitmap selection code for C Git, and I don't think it has been touched since then.

-Peff