Re: Why does send-pack call pack-objects for all remote refs?

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Dec 14, 2015, 22:31 UTC
Message-ID: <20151214223155.GA6594@google.com>
In-Reply-To: <20151214210429.GC14788@sigill.intra.peff.net>

Jeff King wrote:

Show 11 quoted lines

> Hmm. I guess that makes sense. The bitmap we want is the set difference
> between the objects we are sending, and the tips the other side has. If
> we have a bitmap at each ref tip, that's very fast. But if you have a
> very large number of refs, we don't make one for each ref, and it has to
> fallback to walking to the nearest one (and it ends up worse than a
> regular walk, because it's filling in the bitmap for each tree, rather
> than just doing the "good enough" commit walk that we usually do).
>
> I suspect there's room for improvement in the way we select commits to
> store bitmaps for (so that the average walk is smaller). But it's rather
> tricky; there's not a single constant to change to make it work better.

Git gc and JGit GC differ here. JGit partitions the commits being packed by branch and then runs a selection algorithm on each part. Git runs a selection once on a list of all commits.

Some effects:
- JGit selects more bitmaps, so the gc takes longer and the resulting
  bitmap file is larger (bad)
- JGit is more likely to have bitmaps for the commits involved in
  pushes and fetches (good)

The commit selection code, for reference:

https://eclipse.googlesource.com/jgit/jgit/+/86af34e1/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/pack/PackWriterBitmapPreparer.java#151 https://kernel.googlesource.com/pub/scm/git/git/+/ed1c9977/pack-bitmap-write.c#383

Thoughts? Jonathan