Re: 'git repack' and repack.writeBitmaps=true with kept packs

From: Jeff King <peff@peff.net>
Date: Dec 1, 2016, 05:16 UTC
Message-ID: <20161201051640.gsavaexw55mwycza@sigill.intra.peff.net>
In-Reply-To: <CAKbGBLjZ2WLVRM9f=by337xLhPgKCy10T8ra6Qz7OWA=QF-5yA@mail.gmail.com>

On Wed, Nov 30, 2016 at 04:15:33PM -0800, Steven Noonan wrote:

Show 5 quoted lines

> It seems like it's behaving as though I've provided
> --pack-kept-objects. In order to ensure the .bitmap is created, it
> repacks everything, including everything in existing .pack files (not
> respecting .keep). But then it's not deleting the old .pack file
> (oddly, respecting .keep).

Right, that's exactly what's happening.

The bitmaps require a completely reachable set inside the pack, so if you omit some objects that are in .keep packs, we cannot generate the bitmap. So we have to either disable bitmaps, or pack the kept objects. By default, we do the latter (and I'll explain why in a minute).

We can't delete the .keep packfiles because we don't know for sure that we've included all of their contents in the new pack (not to mention that somebody asked to keep them, and we don't know why; we should respect that).

> What I'd expect it to do here is ignore the 'repack.writeBitmaps =
> true' value if there's a .keep that needs to be respected. Is this not
> a correct assumption?

In practice, I think that ends up worse. The .keep files are used by receive-pack as lockfiles for incoming pushes. So imagine you kick off a full repack just as somebody is pushing, and repack sees the temporary .keep file. Your options are:

  1. Disable bitmaps, leaving the repository with no bitmaps at all
     until the next repack (because you're deleting the old bitmaps
     along with the old, non-kept pack).

  2. Duplicate the newly pushed objects in the pack (if they're even
     reachable; you're also racing to see the ref updates). Now you have
     bitmaps, but you're wasting a bit of space to store the racy push
     twice (and it goes away next time you repack).

If you're running a Git server which depends on bitmaps for good performance, then (2) is much better. And that's the default.

If you want to override it, you can pass --no-pack-kept-objects, or set repack.packKeptObjects to false.

I think the documentation for --pack-kept-objects could be a bit more clear for this case. It doesn't mention the default value, nor that what you really want with "-b" is probably "--no-pack-kept-objects".

-Peff