Re: [PATCH] [RFC] Making use of bitmaps for thin objects

From: Jeff King <peff@peff.net>
Date: Jan 7, 2014, 17:26 UTC
Message-ID: <20140107172644.GA19051@sigill.intra.peff.net>
In-Reply-To: <5CDDBDF2D36D9F43B9F5E99003F6A0D4466885EE@PRN-MBX02-1.TheFacebook.com>

On Mon, Jan 06, 2014 at 10:14:30PM +0000, Ben Maurer wrote:

> It looks like for my repo the size win wasn't as big (~10%). Is it
> possible that with the kernel test you got extremely lucky and there
> was some huge binary blob that thin packing turned into a tiny delta?

I don't think so. When I look at the reused-delta numbers, I see that we reused ~3000 extra deltas (and the "compressing" progress meter drops by the same amount. If I do a full clone (or just index-pack the result), it claims ~3000 thin objects completed with local objects (versus 0 in the normal case).

So I think we really are getting a lot of little savings adding up, which makes sense. If there were thousands of changed files, a non-thin pack has to have at least _one_ full version of each file. With thin packs, we might literally have only deltas.

It was a 7-week period, which might make more difference. I'm going to run some experiments with different time periods to see if that changes anything.

It might also be the repo contents. I'm going to try my experiments on a few different repositories. It may be that either the kernel or your repo is unusual in some way.

Or maybe I was just lucky. :)

> When you get a chance, it'd be handy if you could push an updated
> version of your change out to your public github repo. I'd like to see
> if folks here are interested in testing this more, and it'd be good to
> make sure we're testing the diff that is targeted for upstream.

I've pushed it to:

  git://github.com/peff/git.git jk/bitmap-reuse-delta

I'll continue to rebase it forward as time goes on (until a cleaned-up version gets merged upstream), but the tip of that branch should always be in a working state.

-Peff