Re: Horrible re-packing?

From: Nicolas Pitre <nico@cam.org>
Date: Jun 5, 2006, 21:20 UTC
Message-ID: <Pine.LNX.4.64.0606051637490.24152@localhost.localdomain>
In-Reply-To: <Pine.LNX.4.64.0606051155000.5498@g5.osdl.org>

On Mon, 5 Jun 2006, Linus Torvalds wrote:

Show 11 quoted lines

> 
> 
> On this same thread..
> 
> This trivial patch not only simplifies the name hashing, it actually 
> improves packing for both git and the kernel.
> 
> The git archive pack shrinks from 6824090->6622627 bytes (a 3% 
> improvement), and the kernel pack shrinks from 108756213 to 108219021 (a 
> mere 0.5% improvement, but still, it's an improvement from making the 
> hashing much simpler!)

OK here's the scoop. I still have a sample repo (I forget who it was from) that used to exhibit a big packing size regression which was fixed a while ago. I tend to test new packing strategies on that repo as well since it has rather interesting characteristics that makes it pretty sensitive to changes to name hashing and size filtering heuristics.

Before this hashing patch (including the rev-list fix):

$ git repack -a -f
Generating pack...
Done counting 46391 objects.
Deltifying 46391 objects.
 100% (46391/46391) done
Writing 46391 objects.
 100% (46391/46391) done
Total 46391, written 46391 (delta 7457), reused 38934 (delta 0)
Pack pack-7f766f5af5547554bacb28c0294bd562589dc5e7 created.
$ ll .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack
-rw-rw-r--  1 nico nico 39486095 Jun  5 16:28 .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack

Now with this patch applied:

$ git repack -a -f
Generating pack...
Done counting 46391 objects.
Deltifying 46391 objects.
 100% (46391/46391) done
Writing 46391 objects.
 100% (46391/46391) done
Total 46391, written 46391 (delta 9920), reused 36447 (delta 0)
Pack pack-7f766f5af5547554bacb28c0294bd562589dc5e7 created.
$ ll .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack
-rw-rw-r--  1 nico nico 16150417 Jun  5 16:31 .git/objects/pack/pack-7f766f5af5547554bacb28c0294bd562589dc5e7.pack

In other words, the pack shrunk to less than half the size of the previous one !

And yes fsck-objects still pass (I was doubtful at first).

Nicolas