Re: pack operation is thrashing my server

From: Björn Steinbrink <b.steinbrink@gmx.de>
Date: Aug 14, 2008, 23:39 UTC
Message-ID: <20080814233958.GA31225@atjola.homenet>
In-Reply-To: <alpine.LFD.1.10.0808141544150.3324@nehalem.linux-foundation.org>

On 2008.08.14 16:14:26 -0700, Linus Torvalds wrote:

Show 16 quoted lines

> 
> On Thu, 14 Aug 2008, Nicolas Pitre wrote:
> > 
> > Possible.  However, the fact that both the "Compressing objects" and the 
> > "Writing objects" phases during a repack (without -f) together are 
> > _faster_ than the "Counting objects" phase is a sign that something is 
> > more significant than cache misses here, especially when tree 
> > information is a small portion of the total pack data size.
> 
> Hmm. I think I may have clue.
> 
> The size of the delta cache seems to be a sensitive parameter for this 
> thing. Not so much for the git archive, but working on the kernel tree, 
> raising it to 1024 seems to give a 20% performance improvement. That, in 
> turn, implies that we may be unpacking things over and over again because 
> of bad locality wrt delta generation.

Since you mention the delta cache, uau (no idea about his real name) on #git was talking about some delta cache optimizations lately, although he was dealing with "git log -S", maybe it affects rev-list in a similar way. Unfortunately, I can't seem to find any code for that, just a description of what he did and some numbers on the results in the IRC logs.

http://colabti.org/irclogger/irclogger_log/git?date=2008-08-04,Mon#l65

Maybe that helps in some way.

Björn