Re: pack operation is thrashing my server

From: Nicolas Pitre <nico@cam.org>
Date: Aug 14, 2008, 21:50 UTC
Message-ID: <alpine.LFD.1.10.0808141633080.4352@xanadu.home>
In-Reply-To: <alpine.LFD.1.10.0808141215520.3324@nehalem.linux-foundation.org>

On Thu, 14 Aug 2008, Linus Torvalds wrote:

Show 11 quoted lines

> Here's a hint: the cost of a cache miss is generally about a hundred times 
> the cost of just about anything else. 
> 
> So to make a convincing argument, you'd have to show that the actual 
> memory access patterns are also much better.
> 
> No, zlib isn't perfect, and nope, inflate_fast() is no "memcpy()". And 
> yes, I'm sure a pure memcpy would be much faster. But I seriously suspect 
> that a lot of the cost is literally in bringing in the source data to the 
> CPU. Because we just mmap() the whole pack-file, the first access to the 
> data is going to see the cost of the cache misses.

Possible. However, the fact that both the "Compressing objects" and the "Writing objects" phases during a repack (without -f) together are _faster_ than the "Counting objects" phase is a sign that something is more significant than cache misses here, especially when tree information is a small portion of the total pack data size.

Of course we can do further profiling, say with core.compression set to 0 and a full repack, or even hacking the pack-objects code to force a compression level of 0 for tree objects, and possibly commits too since pack v4 intend to deflate only the log text). Tree objects delta very well, but they don't deflate well at all.

OK, so I did, and the quick test for the kernel is:

|nico@xanadu:linux-2.6> time git rev-list --all --objects > /dev/null
|
|real    0m14.737s
|user    0m14.432s
|sys     0m0.296s

That's for 1031404 objects, hence we're now talking around 70k objects/sec instead of 48k objects/sec. _Only_ by removing zlib out of the equation despite the fact that the pack is now larger. So I bet that additional improvements from pack v4 could improve things even more, including the object lookup avoidance optimization I mentioned previously.

Nicolas