Re: [PATCH 2/2] pack-objects: fix threaded load balancing

From: Nicolas Pitre <nico@cam.org>
Date: Dec 10, 2007, 16:03 UTC
Message-ID: <alpine.LFD.0.99999.0712101048230.555@xanadu.home>
In-Reply-To: <9e4733910712092219kd86e576k33b11c93179f6475@mail.gmail.com>

On Mon, 10 Dec 2007, Jon Smirl wrote:

> It seems to be caused by looping over those runs of 100,000 identical
> hash entries.

As mentioned earlier, there simply can't be 100000 identical hash entries.

> 5091699 24.6407 :                       for (entry = index->hash[i]; entry < index->hash[i+1]; entry++) {
> 5301791 25.6575 :                               const unsigned char *ref = entry->ptr;

This loop is the very heart of the delta search code. It is normal to see it high on the graph. In some worst cases, this loop might even be entered for every byte in the source buffer.

And yet I bring you back to your observation about the fact that the source pack used for the repack (whether it is the 2.1GB pack or the 300MB pack) has a great influence on the performance outcome. The code above, though, would be executed the very exact number of times, and the exact amount of memory allocated, regardless of the source pack used (as long as you use -f with 'git repack' of course).

So this cannot be the culprit.

Nicolas