Re: [PATCH] basic threaded delta search

From: Junio C Hamano <gitster@pobox.com>
Date: Sep 6, 2007, 07:01 UTC
Message-ID: <7vwsv4cm6b.fsf@gitster.siamese.dyndns.org>
In-Reply-To: <1189059193250-git-send-email-nico@cam.org>

Nicolas Pitre <nico@cam.org> writes:

Show 8 quoted lines

> this is still rough, hence it is disabled by default.  You need to compile
> with "make THREADED_DELTA_SEARCH=1 ..." at the moment.
>
> Threading is done on different portions of the object list to be
> deltified. This is currently done by spliting the list into n parts and
> then a thread is spawned for each of them.  A better method would consist
> of spliting the list into more smaller parts and have the n threads
> pick the next part available.

Hmmm. I wonder how the result is affected by such a partition; aren't you going to have many objects that could have used somebody else as a delta but gets stored as base because they happen to be a very early part of their partition (and lacking delta base candidates in the window)? You cannot solve it with overlapping partitions without busting the depth limit easily either, I suspect. Also how would this interact with the LRU delta base window we discussed a week or two ago?

Separating the list into different object types would not have any adverse impact coming from the "horizon" of delta base candidates window (because we do not deltify across types), but that is not very useful because we cannot gain much parallerism from such a partition.