Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

From: David Kastrup <dak@gnu.org>
Date: Mar 19, 2014, 21:25 UTC
Message-ID: <87ob11g9st.fsf@fencepost.gnu.org>
In-Reply-To: <xmqq38id3nfs.fsf@gitster.dls.corp.google.com>

Junio C Hamano <gitster@pobox.com> writes:

Show 10 quoted lines

> David Kastrup <dak@gnu.org> writes:
>
>> The default of 16MiB causes serious thrashing for large delta chains
>> combined with large files.
>>
>> Signed-off-by: David Kastrup <dak@gnu.org>
>> ---
>
> Is that a good argument?  Wouldn't the default of 128MiB burden
> smaller machines with bloated processes?

The default file size before Git forgets about delta compression is 512MiB. Unpacking 500MiB files with 16MiB of delta storage is going to be uglier.

Show 9 quoted lines

>> Forgot the signoff.  For the rationale of this patch and the 128MiB
>> choice, see the original patch.
>
> "See the original patch", especially written after three-dash lines
> without a reference, will not help future readers of "git log" who
> later bisects to find that this change hurt their usage and want to
> see why it was done unconditionally (as opposed to encouraging those
> who benefit from this change to configure their Git to use larger
> value for them, without hurting others).

Documentation/config.txt states:

    core.deltaBaseCacheLimit::
            Maximum number of bytes to reserve for caching base objects
            that may be referenced by multiple deltified objects.  By storing the
            entire decompressed base objects in a cache Git is able
            to avoid unpacking and decompressing frequently used base
            objects multiple times.
    +
    Default is 16 MiB on all platforms.  This should be reasonable
    for all users/operating systems, except on the largest projects.
    You probably do not need to adjust this value.

I've seen this seriously screwing performance in several projects of mine that don't really count as "largest projects".

So the description in combination with the current setting is clearly wrong.

Show 5 quoted lines

> While I can personally afford 128MiB, I do *not* think 16MiB was
> chosen more scientifically than the choice of 128MiB this change
> proposes to make, and my gut feeling is that this would not have too
> big a negative impact to anybody, I would prefer to have a reason
> better than gut feeling before accepting a default change.

Shrug. I've set my private default anyway, so I don't have anything to gain from this change. If anybody wants to pick this up and put more science into the exact value: be my guest.

-- 
David Kastrup