Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

From: Junio C Hamano <gitster@pobox.com>
Date: Mar 20, 2014, 17:02 UTC
Message-ID: <xmqqsiqcztu8.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CACsJy8C3=bz1HmVgQuJRdixMhhb-JKouM7b1L7M047L_4PBViA@mail.gmail.com>

Duy Nguyen <pclouds@gmail.com> writes:

Show 26 quoted lines

> On Thu, Mar 20, 2014 at 5:11 AM, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> I know that the 512MiB default for the bitFileThreashold (aka
>> "forget about delta compression") came out of thin air.  It was just
>> "1GB is always too huge for anybody, so let's cut it in half and
>> declare that value the initial version of a sane threashold",
>> nothing more.
>>
>> So it might be that the problem is 512MiB is still too big, relative
>> to the 16MiB of delta base cache, and the former may be what needs
>> to be tweaked.  If a blob close to but below 512MiB is a problem for
>> 16MiB delta base cache, it would still be too big to cause the same
>> problem for 128MiB delta base cache---it would evict all the other
>> objects and then end up not being able to fit in the limit itself,
>> busting the limit immediately, no?
>>
>> I would understand if the change were to update the definition of
>> deltaBaseCacheLimit and link it to the value of bigFileThreashold,
>> for example.  With the presented discussion, I am still not sure if
>> we can say that bumping deltaBaseCacheLimit is the right solution to
>> the "description with the current setting is clearly wrong" (which
>> is a real issue).
>
> I vote make big_file_threshold smaller. 512MB is already unfriendly
> for many smaller machines. I'm thinking somewhere around 32MB-64MB
> (and maybe increase delta cache base limit to match).

These numbers match my gut feeling (e.g. 4k*4k*32-bit uncompressed would be 64MB); delta cash base that is sized to the same as (or perhaps twice as big as) that limit may be a good default.

> The only
> downside I see is large blobs will be packed  undeltified, which could
> increase pack size if you have lots of them.

I think that is something that can be tweaked, unless the user tells us otherwise via command line override, when running the improved "gc --aggressive" ;-)