Re: [PATCH] blame.c: don't drop origin blobs as eagerly

From: David Kastrup <dak@gnu.org>
Date: Apr 3, 2019, 11:08 UTC
Message-ID: <87ftqz5osx.fsf@fencepost.gnu.org>
In-Reply-To: <xmqqv9zvsfay.fsf@gitster-ct.c.googlers.com>

Junio C Hamano <gitster@pobox.com> writes:

Show 24 quoted lines

> David Kastrup <dak@gnu.org> writes:
>
>> When a parent blob already has chunks queued up for blaming, dropping
>> the blob at the end of one blame step will cause it to get reloaded
>> right away, doubling the amount of I/O and unpacking when processing a
>> linear history.
>>
>> Keeping such parent blobs in memory seems like a reasonable optimization
>> that should incur additional memory pressure mostly when processing the
>> merges from old branches.
>
> Thanks for finding an age-old one that dates back to 7c3c7962
> ("blame: drop blob data after passing blame to the parent",
> 2007-12-11).
>
> Interestingly, the said commit claims:
>
>     When passing blame from a parent to its parent (i.e. the
>     grandparent), the blob data for the parent may need to be read
>     again, but it should be relatively cheap, thanks to delta-base
>     cache.
>             
> but perhaps you found a case where the delta-base cache is not all
> that effective in the benchmark?

The most relevant contribution is in a linear history where the diff between commit and parent is followed by the diff between parent and grandparent. It seems wasteful to recreate the blobs in this case. Of course this is also the case where any close cache layers are more likely to still be warm, so the savings may be less apparent. They are likely more for deep delta chains in long histories where the delta-chain cache is more thoroughly exercised.

-- 
David Kastrup