From: Paolo Bonzini <bonzini@gnu.org>
Date: Mon, 22 Feb 2010 12:59:50 GMT
Subject: Re: [PATCH] Teach "git add" and friends to be paranoid
Message-ID: <4B827FC6.1090905@gnu.org>
In-Reply-To: <7vocjnqf5c.fsf@alter.siamese.dyndns.org>

On 02/18/2010 06:36 AM, Junio C Hamano wrote:
> Nicolas Pitre<nico@fluxnic.net>  writes:
>
>> It is likely to have better performance if the buffer is small enough to
>> fit in the CPU L1 cache.  There are two sequencial passes over the
>> buffer: one for the SHA1 computation, and another for the compression,
>> and currently they're sure to trash the L1 cache on each pass.
>
> I did a very unscientific test to hash about 14k paths (arch/ and fs/ from
> the kernel source) using "git-hash-object -w --stdin-paths" into an empty
> repository with varying sizes of paranoia buffer (quarter, 1, 4, 8 and
> 256kB) and saw 8-30% overhead.  256kB did hurt and around 4kB seemed to be
> optimal for my this small sample load.
>
> In any case, with any size of paranoia, this hurts the sane use case

Because by mmaping + memcpying you are getting the worst of both cases: 
you get a page fault per page like with mmap, and touch memory twice 
like with read.

Paolo