Re: [PATCH 0/3] On compresing large index

From: Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Date: Feb 6, 2012, 01:35 UTC
Message-ID: <CACsJy8AnGg11PeCGFs_BxOM3wAjwzs2tOCWJV31_2_KMFTxhDA@mail.gmail.com>
In-Reply-To: <87ehu9ug9i.fsf@thomas.inf.ethz.ch>

2012/2/6 Thomas Rast <trast@inf.ethz.ch>:

Show 7 quoted lines

>> We need to figure out what git uses 4s user time for.
>
> When I worked on the cache-tree stuff, my observation (based on
> profiling, so I had actual data :-) was that computing SHA1s absolutely
> dominates everything in such operations.  It does that when writing the
> index to write the trailing checksum, and also when loading it to verify
> that the index is valid.

You're right. This is on another machine but with same index (2M files), without SHA1 checksum:

$ time ~/w/git/git ls-files --stage|head > /dev/null real 0m1.533s user 0m1.228s sys 0m0.306s

and with SHA-1 checksum:

$ time git ls-files --stage|head > /dev/null real 0m7.525s user 0m7.257s sys 0m0.268s

I guess we could fall back to cheaper digests for such a large index. Still more than one second for doing nothing but reading index is too slow to me.

> ls-files shouldn't be so slow though.  A quick run with callgrind in a
> linux-2.6.git tells me it spends about 45% of its time on SHA1s and a
> whopping 25% in quote_c_style().  I wonder what's so hard about
> quoting...

That's why I put "| head" there, to cut output processing overhead (hopefully).

-- 
Duy