Re: Poor performance of git describe in big repos

From: Alex Bennée <kernel-hacker@bennee.com>
Date: May 30, 2013, 13:09 UTC
Message-ID: <CAJ-05NNAeLUfyk8+NU8PmjKqfTcZ1NT_NPAk3M1QROtzsQKJ8g@mail.gmail.com>
In-Reply-To: <CALkWK0ndKMZRuWgdg6djqPUGxbDAqZPcv2q0qPrv_2b=1NEM5g@mail.gmail.com>

It looks like it's a file caching effect combined with my repo being more pathalogical in size and contents. Note run 1 (cold) vs run 2 on the linux file tree:

13:52 ajb@sloy/x86_64 [linux.git] >time git describe --debug --long
--tags HEAD~10000
searching to describe HEAD~10000
 annotated         57 v2.6.34-rc2
 annotated       1688 v2.6.34-rc1
 annotated       7932 v2.6.33
 annotated       8157 v2.6.33-rc8
 annotated       8381 v2.6.33-rc7
 annotated       8637 v2.6.33-rc6
 annotated       8964 v2.6.33-rc5
 annotated       9493 v2.6.33-rc4
 annotated       9912 v2.6.33-rc3
 annotated      10202 v2.6.33-rc2
traversed 10547 commits
more than 10 tags found; listed 10 most recent
gave up search at 55639353a0035052d9ea6cfe4dde0ac7fcbb2c9f
v2.6.34-rc2-57-gef5da59

real    0m7.332s
user    0m0.308s
sys     0m0.244s
14:03 ajb@sloy/x86_64 [linux.git] >time git describe --debug --long
--tags HEAD~10000
searching to describe HEAD~10000
 annotated         57 v2.6.34-rc2
 annotated       1688 v2.6.34-rc1
 annotated       7932 v2.6.33
 annotated       8157 v2.6.33-rc8
 annotated       8381 v2.6.33-rc7
 annotated       8637 v2.6.33-rc6
 annotated       8964 v2.6.33-rc5
 annotated       9493 v2.6.33-rc4
 annotated       9912 v2.6.33-rc3
 annotated      10202 v2.6.33-rc2
traversed 10547 commits
more than 10 tags found; listed 10 most recent
gave up search at 55639353a0035052d9ea6cfe4dde0ac7fcbb2c9f
v2.6.34-rc2-57-gef5da59

real 0m0.298s user 0m0.244s sys 0m0.036s

Although the perf profile looks subtly different.

First through the linux tree:

 22.35%   git  libz.so.1.2.3.4    [.] inflate
 18.56%   git  libz.so.1.2.3.4    [.] inflate_fast
 17.48%   git  libz.so.1.2.3.4    [.] inflate_table
  7.84%   git  git                [.] hashcmp
  3.93%   git  git                [.] get_sha1_hex
  3.46%   git  libz.so.1.2.3.4    [.] adler32

And through my "special" repo:

 41.58%   git  libcrypto.so.1.0.0  [.] sha1_block_data_order_ssse3
 33.62%   git  libz.so.1.2.3.4     [.] inflate_fast
 10.39%   git  libz.so.1.2.3.4     [.] adler32
  2.03%   git  [kernel.kallsyms]   [k] clear_page_c

 I'm not sure why libcrypto features so highly in the results

 --
 Alex.

On 30 May 2013 12:33, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Show 36 quoted lines

> Alex Bennée wrote:
>>>time /usr/bin/git --no-pager
>> traversed 223 commits
>>
>> real    0m4.817s
>> user    0m4.320s
>> sys     0m0.464s
>
> I'm quite clueless about why it is taking this long: I think it's IO
> because there's nothing to compute?  I really can't trace anything
> unless you can reproduce it on a public repository.  On linux.git with
> my rotating hard disk:
>
> $ time git describe --debug --long --tags HEAD~10000
> searching to describe HEAD~10000
>  annotated       5445 v2.6.33
>  annotated       5660 v2.6.33-rc8
>  annotated       5884 v2.6.33-rc7
>  annotated       6140 v2.6.33-rc6
>  annotated       6467 v2.6.33-rc5
>  annotated       6999 v2.6.33-rc4
>  annotated       7430 v2.6.33-rc3
>  annotated       7746 v2.6.33-rc2
>  annotated       8212 v2.6.33-rc1
>  annotated      13854 v2.6.32
> traversed 18895 commits
> more than 10 tags found; listed 10 most recent
> gave up search at 648f4e3e50c4793d9dbf9a09afa193631f76fa26
> v2.6.33-5445-ge7c84ee
>
> real    0m0.509s
> user    0m0.470s
> sys     0m0.037s
>
> 18k+ commits traversed in half a second here, so I really don't know
> what is going on.

-- 
Alex, homepage: http://www.bennee.com/~alex/