threads / discuss / 31957

'git describe' is very slow on development trees with lots of commits

Subject: 'git describe' is very slow on development trees with lots of commits

## tl;dr

2 messages between Oct 27, 2012 and Oct 31, 2012.

replies: 1people: 2as markdown or json

Ingo Molnar· Oct 27, 2012, 13:33 UTC · lore
(Cc:-ed the Git development list.)
* David Ahern <dsahern@gmail.com> wrote:
> PERF-VERSION-GEN and specifically the git commands are the 
> cause of more delay than the config checks, especially when 
> doing the build in a VM with the kernel source on an NFS 
> mount.
Yes, I have noticed that too.

So, the problem is that we use 'git describe' on the kernel tree to generate the version string, which is very, very slow if we are far away from any tagged release - which is the case for the -tip tree:

 comet:~/tip> perf stat --null --repeat 3 git describe
 v3.7-rc2-2007-g83e8223
 v3.7-rc2-2007-g83e8223
 v3.7-rc2-2007-g83e8223
'git describe' is much faster if we are on or near to a tag:
 $ git checkout v3.6
 $ perf stat --null --repeat 3 git describe
 v3.6
 v3.6
 v3.6
 Performance counter stats for 'git describe' (3 runs):
       0.020171640 seconds time elapsed                                          ( +-  3.64% )
 $ git checkout b34e5f55a1e6
 $ perf stat --null --repeat 3 git describe
 v3.6-41-gb34e5f5
 v3.6-41-gb34e5f5
 v3.6-41-gb34e5f5
 Performance counter stats for 'git describe' (3 runs):
       0.155603676 seconds time elapsed                                          ( +-  0.23% )
The cost on this pretty fast machine is about 1 msecs per commit 
- which adds up to about 2.5 seconds during much of the 
development cycle.

So maybe we should be using a different version string, for example, instead of:

 v3.7-rc2-2007-g83e8223
this would be perfectly fine:
 v3.7-rc2-g83e8223

the 'commit count' is informative but not essential - and in counting the number of off-tag commits is where much of the overhead is:

#
# Overhead  Command       Shared Object                                      Symbol
# ........  .......  ..................  ..........................................
#
    39.79%      git  libz.so.1.2.5       [.] 0x000000000000c1fe                    
    26.39%      git  libz.so.1.2.5       [.] inflate                               
    22.42%      git  git                 [.] 0x000000000009bd1e                    
     2.99%      git  libz.so.1.2.5       [.] adler32                               
     1.23%      git  libc-2.15.so        [.] _int_malloc                           
     0.72%      git  libc-2.15.so        [.] __GI_____strtoull_l_internal          
     0.67%      git  libc-2.15.so        [.] _int_free                             
     0.62%      git  libc-2.15.so        [.] malloc_consolidate                    
     0.54%      git  [kernel.kallsyms]   [k] clear_page_c                          
     0.32%      git  [kernel.kallsyms]   [k] page_fault                            

So by switching to the shorter version string that still embedds the tag and the exact sha1 we'd be able to run this script a *lot* faster.

Thanks,
	Ingo
Pavel Machek· Oct 31, 2012, 17:52 UTC · re: Ingo Molnar · lore

Re: 'git describe' is very slow on development trees with lots of commits

Hi!
Show 10 quoted lines
> (Cc:-ed the Git development list.)
> 
> * David Ahern <dsahern@gmail.com> wrote:
> 
> > PERF-VERSION-GEN and specifically the git commands are the 
> > cause of more delay than the config checks, especially when 
> > doing the build in a VM with the kernel source on an NFS 
> > mount.
> 
> Yes, I have noticed that too.
....
> The cost on this pretty fast machine is about 1 msecs per commit 
> - which adds up to about 2.5 seconds during much of the 
> development cycle.

Well... I noticed my builds when little changed are very slow... and it was due to the computation of version string. Ouch.

pavel@amd:~/mainline-altera/linux$ time git describe fixes-for-linus-506-g71ca8691 0.68user 0.22system 27.82 (0m27.820s) elapsed 3.26%CPU pavel@amd:~/mainline-altera/linux$

(Cached it is more reasonable 3 seconds, but it keeps going out of cache all the time. Uncached clean build is 3 minutes, cached is 9 seconds + time to do git describe).

Thikpad X60.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

← back to recent threads