Re: Poor performance of git describe in big repos

From: Thomas Rast <trast@inf.ethz.ch>
Date: May 30, 2013, 15:33 UTC
Message-ID: <87ehcoeb3t.fsf@linux-k42r.v.cablecom.net>
In-Reply-To: <CAJ-05NNAeLUfyk8+NU8PmjKqfTcZ1NT_NPAk3M1QROtzsQKJ8g@mail.gmail.com>

Alex Bennée <kernel-hacker@bennee.com> writes:

>  41.58%   git  libcrypto.so.1.0.0  [.] sha1_block_data_order_ssse3
>  33.62%   git  libz.so.1.2.3.4     [.] inflate_fast
>  10.39%   git  libz.so.1.2.3.4     [.] adler32
>   2.03%   git  [kernel.kallsyms]   [k] clear_page_c

Do you have any large blobs in the repo that are referenced directly by a tag?

Because this just so happens to exactly reproduce your symptoms:

  # in a random git.git
  $ time git describe --debug
  [...]
  real    0m0.390s
  user    0m0.037s
  sys     0m0.011s
  $ git tag big1 $(dd if=/dev/urandom bs=1M count=512 | git hash-object -w --stdin)
  512+0 records in
  512+0 records out
  536870912 bytes (537 MB) copied, 45.5088 s, 11.8 MB/s
  $ time git describe --debug
  [...]
  real    0m1.875s
  user    0m1.738s
  sys     0m0.129s
  $ git tag big2 $(dd if=/dev/urandom bs=1M count=512 | git hash-object -w --stdin)
  512+0 records in
  512+0 records out
  536870912 bytes (537 MB) copied, 44.972 s, 11.9 MB/s
  $ time git describe --debugsuche zur Beschreibung von HEAD
  [...]
  real    0m3.620s
  user    0m3.357s
  sys     0m0.248s

(I actually ran the git-describe invocations more than once to ensure that they are again cache-hot.)

git-describe should probably be fixed to avoid loading blobs, though I'm not sure off hand if we have any infrastructure to infer the type of a loose object without inflating it. (This could probably be added by inflating only the first block.) We do have this for packed objects, so at least for packed repos there's a speedup to be had.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch