From: Thomas Rast <trast@student.ethz.ch>
Date: Fri, 12 Aug 2011 15:59:24 GMT
Subject: git_checkattr() is inefficient when repeated [Re: [PATCH 00/11] Micro-optimizing lookup_object()]
Message-ID: <201108121759.24884.trast@student.ethz.ch>
In-Reply-To: <1313085196-13249-1-git-send-email-gitster@pobox.com>

Junio C Hamano wrote:
> 4-way cuckoo

Cool stuff!

While looking at the performance of it, I noticed something odd about
packing: stracing the command you gave for your timings

  strace -o pack.trace \
    ./git-pack-objects --count-only --keep-true-parents --honor-pack-keep \
    --non-empty --all --reflog --no-reuse-delta --delta-base-offset \
    --stdout  </dev/null >/dev/null

yields the fairly crazy

  $ grep -c 'open.*attrib' pack.trace 
  4398

including runs such as (with a line of context for clarity)

  munmap(0x7f9cd39f7000, 4096)            = 0
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("compat/.gitattributes", O_RDONLY) = -1 ENOENT (No such file or directory)
  open("t/.gitattributes", O_RDONLY)      = 3
  fstat(3, {st_mode=S_IFREG|0644, st_size=36, ...}) = 0

So calling git_checkattr in a loop is quite inefficient.  Indeed
there's a good optimization opportunity: compiled for 4-way hashing I
have (best of 3)

  6.76user 0.23system 0:07.02elapsed 99%CPU (0avgtext+0avgdata 479792maxresident)

but making no_try_delta() in pack-objects.c a dummy 'return 0' gives

  6.45user 0.13system 0:06.61elapsed 99%CPU (0avgtext+0avgdata 478256maxresident)

Which would be a 4.5% speedup.  Obviously that won't quite be
attainable since we want the attributes mechanism to work, but we
still shouldn't have to open 4398 .gitattributes files when there are
only 8 .gitattributes plus one .git/info/attributes.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch