Re: git status: small difference between stating whole repository and small subdirectory

From: Piotr Krukowiecki <piotr.krukowiecki@gmail.com>
Date: Feb 16, 2012, 13:37 UTC
Message-ID: <CAA01Cso5y23UMguEe0vwOc6kR3-DjuC8-LTMDsMeeOKU4rVGvg@mail.gmail.com>
In-Reply-To: <20120215190318.GA5992@sigill.intra.peff.net>

On Wed, Feb 15, 2012 at 8:03 PM, Jeff King <peff@peff.net> wrote:

Show 15 quoted lines

> On Wed, Feb 15, 2012 at 09:57:29AM +0100, Piotr Krukowiecki wrote:
>>
> I notice that you're still I/O bound even after the repack:
>
>> $ time git status  -- .
>> real    0m2.503s
>> user    0m0.160s
>> sys     0m0.096s
>>
>> $ time git status
>> real    0m9.663s
>> user    0m0.232s
>> sys     0m0.556s
>
> Did you drop caches here, too?

Yes I did - with cache the status takes something like 0.1-0.3s on whole repo.

>  Usually that would not be the case on a
> warm cache. If it is, then it sounds like you are short on memory to
> actually hold the directory tree and object db in cache. If not, what do
> the warm cache numbers look like?

I've got 4GB of ram and I did not hit the swap when doing last performance tests AFAIK. Please see my previous posts for performance results with warm cache and profile results:

http://article.gmane.org/gmane.comp.version-control.git/190397 http://article.gmane.org/gmane.comp.version-control.git/190638

>> - can it be faster without repacking?
>
> Not really. You're showing an I/O problem, and repacking is git's way of
> reducing I/O.

So if I understand correctly, the reason is because git must compare workspace files with packed objects - and the problem is reading/seeking/searching in the packs?

Is there a way to make packs better? I think most operations are on workdir files - so maybe it'd be possible to tell gc/repack/whatever to optimize access to files which I currently have in workdir?

Show 8 quoted lines

>> - even with packed repo, the time on small subdirectory is much higher
>> than I'd expect given time on whole repo and subdirectory size - why?
>
> Hard to say without profiling.  It may be that we reduced the object db
> lookups, saving some time, but still end up stat()ing the whole tree.
> The optimization to stat only the directories of interest was in 688cd6d
> (status: only touch path we may need to check, 2010-01-14), which went
> into v1.7.0. What version of git are you using?

For latest tests I've used 1.7.9.rc0.10.gbeecc, for profiling - 1.7.9.188.g12766

Is there anything else I could do?

-- 
Piotr Krukowiecki