Re: Git performance on OS X
- From
- Pieter de Bie <pdebie@ai.rug.nl>
- Date
- Apr 19, 2008, 23:56 UTC
- Message-ID
- <2F8F3BF2-66F9-473C-BE82-8F784E1FF9A4@ai.rug.nl>
- In-Reply-To
- <alpine.LFD.1.10.0804191619240.2779@woody.linux-foundation.org>
On 20 apr 2008, at 01:26, Linus Torvalds wrote:
Show 16 quoted lines
> > It may be that the problem with OS X is a sucky pathname cache > mechanism. > > The trivial patch cut down the number of stat() calls by a fair > amount, > but the calls that got removed were all of the "do two 'lstat()' > calls on > the exact same pathname consecutively" type. > > Maybe OS X has some very limited pathname caching that catches that, > or > even if not, it just ends up being very nice in the D$, so it's not > a big > deal. And then the real suckiness happens only with bigger workloads. >
Yes, I just tested this.
for (int i = 0; i < 50000; i++) {
sprintf(s, "/Users/pieter/test/perf/%i", i);
int ret = lstat(s, a);
}This loop needs about 3 seconds to run. Replacing the i with 10 in the sprintf reduces it to 0.24seconds.
Show 15 quoted lines
>> Command Mean Std >> git status . 13.970 1.298 >> /Users/pieter/projects/External/git/git-status . 13.759 0.321 > > This is the WebKit archive, right? > > For me, doing a "time git status ." on the WebKit thing I just > cloned from > git://git.webkit.org/WebKit.git is much faster: 1.264s (and it goes > down > by maybe 5-10% with my lstat-avoidance patch). > > Is there any system-level profiler for OS X to get a clue where that > cost > is, in case it's not the lstat() at all?
Yes, that was the webkit repo (the test above was in a dir with 50k files).
Alas, I tried to create a nice profiling for the "git status .". In the Instruments application I can create a sampler, but I see no way to export it. The option to export the script as a dtrace script is greyed out in the menu.
From the sampler, it appears that the lstat calls still account for most of the time. I have uploaded a screenshot to http://ss.frim.nl/==759.png . It actually shows quite nicely when the lstats are being done -- it's when the CPU is idle. Next to the lstats, the read_tree_recursive is also called often.
- Pieter