Re: Git performance on OS X

From: Roman Shaposhnik <rvs@sun.com>
Date: Apr 20, 2008, 00:21 UTC
Message-ID: <F9D9143D-D849-4454-91CD-024D691652C3@sun.com>
In-Reply-To: <alpine.LFD.1.10.0804191658430.2779@woody.linux-foundation.org>

On Apr 19, 2008, at 5:06 PM, Linus Torvalds wrote:

Show 11 quoted lines

> On Sat, 19 Apr 2008, Roman Shaposhnik wrote:
>>>
>>> Is there any system-level profiler for OS X to get a clue where  
>>> that cost
>>> is, in case it's not the lstat() at all?
>>
>> If it happens on Leopard, DTrace would be a perfect way to query  
>> the system:
>
> Well, I'd really like to see a traditional _time_ profile, not system
> call counts.

Good point. I just thought your original question was simply about confirming a hunch of an enormous # of lstat syscalls. Now, if by _time_ profile you mean how much time gets spent in each syscall than the following should help (time is in nanoseconds):

$ dtrace -n 'syscall::*:entry /pid==$target/ { self->ts=timestamp; }  
syscall::*:return /pid==$target/ { @[probefunc]=sum(timestamp-self- 
 >ts); }' -c "echo Hello World"
dtrace: description 'syscall::*:entry ' matched 468 probes
Hello World
dtrace: pid 1400 has exited

   getpid                                                         1392
   sysi86                                                         2799
   getrlimit                                                      2918
   setcontext                                                     4273
   fstat64                                                        6220
   mmap                                                          15419
   munmap                                                        22593
   write                                                         27860
   ioctl                                                         30750

The script can be modified slightly to also profile all of the libc.

On the other hand, if by real system profile you mean a full fledged sampling of the application itself than I can suggest running Git under Shark, and not under Instruments. Although I'm extremly curious about *why* would you need a full fledged *application* level profile of Git as opposed to a profile of how Git interacts with an OS.

Show 13 quoted lines

> The system call profile is trivial - it's generally going to be pretty
> similar under OS X and Linux (modulo library differences, but git  
> doesn't
> really use any really complex libraries that would do system calls).
>
> The problem we've had in the past is that Linux is simply an order of
> magnitude faster (sometimes more) at some operations than OS X is, so
> issues that show up on OS X don't even show up on Linux.
>
> This was the case for doing lots of small "mmap()/munmap()" calls, for
> example, where we literally had a load where OS X was two orders of
> magnitude slower. We switched over to reading the files with "pread()"
> instead of mmap(), and that fixed that particular issue.

Totally understood!

Thanks, Roman.