Re: git performance

From: Matthieu Moy <matthieu.moy@imag.fr>
Date: Oct 23, 2008, 12:16 UTC
Message-ID: <vpq4p33pkv3.fsf@bauges.imag.fr>
In-Reply-To: <000901c93490$e0c40ed0$a24c2c70$@com>

"Edward Ned Harvey" <git@nedharvey.com> writes:

Show 13 quoted lines

>> Yes, it does stat all the files. How many files are you talking about,
>> and what platform?  From a warm cache on Linux, the 23,000 files kernel
>> repo takes about a tenth of a second to stat all files for me (and this
>> on a several year-old machine). And of course many operations don't
>> require stat'ing at all (like looking at logs, or diffs that don't
>> involve the working tree).
>
> No worries.  No solution can meet everyone's needs.
>
> I'm talking about 40-50,000 files, on multi-user production linux,
> which means the cache is never warm, except when I'm benchmarking.
> Specifically RHEL 4 with the files on NFS mount. Cold cache "svn st"
> takes ~10 mins. Warm cache 20-30 sec.

SVN does not only has to stat the files. It also has to read the stat-cache information wich is split in one .svn/ per directory in the working tree. Not sure which operation dominates the performance, though. Best is just to try.

> Out of curiosity, what are they talking about, when they say "git is
> fast?" Just the fact that it's all local disk, or is there more to
> it than that?

Not just local disk: bzr also works locally, and git is much faster on most operations (bzr status can now compete with git, but "git log" and "git commit" can be instantaneous where bzr take 1 minute for example).

For sure, doing most operations locally is the key to being fast, but Git has also been written so that the complexity of algorithms be as low as possible.

Show 5 quoted lines

> I could see - git would probably outperform perforce for versioning
> of large files (let's say iso files) to benefit from sustained local
> disk IO, while perforce would probably outperform anything I can
> think of, operating on thousands of tiny files, because it will
> never walk the tree.

Mercurial has an extension called "inotify" that avoids walking the disk too. AFAIK doesn't have an equivalent in Git (mostly because most people interested find git fast enough).

-- 
Matthieu