Re: git-local-pull

From: Linus Torvalds <torvalds@osdl.org>
Date: Jun 27, 2005, 05:20 UTC
Message-ID: <Pine.LNX.4.58.0506262213220.19755@ppc970.osdl.org>
In-Reply-To: <20050626.200555.39157645.davem@davemloft.net>

On Sun, 26 Jun 2005, David S. Miller wrote:

Show 6 quoted lines

> 
> I tried to start using git-clone-script to clone repositories
> locally.  It crunches on the disk for a couple of seconds,
> that's fine, but then I notice the disk activity stop and
> git-local-pull becomes cpu bound and grows to 80MB in size
> over the course of 5 minutes.

Try adding "-l" to the git-local-pull command line, to make it do hardlinks by default (and fall back to a copy if that fails).

> Is this a side effect of the new pack/unpack stuff?

Nope, nothing uses that yet.

I think the 80MB is just because it built up the object relationship for the whole kernel db there. I don't know what the 5 minutes of CPU are, though, it should just be copying stuff (and at 250MB in and 250MB out, that's a lot of stuff to copy, but it shouldn't be five minutes worth, especially if it's all cached and you see a pegged CPU).

That copy is stupid, btw. Doing a "mmap + write" is a lot slower than doing a "while (data) { read(8kB) + write(8kB) }" loop. The read() will generally do better read-ahead and the small read/write will be better for the caches. But that still doesn't explain 5 minutes worth.

> Compared to what this thing is doing, manually symlinking
> the object database, copying over the HEAD, and building
> the index file is significantly faster.

Well, with "-l" that is what git-local-pull will do too. And if you want symlinks, use "-s". Or use both, in which case it will try a hardlink first, and then a symlink if it gets an EXDEV.

		Linus