# git-local-pull 4 messages from 2005-06-27 to 2005-06-27. Participants: David S. Miller, Linus Torvalds, Jeff Garzik. Thread: https://gitlist.dev/t/1031 ## David S. Miller, 2005-06-27 03:05 Subject: git-local-pull Message-ID: <20050626.200555.39157645.davem@davemloft.net> URL: https://gitlist.dev/e/20050626.200555.39157645.davem%40davemloft.net ``` I tried to start using git-clone-script to clone repositories locally. It crunches on the disk for a couple of seconds, that's fine, but then I notice the disk activity stop and git-local-pull becomes cpu bound and grows to 80MB in size over the course of 5 minutes. Is this a side effect of the new pack/unpack stuff? Compared to what this thing is doing, manually symlinking the object database, copying over the HEAD, and building the index file is significantly faster. Actually, git-clone-script didn't build an index file. So the compute time definitely came from something else entirely. ``` ## Linus Torvalds, 2005-06-27 05:20 Subject: Re: git-local-pull Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.58.0506262213220.19755%40ppc970.osdl.org In-Reply-To: <20050626.200555.39157645.davem@davemloft.net> ``` On Sun, 26 Jun 2005, David S. Miller wrote: > > I tried to start using git-clone-script to clone repositories > locally. It crunches on the disk for a couple of seconds, > that's fine, but then I notice the disk activity stop and > git-local-pull becomes cpu bound and grows to 80MB in size > over the course of 5 minutes. Try adding "-l" to the git-local-pull command line, to make it do hardlinks by default (and fall back to a copy if that fails). > Is this a side effect of the new pack/unpack stuff? Nope, nothing uses that yet. I think the 80MB is just because it built up the object relationship for the whole kernel db there. I don't know what the 5 minutes of CPU are, though, it should just be copying stuff (and at 250MB in and 250MB out, that's a lot of stuff to copy, but it shouldn't be five minutes worth, especially if it's all cached and you see a pegged CPU). That copy is stupid, btw. Doing a "mmap + write" is a lot slower than doing a "while (data) { read(8kB) + write(8kB) }" loop. The read() will generally do better read-ahead and the small read/write will be better for the caches. But that still doesn't explain 5 minutes worth. > Compared to what this thing is doing, manually symlinking > the object database, copying over the HEAD, and building > the index file is significantly faster. Well, with "-l" that is what git-local-pull will do too. And if you want symlinks, use "-s". Or use both, in which case it will try a hardlink first, and then a symlink if it gets an EXDEV. Linus ``` ## Jeff Garzik, 2005-06-27 05:31 Subject: Re: git-local-pull Message-ID: <42BF8F2D.1070808@pobox.com> URL: https://gitlist.dev/e/42BF8F2D.1070808%40pobox.com In-Reply-To: ``` Linus Torvalds wrote: > > On Sun, 26 Jun 2005, David S. Miller wrote: > >>I tried to start using git-clone-script to clone repositories >>locally. It crunches on the disk for a couple of seconds, >>that's fine, but then I notice the disk activity stop and >>git-local-pull becomes cpu bound and grows to 80MB in size >>over the course of 5 minutes. > > > Try adding "-l" to the git-local-pull command line, to make it do > hardlinks by default (and fall back to a copy if that fails). > > >>Is this a side effect of the new pack/unpack stuff? > > > Nope, nothing uses that yet. > > I think the 80MB is just because it built up the object relationship for > the whole kernel db there. I don't know what the 5 minutes of CPU are, > though, it should just be copying stuff (and at 250MB in and 250MB out, > that's a lot of stuff to copy, but it shouldn't be five minutes worth, > especially if it's all cached and you see a pegged CPU). FWIW, I notice that git-diff-tree uses >100MB of RAM, on a large git-pull-script run. It's generating the diffstat output, AFAICS. Jeff ``` ## Linus Torvalds, 2005-06-27 18:46 Subject: Re: git-local-pull Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.58.0506271145410.19755%40ppc970.osdl.org In-Reply-To: <42BF8F2D.1070808@pobox.com> ``` On Mon, 27 Jun 2005, Jeff Garzik wrote: > > FWIW, I notice that git-diff-tree uses >100MB of RAM, on a large > git-pull-script run. It's generating the diffstat output, AFAICS. Ahh, yes, I should make the diffstat be conditional, not everybody wants it, and it's often the most expensive part of the pull.. Linus ```