{"thread":{"id":"1031","subject":"git-local-pull","startedAt":"2005-06-27T03:05:55Z","lastAt":"2005-06-27T18:46:11Z","messageCount":4,"participants":["David S. Miller","Linus Torvalds","Jeff Garzik"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"5296","messageId":"20050626.200555.39157645.davem@davemloft.net","threadId":"1031","inReplyTo":null,"subject":"git-local-pull","fromName":"David S. Miller","fromEmail":"davem@davemloft.net","sentAt":"2005-06-27T03:05:55Z","receivedAt":"2005-06-27T03:05:55Z","isPatch":false,"sender":{"key":"davem@davemloft.net","avatar":null},"body":"\nI tried to start using git-clone-script to clone repositories\nlocally.  It crunches on the disk for a couple of seconds,\nthat's fine, but then I notice the disk activity stop and\ngit-local-pull becomes cpu bound and grows to 80MB in size\nover the course of 5 minutes.\n\nIs this a side effect of the new pack/unpack stuff?\n\nCompared to what this thing is doing, manually symlinking\nthe object database, copying over the HEAD, and building\nthe index file is significantly faster.\n\nActually, git-clone-script didn't build an index file.\nSo the compute time definitely came from something else\nentirely.\n"},{"id":"5297","messageId":"Pine.LNX.4.58.0506262213220.19755@ppc970.osdl.org","threadId":"1031","inReplyTo":"20050626.200555.39157645.davem@davemloft.net","subject":"Re: git-local-pull","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-06-27T05:20:30Z","receivedAt":"2005-06-27T05:20:30Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sun, 26 Jun 2005, David S. Miller wrote:\n> \n> I tried to start using git-clone-script to clone repositories\n> locally.  It crunches on the disk for a couple of seconds,\n> that's fine, but then I notice the disk activity stop and\n> git-local-pull becomes cpu bound and grows to 80MB in size\n> over the course of 5 minutes.\n\nTry adding \"-l\" to the git-local-pull command line, to make it do \nhardlinks by default (and fall back to a copy if that fails).\n\n> Is this a side effect of the new pack/unpack stuff?\n\nNope, nothing uses that yet. \n\nI think the 80MB is just because it built up the object relationship for \nthe whole kernel db there. I don't know what the 5 minutes of CPU are, \nthough, it should just be copying stuff (and at 250MB in and 250MB out, \nthat's a lot of stuff to copy, but it shouldn't be five minutes worth, \nespecially if it's all cached and you see a pegged CPU).\n\nThat copy is stupid, btw. Doing a \"mmap + write\" is a lot slower than\ndoing a \"while (data) { read(8kB) + write(8kB) }\" loop. The read() will\ngenerally do better read-ahead and the small read/write will be better for \nthe caches. But that still doesn't explain 5 minutes worth.\n\n> Compared to what this thing is doing, manually symlinking\n> the object database, copying over the HEAD, and building\n> the index file is significantly faster.\n\nWell, with \"-l\" that is what git-local-pull will do too. And if you want \nsymlinks, use \"-s\". Or use both, in which case it will try a hardlink \nfirst, and then a symlink if it gets an EXDEV.\n\n\t\tLinus\n"},{"id":"5298","messageId":"42BF8F2D.1070808@pobox.com","threadId":"1031","inReplyTo":"Pine.LNX.4.58.0506262213220.19755@ppc970.osdl.org","subject":"Re: git-local-pull","fromName":"Jeff Garzik","fromEmail":"jgarzik@pobox.com","sentAt":"2005-06-27T05:31:25Z","receivedAt":"2005-06-27T05:31:25Z","isPatch":false,"sender":{"key":"jgarzik@pobox.com","avatar":null},"body":"Linus Torvalds wrote:\n> \n> On Sun, 26 Jun 2005, David S. Miller wrote:\n> \n>>I tried to start using git-clone-script to clone repositories\n>>locally.  It crunches on the disk for a couple of seconds,\n>>that's fine, but then I notice the disk activity stop and\n>>git-local-pull becomes cpu bound and grows to 80MB in size\n>>over the course of 5 minutes.\n> \n> \n> Try adding \"-l\" to the git-local-pull command line, to make it do \n> hardlinks by default (and fall back to a copy if that fails).\n> \n> \n>>Is this a side effect of the new pack/unpack stuff?\n> \n> \n> Nope, nothing uses that yet. \n> \n> I think the 80MB is just because it built up the object relationship for \n> the whole kernel db there. I don't know what the 5 minutes of CPU are, \n> though, it should just be copying stuff (and at 250MB in and 250MB out, \n> that's a lot of stuff to copy, but it shouldn't be five minutes worth, \n> especially if it's all cached and you see a pegged CPU).\n\nFWIW, I notice that git-diff-tree uses >100MB of RAM, on a large \ngit-pull-script run.  It's generating the diffstat output, AFAICS.\n\n\tJeff\n"},{"id":"5301","messageId":"Pine.LNX.4.58.0506271145410.19755@ppc970.osdl.org","threadId":"1031","inReplyTo":"42BF8F2D.1070808@pobox.com","subject":"Re: git-local-pull","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2005-06-27T18:46:11Z","receivedAt":"2005-06-27T18:46:11Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 27 Jun 2005, Jeff Garzik wrote:\n> \n> FWIW, I notice that git-diff-tree uses >100MB of RAM, on a large \n> git-pull-script run.  It's generating the diffstat output, AFAICS.\n\nAhh, yes, I should make the diffstat be conditional, not everybody wants \nit, and it's often the most expensive part of the pull..\n\n\t\tLinus\n"}]}