{"thread":{"id":"7401","subject":"Git performance problems with many tags","startedAt":"2007-03-26T04:53:41Z","lastAt":"2007-03-26T06:13:07Z","messageCount":3,"participants":["Tim Allen","Shawn O. Pearce","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"37993","messageId":"20070326045341.GE10545@ws35.commsecure.com.au","threadId":"7401","inReplyTo":null,"subject":"Git performance problems with many tags","fromName":"Tim Allen","fromEmail":"tim@commsecure.com.au","sentAt":"2007-03-26T04:53:41Z","receivedAt":"2007-03-26T04:53:41Z","isPatch":false,"sender":{"key":"tim@commsecure.com.au","avatar":null},"body":"I'm not subscribed, please Cc me on replies.\n\nMy company is considering switching from CVS to a more branch-friendly\nversion-control tool, and so of course we've been playing with git.\nWe imported our CVS repository into git with git-cvsimport, which worked\nwell enough, and resulted in a tree about the same size as the official\nkernel repository: 454121 objects, 334977 deltas.\n\nHowever, operations like 'git-fetch' take much, much longer in our\nrepository than in the kernel repository: a git-fetch that pulls no\nupdates in the kernel repository takes 1.7s, while our repository\n(fetching from one repository to a clone on the same local disk) takes\nabout 20 seconds. After some experimentation, we discovered that\ndeleting all the 5557 imported CVS tags made things fast again.\n(Interestingly, \"git-fetch --no-tags\" was not appreciably quicker, while\nthe tags were still around)\n\nI searched the mailing list archives for similar problems, and the\nclosest thread I could find was this one:\n\n    http://thread.gmane.org/gmane.comp.version-control.git/20682/\n\n...however, that thread seems to have decided that large numbers of\nbinary files were the problem, which is not the case in our repository.\n\nDoes git have known scalability problems with large numbers of tags? Is\nthere anything we can do to mitigate this slowdown, apart from just not\nusing git's tag feature at all? Are there any details I've overlooked or\nmisunderstood?\n\nTim Allen\n"},{"id":"37994","messageId":"20070326060731.GF13247@spearce.org","threadId":"7401","inReplyTo":"20070326045341.GE10545@ws35.commsecure.com.au","subject":"Re: Git performance problems with many tags","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-03-26T06:07:31Z","receivedAt":"2007-03-26T06:07:31Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Tim Allen <tim@commsecure.com.au> wrote:\n> However, operations like 'git-fetch' take much, much longer in our\n> repository than in the kernel repository: a git-fetch that pulls no\n> updates in the kernel repository takes 1.7s, while our repository\n> (fetching from one repository to a clone on the same local disk) takes\n> about 20 seconds. After some experimentation, we discovered that\n> deleting all the 5557 imported CVS tags made things fast again.\n\nYes.  git-fetch in the current stable versions is a Bourne shell\nscript.  Its not very fast as it loops through the refs (tags)\nthat the two ends have.\n\nThere is work in under development (and being tested) that improves\nthis by converting some of the critical parts to C.\n\n> (Interestingly, \"git-fetch --no-tags\" was not appreciably quicker, while\n> the tags were still around)\n\nYes, because that swtich didn't bypass that section of the fetch code\nwhere the slowdown is occuring.\n\n> Does git have known scalability problems with large numbers of tags?\n\nYup.\n\n> Is\n> there anything we can do to mitigate this slowdown, apart from just not\n> using git's tag feature at all?\n\nUpgrade to the upcoming 1.5.1 release.  Junio recently tagged\n1.5.1-rc1.  You can get it by cloning git.git and building the\n'master' branch.  It is quite stable.  I would encourage you to\ngive it a try.\n\n-- \nShawn.\n"},{"id":"37995","messageId":"7vwt14zg5o.fsf@assigned-by-dhcp.cox.net","threadId":"7401","inReplyTo":"20070326045341.GE10545@ws35.commsecure.com.au","subject":"Re: Git performance problems with many tags","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-03-26T06:13:07Z","receivedAt":"2007-03-26T06:13:07Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Tim Allen <tim@commsecure.com.au> writes:\n\n> However, operations like 'git-fetch' take much, much longer in our\n> repository than in the kernel repository: a git-fetch that pulls no\n> updates in the kernel repository takes 1.7s, while our repository\n> (fetching from one repository to a clone on the same local disk) takes\n> about 20 seconds.\n> ...\n> Does git have known scalability problems with large numbers of tags? Is\n> there anything we can do to mitigate this slowdown, apart from just not\n> using git's tag feature at all? Are there any details I've overlooked or\n> misunderstood?\n\nAn optimization for \"fetching between repositories with insane\nnumber of refs\" problem was merged after v1.5.0, so if you try\nthe current tip of 'master' (which happens to be at v1.5.1-rc2\nright now), I suspect you may see significant improvements.\n"}]}