{"thread":{"id":"43175","subject":"Re: git fetch slow as molasses due to tag downloading","startedAt":"2006-12-14T15:40:14Z","lastAt":"2006-12-15T22:27:03Z","messageCount":5,"participants":["Jakub Narebski","Junio C Hamano","Johannes Schindelin","Han-Wen Nienhuys"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"298069","messageId":"elrr8j$a02$1@sea.gmane.org","threadId":"43175","inReplyTo":null,"subject":"git fetch slow as molasses due to tag downloading","fromName":"Han-Wen Nienhuys","fromEmail":"hanwen@xs4all.nl","sentAt":"2006-12-14T15:40:14Z","receivedAt":"2006-12-14T15:40:14Z","isPatch":false,"sender":{"key":"hanwen@google.com","avatar":"https://avatars.githubusercontent.com/u/31547?v=4"},"body":"\nHello,\n\njust upgraded our autobuilder from 1.4.3 to 1.4.4.2.\n\nNow, our standard download command comes to a complete halt. Judging\nfrom the \"ps -ef\" apparently, it does \n\n git-show-ref --verify --quiet -- [TAG]\n\nThis is done for every one of the 1500 tags that are in my repository. \nAt approx 20 tags per second this takes an awful lot of time. \n\n1. Is this necessary? \n\n2. Is this efficient? Wouldn't doing all tags in a single git-show-ref\ninvocation be potentially quicker?\n\n\n-- \n Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen\n"},{"id":"296474","messageId":"Pine.LNX.4.63.0612141701440.3635@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"43175","inReplyTo":"elrr8j$a02$1@sea.gmane.org","subject":"Re: git fetch slow as molasses due to tag downloading","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-12-14T16:05:14Z","receivedAt":"2006-12-14T16:05:14Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 14 Dec 2006, Han-Wen Nienhuys wrote:\n\n> just upgraded our autobuilder from 1.4.3 to 1.4.4.2.\n> \n> Now, our standard download command comes to a complete halt. Judging\n> >from the \"ps -ef\" apparently, it does \n> \n> git-show-ref --verify --quiet -- [TAG]\n> \n> This is done for every one of the 1500 tags that are in my repository. \n> At approx 20 tags per second this takes an awful lot of time. \n> \n> 1. Is this necessary? \n\nYes. The purpose is to check which tags have not yet been fetched.\n\n> 2. Is this efficient? Wouldn't doing all tags in a single git-show-ref \n> invocation be potentially quicker?\n\nIt is not efficient. But it cannot be solved like you propose, since it is \ninside a loop, and a \"continue\" is executed when the tag exists already.\n\nIMHO this should be solved as a filter: \"git-show-ref --stdin \n--show-invalid\". Thus, git does not have to traverse _every_ ref for \n_every_ incoming tag.\n\nCiao,\nDscho\n"},{"id":"294164","messageId":"eltrg8$b4q$2@sea.gmane.org","threadId":"43175","inReplyTo":"Pine.LNX.4.63.0612141701440.3635@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: git fetch slow as molasses due to tag downloading","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-15T09:58:54Z","receivedAt":"2006-12-15T09:58:54Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Johannes Schindelin wrote:\n\n> IMHO this should be solved as a filter: \"git-show-ref --stdin \n> --show-invalid\". Thus, git does not have to traverse _every_ ref for \n> _every_ incoming tag.\n\nCan you make it so? TIA.\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"},{"id":"295505","messageId":"7vtzzwn7su.fsf@assigned-by-dhcp.cox.net","threadId":"43175","inReplyTo":"Pine.LNX.4.63.0612141701440.3635@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: git fetch slow as molasses due to tag downloading","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-12-15T21:55:13Z","receivedAt":"2006-12-15T21:55:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> IMHO this should be solved as a filter: \"git-show-ref --stdin \n> --show-invalid\". Thus, git does not have to traverse _every_ ref for \n> _every_ incoming tag.\n\nThat sounds like a too specialized hack to me. We should first\nspeed up the general \"--verify $ref\" case; as you corrected me\nearlier it still has a useless loop.\n\nIf it is still too slow (which I suspect it could be the case,\nwith fork+exec overhead), we should _also_ pursue the filter\napproach, but even then I think \"filter out the valid ones\" is a\nspecialized hack, if you mean \"show only the invalid one's\nnames\" by --show-invalid.\n\nIt would make sense to do\n\n $ git show-ref --show-invalid v1.0.0 v2.6.18\n f665776185ad074b236c00751d666da7d1977dbe refs/tags/v1.0.0\n - refs/tags/v2.6.18\n\n(and its equivalent to take refs from --stdin) though.\n\n"},{"id":"293974","messageId":"elv7av$n0$1@sea.gmane.org","threadId":"43175","inReplyTo":"7vtzzwn7su.fsf@assigned-by-dhcp.cox.net","subject":"Re: git fetch slow as molasses due to tag downloading","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-12-15T22:27:03Z","receivedAt":"2006-12-15T22:27:03Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n>> IMHO this should be solved as a filter: \"git-show-ref --stdin \n>> --show-invalid\". Thus, git does not have to traverse _every_ ref for \n>> _every_ incoming tag.\n> \n> That sounds like a too specialized hack to me. We should first\n> speed up the general \"--verify $ref\" case; as you corrected me\n> earlier it still has a useless loop.\n> \n> If it is still too slow (which I suspect it could be the case,\n> with fork+exec overhead), we should _also_ pursue the filter\n> approach, but even then I think \"filter out the valid ones\" is a\n> specialized hack, if you mean \"show only the invalid one's\n> names\" by --show-invalid.\n> \n> It would make sense to do\n> \n> $ git show-ref --show-invalid v1.0.0 v2.6.18\n> f665776185ad074b236c00751d666da7d1977dbe refs/tags/v1.0.0\n> - refs/tags/v2.6.18\n> \n> (and its equivalent to take refs from --stdin) though.\n\nNice idea. And having '-' is probably better than using\n00000000000000000000000000000000000000000 as for non-existing objects.\n\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n\n"}]}