# Re: git fetch slow as molasses due to tag downloading 5 messages from 2006-12-14 to 2006-12-15. Participants: Jakub Narebski, Junio C Hamano, Johannes Schindelin, Han-Wen Nienhuys. Thread: https://gitlist.dev/t/43175 ## Han-Wen Nienhuys, 2006-12-14 15:40 Subject: git fetch slow as molasses due to tag downloading Message-ID: URL: https://gitlist.dev/e/elrr8j%24a02%241%40sea.gmane.org ``` Hello, just upgraded our autobuilder from 1.4.3 to 1.4.4.2. Now, our standard download command comes to a complete halt. Judging from the "ps -ef" apparently, it does git-show-ref --verify --quiet -- [TAG] This is done for every one of the 1500 tags that are in my repository. At approx 20 tags per second this takes an awful lot of time. 1. Is this necessary? 2. Is this efficient? Wouldn't doing all tags in a single git-show-ref invocation be potentially quicker? -- Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen ``` ## Johannes Schindelin, 2006-12-14 16:05 Subject: Re: git fetch slow as molasses due to tag downloading Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.63.0612141701440.3635%40wbgn013.biozentrum.uni-wuerzburg.de In-Reply-To: ``` Hi, On Thu, 14 Dec 2006, Han-Wen Nienhuys wrote: > just upgraded our autobuilder from 1.4.3 to 1.4.4.2. > > Now, our standard download command comes to a complete halt. Judging > >from the "ps -ef" apparently, it does > > git-show-ref --verify --quiet -- [TAG] > > This is done for every one of the 1500 tags that are in my repository. > At approx 20 tags per second this takes an awful lot of time. > > 1. Is this necessary? Yes. The purpose is to check which tags have not yet been fetched. > 2. Is this efficient? Wouldn't doing all tags in a single git-show-ref > invocation be potentially quicker? It is not efficient. But it cannot be solved like you propose, since it is inside a loop, and a "continue" is executed when the tag exists already. IMHO this should be solved as a filter: "git-show-ref --stdin --show-invalid". Thus, git does not have to traverse _every_ ref for _every_ incoming tag. Ciao, Dscho ``` ## Jakub Narebski, 2006-12-15 09:58 Subject: Re: git fetch slow as molasses due to tag downloading Message-ID: URL: https://gitlist.dev/e/eltrg8%24b4q%242%40sea.gmane.org In-Reply-To: ``` Johannes Schindelin wrote: > IMHO this should be solved as a filter: "git-show-ref --stdin > --show-invalid". Thus, git does not have to traverse _every_ ref for > _every_ incoming tag. Can you make it so? TIA. -- Jakub Narebski Warsaw, Poland ShadeHawk on #git ``` ## Junio C Hamano, 2006-12-15 21:55 Subject: Re: git fetch slow as molasses due to tag downloading Message-ID: <7vtzzwn7su.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vtzzwn7su.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` Johannes Schindelin writes: > IMHO this should be solved as a filter: "git-show-ref --stdin > --show-invalid". Thus, git does not have to traverse _every_ ref for > _every_ incoming tag. That sounds like a too specialized hack to me. We should first speed up the general "--verify $ref" case; as you corrected me earlier it still has a useless loop. If it is still too slow (which I suspect it could be the case, with fork+exec overhead), we should _also_ pursue the filter approach, but even then I think "filter out the valid ones" is a specialized hack, if you mean "show only the invalid one's names" by --show-invalid. It would make sense to do $ git show-ref --show-invalid v1.0.0 v2.6.18 f665776185ad074b236c00751d666da7d1977dbe refs/tags/v1.0.0 - refs/tags/v2.6.18 (and its equivalent to take refs from --stdin) though. ``` ## Jakub Narebski, 2006-12-15 22:27 Subject: Re: git fetch slow as molasses due to tag downloading Message-ID: URL: https://gitlist.dev/e/elv7av%24n0%241%40sea.gmane.org In-Reply-To: <7vtzzwn7su.fsf@assigned-by-dhcp.cox.net> ``` Junio C Hamano wrote: > Johannes Schindelin writes: > >> IMHO this should be solved as a filter: "git-show-ref --stdin >> --show-invalid". Thus, git does not have to traverse _every_ ref for >> _every_ incoming tag. > > That sounds like a too specialized hack to me. We should first > speed up the general "--verify $ref" case; as you corrected me > earlier it still has a useless loop. > > If it is still too slow (which I suspect it could be the case, > with fork+exec overhead), we should _also_ pursue the filter > approach, but even then I think "filter out the valid ones" is a > specialized hack, if you mean "show only the invalid one's > names" by --show-invalid. > > It would make sense to do > > $ git show-ref --show-invalid v1.0.0 v2.6.18 > f665776185ad074b236c00751d666da7d1977dbe refs/tags/v1.0.0 > - refs/tags/v2.6.18 > > (and its equivalent to take refs from --stdin) though. Nice idea. And having '-' is probably better than using 00000000000000000000000000000000000000000 as for non-existing objects. -- Jakub Narebski Warsaw, Poland ShadeHawk on #git ```