# Re: git clone stable-2.6.25.y fails over HTTP 12 messages from 2008-06-04 to 2008-06-06. Participants: Greg KH, Junio C Hamano, Jakub Narebski, Daniel Barkalow. Thread: https://gitlist.dev/t/13793 ## Greg KH, 2008-06-04 15:45 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <20080604154523.GA25747@kroah.com> URL: https://gitlist.dev/e/20080604154523.GA25747%40kroah.com In-Reply-To: <200806041511.m54FBPL9006783@pogo.cesa.opbu.xerox.com> ``` On Wed, Jun 04, 2008 at 08:11:24AM -0700, Andrew Klossner wrote: > Greg wrote: > > > I don't think I remembered to run 'git-update-server-info' on the > > repository. I just did it, let me know if that works better or not. > > I get the same error. > > -=- Andrew > > % git clone http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2 .6.25.y.git > Initialized empty Git repository in /work/andrew/http-git/linux-2.6.25.y/.git/ > Getting alternates list for http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git > Also look at http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ > Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git > Getting index for pack 6506d57e29d10a0ed9cca269f97296606aeb0b5b > Getting index for pack 067caec25377d65e503d626a6c4f3f7e392406a3 > Getting index for pack 5434817c4def8a894fe4d61f55e8e4117485cbe7 > Getting pack 067caec25377d65e503d626a6c4f3f7e392406a3 > which contains a82a9bfebc74862798a3f6949fe6f6a959c1fd4e > walk a82a9bfebc74862798a3f6949fe6f6a959c1fd4e > walk 15c2419876173c417a119b5665ecc62520e50a6c > Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ > error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git > Cannot obtain needed blob fdcce40226d7d4273a08cc4ef84bb25755a710a4 > while processing commit 15c2419876173c417a119b5665ecc62520e50a6c. Hm, I really do not know, and I can duplicate this myself as well. Does anyone know what's going on here? The stable git trees are created by doing: git clone -s --bare from Linus's main tree to the stable tree repo location. It looks like git over http isn't following the link somehow to the main repo? Or is what is necessary to do is to clone Linus's tree and then pull from the stable tree in this kind of situation (where the user can only use git over http?) thanks, greg k-h ``` ## Junio C Hamano, 2008-06-04 17:09 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <7vr6bdxh3l.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vr6bdxh3l.fsf%40gitster.siamese.dyndns.org In-Reply-To: <20080604154523.GA25747@kroah.com> ``` Greg KH writes: >> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ >> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git > ... > It looks like git over http isn't following the link somehow to the main > repo? Yeah, see above. www.kernel.org does not like to serve /home/ftp/pub/scm path; it wants to serve it as /pub/scm/linux instead. Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/... but in 2.6.25.y it has extra /home/ftp in front of it. A quick fix is obviously to remove the extra prefix but it makes me wonder how these two "stable" repositories were prepared differently. Could this be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's repository slightly differently between them? ``` ## Jakub Narebski, 2008-06-04 18:00 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: URL: https://gitlist.dev/e/m363spyta3.fsf%40localhost.localdomain In-Reply-To: <7vr6bdxh3l.fsf@gitster.siamese.dyndns.org> ``` Junio C Hamano writes: > Greg KH writes: > >>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ >>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git >> ... >> It looks like git over http isn't following the link somehow to the main >> repo? > > Yeah, see above. www.kernel.org does not like to serve /home/ftp/pub/scm > path; it wants to serve it as /pub/scm/linux instead. > > Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/... > but in 2.6.25.y it has extra /home/ftp in front of it. > > A quick fix is obviously to remove the extra prefix but it makes me wonder > how these two "stable" repositories were prepared differently. Could this > be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's > repository slightly differently between them? Errr... isn't it the case for objects/info/http-alternates ? -- Jakub Narebski Poland ShadeHawk on #git ``` ## Junio C Hamano, 2008-06-04 18:02 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <7v63spxem0.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7v63spxem0.fsf%40gitster.siamese.dyndns.org In-Reply-To: ``` Jakub Narebski writes: > Junio C Hamano writes: > >> Greg KH writes: >> >>>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ >>>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git >>> ... >>> It looks like git over http isn't following the link somehow to the main >>> repo? >> >> Yeah, see above. www.kernel.org does not like to serve /home/ftp/pub/scm >> path; it wants to serve it as /pub/scm/linux instead. >> >> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/... >> but in 2.6.25.y it has extra /home/ftp in front of it. >> >> A quick fix is obviously to remove the extra prefix but it makes me wonder >> how these two "stable" repositories were prepared differently. Could this >> be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's >> repository slightly differently between them? > > Errr... isn't it the case for objects/info/http-alternates ? You do not have to use that http specific hack. On these machines, /pub actually points at /home/ftp/pub with a symlink. ``` ## Greg KH, 2008-06-05 04:03 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <20080605040315.GA21798@kroah.com> URL: https://gitlist.dev/e/20080605040315.GA21798%40kroah.com In-Reply-To: <7vr6bdxh3l.fsf@gitster.siamese.dyndns.org> ``` On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote: > Greg KH writes: > > >> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ > >> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git > > ... > > It looks like git over http isn't following the link somehow to the main > > repo? > > Yeah, see above. www.kernel.org does not like to serve /home/ftp/pub/scm > path; it wants to serve it as /pub/scm/linux instead. > > Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/... > but in 2.6.25.y it has extra /home/ftp in front of it. > > A quick fix is obviously to remove the extra prefix but it makes me wonder > how these two "stable" repositories were prepared differently. Could this > be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's > repository slightly differently between them? That's odd. Chris was the one that created the .24 tree and I created the .25 tree, so we must have done so differently, sorry. I've edited the alternates file, so it should work now, sorry about that. thanks, greg k-h ``` ## Junio C Hamano, 2008-06-05 04:12 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <7vabi05xl1.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vabi05xl1.fsf%40gitster.siamese.dyndns.org In-Reply-To: <20080605040315.GA21798@kroah.com> ``` Greg KH writes: > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote: > >> A quick fix is obviously to remove the extra prefix but it makes me wonder >> how these two "stable" repositories were prepared differently. Could this >> be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's >> repository slightly differently between them? > > That's odd. > > Chris was the one that created the .24 tree and I created the .25 tree, > so we must have done so differently, sorry. No need to be sorry. Thanks for raising the issue up. I think I have one possible explanation. When "git clone" was rewritten in C some time ago, it started to dereference symlinks, and newer git would dereference /pub -> /home/ftp/pub when you do: git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y while the scripted version used to keep /pub/scm path. A bit further digging is necessary to make sure if we can fix this easily, but I suspect this is worth fixing. ``` ## Greg KH, 2008-06-05 05:08 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <20080605050855.GA22825@kroah.com> URL: https://gitlist.dev/e/20080605050855.GA22825%40kroah.com In-Reply-To: <7vabi05xl1.fsf@gitster.siamese.dyndns.org> ``` On Wed, Jun 04, 2008 at 09:12:42PM -0700, Junio C Hamano wrote: > Greg KH writes: > > > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote: > > > >> A quick fix is obviously to remove the extra prefix but it makes me wonder > >> how these two "stable" repositories were prepared differently. Could this > >> be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's > >> repository slightly differently between them? > > > > That's odd. > > > > Chris was the one that created the .24 tree and I created the .25 tree, > > so we must have done so differently, sorry. > > No need to be sorry. Thanks for raising the issue up. > > I think I have one possible explanation. When "git clone" was rewritten > in C some time ago, it started to dereference symlinks, and newer git > would dereference /pub -> /home/ftp/pub when you do: > > git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y > > while the scripted version used to keep /pub/scm path. > > A bit further digging is necessary to make sure if we can fix this easily, > but I suspect this is worth fixing. I can duplicate what I did, and it did it again: on master.kernel.org: $ cd /pub/scm/linux/kernel/git/stable/ $ git clone -s --bare ../torvalds/linux-2.6.git/ test $ cat test/objects/info/alternates /home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects I'm guessing this is because /pub/ is a symlink: $ ls -l /pub lrwxrwxrwx 1 root root 13 Feb 4 2006 /pub -> /home/ftp/pub $ git --version git version 1.5.5.3 thanks, greg k-h ``` ## Daniel Barkalow, 2008-06-05 20:53 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: URL: https://gitlist.dev/e/alpine.LNX.1.00.0806051546480.19665%40iabervon.org In-Reply-To: <20080605050855.GA22825@kroah.com> ``` On Wed, 4 Jun 2008, Greg KH wrote: > On Wed, Jun 04, 2008 at 09:12:42PM -0700, Junio C Hamano wrote: > > Greg KH writes: > > > > > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote: > > > > > >> A quick fix is obviously to remove the extra prefix but it makes me wonder > > >> how these two "stable" repositories were prepared differently. Could this > > >> be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's > > >> repository slightly differently between them? > > > > > > That's odd. > > > > > > Chris was the one that created the .24 tree and I created the .25 tree, > > > so we must have done so differently, sorry. > > > > No need to be sorry. Thanks for raising the issue up. > > > > I think I have one possible explanation. When "git clone" was rewritten > > in C some time ago, it started to dereference symlinks, and newer git > > would dereference /pub -> /home/ftp/pub when you do: > > > > git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y > > > > while the scripted version used to keep /pub/scm path. > > > > A bit further digging is necessary to make sure if we can fix this easily, > > but I suspect this is worth fixing. > > I can duplicate what I did, and it did it again: > on master.kernel.org: > $ cd /pub/scm/linux/kernel/git/stable/ > $ git clone -s --bare ../torvalds/linux-2.6.git/ test > $ cat test/objects/info/alternates > /home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects > > I'm guessing this is because /pub/ is a symlink: > $ ls -l /pub > lrwxrwxrwx 1 root root 13 Feb 4 2006 /pub -> /home/ftp/pub We're using getcwd() to make the absolute path, but it looks like we should check whether $PWD is set and names the same directory, and, if so, base on that instead. Actually, make_absolute_path() seems to go to an awful lot of trouble to make sure that there aren't symlinks in the resulting path. Is this actually necessary or even desireable for some reason? Should clone be just pasting $PWD (or, failing that, getcwd()) and the given path, and using that to identify the repo to clone? -Daniel *This .sig left intentionally blank* ``` ## Junio C Hamano, 2008-06-05 21:26 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <7vzlpzzi8c.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vzlpzzi8c.fsf%40gitster.siamese.dyndns.org In-Reply-To: ``` Daniel Barkalow writes: > On Wed, 4 Jun 2008, Greg KH wrote: > ... >> I can duplicate what I did, and it did it again: >> on master.kernel.org: >> $ cd /pub/scm/linux/kernel/git/stable/ >> $ git clone -s --bare ../torvalds/linux-2.6.git/ test >> $ cat test/objects/info/alternates >> /home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects >> >> I'm guessing this is because /pub/ is a symlink: >> $ ls -l /pub >> lrwxrwxrwx 1 root root 13 Feb 4 2006 /pub -> /home/ftp/pub > > We're using getcwd() to make the absolute path, but it looks like we > should check whether $PWD is set and names the same directory, and, if so, > base on that instead. > > Actually, make_absolute_path() seems to go to an awful lot of trouble to > make sure that there aren't symlinks in the resulting path. Is this > actually necessary or even desireable for some reason? Should clone be > just pasting $PWD (or, failing that, getcwd()) and the given path, and > using that to identify the repo to clone? I am not 100% sympathetic if ../torvalds/linux-2.6.git/ was what caused this. Perhaps 30% or so ;-). However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would result in the same breakage, and that is something I consider gravely broken. If both are fixed by rethinking the make_absolute_path(), that would be good. ``` ## Greg KH, 2008-06-06 20:48 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <20080606204814.GE11209@kroah.com> URL: https://gitlist.dev/e/20080606204814.GE11209%40kroah.com In-Reply-To: <7vzlpzzi8c.fsf@gitster.siamese.dyndns.org> ``` On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote: > Daniel Barkalow writes: > > > On Wed, 4 Jun 2008, Greg KH wrote: > > ... > >> I can duplicate what I did, and it did it again: > >> on master.kernel.org: > >> $ cd /pub/scm/linux/kernel/git/stable/ > >> $ git clone -s --bare ../torvalds/linux-2.6.git/ test > >> $ cat test/objects/info/alternates > >> /home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects > >> > >> I'm guessing this is because /pub/ is a symlink: > >> $ ls -l /pub > >> lrwxrwxrwx 1 root root 13 Feb 4 2006 /pub -> /home/ftp/pub > > > > We're using getcwd() to make the absolute path, but it looks like we > > should check whether $PWD is set and names the same directory, and, if so, > > base on that instead. > > > > Actually, make_absolute_path() seems to go to an awful lot of trouble to > > make sure that there aren't symlinks in the resulting path. Is this > > actually necessary or even desireable for some reason? Should clone be > > just pasting $PWD (or, failing that, getcwd()) and the given path, and > > using that to identify the repo to clone? > > I am not 100% sympathetic if ../torvalds/linux-2.6.git/ was what caused > this. Perhaps 30% or so ;-). I agree, if I did something stupid like this, that's my fault. I didn't realize that it was stupid, so I guess a simple warning, "don't use relative paths when cloning with alternates" would have told me so. > However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would > result in the same breakage, and that is something I consider gravely > broken. Yeah, especially as I never realized that /pub/ was a symlink in the first place :) thanks, greg k-h ``` ## Junio C Hamano, 2008-06-06 21:00 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <7vd4muth2e.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vd4muth2e.fsf%40gitster.siamese.dyndns.org In-Reply-To: <20080606204814.GE11209@kroah.com> ``` Greg KH writes: > On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote: > >> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would >> result in the same breakage, and that is something I consider gravely >> broken. > > Yeah, especially as I never realized that /pub/ was a symlink in the > first place :) And it was even worse than that. Even if you knew /pub was a symlink, you could not have done anything about it, short of editing the alternates file in the resulting repository by hand. Up until now, that is. Daniel's patch seems to fix the issue for me and it will be included in tonight's pushout among other fixes. Thanks. ``` ## Greg KH, 2008-06-06 21:54 Subject: Re: git clone stable-2.6.25.y fails over HTTP Message-ID: <20080606215359.GA4914@kroah.com> URL: https://gitlist.dev/e/20080606215359.GA4914%40kroah.com In-Reply-To: <7vd4muth2e.fsf@gitster.siamese.dyndns.org> ``` On Fri, Jun 06, 2008 at 02:00:09PM -0700, Junio C Hamano wrote: > Greg KH writes: > > > On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote: > > > >> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would > >> result in the same breakage, and that is something I consider gravely > >> broken. > > > > Yeah, especially as I never realized that /pub/ was a symlink in the > > first place :) > > And it was even worse than that. Even if you knew /pub was a symlink, you > could not have done anything about it, short of editing the alternates > file in the resulting repository by hand. Heh, true :) > Up until now, that is. > > Daniel's patch seems to fix the issue for me and it will be included in > tonight's pushout among other fixes. Great, thanks a lot for the fix everyone. greg k-h ```