threads / discuss / 13793

Re: git clone stable-2.6.25.y fails over HTTP

Subject: Re: git clone stable-2.6.25.y fails over HTTP

## tl;dr

12 messages between Jun 4, 2008 and Jun 6, 2008.

replies: 11people: 4as markdown or json

Greg KH· Jun 4, 2008, 15:45 UTC · lore
On Wed, Jun 04, 2008 at 08:11:24AM -0700, Andrew Klossner wrote:
Show 25 quoted lines
> Greg wrote:
> 
> > I don't think I remembered to run 'git-update-server-info' on the
> > repository.  I just did it, let me know if that works better or not.
> 
> I get the same error.
> 
>   -=- Andrew
> 
> % git clone http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2 .6.25.y.git
> Initialized empty Git repository in /work/andrew/http-git/linux-2.6.25.y/.git/
> Getting alternates list for http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> Also look at http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> Getting index for pack 6506d57e29d10a0ed9cca269f97296606aeb0b5b
> Getting index for pack 067caec25377d65e503d626a6c4f3f7e392406a3
> Getting index for pack 5434817c4def8a894fe4d61f55e8e4117485cbe7
> Getting pack 067caec25377d65e503d626a6c4f3f7e392406a3
>  which contains a82a9bfebc74862798a3f6949fe6f6a959c1fd4e
> walk a82a9bfebc74862798a3f6949fe6f6a959c1fd4e
> walk 15c2419876173c417a119b5665ecc62520e50a6c
> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> Cannot obtain needed blob fdcce40226d7d4273a08cc4ef84bb25755a710a4
> while processing commit 15c2419876173c417a119b5665ecc62520e50a6c.
Hm, I really do not know, and I can duplicate this myself as well.
<adding the git mailing list>
Does anyone know what's going on here?  The stable git trees are created
by doing:
	git clone -s --bare
from Linus's main tree to the stable tree repo location.

It looks like git over http isn't following the link somehow to the main repo?

Or is what is necessary to do is to clone Linus's tree and then pull from the stable tree in this kind of situation (where the user can only use git over http?)

thanks,
greg k-h
Junio C Hamano· Jun 4, 2008, 17:09 UTC · re: Greg KH · lore
Greg KH <greg@kroah.com> writes:
Show 5 quoted lines
>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> ...
> It looks like git over http isn't following the link somehow to the main
> repo?

Yeah, see above. www.kernel.org does not like to serve /home/ftp/pub/scm path; it wants to serve it as /pub/scm/linux instead.

Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/... but in 2.6.25.y it has extra /home/ftp in front of it.

A quick fix is obviously to remove the extra prefix but it makes me wonder how these two "stable" repositories were prepared differently. Could this be a regression in "git clone"? Or perhaps you "clone -s"'s Linus's repository slightly differently between them?

Jakub Narebski· Jun 4, 2008, 18:00 UTC · re: Junio C Hamano · lore
Junio C Hamano <gitster@pobox.com> writes:
Show 18 quoted lines
> Greg KH <greg@kroah.com> writes:
> 
>>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
>> ...
>> It looks like git over http isn't following the link somehow to the main
>> repo?
> 
> Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
> path; it wants to serve it as /pub/scm/linux instead.
> 
> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
> but in 2.6.25.y it has extra /home/ftp in front of it.
> 
> A quick fix is obviously to remove the extra prefix but it makes me wonder
> how these two "stable" repositories were prepared differently.  Could this
> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> repository slightly differently between them?
Errr... isn't it the case for objects/info/http-alternates ?
-- 
Jakub Narebski
Poland
ShadeHawk on #git
Junio C Hamano· Jun 4, 2008, 18:02 UTC · re: Jakub Narebski · lore
Jakub Narebski <jnareb@gmail.com> writes:
Show 22 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Greg KH <greg@kroah.com> writes:
>> 
>>>> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>>>> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
>>> ...
>>> It looks like git over http isn't following the link somehow to the main
>>> repo?
>> 
>> Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
>> path; it wants to serve it as /pub/scm/linux instead.
>> 
>> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
>> but in 2.6.25.y it has extra /home/ftp in front of it.
>> 
>> A quick fix is obviously to remove the extra prefix but it makes me wonder
>> how these two "stable" repositories were prepared differently.  Could this
>> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
>> repository slightly differently between them?
>
> Errr... isn't it the case for objects/info/http-alternates ?

You do not have to use that http specific hack. On these machines, /pub actually points at /home/ftp/pub with a symlink.

Greg KH· Jun 5, 2008, 04:03 UTC · re: Junio C Hamano · lore
On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
Show 18 quoted lines
> Greg KH <greg@kroah.com> writes:
> 
> >> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> >> error: Unable to find fdcce40226d7d4273a08cc4ef84bb25755a710a4 under http://www.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.25.y.git
> > ...
> > It looks like git over http isn't following the link somehow to the main
> > repo?
> 
> Yeah, see above.  www.kernel.org does not like to serve /home/ftp/pub/scm
> path; it wants to serve it as /pub/scm/linux instead.
> 
> Your objects/info/alternates in 2.6.24.y points at /pub/scm/linux/kernel/git/torvalds/...
> but in 2.6.25.y it has extra /home/ftp in front of it.
> 
> A quick fix is obviously to remove the extra prefix but it makes me wonder
> how these two "stable" repositories were prepared differently.  Could this
> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> repository slightly differently between them?
That's odd.

Chris was the one that created the .24 tree and I created the .25 tree, so we must have done so differently, sorry.

I've edited the alternates file, so it should work now, sorry about that.

thanks,
greg k-h
Junio C Hamano· Jun 5, 2008, 04:12 UTC · re: Greg KH · lore
Greg KH <greg@kroah.com> writes:
Show 11 quoted lines
> On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
>
>> A quick fix is obviously to remove the extra prefix but it makes me wonder
>> how these two "stable" repositories were prepared differently.  Could this
>> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
>> repository slightly differently between them?
>
> That's odd.
>
> Chris was the one that created the .24 tree and I created the .25 tree,
> so we must have done so differently, sorry.
No need to be sorry.  Thanks for raising the issue up.

I think I have one possible explanation. When "git clone" was rewritten in C some time ago, it started to dereference symlinks, and newer git would dereference /pub -> /home/ftp/pub when you do:

	git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y
while the scripted version used to keep /pub/scm path.

A bit further digging is necessary to make sure if we can fix this easily, but I suspect this is worth fixing.

Greg KH· Jun 5, 2008, 05:08 UTC · re: Junio C Hamano · lore
On Wed, Jun 04, 2008 at 09:12:42PM -0700, Junio C Hamano wrote:
Show 26 quoted lines
> Greg KH <greg@kroah.com> writes:
> 
> > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
> >
> >> A quick fix is obviously to remove the extra prefix but it makes me wonder
> >> how these two "stable" repositories were prepared differently.  Could this
> >> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> >> repository slightly differently between them?
> >
> > That's odd.
> >
> > Chris was the one that created the .24 tree and I created the .25 tree,
> > so we must have done so differently, sorry.
> 
> No need to be sorry.  Thanks for raising the issue up.
> 
> I think I have one possible explanation.  When "git clone" was rewritten
> in C some time ago, it started to dereference symlinks, and newer git
> would dereference /pub -> /home/ftp/pub when you do:
> 
> 	git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y
> 
> while the scripted version used to keep /pub/scm path.
> 
> A bit further digging is necessary to make sure if we can fix this easily,
> but I suspect this is worth fixing.
I can duplicate what I did, and it did it again:
	on master.kernel.org:
	$ cd /pub/scm/linux/kernel/git/stable/
	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
	$ cat test/objects/info/alternates
	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
I'm guessing this is because /pub/ is a symlink:
	$ ls -l /pub
	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub
	$ git --version
	git version 1.5.5.3
thanks,
greg k-h
Daniel Barkalow· Jun 5, 2008, 20:53 UTC · re: Greg KH · lore
On Wed, 4 Jun 2008, Greg KH wrote:
Show 38 quoted lines
> On Wed, Jun 04, 2008 at 09:12:42PM -0700, Junio C Hamano wrote:
> > Greg KH <greg@kroah.com> writes:
> > 
> > > On Wed, Jun 04, 2008 at 10:09:02AM -0700, Junio C Hamano wrote:
> > >
> > >> A quick fix is obviously to remove the extra prefix but it makes me wonder
> > >> how these two "stable" repositories were prepared differently.  Could this
> > >> be a regression in "git clone"?  Or perhaps you "clone -s"'s Linus's
> > >> repository slightly differently between them?
> > >
> > > That's odd.
> > >
> > > Chris was the one that created the .24 tree and I created the .25 tree,
> > > so we must have done so differently, sorry.
> > 
> > No need to be sorry.  Thanks for raising the issue up.
> > 
> > I think I have one possible explanation.  When "git clone" was rewritten
> > in C some time ago, it started to dereference symlinks, and newer git
> > would dereference /pub -> /home/ftp/pub when you do:
> > 
> > 	git clone -s --bare /pub/scm/linux/kernel/git/torvalds/linux-2.6/ linux-2.6.2x.y
> > 
> > while the scripted version used to keep /pub/scm path.
> > 
> > A bit further digging is necessary to make sure if we can fix this easily,
> > but I suspect this is worth fixing.
> 
> I can duplicate what I did, and it did it again:
> 	on master.kernel.org:
> 	$ cd /pub/scm/linux/kernel/git/stable/
> 	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
> 	$ cat test/objects/info/alternates
> 	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
> 
> I'm guessing this is because /pub/ is a symlink:
> 	$ ls -l /pub
> 	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub

We're using getcwd() to make the absolute path, but it looks like we should check whether $PWD is set and names the same directory, and, if so, base on that instead.

Actually, make_absolute_path() seems to go to an awful lot of trouble to make sure that there aren't symlinks in the resulting path. Is this actually necessary or even desireable for some reason? Should clone be just pasting $PWD (or, failing that, getcwd()) and the given path, and using that to identify the repo to clone?

	-Daniel
*This .sig left intentionally blank*
Junio C Hamano· Jun 5, 2008, 21:26 UTC · re: Daniel Barkalow · lore
Daniel Barkalow <barkalow@iabervon.org> writes:
Show 22 quoted lines
> On Wed, 4 Jun 2008, Greg KH wrote:
> ...
>> I can duplicate what I did, and it did it again:
>> 	on master.kernel.org:
>> 	$ cd /pub/scm/linux/kernel/git/stable/
>> 	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
>> 	$ cat test/objects/info/alternates
>> 	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
>> 
>> I'm guessing this is because /pub/ is a symlink:
>> 	$ ls -l /pub
>> 	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub
>
> We're using getcwd() to make the absolute path, but it looks like we 
> should check whether $PWD is set and names the same directory, and, if so, 
> base on that instead.
>
> Actually, make_absolute_path() seems to go to an awful lot of trouble to 
> make sure that there aren't symlinks in the resulting path. Is this 
> actually necessary or even desireable for some reason? Should clone be 
> just pasting $PWD (or, failing that, getcwd()) and the given path, and 
> using that to identify the repo to clone?

I am not 100% sympathetic if ../torvalds/linux-2.6.git/ was what caused this. Perhaps 30% or so ;-).

However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would result in the same breakage, and that is something I consider gravely broken.

If both are fixed by rethinking the make_absolute_path(), that would be good.

Greg KH· Jun 6, 2008, 20:48 UTC · re: Junio C Hamano · lore
On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote:
Show 27 quoted lines
> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > On Wed, 4 Jun 2008, Greg KH wrote:
> > ...
> >> I can duplicate what I did, and it did it again:
> >> 	on master.kernel.org:
> >> 	$ cd /pub/scm/linux/kernel/git/stable/
> >> 	$ git clone -s --bare ../torvalds/linux-2.6.git/ test
> >> 	$ cat test/objects/info/alternates
> >> 	/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
> >> 
> >> I'm guessing this is because /pub/ is a symlink:
> >> 	$ ls -l /pub
> >> 	lrwxrwxrwx 1 root root 13 Feb  4  2006 /pub -> /home/ftp/pub
> >
> > We're using getcwd() to make the absolute path, but it looks like we 
> > should check whether $PWD is set and names the same directory, and, if so, 
> > base on that instead.
> >
> > Actually, make_absolute_path() seems to go to an awful lot of trouble to 
> > make sure that there aren't symlinks in the resulting path. Is this 
> > actually necessary or even desireable for some reason? Should clone be 
> > just pasting $PWD (or, failing that, getcwd()) and the given path, and 
> > using that to identify the repo to clone?
> 
> I am not 100% sympathetic if ../torvalds/linux-2.6.git/ was what caused
> this.  Perhaps 30% or so ;-).

I agree, if I did something stupid like this, that's my fault. I didn't realize that it was stupid, so I guess a simple warning, "don't use relative paths when cloning with alternates" would have told me so.

> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
> result in the same breakage, and that is something I consider gravely
> broken.

Yeah, especially as I never realized that /pub/ was a symlink in the first place :)

thanks,
greg k-h
Junio C Hamano· Jun 6, 2008, 21:00 UTC · re: Greg KH · lore
Greg KH <greg@kroah.com> writes:
Show 8 quoted lines
> On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote:
>
>> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
>> result in the same breakage, and that is something I consider gravely
>> broken.
>
> Yeah, especially as I never realized that /pub/ was a symlink in the
> first place :)

And it was even worse than that. Even if you knew /pub was a symlink, you could not have done anything about it, short of editing the alternates file in the resulting repository by hand.

Up until now, that is.

Daniel's patch seems to fix the issue for me and it will be included in tonight's pushout among other fixes.

Thanks.
Greg KH· Jun 6, 2008, 21:54 UTC · re: Junio C Hamano · lore
On Fri, Jun 06, 2008 at 02:00:09PM -0700, Junio C Hamano wrote:
Show 14 quoted lines
> Greg KH <greg@kroah.com> writes:
> 
> > On Thu, Jun 05, 2008 at 02:26:11PM -0700, Junio C Hamano wrote:
> >
> >> However, I think /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ would
> >> result in the same breakage, and that is something I consider gravely
> >> broken.
> >
> > Yeah, especially as I never realized that /pub/ was a symlink in the
> > first place :)
> 
> And it was even worse than that.  Even if you knew /pub was a symlink, you
> could not have done anything about it, short of editing the alternates
> file in the resulting repository by hand.
Heh, true :)
> Up until now, that is.
> 
> Daniel's patch seems to fix the issue for me and it will be included in
> tonight's pushout among other fixes.
Great, thanks a lot for the fix everyone.
greg k-h

← back to recent threads