threads / discuss / 15850

Re: tip tree clone fail

Subject: Re: tip tree clone fail

## tl;dr

21 messages between Oct 10, 2008 and Oct 15, 2008.

replies: 20people: 10as markdown or json

Thomas Gleixner· Oct 10, 2008, 08:56 UTC · lore
On Fri, 10 Oct 2008, Wang Chen wrote:
Show 10 quoted lines
> Thomas Gleixner said the following on 2008-10-10 16:40:
> > On Fri, 10 Oct 2008, Ingo Molnar wrote:
> > 
> >> Cc:-ed Thomas too - Thomas just ran a clean git clone and that succeeded 
> >> for him, both over the http and over the git protocols.
> > 
> > No, I did ssh and git. Will start a http one now.
> 
> OK. I will wait.
> Thanks, Thomas :)
git clone http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git 
fails here as well. 
Getting pack 7c461fa5d92f9e047ae07b70eb30828a63a803f5
 which contains 874e848aefc2d5952b00593c9fd23d748259eec7
walk 874e848aefc2d5952b00593c9fd23d748259eec7
Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/sfr/linux-next.git/
error: Unable to find 85ba94ba0592296053f7f2846812173424afe1cb under http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
Cannot obtain needed object 85ba94ba0592296053f7f2846812173424afe1cb
while processing commit 874e848aefc2d5952b00593c9fd23d748259eec7.
fatal: Fetch failed.

The needed object 85ba94ba0592296053f7f2846812173424afe1cb is a commit in linus tree.

So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using 1.5.3.x)

Junio, can you please have a look ? I know it's http again, but we can not sanitze the corporate firewall folks by any means.

Thanks,
	tglx
Wang Chen· Oct 10, 2008, 08:57 UTC · re: Thomas Gleixner · lore
Thomas Gleixner said the following on 2008-10-10 16:56:
Show 30 quoted lines
> On Fri, 10 Oct 2008, Wang Chen wrote:
>> Thomas Gleixner said the following on 2008-10-10 16:40:
>>> On Fri, 10 Oct 2008, Ingo Molnar wrote:
>>>
>>>> Cc:-ed Thomas too - Thomas just ran a clean git clone and that succeeded 
>>>> for him, both over the http and over the git protocols.
>>> No, I did ssh and git. Will start a http one now.
>> OK. I will wait.
>> Thanks, Thomas :)
> 
> git clone http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git 
> 
> fails here as well. 
> 
> Getting pack 7c461fa5d92f9e047ae07b70eb30828a63a803f5
>  which contains 874e848aefc2d5952b00593c9fd23d748259eec7
> walk 874e848aefc2d5952b00593c9fd23d748259eec7
> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> Getting pack list for http://www.kernel.org/home/ftp/pub/scm/linux/kernel/git/sfr/linux-next.git/
> error: Unable to find 85ba94ba0592296053f7f2846812173424afe1cb under http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
> Cannot obtain needed object 85ba94ba0592296053f7f2846812173424afe1cb
> while processing commit 874e848aefc2d5952b00593c9fd23d748259eec7.
> fatal: Fetch failed.
> 
> The needed object 85ba94ba0592296053f7f2846812173424afe1cb is a commit
> in linus tree.
> 
> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
> 1.5.3.x)
> 
My git version is 1.5.5.1, although it doesn't matter ;)
> Junio, can you please have a look ? I know it's http again, but we can
> not sanitze the corporate firewall folks by any means.
>
H. Peter Anvin· Oct 10, 2008, 15:58 UTC · re: Wang Chen · lore
Wang Chen wrote:
Show 6 quoted lines
>>
>> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
>> 1.5.3.x)
> 
> My git version is 1.5.5.1, although it doesn't matter ;)
>

http transport requires that "git update-server-info" is done after each push. Otherwise, it ends up in a trainwreck.

Shawn Pierce at Google is working on a long-term solution, but in the meantime, if you have to use http transport, murder your IT people.

	-hpa
Wang Chen· Oct 12, 2008, 11:28 UTC · re: H. Peter Anvin · lore
H. Peter Anvin said the following on 2008-10-10 23:58:
Show 14 quoted lines
> Wang Chen wrote:
>>>
>>> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
>>> 1.5.3.x)
>>
>> My git version is 1.5.5.1, although it doesn't matter ;)
>>
> 
> http transport requires that "git update-server-info" is done after each
> push.  Otherwise, it ends up in a trainwreck.
> 
> Shawn Pierce at Google is working on a long-term solution, but in the
> meantime, if you have to use http transport, murder your IT people.
> 
So, the current solutions are:
1. Waiting for Shawn fixing it.
2. Asking Ingo to "git update-server-info" after each push.
3. Murdering our IT people.
I'd rather to do the second solution than the third one :)

Ingo, will you do "git update-server-info" after each push before Shawn fix this problem?

Lars Hjemli· Oct 12, 2008, 12:04 UTC · re: Wang Chen · lore
On Sun, Oct 12, 2008 at 13:28, Wang Chen <wangchen@cn.fujitsu.com> wrote:
Show 22 quoted lines
>
> H. Peter Anvin said the following on 2008-10-10 23:58:
> > Wang Chen wrote:
> >>>
> >>> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
> >>> 1.5.3.x)
> >>
> >> My git version is 1.5.5.1, although it doesn't matter ;)
> >>
> >
> > http transport requires that "git update-server-info" is done after each
> > push.  Otherwise, it ends up in a trainwreck.
> >
> > Shawn Pierce at Google is working on a long-term solution, but in the
> > meantime, if you have to use http transport, murder your IT people.
> >
>
>
> So, the current solutions are:
> 1. Waiting for Shawn fixing it.
> 2. Asking Ingo to "git update-server-info" after each push.
> 3. Murdering our IT people.
4. Install cgit, which removes the need for `git update-server-info`

-- larsh

H. Peter Anvin· Oct 12, 2008, 14:15 UTC · re: Wang Chen · lore
Wang Chen wrote:
Show 8 quoted lines
> 
> So, the current solutions are:
> 1. Waiting for Shawn fixing it.
> 2. Asking Ingo to "git update-server-info" after each push.
> 3. Murdering our IT people.
> 
> I'd rather to do the second solution than the third one :)
>

Actually, the third solution is the proper one given the circumstances you find yourself in. None of this would be a problem if there wasn't a bunch of companies with incompetent network management.

	-hpa
Ingo Molnar· Oct 12, 2008, 12:41 UTC · re: H. Peter Anvin · lore
* H. Peter Anvin <hpa@zytor.com> wrote:
Show 10 quoted lines
> Wang Chen wrote:
>>>
>>> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
>>> 1.5.3.x)
>>
>> My git version is 1.5.5.1, although it doesn't matter ;)
>>
>
> http transport requires that "git update-server-info" is done after 
> each push.  Otherwise, it ends up in a trainwreck.

can i simply put "git update-server-info" into .git/hooks/post-receive to solve this problem?

	Ingo
Miklos Vajna· Oct 12, 2008, 12:59 UTC · re: Ingo Molnar · lore
On Sun, Oct 12, 2008 at 02:41:05PM +0200, Ingo Molnar <mingo@elte.hu> wrote:
> can i simply put "git update-server-info" into .git/hooks/post-receive 
> to solve this problem?
Yes. And don't forget the chmod +x it, if it's not done yet.
H. Peter Anvin· Oct 12, 2008, 14:14 UTC · re: Ingo Molnar · lore
Ingo Molnar wrote:
Show 13 quoted lines
> * H. Peter Anvin <hpa@zytor.com> wrote:
> 
>> Wang Chen wrote:
>>>> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
>>>> 1.5.3.x)
>>> My git version is 1.5.5.1, although it doesn't matter ;)
>>>
>> http transport requires that "git update-server-info" is done after 
>> each push.  Otherwise, it ends up in a trainwreck.
> 
> can i simply put "git update-server-info" into .git/hooks/post-receive 
> to solve this problem?
> 
post-update, rather than post-receive.
The standard post-update.sample contains it, it just needs a rename.
	-hpa
Ingo Molnar· Oct 12, 2008, 15:24 UTC · re: H. Peter Anvin · lore
* H. Peter Anvin <hpa@zytor.com> wrote:
Show 18 quoted lines
> Ingo Molnar wrote:
>> * H. Peter Anvin <hpa@zytor.com> wrote:
>>
>>> Wang Chen wrote:
>>>>> So http transport is wreckaged. (git version 1.6.0.1 here, Wang is using
>>>>> 1.5.3.x)
>>>> My git version is 1.5.5.1, although it doesn't matter ;)
>>>>
>>> http transport requires that "git update-server-info" is done after  
>>> each push.  Otherwise, it ends up in a trainwreck.
>>
>> can i simply put "git update-server-info" into .git/hooks/post-receive  
>> to solve this problem?
>>
>
> post-update, rather than post-receive.
>
> The standard post-update.sample contains it, it just needs a rename.

hm, -tip's .git/hooks/post-update already contained this, for the last 2 months:

  exec git update-server-info

so ... _despite_ us having this in the git repo, the HTTP protocol still does not work. Why?

	Ingo
Teemu Likonen· Oct 12, 2008, 15:36 UTC · re: Ingo Molnar · lore
Ingo Molnar <mingo@elte.hu> writes:
Show 12 quoted lines
> * H. Peter Anvin <hpa@zytor.com> wrote:
>> post-update, rather than post-receive.
>>
>> The standard post-update.sample contains it, it just needs a rename.
>
> hm, -tip's .git/hooks/post-update already contained this, for the last 2 
> months:
>
>   exec git update-server-info
>
> so ... _despite_ us having this in the git repo, the HTTP protocol
> still does not work. Why?

I don't know why but I have experienced the same: post-update hook just doesn't work. I described it here:

    http://thread.gmane.org/gmane.comp.version-control.git/82351
Fortunately post-receive hooks works, so you could to try it instead.
Petr Baudis· Oct 12, 2008, 15:39 UTC · re: Ingo Molnar · lore
On Sun, Oct 12, 2008 at 05:24:27PM +0200, Ingo Molnar wrote:
Show 7 quoted lines
> hm, -tip's .git/hooks/post-update already contained this, for the last 2 
> months:
> 
>   exec git update-server-info
> 
> so ... _despite_ us having this in the git repo, the HTTP protocol still 
> does not work. Why?

I think your problem is that HTTP does not know where to look for objects coming from alternates; IIRC this would work if you used relative paths in objects/info/alternates, or you can create objects/info/http-alternates like

	/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
	/pub/scm/linux/kernel/git/sfr/linux-next.git/objects
-- 
				Petr "Pasky" Baudis
People who take cold baths never have rheumatism, but they have
cold baths.
Ingo Molnar· Oct 12, 2008, 16:59 UTC · re: Petr Baudis · lore
* Petr Baudis <pasky@suse.cz> wrote:
Show 16 quoted lines
> On Sun, Oct 12, 2008 at 05:24:27PM +0200, Ingo Molnar wrote:
> > hm, -tip's .git/hooks/post-update already contained this, for the last 2 
> > months:
> > 
> >   exec git update-server-info
> > 
> > so ... _despite_ us having this in the git repo, the HTTP protocol still 
> > does not work. Why?
> 
> I think your problem is that HTTP does not know where to look for
> objects coming from alternates; IIRC this would work if you used
> relative paths in objects/info/alternates, or you can create
> objects/info/http-alternates like
> 
> 	/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
> 	/pub/scm/linux/kernel/git/sfr/linux-next.git/objects
ok, i've now set it up like this:
 $ pwd
 /pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
 $ cat objects/info/alternates
 /home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
 /home/ftp/pub/scm/linux/kernel/git/sfr/linux-next.git/objects
 $ cat objects/info/http-alternates
 /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
 /pub/scm/linux/kernel/git/sfr/linux-next.git/objects

and i've added "git update-server-info" to hooks/post-receive and made it chmod +x.

that should be golden, right? I'm wondering why this isnt in the default setup - i've been behind a limited corporate firewall in a former life and having HTTP access is indeed very handy and pragmatic. Often hotel WLANs are HTTP only as well.

Soapbox: in fact it would be outright stupid to limit the kernel 
source's availability artificially by not making HTTP a tier-one access 
method.
Fighting against HTTP-only firewalls is like constantly pointing it out 
to the popular press that they should say 'cracker' instead of 'hacker'. 
It is pointless and only hurts the availability our own project.
	
	Ingo
Jakub Narebski· Oct 12, 2008, 17:37 UTC · re: Ingo Molnar · lore
Ingo Molnar <mingo@elte.hu> writes:
Show 7 quoted lines
> Soapbox: in fact it would be outright stupid to limit the kernel 
> source's availability artificially by not making HTTP a tier-one access 
> method.
> 
> Fighting against HTTP-only firewalls is like constantly pointing it out 
> to the popular press that they should say 'cracker' instead of 'hacker'. 
> It is pointless and only hurts the availability our own project.

Hopefully there should soon be here "smart" HTTP server (as for example CGI script), which encapsulates git protocol, modified for the fact that HTTP is stateless protocol, in HTTP. It would unfortunately require git installed on server.

It is WIP, but I'm not sure how far it is from completion.
-- 
Jakub Narebski
Poland
ShadeHawk on #git
Shawn O. Pearce· Oct 12, 2008, 17:59 UTC · re: Jakub Narebski · lore
Jakub Narebski <jnareb@gmail.com> wrote:
Show 9 quoted lines
> Ingo Molnar <mingo@elte.hu> writes:
> 
> > Soapbox: in fact it would be outright stupid to limit the kernel 
> > source's availability artificially by not making HTTP a tier-one access 
> > method.
> 
> Hopefully there should soon be here "smart" HTTP server [...]
> 
> It is WIP, but I'm not sure how far it is from completion.

Its still not even prototyped. I'm supposed to be spending my 20% time at Google on the Git-in-HTTP documentation and implementation, so I can feed patches to the list for review.

Sadly, my current 80% project has been demanding >120% of my time these past 5 weeks, so I have not been able to touch the Git-in-HTTP concept in that time period.

Most of my 80% project has to be wrapped up by this time next week. Afterwards I'm going to try and dedicate at least a week to Git-in-HTTP and get caught up. I sort of have to; its part of my goals for my performance evaluation. Even if the list winds up rejecting my implementation, I still have to put it out there for discussion.

-- 
Shawn.
H. Peter Anvin· Oct 13, 2008, 04:09 UTC · re: Ingo Molnar · lore
Ingo Molnar wrote:
Show 9 quoted lines
> 
> Soapbox: in fact it would be outright stupid to limit the kernel 
> source's availability artificially by not making HTTP a tier-one access 
> method.
> 
> Fighting against HTTP-only firewalls is like constantly pointing it out 
> to the popular press that they should say 'cracker' instead of 'hacker'. 
> It is pointless and only hurts the availability our own project.
>

Yes, and Shawn Pierce is working on making HTTP a proper "smart" access method, based on a proposal I submitted. That is the proper solution, since it avoids all the braindamage of "dumb" access methods. We [kernel.org] intend to deploy it as soon as it is available.

	-hpa
Wang Chen· Oct 13, 2008, 09:40 UTC · re: Ingo Molnar · lore
Ingo Molnar said the following on 2008-10-13 0:59:
Show 47 quoted lines
> * Petr Baudis <pasky@suse.cz> wrote:
> 
>> On Sun, Oct 12, 2008 at 05:24:27PM +0200, Ingo Molnar wrote:
>>> hm, -tip's .git/hooks/post-update already contained this, for the last 2 
>>> months:
>>>
>>>   exec git update-server-info
>>>
>>> so ... _despite_ us having this in the git repo, the HTTP protocol still 
>>> does not work. Why?
>> I think your problem is that HTTP does not know where to look for
>> objects coming from alternates; IIRC this would work if you used
>> relative paths in objects/info/alternates, or you can create
>> objects/info/http-alternates like
>>
>> 	/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
>> 	/pub/scm/linux/kernel/git/sfr/linux-next.git/objects
> 
> ok, i've now set it up like this:
> 
>  $ pwd
>  /pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
> 
>  $ cat objects/info/alternates
>  /home/ftp/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
>  /home/ftp/pub/scm/linux/kernel/git/sfr/linux-next.git/objects
> 
>  $ cat objects/info/http-alternates
>  /pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
>  /pub/scm/linux/kernel/git/sfr/linux-next.git/objects
> 
> and i've added "git update-server-info" to hooks/post-receive and made 
> it chmod +x.
> 
> that should be golden, right? I'm wondering why this isnt in the default 
> setup - i've been behind a limited corporate firewall in a former life 
> and having HTTP access is indeed very handy and pragmatic. Often hotel 
> WLANs are HTTP only as well.
> 
> Soapbox: in fact it would be outright stupid to limit the kernel 
> source's availability artificially by not making HTTP a tier-one access 
> method.
> 
> Fighting against HTTP-only firewalls is like constantly pointing it out 
> to the popular press that they should say 'cracker' instead of 'hacker'. 
> It is pointless and only hurts the availability our own project.
>

Ingo, thank you for your work. I can clone more, but error still occurs:

Getting alternates list for http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git Also look at http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ Also look at http://www.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git/ Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git error: transfer closed with 8280 bytes remaining to read Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git/ error: Unable to find 95630fe2917f805a26f8d8beaafb80cd2f729eb5 under http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git Cannot obtain needed object 95630fe2917f805a26f8d8beaafb80cd2f729eb5

H. Peter Anvin· Oct 13, 2008, 15:59 UTC · re: Wang Chen · lore
Wang Chen wrote:
Show 13 quoted lines
> 
> Ingo, thank you for your work.
> I can clone more, but error still occurs:
> 
> Getting alternates list for http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
> Also look at http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> Also look at http://www.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git/
> Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
> error: transfer closed with 8280 bytes remaining to read
> Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> Getting pack list for http://www.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git/
> error: Unable to find 95630fe2917f805a26f8d8beaafb80cd2f729eb5 under http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
> Cannot obtain needed object 95630fe2917f805a26f8d8beaafb80cd2f729eb5
I cleaned it up and it should work now.
	-hpa
Wang Chen· Oct 15, 2008, 03:43 UTC · re: H. Peter Anvin · lore
H. Peter Anvin said the following on 2008-10-13 23:59:
Show 24 quoted lines
> Wang Chen wrote:
>>
>> Ingo, thank you for your work.
>> I can clone more, but error still occurs:
>>
>> Getting alternates list for
>> http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
>> Also look at
>> http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>> Also look at
>> http://www.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git/
>> Getting pack list for
>> http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
>> error: transfer closed with 8280 bytes remaining to read
>> Getting pack list for
>> http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>> Getting pack list for
>> http://www.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git/
>> error: Unable to find 95630fe2917f805a26f8d8beaafb80cd2f729eb5 under
>> http://www.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-tip.git
>> Cannot obtain needed object 95630fe2917f805a26f8d8beaafb80cd2f729eb5
> 
> I cleaned it up and it should work now.
>

Yes. My clone is successful. Thanks, Peter. But, after cloning, git-pull failed because of conflict. I think maybe because Ingo rebased his tree? --- Auto-merged init/main.c CONFLICT (content): Merge conflict in init/main.c Auto-merged scripts/bootgraph.pl CONFLICT (add/add): Merge conflict in scripts/bootgraph.pl Removed sound/soc/at91/eti_b1_wm8731.c Automatic merge failed; fix conflicts and then commit the result. ---

H. Peter Anvin· Oct 15, 2008, 05:25 UTC · re: Wang Chen · lore
Wang Chen wrote:
Show 12 quoted lines
> 
> Yes. My clone is successful. Thanks, Peter.
> But, after cloning, git-pull failed because of conflict.
> I think maybe because Ingo rebased his tree?
> ---
> Auto-merged init/main.c
> CONFLICT (content): Merge conflict in init/main.c
> Auto-merged scripts/bootgraph.pl
> CONFLICT (add/add): Merge conflict in scripts/bootgraph.pl
> Removed sound/soc/at91/eti_b1_wm8731.c
> Automatic merge failed; fix conflicts and then commit the result.
> ---
Yes, the master branch on tip is not rebase-free.
	-hpa
H. Peter Anvin· Oct 13, 2008, 04:07 UTC · re: Petr Baudis · lore
Petr Baudis wrote:
Show 17 quoted lines
> On Sun, Oct 12, 2008 at 05:24:27PM +0200, Ingo Molnar wrote:
>> hm, -tip's .git/hooks/post-update already contained this, for the last 2 
>> months:
>>
>>   exec git update-server-info
>>
>> so ... _despite_ us having this in the git repo, the HTTP protocol still 
>> does not work. Why?
> 
> I think your problem is that HTTP does not know where to look for
> objects coming from alternates; IIRC this would work if you used
> relative paths in objects/info/alternates, or you can create
> objects/info/http-alternates like
> 
> 	/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/objects
> 	/pub/scm/linux/kernel/git/sfr/linux-next.git/objects
> 
Yes, alternates must not contain paths referencing /home/ftp.
	-hpa

← back to recent threads