threads / discuss / 5011

Git clone stalls at a read(3, ...) saw using strace

Subject: Git clone stalls at a read(3, ...) saw using strace

## tl;dr

14 messages between Jul 27, 2006 and Jul 28, 2006.

replies: 13people: 4as markdown or json

André Goddard Rosa· Jul 27, 2006, 12:16 UTC · lore
Hi, all!
    Please CC me as I'm not subscribed.
    Using  GIT 1.4.1, I tried to get this repository:
# git clone git://source.mvista.com/git/linux-davinci-2.6.git
    After some time the download stalled completely. I used ps and
saw two git processes.
    Using strace, I saw that the first one, "git clone", was waiting
with waitpid()
for its child termination.
    Its child, "git pack --all -k", was stalled at a "read(3, ...)"
    I went to /proc and saw that fd 3 was a pipe. Anyone know why this
is happening?
    I found an issue somewhat related in google (not sure that is the
same problem):
http://groups.google.com/group/fa.linux.kernel/browse_thread/thread/e64a92f28def32a7/93bd8bf5df1fcc66?lnk=st&q=&rnum=1#93bd8bf5df1fcc66
    It stalled at the first 60 Mb transfered. After CTRL+C it deletes
and I have to start again from scratch.
Thank you for the great alternative to bitkeeper and CVS,
-- 
[]s,
André Goddard
Pavel Roskin· Jul 27, 2006, 16:38 UTC · re: André Goddard Rosa · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On Thu, 2006-07-27 at 08:16 -0400, André Goddard Rosa wrote:
Show 5 quoted lines
> Hi, all!
> 
>     Please CC me as I'm not subscribed.
> 
>     Using  GIT 1.4.1, I tried to get this repository:
Please try git 1.4.1.1, it fixes git-clone.
-- 
Regards,
Pavel Roskin
André Goddard Rosa· Jul 27, 2006, 16:43 UTC · re: Pavel Roskin · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
Show 8 quoted lines
> On Thu, 2006-07-27 at 08:16 -0400, André Goddard Rosa wrote:
> > Hi, all!
> >
> >     Please CC me as I'm not subscribed.
> >
> >     Using  GIT 1.4.1, I tried to get this repository:
>
> Please try git 1.4.1.1, it fixes git-clone.
Thank you so much, Pavel!
Downloading it right now!
Thanks a lot,
-- 
[]s,
André Goddard
Linus Torvalds· Jul 27, 2006, 16:54 UTC · re: André Goddard Rosa · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On Thu, 27 Jul 2006, André Goddard Rosa wrote:
Show 6 quoted lines
> > 
> > Please try git 1.4.1.1, it fixes git-clone.
> 
> Thank you so much, Pavel!
> 
> Downloading it right now!

The problem is that at least the timeout-fix is a server-side thing, it won't fix the client. So if a server is running the broken 1.4.1 release, it will time out the clients after 10 minutes. It wasn't obvious breakage, because most people do smaller fetches, and never even got close to that timeout (most of my connections tend to be in the seconds ;).

I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this week, so I'm bcc'ing the other admins here to ask them to upgrade to 1.4.1.1. That won't fix the montavista site, though.

That said, I don't think the timeout problem should have caused a _stall_ of the connection, it should have caused an early EOF and thus a unpack error.

		Linus
André Goddard Rosa· Jul 27, 2006, 17:17 UTC · re: Linus Torvalds · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/27/06, Linus Torvalds <torvalds@osdl.org> wrote:
Show 21 quoted lines
> On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> > >
> > > Please try git 1.4.1.1, it fixes git-clone.
> >
> > Thank you so much, Pavel!
> >
> > Downloading it right now!
>
> The problem is that at least the timeout-fix is a server-side thing, it
> won't fix the client. So if a server is running the broken 1.4.1 release,
> it will time out the clients after 10 minutes. It wasn't obvious breakage,
> because most people do smaller fetches, and never even got close to that
> timeout (most of my connections tend to be in the seconds ;).
>
> I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this
> week, so I'm bcc'ing the other admins here to ask them to upgrade to
> 1.4.1.1. That won't fix the montavista site, though.
>
> That said, I don't think the timeout problem should have caused a _stall_
> of the connection, it should have caused an early EOF and thus a unpack
> error.
Great! Thank you so much, I will try to contact Montavista.
Best regards,
-- 
[]s,
André Goddard
André Goddard Rosa· Jul 27, 2006, 17:25 UTC · re: André Goddard Rosa · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
Show 24 quoted lines
> On 7/27/06, Linus Torvalds <torvalds@osdl.org> wrote:
> > On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> > > >
> > > > Please try git 1.4.1.1, it fixes git-clone.
> > >
> > > Thank you so much, Pavel!
> > >
> > > Downloading it right now!
> >
> > The problem is that at least the timeout-fix is a server-side thing, it
> > won't fix the client. So if a server is running the broken 1.4.1 release,
> > it will time out the clients after 10 minutes. It wasn't obvious breakage,
> > because most people do smaller fetches, and never even got close to that
> > timeout (most of my connections tend to be in the seconds ;).
> >
> > I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this
> > week, so I'm bcc'ing the other admins here to ask them to upgrade to
> > 1.4.1.1. That won't fix the montavista site, though.
> >
> > That said, I don't think the timeout problem should have caused a _stall_
> > of the connection, it should have caused an early EOF and thus a unpack
> > error.
>
> Great! Thank you so much, I will try to contact Montavista.
Hi, Linus!

The problem happened again, and it stalls AFAICS. Can you reproduce there?

# git clone git://source.mvista.com/git/linux-davinci-2.6.git
Here I have a constant 55 kbps and it stalls with about 117Mb downloaded.
I will try to contact Montavista.
Thanks you so much for the support,
-- 
[]s,
André Goddard
Linus Torvalds· Jul 27, 2006, 17:50 UTC · re: André Goddard Rosa · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On Thu, 27 Jul 2006, André Goddard Rosa wrote:
Show 5 quoted lines
> 
> The problem happened again, and it stalls AFAICS.
> Can you reproduce there?
> 
> # git clone git://source.mvista.com/git/linux-davinci-2.6.git
Nope. I have a fairly constant 120kbps, and:
[torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
Checking files out...)      
 100% (19754/19754) done
here
		Linus
André Goddard Rosa· Jul 27, 2006, 17:57 UTC · re: Linus Torvalds · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/27/06, Linus Torvalds <torvalds@osdl.org> wrote:
Show 15 quoted lines
>
>
> On Thu, 27 Jul 2006, André Goddard Rosa wrote:
> >
> > The problem happened again, and it stalls AFAICS.
> > Can you reproduce there?
> >
> > # git clone git://source.mvista.com/git/linux-davinci-2.6.git
>
> Nope. I have a fairly constant 120kbps, and:
>
> [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> Checking files out...)
>  100% (19754/19754) done
> here

Do you think you can reproduce the server timeout by CTRL + Z and stopping the process for a while?

I sent an email to the last one who commited in that tree asking for updating the git version there. I also sent and email to the maillist owner asking this.

Thanks again,
-- 
[]s,
André Goddard
Pavel Roskin· Jul 27, 2006, 18:41 UTC · re: Linus Torvalds · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
Show 5 quoted lines
> Nope. I have a fairly constant 120kbps, and:
> 
> [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> Checking files out...)      
>  100% (19754/19754) done
Same thing here.  Current git from the master branch.
-- 
Regards,
Pavel Roskin
André Goddard Rosa· Jul 27, 2006, 19:16 UTC · re: Pavel Roskin · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
Show 8 quoted lines
> On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > Nope. I have a fairly constant 120kbps, and:
> >
> > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > Checking files out...)
> >  100% (19754/19754) done
>
> Same thing here.  Current git from the master branch.

Forgot to say that we are using this script in GIT_PROXY_COMMAND environment variable:

(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
| (read a; read a; cat )

The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and the second removes an empty line as described here:

http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
I will try from home later again.
Thanks,
-- 
[]s,
André Goddard
André Goddard Rosa· Jul 28, 2006, 09:56 UTC · re: André Goddard Rosa · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
Show 22 quoted lines
> On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > > Nope. I have a fairly constant 120kbps, and:
> > >
> > > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > > Checking files out...)
> > >  100% (19754/19754) done
> >
> > Same thing here.  Current git from the master branch.
>
> Forgot to say that we are using this script in GIT_PROXY_COMMAND
> environment variable:
>
> (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
> | (read a; read a; cat )
>
> The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
> the second removes an empty line as described here:
>
> http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
>
> I will try from home later again.

Okey, I tried from home (without the proxy trick) and it behaved a lot better but my disc went full in the process and I got these messages: ... ... ... error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c error: git-checkout-index: unable to create file drivers/scsi/mac_scsi.c (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/mac_scsi.h (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/mca_53c9x.c (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/megaraid.c (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/megaraid.h (No space left on device) fatal: cannot create directory at drivers/scsi/megaraid

And it finished keeping the downloaded files, but I still cannot see these files listed above. I tried to pull but it says that I'm up-to-date:

doctorture:/opt/downloads/mvista/linux-mvista # git-pull Already up-to-date.

I remember that using CVS I just used 'cvs update' after checkout and it would bring the missing files to me.

What I'm doing wrong here?
Thank you so much,
-- 
[]s,
André Goddard
André Goddard Rosa· Jul 28, 2006, 09:58 UTC · re: André Goddard Rosa · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/28/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
Show 55 quoted lines
> On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> > On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > > > Nope. I have a fairly constant 120kbps, and:
> > > >
> > > > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > > > Checking files out...)
> > > >  100% (19754/19754) done
> > >
> > > Same thing here.  Current git from the master branch.
> >
> > Forgot to say that we are using this script in GIT_PROXY_COMMAND
> > environment variable:
> >
> > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
> > | (read a; read a; cat )
> >
> > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
> > the second removes an empty line as described here:
> >
> > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
> >
> > I will try from home later again.
>
> Okey, I tried from home (without the proxy trick) and it behaved a lot
> better but my disc went full in the process and I got these messages:
> ...
> ...
> ...
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h
> error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.h (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mca_53c9x.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.h (No space left on device)
> fatal: cannot create directory at drivers/scsi/megaraid
>
> And it finished keeping the downloaded files, but I still cannot see
> these files listed above.
> I tried to pull but it says that I'm up-to-date:
>
> doctorture:/opt/downloads/mvista/linux-mvista # git-pull
> Already up-to-date.
>
> I remember that using CVS I just used 'cvs update' after checkout and
> it would bring the missing files to me.
>
> What I'm doing wrong here?
I'm also receiving these messages when trying to change branches:

doctorture:/opt/downloads/mvista/linux-mvista # git-checkout origin fatal: Untracked working tree file '.gitignore' would be overwritten by merge.

Perhaps I will need to download using git-clone again?
Thanks,
-- 
[]s,
André Goddard
Ribeiro, Humberto Plinio· Jul 28, 2006, 13:36 UTC · re: André Goddard Rosa · lore

RE: Git clone stalls at a read(3, ...) saw using strace

Hi, Andre.
After a "ps -ef" I saw two instances of the script defined by GIT_PROXY_COMMAND. Strangely one of those instances was child of the other. A strace showed the scripts were blocked in a "waitpid(-1,". I've killed the child script and the git clone resumed the process. 
I didn't understand why this blocking on git happened though. The creation of these two instances (one child of the other) was also strange. 

Regards, Humberto

---
Obrigado/Best regards/Mit freundlichen Grüßen/Pozdrawiam
Humberto Ribeiro, MSc.
SIEMENS Home and Office Communication
Phone: +55 92 2129 6205
Mobile: +55 92 8146 3147
-----Original Message-----
From: André Goddard Rosa [mailto:andre.goddard@gmail.com] 
Sent: sexta-feira, 28 de julho de 2006 05:58
To: Pavel Roskin
Cc: Git Mailing List; Linus Torvalds; Ribeiro, Humberto Plinio
Subject: Re: Git clone stalls at a read(3, ...) saw using strace
On 7/28/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
Show 55 quoted lines
> On 7/27/06, André Goddard Rosa <andre.goddard@gmail.com> wrote:
> > On 7/27/06, Pavel Roskin <proski@gnu.org> wrote:
> > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote:
> > > > Nope. I have a fairly constant 120kbps, and:
> > > >
> > > > [torvalds@g5 ~]$  git clone git://source.mvista.com/git/linux-davinci-2.6.git
> > > > Checking files out...)
> > > >  100% (19754/19754) done
> > >
> > > Same thing here.  Current git from the master branch.
> >
> > Forgot to say that we are using this script in GIT_PROXY_COMMAND
> > environment variable:
> >
> > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc <proxy_add> <portnum>
> > | (read a; read a; cat )
> >
> > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and
> > the second removes an empty line as described here:
> >
> > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html
> >
> > I will try from home later again.
>
> Okey, I tried from home (without the proxy trick) and it behaved a lot
> better but my disc went full in the process and I got these messages:
> ...
> ...
> ...
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c
> error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h
> error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mac_scsi.h (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/mca_53c9x.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.c (No space left on device)
> error: git-checkout-index: unable to create file
> drivers/scsi/megaraid.h (No space left on device)
> fatal: cannot create directory at drivers/scsi/megaraid
>
> And it finished keeping the downloaded files, but I still cannot see
> these files listed above.
> I tried to pull but it says that I'm up-to-date:
>
> doctorture:/opt/downloads/mvista/linux-mvista # git-pull
> Already up-to-date.
>
> I remember that using CVS I just used 'cvs update' after checkout and
> it would bring the missing files to me.
>
> What I'm doing wrong here?
I'm also receiving these messages when trying to change branches:

doctorture:/opt/downloads/mvista/linux-mvista # git-checkout origin fatal: Untracked working tree file '.gitignore' would be overwritten by merge.

Perhaps I will need to download using git-clone again?
Thanks,
-- 
[]s,
André Goddard
André Goddard Rosa· Jul 28, 2006, 13:57 UTC · re: Ribeiro, Humberto Plinio · lore

Re: Git clone stalls at a read(3, ...) saw using strace

On 7/28/06, Ribeiro, Humberto Plinio <humberto.ribeiro@siemens.com> wrote:
Show 11 quoted lines
> Hi, Andre.
>
> After a "ps -ef" I saw two instances of the script defined by
> GIT_PROXY_COMMAND. Strangely one of those instances was child
> of the other. A strace showed the scripts were blocked in a
>"waitpid(-1,". I've killed the child script and the git clone resumed the
>process.
>
> I didn't understand why this blocking on git happened though. The
> creation of these two instances (one child of the other) was also
> strange.
Yes,  it worked to me too, but I also don't know why.

Let me explain exactly what we had to do so someone perhaps can explain why we had to do this:

I configured GIT_PROXY_COMMAND pointing to a script with this content:

(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc 172.29.0.6 3128 | (read a; read a; cat )

Later:
git clone git://source.mvista.com/git/linux-davinci-2.6.git

After some time (downloading 117 Mb) it stalled (git-fetch on a pipe read) and we had to do this to resume and let it finish:

[root@mao2wx23 tmp]# ps -ef | grep git | grep -v grep opb694 9975 1451 0 01:39 pts/11 00:00:00 /bin/sh /usr/bin/git-clone git://source.mvista.com/git/linux-davinci-2.6.git opb694 9985 9975 3 01:39 pts/11 00:01:56 git-fetch-pack --all -k git //source.mvista.com git/linux-davinci-2.6.git [root@mao2wx23 tmp]# ps -ef | grep proxy | grep -v grep opb694 9986 9985 0 01:39 pts/11 00:00:00 /bin/sh /home/opb694/proxy-cmd.sh source.mvista.com 9418 opb694 9987 9986 0 01:39 pts/11 00:00:00 /bin/sh /home/opb694/proxy-cmd.sh source.mvista.com 9418 [root@mao2wx23 tmp]# kill 9987

Thanks again for the patience,
-- 
[]s,
André Goddard

← back to recent threads