# Git clone stalls at a read(3, ...) saw using strace 14 messages from 2006-07-27 to 2006-07-28. Participants: André Goddard Rosa, Pavel Roskin, Linus Torvalds, Ribeiro, Humberto Plinio. Thread: https://gitlist.dev/t/5011 ## André Goddard Rosa, 2006-07-27 12:16 Subject: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607270516i7fbd8844he03e107b15fd2ed7%40mail.gmail.com ``` Hi, all! Please CC me as I'm not subscribed. Using GIT 1.4.1, I tried to get this repository: # git clone git://source.mvista.com/git/linux-davinci-2.6.git After some time the download stalled completely. I used ps and saw two git processes. Using strace, I saw that the first one, "git clone", was waiting with waitpid() for its child termination. Its child, "git pack --all -k", was stalled at a "read(3, ...)" I went to /proc and saw that fd 3 was a pipe. Anyone know why this is happening? I found an issue somewhat related in google (not sure that is the same problem): http://groups.google.com/group/fa.linux.kernel/browse_thread/thread/e64a92f28def32a7/93bd8bf5df1fcc66?lnk=st&q=&rnum=1#93bd8bf5df1fcc66 It stalled at the first 60 Mb transfered. After CTRL+C it deletes and I have to start again from scratch. Thank you for the great alternative to bitkeeper and CVS, -- []s, André Goddard ``` ## Pavel Roskin, 2006-07-27 16:38 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: <1154018302.13273.0.camel@dv> URL: https://gitlist.dev/e/1154018302.13273.0.camel%40dv In-Reply-To: ``` On Thu, 2006-07-27 at 08:16 -0400, André Goddard Rosa wrote: > Hi, all! > > Please CC me as I'm not subscribed. > > Using GIT 1.4.1, I tried to get this repository: Please try git 1.4.1.1, it fixes git-clone. -- Regards, Pavel Roskin ``` ## André Goddard Rosa, 2006-07-27 16:43 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607270943w562ec21fuab0eb882b3ccffeb%40mail.gmail.com In-Reply-To: <1154018302.13273.0.camel@dv> ``` On 7/27/06, Pavel Roskin wrote: > On Thu, 2006-07-27 at 08:16 -0400, André Goddard Rosa wrote: > > Hi, all! > > > > Please CC me as I'm not subscribed. > > > > Using GIT 1.4.1, I tried to get this repository: > > Please try git 1.4.1.1, it fixes git-clone. Thank you so much, Pavel! Downloading it right now! Thanks a lot, -- []s, André Goddard ``` ## Linus Torvalds, 2006-07-27 16:54 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0607270947540.4168%40g5.osdl.org In-Reply-To: ``` On Thu, 27 Jul 2006, André Goddard Rosa wrote: > > > > Please try git 1.4.1.1, it fixes git-clone. > > Thank you so much, Pavel! > > Downloading it right now! The problem is that at least the timeout-fix is a server-side thing, it won't fix the client. So if a server is running the broken 1.4.1 release, it will time out the clients after 10 minutes. It wasn't obvious breakage, because most people do smaller fetches, and never even got close to that timeout (most of my connections tend to be in the seconds ;). I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this week, so I'm bcc'ing the other admins here to ask them to upgrade to 1.4.1.1. That won't fix the montavista site, though. That said, I don't think the timeout problem should have caused a _stall_ of the connection, it should have caused an early EOF and thus a unpack error. Linus ``` ## André Goddard Rosa, 2006-07-27 17:17 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607271017p22fa908bt82a564a4a1a15a79%40mail.gmail.com In-Reply-To: ``` On 7/27/06, Linus Torvalds wrote: > On Thu, 27 Jul 2006, André Goddard Rosa wrote: > > > > > > Please try git 1.4.1.1, it fixes git-clone. > > > > Thank you so much, Pavel! > > > > Downloading it right now! > > The problem is that at least the timeout-fix is a server-side thing, it > won't fix the client. So if a server is running the broken 1.4.1 release, > it will time out the clients after 10 minutes. It wasn't obvious breakage, > because most people do smaller fetches, and never even got close to that > timeout (most of my connections tend to be in the seconds ;). > > I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this > week, so I'm bcc'ing the other admins here to ask them to upgrade to > 1.4.1.1. That won't fix the montavista site, though. > > That said, I don't think the timeout problem should have caused a _stall_ > of the connection, it should have caused an early EOF and thus a unpack > error. Great! Thank you so much, I will try to contact Montavista. Best regards, -- []s, André Goddard ``` ## André Goddard Rosa, 2006-07-27 17:25 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607271025m1a611006x65a900e9e487ce1b%40mail.gmail.com In-Reply-To: ``` On 7/27/06, André Goddard Rosa wrote: > On 7/27/06, Linus Torvalds wrote: > > On Thu, 27 Jul 2006, André Goddard Rosa wrote: > > > > > > > > Please try git 1.4.1.1, it fixes git-clone. > > > > > > Thank you so much, Pavel! > > > > > > Downloading it right now! > > > > The problem is that at least the timeout-fix is a server-side thing, it > > won't fix the client. So if a server is running the broken 1.4.1 release, > > it will time out the clients after 10 minutes. It wasn't obvious breakage, > > because most people do smaller fetches, and never even got close to that > > timeout (most of my connections tend to be in the seconds ;). > > > > I asked Kees to upgrade kernel.org, but he's apparently busy at OSCON this > > week, so I'm bcc'ing the other admins here to ask them to upgrade to > > 1.4.1.1. That won't fix the montavista site, though. > > > > That said, I don't think the timeout problem should have caused a _stall_ > > of the connection, it should have caused an early EOF and thus a unpack > > error. > > Great! Thank you so much, I will try to contact Montavista. Hi, Linus! The problem happened again, and it stalls AFAICS. Can you reproduce there? # git clone git://source.mvista.com/git/linux-davinci-2.6.git Here I have a constant 55 kbps and it stalls with about 117Mb downloaded. I will try to contact Montavista. Thanks you so much for the support, -- []s, André Goddard ``` ## Linus Torvalds, 2006-07-27 17:50 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0607271049460.4168%40g5.osdl.org In-Reply-To: ``` On Thu, 27 Jul 2006, André Goddard Rosa wrote: > > The problem happened again, and it stalls AFAICS. > Can you reproduce there? > > # git clone git://source.mvista.com/git/linux-davinci-2.6.git Nope. I have a fairly constant 120kbps, and: [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git Checking files out...) 100% (19754/19754) done here Linus ``` ## André Goddard Rosa, 2006-07-27 17:57 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607271057i22c700e0hde1c0893cd638f25%40mail.gmail.com In-Reply-To: ``` On 7/27/06, Linus Torvalds wrote: > > > On Thu, 27 Jul 2006, André Goddard Rosa wrote: > > > > The problem happened again, and it stalls AFAICS. > > Can you reproduce there? > > > > # git clone git://source.mvista.com/git/linux-davinci-2.6.git > > Nope. I have a fairly constant 120kbps, and: > > [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git > Checking files out...) > 100% (19754/19754) done > here Do you think you can reproduce the server timeout by CTRL + Z and stopping the process for a while? I sent an email to the last one who commited in that tree asking for updating the git version there. I also sent and email to the maillist owner asking this. Thanks again, -- []s, André Goddard ``` ## Pavel Roskin, 2006-07-27 18:41 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: <1154025679.13273.5.camel@dv> URL: https://gitlist.dev/e/1154025679.13273.5.camel%40dv In-Reply-To: ``` On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote: > Nope. I have a fairly constant 120kbps, and: > > [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git > Checking files out...) > 100% (19754/19754) done Same thing here. Current git from the master branch. -- Regards, Pavel Roskin ``` ## André Goddard Rosa, 2006-07-27 19:16 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607271216i5b1d8d34n900ffeacbe81f380%40mail.gmail.com In-Reply-To: <1154025679.13273.5.camel@dv> ``` On 7/27/06, Pavel Roskin wrote: > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote: > > Nope. I have a fairly constant 120kbps, and: > > > > [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git > > Checking files out...) > > 100% (19754/19754) done > > Same thing here. Current git from the master branch. Forgot to say that we are using this script in GIT_PROXY_COMMAND environment variable: (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc | (read a; read a; cat ) The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and the second removes an empty line as described here: http://www.gelato.unsw.edu.au/archives/git/0605/20664.html I will try from home later again. Thanks, -- []s, André Goddard ``` ## André Goddard Rosa, 2006-07-28 09:56 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607280256q485b7ae9p9cdedf9b621a0e9b%40mail.gmail.com In-Reply-To: ``` On 7/27/06, André Goddard Rosa wrote: > On 7/27/06, Pavel Roskin wrote: > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote: > > > Nope. I have a fairly constant 120kbps, and: > > > > > > [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git > > > Checking files out...) > > > 100% (19754/19754) done > > > > Same thing here. Current git from the master branch. > > Forgot to say that we are using this script in GIT_PROXY_COMMAND > environment variable: > > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc > | (read a; read a; cat ) > > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and > the second removes an empty line as described here: > > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html > > I will try from home later again. Okey, I tried from home (without the proxy trick) and it behaved a lot better but my disc went full in the process and I got these messages: ... ... ... error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c error: git-checkout-index: unable to create file drivers/scsi/mac_scsi.c (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/mac_scsi.h (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/mca_53c9x.c (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/megaraid.c (No space left on device) error: git-checkout-index: unable to create file drivers/scsi/megaraid.h (No space left on device) fatal: cannot create directory at drivers/scsi/megaraid And it finished keeping the downloaded files, but I still cannot see these files listed above. I tried to pull but it says that I'm up-to-date: doctorture:/opt/downloads/mvista/linux-mvista # git-pull Already up-to-date. I remember that using CVS I just used 'cvs update' after checkout and it would bring the missing files to me. What I'm doing wrong here? Thank you so much, -- []s, André Goddard ``` ## André Goddard Rosa, 2006-07-28 09:58 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607280258s421faf65o11c5dd241e7a27c6%40mail.gmail.com In-Reply-To: ``` On 7/28/06, André Goddard Rosa wrote: > On 7/27/06, André Goddard Rosa wrote: > > On 7/27/06, Pavel Roskin wrote: > > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote: > > > > Nope. I have a fairly constant 120kbps, and: > > > > > > > > [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git > > > > Checking files out...) > > > > 100% (19754/19754) done > > > > > > Same thing here. Current git from the master branch. > > > > Forgot to say that we are using this script in GIT_PROXY_COMMAND > > environment variable: > > > > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc > > | (read a; read a; cat ) > > > > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and > > the second removes an empty line as described here: > > > > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html > > > > I will try from home later again. > > Okey, I tried from home (without the proxy trick) and it behaved a lot > better but my disc went full in the process and I got these messages: > ... > ... > ... > error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c > error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h > error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c > error: git-checkout-index: unable to create file > drivers/scsi/mac_scsi.c (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/mac_scsi.h (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/mca_53c9x.c (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/megaraid.c (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/megaraid.h (No space left on device) > fatal: cannot create directory at drivers/scsi/megaraid > > And it finished keeping the downloaded files, but I still cannot see > these files listed above. > I tried to pull but it says that I'm up-to-date: > > doctorture:/opt/downloads/mvista/linux-mvista # git-pull > Already up-to-date. > > I remember that using CVS I just used 'cvs update' after checkout and > it would bring the missing files to me. > > What I'm doing wrong here? I'm also receiving these messages when trying to change branches: doctorture:/opt/downloads/mvista/linux-mvista # git-checkout origin fatal: Untracked working tree file '.gitignore' would be overwritten by merge. Perhaps I will need to download using git-clone again? Thanks, -- []s, André Goddard ``` ## Ribeiro, Humberto Plinio, 2006-07-28 13:36 Subject: RE: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/EA24FE8B11562848B4760FD5EB4D2E9401B3F3C2%40SAO1016V.ww101.siemens.net In-Reply-To: ``` Hi, Andre. After a "ps -ef" I saw two instances of the script defined by GIT_PROXY_COMMAND. Strangely one of those instances was child of the other. A strace showed the scripts were blocked in a "waitpid(-1,". I've killed the child script and the git clone resumed the process. I didn't understand why this blocking on git happened though. The creation of these two instances (one child of the other) was also strange. Regards, Humberto --- Obrigado/Best regards/Mit freundlichen Grüßen/Pozdrawiam Humberto Ribeiro, MSc. SIEMENS Home and Office Communication Phone: +55 92 2129 6205 Mobile: +55 92 8146 3147 -----Original Message----- From: André Goddard Rosa [mailto:andre.goddard@gmail.com] Sent: sexta-feira, 28 de julho de 2006 05:58 To: Pavel Roskin Cc: Git Mailing List; Linus Torvalds; Ribeiro, Humberto Plinio Subject: Re: Git clone stalls at a read(3, ...) saw using strace On 7/28/06, André Goddard Rosa wrote: > On 7/27/06, André Goddard Rosa wrote: > > On 7/27/06, Pavel Roskin wrote: > > > On Thu, 2006-07-27 at 10:50 -0700, Linus Torvalds wrote: > > > > Nope. I have a fairly constant 120kbps, and: > > > > > > > > [torvalds@g5 ~]$ git clone git://source.mvista.com/git/linux-davinci-2.6.git > > > > Checking files out...) > > > > 100% (19754/19754) done > > > > > > Same thing here. Current git from the master branch. > > > > Forgot to say that we are using this script in GIT_PROXY_COMMAND > > environment variable: > > > > (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc > > | (read a; read a; cat ) > > > > The first 'read a' removes the 'CONNECT SUCCESS HTTP RESPONSE 200' and > > the second removes an empty line as described here: > > > > http://www.gelato.unsw.edu.au/archives/git/0605/20664.html > > > > I will try from home later again. > > Okey, I tried from home (without the proxy trick) and it behaved a lot > better but my disc went full in the process and I got these messages: > ... > ... > ... > error: git-checkout-index: unable to write file drivers/scsi/mac53c94.c > error: git-checkout-index: unable to write file drivers/scsi/mac53c94.h > error: git-checkout-index: unable to write file drivers/scsi/mac_esp.c > error: git-checkout-index: unable to create file > drivers/scsi/mac_scsi.c (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/mac_scsi.h (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/mca_53c9x.c (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/megaraid.c (No space left on device) > error: git-checkout-index: unable to create file > drivers/scsi/megaraid.h (No space left on device) > fatal: cannot create directory at drivers/scsi/megaraid > > And it finished keeping the downloaded files, but I still cannot see > these files listed above. > I tried to pull but it says that I'm up-to-date: > > doctorture:/opt/downloads/mvista/linux-mvista # git-pull > Already up-to-date. > > I remember that using CVS I just used 'cvs update' after checkout and > it would bring the missing files to me. > > What I'm doing wrong here? I'm also receiving these messages when trying to change branches: doctorture:/opt/downloads/mvista/linux-mvista # git-checkout origin fatal: Untracked working tree file '.gitignore' would be overwritten by merge. Perhaps I will need to download using git-clone again? Thanks, -- []s, André Goddard ``` ## André Goddard Rosa, 2006-07-28 13:57 Subject: Re: Git clone stalls at a read(3, ...) saw using strace Message-ID: URL: https://gitlist.dev/e/b8bf37780607280657x1c4c88d1ofd0734ddc4642d02%40mail.gmail.com In-Reply-To: ``` On 7/28/06, Ribeiro, Humberto Plinio wrote: > Hi, Andre. > > After a "ps -ef" I saw two instances of the script defined by > GIT_PROXY_COMMAND. Strangely one of those instances was child > of the other. A strace showed the scripts were blocked in a >"waitpid(-1,". I've killed the child script and the git clone resumed the >process. > > I didn't understand why this blocking on git happened though. The > creation of these two instances (one child of the other) was also > strange. Yes, it worked to me too, but I also don't know why. Let me explain exactly what we had to do so someone perhaps can explain why we had to do this: I configured GIT_PROXY_COMMAND pointing to a script with this content: (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | nc 172.29.0.6 3128 | (read a; read a; cat ) Later: git clone git://source.mvista.com/git/linux-davinci-2.6.git After some time (downloading 117 Mb) it stalled (git-fetch on a pipe read) and we had to do this to resume and let it finish: [root@mao2wx23 tmp]# ps -ef | grep git | grep -v grep opb694 9975 1451 0 01:39 pts/11 00:00:00 /bin/sh /usr/bin/git-clone git://source.mvista.com/git/linux-davinci-2.6.git opb694 9985 9975 3 01:39 pts/11 00:01:56 git-fetch-pack --all -k git //source.mvista.com git/linux-davinci-2.6.git [root@mao2wx23 tmp]# ps -ef | grep proxy | grep -v grep opb694 9986 9985 0 01:39 pts/11 00:00:00 /bin/sh /home/opb694/proxy-cmd.sh source.mvista.com 9418 opb694 9987 9986 0 01:39 pts/11 00:00:00 /bin/sh /home/opb694/proxy-cmd.sh source.mvista.com 9418 [root@mao2wx23 tmp]# kill 9987 Thanks again for the patience, -- []s, André Goddard ```