# git clone failing over slow links? 6 messages from 2009-06-09 to 2009-06-09. Participants: skillzero@gmail.com, Robin H. Johnson, Nicolas Pitre. Thread: https://gitlist.dev/t/19761 ## skillzero@gmail.com, 2009-06-09 15:52 Subject: git clone failing over slow links? Message-ID: <2729632a0906090852r124eff8eo8391ae90cbda5872@mail.gmail.com> URL: https://gitlist.dev/e/2729632a0906090852r124eff8eo8391ae90cbda5872%40mail.gmail.com ``` People with slower connections are having problems cloning my repository. The server seems to just drop the connection. This repository is about 300 MB. The server is using git 1.6.1. This is using an ssh URL. They've tried many times and it usually dies at different places in the clone (5%, 80%, etc., with one successful clone so far for them). remote: Counting objects: 121290, done. remote: Compressing objects: 100% (82778/82778), done. fatal: The remote end hung up unexpectedly62.14 MiB | 147 KiB/s fatal: early EOF fatal: index-pack failed Are there any known issues with git 1.6.1 that might cause this? I can try to get the server updated to 1.6.2.3, but I was curious if anyone knew of any specific issues like this. The server is running Mac OS X, if that matters. ``` ## Robin H. Johnson, 2009-06-09 17:12 Subject: Re: git clone failing over slow links? Message-ID: URL: https://gitlist.dev/e/robbat2-20090609T171032-419739557Z%40orbis-terrarum.net In-Reply-To: <2729632a0906090852r124eff8eo8391ae90cbda5872@mail.gmail.com> ``` On Tue, Jun 09, 2009 at 08:52:10AM -0700, skillzero@gmail.com wrote: > People with slower connections are having problems cloning my > repository. The server seems to just drop the connection. This > repository is about 300 MB. The server is using git 1.6.1. This is > using an ssh URL. They've tried many times and it usually dies at > different places in the clone (5%, 80%, etc., with one successful > clone so far for them). For Gentoo, we saw what I think was same problem early on with our conversion experiments. In our case however, it's going to be a non-problem as we intend on disabling initial clone and requiring use of a bundle. -- Robin Hugh Johnson Gentoo Linux Developer & Infra Guy E-Mail : robbat2@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 ``` ## Nicolas Pitre, 2009-06-09 18:42 Subject: Re: git clone failing over slow links? Message-ID: URL: https://gitlist.dev/e/alpine.LFD.2.00.0906091440100.31536%40xanadu.home In-Reply-To: ``` On Tue, 9 Jun 2009, Robin H. Johnson wrote: > On Tue, Jun 09, 2009 at 08:52:10AM -0700, skillzero@gmail.com wrote: > > People with slower connections are having problems cloning my > > repository. The server seems to just drop the connection. This > > repository is about 300 MB. The server is using git 1.6.1. This is > > using an ssh URL. They've tried many times and it usually dies at > > different places in the clone (5%, 80%, etc., with one successful > > clone so far for them). > For Gentoo, we saw what I think was same problem early on with our > conversion experiments. In our case however, it's going to be a > non-problem as we intend on disabling initial clone and requiring use of > a bundle. In the Gentoo case, I think your problem was rather about the memory usage impact of an initial clone and nothing to do with the speed of the connection. Nicolas ``` ## Robin H. Johnson, 2009-06-09 18:49 Subject: Re: git clone failing over slow links? Message-ID: URL: https://gitlist.dev/e/robbat2-20090609T184834-913673685Z%40orbis-terrarum.net In-Reply-To: ``` On Tue, Jun 09, 2009 at 02:42:46PM -0400, Nicolas Pitre wrote: > > For Gentoo, we saw what I think was same problem early on with our > > conversion experiments. In our case however, it's going to be a > > non-problem as we intend on disabling initial clone and requiring use of > > a bundle. > In the Gentoo case, I think your problem was rather about the memory > usage impact of an initial clone and nothing to do with the speed of the > connection. That was true early on, but it still persists for users on slow connections. -- Robin Hugh Johnson Gentoo Linux Developer & Infra Guy E-Mail : robbat2@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 ``` ## Nicolas Pitre, 2009-06-09 18:55 Subject: Re: git clone failing over slow links? Message-ID: URL: https://gitlist.dev/e/alpine.LFD.2.00.0906091443050.31536%40xanadu.home In-Reply-To: <2729632a0906090852r124eff8eo8391ae90cbda5872@mail.gmail.com> ``` On Tue, 9 Jun 2009, skillzero@gmail.com wrote: > People with slower connections are having problems cloning my > repository. The server seems to just drop the connection. This > repository is about 300 MB. The server is using git 1.6.1. This is > using an ssh URL. They've tried many times and it usually dies at > different places in the clone (5%, 80%, etc., with one successful > clone so far for them). Do you clearly have a set of people for whom the clone works and another set with issues? > remote: Counting objects: 121290, done. > remote: Compressing objects: 100% (82778/82778), done. > fatal: The remote end hung up unexpectedly62.14 MiB | 147 KiB/s 147 KiB/s is not particularly slow either. What is fast enough for the clone to usually succeed? Also, since this is through an ssh connection, do you have anything in your syslog from sshd (usually in /var/log/secure) about anything unusual for the connection to go down? > Are there any known issues with git 1.6.1 that might cause this? I can > try to get the server updated to 1.6.2.3, but I was curious if anyone > knew of any specific issues like this. The server is running Mac OS X, > if that matters. Maybe it does. Major sites using git often use Linux and we would have heard from them if anything like your problem was seen there. Nicolas ``` ## skillzero@gmail.com, 2009-06-09 19:58 Subject: Re: git clone failing over slow links? Message-ID: <2729632a0906091258v7ec334faue725c71bdd11165c@mail.gmail.com> URL: https://gitlist.dev/e/2729632a0906091258v7ec334faue725c71bdd11165c%40mail.gmail.com In-Reply-To: ``` On Tue, Jun 9, 2009 at 11:55 AM, Nicolas Pitre wrote: > > Do you clearly have a set of people for whom the clone works and another > set with issues? Yes. Several us are able to reliably clone every time. A few users are able to clone once or twice (maybe 1 out of every 10 times), but otherwise, it fails when the server drops the connection. From what they said, it looked like read_in_full() (in wrapper.c) isn't getting a full buffer. In two cases, 4077 or 4082 out of 4097 bytes. xread is returning 0 as you'd expect for an EOF. > 147 KiB/s is not particularly slow either.  What is fast enough for the > clone to usually succeed? Mine is 500 KB/sec where it always succeeds. I can try using dummynet to simulate a slow link though. > Also, since this is through an ssh connection, do you have anything in > your syslog from sshd (usually in /var/log/secure) about anything > unusual for the connection to go down? I'm trying to get that from the server admin now. ```