threads / discuss / 19761

git clone failing over slow links?

Subject: git clone failing over slow links?

## tl;dr

6 messages between Jun 9, 2009 and Jun 9, 2009.

replies: 5people: 3as markdown or json

skillzero@gmail.com· Jun 9, 2009, 15:52 UTC · lore

People with slower connections are having problems cloning my repository. The server seems to just drop the connection. This repository is about 300 MB. The server is using git 1.6.1. This is using an ssh URL. They've tried many times and it usually dies at different places in the clone (5%, 80%, etc., with one successful clone so far for them).

remote: Counting objects: 121290, done. remote: Compressing objects: 100% (82778/82778), done. fatal: The remote end hung up unexpectedly62.14 MiB | 147 KiB/s fatal: early EOF fatal: index-pack failed

Are there any known issues with git 1.6.1 that might cause this? I can try to get the server updated to 1.6.2.3, but I was curious if anyone knew of any specific issues like this. The server is running Mac OS X, if that matters.

Robin H. Johnson· Jun 9, 2009, 17:12 UTC · re: skillzero@gmail.com · lore

Re: git clone failing over slow links?

On Tue, Jun 09, 2009 at 08:52:10AM -0700, skillzero@gmail.com wrote:

Show 6 quoted lines

> People with slower connections are having problems cloning my
> repository. The server seems to just drop the connection. This
> repository is about 300 MB. The server is using git 1.6.1. This is
> using an ssh URL. They've tried many times and it usually dies at
> different places in the clone (5%, 80%, etc., with one successful
> clone so far for them).

For Gentoo, we saw what I think was same problem early on with our conversion experiments. In our case however, it's going to be a non-problem as we intend on disabling initial clone and requiring use of a bundle.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

Nicolas Pitre· Jun 9, 2009, 18:42 UTC · re: Robin H. Johnson · lore

Re: git clone failing over slow links?

On Tue, 9 Jun 2009, Robin H. Johnson wrote:

Show 11 quoted lines

> On Tue, Jun 09, 2009 at 08:52:10AM -0700, skillzero@gmail.com wrote:
> > People with slower connections are having problems cloning my
> > repository. The server seems to just drop the connection. This
> > repository is about 300 MB. The server is using git 1.6.1. This is
> > using an ssh URL. They've tried many times and it usually dies at
> > different places in the clone (5%, 80%, etc., with one successful
> > clone so far for them).
> For Gentoo, we saw what I think was same problem early on with our
> conversion experiments. In our case however, it's going to be a
> non-problem as we intend on disabling initial clone and requiring use of
> a bundle.

In the Gentoo case, I think your problem was rather about the memory usage impact of an initial clone and nothing to do with the speed of the connection.

Nicolas

Robin H. Johnson· Jun 9, 2009, 18:49 UTC · re: Nicolas Pitre · lore

Re: git clone failing over slow links?

On Tue, Jun 09, 2009 at 02:42:46PM -0400, Nicolas Pitre wrote:

Show 7 quoted lines

> > For Gentoo, we saw what I think was same problem early on with our
> > conversion experiments. In our case however, it's going to be a
> > non-problem as we intend on disabling initial clone and requiring use of
> > a bundle.
> In the Gentoo case, I think your problem was rather about the memory 
> usage impact of an initial clone and nothing to do with the speed of the 
> connection.

That was true early on, but it still persists for users on slow connections.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Infra Guy
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

Nicolas Pitre· Jun 9, 2009, 18:55 UTC · re: skillzero@gmail.com · lore

Re: git clone failing over slow links?

On Tue, 9 Jun 2009, skillzero@gmail.com wrote:

Show 6 quoted lines

> People with slower connections are having problems cloning my
> repository. The server seems to just drop the connection. This
> repository is about 300 MB. The server is using git 1.6.1. This is
> using an ssh URL. They've tried many times and it usually dies at
> different places in the clone (5%, 80%, etc., with one successful
> clone so far for them).

Do you clearly have a set of people for whom the clone works and another set with issues?

> remote: Counting objects: 121290, done.
> remote: Compressing objects: 100% (82778/82778), done.
> fatal: The remote end hung up unexpectedly62.14 MiB | 147 KiB/s

147 KiB/s is not particularly slow either. What is fast enough for the clone to usually succeed?

Also, since this is through an ssh connection, do you have anything in your syslog from sshd (usually in /var/log/secure) about anything unusual for the connection to go down?

> Are there any known issues with git 1.6.1 that might cause this? I can
> try to get the server updated to 1.6.2.3, but I was curious if anyone
> knew of any specific issues like this. The server is running Mac OS X,
> if that matters.

Maybe it does. Major sites using git often use Linux and we would have heard from them if anything like your problem was seen there.

Nicolas

skillzero@gmail.com· Jun 9, 2009, 19:58 UTC · re: Nicolas Pitre · lore

Re: git clone failing over slow links?

On Tue, Jun 9, 2009 at 11:55 AM, Nicolas Pitre<nico@cam.org> wrote:

>
> Do you clearly have a set of people for whom the clone works and another
> set with issues?

Yes. Several us are able to reliably clone every time. A few users are able to clone once or twice (maybe 1 out of every 10 times), but otherwise, it fails when the server drops the connection.

From what they said, it looked like read_in_full() (in wrapper.c) isn't getting a full buffer. In two cases, 4077 or 4082 out of 4097 bytes. xread is returning 0 as you'd expect for an EOF.

> 147 KiB/s is not particularly slow either.  What is fast enough for the
> clone to usually succeed?

Mine is 500 KB/sec where it always succeeds. I can try using dummynet to simulate a slow link though.

> Also, since this is through an ssh connection, do you have anything in
> your syslog from sshd (usually in /var/log/secure) about anything
> unusual for the connection to go down?

I'm trying to get that from the server admin now.