{"thread":{"id":"19761","subject":"git clone failing over slow links?","startedAt":"2009-06-09T15:52:10Z","lastAt":"2009-06-09T19:58:57Z","messageCount":6,"participants":["skillzero@gmail.com","Robin H. Johnson","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"115910","messageId":"2729632a0906090852r124eff8eo8391ae90cbda5872@mail.gmail.com","threadId":"19761","inReplyTo":null,"subject":"git clone failing over slow links?","fromName":"","fromEmail":"skillzero@gmail.com","sentAt":"2009-06-09T15:52:10Z","receivedAt":"2009-06-09T15:52:10Z","isPatch":false,"sender":{"key":"skillzero@gmail.com","avatar":null},"body":"People with slower connections are having problems cloning my\nrepository. The server seems to just drop the connection. This\nrepository is about 300 MB. The server is using git 1.6.1. This is\nusing an ssh URL. They've tried many times and it usually dies at\ndifferent places in the clone (5%, 80%, etc., with one successful\nclone so far for them).\n\nremote: Counting objects: 121290, done.\nremote: Compressing objects: 100% (82778/82778), done.\nfatal: The remote end hung up unexpectedly62.14 MiB | 147 KiB/s\nfatal: early EOF\nfatal: index-pack failed\n\nAre there any known issues with git 1.6.1 that might cause this? I can\ntry to get the server updated to 1.6.2.3, but I was curious if anyone\nknew of any specific issues like this. The server is running Mac OS X,\nif that matters.\n"},{"id":"115923","messageId":"robbat2-20090609T171032-419739557Z@orbis-terrarum.net","threadId":"19761","inReplyTo":"2729632a0906090852r124eff8eo8391ae90cbda5872@mail.gmail.com","subject":"Re: git clone failing over slow links?","fromName":"Robin H. Johnson","fromEmail":"robbat2@gentoo.org","sentAt":"2009-06-09T17:12:32Z","receivedAt":"2009-06-09T17:12:32Z","isPatch":false,"sender":{"key":"robbat2@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/373898?v=4"},"body":"On Tue, Jun 09, 2009 at 08:52:10AM -0700, skillzero@gmail.com wrote:\n> People with slower connections are having problems cloning my\n> repository. The server seems to just drop the connection. This\n> repository is about 300 MB. The server is using git 1.6.1. This is\n> using an ssh URL. They've tried many times and it usually dies at\n> different places in the clone (5%, 80%, etc., with one successful\n> clone so far for them).\nFor Gentoo, we saw what I think was same problem early on with our\nconversion experiments. In our case however, it's going to be a\nnon-problem as we intend on disabling initial clone and requiring use of\na bundle.\n\n-- \nRobin Hugh Johnson\nGentoo Linux Developer & Infra Guy\nE-Mail     : robbat2@gentoo.org\nGnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85\n"},{"id":"115934","messageId":"alpine.LFD.2.00.0906091440100.31536@xanadu.home","threadId":"19761","inReplyTo":"robbat2-20090609T171032-419739557Z@orbis-terrarum.net","subject":"Re: git clone failing over slow links?","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2009-06-09T18:42:46Z","receivedAt":"2009-06-09T18:42:46Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 9 Jun 2009, Robin H. Johnson wrote:\n\n> On Tue, Jun 09, 2009 at 08:52:10AM -0700, skillzero@gmail.com wrote:\n> > People with slower connections are having problems cloning my\n> > repository. The server seems to just drop the connection. This\n> > repository is about 300 MB. The server is using git 1.6.1. This is\n> > using an ssh URL. They've tried many times and it usually dies at\n> > different places in the clone (5%, 80%, etc., with one successful\n> > clone so far for them).\n> For Gentoo, we saw what I think was same problem early on with our\n> conversion experiments. In our case however, it's going to be a\n> non-problem as we intend on disabling initial clone and requiring use of\n> a bundle.\n\nIn the Gentoo case, I think your problem was rather about the memory \nusage impact of an initial clone and nothing to do with the speed of the \nconnection.\n\n\nNicolas\n"},{"id":"115935","messageId":"robbat2-20090609T184834-913673685Z@orbis-terrarum.net","threadId":"19761","inReplyTo":"alpine.LFD.2.00.0906091440100.31536@xanadu.home","subject":"Re: git clone failing over slow links?","fromName":"Robin H. Johnson","fromEmail":"robbat2@gentoo.org","sentAt":"2009-06-09T18:49:18Z","receivedAt":"2009-06-09T18:49:18Z","isPatch":false,"sender":{"key":"robbat2@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/373898?v=4"},"body":"On Tue, Jun 09, 2009 at 02:42:46PM -0400, Nicolas Pitre wrote:\n> > For Gentoo, we saw what I think was same problem early on with our\n> > conversion experiments. In our case however, it's going to be a\n> > non-problem as we intend on disabling initial clone and requiring use of\n> > a bundle.\n> In the Gentoo case, I think your problem was rather about the memory \n> usage impact of an initial clone and nothing to do with the speed of the \n> connection.\nThat was true early on, but it still persists for users on slow\nconnections.\n\n-- \nRobin Hugh Johnson\nGentoo Linux Developer & Infra Guy\nE-Mail     : robbat2@gentoo.org\nGnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85\n"},{"id":"115937","messageId":"alpine.LFD.2.00.0906091443050.31536@xanadu.home","threadId":"19761","inReplyTo":"2729632a0906090852r124eff8eo8391ae90cbda5872@mail.gmail.com","subject":"Re: git clone failing over slow links?","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2009-06-09T18:55:29Z","receivedAt":"2009-06-09T18:55:29Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Tue, 9 Jun 2009, skillzero@gmail.com wrote:\n\n> People with slower connections are having problems cloning my\n> repository. The server seems to just drop the connection. This\n> repository is about 300 MB. The server is using git 1.6.1. This is\n> using an ssh URL. They've tried many times and it usually dies at\n> different places in the clone (5%, 80%, etc., with one successful\n> clone so far for them).\n\nDo you clearly have a set of people for whom the clone works and another \nset with issues?\n\n> remote: Counting objects: 121290, done.\n> remote: Compressing objects: 100% (82778/82778), done.\n> fatal: The remote end hung up unexpectedly62.14 MiB | 147 KiB/s\n\n147 KiB/s is not particularly slow either.  What is fast enough for the \nclone to usually succeed?\n\nAlso, since this is through an ssh connection, do you have anything in \nyour syslog from sshd (usually in /var/log/secure) about anything \nunusual for the connection to go down?\n\n> Are there any known issues with git 1.6.1 that might cause this? I can\n> try to get the server updated to 1.6.2.3, but I was curious if anyone\n> knew of any specific issues like this. The server is running Mac OS X,\n> if that matters.\n\nMaybe it does.  Major sites using git often use Linux and we would have \nheard from them if anything like your problem was seen there.\n\n\nNicolas\n"},{"id":"115948","messageId":"2729632a0906091258v7ec334faue725c71bdd11165c@mail.gmail.com","threadId":"19761","inReplyTo":"alpine.LFD.2.00.0906091443050.31536@xanadu.home","subject":"Re: git clone failing over slow links?","fromName":"","fromEmail":"skillzero@gmail.com","sentAt":"2009-06-09T19:58:57Z","receivedAt":"2009-06-09T19:58:57Z","isPatch":false,"sender":{"key":"skillzero@gmail.com","avatar":null},"body":"On Tue, Jun 9, 2009 at 11:55 AM, Nicolas Pitre<nico@cam.org> wrote:\n>\n> Do you clearly have a set of people for whom the clone works and another\n> set with issues?\n\nYes. Several us are able to reliably clone every time. A few users are\nable to clone once or twice (maybe 1 out of every 10 times), but\notherwise, it fails when the server drops the connection.\n\nFrom what they said, it looked like read_in_full() (in wrapper.c)\nisn't getting a full buffer. In two cases, 4077 or 4082 out of 4097\nbytes. xread is returning 0 as you'd expect for an EOF.\n\n> 147 KiB/s is not particularly slow either.  What is fast enough for the\n> clone to usually succeed?\n\nMine is 500 KB/sec where it always succeeds. I can try using dummynet\nto simulate a slow link though.\n\n> Also, since this is through an ssh connection, do you have anything in\n> your syslog from sshd (usually in /var/log/secure) about anything\n> unusual for the connection to go down?\n\nI'm trying to get that from the server admin now.\n"}]}