{"thread":{"id":"44558","subject":"Partial fetch?","startedAt":"2016-11-28T20:34:57Z","lastAt":"2016-11-30T00:30:50Z","messageCount":3,"participants":["Dāvis Mosāns","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"306517","messageId":"CAOE4rSzTq6DVR2ch+as9Pbo35NjKP5b1+Ub1XZWEnwJTahqEfg@mail.gmail.com","threadId":"44558","inReplyTo":null,"subject":"Partial fetch?","fromName":"Dāvis Mosāns","fromEmail":"davispuh@gmail.com","sentAt":"2016-11-28T20:34:51Z","receivedAt":"2016-11-28T20:34:57Z","isPatch":false,"sender":{"key":"davispuh@gmail.com","avatar":null},"body":"I'm trying to fetch a remote repository over https but sadly it\ntimeouts too soon.\n\n$ git fetch -v upstream\nPOST git-upload-pack (gzip 1148 to 641 bytes)\nPOST git-upload-pack (gzip 1148 to 644 bytes)\nPOST git-upload-pack (gzip 1948 to 1038 bytes)\nPOST git-upload-pack (gzip 3548 to 1845 bytes)\nPOST git-upload-pack (gzip 6748 to 3431 bytes)\nPOST git-upload-pack (gzip 13148 to 6659 bytes)\nPOST git-upload-pack (gzip 25948 to 13084 bytes)\nPOST git-upload-pack (gzip 51548 to 25997 bytes)\nPOST git-upload-pack (gzip 102748 to 51375 bytes)\nPOST git-upload-pack (gzip 205148 to 101973 bytes)\nPOST git-upload-pack (gzip 409948 to 203200 bytes)\nPOST git-upload-pack (gzip 82248 to 41185 bytes)\nPOST git-upload-pack (gzip 90448 to 45257 bytes)\nPOST git-upload-pack (gzip 99448 to 49719 bytes)\nPOST git-upload-pack (gzip 109348 to 54623 bytes)\nPOST git-upload-pack (gzip 120248 to 59937 bytes)\nPOST git-upload-pack (gzip 132248 to 65839 bytes)\nPOST git-upload-pack (gzip 145448 to 72450 bytes)\nPOST git-upload-pack (gzip 159948 to 79682 bytes)\nPOST git-upload-pack (gzip 175898 to 87436 bytes)\nPOST git-upload-pack (gzip 193448 to 96171 bytes)\nPOST git-upload-pack (gzip 212748 to 105783 bytes)\nPOST git-upload-pack (gzip 233998 to 116207 bytes)\nPOST git-upload-pack (gzip 257398 to 127793 bytes)\nPOST git-upload-pack (gzip 283098 to 140511 bytes)\nPOST git-upload-pack (gzip 311348 to 154447 bytes)\nPOST git-upload-pack (gzip 342448 to 169815 bytes)\nPOST git-upload-pack (gzip 376648 to 186850 bytes)\nPOST git-upload-pack (gzip 414298 to 205408 bytes)\nPOST git-upload-pack (gzip 455698 to 225879 bytes)\nPOST git-upload-pack (gzip 501248 to 248389 bytes)\nPOST git-upload-pack (gzip 551298 to 273131 bytes)\nPOST git-upload-pack (gzip 606398 to 300334 bytes)\nPOST git-upload-pack (gzip 666998 to 330414 bytes)\nPOST git-upload-pack (gzip 733698 to 363387 bytes)\nPOST git-upload-pack (gzip 807048 to 399647 bytes)\nPOST git-upload-pack (gzip 689453 to 341449 bytes)\nerror: RPC failed; HTTP 504 curl 22 The requested URL returned error:\n504 Gateway Time-out\nfatal: The remote end hung up unexpectedly\n\nIs there some way to fetch partially by smaller chunks and then repeat\nthat again till everything is fetched?\n\nThanks!\n"},{"id":"306578","messageId":"20161129215538.vhgmxpq4rasri4vm@sigill.intra.peff.net","threadId":"44558","inReplyTo":"CAOE4rSzTq6DVR2ch+as9Pbo35NjKP5b1+Ub1XZWEnwJTahqEfg@mail.gmail.com","subject":"Re: Partial fetch?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-11-29T21:55:38Z","receivedAt":"2016-11-29T21:56:47Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Nov 28, 2016 at 10:34:51PM +0200, Dāvis Mosāns wrote:\n\n> I'm trying to fetch a remote repository over https but sadly it\n> timeouts too soon.\n> \n> $ git fetch -v upstream\n> POST git-upload-pack (gzip 1148 to 641 bytes)\n> POST git-upload-pack (gzip 1148 to 644 bytes)\n> [...]\n> Is there some way to fetch partially by smaller chunks and then repeat\n> that again till everything is fetched?\n\nNot an easy one. The series of POSTs is an indication that the fetch\nnegotiation is going on for a long time, which probably means you have a\nlot of commits in your local repository that aren't in the remote, or\nvice versa.\n\nHere are the things I might try:\n\n  - git v2.10.2 has commit 06b3d386e (fetch-pack: do not reset in_vain\n    on non-novel acks, 2016-09-23), which may help with this.\n\n  - HTTP, because the server is stateless, performs less well than other\n    protocols. If you can fetch over ssh or git://, it will probably\n    just work.\n\n  - If this is a one-time thing to fetch unrelated history from another\n    repository, you can \"clone --mirror\" instead of fetching,\n    then fetch from the mirror locally. Subsequent fetches should be\n    fast.\n\nIf you do try v2.10.2 and it improves things, I'd be interested to hear\nabout it as a data point.\n\n-Peff\n"},{"id":"306584","messageId":"CAOE4rSxLNh0eWndroNnWsutR3FfLa=QzKPwFPcKOhwSqkKtjLw@mail.gmail.com","threadId":"44558","inReplyTo":"20161129215538.vhgmxpq4rasri4vm@sigill.intra.peff.net","subject":"Re: Partial fetch?","fromName":"Dāvis Mosāns","fromEmail":"davispuh@gmail.com","sentAt":"2016-11-30T00:30:44Z","receivedAt":"2016-11-30T00:30:50Z","isPatch":false,"sender":{"key":"davispuh@gmail.com","avatar":null},"body":"2016-11-29 23:55 GMT+02:00 Jeff King <peff@peff.net>:\n> On Mon, Nov 28, 2016 at 10:34:51PM +0200, Dāvis Mosāns wrote:\n>\n>> I'm trying to fetch a remote repository over https but sadly it\n>> timeouts too soon.\n>>\n>> $ git fetch -v upstream\n>> POST git-upload-pack (gzip 1148 to 641 bytes)\n>> POST git-upload-pack (gzip 1148 to 644 bytes)\n>> [...]\n>> Is there some way to fetch partially by smaller chunks and then repeat\n>> that again till everything is fetched?\n>\n> Not an easy one. The series of POSTs is an indication that the fetch\n> negotiation is going on for a long time, which probably means you have a\n> lot of commits in your local repository that aren't in the remote, or\n> vice versa.\n>\n> Here are the things I might try:\n>\n>   - git v2.10.2 has commit 06b3d386e (fetch-pack: do not reset in_vain\n>     on non-novel acks, 2016-09-23), which may help with this.\n>\n\nThat output and this is already with git v2.10.2\n\n>   - HTTP, because the server is stateless, performs less well than other\n>     protocols. If you can fetch over ssh or git://, it will probably\n>     just work.\n>\n\nIt's only available under https://git.replicant.us/replicant/frameworks_base.git\n\n>   - If this is a one-time thing to fetch unrelated history from another\n>     repository, you can \"clone --mirror\" instead of fetching,\n>     then fetch from the mirror locally. Subsequent fetches should be\n>     fast.\n>\n\nLooks like something is wrong with their server/setup since currently\neven clone doesn't work for me.\n\n Cloning into bare repository 'frameworks_base.git'...\nremote: Counting objects: 739930, done.\nremote: Compressing objects: 100% (196567/196567), done.\nerror: RPC failed; curl 18 transfer closed with outstanding read data remaining\nfatal: The remote end hung up unexpectedly\nfatal: early EOF\nfatal: index-pack failed\n\nBut someone have made a mirror for it and there clone works only still\nsame issue with fetch.\nOnce I will have this cloned I will try local fetch.\n\nThanks!\n"}]}