{"thread":{"id":"2084","subject":"maybe breakage with latest git-pull and http protocol","startedAt":"2005-10-13T00:53:03Z","lastAt":"2005-10-20T17:43:24Z","messageCount":17,"participants":["Randal L. Schwartz","Junio C Hamano","Daniel Barkalow","Nick Hengeveld"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"10049","messageId":"867jciz18w.fsf@blue.stonehenge.com","threadId":"2084","inReplyTo":null,"subject":"maybe breakage with latest git-pull and http protocol","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2005-10-13T00:53:03Z","receivedAt":"2005-10-13T00:53:03Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":"\nI updated git to d06b689a933f6d2130f8afdf1ac0ddb83eeb59ab,\nthen compiled and installed.\n\nWhen I went to \"git-pull\" on my cogito archive (which I had edited\nto use HTTP instead of RSYNC), I got into trouble.  Unfortunately,\nI changed it to rsync to force cogito into a sane state before\nI realized that this would be a good bug report. :)\n\nThis is perhaps just a heads-up that the recent git-pull might be\nbroken with respect to http updates.\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\nSee PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"10054","messageId":"7vr7aq55kd.fsf@assigned-by-dhcp.cox.net","threadId":"2084","inReplyTo":"867jciz18w.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-13T05:50:10Z","receivedAt":"2005-10-13T05:50:10Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"merlyn@stonehenge.com (Randal L. Schwartz) writes:\n\n> I updated git to d06b689a933f6d2130f8afdf1ac0ddb83eeb59ab,\n> then compiled and installed.\n>\n> When I went to \"git-pull\" on my cogito archive (which I had edited\n> to use HTTP instead of RSYNC), I got into trouble.  Unfortunately,\n> I changed it to rsync to force cogito into a sane state before\n> I realized that this would be a good bug report. :)\n\nIndeed I wish we could see the set of refs you had and output\nfrom fsck-objects before the failed git-pull and after.\n\nOne thing I am aware of is that cogito repository at kernel.org\nis not set up to be HTTP friendly -- it lacks info/refs file\ngit-clone uses for discovery of the available refs.\n\nCogito's clone/fetch over HTTP uses recursive wget for\ndiscovery, and I presume that is one of the reasons nobody\nnoticed this.  Another reason may probably be that more people\nuse rsync transport.\n"},{"id":"10098","messageId":"864q7kqsa4.fsf@blue.stonehenge.com","threadId":"2084","inReplyTo":"867jciz18w.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2005-10-14T10:58:27Z","receivedAt":"2005-10-14T10:58:27Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Randal\" == Randal L Schwartz <merlyn@stonehenge.com> writes:\n\nRandal> I updated git to d06b689a933f6d2130f8afdf1ac0ddb83eeb59ab,\nRandal> then compiled and installed.\n\nRandal> When I went to \"git-pull\" on my cogito archive (which I had edited\nRandal> to use HTTP instead of RSYNC), I got into trouble.  Unfortunately,\nRandal> I changed it to rsync to force cogito into a sane state before\nRandal> I realized that this would be a good bug report. :)\n\nRandal> This is perhaps just a heads-up that the recent git-pull might be\nRandal> broken with respect to http updates.\n\nEven after updating git this morning, git-pull still seems to be broken\nwith respect to http://www.kernel.org/.\nIs http pulling broken for good now?  Or is someone looking at this?\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\nSee PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"10102","messageId":"7vhdbkt8ad.fsf@assigned-by-dhcp.cox.net","threadId":"2084","inReplyTo":"864q7kqsa4.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-14T15:42:02Z","receivedAt":"2005-10-14T15:42:02Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"merlyn@stonehenge.com (Randal L. Schwartz) writes:\n\n> Even after updating git this morning, git-pull still seems to be broken\n> with respect to http://www.kernel.org/.\n> Is http pulling broken for good now?  Or is someone looking at this?\n\nSorry, but this is not a description of your problem helpful\nenough for someone who is willing to look at it, I am afraid.\nhttp://www.kernel.org/ has 80 or so repos (I counted about a\nmonth ago so it may probably have more by now) --- which ones?\n\nI have local repositories used only to test pulling into them,\nand I pull from Linus 2.6 kernel, and my own git repository,\nevery other day or so, but haven't seen breakage, so I do not\nthink it is http://www.kernel.org/. in general.  If some\nparticular repository is not set up HTTP friendly I would\nunderstand.\n\nAlso how does it fail?  Does cloning from scratch succeed but\nupdating a repo that was in sync a few days ago fail?  Does it\ndie silently and you find the breakage by running fsck-object,\nor does it fail loudly with error messages?  If the latter what\ndoes it say?\n"},{"id":"10104","messageId":"86achcoyvz.fsf@blue.stonehenge.com","threadId":"2084","inReplyTo":"7vhdbkt8ad.fsf@assigned-by-dhcp.cox.net","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2005-10-14T16:18:40Z","receivedAt":"2005-10-14T16:18:40Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Junio\" == Junio C Hamano <junkio@cox.net> writes:\n\nJunio> merlyn@stonehenge.com (Randal L. Schwartz) writes:\n>> Even after updating git this morning, git-pull still seems to be broken\n>> with respect to http://www.kernel.org/.\n>> Is http pulling broken for good now?  Or is someone looking at this?\n\nJunio> Sorry, but this is not a description of your problem helpful\nJunio> enough for someone who is willing to look at it, I am afraid.\nJunio> http://www.kernel.org/ has 80 or so repos (I counted about a\nJunio> month ago so it may probably have more by now) --- which ones?\n\nMy previous message mentioned cogito.git breaking.  This message\nwas triggered because git.git itself broke.\n\nUnfortunately, git.git hasn't been updated since the last time I\nworked around the problem by switching to rsync again, but when it\ndoes, I'll capture the breakage this time.\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\nSee PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"10114","messageId":"Pine.LNX.4.63.0510141543030.23242@iabervon.org","threadId":"2084","inReplyTo":"86achcoyvz.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-10-14T19:56:41Z","receivedAt":"2005-10-14T19:56:41Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Fri, 14 Oct 2005, Randal L. Schwartz wrote:\n\n> My previous message mentioned cogito.git breaking.  This message\n> was triggered because git.git itself broke.\n> \n> Unfortunately, git.git hasn't been updated since the last time I\n> worked around the problem by switching to rsync again, but when it\n> does, I'll capture the breakage this time.\n\nCan you give a general description of what happens? I've noticed that I \nsometimes get spurious error messages that don't actually affect the \ndownload, which I haven't tracked down yet.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"10124","messageId":"863bn3lypn.fsf@blue.stonehenge.com","threadId":"2084","inReplyTo":"Pine.LNX.4.63.0510141543030.23242@iabervon.org","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2005-10-15T13:03:00Z","receivedAt":"2005-10-15T13:03:00Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Daniel\" == Daniel Barkalow <barkalow@iabervon.org> writes:\n\nDaniel> Can you give a general description of what happens? I've\nDaniel> noticed that I sometimes get spurious error messages that\nDaniel> don't actually affect the download, which I haven't tracked\nDaniel> down yet.\n\nOK, it happened this morning.  While syncing to update from\nyesterday's version, I got:\n\n    localhost:~/MIRROR/git-GIT % git-pull\n    Fetching refs/heads/master from http://www.kernel.org/pub/scm/git/git.git using http\n    Getting alternates list\n    got 4546738b58a0134eef154231b07d60fc174d56e3\n    walk 4546738b58a0134eef154231b07d60fc174d56e3\n    got d402d5566fdf226697a386dfb9858e5d954e9b91\n    got 873d8e5652c06c3891278f33546c437efc209c2d\n    walk d402d5566fdf226697a386dfb9858e5d954e9b91\n    error: \n    Getting pack list\n    got 0207ab18a3876249a928e7539d8f594a4f6921f1\n    got 9f7534accdf34b980a2de670cb1009dd84ee56c4\n    error: Unable to find 5ad4a2766d34569f3a1278544ab64978fab14cc8 under http://www.kernel.org/pub/scm/git/git.git/\n\n    Cannot obtain needed blob 5ad4a2766d34569f3a1278544ab64978fab14cc8\n    while processing commit d402d5566fdf226697a386dfb9858e5d954e9b91.\n\nDefinitely broken.  But I can \"rsync\" just fine.\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\nSee PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"10125","messageId":"7vd5m6py11.fsf@assigned-by-dhcp.cox.net","threadId":"2084","inReplyTo":"863bn3lypn.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-15T16:04:10Z","receivedAt":"2005-10-15T16:04:10Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"merlyn@stonehenge.com (Randal L. Schwartz) writes:\n\n> OK, it happened this morning.  While syncing to update from\n> yesterday's version, I got:\n\nThanks.\n\n>     localhost:~/MIRROR/git-GIT % git-pull\n>     Fetching refs/heads/master from http://www.kernel.org/pub/scm/git/git.git using http\n>     Getting alternates list\n>     got 4546738b58a0134eef154231b07d60fc174d56e3\n>     walk 4546738b58a0134eef154231b07d60fc174d56e3\n>     got d402d5566fdf226697a386dfb9858e5d954e9b91\n>     got 873d8e5652c06c3891278f33546c437efc209c2d\n>     walk d402d5566fdf226697a386dfb9858e5d954e9b91\n>     error: \n>     Getting pack list\n>     got 0207ab18a3876249a928e7539d8f594a4f6921f1\n\nHere is the beginning of a session that succeeded:\n\n    : siamese; GIT_DIR=. git-http-fetch -v -a heads/master \\\n               http://www.kernel.org/pub/scm/git/git.git/\n    Getting alternates list\n    got 4546738b58a0134eef154231b07d60fc174d56e3\n    walk 4546738b58a0134eef154231b07d60fc174d56e3\n    got d402d5566fdf226697a386dfb9858e5d954e9b91\n    got 873d8e5652c06c3891278f33546c437efc209c2d\n    got 5ad4a2766d34569f3a1278544ab64978fab14cc8\n    walk d402d5566fdf226697a386dfb9858e5d954e9b91\n    ...\n\nThe difference is that this log gets 5ad4a2 blob, before it\nstarts walking d402d5 commit, while Merlyn's log shows we tried\nto walk that commit before getting the blob.  I think what is\nhappening is:\n\n    - we request 454673 commit, and get it.\n\n    - we start requesting trees, blobs, and parent commit\n      reachable from it.  Especially, 5ad4a2 blob and d402d5\n      commit are asked.\n\n    - as soon as d402d5 commit arrives we walk and find out we\n      need 5ad4a2 blob.  In the case that happened to work, that\n      blob has already arrived because it was also part of the\n      454673 commit, but in Merlyn's case that blob has not\n      arrived yet. \"Getting pack list\" on the next line is an\n      indication that the fetch_object incorrectly decided that\n      the object we are waiting for is not available unpacked,\n      which does not (and should not) happen in the case we got\n      the blob object in time.\n\nI have a suspicion that the recent multi-fetch work has some\ninteresting interaction with the assumption Sergey's fetch.c\noptimization makes.\n"},{"id":"10126","messageId":"7v1x2mpx6m.fsf@assigned-by-dhcp.cox.net","threadId":"2084","inReplyTo":"7vd5m6py11.fsf@assigned-by-dhcp.cox.net","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-15T16:22:25Z","receivedAt":"2005-10-15T16:22:25Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <junkio@cox.net> writes:\n\n> merlyn@stonehenge.com (Randal L. Schwartz) writes:\n>\n>> OK, it happened this morning.  While syncing to update from\n>> yesterday's version, I got:\n>\n> Thanks.\n>\n>>     localhost:~/MIRROR/git-GIT % git-pull\n>>     Fetching refs/heads/master from http://www.kernel.org/pub/scm/git/git.git using http\n>>     Getting alternates list\n>>     got 4546738b58a0134eef154231b07d60fc174d56e3\n>>     walk 4546738b58a0134eef154231b07d60fc174d56e3\n>>     got d402d5566fdf226697a386dfb9858e5d954e9b91\n>>     got 873d8e5652c06c3891278f33546c437efc209c2d\n>>     walk d402d5566fdf226697a386dfb9858e5d954e9b91\n>>     error: \n>>     Getting pack list\n>>     got 0207ab18a3876249a928e7539d8f594a4f6921f1\n\nMore interesting is this \"error:\" without error message.\n\"Getting pack list\" is a signal that we fell back to\nfetch_pack(), so this is coming from fetch_object().\n\nI see this line could emit an empty error message, if errorstr\nis empty.\n\n\tif (request->curl_result != CURLE_OK && request->http_code != 416) {\n\t\tret = error(\"%s\", request->errorstr);\n\t\trelease_request(request);\n\t\treturn ret;\n\t}\n\nSo if that is the case maybe my previous speculation that we\nsometimes forget to issue a necessary request was wrong.  We\nasked for that object and got an error from cURL library...\n\nBTW, I do not think this is related to git.git repository\nproblem, but I wonder why we do not do fetch_object() against\neach altbase in http-fetch.c::fetch(); nobody said you cannot\nborrow unpacked object from your neighbour.\n"},{"id":"10127","messageId":"7vwtkeohws.fsf@assigned-by-dhcp.cox.net","threadId":"2084","inReplyTo":"863bn3lypn.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-15T16:37:39Z","receivedAt":"2005-10-15T16:37:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"merlyn@stonehenge.com (Randal L. Schwartz) writes:\n\n>>>>>> \"Daniel\" == Daniel Barkalow <barkalow@iabervon.org> writes:\n>\n> Daniel> Can you give a general description of what happens? I've\n> Daniel> noticed that I sometimes get spurious error messages that\n> Daniel> don't actually affect the download, which I haven't tracked\n> Daniel> down yet.\n>\n> OK, it happened this morning.  While syncing to update from\n> yesterday's version, I got:\n>\n>     localhost:~/MIRROR/git-GIT % git-pull\n>     Fetching refs/heads/master from http://www.kernel.org/pub/scm/git/git.git using http\n>     Getting alternates list\n>     got 4546738b58a0134eef154231b07d60fc174d56e3\n>     walk 4546738b58a0134eef154231b07d60fc174d56e3\n>     got d402d5566fdf226697a386dfb9858e5d954e9b91\n>     got 873d8e5652c06c3891278f33546c437efc209c2d\n>     walk d402d5566fdf226697a386dfb9858e5d954e9b91\n>     error: \n>     Getting pack list\n>     got 0207ab18a3876249a928e7539d8f594a4f6921f1\n>     got 9f7534accdf34b980a2de670cb1009dd84ee56c4\n>     error: Unable to find 5ad4a2766d34569f3a1278544ab64978fab14cc8 under http://www.kernel.org/pub/scm/git/git.git/\n>\n>     Cannot obtain needed blob 5ad4a2766d34569f3a1278544ab64978fab14cc8\n>     while processing commit d402d5566fdf226697a386dfb9858e5d954e9b91.\n>\n> Definitely broken.  But I can \"rsync\" just fine.\n>\n> -- \n> Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n> <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\n> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.\n> See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"10128","messageId":"7v1x2mog9n.fsf@assigned-by-dhcp.cox.net","threadId":"2084","inReplyTo":"863bn3lypn.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-15T17:13:08Z","receivedAt":"2005-10-15T17:13:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"merlyn@stonehenge.com (Randal L. Schwartz) writes:\n\n>>>>>> \"Daniel\" == Daniel Barkalow <barkalow@iabervon.org> writes:\n>\n> Daniel> Can you give a general description of what happens? I've\n> Daniel> noticed that I sometimes get spurious error messages that\n> Daniel> don't actually affect the download, which I haven't tracked\n> Daniel> down yet.\n>\n> OK, it happened this morning.  While syncing to update from\n> yesterday's version, I got:\n>\n>     localhost:~/MIRROR/git-GIT % git-pull\n>     Fetching refs/heads/master from http://www.kernel.org/pub/scm/git/git.git using http\n>     Getting alternates list\n>     got 4546738b58a0134eef154231b07d60fc174d56e3\n>     walk 4546738b58a0134eef154231b07d60fc174d56e3\n>     got d402d5566fdf226697a386dfb9858e5d954e9b91\n>     got 873d8e5652c06c3891278f33546c437efc209c2d\n>     walk d402d5566fdf226697a386dfb9858e5d954e9b91\n>     error: \n>     Getting pack list\n\nSorry for an empty message.\n\nCould you do the following to establish a reliable way to\nreproduce this problem in your setup, so we can diagnose it\nbetter?\n\nFirst please apply this patch to http-fetch.c and rebuild your\ngit.\n\ndiff --git a/http-fetch.c b/http-fetch.c\nindex 784aedf..2c1f72c 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -1009,7 +1009,10 @@ static int fetch_object(struct alt_base \n \t}\n \n \tif (request->curl_result != CURLE_OK && request->http_code != 416) {\n-\t\tret = error(\"%s\", request->errorstr);\n+\t\tret = error(\"%s (curl_result = %d, http_code = %ld, sha1 = %s)\",\n+\t\t\t    request->errorstr, request->curl_result,\n+\t\t\t    request->http_code,\n+\t\t\t    sha1_to_hex(request->sha1));\n \t\trelease_request(request);\n \t\treturn ret;\n \t}\n\nThen, using that:\n\n$ cd ~/MIRROR\n$ rm -fr git-BROKEN\n$ git-clone -n http://www.kernel.org/pub/scm/git/git.git git-BROKEN\n\nIf you got any error here, then we know we already have a way\nto reproduce.  Please give us a log.\n\nIf the above succeeds, then please run\n\n$ git-fsck-objects --full\n\nto make sure we got things OK.  This should not report\nanything.\n\nNow, I'd like to make this repository as close as I can make it\nto \"after yesterday's pull\", by rewining its top commit back to \nthis one:\n\n    commit f7aac2eac96b637755c89c2405a41749c8f6033a\n    Author: Linus Torvalds <torvalds@osdl.org>\n    Date:   Thu Oct 13 15:38:29 2005 -0700\n    Add \"-l\" flag for repacking only local packs\n\n$ git-reset --hard f7aac2eac96b637755c89c2405a41749c8f6033a\n$ git-update-ref refs/heads/origin f7aac2eac96b637755c89c2405a41749c8f6033a\n\nSince proposed updates (pu) branch usually contains everything\nthat is in the master, we need to get rid of it for this\nexperiment:\n\n$ rm -f .git/refs/heads/pu\n\nand the we get rid of objects that were added since then:\n\n$ git-prune\n$ git-fsck-objects --full\n\nNow we are hopefully as close as \"after yesterday's pull\" state.\n\n$ git-http-fetch -a -v heads/master http://www.kernel.org/pub/scm/git/git.git/\n\nThis will start with something like:\n\n        Getting alternates list\n        got 4546738b58a0134eef154231b07d60fc174d56e3\n        walk 4546738b58a0134eef154231b07d60fc174d56e3\n        got d402d5566fdf226697a386dfb9858e5d954e9b91\n        got 873d8e5652c06c3891278f33546c437efc209c2d\n        walk d402d5566fdf226697a386dfb9858e5d954e9b91\n        ...\n\nif you see the same error, instead of the empty \"error:\" up\nabove, we may see a bit more information.\n"},{"id":"10133","messageId":"Pine.LNX.4.63.0510151502290.23242@iabervon.org","threadId":"2084","inReplyTo":"7v1x2mpx6m.fsf@assigned-by-dhcp.cox.net","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-10-15T19:41:35Z","receivedAt":"2005-10-15T19:41:35Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Sat, 15 Oct 2005, Junio C Hamano wrote:\n\n> Junio C Hamano <junkio@cox.net> writes:\n> \n> > merlyn@stonehenge.com (Randal L. Schwartz) writes:\n> >\n> >> OK, it happened this morning.  While syncing to update from\n> >> yesterday's version, I got:\n> >\n> > Thanks.\n> >\n> >>     localhost:~/MIRROR/git-GIT % git-pull\n> >>     Fetching refs/heads/master from http://www.kernel.org/pub/scm/git/git.git using http\n> >>     Getting alternates list\n> >>     got 4546738b58a0134eef154231b07d60fc174d56e3\n> >>     walk 4546738b58a0134eef154231b07d60fc174d56e3\n> >>     got d402d5566fdf226697a386dfb9858e5d954e9b91\n> >>     got 873d8e5652c06c3891278f33546c437efc209c2d\n> >>     walk d402d5566fdf226697a386dfb9858e5d954e9b91\n> >>     error: \n> >>     Getting pack list\n> >>     got 0207ab18a3876249a928e7539d8f594a4f6921f1\n> \n> More interesting is this \"error:\" without error message.\n> \"Getting pack list\" is a signal that we fell back to\n> fetch_pack(), so this is coming from fetch_object().\n> \n> I see this line could emit an empty error message, if errorstr\n> is empty.\n> \n> \tif (request->curl_result != CURLE_OK && request->http_code != 416) {\n> \t\tret = error(\"%s\", request->errorstr);\n> \t\trelease_request(request);\n> \t\treturn ret;\n> \t}\n> \n> So if that is the case maybe my previous speculation that we\n> sometimes forget to issue a necessary request was wrong.  We\n> asked for that object and got an error from cURL library...\n\nIt looks like we didn't get an error from the cURL library, actually, or \nit would have printed something. My guess is that it is getting to the \npoint about while the request is still in progress, but I'm not seeing how \nthat could happen.\n\n> BTW, I do not think this is related to git.git repository\n> problem, but I wonder why we do not do fetch_object() against\n> each altbase in http-fetch.c::fetch(); nobody said you cannot\n> borrow unpacked object from your neighbour.\n\nI believe the code is doing that, but elsewhere.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"10138","messageId":"20051015212154.GB5509@reactrix.com","threadId":"2084","inReplyTo":"7v1x2mpx6m.fsf@assigned-by-dhcp.cox.net","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Nick Hengeveld","fromEmail":"nickh@reactrix.com","sentAt":"2005-10-15T21:21:54Z","receivedAt":"2005-10-15T21:21:54Z","isPatch":false,"sender":{"key":"nickh@reactrix.com","avatar":null},"body":"On Sat, Oct 15, 2005 at 09:22:25AM -0700, Junio C Hamano wrote:\n\n> More interesting is this \"error:\" without error message.\n> \"Getting pack list\" is a signal that we fell back to\n> fetch_pack(), so this is coming from fetch_object().\n\nI've seen that happen before fetch_pack used the active queue; open object\nrequests in the active queue would not be processed while fetch_pack was\ntransferring the pack, and in some cases a server had timed out a connection\nfrom one of these requests by the time the active queue started processing \nagain.  I worked around this at one point by detecting an empty server\nresponse and retrying the request.\n\nChanging all requests to run through the active queue seemed to fix the\nproblem, but it's possible that something is still holding up processing\nlong enough to cause a server timeout.\n\n> BTW, I do not think this is related to git.git repository\n> problem, but I wonder why we do not do fetch_object() against\n> each altbase in http-fetch.c::fetch(); nobody said you cannot\n> borrow unpacked object from your neighbour.\n\nIt doesn't look like there are any alternates defined in the git.git\nrepository.  I've seen alternates used during testing when I deliberately\nremoved objects and packs from the primary repository.\n\n-- \nFor a successful technology, reality must take precedence over public\nrelations, for nature cannot be fooled.\n"},{"id":"10140","messageId":"20051015215648.GC5509@reactrix.com","threadId":"2084","inReplyTo":"Pine.LNX.4.63.0510151502290.23242@iabervon.org","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Nick Hengeveld","fromEmail":"nickh@reactrix.com","sentAt":"2005-10-15T21:56:48Z","receivedAt":"2005-10-15T21:56:48Z","isPatch":false,"sender":{"key":"nickh@reactrix.com","avatar":null},"body":"On Sat, Oct 15, 2005 at 03:41:35PM -0400, Daniel Barkalow wrote:\n\n> > BTW, I do not think this is related to git.git repository\n> > problem, but I wonder why we do not do fetch_object() against\n> > each altbase in http-fetch.c::fetch(); nobody said you cannot\n> > borrow unpacked object from your neighbour.\n> \n> I believe the code is doing that, but elsewhere.\n\nEr, never mind what I said in my previous message.  Daniel is correct,\nprocess_curl_messages() will try the next altbase if a request 404s.\n\n-- \nFor a successful technology, reality must take precedence over public\nrelations, for nature cannot be fooled.\n"},{"id":"10139","messageId":"20051015215754.GD5509@reactrix.com","threadId":"2084","inReplyTo":"863bn3lypn.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Nick Hengeveld","fromEmail":"nickh@reactrix.com","sentAt":"2005-10-15T21:57:54Z","receivedAt":"2005-10-15T21:57:54Z","isPatch":false,"sender":{"key":"nickh@reactrix.com","avatar":null},"body":"On Sat, Oct 15, 2005 at 06:03:00AM -0700, Randal L. Schwartz wrote:\n\n> OK, it happened this morning.  While syncing to update from\n> yesterday's version, I got:\n> ...\n> Definitely broken.  But I can \"rsync\" just fine.\n\nWhich version of curl did you build with?\n\n-- \nFor a successful technology, reality must take precedence over public\nrelations, for nature cannot be fooled.\n"},{"id":"10141","messageId":"86irvyigig.fsf@blue.stonehenge.com","threadId":"2084","inReplyTo":"20051015215754.GD5509@reactrix.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Randal L. Schwartz","fromEmail":"merlyn@stonehenge.com","sentAt":"2005-10-15T22:04:23Z","receivedAt":"2005-10-15T22:04:23Z","isPatch":false,"sender":{"key":"merlyn@stonehenge.com","avatar":"https://gravatar.com/avatar/dc528d210743ff0333e6213f9ee7b33b23f1b7bc1f3c5a8c2d819074ecd7ab19?d=mp&s=160"},"body":">>>>> \"Nick\" == Nick Hengeveld <nickh@reactrix.com> writes:\n\nNick> On Sat, Oct 15, 2005 at 06:03:00AM -0700, Randal L. Schwartz wrote:\n>> OK, it happened this morning.  While syncing to update from\n>> yesterday's version, I got:\n>> ...\n>> Definitely broken.  But I can \"rsync\" just fine.\n\nNick> Which version of curl did you build with?\n\ncurl 7.13.1 (powerpc-apple-darwin8.0) libcurl/7.13.1 OpenSSL/0.9.7g zlib/1.2.3\n\n-- \nRandal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095\n<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>\nPerl/Unix/security consulting, Technical writing, Comedy, etc. etc.\nSee PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!\n"},{"id":"10364","messageId":"20051020174324.GD6160@reactrix.com","threadId":"2084","inReplyTo":"867jciz18w.fsf@blue.stonehenge.com","subject":"Re: maybe breakage with latest git-pull and http protocol","fromName":"Nick Hengeveld","fromEmail":"nickh@reactrix.com","sentAt":"2005-10-20T17:43:24Z","receivedAt":"2005-10-20T17:43:24Z","isPatch":false,"sender":{"key":"nickh@reactrix.com","avatar":null},"body":"Can you try using the following patch?  This cleans up curl handles for\nactive request slots that are no longer in use.  If all other prefetched\nrequests finish while a large transfer is happening, the active slots\ncurrently keep open connections to the server and I'm guessing that\nperhaps by the time additional objects are prefetched the server has\ntimed out some of those keepalive connections.\n\n\n---\n\n http-fetch.c |   26 ++++++++++++++++++++------\n 1 files changed, 20 insertions(+), 6 deletions(-)\n\napplies-to: ce9a5a0fdd52a29e370d849a132b4509c844aca1\n04b4353279eaceb7e7c3d73a9565b219aa7a10a9\ndiff --git a/http-fetch.c b/http-fetch.c\nindex a7dc2cc..d26fae8 100644\n--- a/http-fetch.c\n+++ b/http-fetch.c\n@@ -291,11 +291,7 @@ static struct active_request_slot *get_a\n \t}\n \tif (slot == NULL) {\n \t\tnewslot = xmalloc(sizeof(*newslot));\n-#ifdef NO_CURL_EASY_DUPHANDLE\n-\t\tnewslot->curl = get_curl_handle();\n-#else\n-\t\tnewslot->curl = curl_easy_duphandle(curl_default);\n-#endif\n+\t\tnewslot->curl = NULL;\n \t\tnewslot->in_use = 0;\n \t\tnewslot->next = NULL;\n \n@@ -311,6 +307,14 @@ static struct active_request_slot *get_a\n \t\tslot = newslot;\n \t}\n \n+\tif (slot->curl == NULL) {\n+#ifdef NO_CURL_EASY_DUPHANDLE\n+\t\tslot->curl = get_curl_handle();\n+#else\n+\t\tslot->curl = curl_easy_duphandle(curl_default);\n+#endif\n+\t}\n+\n \tactive_requests++;\n \tslot->in_use = 1;\n \tslot->done = 0;\n@@ -612,6 +616,7 @@ void process_curl_messages(void)\n void process_request_queue(void)\n {\n \tstruct transfer_request *request = request_queue_head;\n+\tstruct active_request_slot *slot = active_queue_head;\n \tint num_transfers;\n \n \twhile (active_requests < max_requests && request != NULL) {\n@@ -624,6 +629,14 @@ void process_request_queue(void)\n \t\t}\n \t\trequest = request->next;\n \t}\n+\n+\twhile (slot != NULL) {\n+\t\tif (!slot->in_use && slot->curl != NULL) {\n+\t\t\tcurl_easy_cleanup(slot->curl);\n+\t\t\tslot->curl = NULL;\n+\t\t}\n+\t\tslot = slot->next;\n+\t}\t\t\t\t\n }\n #endif\n \n@@ -1297,7 +1310,8 @@ int main(int argc, char **argv)\n #endif\n \tslot = active_queue_head;\n \twhile (slot != NULL) {\n-\t\tcurl_easy_cleanup(slot->curl);\n+\t\tif (slot->curl != NULL)\n+\t\t\tcurl_easy_cleanup(slot->curl);\n \t\tslot = slot->next;\n \t}\n #ifdef USE_CURL_MULTI\n---\n0.99.8.GIT\n\n-- \nFor a successful technology, reality must take precedence over public\nrelations, for nature cannot be fooled.\n"}]}