{"thread":{"id":"28358","subject":"How do I investigate apparently random git clone reports of \"error: File ... has bad hash\"?","startedAt":"2011-09-11T19:03:03Z","lastAt":"2011-10-06T02:34:33Z","messageCount":4,"participants":["Thorkil Naur","Andreas Schwab"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"175300","messageId":"loom.20110911T210035-693@post.gmane.org","threadId":"28358","inReplyTo":null,"subject":"How do I investigate apparently random git clone reports of \"error: File ... has bad hash\"?","fromName":"Thorkil Naur","fromEmail":"naur@post11.tele.dk","sentAt":"2011-09-11T19:03:03Z","receivedAt":"2011-09-11T19:03:03Z","isPatch":false,"sender":{"key":"naur@post11.tele.dk","avatar":null},"body":"Hello,\n\nOn a Mac OS X 10.5.8 with\n\n> $ uname -a\n> Darwin thorkil-naurs-intel-mac-mini.local 9.8.0 Darwin Kernel Version 9.8.0:\nWed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386\n> $ git --version\n> git version 1.7.4.1\n> $ \n\nthat I use as a so-called buildbot slave (http://trac.buildbot.net/) for GHC\n(http://www.haskell.org/ghc/), git clone is used repeatedly to fetch the main\nrepository and several other (library) repositories on which the main repository\ndepends. In recent months, I have experienced apparently random failures in\nwhich git clone reports, for example:\n\n> $ git clone http://darcs.haskell.org/ghc.git/ build8\n> Cloning into build8...\n> error: File 42988feeeb76f5cb92b541e9dac277e073bcb3ef has bad hash\n> error: Unable to find 42988feeeb76f5cb92b541e9dac277e073bcb3ef under\nhttp://darcs.haskell.org/ghc.git\n> Cannot obtain needed blob 42988feeeb76f5cb92b541e9dac277e073bcb3ef\n> while processing commit ffb2e81c03a01e74825b3a0223e214df59241fab.\n> error: Fetch failed.\n\nThere are variations of this, such as\n\n> $ git clone http://darcs.haskell.org/ghc.git/ build6\n> Cloning into build6...\n> error: inflate: data stream error (incorrect header check)\n> error: inflate: data stream error (incorrect header check)\n> error: inflate: data stream error (incorrect header check)\n> error: inflate: data stream error (incorrect header check)\n> error: inflate: data stream error (incorrect header check)\n> error: File 280db842d43e7e68c09bf01695434fc0f8fff796\n(http://darcs.haskell.org/ghc.git/objects/c8/ba6e77a6202e7df588c3b1fa3ada5142482036)\ncorrupt\n> error: Unable to find c8ba6e77a6202e7df588c3b1fa3ada5142482036 under\nhttp://darcs.haskell.org/ghc.git\n> Cannot obtain needed blob c8ba6e77a6202e7df588c3b1fa3ada5142482036\n> while processing commit ffb2e81c03a01e74825b3a0223e214df59241fab.\n> error: Fetch failed.\n\nIn most cases, the git clone succeeds, but failure nevertheless happens\nsufficiently often to break most builds. See, for example\nhttp://darcs.haskell.org/ghcBuilder/builders/tn23/437/3.html.\n\nI tried the clone of the main repository on my Ubuntu Linux machine that says\n\n> $ uname -a\n> Linux tn24 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:08:37 UTC 2011 i686\nGNU/Linux\n> $ git --version\n> git version 1.7.0.4\n> $ \n\n(which is connected to the same network as the Mac) and was not able to\nreproduce the error.\n\nI have looked for git options (debug, verbosity) for increasing the amount of\ninformation reported, but have not managed to find anything that seemed useful\nin this situation. The apparent randomness of the failures points to some kind\nof network problem, but I have not noticed other indications that my network\nconnection is unstable.\n\nI am not a git expert at all, I am not using git for anything but GHC building.\nI would very much appreciate some indication of how I might investigate this\nproblem.\n\nBest regards\nThorkil\n"},{"id":"175310","messageId":"m2r53metpo.fsf@igel.home","threadId":"28358","inReplyTo":"loom.20110911T210035-693@post.gmane.org","subject":"Re: How do I investigate apparently random git clone reports of \"error: File ... has bad hash\"?","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2011-09-11T19:59:15Z","receivedAt":"2011-09-11T19:59:15Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Thorkil Naur writes:\n\n>> $ git clone http://darcs.haskell.org/ghc.git/ build8\n>> Cloning into build8...\n>> error: File 42988feeeb76f5cb92b541e9dac277e073bcb3ef has bad hash\n>> error: Unable to find 42988feeeb76f5cb92b541e9dac277e073bcb3ef under\n> http://darcs.haskell.org/ghc.git\n>> Cannot obtain needed blob 42988feeeb76f5cb92b541e9dac277e073bcb3ef\n>> while processing commit ffb2e81c03a01e74825b3a0223e214df59241fab.\n>> error: Fetch failed.\n\nI just tried to clone it and got this error:\n\n$ git clone http://darcs.haskell.org/ghc.git\nCloning into ghc...\nerror: Recv failure: Connection reset by peer (curl_result = 56, http_code = 0, sha1 = be6810bb027643bf0697b3d237426110f064aba1)\nerror: Unable to find be6810bb027643bf0697b3d237426110f064aba1 under http://darcs.haskell.org/ghc.git\nCannot obtain needed commit be6810bb027643bf0697b3d237426110f064aba1\nwhile processing commit 6942b112082fbcdff5c66f06f56fdd336861da47.\nerror: Fetch failed.\n\nIt looks like this is just a network problem.\n\nBtw, the repo is rather strange. It's not a bare repo, but does not\ncontain a .git directory. Instead the files that are normally under\n.git are placed directly in the working tree.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"175311","messageId":"m2mxeaet2h.fsf@igel.home","threadId":"28358","inReplyTo":"loom.20110911T210035-693@post.gmane.org","subject":"Re: How do I investigate apparently random git clone reports of \"error: File ... has bad hash\"?","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2011-09-11T20:13:10Z","receivedAt":"2011-09-11T20:13:10Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Thorkil Naur writes:\n\n> I have looked for git options (debug, verbosity) for increasing the amount of\n> information reported, but have not managed to find anything that seemed useful\n> in this situation.\n\nTry adding -v -v when cloning. That also shows you that the server does\nnot support smart-http, which is suboptimal.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"176995","messageId":"20111006023425.GA2554@tn24","threadId":"28358","inReplyTo":"m2r53metpo.fsf@igel.home","subject":"Re: How do I investigate apparently random git clone reports of \"error: File ... has bad hash\"?","fromName":"Thorkil Naur","fromEmail":"naur@post11.tele.dk","sentAt":"2011-10-06T02:34:33Z","receivedAt":"2011-10-06T02:34:33Z","isPatch":false,"sender":{"key":"naur@post11.tele.dk","avatar":null},"body":"Hello Andreas,\n\nThank you very much for your response. I have separately received the\nadvice of upgrading my curl installation:\n\n> Date: Fri, 30 Sep 2011 09:08:56 +0200\n> From: Karel Gardas \n> To: cvs-ghc \n> Subject: Re: tn23 (x86 OSX HEAD), build 442, Failure\n> ...\n> Hello,\n>\n> my opensolaris builder machine also suffered from the same issue like\n> tn23 and sometimes even mbolingbroke and others. Symptoms are you are\n> not able to grab the ghc code or subrepos code. The solution is\n> simple: (1) either remove curl from your path or (2) update curl to\n> the latest version (7.21.7 works for me) and make sure it is really\n> using its latest libcurl. Once I did (2) here I've never seen the\n> issue again.\n> ...\n\nAdditional details:\n\n> http://www.haskell.org/pipermail/cvs-ghc/2011-October/066434.html\n\nSo it appears that upgrading curl has removed the problem.\n\nBest regards\nThorkil\n\nOn Sun, Sep 11, 2011 at 09:59:15PM +0200, Andreas Schwab wrote:\n> Thorkil Naur writes:\n> \n> >> $ git clone http://darcs.haskell.org/ghc.git/ build8\n> >> Cloning into build8...\n> >> error: File 42988feeeb76f5cb92b541e9dac277e073bcb3ef has bad hash\n> >> error: Unable to find 42988feeeb76f5cb92b541e9dac277e073bcb3ef under\n> > http://darcs.haskell.org/ghc.git\n> >> Cannot obtain needed blob 42988feeeb76f5cb92b541e9dac277e073bcb3ef\n> >> while processing commit ffb2e81c03a01e74825b3a0223e214df59241fab.\n> >> error: Fetch failed.\n> \n> I just tried to clone it and got this error:\n> \n> $ git clone http://darcs.haskell.org/ghc.git\n> Cloning into ghc...\n> error: Recv failure: Connection reset by peer (curl_result = 56, http_code = 0, sha1 = be6810bb027643bf0697b3d237426110f064aba1)\n> error: Unable to find be6810bb027643bf0697b3d237426110f064aba1 under http://darcs.haskell.org/ghc.git\n> Cannot obtain needed commit be6810bb027643bf0697b3d237426110f064aba1\n> while processing commit 6942b112082fbcdff5c66f06f56fdd336861da47.\n> error: Fetch failed.\n> \n> It looks like this is just a network problem.\n> \n> Btw, the repo is rather strange. It's not a bare repo, but does not\n> contain a .git directory. Instead the files that are normally under\n> .git are placed directly in the working tree.\n> \n> Andreas.\n> \n> -- \n> Andreas Schwab, schwab@linux-m68k.org\n> GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n> \"And now for something completely different.\"\n"}]}