git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git fetch over http:// left my repo broken

From
Christian Halstrick <christian.halstrick@gmail.com>
Date
Apr 15, 2010, 09:51 UTC
Message-ID
<g2y8c627c4f1004150251l3dc2ad17n352b149ac739d309@mail.gmail.com>
Hi,

some days back I fetched from a github repo with http protocol and afterwards my local repo was broken. Since the fetch was done by a cronjob I don't know whether the fetch reported an error. Problem is that one pack file was corrupted because the github servers put the repo I wanted to clone into some maintenance mode while I was fetching. The pack file includes at the end the html source code - which makes these files clearly corrupted.

Git should detect this error and let the fetch fail, right?

All this done on Linux (CentOS) with git version 1.6.6.1. Github repo was http://github.com/sonatype/sonatype-tycho.git. This is not easy to reproduce - because you have to hit the maintenance times of github.

Here is what I did:
> git --version
git version 1.6.6.1
> uname --all

Linux wdfd00220954a.wdf.sap.corp 2.6.18-164.15.1.el5 #1 SMP Wed Mar 17 11:30:06 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

> git remote -v

origin  http://github.com/sonatype/sonatype-tycho.git (fetch) origin  http://github.com/sonatype/sonatype-tycho.git (push)

> git fetch origin
...
> git fsck --full

error: packfile ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack does not match index error: packfile ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack cannot be accessed error: packfile ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack does not match index fatal: packfile ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack cannot be accessed

# get the size of the corrupted pack file
> ls -l ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack

-rw-r--r-- 1 git git 766920900 Apr 13 16:22 ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack

# dump the start of the corrupted file: looks ok
> xxd -l 64 ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack

0000000: 5041 434b 0000 0002 0000 4bef 962b 789c  PACK......K..+x. 0000010: 9591 cb72 dc20 1045 f77c 457f 8067 8a41  ...r. .E.|E..g.A 0000020: 1a21 a552 a954 5cce 63e5 2adb 9b2c 7934  .!.R.T\.c.*..,y4 0000030: 2332 8856 00d9 99bf 37a3 781c 2f9d 0d05  #2.V....7.x./...

# dump a section near the end of the corrupted file: html source code included here?

>  xxd -s 766911990 -l 128 ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack

2db625f6:d64b 2752 e70f 8f1a 6161 3c21 444f 4354  .K'R....aa<!DOCT 2db62606:5950 4520 6874 6d6c 2050 5542 4c49 4320  YPE html PUBLIC 2db62616:222d 2f2f 5733 432f 2f44 5444 2058 4854  "-//W3C//DTD XHT 2db62626:4d4c 2031 2e30 2054 7261 6e73 6974 696f  ML 1.0 Transitio 2db62636:6e61 6c2f 2f45 4e22 0a20 2022 6874 7470  nal//EN".  "http 2db62646:3a2f 2f77 7777 2e77 332e 6f72 672f 5452  ://www.w3.org/TR 2db62656:2f78 6874 6d6c 312f 4454 442f 7868 746d  /xhtml1/DTD/xhtm 2db62666:6c31 2d74 7261 6e73 6974 696f 6e61 6c2e  l1-transitional.

# search for the string "maintenance" in the binary pack file
> grep -C5 -a "maintenance" ./objects/pack/pack-b5dceb0bae390d6540f881be686839f7e691fa0b.pack
<div id="error" class="status404">
  <img alt="Repo unavailable due to maintenanace" height="219"
src="http://assets3.github.com/images/error/octocat_construction.gif?5cb2cfa6f35ac1b0c322a78a129d7531177b36d7"
width="243" />
  <h1>Repository temporarily unavailable.</h1>
  <p>The backend storage is temporarily offline. Usually this means the<br />
     storage server is undergoing maintenance. Your repository should <br />
     be available again very soon.</p>
</div>
        </div>
>
Ciao
 Chris
Next: Michael J Gruber
Message 1 of 46 in “git fetch over http:// left my repo broken”
  1. Christian HalstrickApr 15, 2010
  2. Michael J GruberApr 15, 2010
  3. Ilari LiusvaaraApr 15, 2010
  4. Shawn O. PearceApr 15, 2010
  5. 0/6 detect dumb HTTP pack file corruptionShawn O. Pearce, Apr 15, 2010
  6. Junio C HamanoApr 17, 2010
  7. Shawn O. PearceApr 17, 2010
  8. 1/6 http.c: Remove bad free of static blockShawn O. Pearce, Apr 15, 2010
  9. 2/6 t5550-http-fetch: Use subshell for repository operationsShawn O. Pearce, Apr 15, 2010
  10. 3/6 http.c: Tiny refactoring of finish_http_pack_requestShawn O. Pearce, Apr 15, 2010
  11. 4/6 http.c: Drop useless != NULL test in finish_http_pack_requestShawn O. Pearce, Apr 15, 2010
  12. 5/6 http-fetch: Use index-pack rather than verify-pack to check packsShawn O. Pearce, Apr 15, 2010
  13. Johannes SixtApr 15, 2010
  14. 5/6 http-fetch: Use index-pack rather than verify-pack to check packsShawn O. Pearce, Apr 15, 2010
  15. Tay Ray ChuanApr 16, 2010
  16. Shawn O. PearceApr 17, 2010
  17. 6/6 http-fetch: Use temporary files for pack-*.idx until verifiedShawn O. Pearce, Apr 15, 2010
  18. Tay Ray ChuanApr 16, 2010
  19. 01/11 http.c: Remove bad free of static blockShawn O. Pearce, Apr 17, 2010
  20. 02/11 t5550-http-fetch: Use subshell for repository operationsShawn O. Pearce, Apr 17, 2010
  21. 03/11 http.c: Tiny refactoring of finish_http_pack_requestShawn O. Pearce, Apr 17, 2010
  22. 04/11 http.c: Drop useless != NULL test in finish_http_pack_requestShawn O. Pearce, Apr 17, 2010
  23. 05/11 http.c: Don't store destination name in request structuresShawn O. Pearce, Apr 17, 2010
  24. Tay Ray ChuanApr 18, 2010
  25. 06/11 http.c: Remove unnecessary strdup of sha1_to_hex resultShawn O. Pearce, Apr 17, 2010
  26. Tay Ray ChuanApr 18, 2010
  27. 07/11 Introduce close_pack_index to permit replacementShawn O. Pearce, Apr 17, 2010
  28. 08/11 Extract verify_pack_index for reuse from verify_packShawn O. Pearce, Apr 17, 2010
  29. 09/11 Allow parse_pack_index on temporary filesShawn O. Pearce, Apr 17, 2010
  30. 10/11 http-fetch: Use index-pack rather than verify-pack to check packsShawn O. Pearce, Apr 17, 2010
  31. Tay Ray ChuanApr 18, 2010
  32. 11/11 http-fetch: Use temporary files for pack-*.idx until verifiedShawn O. Pearce, Apr 17, 2010
  33. Tay Ray ChuanApr 18, 2010
  34. 00/11 Resend sp/maint-dumb-http-pack-reidxShawn O. Pearce, Apr 19, 2010
  35. Tay Ray ChuanApr 19, 2010
  36. Shawn O. PearceApr 19, 2010
  37. Tay Ray ChuanApr 20, 2010
  38. 06/11 http.c: Remove unnecessary strdup of sha1_to_hex resultShawn O. Pearce, Apr 19, 2010
  39. 07/11 Introduce close_pack_index to permit replacementShawn O. Pearce, Apr 19, 2010
  40. 08/11 Extract verify_pack_index for reuse from verify_packShawn O. Pearce, Apr 19, 2010
  41. 09/11 Allow parse_pack_index on temporary filesShawn O. Pearce, Apr 19, 2010
  42. 10/11 http-fetch: Use index-pack rather than verify-pack to check packsShawn O. Pearce, Apr 19, 2010
  43. Tay Ray ChuanApr 19, 2010
  44. 11/11 http-fetch: Use temporary files for pack-*.idx until verifiedShawn O. Pearce, Apr 19, 2010
  45. 6/6 http-fetch: Use temporary files for pack-*.idx until verifiedShawn O. Pearce, Apr 15, 2010
  46. Ilari LiusvaaraApr 15, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.