{"thread":{"id":"24860","subject":"Remote corruption issue, linked to thin pack code?","startedAt":"2010-08-25T20:53:26Z","lastAt":"2010-08-26T10:13:04Z","messageCount":4,"participants":["Thomas Rast","Nicolas Pitre","Jonas Thiem"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"148995","messageId":"201008252253.26521.trast@student.ethz.ch","threadId":"24860","inReplyTo":null,"subject":"Remote corruption issue, linked to thin pack code?","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2010-08-25T20:53:26Z","receivedAt":"2010-08-25T20:53:26Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Hi *,\n\nThis one sounds fairly similar to\n\n  http://thread.gmane.org/gmane.comp.version-control.git/123244/\n\nwhich was never resolved.\n\nJonas reported repo corruption on IRC along the lines of\n\n  $ make gitpush\n  cd dist && git push ssh://<user>@eloxoph.com/repos/mainrepo/bla.git master\n   \n    -  W e l c o m e   t o   E L O X O P H  -\n               friendly landlord\n\n  <user>@eloxoph.com's password: \n  Counting objects: 201, done.\n  Delta compression using up to 2 threads.\n  Compressing objects: 100% (132/132), done.\n  Writing objects: 100% (133/133), 1.01 MiB, done.\n  Total 133 (delta 118), reused 0 (delta 0)\n  fatal: pack has 114 unresolved deltas\n  error: unpack failed: index-pack abnormal exit\n  To ssh://<user>@eloxoph.com/repos/mainrepo/bla.git\n   ! [remote rejected] master -> master (n/a (unpacker error))\n  error: failed to push some refs to 'ssh://<user>@eloxoph.com/repos/mainrepo/bla.git'\n  make: *** [gitpush] Error 1\n\nThe respective repos show no errors with git-fsck.  Jonas kindly\nprovides a download link for both:\n\n  http://eloxoph.com/localrepo.zip\n  http://eloxoph.com/remoterepo.zip\n\nVersions are 1.7.2.1 locally and 1.5.4.3 remotely.\n\nBeware, there are hooks in both.  Disabling them, however, only\nchanges the error slightly:\n\n  [...]\n  Total 133 (delta 118), reused 0 (delta 0)\n  fatal: pack has 114 unresolved deltas\n  error: unpack failed: index-pack abnormal exit\n  To ssh://mainrepo@eloxoph.com/repos/mainrepo/bla.git\n   ! [remote rejected] master -> master (n/a (unpacker error))\n  error: failed to push some refs to 'ssh://mainrepo@eloxoph.com/repos/mainrepo/bla.git'\n  make: *** [gitpush] Error 1\n\nWhat's even stranger is that fetching from the repo is also not\npossible:\n\n  fatal: git-upload-pack: cannot find object e28ae6b61c384732c506544626c5083557dd2d75:\n  fatal: The remote end hung up unexpectedly\n\ndespite the object being there.\n\nWhat's also strange is that while there is a temporary pack inside\nobjects/, I get\n\n  $ git index-pack --stdin < objects/tmp_pack_oEUkIc\n  fatal: pack has 114 unresolved deltas\n  $ git index-pack --fix-thin -v --stdin < objects/tmp_pack_oEUkIc\n  Receiving objects: 100% (133/133), 1.01 MiB, done.\n  Resolving deltas: 100% (118/118), completed with 63 local objects.\n  pack    061120577b0a1fec7ba636d6e3162f95f83543aa\n\nSo it seems the remote side got a thin pack and can't cope.  But\na4503a1 (Make --no-thin the default in git-push to save server\nresources, 2007-09-09), merged way back in 1.5.3.2, claims to enable\n--no-thin all the time.  So how did a thin pack get there?\n\nAny ideas?\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"149025","messageId":"alpine.LFD.2.00.1008252107070.622@xanadu.home","threadId":"24860","inReplyTo":"201008252253.26521.trast@student.ethz.ch","subject":"Re: Remote corruption issue, linked to thin pack code?","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-08-26T02:42:51Z","receivedAt":"2010-08-26T02:42:51Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Wed, 25 Aug 2010, Thomas Rast wrote:\n\n> Hi *,\n> \n> This one sounds fairly similar to\n> \n>   http://thread.gmane.org/gmane.comp.version-control.git/123244/\n> \n> which was never resolved.\n\nI don't think it actually is the same issue.\n\n> Jonas reported repo corruption on IRC along the lines of\n> \n>   $ make gitpush\n>   cd dist && git push ssh://<user>@eloxoph.com/repos/mainrepo/bla.git master\n>    \n>     -  W e l c o m e   t o   E L O X O P H  -\n>                friendly landlord\n> \n>   <user>@eloxoph.com's password: \n>   Counting objects: 201, done.\n>   Delta compression using up to 2 threads.\n>   Compressing objects: 100% (132/132), done.\n>   Writing objects: 100% (133/133), 1.01 MiB, done.\n>   Total 133 (delta 118), reused 0 (delta 0)\n>   fatal: pack has 114 unresolved deltas\n\nSo... out of 118 deltas, 114 were relying on the remote to already have \nbase objects for them (or some of them are deltas against unresolved \ndeltas).  In any case, the remote is missing some objects it advertised \nit should have.\n\n> The respective repos show no errors with git-fsck.\n\nWhat about 'git fsck --full'?  Given git v1.5.4 on the remote, you \nshould try --full with 'git fsck' as this wasn't the default back then.\n\n> Jonas kindly\n> provides a download link for both:\n> \n>   http://eloxoph.com/localrepo.zip\n>   http://eloxoph.com/remoterepo.zip\n\n$ wget http://eloxoph.com/remoterepo.zip\n--2010-08-25 21:24:52--  http://eloxoph.com/remoterepo.zip\nResolving eloxoph.com... 85.214.104.74\nConnecting to eloxoph.com|85.214.104.74|:80... connected.\nHTTP request sent, awaiting response... 404 Not Found\n2010-08-25 21:24:52 ERROR 404: Not Found.\n\n> What's even stranger is that fetching from the repo is also not\n> possible:\n> \n>   fatal: git-upload-pack: cannot find object e28ae6b61c384732c506544626c5083557dd2d75:\n>   fatal: The remote end hung up unexpectedly\n> \n> despite the object being there.\n\nHow do you know it is there while git (on the remote) is telling you it \nis not?\n\nWhat about 'git cat-file -t e28ae6b61c384732c506' ?\n\n> What's also strange is that while there is a temporary pack inside\n> objects/, I get\n> \n>   $ git index-pack --stdin < objects/tmp_pack_oEUkIc\n>   fatal: pack has 114 unresolved deltas\n>   $ git index-pack --fix-thin -v --stdin < objects/tmp_pack_oEUkIc\n>   Receiving objects: 100% (133/133), 1.01 MiB, done.\n>   Resolving deltas: 100% (118/118), completed with 63 local objects.\n>   pack    061120577b0a1fec7ba636d6e3162f95f83543aa\n> \n> So it seems the remote side got a thin pack and can't cope. \n\nHmmm... in this case it must be because git v1.5.4 is buggy.\n\n/me looks\n\nWell... apparently no.  In receive-pack.c:unpack() there is this code:\n\n                keeper[0] = \"index-pack\";\n                keeper[1] = \"--stdin\";\n                keeper[2] = \"--fix-thin\";\n                keeper[3] = hdr_arg;\n                keeper[4] = keep_arg;\n                keeper[5] = NULL;\n\nso --fix-thin is always unconditionally passed to index-pack.  This is \nalso more or less the same code in v1.7.2.\n\n> But a4503a1 (Make --no-thin the default in git-push to save server \n> resources, 2007-09-09), merged way back in 1.5.3.2, claims to enable \n> --no-thin all the time.  So how did a thin pack get there?\n\nWell, this was added back by accident a long time ago, and since then I \nalso changed my mind about this, especially since 'git gc --auto' is \nexecuted on the remote after a push these days, so I never pushed a \n\"fix\" for it.\n\nRegardless, the receiving end of a push shouldn't care either ways and \nalways complete a thin pack.\n\nSo there are 2 mysteries here:\n\n1) Why can't the remote repo be fetched? The error from git-upload-pack \n   is straight forward and has nothing to do with any thin packs.\n\n2) Why adding --fix-thin to a manual invocation of index-pack does work \n   while the invocation from receive_pack which should have it doesn't?\n\nAlso, what is the OS version and filesystem used on the remote machine?\n\n> Any ideas?\n\nMaybe after I can reproduce this issue locally.\n\n\nNicolas\n"},{"id":"149051","messageId":"201008260914.00741.trast@student.ethz.ch","threadId":"24860","inReplyTo":"alpine.LFD.2.00.1008252107070.622@xanadu.home","subject":"Re: Remote corruption issue, linked to thin pack code?","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2010-08-26T07:13:59Z","receivedAt":"2010-08-26T07:13:59Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Nicolas Pitre wrote:\n> > The respective repos show no errors with git-fsck.\n> \n> What about 'git fsck --full'?  Given git v1.5.4 on the remote, you \n> should try --full with 'git fsck' as this wasn't the default back then.\n\nWell, I instructed him to try --full, as you can see in the\nconversation:\n\n  http://colabti.org/irclogger/irclogger_log/git?date=2010-08-25#l2196\n\nNothing interesting.\n\n> >   http://eloxoph.com/localrepo.zip\n> >   http://eloxoph.com/remoterepo.zip\n> \n> 2010-08-25 21:24:52 ERROR 404: Not Found.\n\nHrm.  Apparently a mis-edit on my part, it should be\n\n  http://eloxoph.com/remoteserverrepo.zip\n\n> How do you know it is there while git (on the remote) is telling you it \n> is not?\n> \n> What about 'git cat-file -t e28ae6b61c384732c506' ?\n\nIt's fine in my unpacked copy.  Jonas, what were the results on the\nserver?\n\n> Maybe after I can reproduce this issue locally.\n\nAs a side note, I couldn't reproduce locally with the above repos but\nthen I didn't have the time to downgrade to 1.5.4.\n\nJonas also has to answer the rest of the questions for obvious\nreasons.  (I'm away from the net for a few days anyway...)\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"149060","messageId":"4C763E30.2050407@eloxoph.com","threadId":"24860","inReplyTo":"alpine.LFD.2.00.1008252107070.622@xanadu.home","subject":"Re: Remote corruption issue, linked to thin pack code?","fromName":"Jonas Thiem","fromEmail":"contact@eloxoph.com","sentAt":"2010-08-26T10:13:04Z","receivedAt":"2010-08-26T10:13:04Z","isPatch":false,"sender":{"key":"contact@eloxoph.com","avatar":null},"body":"-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA256\n\nHi,\n\nI'll try to provide the missing information as good as I can:\n\n> What about 'git fsck --full'?  Given git v1.5.4 on the remote, you \n> should try --full with 'git fsck' as this wasn't the default back then.\n$ GIT_DIR=./ git fsck --full\ndangling commit d298351127861349846fe626a320c66101821d72\ndangling commit a71ea8dd4c837c39ef26765574d515253ea2fd5a\n\n> What about 'git cat-file -t e28ae6b61c384732c506' ?\n$ GIT_DIR=./ git cat-file -t e28ae6b61c384732c506\ncommit\n\n> Also, what is the OS version and filesystem used on the remote machine?\nOS is Ubuntu 8.04 LTS. It's a virtual server by a hosting company,\ntherefore I don't know much about the filesystem setup, but maybe that\nhelps:\n\n$ df -T:\nFilesystem    Type   1K-blocks      Used Available Use% Mounted on\n/dev/vzfs reiserfs    10485760   4056808   6428952  39% /\n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1.4.10 (GNU/Linux)\nComment: Using GnuPG with Fedora - http://enigmail.mozdev.org/\n\niF4EAREIAAYFAkx2Ph8ACgkQVizsUChSmww9TAEAptTyg+0yqidArbg2wMLmgac+\nPrK9T3g4cU20EBOEuXkBALlqLapTQLlyO8UoFi+I3Mq64sDBBdjaEqH8bi6LcTor\n=ahRa\n-----END PGP SIGNATURE-----\n"}]}