{"thread":{"id":"20109","subject":"Recover broken git repository?","startedAt":"2009-07-14T13:20:32Z","lastAt":"2009-07-14T19:43:10Z","messageCount":9,"participants":["Florian Breitwieser","Alex Riesen","Nicolas Sebrecht","Jakub Narebski","Linus Torvalds","Stefan-W. Hahn","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"117957","messageId":"1247577632053-3255106.post@n2.nabble.com","threadId":"20109","inReplyTo":null,"subject":"Recover broken git repository?","fromName":"Florian Breitwieser","fromEmail":"florian.bw@gmail.com","sentAt":"2009-07-14T13:20:32Z","receivedAt":"2009-07-14T13:20:32Z","isPatch":false,"sender":{"key":"florian.bw@gmail.com","avatar":null},"body":"\nHi, \nI have problems with my git repository, attached below are the steps I tried\nto resolve it. But now I am stuck. Is there any good way to recover?\n\nThank you\nFlorian\n\n-- \nFlorian Breitwieser\nPhD Student\n\nResearch Center of Molecular Medicine\nof the Austrian Academy of Science\n\n-------------------\n\n$ git commit -m \"Some message\"\nerror: invalid object 1086b1c606a04bcb78b92d1d411a299d20d18034\nfatal: Error building trees\n\n$ git-fsck --full\ndangling tree c2549a3cdd83098a523cb707f217f4656cde7eb5\n\n$ git prune\n$ git commit -m \"Some message\"\nCreated commit e32d5dd: Some message\n 2 files changed, 167 insertions(+), 10 deletions(-)\n\n$ git push\nCounting objects: 15, done.\nCompressing objects: 100% (6/6), done.\nWriting objects: 100% (8/8), 2.44 KiB, done.\nTotal 8 (delta 4), reused 0 (delta 0)\nUnpacking objects: 100% (8/8), done.\nfatal: unresolved deltas left after unpacking\nerror: unpack failed: unpacker exited with error code\nTo /mnt/extstore2/bioinformatic/git/ppi.git\n ! [remote rejected] master -> master (n/a (unpacker error))\nerror: failed to push some refs to\n'/mnt/extstore2/bioinformatic/git/ppi.git'\n\n-- \nView this message in context: http://n2.nabble.com/Recover-broken-git-repository--tp3255106p3255106.html\nSent from the git mailing list archive at Nabble.com.\n"},{"id":"117964","messageId":"81b0412b0907140740j4fa1e12csc4c3081719f9090f@mail.gmail.com","threadId":"20109","inReplyTo":"1247577632053-3255106.post@n2.nabble.com","subject":"Re: Recover broken git repository?","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2009-07-14T14:40:08Z","receivedAt":"2009-07-14T14:40:08Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"On Tue, Jul 14, 2009 at 15:20, Florian Breitwieser<florian.bw@gmail.com> wrote:\n> I have problems with my git repository, attached below are the steps I tried\n> to resolve it. But now I am stuck. Is there any good way to recover?\n\nThere are recovery instructions all over the web, and on this mailing list.\nTry googling for \"git repository corruption recovery\".\nThere is even one in Git's repository:\n\n  Documentation/howto/recover-corrupted-blob-object.txt\n\n> $ git commit -m \"Some message\"\n> error: invalid object 1086b1c606a04bcb78b92d1d411a299d20d18034\n\nLooks like you have hit a genuine repository corruption.\n\nYou can try to look for the object in all copies of the project (or just in any\nprojects you possibly have laying around) and copy it into you repo.\nTo test if the object is present in a repo:\n\n  git cat-file -t 1086b1c606a04bcb78b92d1d411a299d20d18034\n\nYou can just copy (or reference in .git/objects/info/alternates) the whole\nrepo containing the object in the broken repo. You can cleanup unused\nobjects afterwards.\n\n> fatal: Error building trees\n\nYou can try \"git log --pretty=raw --raw --no-abbrev HEAD | less\" and\nlook for the first commit which introduced the invalid object. The history\nup to this commit must be intact, BTW. Try to find out what object it was.\nIf it was a file, than maybe you have the original source somewhere,\nor the latest version is present and good enough. Then you can checkout\nthe good commit, introduce the file from the source, commit, and cherry-pick\nall the newer commits on top of it.\nIf it was a tree, you can try to guess what files did it contain in the\ncommit which references it and try to rebuild it by using git add and\ngit write-tree (the latter prints the SHA1 of the generated tree on\nstdout, so you can retry indefinitely).\nIf it happens to be a commit (you'll find the SHA1 in parents of\nthe good commits, but it does not look like it is the case), you can\ntry git fsck --lost-found to find all the commits which are not\nreferenced, try connecting them (by using grafts, for example.\nOr checkout and cherry-pick described above).\n\n> $ git-fsck --full\n> dangling tree c2549a3cdd83098a523cb707f217f4656cde7eb5\n>\n> $ git prune\n\nNow that was dangerous. When something was unreferenced\n(but useful) now it is lost.\n"},{"id":"117966","messageId":"20090714145432.GA12928@vidovic","threadId":"20109","inReplyTo":"1247577632053-3255106.post@n2.nabble.com","subject":"Re: Recover broken git repository?","fromName":"Nicolas Sebrecht","fromEmail":"nicolas.s.dev@gmx.fr","sentAt":"2009-07-14T14:54:32Z","receivedAt":"2009-07-14T14:54:32Z","isPatch":false,"sender":{"key":"nicolas.s.dev@gmx.fr","avatar":null},"body":"The 14/07/09, Florian Breitwieser wrote:\n\n> I have problems with my git repository, attached below are the steps I tried\n> to resolve it. But now I am stuck. Is there any good way to recover?\n\nYou could git-format-patch your topics, clone a new repo and git-am the\npatches.\n\n-- \nNicolas Sebrecht\n"},{"id":"117968","messageId":"m3eisjuw8d.fsf@localhost.localdomain","threadId":"20109","inReplyTo":"20090714145432.GA12928@vidovic","subject":"Re: Recover broken git repository?","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2009-07-14T15:20:57Z","receivedAt":"2009-07-14T15:20:57Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Nicolas Sebrecht <nicolas.s.dev@gmx.fr> writes:\n> The 14/07/09, Florian Breitwieser wrote:\n> \n> > I have problems with my git repository, attached below are the steps I tried\n> > to resolve it. But now I am stuck. Is there any good way to recover?\n> \n> You could git-format-patch your topics, clone a new repo and git-am the\n> patches.\n\nBetter solution, if possible, would be to use git-fast-export /\ngit-fast-import pair.\n\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"117972","messageId":"alpine.LFD.2.01.0907141019380.13838@localhost.localdomain","threadId":"20109","inReplyTo":"1247577632053-3255106.post@n2.nabble.com","subject":"Re: Recover broken git repository?","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2009-07-14T17:33:58Z","receivedAt":"2009-07-14T17:33:58Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 14 Jul 2009, Florian Breitwieser wrote:\n>\n> $ git commit -m \"Some message\"\n> error: invalid object 1086b1c606a04bcb78b92d1d411a299d20d18034\n> fatal: Error building trees\n\nHmm. That \"invalid object\" error comes from 'update_one()' when it cannot \nfind the object it is looking for. That, in turn, tends to be an issue of \nthe index containing a SHA1 that does not exist in the database.\n\nDo you trust your filesystem? The symptoms really sound like you did a \n\"git add ..\" earlier, and populated the index, but now the object that got \npopulated is no longer found.\n\nWhat OS, what filesystem?  Oh, and what version of git?\n\n> $ git-fsck --full\n> dangling tree c2549a3cdd83098a523cb707f217f4656cde7eb5\n\nThe fsck seems to imply that things are ok. The fsck _should_ have checked \nthe index too.\n\nThe 'git commit' obviously disagrees.\n\nTwo things to check:\n\n - do you use grafts to hide old history, rather than connect additional \n   history?\n\n   That can be really dangerous, and will make fsck (and other tools) not \n   look at the hidden state. \n\n - do you have alternates (is multiple _different_ repositories tat point \n   to each other)? That can cause problems if you then do things like git \n   prune in them. It wouldn't explain this particular case, but it's \n   something I look out for when I hear about corruption.\n\n> $ git prune\n\nNot a good idea. When you suspect corruption, the _last_ thing you want to \ndo is prune things. Who knows what happened?\n\n> $ git commit -m \"Some message\"\n> Created commit e32d5dd: Some message\n>  2 files changed, 167 insertions(+), 10 deletions(-)\n\nSo now the same commit worked? It _really_ sounds like your filesystem has \na hard time finding the files it just created. Some local caching issue.\n\nIs it a network filesystem? If so, what's the filesystem server and \nversion, if you can find it out?\n\n> $ git push\n> Counting objects: 15, done.\n> Compressing objects: 100% (6/6), done.\n> Writing objects: 100% (8/8), 2.44 KiB, done.\n> Total 8 (delta 4), reused 0 (delta 0)\n> Unpacking objects: 100% (8/8), done.\n> fatal: unresolved deltas left after unpacking\n> error: unpack failed: unpacker exited with error code\n> To /mnt/extstore2/bioinformatic/git/ppi.git\n>  ! [remote rejected] master -> master (n/a (unpacker error))\n> error: failed to push some refs to\n> '/mnt/extstore2/bioinformatic/git/ppi.git'\n\nHere the other end doesn't have some object that we expect it to have, and \nthat we sent a delta to it against. Sounds like potentially the exact same \nproblem (it created some new file, but then couldn't see it). Is the thign \nyou are pushing to a similar machine with similar filesystems?\n\nWe've had issues with both CIFS and NFS. Using\n\n\t[core]\n\t\tfsyncobjectfiles = true\n\nmight work around some issues where the filesystem does crazy things. But \nI'd really like to hear what OS and filesystem versions you're running.\n\n\t\t\tLinus\n"},{"id":"117974","messageId":"20090714174819.GC9919@scotty.home","threadId":"20109","inReplyTo":"alpine.LFD.2.01.0907141019380.13838@localhost.localdomain","subject":"Re: Recover broken git repository?","fromName":"Stefan-W. Hahn","fromEmail":"stefan.hahn@s-hahn.de","sentAt":"2009-07-14T17:48:19Z","receivedAt":"2009-07-14T17:48:19Z","isPatch":false,"sender":{"key":"stefan.hahn@s-hahn.de","avatar":null},"body":"Also sprach Linus Torvalds am Tue, 14 Jul 2009 at 10:33:58 -0700:\n> \n> \n> On Tue, 14 Jul 2009, Florian Breitwieser wrote:\n> >\n> > $ git commit -m \"Some message\"\n> > error: invalid object 1086b1c606a04bcb78b92d1d411a299d20d18034\n> > fatal: Error building trees\n> \n> Hmm. That \"invalid object\" error comes from 'update_one()' when it cannot \n> find the object it is looking for. That, in turn, tends to be an issue of \n> the index containing a SHA1 that does not exist in the database.\n> \n> Do you trust your filesystem? The symptoms really sound like you did a \n> \"git add ..\" earlier, and populated the index, but now the object that got \n> populated is no longer found.\n> \n> What OS, what filesystem?  Oh, and what version of git?\n> \n> > $ git-fsck --full\n> > dangling tree c2549a3cdd83098a523cb707f217f4656cde7eb5\n> \n> The fsck seems to imply that things are ok. The fsck _should_ have checked \n> the index too.\n> \n> The 'git commit' obviously disagrees.\n> \n> Two things to check:\n> \n>  - do you use grafts to hide old history, rather than connect additional \n>    history?\n> \n>    That can be really dangerous, and will make fsck (and other tools) not \n>    look at the hidden state. \n> \n>  - do you have alternates (is multiple _different_ repositories tat point \n>    to each other)? That can cause problems if you then do things like git \n>    prune in them. It wouldn't explain this particular case, but it's \n>    something I look out for when I hear about corruption.\n> \n> > $ git prune\n> \n> Not a good idea. When you suspect corruption, the _last_ thing you want to \n> do is prune things. Who knows what happened?\n> \n> > $ git commit -m \"Some message\"\n> > Created commit e32d5dd: Some message\n> >  2 files changed, 167 insertions(+), 10 deletions(-)\n> \n> So now the same commit worked? It _really_ sounds like your filesystem has \n> a hard time finding the files it just created. Some local caching issue.\n> \n> Is it a network filesystem? If so, what's the filesystem server and \n> version, if you can find it out?\n> \n> > $ git push\n> > Counting objects: 15, done.\n> > Compressing objects: 100% (6/6), done.\n> > Writing objects: 100% (8/8), 2.44 KiB, done.\n> > Total 8 (delta 4), reused 0 (delta 0)\n> > Unpacking objects: 100% (8/8), done.\n> > fatal: unresolved deltas left after unpacking\n> > error: unpack failed: unpacker exited with error code\n> > To /mnt/extstore2/bioinformatic/git/ppi.git\n> >  ! [remote rejected] master -> master (n/a (unpacker error))\n> > error: failed to push some refs to\n> > '/mnt/extstore2/bioinformatic/git/ppi.git'\n> \n> Here the other end doesn't have some object that we expect it to have, and \n> that we sent a delta to it against. Sounds like potentially the exact same \n> problem (it created some new file, but then couldn't see it). Is the thign \n> you are pushing to a similar machine with similar filesystems?\n> \n> We've had issues with both CIFS and NFS. Using\n> \n> \t[core]\n> \t\tfsyncobjectfiles = true\n> \n> might work around some issues where the filesystem does crazy things. But \n> I'd really like to hear what OS and filesystem versions you're running.\n> \n> \t\t\tLinus\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n\n-- \nStefan-W. Hahn                          It is easy to make things.\n/ mailto:stefan.hahn@s-hahn.de /        It is hard to make things simple.\t\t\t\n\nPlease note that according to the German law on data retention,\ninformation on every electronic information exchange with me is\nretained for a period of six months.\nBundesgesetzblatt:\nhttp://www.bgblportal.de/BGBL/bgbl1f/bgbl107s3198.pdf\n"},{"id":"117975","messageId":"20090714175245.GD9919@scotty.home","threadId":"20109","inReplyTo":"alpine.LFD.2.01.0907141019380.13838@localhost.localdomain","subject":"Re: Recover broken git repository?","fromName":"Stefan-W. Hahn","fromEmail":"stefan.hahn@s-hahn.de","sentAt":"2009-07-14T17:52:45Z","receivedAt":"2009-07-14T17:52:45Z","isPatch":false,"sender":{"key":"stefan.hahn@s-hahn.de","avatar":null},"body":"Also sprach Linus Torvalds am Tue, 14 Jul 2009 at 10:33:58 -0700:\n\nHi,\n\n> > $ git commit -m \"Some message\"\n> > error: invalid object 1086b1c606a04bcb78b92d1d411a299d20d18034\n> > fatal: Error building trees\n\nI've trapped in a similar problem few days ago on windows using\ncygwin.\n\nA simple \"git status\" told me an invalid object. Just doing\ngit status corrupted the index file and only this. The reason was a\nwrong configured (unseen reconfigured from another script) mount via\ncygwin using \"textmode\" not \"binary mode\".\n\nAfter remounting with \"binary mode\" a simple \"git read-tree HEAD\"\nrepaired it. Just the index file had been corrupted.\n\n\nStefan\n\n-- \nStefan-W. Hahn                          It is easy to make things.\n/ mailto:stefan.hahn@s-hahn.de /        It is hard to make things simple.\t\t\t\n\nPlease note that according to the German law on data retention,\ninformation on every electronic information exchange with me is\nretained for a period of six months.\nBundesgesetzblatt:\nhttp://www.bgblportal.de/BGBL/bgbl1f/bgbl107s3198.pdf\n"},{"id":"117979","messageId":"alpine.LFD.2.01.0907141119190.13838@localhost.localdomain","threadId":"20109","inReplyTo":"alpine.LFD.2.01.0907141019380.13838@localhost.localdomain","subject":"[PATCH] Improve on the 'invalid object' error message at commit time","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2009-07-14T18:25:17Z","receivedAt":"2009-07-14T18:25:17Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\nNot that anybody should ever get it, but somebody did (probably because\nof a flaky filesystem, but whatever).  And each time I see an error\nmessage that I haven't seen before, I decide that next time it will look\nbetter.\n\nSo this makes us write more relevant information about exactly which\nfile ended up having issues with a missing object.  Which will tell\nwhether it was a tree object, for example, or just a regular file in the\nindex (and which one).\n\nSigned-off-by: Linus Torvalds <torvalds@linux-foundation.org>\n---\n\nNot a big deal. But since somebody actually saw this message, let's just \nmake it more informative. \n\nA lot of the \"these can't happen unless you're seriously screwed\" messages \naren't very good, because there is little upside when doing development. \nBut let's try to improve on them in case they happen in the future.\n\nNot that this would make debugging much easier, but one thing that I \nstarted wondering about was whether the problem Florian saw was about one \nof the files he had done \"git add\" on, or whether it was a tree entry that \nwas the result of \"find_subtree()\".\n\n cache-tree.c |    3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/cache-tree.c b/cache-tree.c\nindex 16a65df..d917437 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -329,7 +329,8 @@ static int update_one(struct cache_tree *it,\n \t\t\tentlen = pathlen - baselen;\n \t\t}\n \t\tif (mode != S_IFGITLINK && !missing_ok && !has_sha1_file(sha1))\n-\t\t\treturn error(\"invalid object %s\", sha1_to_hex(sha1));\n+\t\t\treturn error(\"invalid object %06o %s for '%.*s'\",\n+\t\t\t\tmode, sha1_to_hex(sha1), entlen+baselen, path);\n \n \t\tif (ce->ce_flags & CE_REMOVE)\n \t\t\tcontinue; /* entry being removed */\n"},{"id":"117982","messageId":"7vab37giep.fsf@alter.siamese.dyndns.org","threadId":"20109","inReplyTo":"alpine.LFD.2.01.0907141119190.13838@localhost.localdomain","subject":"Re: [PATCH] Improve on the 'invalid object' error message at commit time","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-07-14T19:43:10Z","receivedAt":"2009-07-14T19:43:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> writes:\n\n> ... one thing that I \n> started wondering about was whether the problem Florian saw was about one \n> of the files he had done \"git add\" on, or whether it was a tree entry that \n> was the result of \"find_subtree()\".\n\nSame here.\n\nHistorically, cache-tree had issues that same corner case codepaths forgot\ninvalidating subtrees when they should that could have led to kinds of\nbugs that are hard to diagnose.  I think the API got harder to make that\nkind of mistakes while we squashed those bugs, though.\n"}]}