{"thread":{"id":"17900","subject":"Recovering from missing objects?","startedAt":"2009-02-19T14:08:23Z","lastAt":"2009-02-20T14:31:22Z","messageCount":6,"participants":["Geert Uytterhoeven","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"105465","messageId":"alpine.LRH.2.00.0902191447040.16988@vixen.sonytel.be","threadId":"17900","inReplyTo":null,"subject":"Recovering from missing objects?","fromName":"Geert Uytterhoeven","fromEmail":"geert.uytterhoeven@sonycom.com","sentAt":"2009-02-19T14:08:23Z","receivedAt":"2009-02-19T14:08:23Z","isPatch":false,"sender":{"key":"geert.uytterhoeven@sonycom.com","avatar":null},"body":"\tHi all,\n\nI start to see suspicious messages about missing objects in one of my working\nrepositories:\n\n| vixen$ git gc\n| error: Could not read c406ab0be69c912ea59233595a071478103cdad8\n| fatal: bad tree object c406ab0be69c912ea59233595a071478103cdad8\n| error: failed to run repack\n| vixen$ \n\nMy setup:\n - I have one reference repository (cloned from Linus' linux-2.6.git)\n - I have several working repositories, cloned using --reference to my\n reference repository. A working repository has several remotes (cloned from\n other Linux kernel repositories).\n\nI always do a `git pull' in the reference repository, before doing a `git\nremote update' in a working repository. When I do `git gc' in a working\nrepository, it cleans up all objects that are not in the reference repository.\nHence I only need to care about backup of the .git directories in the working\nrepositories (the stuff I'm working on), and not about the reference\nrepository (its objects are publicly available and replicated all over the\nworld).\n\nI identified the missing object listed above to be part of a remote repository.\nDoing a `git remote update' doesn't fetch it again, as git is too smart and\nthinks I already have everything.\n\nIf I clone the remote repository, I have the object in the new clone.\nHowever, how do I get the missing object back into the .git directory of my\nworking repository? Is there an easy way to do that, or should I just recreate\nmy working repository from scratch, and reimport anything I can recover?\n\nAs for the cause of the problem: most probably I once added the remote\nrepository to my reference repository, and removed it later, causing all\nobjects to be removed during a subsequent gc.\n\nThanks for your answers!\n\nWith kind regards,\n\nGeert Uytterhoeven\nSoftware Architect\n\nSony Techsoft Centre Europe\nThe Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium\n\nPhone: +32 (0)2 700 8453\nFax: +32 (0)2 700 8622\nE-mail: Geert.Uytterhoeven@sonycom.com\nInternet: http://www.sony-europe.com/\n\nA division of Sony Europe (Belgium) N.V.\nVAT BE 0413.825.160 · RPR Brussels\nFortis · BIC GEBABEBB · IBAN BE41293037680010\n"},{"id":"105536","messageId":"7vvdr6j6hz.fsf@gitster.siamese.dyndns.org","threadId":"17900","inReplyTo":"alpine.LRH.2.00.0902191447040.16988@vixen.sonytel.be","subject":"Re: Recovering from missing objects?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-20T00:29:12Z","receivedAt":"2009-02-20T00:29:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Geert Uytterhoeven writes:\n\n> \tHi all,\n>\n> I start to see suspicious messages about missing objects in one of my working\n> repositories:\n>\n> | vixen$ git gc\n> | error: Could not read c406ab0be69c912ea59233595a071478103cdad8\n> | fatal: bad tree object c406ab0be69c912ea59233595a071478103cdad8\n> | error: failed to run repack\n> | vixen$ \n>\n> My setup:\n> - I have one reference repository (cloned from Linus' linux-2.6.git)\n> - I have several working repositories, cloned using --reference to my\n> reference repository. A working repository has several remotes (cloned from\n> other Linux kernel repositories).\n>\n> I always do a `git pull' in the reference repository, before doing a `git\n> remote update' in a working repository. When I do `git gc' in a working\n> repository, it cleans up all objects that are not in the reference repository.\n> Hence I only need to care about backup of the .git directories in the working\n> repositories (the stuff I'm working on), and not about the reference\n> repository (its objects are publicly available and replicated all over the\n> world).\n>\n> I identified the missing object listed above to be part of a remote repository.\n> Doing a `git remote update' doesn't fetch it again, as git is too smart and\n> thinks I already have everything.\n>\n> If I clone the remote repository, I have the object in the new clone.\n> However, how do I get the missing object back into the .git directory of my\n> working repository?\n\nIn the new clone:\n\n\t$ IT=c406ab0be69c912ea59233595a071478103cdad8\n\t$ TYPE=$(git cat-file -t $IT)\n $ git cat-file $TYPE $IT >/var/tmp/$IT.raw\n\nGo to the repository that lacks the object and then\n\n\t$ git hash-object -t $TYPE -w --stdin > If I clone the remote repository, I have the object in the new clone.\n> > However, how do I get the missing object back into the .git directory of my\n> > working repository?\n> [...]\n> In the new clone:\n> \n> \t$ IT=c406ab0be69c912ea59233595a071478103cdad8\n> \t$ H=$(git rev-list --objects $IT | git pack-objects mine)\n> $ mv mine-$H.pack /var/tmp\n> \n> Go to the repository that lacks the object and then\n> \n> \t$ git unpack-objects writes:\n\n> Might it not be simpler to just copy or hardlink the pack from the new\n> clone into the old directory's .git/objects/pack? That will get more\n> than you need, but things should start working, at which point a \"git\n> repack -a -d\" will make it small again.\n>\n> Or am I misunderstanding something?\n\nIf you already have a good clone and a half-broken one locally, of course,\nwhat you said is the easiest.\n\nI just assumed that Geert did not really want to copy the whole pack.\nMaybe he doesn't mind in this particular case, but the next breakage may\ninvolve gigapacks he'd rather not re-clone.\n\nI also assumed that anybody who is reading the message can easily guess\nthat the copy I was demonstrating in the description could be done across\nmachines, instead of via local /var/tmp/, and \"In the new clone\" steps\ncould even be done in the original one. Presenting the knowledge that\nway, the solution hopefully would be adjustable for more people who are\nreading this thread.\n"},{"id":"105564","messageId":"20090220061046.GA28122@coredump.intra.peff.net","threadId":"17900","inReplyTo":"7vmychiqr3.fsf@gitster.siamese.dyndns.org","subject":"Re: Recovering from missing objects?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-02-20T06:10:46Z","receivedAt":"2009-02-20T06:10:46Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 19, 2009 at 10:09:20PM -0800, Junio C Hamano wrote:\n\n> > Might it not be simpler to just copy or hardlink the pack from the new\n> > clone into the old directory's .git/objects/pack? That will get more\n> > than you need, but things should start working, at which point a \"git\n> > repack -a -d\" will make it small again.\n> >\n> > Or am I misunderstanding something?\n> \n> If you already have a good clone and a half-broken one locally, of course,\n> what you said is the easiest.\n> \n> I just assumed that Geert did not really want to copy the whole pack.\n> Maybe he doesn't mind in this particular case, but the next breakage may\n> involve gigapacks he'd rather not re-clone.\n\nAh, OK. I had the impression that he had already made a new valid clone\non the local box.\n\n> I also assumed that anybody who is reading the message can easily guess\n> that the copy I was demonstrating in the description could be done across\n> machines, instead of via local /var/tmp/, and \"In the new clone\" steps\n> could even be done in the original one. Presenting the knowledge that\n> way, the solution hopefully would be adjustable for more people who are\n> reading this thread.\n\nMakes sense. Thanks.\n\n-Peff\n"},{"id":"105612","messageId":"alpine.LRH.2.00.0902201440140.18017@vixen.sonytel.be","threadId":"17900","inReplyTo":"7vvdr6j6hz.fsf@gitster.siamese.dyndns.org","subject":"Re: Recovering from missing objects?","fromName":"Geert Uytterhoeven","fromEmail":"geert.uytterhoeven@sonycom.com","sentAt":"2009-02-20T14:31:22Z","receivedAt":"2009-02-20T14:31:22Z","isPatch":false,"sender":{"key":"geert.uytterhoeven@sonycom.com","avatar":null},"body":"\tHi Junio,\n\nOn Thu, 19 Feb 2009, Junio C Hamano wrote:\n> Geert Uytterhoeven writes:\n> > I start to see suspicious messages about missing objects in one of my working\n> > repositories:\n> >\n> > | vixen$ git gc\n> > | error: Could not read c406ab0be69c912ea59233595a071478103cdad8\n> > | fatal: bad tree object c406ab0be69c912ea59233595a071478103cdad8\n> > | error: failed to run repack\n> > | vixen$ \n> >\n> > I identified the missing object listed above to be part of a remote repository.\n> > Doing a `git remote update' doesn't fetch it again, as git is too smart and\n> > thinks I already have everything.\n> >\n> > If I clone the remote repository, I have the object in the new clone.\n> > However, how do I get the missing object back into the .git directory of my\n> > working repository?\n> \n> In the new clone:\n> \n> \t$ IT=c406ab0be69c912ea59233595a071478103cdad8\n> \t$ TYPE=$(git cat-file -t $IT)\n> $ git cat-file $TYPE $IT >/var/tmp/$IT.raw\n> \n> Go to the repository that lacks the object and then\n> \n> \t$ git hash-object -t $TYPE -w --stdin \n> After that you may find objects that $IT needs to reference. You can\n> obviously repeat the above procedure until you have nothing missing.\n\nThanks! It worked fine for most of the missing objects.\n\nFor some of them (type commit), I still get:\n\n| vixen$ git show 32582324956483840d1ae90726bbe879cc48f63d\n| fatal: unable to read destination tree (32582324956483840d1ae90726bbe879cc48f63d\n| vixen$ \n\nafter importing it from the other repository.\n\n> I also suspect you could do this instead; I haven't thought things through\n> and that is why I say \"suspect\" but this is safe (i.e. not destructive)\n> and may worth a try.\n> \n> In the new clone:\n> \n> \t$ IT=c406ab0be69c912ea59233595a071478103cdad8\n> \t$ H=$(git rev-list --objects $IT | git pack-objects mine)\n> $ mv mine-$H.pack /var/tmp\n> \n> Go to the repository that lacks the object and then\n> \n> \t$ git unpack-objects