Recovering from missing objects?

	Hi all,

I start to see suspicious messages about missing objects in one of my working repositories:

| vixen$ git gc
| error: Could not read c406ab0be69c912ea59233595a071478103cdad8
| fatal: bad tree object c406ab0be69c912ea59233595a071478103cdad8
| error: failed to run repack
| vixen$ 

My setup:
  - I have one reference repository (cloned from Linus' linux-2.6.git)
  - I have several working repositories, cloned using --reference to my
    reference repository. A working repository has several remotes (cloned from
    other Linux kernel repositories).

I always do a `git pull' in the reference repository, before doing a `git remote update' in a working repository. When I do `git gc' in a working repository, it cleans up all objects that are not in the reference repository. Hence I only need to care about backup of the .git directories in the working repositories (the stuff I'm working on), and not about the reference repository (its objects are publicly available and replicated all over the world).

I identified the missing object listed above to be part of a remote repository. Doing a `git remote update' doesn't fetch it again, as git is too smart and thinks I already have everything.

If I clone the remote repository, I have the object in the new clone. However, how do I get the missing object back into the .git directory of my working repository? Is there an easy way to do that, or should I just recreate my working repository from scratch, and reimport anything I can recover?

As for the cause of the problem: most probably I once added the remote repository to my reference repository, and removed it later, causing all objects to be removed during a subsequent gc.

Thanks for your answers!

With kind regards,

Geert Uytterhoeven Software Architect

Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010

Geert Uytterhoeven <Geert.Uytterhoeven@sonycom.com> writes:

In the new clone:

	$ IT=c406ab0be69c912ea59233595a071478103cdad8
	$ TYPE=$(git cat-file -t $IT)
        $ git cat-file $TYPE $IT >/var/tmp/$IT.raw

Go to the repository that lacks the object and then

	$ git hash-object -t $TYPE -w --stdin </var/tmp/$IT.raw

After that you may find objects that $IT needs to reference. You can obviously repeat the above procedure until you have nothing missing.

I also suspect you could do this instead; I haven't thought things through and that is why I say "suspect" but this is safe (i.e. not destructive) and may worth a try.

In the new clone:

	$ IT=c406ab0be69c912ea59233595a071478103cdad8
	$ H=$(git rev-list --objects $IT | git pack-objects mine)
        $ mv mine-$H.pack /var/tmp

Go to the repository that lacks the object and then

	$ git unpack-objects </var/tmp/mine-$H.pack

On Thu, Feb 19, 2009 at 04:29:12PM -0800, Junio C Hamano wrote:

Might it not be simpler to just copy or hardlink the pack from the new clone into the old directory's .git/objects/pack? That will get more than you need, but things should start working, at which point a "git repack -a -d" will make it small again.

Or am I misunderstanding something?

-Peff

Jeff King <peff@peff.net> writes:

If you already have a good clone and a half-broken one locally, of course, what you said is the easiest.

I just assumed that Geert did not really want to copy the whole pack. Maybe he doesn't mind in this particular case, but the next breakage may involve gigapacks he'd rather not re-clone.

I also assumed that anybody who is reading the message can easily guess that the copy I was demonstrating in the description could be done across machines, instead of via local /var/tmp/, and "In the new clone" steps could even be done in the original one. Presenting the knowledge that way, the solution hopefully would be adjustable for more people who are reading this thread.

On Thu, Feb 19, 2009 at 10:09:20PM -0800, Junio C Hamano wrote:

Ah, OK. I had the impression that he had already made a new valid clone on the local box.

Makes sense. Thanks.

-Peff

	Hi Junio,

On Thu, 19 Feb 2009, Junio C Hamano wrote:

Thanks! It worked fine for most of the missing objects.

For some of them (type commit), I still get:

| vixen$ git show 32582324956483840d1ae90726bbe879cc48f63d
| fatal: unable to read destination tree (32582324956483840d1ae90726bbe879cc48f63d
| vixen$ 

after importing it from the other repository.

That one did the trick for the "stubborn" commits I couldn't get imported the other way.

BTW, I kept a copy of the repository with the missing commit 32582324956483840d1ae90726bbe879cc48f63d, just in case you have another idea to revive that single one without creating a (big) pack ;-)

Anyway, I think it wouldn't hurt to have an option for "git remote update" to retrieve all lost commits from the remote...

With kind regards,

Geert Uytterhoeven Software Architect

Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010