threads / discuss / 12706

Re: Repository corruption

Subject: Re: Repository corruption

## tl;dr

4 messages between Mar 15, 2008 and Mar 16, 2008.

replies: 3people: 4as markdown or json

Yann Dirson· Mar 15, 2008, 13:04 UTC · lore

On Sat, Mar 15, 2008 at 01:19:03PM +0100, Petr Baudis wrote:

Show 9 quoted lines

>   Hi,
> 
> On Mon, Sep 10, 2007 at 11:25:50PM +0200, Yann Dirson wrote:
> > Not sure if you saw my note in IRC: it looks like tagua/ydirson.git is
> > definitely corrupted - not sure it would be useful to keep a copy
> > around for investigation, but I'd appreciate if you could nuke the
> > contents, so I can push again to it.
> 
>   I have nuked it now. :-)

Thanks :)

Any idea what caused the corruption ?

BTW - although I have not reported it on the list, I recently had a repo corrupted by a "git gc" with 1.5.4.3. Not sure there would be a link between those 2 events, but who knows...

It may not be easy to investigate, since I do not have an exact copy of it before the corruption (fortunately I was able to salvage most of the work that had no backup copy), since that repo is a private one holding source code I am not allowed to publish, and since I will be leaving in 2 weeks the position that gives me access to that code.

The corruptions were of several kinds:
- commits for unapplied stgit patches were all lost, although they
were reachable from refs/patches/master/*
- seemingly-random objects in the history (commits, trees, probably
blobs as well) were missing

Does anyone have ideas about post-mortem analysis on that repo ?

That tree was managed with stgit, using a rebase/commit workflow, and accumulated objects reported by git-count-objects over the months kept growing, even after "git repack -a" and "git prune", which always looked suspect to me.

Although it is possible that I had already run git-gc on that tree, that would not be recently - it was basically a "let's try it and see if we perform better afterwards".

Best regards,

-- 
Yann

Christian Couder· Mar 16, 2008, 05:49 UTC · re: Yann Dirson · lore

Hi Yann,

Le samedi 15 mars 2008, Yann Dirson a écrit :

Show 7 quoted lines

> The corruptions were of several kinds:
> - commits for unapplied stgit patches were all lost, although they
> were reachable from refs/patches/master/*
> - seemingly-random objects in the history (commits, trees, probably
> blobs as well) were missing
>
> Does anyone have ideas about post-mortem analysis on that repo ?

Did you have a look at:

http://git.kernel.org/?p=git/git.git;a=blob;f=Documentation/howto/recover-corrupted-blob-object.txt;h=323b513ed0e0ce8b749672f589a375073a050b97;hb=HEAD

and

http://git.or.cz/gitwiki/GitFaq#head-ac11406480d09e2df98588e800e41b7256602074

Best regards, Christian.

Petr Baudis· Mar 16, 2008, 23:01 UTC · re: Yann Dirson · lore

On Sat, Mar 15, 2008 at 02:04:41PM +0100, Yann Dirson wrote:

Show 14 quoted lines

> On Sat, Mar 15, 2008 at 01:19:03PM +0100, Petr Baudis wrote:
> >   Hi,
> > 
> > On Mon, Sep 10, 2007 at 11:25:50PM +0200, Yann Dirson wrote:
> > > Not sure if you saw my note in IRC: it looks like tagua/ydirson.git is
> > > definitely corrupted - not sure it would be useful to keep a copy
> > > around for investigation, but I'd appreciate if you could nuke the
> > > contents, so I can push again to it.
> > 
> >   I have nuked it now. :-)
> 
> Thanks :)
> 
> Any idea what caused the corruption ?

My idea I got few days ago when looking at the borken repositories is that it was caused by interrupted git-repack, since there were stale tmp_pack* files left in the repository - it seems like it removed old "redundant" packs even though the new one weren't properly completed. But that is just a speculation, and I did not get around to look at the script yet whether this could really happen, or if something like this got fixed recently.

-- 
				Petr "Pasky" Baudis
Whatever you can do, or dream you can, begin it.
Boldness has genius, power, and magic in it.	-- J. W. von Goethe

Junio C Hamano· Mar 16, 2008, 23:14 UTC · re: Petr Baudis · lore

Petr Baudis <pasky@suse.cz> writes:

Show 7 quoted lines

> My idea I got few days ago when looking at the borken repositories is
> that it was caused by interrupted git-repack, since there were stale
> tmp_pack* files left in the repository - it seems like it removed old
> "redundant" packs even though the new one weren't properly completed.
> But that is just a speculation, and I did not get around to look at the
> script yet whether this could really happen, or if something like this
> got fixed recently.

Hmm, unless you are talking about ef07618 (git-repack: Properly abort in corrupt repository, 2005-11-21) as "recent", I do not see anything suspicious.

We may want to refine the first comment in the remove-redundant loop to clarify, though. We do check if elements of $existing are actually redundant or not in the loop against $fullbases these days, since ce85907 (Only repack active packs by skipping over kept packs., 2006-10-29).