Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file

From: Nicolas Pitre <nico@cam.org>
Date: Jan 7, 2009, 01:25 UTC
Message-ID: <alpine.LFD.2.00.0901062005290.26118@xanadu.home>
In-Reply-To: <1231282320.8870.52.camel@starfruit>

On Tue, 6 Jan 2009, R. Tyler Ballance wrote:

Show 28 quoted lines

> On Tue, 2008-12-09 at 09:36 +0100, Jan Krüger wrote:
> > For fixing a corrupted repository by using backup copies of individual
> > files, allow write_sha1_file() to write loose files even if the object
> > already exists in a pack file, but only if the existing entry is marked
> > as corrupted.
> 
> I figured I'd reply to this again, since the issue cropped up again.
> 
> We started experiencing *large* numbers of corruptions like the ones
> that started the thread (one developer was receiving them once or twice
> a day) with v1.6.0.4
> 
> We went ahead and upgraded to a custom build of v1.6.1 with Jan's patch
> (below) and the issues /seem/ to have resolved themselves. I'm not
> certain whether Jan's patch was really responsible, or if there was
> another issue that caused this to correct itself in v1.6.1. 
> 
> As it stands, I think it's safe to assume that given the frequency of
> the occurances that they were not tied to a memory or disk error (or
> other levels of the machine's stack would be suffering as well). The
> only thing I can think of is that /some/ developers who've experienced
> the issue are using Samba mount points and changing files in Mac OS X,
> but using Git on the mounted share (i.e. TextMate changes a file hosted
> on Samba, changes are committed in an SSH session on that machine), but
> that doesn't account for everything.
> 
> If there was something else included in the v1.6.1 release please let me
> know so I can back Jan's patch out.

Please back it out. As it stands, that patch is a no op because of the way git is used, and even if the patch was to work as intended, its purpose is not to magically fix corruptions without special action from your part. If you have corruption problems coming back only because of the removal of this patch then something is really really fishy and I would really like to know about it.

There were indeed many changes between v1.6.0.4 and v1.6.1: the exact number is 1029. A couple of them are especially addressing increased robustness against some kind of pack corruptions. But in any case you still should see error messages appearing about them.

And don't underestimate the power of disk corruptions. I started to work on git corruption resilience simply because I ended up with a corrupted pack at some point. Then a while later I got another corrupted pack. Then another while later I lost my filesystem entirely and had to reinstall my system (after buying a new disk). Turns out that my old disk is silently corrupting data without signaling any errors to the host.

Nicolas