Re: Error converting from 1.4.4.1 to 1.5.0?

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Feb 14, 2007, 18:42 UTC
Message-ID: <Pine.LNX.4.64.0702141033400.3604@woody.linux-foundation.org>
In-Reply-To: <Pine.LNX.4.64.0702140958440.3604@woody.linux-foundation.org>

On Wed, 14 Feb 2007, Linus Torvalds wrote:

> 
> And if you can make the git history available to outsiders, I'd love to 
> see the corrupt tar-file (it doesn't have to be *public*, if you just can 
> trust me and perhaps a few other people with the data).

Side note: one reason why this is nice - even if you don't care about the corruption and can fix it other ways - is that the last time we had the one-bit corruption is also the reason why we now have the "-r" option to git-unpack-objects.

In other words, real-life corruption is not just a really nasty event, it's also a good way for *us* to verify that our recovery tools do as good a job as they possibly can. Maybe there are other things like that "-r" option where we could possibly do even better.

The git data structures are designed to be extremely robust, but there's nothing they can do about "corruption after the fact". The same way that a logging filesystem doesn't help if the disk itself starts getting read errors, the git data structures aren't going to guarantee that you can't lose data if you have actual disk or memory corruption going on.

The things git can do is:

 - detection. The SHA1's should basically guarantee that you will never 
   ever have an _undetectable_ corruption anywhere (which is really really 
   easy with just about any other SCM)

 - make replication easy (so that once you've detected corruption, you 
   have mirrors you can trust).

 - and finally: in the absense of replication, we can do  our damndest to 
   try to figure out what the data was. But in many ways, the fact that we 
   are really really good at compressing data (people do love their small 
   repositories) also means that we have basically no redundancy anywhere, 
   because redundancy is what compression gets rid of (both delta- and 
   zlib compression do it - it's very fundamentally what any compression 
   is based on)

but it's always interesting to have real-life corruption cases to verify.

			Linus