Re: [PATCH 0/3] don't let disk corruptions escape pack SHA1 checksum

From: Shawn O. Pearce <spearce@spearce.org>
Date: Aug 29, 2008, 04:55 UTC
Message-ID: <20080829045535.GB28492@spearce.org>
In-Reply-To: <alpine.LFD.1.10.0808282142490.1624@xanadu.home>

Nicolas Pitre <nico@cam.org> wrote:

Show 16 quoted lines

> 
> So here is 3 patches for this in C git (Shawn already did it for jgit).
> 
> There are 3 spots where this needs to be plugged: in pack-objects, 
> index-pack and fast-import.  I did the first two.
> 
> I don't think this is that pertinent in fast-import because:
> 
>  - we want fast-import to be fast
> 
>  - after fast-import is done, a full repack is typically done which 
>    would revalidate everything (maybe fast-import should use index v2 
>    with its per object CRC32 though)
> 
> But maybe I'm wrong about that. In any case, I am leaving the 
> fast-import to someone more knowledgeable about its code than I do.

fast-import is the backend to a number of incremental import tools. They stream data in from foreign VCS systems (notably Perforce) and write into a live repository. That repository may not get repacked for days/weeks. We should treat it just like index-pack.

So I think we should plug it. The cost to keep a running SHA-1 for what we have written is pretty low. I'd rather trade off a minute or so on an hour long import for data safety than have a chance for data corruption going unnoticed by the end user.

And now that index v2 is available, yea, fast-import should write its index in that format so that a later repack can safely delta reuse if it (by rare chance) decides the current packed form is the best representation. This is especially true with "git gc --auto" triggering a repack every so often due to the incremental import packs I just metioned above.

I've just been too busy with other things to add index v2 support.

-- 
Shawn.