[PATCH 0/3] don't let disk corruptions escape pack SHA1 checksum

From: Nicolas Pitre <nico@cam.org>
Date: Aug 29, 2008, 02:05 UTC
Message-ID: <alpine.LFD.1.10.0808282142490.1624@xanadu.home>
In-Reply-To: <alpine.LFD.1.10.0808271627540.1624@xanadu.home>

On Wed, 27 Aug 2008, Nicolas Pitre wrote:

Show 19 quoted lines

> On Wed, 27 Aug 2008, Linus Torvalds wrote:
> 
> > However, we do have this odd case of fixing up the pack after-the-fact 
> > when we receive it from somebody else (because we get a thin pack and 
> > don't know how many objects the final result will have). And that case 
> > seems to be not as safe, because it
> > 
> >  - re-reads the file to recompute the SHA1
> > 
> >    This is understandable, and it's fairly ok, but it does mean that there 
> >    is a bigger chance of the SHA1 matching if something has corrupted the 
> >    file in the meantime!
> 
> I think that can be fixed.  When reading the file back, it is possible 
> to compute 2 sha1s: one to compare with the recieved one using original 
> pack header, and the second which would be the final one.  FRom a 
> certain offset, new objects were added, so that first sha1 is validated 
> against the received one and reset, and at the end, it should correspond 
> to the sha1 of added objects that we should compute when writing them.

So here is 3 patches for this in C git (Shawn already did it for jgit).

There are 3 spots where this needs to be plugged: in pack-objects, index-pack and fast-import. I did the first two.

I don't think this is that pertinent in fast-import because:

 - we want fast-import to be fast

 - after fast-import is done, a full repack is typically done which 
   would revalidate everything (maybe fast-import should use index v2 
   with its per object CRC32 though)

But maybe I'm wrong about that. In any case, I am leaving the fast-import to someone more knowledgeable about its code than I do.

Nicolas Pitre (3):
      improve reliability of fixup_pack_header_footer()
      pack-objects: use fixup_pack_header_footer()'s validation mode
      index-pack: use fixup_pack_header_footer()'s validation mode

Nicolas