From: Dave Zarzycki <zarzycki@apple.com>
Date: Tue, 07 Feb 2012 04:04:59 GMT
Subject: Re: [PATCH 5/6] Allow to use crc32 as a lighter checksum on index
Message-ID: <159A2D07-0B02-4E85-B7AA-C668FDA9F382@apple.com>
In-Reply-To: <CAJo=hJvSyhv8EUh=6ROotc3Q=zQo7vbww_ShQJP3tf1T7s889g@mail.gmail.com>

On Feb 6, 2012, at 7:17 PM, Shawn Pearce <spearce@spearce.org> wrote:

> I haven't really put a lot of thought into this. But I suspect CRC-32
> is sufficient on the index file, until it gets so big that the
> probability of a bit flip going undetected is too high due to the size
> of the file, but then we are into the "huge" index size range that has
> you trying to swap out SHA-1 for CRC-32 because SHA-1 is too slow. Uhm
> no.

CRCs are designed to be implemented in hardware and provide basic single-bit error checking for networking packets of disk blocks. With a good polynomial, they're reasonably effective at detecting a single-bit error within 8 or 16 kilobytes:

    http://www.ece.cmu.edu/~koopman/networks/dsn02/dsn02_koopman.pdf