From: George Spelvin <linux@horizon.com>
Date: Tue, 04 Aug 2009 05:01:38 GMT
Subject: Re: x86 SHA1: Faster than OpenSSL
Message-ID: <20090804050138.13256.qmail@science.horizon.com>
In-Reply-To: <9e4733910908032007td74ef9fp669d0d958df67c1@mail.gmail.com>

> Would there happen to be a SHA1 implementation around that can compute
> the SHA1 without first decompressing the data? Databases gain a lot of
> speed by using special algorithms that can directly operate on the
> compressed data.

I can't imagine how.  In general, this requires that the compression
be carefully designed to be compatible with the algorithms, and SHA1
is specifically designed to depend on every bit of the input in
an un-analyzable way.

Also, git normally avoids hashing objects that it doesn't need
uncompressed for some other reason.  git-fsck is a notable exception,
but I think the idea of creating special optimized code paths for that
interferes with its reliability and robustness goals.