Re: x86 SHA1: Faster than OpenSSL

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Aug 6, 2009, 01:18 UTC
Message-ID: <alpine.LFD.2.01.0908051800030.3390@localhost.localdomain>
In-Reply-To: <alpine.LFD.2.01.0908051545000.3390@localhost.localdomain>

On Wed, 5 Aug 2009, Linus Torvalds wrote:

Show 20 quoted lines

> 
> Timings for "git fsck --full" on the git directory:
> 
>  - Mozilla SHA1 portable C-code (sucky sucky): MOZILLA_SHA1=1
> 
> 	real	0m38.194s
> 	user	0m37.838s
> 	sys	0m0.356s
> 
>  - This code ("half-portable C code"): BLK_SHA1=1
> 
> 	real	0m28.120s
> 	user	0m27.930s
> 	sys	0m0.192s
> 
>  - OpenSSL assembler code:
> 
> 	real	0m26.327s
> 	user	0m26.194s
> 	sys	0m0.136s

Ok, I installed the 32-bit libraries too, to see what it looks like for that case. As expected, the compiler is not able to do a great job due to it being somewhat register starved, but on the other hand, the old Mozilla code did even worse, so..

 - Mozilla SHA:

	real	0m47.063s
	user	0m46.815s
	sys	0m0.252s

 - BLK_SHA1=1

	real	0m34.705s
	user	0m34.394s
	sys	0m0.312s

 - OPENSSL:

	real	0m29.754s
	user	0m29.446s
	sys	0m0.288s

so the tuned asm from OpenSSL does kick ass, but the C code version isn't _that_ far away. It's quite a reasonable alternative if you don't have the OpenSSL libraries installed, for example.

I note that MINGW does NO_OPENSSL by default, for example, and maybe the MINGW people want to test the patch out and enable BLK_SHA1 rather than the original Mozilla one.

But while looking at 32-bit issues, I noticed that I really should also cast 'len' when shifting it. Otherwise the thing is limited to fairly small areas (28 bits - 256MB). This is not just a 32-bit problem ("int" is a signed 32-bit thing even in a 64-bit build), but I only noticed it when looking at 32-bit issues.

So here's an incremental patch to fix that.

		Linus

---
 block-sha1/sha1.c |    4 ++--
 block-sha1/sha1.h |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/block-sha1/sha1.c b/block-sha1/sha1.c
index 8fd90b0..eef32f7 100644
--- a/block-sha1/sha1.c
+++ b/block-sha1/sha1.c
@@ -27,11 +27,11 @@ void blk_SHA1_Init(blk_SHA_CTX *ctx)
 }
 
 
-void blk_SHA1_Update(blk_SHA_CTX *ctx, const void *data, int len)
+void blk_SHA1_Update(blk_SHA_CTX *ctx, const void *data, unsigned long len)
 {
 	int lenW = ctx->lenW;
 
-	ctx->size += len << 3;
+	ctx->size += (unsigned long long) len << 3;
 
 	/* Read the data into W and process blocks as they get full
 	 */
diff --git a/block-sha1/sha1.h b/block-sha1/sha1.h
index dbc719f..7be2d93 100644
--- a/block-sha1/sha1.h
+++ b/block-sha1/sha1.h
@@ -12,7 +12,7 @@ typedef struct {
 } blk_SHA_CTX;
 
 void blk_SHA1_Init(blk_SHA_CTX *ctx);
-void blk_SHA1_Update(blk_SHA_CTX *ctx, const void *dataIn, int len);
+void blk_SHA1_Update(blk_SHA_CTX *ctx, const void *dataIn, unsigned long len);
 void blk_SHA1_Final(unsigned char hashout[20], blk_SHA_CTX *ctx);
 
 #define git_SHA_CTX	blk_SHA_CTX