Re: SHA-256 transition

From: Jeff King <peff@peff.net>
Date: Jul 1, 2022, 18:00 UTC
Message-ID: <Yr82LITalM8JX753@coredump.intra.peff.net>
In-Reply-To: <YrbNIUnftj+Ooumo@tapette.crustytoothpaste.net>

On Sat, Jun 25, 2022 at 08:53:53AM +0000, brian m. carlson wrote:

Show 8 quoted lines

> > I'm curious if you have numbers on this. I naively converted linux.git
> > to sha256 by doing "fast-export | fast-import" (the latter in a sha256
> > repo, of course, and then both repacked with "-f --window=250" to get
> > reasonable apples-to-apples packs).
> 
> I did the same thing, except I just did a regular gc and not a custom
> repack, and I created both a SHA-1 and SHA-256 repo from the same
> original.

That _might_ influence your timings a bit, just because the fast-import packs have lousy deltas. I think my linux.git was something like 6GB from fast-import, packed down to 1.5GB after "repack -f".

But I'm not sure if it would change the direction of the trend of what you were measuring, only the magnitude. We'll hash the same bytes in either case, but in the fast-import pack we'd spend more time on zlib inflating and less time on delta reconstruction. Which one is more expensive probably depends on a lot of factors, but it's entirely possible that running your test after a "repack -f" would actually show a greater change between the two cases.

Show 14 quoted lines

> Here are my results:
> 
> [sha256]
> time ~/checkouts/git/git index-pack --verify .git/objects/pack/pack-*.pack
> ~/checkouts/git/git index-pack --verify .git/objects/pack/pack-*.pack  2768.42s user 181.00s system 185% cpu 26:31.70 total
> 
> [sha1dc]
> time ~/checkouts/git/git index-pack --verify .git/objects/pack/pack-*.pack
> ~/checkouts/git/git index-pack --verify .git/objects/pack/pack-*.pack  3041.28s user 184.84s system 199% cpu 26:54.74 total
> 
> Note that in my case, I'm using an accelerated hardware-based SHA-256
> implementation (Nettle, which I will send a patch for soon).  This is a
> brand new ThinkPad X1 Carbon Gen 10 with an i7-1280P (with 20 "cores" of
> different sizes).

OK, that probably explains the difference in results we saw. Thanks for sharing your numbers. I think that's pretty "apples to apples" since we'd hope that sha256 will eventually be accelerated, but sha1dc never will be.

-Peff