Re: SHA-accelerated Git

From: Jeffrey Walton <noloader@gmail.com>
Date: Jun 2, 2019, 11:34 UTC
Message-ID: <CAH8yC8n8XN1wKiF4FgTm6xfnRHUDi+1=iuWXE8F91P=7mLNsLg@mail.gmail.com>
In-Reply-To: <20190602132150.4c1b320f@naga.suse.cz>

On Sun, Jun 2, 2019 at 7:21 AM Michal Suchánek <msuchanek@suse.de> wrote:

Show 16 quoted lines

>
> Hello,
>
> On Sun, 2 Jun 2019 06:43:07 -0400
> Jeffrey Walton <noloader@gmail.com> wrote:
>
> > Hi Everyone,
> >
> > I have a lot of experience with cutting in SHA acceleration. I have no
> > experience with Git.
>
> sha1 is dead. Git is planning to move away from it. It will not happen
> overnight but it probably does not make sense to accelerate it at this
> point.
>
> https://github.com/git/git/blob/next/Documentation/technical/hash-function-transition.txt

SHA-1 has been dead for years. That has not stopped Git yet :)

My guess is, Git will have to carry around a SHA-1 implementation for a decade or so for interop purposes.

Show 31 quoted lines

> > If someone would setup a fork and provide a call like:
> >
> >     // https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/cpufeature.h
> >     // Or whatever the project prefers
> >     has_x86_sha = cpu_feature_enabled(X86_FEATURE_SHA_NI);
> >
> >     if (has_x86_sha)
> >     {
> >         sha1_hash_block(state, ptr, size);
> >     }
> >
> > And provide a stub:
> >
> >     sha1_hash_block(uint32_t* state, uint8_t* ptr, size_t size)
> >     {
> >     }
> >
> > Then I would be happy to fill in the pieces.
> >
> > I can also help with ARMv8.
> >
> > In fact, if you want to use Andy Polyakov's asm from Cryptogams
> > (https://www.openssl.org/~appro/cryptogams/), then I can help with
> > that, too. Andy's skills are renowned and his code is used in the
> > kernel.
>
> And buggy. I am not referring to the sha1 implementation in particular
> but to the hand-crafted assembly in general.
>
> Do you have some performance data that shows significant improvement of
> common tasks that makes maintaining this hackery worthwhile?

Well, considering Git carries along an SHA-1 implementation with undefined behavior, anything would be an improvement since it can't pass acceptance testing out of the box. And Git does that for a 0.01 to 0.03 cpb increase in spperformance.

For SHA-1 intrinsics on x86, performance increases to about 1.7 cpb. For SHA-256 performance increases to 3.8 cpb. On ARMv8, performance will increase to about 2.0 cpb on A-53 cpu's, and about 1.5 to 1.0 cpb on A-57 cpus.

Comparing against a standard C/C++ implementation, Git will see a 3x to 6x speedup depending on the platform and algorithm.

Jeff