Re: [PATCH 0/4] faster SHA-1 collision detection

>
> Thanks for reeling me in, Scott!
>
> First of all, I am very happy to see that overall, everyone here is
> making an effort to find a way to speed up SHA-1dc again.
> It's so impactful!
>
>
> It really did hurt when I finally had to add SHA-1dc to Gitoxide and see
> the performance of clones plummet. And it still hurts me knowing that
> an incredible amount of CPU time is wasted doing something that we now
> know can be done much faster. At GitHub scale, this must be more than
> a blip.
>
> While it's my dream to one day have a GitHub action that uses `gix` to
> clone and safe even more power, I think Git is in a far better spot
> to achieve significant savings much sooner.
>
> On 07.10.26 20:13, Scott Chacon wrote:
> > Hey,
> >
> > On Wed, Oct 7, 2026 at 7:23 PM Junio C Hamano <gitster@pobox.com> wrote:
> >>> This series ports the approach of Sam Reis's sha1dc Rust crate [1],
> >>> which gitoxide recently switched to [2], to C.
> >>
> >> Which means license-wise the original is compatible with us, I
> >> presume, as they are "Apache2 or MIT, your choice".
> >>
> >> How can you/we be sure, with respect to the current AI policy in
> >> SubmittingPatches (which by the way was vetted by SFC lawyers), that
> >> your "AI generated" code did not "borrow" from places that gets
> >> you/us into trouble?
> >
> > It's a good question. I actually just submitted a proposed update to
> > that policy based on SFC's updated guidelines, but either way, I
> > learned about this from Sam and have talked to him about the port and
> > he seemed excited about it. I can triple check, but I'm fairly
> > confident that he's fine with this and I am fine signing off on it
> > under the terms of the DCO language.
> >
> > Of course, he in turn used AI tooling to produce _his_ library, but
> > within the guidelines of the updated SFC guidelines. Johannes's
> > alternative series is the original Rust code of Sam that my agent
> > looked at to produce this (in addition to his blog post explaining
> > it), so I'm not sure how that might be materially different.
> >
> >>> The end result hashes roughly 2.7x faster on the Xeon and 2.85x faster
> >>> on the M5 Max. Single-threaded index-pack of git.git goes from 24.3s to
> >>> 12.7s on the Xeon, and from 16.1s to 8.7s on the M5 Max.
> >>>
> >>> Hashing throughput on the Xeon, in MiB/s:
> >>>
> >>>                                  16KiB    1MiB   vs OpenSSL
> >>>    OpenSSL SHA-1 (no detection)   1234    1129      1.00x
> >>>    sha1dc/ (today)                 435     450      2.67x
> >>>    shani+avx2 (default here)      1002     901      1.24x
> >>>    shani+sse2                     1075    1008      1.13x
> >>>    portable+avx2                   553     654      1.96x
> >>>    portable+sse2                   603     681      1.84x
> >>>    portable                        466     565      2.29x
> >>>
> >>> In other words, currently collision detection costs about 1.5–2.5x on
> >>> top of the hashing itself today, but only about 0.2x with the series.
> >>
> >> Thanks for these numbers.
> >
> > It would have been better had I provided the same relative scale (it
> > should be 1.5-2.5x vs 1.2x, but whatever, you probably get it. It's
> > 20% overhead here vs 50%-150% overhead previously).
> >
> >>> [1] https://sam.dev/blog/faster-sha1-collision-detection
> >>> [2] https://github.com/GitoxideLabs/gitoxide/pull/3008
> >>
> >> And the pointers to the original sources.
> >
> > CC'ing Sam (sha1dc rust guy) and Sebastian (Gitoxide) on this, just in
> > case they have an opinion but I'm pretty sure they would be more than
> > happy for this to be integrated.
> >
> > Scott
>