Re: [PATCH v3 0/5] Avoid spawning gzip in git archive

From: René Scharfe <l.s.r@web.de>
Date: Jun 14, 2022, 20:05 UTC
Message-ID: <0aa5c101-06bf-325c-efbc-6b4ef38616c5@web.de>
In-Reply-To: <nycvar.QRO.7.76.6.2206141043150.353@tvgsbejvaqbjf.bet>

Am 14.06.22 um 13:28 schrieb Johannes Schindelin:

Show 22 quoted lines

>
> By the way, the main reason why I did not work more is that in
> http://madler.net/pipermail/zlib-devel_madler.net/2019-December/003308.html,
> Mark Adler (the zlib maintainer) announced that...
>
>> [...] There are many well-tested performance improvements in zlib
>> waiting in the wings that will be incorporated over the next several
>> months. [...]
>
> This was in December 2019. And now it's June 2022 and I kind of wonder
> whether those promised improvements will still come.
>
> In the meantime, however, a viable alternative seems to have cropped up:
> https://github.com/zlib-ng/zlib-ng. Essentially, it looks as if it is what
> zlib should have become after above-quoted announcement.
>
> In particular the CPU intrinsics support (think MMX, SSE2/3, etc) seem to
> be very interesting and I would not be completely surprised if building
> Git with your patches and linking against zlib-ng would paint a very
> favorable picture not only in terms of CPU time but also in terms of
> wallclock time. Sadly, I have not been able to set aside time to look into
> that angle, but maybe I can peak your interest?

I was unable to preload zlib-ng using DYLD_INSERT_LIBRARIES on macOS 12.4 so far. The included demo proggy looks impressive, though:

$ hyperfine -w3 -L gzip gzip,../zlib-ng/minigzip "git -C ../linux archive --format=tar HEAD | {gzip} -c"
Benchmark #1: git -C ../linux archive --format=tar HEAD | gzip -c
  Time (mean ± σ):     20.424 s ±  0.006 s    [User: 23.964 s, System: 0.432 s]
  Range (min … max):   20.414 s … 20.434 s    10 runs

Benchmark #2: git -C ../linux archive --format=tar HEAD | ../zlib-ng/minigzip -c
  Time (mean ± σ):     12.158 s ±  0.006 s    [User: 13.908 s, System: 0.376 s]
  Range (min … max):   12.145 s … 12.166 s    10 runs

Summary
  'git -C ../linux archive --format=tar HEAD | ../zlib-ng/minigzip -c' ran
    1.68 ± 0.00 times faster than 'git -C ../linux archive --format=tar HEAD | gzip -c'