Re: [PATCH 2/2] archive: avoid spawning `gzip`

From: René Scharfe <l.s.r@web.de>
Date: May 1, 2019, 17:45 UTC
Message-ID: <05e97774-5dd1-7224-940f-e50558118d93@web.de>
In-Reply-To: <nycvar.QRO.7.76.6.1904291720120.45@tvgsbejvaqbjf.bet>

Hello Dscho,

Am 29.04.19 um 23:25 schrieb Johannes Schindelin:

Show 7 quoted lines

> On Sat, 27 Apr 2019, René Scharfe wrote:
>> The simplest solution is of course to not touch the archive code.
>
> We could do that, of course, and we could avoid adding a new command that
> we have to support for eternity by introducing a command mode for `git
> archive` instead (think: `git archive --gzip -9`), and marking that
> command mode clearly as an internal implementation detail.

adding gzip as the 142nd git command and 18th pure helper *would* be a bit embarrassing, in particular for a command that's not directly related to version control and readily available on all platforms. Exposing it as a (hidden?) archive sub-command might be better.

> But since the performance is still not quite on par with `gzip`, I would
> actually rather not, and really, just punt on that one, stating that
> people interested in higher performance should use `pigz`.

Here are my performance numbers for generating .tar.gz files again:

master, using gzip(1):
  Time (mean ± σ):     16.683 s ±  0.451 s    [User: 20.230 s, System: 0.375 s]
  Range (min … max):   16.308 s … 17.852 s    10 runs

using zlib sequentially:
  Time (mean ± σ):     19.898 s ±  0.228 s    [User: 19.825 s, System: 0.073 s]
  Range (min … max):   19.627 s … 20.355 s    10 runs

using zlib asynchronously:
  Time (mean ± σ):     17.300 s ±  0.198 s    [User: 20.825 s, System: 0.356 s]
  Range (min … max):   17.042 s … 17.638 s    10 runs

using a gzip-lookalike:
  Time (mean ± σ):     17.256 s ±  0.299 s    [User: 20.380 s, System: 0.294 s]
  Range (min … max):   16.940 s … 17.804 s    10 runs

The last two have comparable system time, ca. 1% more user time and ca. 5% longer duration. The second one has much better system time and 2% less user time and 19% longer duration. Hmm.

> And who knows, maybe nobody will complain at all about the performance?

Probably.  And popular tarballs would be cached anyway, I guess.

So I'll send comments on your series later this week.

René