threads / discuss / 48740

t5562: gzip -k is not portable

Subject: t5562: gzip -k is not portable

## tl;dr

10 messages between Jun 19, 2018 and Jun 20, 2018.

replies: 9people: 5as markdown or json

Torsten Bögershausen· Jun 19, 2018, 17:25 UTC · lore

Hej Max,

t5562 fails here under MacOS: "gzip -k" is not portable.

The following works (there may be better solutions, I didn't dig into the test code)

diff --git a/t/t5562-http-backend-content-length.sh 
b/t/t5562-http-backend-content-length.sh
index 8040d80e04..7befe3885c 100755
--- a/t/t5562-http-backend-content-length.sh
+++ b/t/t5562-http-backend-content-length.sh
@@ -61,9 +61,13 @@ test_expect_success 'setup' '
  '

  test_expect_success GZIP 'setup, compression related' '
-       gzip -k fetch_body &&
+       cp fetch_body f &&
+       gzip fetch_body &&
+       mv f fetch_body &&
         test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
-       gzip -k push_body &&
+       cp push_body f &&
+       gzip push_body &&
+       mv f push_body &&
         test_copy_bytes 10 <push_body.gz >push_body.gz.trunc

Junio C Hamano· Jun 19, 2018, 18:06 UTC · re: Torsten Bögershausen · lore

Re: t5562: gzip -k is not portable

Torsten Bögershausen <tboegi@web.de> writes:

> Hej Max,
>
> t5562 fails here under MacOS:
> "gzip -k"  is not portable.

Sigh.  Perhaps -c would help.  Or do BSD implementations also lack -c?

 t/t5562-http-backend-content-length.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
index 8040d80e04..98b6543827 100755
--- a/t/t5562-http-backend-content-length.sh
+++ b/t/t5562-http-backend-content-length.sh
@@ -61,9 +61,9 @@ test_expect_success 'setup' '
 '
 
 test_expect_success GZIP 'setup, compression related' '
-	gzip -k fetch_body &&
+	gzip -c fetch_body >fetch_body.gz &&
 	test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
-	gzip -k push_body &&
+	gzip -c push_body >push_body.gz &&
 	test_copy_bytes 10 <push_body.gz >push_body.gz.trunc
 '

Eric Sunshine· Jun 19, 2018, 18:22 UTC · re: Junio C Hamano · lore

Re: t5562: gzip -k is not portable

On Tue, Jun 19, 2018 at 2:06 PM Junio C Hamano <gitster@pobox.com> wrote:

> Torsten Bögershausen <tboegi@web.de> writes:
> > t5562 fails here under MacOS:
> > "gzip -k"  is not portable.

Very odd. Stock /usr/bin/gzip on my MacOS 10.12.6 _does_ recognize -k, and the test does pass.

> Sigh.  Perhaps -c would help.  Or do BSD implementations also lack -c?

MacOS and BSD do support -c, so this solution would also work (and is "cleaner" the the other proposal).

Show 10 quoted lines

> diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
> @@ -61,9 +61,9 @@ test_expect_success 'setup' '
>  test_expect_success GZIP 'setup, compression related' '
> -       gzip -k fetch_body &&
> +       gzip -c fetch_body >fetch_body.gz &&
>         test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
> -       gzip -k push_body &&
> +       gzip -c push_body >push_body.gz &&
>         test_copy_bytes 10 <push_body.gz >push_body.gz.trunc
>  '

Torsten Bögershausen· Jun 19, 2018, 20:40 UTC · re: Eric Sunshine · lore

Re: t5562: gzip -k is not portable

On 06/19/2018 08:22 PM, Eric Sunshine wrote:

Show 6 quoted lines

> On Tue, Jun 19, 2018 at 2:06 PM Junio C Hamano <gitster@pobox.com> wrote:
>> Torsten Bögershausen <tboegi@web.de> writes:
>>> t5562 fails here under MacOS:
>>> "gzip -k"  is not portable.
> Very odd. Stock /usr/bin/gzip on my MacOS 10.12.6 _does_ recognize -k,
> and the test does pass.

This is the test box running Mac OS X 10.6 speaking. The -c seems to need even -f. But this doesn't work either:

gzip 1.3.12 Copyright (C) 2007 Free Software Foundation, Inc. Copyright (C) 1993 Jean-loup Gailly. This is free software. You may redistribute copies of it under the terms of the GNU General Public License <http://www.gnu.org/licenses/gpl.html>. There is NO WARRANTY, to the extent permitted by law.

Written by Jean-loup Gailly.
prerequisite GZIP ok
expecting success:
     gzip -f -c fetch_body >fetch_body.gz &&
     test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
     gzip -f -c push_body >push_body,gz &&
     test_copy_bytes 10 <push_body.gz >push_body.gz.trunc

./test-lib.sh: line 632: push_body.gz: No such file or directory not ok 2 - setup, compression related # # gzip -f -c fetch_body >fetch_body.gz && # test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc && # gzip -f -c push_body >push_body,gz && # test_copy_bytes 10 <push_body.gz >push_body.gz.trunc #

Show 14 quoted lines

>> Sigh.  Perhaps -c would help.  Or do BSD implementations also lack -c?
> MacOS and BSD do support -c, so this solution would also work (and is
> "cleaner" the the other proposal).
>
>> diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
>> @@ -61,9 +61,9 @@ test_expect_success 'setup' '
>>   test_expect_success GZIP 'setup, compression related' '
>> -       gzip -k fetch_body &&
>> +       gzip -c fetch_body >fetch_body.gz &&
>>          test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
>> -       gzip -k push_body &&
>> +       gzip -c push_body >push_body.gz &&
>>          test_copy_bytes 10 <push_body.gz >push_body.gz.trunc
>>   '

Jeff King· Jun 19, 2018, 20:53 UTC · re: Torsten Bögershausen · lore

Re: t5562: gzip -k is not portable

On Tue, Jun 19, 2018 at 10:40:16PM +0200, Torsten Bögershausen wrote:

Show 20 quoted lines

> 
> 
> On 06/19/2018 08:22 PM, Eric Sunshine wrote:
> > On Tue, Jun 19, 2018 at 2:06 PM Junio C Hamano <gitster@pobox.com> wrote:
> > > Torsten Bögershausen <tboegi@web.de> writes:
> > > > t5562 fails here under MacOS:
> > > > "gzip -k"  is not portable.
> > Very odd. Stock /usr/bin/gzip on my MacOS 10.12.6 _does_ recognize -k,
> > and the test does pass.
> 
> This is the test box running Mac OS X 10.6 speaking.
> The -c seems to need even -f.
> But this doesn't work either:
> 
> gzip 1.3.12
> Copyright (C) 2007 Free Software Foundation, Inc.
> Copyright (C) 1993 Jean-loup Gailly.
> This is free software.  You may redistribute copies of it under the terms of
> the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
> There is NO WARRANTY, to the extent permitted by law.

Ah, that's it. "-k" came about in gzip v1.6. That's 5 years old, but obviously some people are still running it.

"-c" goes back quite a while and should be safe, I think.

Show 7 quoted lines

> expecting success:
>     gzip -f -c fetch_body >fetch_body.gz &&
>     test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
>     gzip -f -c push_body >push_body,gz &&
>     test_copy_bytes 10 <push_body.gz >push_body.gz.trunc
> 
> ./test-lib.sh: line 632: push_body.gz: No such file or directory

Typo in the ">" redirect (comma instead of period)?

-Peff

Torsten Bögershausen· Jun 20, 2018, 06:13 UTC · re: Jeff King · lore

Re: t5562: gzip -k is not portable

On Tue, Jun 19, 2018 at 04:53:10PM -0400, Jeff King wrote:

Show 39 quoted lines

> On Tue, Jun 19, 2018 at 10:40:16PM +0200, Torsten Bögershausen wrote:
> 
> > 
> > 
> > On 06/19/2018 08:22 PM, Eric Sunshine wrote:
> > > On Tue, Jun 19, 2018 at 2:06 PM Junio C Hamano <gitster@pobox.com> wrote:
> > > > Torsten Bögershausen <tboegi@web.de> writes:
> > > > > t5562 fails here under MacOS:
> > > > > "gzip -k"  is not portable.
> > > Very odd. Stock /usr/bin/gzip on my MacOS 10.12.6 _does_ recognize -k,
> > > and the test does pass.
> > 
> > This is the test box running Mac OS X 10.6 speaking.
> > The -c seems to need even -f.
> > But this doesn't work either:
> > 
> > gzip 1.3.12
> > Copyright (C) 2007 Free Software Foundation, Inc.
> > Copyright (C) 1993 Jean-loup Gailly.
> > This is free software.  You may redistribute copies of it under the terms of
> > the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
> > There is NO WARRANTY, to the extent permitted by law.
> 
> Ah, that's it. "-k" came about in gzip v1.6. That's 5 years old, but
> obviously some people are still running it.
> 
> "-c" goes back quite a while and should be safe, I think.
> 
> > expecting success:
> >     gzip -f -c fetch_body >fetch_body.gz &&
> >     test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
> >     gzip -f -c push_body >push_body,gz &&
> >     test_copy_bytes 10 <push_body.gz >push_body.gz.trunc
> > 
> > ./test-lib.sh: line 632: push_body.gz: No such file or directory
> 
> Typo in the ">" redirect (comma instead of period)?
> 
> -Peff

Good eyes, thanks. The "-f -c" combo works:

-       gzip -k fetch_body &&
+       gzip -f -c fetch_body >fetch_body.gz &&
        test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
-       gzip -k push_body &&
+       gzip -f -c push_body >push_body.gz &&

Jeff King· Jun 20, 2018, 12:40 UTC · re: Torsten Bögershausen · lore

Re: t5562: gzip -k is not portable

On Wed, Jun 20, 2018 at 08:13:06AM +0200, Torsten Bögershausen wrote:

Show 8 quoted lines

> Good eyes, thanks.
> The "-f -c" combo works:
> 
> -       gzip -k fetch_body &&
> +       gzip -f -c fetch_body >fetch_body.gz &&
>         test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
> -       gzip -k push_body &&
> +       gzip -f -c push_body >push_body.gz &&

Do we still need "-f"? With "-c" gzip is only writing to stdout, so we should not need to force anything.

-Peff

Torsten Bögershausen· Jun 20, 2018, 15:41 UTC · re: Jeff King · lore

Re: t5562: gzip -k is not portable

On Wed, Jun 20, 2018 at 08:40:06AM -0400, Jeff King wrote:

Show 15 quoted lines

> On Wed, Jun 20, 2018 at 08:13:06AM +0200, Torsten Bögershausen wrote:
> 
> > Good eyes, thanks.
> > The "-f -c" combo works:
> > 
> > -       gzip -k fetch_body &&
> > +       gzip -f -c fetch_body >fetch_body.gz &&
> >         test_copy_bytes 10 <fetch_body.gz >fetch_body.gz.trunc &&
> > -       gzip -k push_body &&
> > +       gzip -f -c push_body >push_body.gz &&
> 
> Do we still need "-f"?  With "-c" gzip is only writing to stdout, so we
> should not need to force anything.
> 
> -Peff

You are rigth, -f is not needed.

I was confusing stdout with terminal :-( Most often stdout is the terminal, but not here of course.

Rodrigo Campos· Jun 19, 2018, 20:53 UTC · re: Torsten Bögershausen · lore

Re: t5562: gzip -k is not portable

On Tue, Jun 19, 2018 at 10:40:16PM +0200, Torsten Bögershausen wrote:

Show 11 quoted lines

> 
> 
> On 06/19/2018 08:22 PM, Eric Sunshine wrote:
> > On Tue, Jun 19, 2018 at 2:06 PM Junio C Hamano <gitster@pobox.com> wrote:
> > > Torsten Bögershausen <tboegi@web.de> writes:
> > > > t5562 fails here under MacOS:
> > > > "gzip -k"  is not portable.
> > Very odd. Stock /usr/bin/gzip on my MacOS 10.12.6 _does_ recognize -k,
> > and the test does pass.
> 
> gzip 1.3.12

Oh, this seems to be the issue. As I've said in my prvious email, the first version to include this was 1.6 IIRC.

The commit adding it is here[1]. But yeah, it was not added 20 years ago, and it seems there are systems with such old versions of gzip (although can probably update? Don't know much about mac) :-(

[1]: http://git.savannah.gnu.org/cgit/gzip.git/commit/?id=0192f02e26ac9fa0a27ed177263ee3ea73d5e95c

Rodrigo Campos· Jun 19, 2018, 20:48 UTC · re: Torsten Bögershausen · lore

Re: t5562: gzip -k is not portable

On Tue, Jun 19, 2018 at 07:25:15PM +0200, Torsten Bögershausen wrote:

> Hej Max,
> 
> t5562 fails here under MacOS:
> "gzip -k"  is not portable.

What do you mean with is not portable?

I wrote the patch for gzip[1]. That was in 2013, and is included since version 1.6 IIUC:

	$ git tag --contains 0192f02e26ac9fa0a27ed177263ee3ea73d5e95c
	v1.6
	v1.7
	v1.8
	v1.9

But that maybe is too new? Or do you mean something else?

[1]: http://git.savannah.gnu.org/cgit/gzip.git/commit/?id=0192f02e26ac9fa0a27ed177263ee3ea73d5e95c