# [PATCH] supply '-n' to gzip to produce identical tarballs 6 messages from 2011-04-10 to 2011-04-11. Participants: Fraser Tweedale, Jakub Narebski, Junio C Hamano. Thread: https://gitlist.dev/t/27048 ## Fraser Tweedale, 2011-04-10 06:12 Subject: [PATCH] supply '-n' to gzip to produce identical tarballs Message-ID: <1302415936-7362-1-git-send-email-frase@frase.id.au> URL: https://gitlist.dev/e/1302415936-7362-1-git-send-email-frase%40frase.id.au ``` Without the '-n' ('--no-name') argument, gzip includes timestamp in output which results in different files. Important systems like FreeBSD ports and perhaps many others hash/checksum downloaded files to ensure integrity. For projects that do not release official archives, gitweb's snapshot feature would be an excellent stand-in but for the fact that the files it produces are not identical. Supply '-n' to gzip to exclude timestamp from output and produce idential output every time. Signed-off-by: Fraser Tweedale --- gitweb/gitweb.perl | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index 46186ab..2ab08da 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -186,7 +186,7 @@ our %known_snapshot_formats = ( 'type' => 'application/x-gzip', 'suffix' => '.tar.gz', 'format' => 'tar', - 'compressor' => ['gzip']}, + 'compressor' => ['gzip', '-n']}, 'tbz2' => { 'display' => 'tar.bz2', -- 1.7.4.3 ``` ## Jakub Narebski, 2011-04-10 07:38 Subject: Re: [PATCH] supply '-n' to gzip to produce identical tarballs Message-ID: URL: https://gitlist.dev/e/m362qm4kvu.fsf%40localhost.localdomain In-Reply-To: <1302415936-7362-1-git-send-email-frase@frase.id.au> ``` Fraser Tweedale writes: > Subject: [PATCH] supply '-n' to gzip to produce identical tarballs > > Without the '-n' ('--no-name') argument, gzip includes timestamp in > output which results in different files. Important systems like FreeBSD > ports and perhaps many others hash/checksum downloaded files to ensure > integrity. For projects that do not release official archives, gitweb's > snapshot feature would be an excellent stand-in but for the fact that the > files it produces are not identical. > > Supply '-n' to gzip to exclude timestamp from output and produce idential > output every time. > > Signed-off-by: Fraser Tweedale Very good description, except subject line should denote which subsystem this commit affects, i.e.: gitweb: supply '-n' to gzip to produce identical tarballs Hmmm... gzip in gitweb's 'snapshot' action gets data compressed from standard input, not from filesystem. Isn't -n / --no-name no-op then? Just asking... > --- > gitweb/gitweb.perl | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl > index 46186ab..2ab08da 100755 > --- a/gitweb/gitweb.perl > +++ b/gitweb/gitweb.perl > @@ -186,7 +186,7 @@ our %known_snapshot_formats = ( > 'type' => 'application/x-gzip', > 'suffix' => '.tar.gz', > 'format' => 'tar', > - 'compressor' => ['gzip']}, > + 'compressor' => ['gzip', '-n']}, Perhaps it would be more clear to use + 'compressor' => ['gzip', '--no-name']}, > > 'tbz2' => { > 'display' => 'tar.bz2', > -- > 1.7.4.3 > -- Jakub Narebski Poland ShadeHawk on #git ``` ## Fraser Tweedale, 2011-04-10 10:13 Subject: Re: [PATCH] supply '-n' to gzip to produce identical tarballs Message-ID: <20110410101345.GD1521@bacardi.wooloowin.frase.id.au> URL: https://gitlist.dev/e/20110410101345.GD1521%40bacardi.wooloowin.frase.id.au In-Reply-To: ``` On Sun, Apr 10, 2011 at 12:38:32AM -0700, Jakub Narebski wrote: > Fraser Tweedale writes: > > > Subject: [PATCH] supply '-n' to gzip to produce identical tarballs > > > > Without the '-n' ('--no-name') argument, gzip includes timestamp in > > output which results in different files. Important systems like FreeBSD > > ports and perhaps many others hash/checksum downloaded files to ensure > > integrity. For projects that do not release official archives, gitweb's > > snapshot feature would be an excellent stand-in but for the fact that the > > files it produces are not identical. > > > > Supply '-n' to gzip to exclude timestamp from output and produce idential > > output every time. > > > > Signed-off-by: Fraser Tweedale > > Very good description, except subject line should denote which > subsystem this commit affects, i.e.: > > gitweb: supply '-n' to gzip to produce identical tarballs > Thank you. Do I need to amend the message and resubmit the patch? (first time submitting a patch to git; I used git send-email). > Hmmm... gzip in gitweb's 'snapshot' action gets data compressed from > standard input, not from filesystem. Isn't -n / --no-name no-op then? > Just asking... > It is not no-op; I have tested to confirm this. I'm not sure whether a file name is recorded in the stdin case, or if so what it is, but the timestamp is recorded and that makes the difference. > > --- > > gitweb/gitweb.perl | 2 +- > > 1 files changed, 1 insertions(+), 1 deletions(-) > > > > diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl > > index 46186ab..2ab08da 100755 > > --- a/gitweb/gitweb.perl > > +++ b/gitweb/gitweb.perl > > @@ -186,7 +186,7 @@ our %known_snapshot_formats = ( > > 'type' => 'application/x-gzip', > > 'suffix' => '.tar.gz', > > 'format' => 'tar', > > - 'compressor' => ['gzip']}, > > + 'compressor' => ['gzip', '-n']}, > > Perhaps it would be more clear to use > > + 'compressor' => ['gzip', '--no-name']}, > > > > > 'tbz2' => { > > 'display' => 'tar.bz2', > > -- > > 1.7.4.3 > > > Definitely, if the argument is the same (or even present) on all systems. On FreeBSD and GNU both '-n' and '--no-name' are do the job, but an audit of other systems should be done to ensure they don't break. I chose '-n' as it seemed the more conservative choice. > -- > Jakub Narebski > Poland > ShadeHawk on #git ``` ## Jakub Narebski, 2011-04-10 13:55 Subject: Re: [PATCH] supply '-n' to gzip to produce identical tarballs Message-ID: <201104101555.09798.jnareb@gmail.com> URL: https://gitlist.dev/e/201104101555.09798.jnareb%40gmail.com In-Reply-To: <20110410101345.GD1521@bacardi.wooloowin.frase.id.au> ``` On Sun, 10 Apr 2011, Fraser Tweedale wrote: > On Sun, Apr 10, 2011 at 12:38:32AM -0700, Jakub Narebski wrote: > > Fraser Tweedale writes: > > > > > Subject: [PATCH] supply '-n' to gzip to produce identical tarballs > > Very good description, except subject line should denote which > > subsystem this commit affects, i.e.: > > > > gitweb: supply '-n' to gzip to produce identical tarballs > > Thank you. Do I need to amend the message and resubmit the patch? (first > time submitting a patch to git; I used git send-email). I don't think so. I guess that Junio can do such trivial amend when applying, at the time he is adding his signoff. > > Hmmm... gzip in gitweb's 'snapshot' action gets data compressed from > > standard input, not from filesystem. Isn't -n / --no-name no-op then? > > Just asking... > > It is not no-op; I have tested to confirm this. I'm not sure whether > a file name is recorded in the stdin case, or if so what it is, but the > timestamp is recorded and that makes the difference. Thanks for the clarification. For what it is worth: Acked-by: Jakub Narebski > > > - 'compressor' => ['gzip']}, > > > + 'compressor' => ['gzip', '-n']}, > > > > Perhaps it would be more clear to use > > > > + 'compressor' => ['gzip', '--no-name']}, > Definitely, if the argument is the same (or even present) on all systems. > On FreeBSD and GNU both '-n' and '--no-name' are do the job, but an audit > of other systems should be done to ensure they don't break. I chose '-n' > as it seemed the more conservative choice. So you choose '-n' because it has more chance of being widely supported, isn't it? Good enough for me. -- Jakub Narebski Poland ``` ## Junio C Hamano, 2011-04-11 19:24 Subject: Re: [PATCH] supply '-n' to gzip to produce identical tarballs Message-ID: <7vei58wq0a.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vei58wq0a.fsf%40alter.siamese.dyndns.org In-Reply-To: <201104101555.09798.jnareb@gmail.com> ``` Jakub Narebski writes: >> > Perhaps it would be more clear to use >> > >> > + 'compressor' => ['gzip', '--no-name']}, > >> Definitely, if the argument is the same (or even present) on all systems. >> On FreeBSD and GNU both '-n' and '--no-name' are do the job, but an audit >> of other systems should be done to ensure they don't break. I chose '-n' >> as it seemed the more conservative choice. > > So you choose '-n' because it has more chance of being widely supported, > isn't it? Good enough for me. Interesting. "gzip URL: https://gitlist.dev/e/20110411205934.GI1521%40bacardi.wooloowin.frase.id.au In-Reply-To: <7vei58wq0a.fsf@alter.siamese.dyndns.org> ``` On Mon, Apr 11, 2011 at 12:24:05PM -0700, Junio C Hamano wrote: > Jakub Narebski writes: > > >> > Perhaps it would be more clear to use > >> > > >> > + 'compressor' => ['gzip', '--no-name']}, > > > >> Definitely, if the argument is the same (or even present) on all systems. > >> On FreeBSD and GNU both '-n' and '--no-name' are do the job, but an audit > >> of other systems should be done to ensure they don't break. I chose '-n' > >> as it seemed the more conservative choice. > > > > So you choose '-n' because it has more chance of being widely supported, > > isn't it? Good enough for me. > > Interesting. "gzip fstat to get the timestamp, but "cat COPYING | gzip" does change its > output every time it is run. Good catch and a solution. Thanks, both. > > So I should expect a pull request sometime after 1.7.5 final from you, > with "an audit of other systems" done by others on the list noted in the > final commit message? > > Sure, no problem. Thanks, Fraser ```