Re: [PATCH v2 4/4] upload-archive: use start_command instead of fork

From: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Date: Aug 2, 2011, 16:46 UTC
Message-ID: <4E3829DC.8070802@lsrfire.ath.cx>
In-Reply-To: <20110802040004.GC17494@sigill.intra.peff.net>

Am 02.08.2011 06:00, schrieb Jeff King:

Show 35 quoted lines

> On Mon, Aug 01, 2011 at 11:52:43PM +0200, René Scharfe wrote:
> 
>> Am 01.08.2011 23:20, schrieb Johannes Sixt:
>>> Am 01.08.2011 22:48, schrieb René Scharfe:
>>>> So git archive gives the right results when writing to a pipe, but
>>>> always the same wrong result when writing directly to a file.
>>>
>>> This could indeed be a CRLF issue. archive-tar.c runs gzip to let it
>>> write to the original fd 1 (stdout). gzip is an MSYS program, and MSYS
>>> is "clever" and sets up the channel in text mode (CRLF conversion) if it
>>> is a regular file, but in binary mode if it is a pipe.
>>>
>>> Without the gzip filter, git-archive writes to stdout itself. Since we
>>> have set up all our channels in binary mode, we do not suffer from the
>>> same problem for plain tar format.
>>>
>>> So, I don't think we can do a lot about it, short of patching MSYS again...
>>
>> Or we could pipe the output through us, i.e. attach a builtin version of
>> cat at the output end of the called command.  Only on Windows, of
>> course.  Better ugly and limping then wrong, right?
> 
> Yeah, that would work. But I am confused. If what Johannes says is true,
> isn't MSYS gzip totally broken for:
> 
>   # works
>   echo foo | gzip -c | cat >foo.gz
> 
>   # broken; introduces CR
>   echo foo | gzip -c >foo.gz
> 
> ? (The "works" and "broken" there are my guesses; I don't have a Windows
> box to test on). IOW, is it simply gzip that is broken, and any fix we
> do is simply working around a bug in gzip? And therefore the right
> solution is for MSYS people to fix gzip?

"foo" may be too short to trigger the issue as the small resulting gz file has a low probability of containing LFs.

The output of gzip is not simply always mangled, though (taken from my earlier email, all three are working):

	$ git archive v1.7.6 | gzip -cn | md5sum
	a0ca1c873a533a5fcd41d248fb325a5b *-

	$ git archive --format=tar.gz v1.7.6 | md5sum
	a0ca1c873a533a5fcd41d248fb325a5b *-

	$ git archive v1.7.6 | gzip -cn >a.tgz && md5sum <a.tgz
	a0ca1c873a533a5fcd41d248fb325a5b *-

It's only broken if we call it from git archive:

	$ git archive --format=tar.gz v1.7.6 >a.tgz && md5sum <a.tgz
	30886283af1aed05ae6a36fc5aeda077 *-

	$ git archive -o a.tgz v1.7.6 && md5sum <a.tgz
	30886283af1aed05ae6a36fc5aeda077 *-

But not if we stuff the result into a pipe instead of a file:

	$ git archive --format=tar.gz v1.7.6 | cat >a.tgz && md5sum <a.tgz
	a0ca1c873a533a5fcd41d248fb325a5b *-

It _is_ confusing.

René