Re: [PATCH 2/3] sha1_file: add the ability to parse objects in "pack file format"

From: Junio C Hamano <junkio@cox.net>
Date: Jul 12, 2006, 00:46 UTC
Message-ID: <7vveq31wgo.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <Pine.LNX.4.64.0607111656250.5623@g5.osdl.org>

Linus Torvalds <torvalds@osdl.org> writes:

> It's actually really part of the specs, and not just happenstance.

Show 5 quoted lines

> Well, I normally would agree with you if it was a "oh, all our zlib 
> objects seem to start with 0x78" thing, but after having dug into both the 
> zlib standard (which is actually an RFC, not just some random thing), and 
> looked at the sources, it's definitely the case that the "0x78" byte isn't 
> just an implementation detail.

Ok, I do not think we would worry about casting use of deflate + 32k windowsize in stone that much, and being able to check the size and type without inflating certainly is attractive. Validating FCHECK bits is surely a nice touch. Thanks.

Show 7 quoted lines

> Anyway, I think this following patch replaces the old 2/3 and 3/3 (it 
> still depends on the original [1/3] cleanup.
>
> (It also renames and reverses the meaning of the config file option: it's 
> now "[core] LegacyHeaders = true" for using legacy headers.)
>
> Not heavily tested, but seems ok.

I'd queue it in "pu" with reversed default and then move it to "next" later.

Show 16 quoted lines

>  static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size)
>  {
>  	int bytes = strlen(buffer) + 1;
>  	unsigned char *buf = xmalloc(1+size);
> +	unsigned long n;
>  
> -	memcpy(buf, (char *) buffer + bytes, stream->total_out - bytes);
> -	bytes = stream->total_out - bytes;
> +	n = stream->total_out - bytes;
> +	if (n > size)
> +		n = size;
> +	memcpy(buf, (char *) buffer + bytes, n);
> +	bytes = n;
>  	if (bytes < size) {
>  		stream->next_out = buf + bytes;
>  		stream->avail_out = size - bytes;

This one looks like an independent fix for a well spotted bug.

Show 12 quoted lines

>  int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *returnsha1)
>  {
>  	int size;
> @@ -1459,7 +1550,7 @@ int write_sha1_file(void *buf, unsigned 
>  	/* Set it up */
>  	memset(&stream, 0, sizeof(stream));
>  	deflateInit(&stream, zlib_compression_level);
> -	size = deflateBound(&stream, len+hdrlen);
> +	size = 8 + deflateBound(&stream, len+hdrlen);
>  	compressed = xmalloc(size);
>  
>  	/* Compress it */

I am wondring what this eight is. You would pack 7 7-bit length plus 4-bit totalling 49+4 = 53-bit length (plus 4-bit type). Is it an unwritten decision that the format would not deal with objects larger than 2^53 (which is probably fine but looks magic)?