Re: [PATCH 2/3] sha1_file: add the ability to parse objects in "pack file format"

From: Linus Torvalds <torvalds@osdl.org>
Date: Jul 11, 2006, 22:09 UTC
Message-ID: <Pine.LNX.4.64.0607111449190.5623@g5.osdl.org>
In-Reply-To: <44B4172B.3070503@stephan-feder.de>

On Tue, 11 Jul 2006, sf wrote:

> 
> And in the traditional format type and length are compressed whereas in
> the pack-file format they are not.

Ahh. Yes.

Show 5 quoted lines

> > This should probably be applied to the main tree asap if we think
> > this is at all a worthwhile exercise. But somebody should verify that I 
> > got the format right first!
> 
> Sorry but see above.

Good catch, thanks indeed.

Doing that for unpacked objects would in fact make a lot of things much simpler, so it would be good to do. The _bad_ part is that this also makes it a lot harder to see the difference between a "binary header" and a "compressed ASCII header". The two are not "obviously different" any more.

The common byte sequence for a compressed stream is

	78 9c ...

where the first byte is the CMF byte (compression info and method).

But it's not the only possible such sequence according to the zlib format.

(The 16-bit hex number in MSB format, ie 0x789c above, is defined to have a built-in checksum, so that it must be a multiple of 31 according to the standard: 0x789c = 996 * 31).

So if we have a uncompressed header, we'd need to add a separate 2-byte fingerprint to it _before_ the real header that isn't divisible by 31, and use that as the thing to test.

Ho humm. I'll see what I can come up with.

		Linus