Re: Request for detailed documentation of git pack protocol

>>>> It is a bit pity that git protocol was not created with
>>>> extendability (like capabilities) in mind...
>>> 
>>> Yes, no doubt.  There are many things I would have done differently,
>>> given that I now have 20/20 hindsight vision into the past's future.
>>> 
>>> :-)
>>> 
>>> The protocol (mostly) works fine as-is.  Its widely distributed in
>>> terms of clients using it on a daily basis.  Its likely to continue
>>> to serve our needs well into the future.  So, it is what it is.
>> 
>> I do wonder if existing Internet Standard (in the meaning of RFC) 
>> protocols also have such kludges and hacks...

> 
> I'm sure they have some... oddities.  But perhaps not as bad as git.
> 
> We have a history of not leaving ourselves room for future expansion,
> and then needing to find a backdoor in the canonical implementation
> parser in order to make it work.
> 
> In the protocol suite, its been the strlen() < pktlen trick which
> has generally worked.  Oh, and also sticking stuff after a fixed
> length record, where we didn't care.

> 
> Oh, and send-pack/receive-pack protocol now has ".have" refs, which
> work for C Git because the send-pack client was always calling
> check_ref_format() on each thing sent by the server, and ".have"
> isn't a valid ref name.  Why the hell the send-pack client was doing
> that, I have no idea.  But, when the ref failed it was a silent
> failure, so we were able to use ".have" for some new capability.
> 
> It also broke JGit, which wasn't doing this seemingly pointless
> check_ref_format() and silently fail business.  Oh, and IIRC,
> GitHub may have been burned around the same time somehow.

> 
> In packed-refs, Junio had a hard time adding the "peeled-refs"
> support, because the first version of the parser was so strict.
> But again, somehow he managed to find a backdoor in the old parser,
> and that backdoor is why that file looks the way it does today.

> 
> In the loose object format, when we added new-style loose objects
> we found a backdoor in the way libz deflate formats the first 2
> bytes of the file... and encoded something that shouldn't appear
> there to signal it was a new "pack style" loose object.
> 
> Pack index v2 uses a hole where old clients would barf on the
> '\377t0c' followed by the version '2' not being monotonically
> increasing.

>> P.S. By the way, is pkt-line format original invention, or was it 
>> 'borrowed' from some other standard or protocol?
> 
> No clue.  I find it f'king odd that the length is in hex.  There
> isn't much value to the protocol being human readable.  The PACK
> part of the stream sure as hell ain't.  You aren't going to type
> out a sequence of "have" lines against the remote, like you could
> with say an HTTP GET.  *shrug*