Git network protocol

Hello,

I'm trying to implement the git protocol, and I am having a bit of an issue with the lack of information available about it (please correct me if I missed some source of information.)

I understand the basic format of the protocol, however I'm not sure what "command" can follow what. I also noticed some odd inconsistencies (or maybe I just don't see the pattern yet.) For example, a git-clone generates this traffic:

C: git-upload-pack ....
S: SHA1 HEAD....
S: SHA1 refs/heads/master
S: flush
C: want SHA1...
C: want SHA1... (it wants the same SHA1 twice!)
C: flush
C: done
S: NAK
S: the pack...

Then, when it is time to git-fetch a new commit, I get:

C: git-upload-pack ....
S: SHA1 HEAD....
S: SHA1 refs/heads/master
S: flush
C: want SHA1...
C: flush
C: have SHA1
C: done
S: ACK SHA1 continue
S: ACK SHA1 (same hash)
S: the pack...

Then, if I try to git-fetch but there is nothing new, I get:

C: git-upload-pack ....
S: SHA1 HEAD....
S: SHA1 refs/heads/master
S: flush
C: flush
<client closes connection>

So, I can _assume_ that "done" tells the server that it is time to make a pack. Why does the server use NAK during the clone operation, but ACK during fetch? Why does the server ACK the same SHA1 twice? And why does the client "want" the same SHA1 twice? It just seems odd.

I think it would be great if there was some kind of description somewhere that detailed the protocol. Also, the daemon source isn't the prettiest thing in the world.

Thanks, Josef "Jeff" Sipek.

jeffpc@josefsipek.net (Josef "Jeff" Sipek) writes:

Documentation/technical/pack-protocol.txt, and "git show 1bd8c8f0".

During the initial "SHA-1 name"/"want" exchange, the server and the client negotiate the protocol extension (the document needs to be updated at least for "multi-ack" extension).

After that, the client and the server try to determine what commits the client has that are recent ancestors of "want" commits. This exchange is done by client sending bunch of "have" to the server. The server responds "ACK" (in the original protocol) to say "I've seen enough and know a common ancestor to use". In multi-ack protocol (which is used in modern git, v0.99.9 and later), the server can respond "ACK continue" to say "I've seen enough on that branch so do not bother sending 'have's for its ancestors, but do keep sending from other branch." If the server does not feel it saw enough, it does not send either.

The client can send "flush" -- this asks the server to give NAK if more "have"s need to be sent. If there is no more "have"s to be sent, "done" is sent.

The protocol streams; if you see an ACK from server it does not usually mean it is ACKing the last 'have' client has sent.

On Sun, Aug 13, 2006 at 11:42:33PM -0700, Junio C Hamano wrote:

This is pretty much a vague example - or at least it feels vague if you don't know the protocol. After reading the source, and your description, the example makes a lot more sense.

Neat.

So, if I understand this correctly, multi_ack allows for multiple branches to be fetched using the same connection?

Yeah, makes sense. Doing things over lo does things like that - I'm not sure why I didn't realize it earlier.

Thanks, Josef "Jeff" Sipek.

jeffpc@josefsipek.net (Josef "Jeff" Sipek) writes:

The original protocol without extension already allowed it. Suppose the global history was like this:

       o-----------------------o---o---o---o---o---o---o---c---c---c
      /
 o---o---o---o---o---o---o---o---s---s---s

where the server side had 's', the client side had 'c', and both of them had 'o'. The objective is to update client with three 's' commits.

The exchange would go like this without multi-ack:

	S: SHA-1 name1 -- for the rightmost 'o' on the top branch
        S: SHA-2 name2 -- for the rightmost 's' on the bottom branch
	C: want SHA-2 -- ask for the second branch tip
	C: have SHA1 -- the rightmost 'c' on the top branch
	C: have SHA1 -- the parent of the above
	C: have SHA1 -- the parent of the above
        C: have SHA1 -- the parent of the above, rightmost 'o' on the top.
        C: ... more have from the top branch.
	S: ACK -- for the rightmost 'o' on the top branch

During this exchange, the server learns that the client and the server shares the rightmost 'o' on the top branch, but does not learn about all the 'o' commits on the bottom branch, so it ends up sending everything from the fork point to complete three 's' commits.

The multi-ack extension was invented by Johannes to improve this exchange. It changes the protocol to let the server ACK with "ACK continue". After that, the client is expected to stop traversing the parents of ACK-continue'd commit -- so it has a chance to send the rightmost 'o' on the bottom branch. When the server sees it, it again gives an ACK, and the client soon runs out have's to send and says "done". In this case, the server would have the rightmost 'o' commits on both branches to work out the minimum set of objects to complete three 's' commits that are missing from the client.

On Mon, Aug 14, 2006 at 05:59:37PM -0700, Junio C Hamano wrote:

...

Thanks. That helped a lot.

Josef "Jeff" Sipek.