# Git network protocol 5 messages from 2006-08-14 to 2006-08-15. Participants: Josef "Jeff" Sipek, Junio C Hamano. Thread: https://gitlist.dev/t/5239 ## Josef "Jeff" Sipek, 2006-08-14 06:21 Subject: Git network protocol Message-ID: <20060814062117.GC10476@josefsipek.net> URL: https://gitlist.dev/e/20060814062117.GC10476%40josefsipek.net ``` Hello, I'm trying to implement the git protocol, and I am having a bit of an issue with the lack of information available about it (please correct me if I missed some source of information.) I understand the basic format of the protocol, however I'm not sure what "command" can follow what. I also noticed some odd inconsistencies (or maybe I just don't see the pattern yet.) For example, a git-clone generates this traffic: C: git-upload-pack .... S: SHA1 HEAD.... S: SHA1 refs/heads/master S: flush C: want SHA1... C: want SHA1... (it wants the same SHA1 twice!) C: flush C: done S: NAK S: the pack... Then, when it is time to git-fetch a new commit, I get: C: git-upload-pack .... S: SHA1 HEAD.... S: SHA1 refs/heads/master S: flush C: want SHA1... C: flush C: have SHA1 C: done S: ACK SHA1 continue S: ACK SHA1 (same hash) S: the pack... Then, if I try to git-fetch but there is nothing new, I get: C: git-upload-pack .... S: SHA1 HEAD.... S: SHA1 refs/heads/master S: flush C: flush So, I can _assume_ that "done" tells the server that it is time to make a pack. Why does the server use NAK during the clone operation, but ACK during fetch? Why does the server ACK the same SHA1 twice? And why does the client "want" the same SHA1 twice? It just seems odd. I think it would be great if there was some kind of description somewhere that detailed the protocol. Also, the daemon source isn't the prettiest thing in the world. Thanks, Josef "Jeff" Sipek. -- Reality is merely an illusion, albeit a very persistent one. - Albert Einstein ``` ## Junio C Hamano, 2006-08-14 06:42 Subject: Re: Git network protocol Message-ID: <7vk65bg6na.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vk65bg6na.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <20060814062117.GC10476@josefsipek.net> ``` jeffpc@josefsipek.net (Josef "Jeff" Sipek) writes: > I'm trying to implement the git protocol, and I am having a bit of an issue > with the lack of information available about it (please correct me if I > missed some source of information.) Documentation/technical/pack-protocol.txt, and "git show 1bd8c8f0". > So, I can _assume_ that "done" tells the server that it is time to make a > pack. Why does the server use NAK during the clone operation, but ACK > during fetch? Why does the server ACK the same SHA1 twice? And why does the > client "want" the same SHA1 twice? It just seems odd. During the initial "SHA-1 name"/"want" exchange, the server and the client negotiate the protocol extension (the document needs to be updated at least for "multi-ack" extension). After that, the client and the server try to determine what commits the client has that are recent ancestors of "want" commits. This exchange is done by client sending bunch of "have" to the server. The server responds "ACK" (in the original protocol) to say "I've seen enough and know a common ancestor to use". In multi-ack protocol (which is used in modern git, v0.99.9 and later), the server can respond "ACK continue" to say "I've seen enough on that branch so do not bother sending 'have's for its ancestors, but do keep sending from other branch." If the server does not feel it saw enough, it does not send either. The client can send "flush" -- this asks the server to give NAK if more "have"s need to be sent. If there is no more "have"s to be sent, "done" is sent. The protocol streams; if you see an ACK from server it does not usually mean it is ACKing the last 'have' client has sent. ``` ## Josef "Jeff" Sipek, 2006-08-14 23:48 Subject: Re: Git network protocol Message-ID: <20060814234826.GA21861@josefsipek.net> URL: https://gitlist.dev/e/20060814234826.GA21861%40josefsipek.net In-Reply-To: <7vk65bg6na.fsf@assigned-by-dhcp.cox.net> ``` On Sun, Aug 13, 2006 at 11:42:33PM -0700, Junio C Hamano wrote: > jeffpc@josefsipek.net (Josef "Jeff" Sipek) writes: > > > I'm trying to implement the git protocol, and I am having a bit of an issue > > with the lack of information available about it (please correct me if I > > missed some source of information.) > > Documentation/technical/pack-protocol.txt This is pretty much a vague example - or at least it feels vague if you don't know the protocol. After reading the source, and your description, the example makes a lot more sense. > "git show 1bd8c8f0". Neat. > After that, the client and the server try to determine what > commits the client has that are recent ancestors of "want" > commits. This exchange is done by client sending bunch of > "have" to the server. The server responds "ACK" (in the > original protocol) to say "I've seen enough and know a common > ancestor to use". In multi-ack protocol (which is used in > modern git, v0.99.9 and later), the server can respond "ACK > continue" to say "I've seen enough on that branch so do not > bother sending 'have's for its ancestors, but do keep sending > from other branch." If the server does not feel it saw enough, > it does not send either. So, if I understand this correctly, multi_ack allows for multiple branches to be fetched using the same connection? > The protocol streams; if you see an ACK from server it does not > usually mean it is ACKing the last 'have' client has sent. Yeah, makes sense. Doing things over lo does things like that - I'm not sure why I didn't realize it earlier. Thanks, Josef "Jeff" Sipek. -- Don't drink and derive. Alcohol and algebra don't mix. ``` ## Junio C Hamano, 2006-08-15 00:59 Subject: Re: Git network protocol Message-ID: <7vwt9aak5i.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vwt9aak5i.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <20060814234826.GA21861@josefsipek.net> ``` jeffpc@josefsipek.net (Josef "Jeff" Sipek) writes: > So, if I understand this correctly, multi_ack allows for multiple branches > to be fetched using the same connection? The original protocol without extension already allowed it. Suppose the global history was like this: o-----------------------o---o---o---o---o---o---o---c---c---c / o---o---o---o---o---o---o---o---s---s---s where the server side had 's', the client side had 'c', and both of them had 'o'. The objective is to update client with three 's' commits. The exchange would go like this without multi-ack: S: SHA-1 name1 -- for the rightmost 'o' on the top branch S: SHA-2 name2 -- for the rightmost 's' on the bottom branch C: want SHA-2 -- ask for the second branch tip C: have SHA1 -- the rightmost 'c' on the top branch C: have SHA1 -- the parent of the above C: have SHA1 -- the parent of the above C: have SHA1 -- the parent of the above, rightmost 'o' on the top. C: ... more have from the top branch. S: ACK -- for the rightmost 'o' on the top branch During this exchange, the server learns that the client and the server shares the rightmost 'o' on the top branch, but does not learn about all the 'o' commits on the bottom branch, so it ends up sending everything from the fork point to complete three 's' commits. The multi-ack extension was invented by Johannes to improve this exchange. It changes the protocol to let the server ACK with "ACK continue". After that, the client is expected to stop traversing the parents of ACK-continue'd commit -- so it has a chance to send the rightmost 'o' on the bottom branch. When the server sees it, it again gives an ACK, and the client soon runs out have's to send and says "done". In this case, the server would have the rightmost 'o' commits on both branches to work out the minimum set of objects to complete three 's' commits that are missing from the client. ``` ## Josef "Jeff" Sipek, 2006-08-15 09:14 Subject: Re: Git network protocol Message-ID: <20060815091410.GA28369@josefsipek.net> URL: https://gitlist.dev/e/20060815091410.GA28369%40josefsipek.net In-Reply-To: <7vwt9aak5i.fsf@assigned-by-dhcp.cox.net> ``` On Mon, Aug 14, 2006 at 05:59:37PM -0700, Junio C Hamano wrote: > jeffpc@josefsipek.net (Josef "Jeff" Sipek) writes: > > > So, if I understand this correctly, multi_ack allows for multiple branches > > to be fetched using the same connection? > > The original protocol without extension already allowed it. > Suppose the global history was like this: ... Thanks. That helped a lot. Josef "Jeff" Sipek. -- Linux, n.: Generous programmers from around the world all join forces to help you shoot yourself in the foot for free. ```