{"thread":{"id":"2237","subject":"[RFC] multi_ack protocol v2","startedAt":"2005-10-27T00:16:45Z","lastAt":"2005-10-27T19:15:14Z","messageCount":10,"participants":["Johannes Schindelin","Junio C Hamano","Sergey Vlasov","Paul Mackerras"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"10688","messageId":"Pine.LNX.4.63.0510270149590.12163@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2237","inReplyTo":null,"subject":"[RFC] multi_ack protocol v2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-10-27T00:16:45Z","receivedAt":"2005-10-27T00:16:45Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nthe fetch-pack/upload-pack protocol as of now goes like this:\n\n- server sends the refs it has, then an empty packet\n- client sends what it wants, then an empty packet\n- client sends a rev-list via \"have\" lines in the hope to find a common\n  rev. Interspersed, it sends empty packets.\n- server answers to those empty packets with a NAK message, until it\n  receives a \"have\" line for a commit it has (and which is therefore a\n  common commit). This is answered by an ACK message, and no NAK or ACK is \n  sent after that\n- client sends \"done\"\n- server only responds if no common commit was found, with a NAK\n- server sends pack\n\nafter thinking about my earlier approach, I think there's a better, less \nintrusive, and all in all just simpler approach:\n\n- client asks for multi_ack protocol by sending \" multi_ack\" at the end\n  of at least one \"want\" line\n- server appends \" continue\" to the ACK message, but continues sending\n  ACK (but not NAK) messages\n- after receiving \"done\", server repeats last ACK message without \n  \" continue\" appended\n\nAfter much fun with non-working, fragile code which had to be retracted \nfrom master, I hope that this approach is less prone to errors.\n\nNote that this is incompatible to the multi_ack protocol I described \nearlier, but given that my patches were buggy anyway, I'd say it does not \nmatter at all.\n\nThoughts, comments, objections?\n\nCiao,\nDscho\n"},{"id":"10691","messageId":"7vzmov4elu.fsf@assigned-by-dhcp.cox.net","threadId":"2237","inReplyTo":"Pine.LNX.4.63.0510270149590.12163@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-27T07:13:17Z","receivedAt":"2005-10-27T07:13:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> after thinking about my earlier approach, I think there's a better, less \n> intrusive, and all in all just simpler approach:\n>\n> - client asks for multi_ack protocol by sending \" multi_ack\" at the end\n>   of at least one \"want\" line\n> - server appends \" continue\" to the ACK message, but continues sending\n>   ACK (but not NAK) messages\n> - after receiving \"done\", server repeats last ACK message without \n>   \" continue\" appended\n>\n> After much fun with non-working, fragile code which had to be retracted \n> from master, I hope that this approach is less prone to errors.\n\nSorry for a late reply -- real life interrupts ;-).\n\nI do not necessarily think the last round (still found in the\nproposed updates branch) was a failure.  One valuable thing you\nfound out was that there is a way to extend the protocol with\nthe appended \"multi-ack\" trick.\n\nSending \"have you found a common yet\" every 32 \"have\" like the\noriginal protocol does has a nice batching property (I think we\ncould even tweak pkt-line.c::packet_write() to actually buffer\nthese \"have\"s and write them out in one-go, to put them in a\nsingle network packet) and I do not want to lose it, but other\nthan that, since you have found a good way to extend the initial\nhandshake to find out if both ends can do a v2 protocol without\nbreaking older server nor clients, I have no objection against\ndoing things drastically differently in the v2 protocol.\n\nI noticed the sketch in Documentation/pack-protocol.txt is\nsomewhat buggy.  Here is my attempt to correct it:\n\nupload-pack (S) | fetch/clone-pack (C) protocol (v1):\n\n\t# Tell the puller what commits we have and what their names are\n\tS: SHA1 name\n\tS: ...\n\tS: SHA1 name\n\tS: # flush -- it's your turn\n\t# Tell the pusher what commits we want, and what we have\n\tC: want SHA1\n\tC: ..\n\tC: want SHA1\n\tC: # flush -- done with \"want\" lines.\n\tC: have SHA1\n\tC: have SHA1\n\tC: ...\n\tC: # flush -- this occasionally asks \"had enough?\"\n\tS: NAK\n        # and the server answers \"notyet\"\n\tC: have SHA1\n\tC: ...\n\tC: have SHA1\n\tS: ACK SHA1\n\tC: done\n\tS: XXXXXXX -- packfile contents.\n\nI wrote as if \"want\" sends names, but it actually sends SHA1s.\nAlso I missed a place where \"flush\" was needed.\n\nSo let's illustrate the v2 the same way as I understand it.\n\nupload-pack (S) | fetch/clone-pack (C) protocol (v2):\n\n\t# Tell the puller what commits we have and what their names are\n\tS: SHA1 name\n\tS: ...\n\tS: SHA1 name\n\tS: # flush -- it's your turn\n\t# Tell the pusher what commits we want, and what we have.\n        # In addition, we tell the other end that we support protocol\n\t# extensions, without breaking the old servers.\n\tC: want SHA1 extended\n\tC: ..\n\tC: want SHA1\n\tC: # flush -- done with \"want\" lines.\n\nNotice that until we hear from the server, we cannot tell if our\n\"extended\" protocol wish will be granted, and in the original\nprotocol, \"NAK\" will come in fixed length, and the only thing we\ncould tack arbitrary garbage to was \"ACK SHA1\".  That's why your\n\"ACK SHA1 continue\" works nicely, but at the time you could not\nfind out if you are talking with updated server until you get at\nleast one ACK.\n\nHowever, at this point, we *could* force the server to reveal\nwhat it supports, by doing an extra flush here, before sending\n*ANY* \"have\" lines yet:\n\n\tC: # flush -- this is another one after \"I'm done with wants\".\n\nUpon receiving this, if we were talking with an old upload-pack,\nwe would certanly get an NAK.  Note that the server already\nknows that we support extended protocol at this point, so our\nupdated server can send anything here to say it knows what\nprotocol extensions it supports.  Let's say it says something\nlike this:\n\n\tS: proto v2 v3 v5\n\nto tell the puller it understands protocol v2, v3, and v5, to\nwhich the puller responds:\n\n\tC: proto v2\n\nAfter this exchange, both ends know they understand and would\nwant to talk at protocol level v2.  This leaves door open for\nfuture protocol extension, but more importantly, I think this\narrangement would make things safer.\n\nBoth the server side and the client side code can be modified\nfrom the current one by splitting the above handshake part and\nthe current trusty \"only one ACK is supported\" code, and the\nexchange after this protocol negotiation part can be implemented\nin totally separate functions.  We could also give command line\noption and/or .git/config item to limit the protocol level each\nend supports when we find the v2 protocol implementation of the\nday was buggy, in order to work problems around without\nrecompilation, reducing the risk of breaking things too much.\n"},{"id":"10703","messageId":"17248.39333.181379.615274@cargo.ozlabs.ibm.com","threadId":"2237","inReplyTo":"Pine.LNX.4.63.0510270149590.12163@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Paul Mackerras","fromEmail":"paulus@samba.org","sentAt":"2005-10-27T09:11:01Z","receivedAt":"2005-10-27T09:11:01Z","isPatch":false,"sender":{"key":"paulus@samba.org","avatar":"https://avatars.githubusercontent.com/u/1606439?v=4"},"body":"Johannes Schindelin writes:\n\n> after thinking about my earlier approach, I think there's a better, less \n> intrusive, and all in all just simpler approach:\n\nHow many round-trips does it take?  When you're on the other side of\nthe Pacific from the server you care about round-trips. :)  One of the\nnice things about rsync is that it does everything with only 1.5\nround-trips.\n\nPaul.\n"},{"id":"10698","messageId":"Pine.LNX.4.63.0510271107570.5970@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2237","inReplyTo":"7vzmov4elu.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-10-27T09:37:29Z","receivedAt":"2005-10-27T09:37:29Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 27 Oct 2005, Junio C Hamano wrote:\n\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n> \n> > after thinking about my earlier approach, I think there's a better, less \n> > intrusive, and all in all just simpler approach:\n> >\n> > - client asks for multi_ack protocol by sending \" multi_ack\" at the end\n> >   of at least one \"want\" line\n> > - server appends \" continue\" to the ACK message, but continues sending\n> >   ACK (but not NAK) messages\n\nHere must be a correction: server continues to send NAK messages. Else we \nmust introduce some select() or poll() crud, or the client hangs forever.\n\n> > - after receiving \"done\", server repeats last ACK message without \n> >   \" continue\" appended\n\nIn effect, an ACK message with \"continue\" switches into v2, which \njust means: \"I'll keep sendin'\", and an ACK *without* \"continue\" switches \nback to v1.\n\nThis way the patch looks almost trivial. (Will send soon).\n\n> > After much fun with non-working, fragile code which had to be retracted \n> > from master, I hope that this approach is less prone to errors.\n> \n> Sorry for a late reply -- real life interrupts ;-).\n\nNo problem (hope you enjoyed it ;-). I tried to get some Zs anyway.\n\n> Sending \"have you found a common yet\" every 32 \"have\" like the\n> original protocol does has a nice batching property (I think we\n> could even tweak pkt-line.c::packet_write() to actually buffer\n> these \"have\"s and write them out in one-go, to put them in a\n> single network packet) and I do not want to lose it, but other\n> than that, since you have found a good way to extend the initial\n> handshake to find out if both ends can do a v2 protocol without\n> breaking older server nor clients, I have no objection against\n> doing things drastically differently in the v2 protocol.\n\nI think that we don't need to extend packet_write(). Let's leave it like \nthat.\n\n> upload-pack (S) | fetch/clone-pack (C) protocol (v1):\n> \n> \t# Tell the puller what commits we have and what their names are\n> \tS: SHA1 name\n> \tS: ...\n> \tS: SHA1 name\n> \tS: # flush -- it's your turn\n> \t# Tell the pusher what commits we want, and what we have\n> \tC: want SHA1\n> \tC: ..\n> \tC: want SHA1\n> \tC: # flush -- done with \"want\" lines.\n> \tC: have SHA1\n> \tC: have SHA1\n> \tC: ...\n> \tC: # flush -- this occasionally asks \"had enough?\"\n> \tS: NAK\n>         # and the server answers \"notyet\"\n> \tC: have SHA1\n> \tC: ...\n> \tC: have SHA1\n> \tS: ACK SHA1\n> \tC: done\n> \tS: XXXXXXX -- packfile contents.\n\nalternatively, when no \"ACK\" was sent, the response to \"done\" is \"NAK\".\n\n> I wrote as if \"want\" sends names, but it actually sends SHA1s.\n\nYou also ask if it is sensible in upload-pack.c to accept names. I think \nnot. SHA1s are supposed to be unique, names not (master~10 is likely to \ndenote different commits on server and client).\n\n> Also I missed a place where \"flush\" was needed.\n\nBetween \"want\" and \"have\". It is sent so that the server can handle them \nin separate functions.\n\n> upload-pack (S) | fetch/clone-pack (C) protocol (v2):\n> \n> \t# Tell the puller what commits we have and what their names are\n> \tS: SHA1 name\n> \tS: ...\n> \tS: SHA1 name\n> \tS: # flush -- it's your turn\n> \t# Tell the pusher what commits we want, and what we have.\n>         # In addition, we tell the other end that we support protocol\n> \t# extensions, without breaking the old servers.\n> \tC: want SHA1 extended\n> \tC: ..\n> \tC: want SHA1\n> \tC: # flush -- done with \"want\" lines.\n> \n> Notice that until we hear from the server, we cannot tell if our\n> \"extended\" protocol wish will be granted, and in the original\n> protocol, \"NAK\" will come in fixed length, and the only thing we\n> could tack arbitrary garbage to was \"ACK SHA1\".  That's why your\n> \"ACK SHA1 continue\" works nicely, but at the time you could not\n> find out if you are talking with updated server until you get at\n> least one ACK.\n> \n> However, at this point, we *could* force the server to reveal\n> what it supports, by doing an extra flush here, before sending\n> *ANY* \"have\" lines yet:\n> \n> \tC: # flush -- this is another one after \"I'm done with wants\".\n\nWe don't need a second flush. The first can do. The client just has to \nexpect either \"NAK\" or \"VER blabla blibli multi_ack\". \"VER\" must be sent \nby the server only when the client requested an extension the server has,\nthough.\n\n> \tS: proto v2 v3 v5\n\nAs written above, I'd prefer something like \"VER\" in order to stay with \nthe 3 capital letters convention. Also, I like to read what it is about, \ni.e. \"VER multi_ack\" instead of \"VER v2\".\n\n> \tC: proto v2\n\nI'd say that this is not necessary. Client can send that in the \"want\" \nlines.\n\n> After this exchange, both ends know they understand and would\n> want to talk at protocol level v2.  This leaves door open for\n> future protocol extension, but more importantly, I think this\n> arrangement would make things safer.\n\nYes. Nice.\n\nI especially like that the client learns early that multi_ack protocol it \nis. In my tests, non-multi_ack would only be happy when fed with commits \nsorted by date, while multi_ack is much more efficient when fed commits \nsorted by distance-to-tip (I cannot yet explain why this is so).\n\nCiao,\nDscho\n"},{"id":"10702","messageId":"20051027141619.0e8029f2.vsu@altlinux.ru","threadId":"2237","inReplyTo":"7vzmov4elu.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Sergey Vlasov","fromEmail":"vsu@altlinux.ru","sentAt":"2005-10-27T10:16:19Z","receivedAt":"2005-10-27T10:16:19Z","isPatch":false,"sender":{"key":"vsu@altlinux.ru","avatar":"https://avatars.githubusercontent.com/u/616082?v=4"},"body":"On Thu, 27 Oct 2005 00:13:17 -0700 Junio C Hamano wrote:\n\n[skip]\n> So let's illustrate the v2 the same way as I understand it.\n> \n> upload-pack (S) | fetch/clone-pack (C) protocol (v2):\n> \n> \t# Tell the puller what commits we have and what their names are\n> \tS: SHA1 name\n> \tS: ...\n> \tS: SHA1 name\n> \tS: # flush -- it's your turn\n> \t# Tell the pusher what commits we want, and what we have.\n>         # In addition, we tell the other end that we support protocol\n> \t# extensions, without breaking the old servers.\n> \tC: want SHA1 extended\n> \tC: ..\n> \tC: want SHA1\n> \tC: # flush -- done with \"want\" lines.\n> \n> Notice that until we hear from the server, we cannot tell if our\n> \"extended\" protocol wish will be granted, and in the original\n> protocol, \"NAK\" will come in fixed length, and the only thing we\n> could tack arbitrary garbage to was \"ACK SHA1\".  That's why your\n> \"ACK SHA1 continue\" works nicely, but at the time you could not\n> find out if you are talking with updated server until you get at\n> least one ACK.\n\nActually, there is another way to pass some data from the server\nwhich would be ignored by older clients - at the first stage,\nwhen upload-pack sends the list of refs to the client:\n\n\tpacket_write(1, \"%s %s%c%s\\n\", sha1_to_hex(sha1), refname, '\\0',\n\t\t     server_capabilities);\n\nOld clients will ignore the additional data (functions which\nwork with the received name will happily stop at the added NUL\ncharacter; just some memory will be wasted), but the new\nimplementation of get_remote_heads() could look for that NUL\ninside the received packet and find extended server capabilities\nafter it.  Then the client could just use new commands at the\n\"want\" stage.\n\n> However, at this point, we *could* force the server to reveal\n> what it supports, by doing an extra flush here, before sending\n> *ANY* \"have\" lines yet:\n> \n> \tC: # flush -- this is another one after \"I'm done with wants\".\n> \n> Upon receiving this, if we were talking with an old upload-pack,\n> we would certanly get an NAK.  Note that the server already\n> knows that we support extended protocol at this point, so our\n> updated server can send anything here to say it knows what\n> protocol extensions it supports.  Let's say it says something\n> like this:\n> \n> \tS: proto v2 v3 v5\n> \n> to tell the puller it understands protocol v2, v3, and v5, to\n> which the puller responds:\n> \n> \tC: proto v2\n> \n> After this exchange, both ends know they understand and would\n> want to talk at protocol level v2.  This leaves door open for\n> future protocol extension, but more importantly, I think this\n> arrangement would make things safer.\n\nThis looks cleaner (no NUL bytes in the protocol), but does not\nallow to replace the \"want\" stage with something entirely\ndifferent (if we ever want to do that).\n"},{"id":"10704","messageId":"Pine.LNX.4.63.0510271227490.2724@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2237","inReplyTo":"20051027141619.0e8029f2.vsu@altlinux.ru","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-10-27T10:47:24Z","receivedAt":"2005-10-27T10:47:24Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 27 Oct 2005, Sergey Vlasov wrote:\n\n> Actually, there is another way to pass some data from the server\n> which would be ignored by older clients - at the first stage,\n> when upload-pack sends the list of refs to the client:\n> \n> \tpacket_write(1, \"%s %s%c%s\\n\", sha1_to_hex(sha1), refname, '\\0',\n> \t\t     server_capabilities);\n\nThat exploits that packet_write() uses vnsprintf() to find out the length,\nnot strlen(). Sweet.\n\nget_remote_heads() would need to store the server_capabilities, maybe with \na function \"server_supports(const char *extension_string)\" \n\nActually, I like that even more than Junio's proposal. I was thinking hard \nabout how to enhance get_ack() to understand -- and most importantly to \nreturn -- the capabilities. I could not think of an elegant way.\n\nSo, unless people complain, I'll go with that approach.\n\n> This looks cleaner (no NUL bytes in the protocol), but does not\n> allow to replace the \"want\" stage with something entirely\n> different (if we ever want to do that).\n\nAlso it feels wrong that a client should ask for capabilities, where it \ndoes not yet know if the server supports them.\n\nCiao,\nDscho\n"},{"id":"10705","messageId":"Pine.LNX.4.63.0510271248590.3663@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2237","inReplyTo":"17248.39333.181379.615274@cargo.ozlabs.ibm.com","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-10-27T10:54:04Z","receivedAt":"2005-10-27T10:54:04Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 27 Oct 2005, Paul Mackerras wrote:\n\n> Johannes Schindelin writes:\n> \n> > after thinking about my earlier approach, I think there's a better, less \n> > intrusive, and all in all just simpler approach:\n> \n> How many round-trips does it take?  When you're on the other side of\n> the Pacific from the server you care about round-trips. :)  One of the\n> nice things about rsync is that it does everything with only 1.5\n> round-trips.\n\nThe plan is to reuse the existing framework. It sends 32 \"have\" messages, \nthen a flush, then again 32 \"have\" messages, flush again, then reads all \nanswers until a \"NAK\". The last 3 steps are repeated over and over again.\n\nThat means that there are always at least 32 \"have\" messages on the wire \n(which is described in the source code as \"one window ahead\"), i.e. it is \nabout 32x faster than simple \"have\" -> \"NAK/ACK\" handshakes.\n\nCiao,\nDscho\n"},{"id":"10711","messageId":"7vwtjy3lch.fsf@assigned-by-dhcp.cox.net","threadId":"2237","inReplyTo":"Pine.LNX.4.63.0510271227490.2724@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-27T17:45:18Z","receivedAt":"2005-10-27T17:45:18Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> On Thu, 27 Oct 2005, Sergey Vlasov wrote:\n>\n>> Actually, there is another way to pass some data from the server\n>> which would be ignored by older clients - at the first stage,\n>> when upload-pack sends the list of refs to the client:\n>> \n>> \tpacket_write(1, \"%s %s%c%s\\n\", sha1_to_hex(sha1), refname, '\\0',\n>> \t\t     server_capabilities);\n>\n> That exploits that packet_write() uses vnsprintf() to find out the length,\n> not strlen(). Sweet.\n\nOK.\n\n> get_remote_heads() would need to store the server_capabilities, maybe with \n> a function \"server_supports(const char *extension_string)\" \n\nAnother thing that would probably be helpful is to redo\nget_remote_heads() slightly differently, so that it can return\ninformation on *all* refs the other end has.  We need to extend\n\"struct ref\" to mark which one was actually matched/ignored by\npath_match() and ignore_funny.  An completely untested patch is\nattached, based on fetch-pack that still runs rev-list as an\nexternal process, to outline the idea.\n\n---\ndiff --git a/cache.h b/cache.h\nindex 2e36cc5..d39a006 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -329,6 +329,7 @@ struct ref {\n \tunsigned char old_sha1[20];\n \tunsigned char new_sha1[20];\n \tunsigned char force;\n+\tunsigned char matched; /* when using REPORT_ALL */\n \tstruct ref *peer_ref; /* when renaming */\n \tchar name[0];\n };\n@@ -339,7 +340,9 @@ extern int path_match(const char *path, \n extern int match_refs(struct ref *src, struct ref *dst, struct ref ***dst_tail,\n \t\t      int nr_refspec, char **refspec, int all);\n extern int get_ack(int fd, unsigned char *result_sha1);\n-extern struct ref **get_remote_heads(int in, struct ref **list, int nr_match, char **match, int ignore_funny);\n+#define GET_REMOTE_HEADS_IGNORE_FUNNY\t1\n+#define GET_REMOTE_HEADS_REPORT_ALL\t2\n+extern struct ref **get_remote_heads(int in, struct ref **list, int nr_match, char **match, int match_options);\n \n extern struct packed_git *parse_pack_index(unsigned char *sha1);\n extern struct packed_git *parse_pack_index_file(const unsigned char *sha1,\ndiff --git a/clone-pack.c b/clone-pack.c\nindex 9609219..8b63183 100644\n--- a/clone-pack.c\n+++ b/clone-pack.c\n@@ -250,7 +250,7 @@ static int clone_pack(int fd[2], int nr_\n \tstruct ref *refs;\n \tint status;\n \n-\tget_remote_heads(fd[0], &refs, nr_match, match, 1);\n+\tget_remote_heads(fd[0], &refs, nr_match, match, GET_REMOTE_HEADS_IGNORE_FUNNY);\n \tif (!refs) {\n \t\tpacket_flush(fd[1]);\n \t\tdie(\"no matching remote head\");\ndiff --git a/connect.c b/connect.c\nindex b171c5d..f7a3550 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -12,8 +12,11 @@\n  * Read all the refs from the other end\n  */\n struct ref **get_remote_heads(int in, struct ref **list,\n-\t\t\t      int nr_match, char **match, int ignore_funny)\n+\t\t\t      int nr_match, char **match, int match_options)\n {\n+\tint ignore_funny = match_options & GET_REMOTE_HEADS_IGNORE_FUNNY;\n+\tint report_all = match_options & GET_REMOTE_HEADS_REPORT_ALL;\n+\n \t*list = NULL;\n \tfor (;;) {\n \t\tstruct ref *ref;\n@@ -21,6 +24,7 @@ struct ref **get_remote_heads(int in, st\n \t\tstatic char buffer[1000];\n \t\tchar *name;\n \t\tint len;\n+\t\tint matched = 1;\n \n \t\tlen = packet_read_line(in, buffer, sizeof(buffer));\n \t\tif (!len)\n@@ -28,17 +32,22 @@ struct ref **get_remote_heads(int in, st\n \t\tif (buffer[len-1] == '\\n')\n \t\t\tbuffer[--len] = 0;\n \n-\t\tif (len < 42 || get_sha1_hex(buffer, old_sha1) || buffer[40] != ' ')\n-\t\t\tdie(\"protocol error: expected sha/ref, got '%s'\", buffer);\n+\t\tif (len < 42 || get_sha1_hex(buffer, old_sha1) ||\n+\t\t    buffer[40] != ' ')\n+\t\t\tdie(\"protocol error: expected sha/ref, got '%s'\",\n+\t\t\t    buffer);\n \t\tname = buffer + 41;\n \n-\t\tif (ignore_funny && 45 < len && !memcmp(name, \"refs/\", 5) &&\n-\t\t    check_ref_format(name + 5))\n-\t\t\tcontinue;\n+\t\tif ((ignore_funny && 45 < len && !memcmp(name, \"refs/\", 5) &&\n+\t\t     check_ref_format(name + 5)) ||\n+\t\t    (nr_match && !path_match(name, nr_match, match))) {\n+\t\t\tif (!report_all)\n+\t\t\t\tcontinue;\n+\t\t\tmatched = 0;\n+\t\t}\n \n-\t\tif (nr_match && !path_match(name, nr_match, match))\n-\t\t\tcontinue;\n \t\tref = xcalloc(1, sizeof(*ref) + len - 40);\n+\t\tref->matched = matched;\n \t\tmemcpy(ref->old_sha1, old_sha1, 20);\n \t\tmemcpy(ref->name, buffer + 41, len - 40);\n \t\t*list = ref;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 8566ab1..11fb1c1 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -13,6 +13,7 @@ static const char fetch_pack_usage[] =\n static const char *exec = \"git-upload-pack\";\n \n #define COMPLETE\t(1U << 0)\n+#define EXCLUDE\t\t(1U << 1)\n \n static int find_common(int fd[2], unsigned char *result_sha1,\n \t\t       struct ref *refs)\n@@ -52,17 +53,24 @@ static int find_common(int fd[2], unsign\n \t\t\tp = commit->parents;\n \t\t\twhile (p &&\n \t\t\t       rev_command_len + 44 < sizeof(rev_command)) {\n-\t\t\t\tsnprintf(rev_command + rev_command_len, 44,\n-\t\t\t\t\t \" ^%s\",\n-\t\t\t\t\t sha1_to_hex(p->item->object.sha1));\n-\t\t\t\trev_command_len += 43;\n+\t\t\t\tstruct object *pobj = &(p->item->object);\n+\t\t\t\tif (!pobj->flags & EXCLUDE) {\n+\t\t\t\t\tsnprintf(rev_command +\n+\t\t\t\t\t\t rev_command_len, 44,\n+\t\t\t\t\t\t \" ^%s\",\n+\t\t\t\t\t\t sha1_to_hex(pobj->sha1));\n+\t\t\t\t\trev_command_len += 43;\n+\t\t\t\t}\n+\t\t\t\tpobj->flags |= EXCLUDE;\n \t\t\t\tp = p->next;\n \t\t\t}\n \t\t\tcontinue;\n \t\t}\n \trepair:\n-\t\tpacket_write(fd[1], \"want %s\\n\", sha1_to_hex(remote));\n-\t\tfetching++;\n+\t\tif (refs->matched) {\n+\t\t\tpacket_write(fd[1], \"want %s\\n\", sha1_to_hex(remote));\n+\t\t\tfetching = 1;\n+\t\t}\n \t}\n \tpacket_flush(fd[1]);\n \tif (!fetching)\n@@ -183,6 +191,8 @@ static int everything_local(struct ref *\n \t\tunsigned char local[20];\n \t\tstruct object *o;\n \n+\t\tif (!refs->matched)\n+\t\t\tcontinue;\n \t\to = parse_object(remote);\n \t\tif (!o || !(o->flags & COMPLETE)) {\n \t\t\tretval = 0;\n@@ -204,6 +214,16 @@ static int everything_local(struct ref *\n \treturn retval;\n }\n \n+static int no_matching_remote(struct ref *ref)\n+{\n+\twhile (ref)\n+\t\tif (ref->matched)\n+\t\t\treturn 0;\n+\t\telse\n+\t\t\tref = ref->next;\n+\treturn 1;\n+}\n+\n static int fetch_pack(int fd[2], int nr_match, char **match)\n {\n \tstruct ref *ref;\n@@ -211,8 +231,10 @@ static int fetch_pack(int fd[2], int nr_\n \tint status;\n \tpid_t pid;\n \n-\tget_remote_heads(fd[0], &ref, nr_match, match, 1);\n-\tif (!ref) {\n+\tget_remote_heads(fd[0], &ref, nr_match, match,\n+\t\t\t GET_REMOTE_HEADS_IGNORE_FUNNY |\n+\t\t\t GET_REMOTE_HEADS_REPORT_ALL);\n+\tif (no_matching_remote(ref)) {\n \t\tpacket_flush(fd[1]);\n \t\tdie(\"no matching remote head\");\n \t}\n@@ -245,8 +267,9 @@ static int fetch_pack(int fd[2], int nr_\n \t\t\tdie(\"git-unpack-objects died with error code %d\", code);\n all_done:\n \t\twhile (ref) {\n-\t\t\tprintf(\"%s %s\\n\",\n-\t\t\t       sha1_to_hex(ref->old_sha1), ref->name);\n+\t\t\tif (ref->matched)\n+\t\t\t\tprintf(\"%s %s\\n\",\n+\t\t\t\t       sha1_to_hex(ref->old_sha1), ref->name);\n \t\t\tref = ref->next;\n \t\t}\n \t\treturn 0;\ndiff --git a/send-pack.c b/send-pack.c\nindex 9f9a6e7..d7bb6c1 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -181,7 +181,8 @@ static int send_pack(int in, int out, in\n \tint new_refs;\n \n \t/* No funny business with the matcher */\n-\tremote_tail = get_remote_heads(in, &remote_refs, 0, NULL, 1);\n+\tremote_tail = get_remote_heads(in, &remote_refs, 0, NULL,\n+\t\t\t\t       GET_REMOTE_HEADS_IGNORE_FUNNY);\n \tget_local_heads();\n \n \t/* match them up */\n"},{"id":"10714","messageId":"Pine.LNX.4.63.0510272003060.22251@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"2237","inReplyTo":"7vwtjy3lch.fsf@assigned-by-dhcp.cox.net","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2005-10-27T18:04:30Z","receivedAt":"2005-10-27T18:04:30Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 27 Oct 2005, Junio C Hamano wrote:\n\n> Another thing that would probably be helpful is to redo\n> get_remote_heads() slightly differently, so that it can return\n> information on *all* refs the other end has.  We need to extend\n> \"struct ref\" to mark which one was actually matched/ignored by\n> path_match() and ignore_funny.  An completely untested patch is\n> attached, based on fetch-pack that still runs rev-list as an\n> external process, to outline the idea.\n\nA completely tested patch (unfortunately in the middle of my work) does \nnot touch get_remote_heads() at all:\n\ndiff --git a/fetch-pack.c b/fetch-pack.c\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -274,7 +290,27 @@ static void mark_recent_complete_commits\n \t}\n }\n \n-static int everything_local(struct ref *refs)\n+static void filter_refs(struct ref **refs, int nr_match, char **match)\n+{\n+\tstruct ref *prev, *current, *next;\n+\n+\tif (!nr_match)\n+\t\treturn;\n+\n+\tfor (prev = NULL, current = *refs; current; current = next) {\n+\t\tnext = current->next;\n+\t\tif (!path_match(current->name, nr_match, match)) {\n+\t\t\tif (prev == NULL)\n+\t\t\t\t*refs = next;\n+\t\t\telse\n+\t\t\t\tprev->next = next;\n+\t\t\tfree(current);\n+\t\t} else\n+\t\t\tprev = current;\n+\t}\n+}\n+\n+static int everything_local(struct ref **refs, int nr_match, char **match)\n {\n \tstruct ref *ref;\n \tint retval;\n@@ -283,7 +319,7 @@ static int everything_local(struct ref *\n \ttrack_object_refs = 0;\n \tsave_commit_buffer = 0;\n \n-\tfor (ref = refs; ref; ref = ref->next) {\n+\tfor (ref = *refs; ref; ref = ref->next) {\n \t\tstruct object *o;\n \n \t\to = parse_object(ref->old_sha1);\n@@ -305,8 +341,10 @@ static int everything_local(struct ref *\n \tif (cutoff)\n \t\tmark_recent_complete_commits(cutoff);\n \n-\tfor (retval = 1; refs ; refs = refs->next) {\n-\t\tconst unsigned char *remote = refs->old_sha1;\n+\tfilter_refs(refs, nr_match, match);\n+\n+\tfor (retval = 1, ref = *refs; ref ; ref = ref->next) {\n+\t\tconst unsigned char *remote = ref->old_sha1;\n \t\tunsigned char local[20];\n \t\tstruct object *o;\n \n@@ -317,16 +355,16 @@ static int everything_local(struct ref *\n \t\t\t\tcontinue;\n \t\t\tfprintf(stderr,\n \t\t\t\t\"want %s (%s)\\n\", sha1_to_hex(remote),\n-\t\t\t\trefs->name);\n+\t\t\t\tref->name);\n \t\t\tcontinue;\n \t\t}\n \n-\t\tmemcpy(refs->new_sha1, local, 20);\n+\t\tmemcpy(ref->new_sha1, local, 20);\n \t\tif (!verbose)\n \t\t\tcontinue;\n \t\tfprintf(stderr,\n \t\t\t\"already have %s (%s)\\n\", sha1_to_hex(remote),\n-\t\t\trefs->name);\n+\t\t\tref->name);\n \t}\n \treturn retval;\n }\n@@ -338,12 +376,12 @@ static int fetch_pack(int fd[2], int nr_\n \tint status;\n \tpid_t pid;\n \n-\tget_remote_heads(fd[0], &ref, nr_match, match, 1);\n+\tget_remote_heads(fd[0], &ref, 0, NULL, 1);\n \tif (!ref) {\n \t\tpacket_flush(fd[1]);\n \t\tdie(\"no matching remote head\");\n \t}\n-\tif (everything_local(ref)) {\n+\tif (everything_local(&ref, nr_match, match)) {\n \t\tpacket_flush(fd[1]);\n \t\tgoto all_done;\n \t}\n"},{"id":"10716","messageId":"7v4q723h6l.fsf@assigned-by-dhcp.cox.net","threadId":"2237","inReplyTo":"Pine.LNX.4.63.0510272003060.22251@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC] multi_ack protocol v2","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-27T19:15:14Z","receivedAt":"2005-10-27T19:15:14Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> A completely tested patch (unfortunately in the middle of my work) does \n> not touch get_remote_heads() at all:\n\nMuch nicer.\n"}]}