{"thread":{"id":"55376","subject":"[PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","startedAt":"2021-03-24T05:38:10Z","lastAt":"2021-03-29T10:41:06Z","messageCount":8,"participants":["lilinchao@oschina.cn","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"420102","messageId":"006547b28c6311eb93820024e87935e7@oschina.cn","threadId":"55376","inReplyTo":"20210324053648.25584-1-lilinchao@oschina.cn","subject":"[PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"","fromEmail":"lilinchao@oschina.cn","sentAt":"2021-03-24T05:36:48Z","receivedAt":"2021-03-24T05:38:10Z","isPatch":true,"sender":{"key":"lilinchao@oschina.cn","avatar":null},"body":"From: lilinchao \n\nWhen clone with http protocol version 1, the server side\njust tells client that \"invalid server response, got version 1\",\nthis is not clear enough, because version 0 is ok, and\nversion 2 is ok, then version 1 should be ok too intuitively,\nbut the other side just treat it as \"invalid response\", this\ncan't explain why is not ok.\n\n>From receive-pack/upload-pack, there is a comment which I think,\ncan explain it:\n\"v1 is just the original protocol with a version string\".\nSo I made this patch to try to fix it.\n\nSigned-off-by: lilinchao \n---\n remote-curl.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 0290b04891..1fe1f3c475 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -434,7 +434,8 @@ static void check_smart_http(struct discovery *d, const char *service,\n \t\t * be handled elsewhere.\n \t\t */\n \t\td->proto_git = 1;\n-\n+\t} else if (!strcmp(reader.line, \"version 1\")) {\n+\t\tdie(_(\"v1 is just the original protocol with a version string, use v0 or v2 instead.\"));\n \t} else {\n \t\tdie(_(\"invalid server response; got '%s'\"), reader.line);\n \t}\n-- \n2.30.0.1006.g4a81e96670\n\n"},{"id":"420145","messageId":"xmqq7dlwxpn3.fsf@gitster.g","threadId":"55376","inReplyTo":"006547b28c6311eb93820024e87935e7@oschina.cn","subject":"Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-24T20:28:32Z","receivedAt":"2021-03-24T20:29:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"lilinchao@oschina.cn writes:\n\n> \t\t * be handled elsewhere.\n> \t\t */\n> \t\td->proto_git = 1;\n> -\n\nUnrelated removal.\n\n> +\t} else if (!strcmp(reader.line, \"version 1\")) {\n> +\t\tdie(_(\"v1 is just the original protocol with a version string, use v0 or v2 instead.\"));\n\nThe user may no longer get \"invalid response; got 'version 1'\", but\nthe above does not still explain why v1 is bad and v0 or v2 is\nwelcome, either. IOW, I do not think the patch improves the message\nto achieve what it attempted to do, i.e.\n\n ... but the other side just treat it as \"invalid response\", this\n can't explain why is not ok.\n\nI wonder if it is a sensible and better alternative to treat v1\nresponse as if we got v0 (if v1 is truly the same as v0 except for\nthe initial version advertisement).\n\nInput from those who are familiar with the protocol versions is very\nmuch appreciated.\n\nThanks.\n\n> \t} else {\n> \t\tdie(_(\"invalid server response; got '%s'\"), reader.line);\n> \t}\n"},{"id":"420146","messageId":"xmqq35wkxper.fsf@gitster.g","threadId":"55376","inReplyTo":"xmqq7dlwxpn3.fsf@gitster.g","subject":"Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-03-24T20:33:32Z","receivedAt":"2021-03-24T20:34:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n>> +\t} else if (!strcmp(reader.line, \"version 1\")) {\n>> +\t\tdie(_(\"v1 is just the original protocol with a version string, use v0 or v2 instead.\"));\n>\n> The user may no longer get \"invalid response; got 'version 1'\", but\n> the above does not still explain why v1 is bad and v0 or v2 is\n> welcome, either. IOW, I do not think the patch improves the message\n> to achieve what it attempted to do, i.e.\n>\n> ... but the other side just treat it as \"invalid response\", this\n> can't explain why is not ok.\n\nAlternatively\n\n\tv1 is not supported; use v0 or v2\n\nwould explain why the connection is refused. It explains why it is\nnot ok much clearly than \"just the original with a version string\".\n\n> I wonder if it is a sensible and better alternative to treat v1\n> response as if we got v0 (if v1 is truly the same as v0 except for\n> the initial version advertisement).\n>\n> Input from those who are familiar with the protocol versions is very\n> much appreciated.\n\nThis still stands; we reject because we don't support, but is it\neasy to support it instead, if there is no difference?\n"},{"id":"420176","messageId":"c0cd24e88d2111ebbdf30026b95c99cc@oschina.cn","threadId":"55376","inReplyTo":"388751448ce011ebaaead4ae5278bc1265898@pobox.com","subject":"Re: Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"lilinchao@oschina.cn","fromEmail":"lilinchao@oschina.cn","sentAt":"2021-03-25T04:22:45Z","receivedAt":"2021-03-25T04:23:52Z","isPatch":true,"sender":{"key":"lilinchao@oschina.cn","avatar":null},"body":"--------------\nlilinchao@oschina.cn\n>Junio C Hamano writes:\n>\n>>> +\t} else if (!strcmp(reader.line, \"version 1\")) {\n>>> +\tdie(_(\"v1 is just the original protocol with a version string, use v0 or v2 instead.\"));\n>>\n>> The user may no longer get \"invalid response; got 'version 1'\", but\n>> the above does not still explain why v1 is bad and v0 or v2 is\n>> welcome, either.  IOW, I do not think the patch improves the message\n>> to achieve what it attempted to do, i.e.\n>>\n>>     ... but the other side just treat it as \"invalid response\", this\n>>     can't explain why is not ok.\n>\n>Alternatively\n>\n>\tv1 is not supported; use v0 or v2\n>\n>would explain why the connection is refused.  It explains why it is\n>not ok much clearly than \"just the original with a version string\".\n>\n>> I wonder if it is a sensible and better alternative to treat v1\n>> response as if we got v0 (if v1 is truly the same as v0 except for\n>> the initial version advertisement).\n>>\n>> Input from those who are familiar with the protocol versions is very\n>> much appreciated.\n>\n>This still stands; we reject because we don't support, but is it\n>easy to support it instead, if there is no difference? \n\nYes, if there is no difference, just to support it too. So I don't know\nwhy it is not support until now.\n\nThanks."},{"id":"420233","messageId":"YF1+AjgfA6gnAGga@coredump.intra.peff.net","threadId":"55376","inReplyTo":"xmqq7dlwxpn3.fsf@gitster.g","subject":"Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-03-26T06:24:02Z","receivedAt":"2021-03-26T06:25:05Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Mar 24, 2021 at 01:28:32PM -0700, Junio C Hamano wrote:\n\n> > +\t} else if (!strcmp(reader.line, \"version 1\")) {\n> > +\t\tdie(_(\"v1 is just the original protocol with a version string, use v0 or v2 instead.\"));\n> \n> The user may no longer get \"invalid response; got 'version 1'\", but\n> the above does not still explain why v1 is bad and v0 or v2 is\n> welcome, either. IOW, I do not think the patch improves the message\n> to achieve what it attempted to do, i.e.\n> \n> ... but the other side just treat it as \"invalid response\", this\n> can't explain why is not ok.\n> \n> I wonder if it is a sensible and better alternative to treat v1\n> response as if we got v0 (if v1 is truly the same as v0 except for\n> the initial version advertisement).\n> \n> Input from those who are familiar with the protocol versions is very\n> much appreciated.\n\nYes, \"v1\" is supposed to behave just like v0, except with the version\nadvertisement (it is true that there is no point in normal people using\nit, but the purpose was to make sure the version advertisement worked).\n\nI am not sure who is rejecting it, though. Our test suite passes with\nGIT_TEST_PROTOCOL_VERSION=1. Running something like:\n\n $ GIT_TRACE_PACKET=1 git -c protocol.version=1 ls-remote https://github.com/git/git\n\nyields a conversation like (cut down for clarity):\n\n git< # service=git-upload-pack\n git< 0000\n git< version 1\n git< 1234abcd[...etc, this is a normal v0/v1 advertisement]\n\nSo the version string is there, but it does not trigger the problem\ndescribed by this patch. That's because check_smart_http(), after seeing\nthe \"# service\" line and the flush, takes all the rest of the packetized\ndata and gives it to parse_git_refs(), which handles the version field\nline via discover_version().\n\n Aside: on gitlab.com, the v1 response looks like a v0 response, with\n no extra header. I guess they did not bother to implement v1, which is\n OK, since it was not useful after the initial experiment.\n\nSo everything seems to be working as intended. Is there some particular\nserver that returns \"version 1\" in the wrong way, triggering the die()?\n\nOne curiosity is that for v2, the response from github.com does include\nthe \"service\" line. So it follows the same path as v1, and never hits\nthe \"version 2\" line check here. But http-backend omits the \"service\"\nline, due to 237ffedd46 (http: eliminate \"# service\" line when using\nprotocol v2, 2018-03-15).\n\nSo it's interesting that GitHub behaves differently than http-backend\nhere. It's not surprising, since the HTTP framing is all done by a\ncustom server there, which implemented off the spec. What _is_\nsurprising is that the client seems perfectly happy to see either form,\nand nobody has noticed the difference until just now.\n\nIMHO the spec is very unclear here; it says \"client makes a smart\ninfo/refs request as described in http-protocol.txt\", but doesn't call\nout the difference in the response. It's only implied by the example:\n\n A v2 server would reply:\n\n S: 200 OK\n S: \n S: ...\n S:\n S: 000eversion 2\\n\n S: \n\nwhere it is unclear whether the blank line is separating HTTP headers\nfrom the body (and thus \"...\" is some headers), or if it is separating\nthe \"# service\" line and matching flush from the rest of the response\nbody.\n\nI note that gitlab.com also returns the \"service\" line for v2 (I don't\nknow anything about their implementation, but I would not be at all\nsurprised if they also use a custom HTTP endpoint; apache+http-backend\nis not very flexible or scalable).\n\nAnyway, that's all just an interesting side note. The client is happy\nwith either form (though it might be nice if we had tests for the \"#\nservice\" form; I suspect our tests don't cover that because they are all\nusing http-backend).\n\nGetting back to the patch at hand, if there is a server saying \"version\n1\" without a \"service\" line, then I think that is a bug in that server.\n\n-Peff\n"},{"id":"420236","messageId":"YF2FdzuQJV5Zb/y1@coredump.intra.peff.net","threadId":"55376","inReplyTo":"YF1+AjgfA6gnAGga@coredump.intra.peff.net","subject":"Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-03-26T06:55:51Z","receivedAt":"2021-03-26T06:56:51Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 26, 2021 at 02:24:03AM -0400, Jeff King wrote:\n\n> One curiosity is that for v2, the response from github.com does include\n> the \"service\" line. So it follows the same path as v1, and never hits\n> the \"version 2\" line check here. But http-backend omits the \"service\"\n> line, due to 237ffedd46 (http: eliminate \"# service\" line when using\n> protocol v2, 2018-03-15).\n> \n> So it's interesting that GitHub behaves differently than http-backend\n> here. It's not surprising, since the HTTP framing is all done by a\n> custom server there, which implemented off the spec. What _is_\n> surprising is that the client seems perfectly happy to see either form,\n> and nobody has noticed the difference until just now.\n> \n> IMHO the spec is very unclear here; it says \"client makes a smart\n> info/refs request as described in http-protocol.txt\", but doesn't call\n> out the difference in the response. It's only implied by the example:\n> \n> A v2 server would reply:\n> \n> S: 200 OK\n> S: \n> S: ...\n> S:\n> S: 000eversion 2\\n\n> S: \n> \n> where it is unclear whether the blank line is separating HTTP headers\n> from the body (and thus \"...\" is some headers), or if it is separating\n> the \"# service\" line and matching flush from the rest of the response\n> body.\n> \n> I note that gitlab.com also returns the \"service\" line for v2 (I don't\n> know anything about their implementation, but I would not be at all\n> surprised if they also use a custom HTTP endpoint; apache+http-backend\n> is not very flexible or scalable).\n\nI wondered two things:\n\n - how other servers behave; jgit is the obvious other one to check. It\n seems to match http-backend in omitting the \"service\" line. I also\n checked its v1 behavior. It seems to ignore it totally and behave\n like v0 (which again, is OK, since it's not useful). This was based\n on testing against https://android.googlesource.com. In v0, it\n claims agent=JGit/4-google, though curiously in v2 it does not\n advertise an agent at all. :)\n\n - whether other v2 clients are equally forgiving of either format.\n Again, jgit is probably the most interesting here (libgit2 does not\n speak v2 at all yet). And indeed, it seems to be happy with either\n format (which is not surprising, given how common both types of\n server are).\n\n-Peff\n"},{"id":"420438","messageId":"68765f14907111eb8e180024e87935e7@oschina.cn","threadId":"55376","inReplyTo":"e4d6bef08dfb11eb90f0a4badb2c2b1115536@peff.net","subject":"Re: Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"lilinchao@oschina.cn","fromEmail":"lilinchao@oschina.cn","sentAt":"2021-03-29T09:30:29Z","receivedAt":"2021-03-29T09:31:18Z","isPatch":true,"sender":{"key":"lilinchao@oschina.cn","avatar":null},"body":"\n\n\n--------------\nlilinchao@oschina.cn\n>On Wed, Mar 24, 2021 at 01:28:32PM -0700, Junio C Hamano wrote:\n>\n>> > +\t} else if (!strcmp(reader.line, \"version 1\")) {\n>> > +\tdie(_(\"v1 is just the original protocol with a version string, use v0 or v2 instead.\"));\n>>\n>> The user may no longer get \"invalid response; got 'version 1'\", but\n>> the above does not still explain why v1 is bad and v0 or v2 is\n>> welcome, either.  IOW, I do not think the patch improves the message\n>> to achieve what it attempted to do, i.e.\n>>\n>>     ... but the other side just treat it as \"invalid response\", this\n>>     can't explain why is not ok.\n>>\n>> I wonder if it is a sensible and better alternative to treat v1\n>> response as if we got v0 (if v1 is truly the same as v0 except for\n>> the initial version advertisement).\n>>\n>> Input from those who are familiar with the protocol versions is very\n>> much appreciated.\n>\n>Yes, \"v1\" is supposed to behave just like v0, except with the version\n>advertisement (it is true that there is no point in normal people using\n>it, but the purpose was to make sure the version advertisement worked).\n>\n>I am not sure who is rejecting it, though. Our test suite passes with\n>GIT_TEST_PROTOCOL_VERSION=1. Running something like:\n>\n> $ GIT_TRACE_PACKET=1 git -c protocol.version=1 ls-remote https://github.com/git/git\n>\n>yields a conversation like (cut down for clarity):\n>\n>  git< # service=git-upload-pack\n>  git< 0000\n>  git< version 1\n>  git< 1234abcd[...etc, this is a normal v0/v1 advertisement]\n>\n>So the version string is there, but it does not trigger the problem\n>described by this patch. That's because check_smart_http(), after seeing\n>the \"# service\" line and the flush, takes all the rest of the packetized\n>data and gives it to parse_git_refs(), which handles the version field\n>line via discover_version().\n>\n>  Aside: on gitlab.com, the v1 response looks like a v0 response, with\n>  no extra header. I guess they did not bother to implement v1, which is\n>  OK, since it was not useful after the initial experiment.\n>\n>So everything seems to be working as intended. Is there some particular\n>server that returns \"version 1\" in the wrong way, triggering the die()?\n> \nOn gitee.com, I got \"version 1\", and the process died here.\n\n>One curiosity is that for v2, the response from github.com does include\n>the \"service\" line. So it follows the same path as v1, and never hits\n>the \"version 2\" line check here. But http-backend omits the \"service\"\n>line, due to 237ffedd46 (http: eliminate \"# service\" line when using\n>protocol v2, 2018-03-15).\n> \nKeen observation :)\n\n>So it's interesting that GitHub behaves differently than http-backend\n>here. It's not surprising, since the HTTP framing is all done by a\n>custom server there, which implemented off the spec.  What _is_\n>surprising is that the client seems perfectly happy to see either form,\n>and nobody has noticed the difference until just now.\n>\n>IMHO the spec is very unclear here; it says \"client makes a smart\n>info/refs request as described in http-protocol.txt\", but doesn't call\n>out the difference in the response. It's only implied by the example:\n>\n>  A v2 server would reply:\n>\n>     S: 200 OK\n>     S: \n>     S: ...\n>     S:\n>     S: 000eversion 2\\n\n>     S: \n>\n>where it is unclear whether the blank line is separating HTTP headers\n>from the body (and thus \"...\" is some headers), or if it is separating\n>the \"# service\" line and matching flush from the rest of the response\n>body.\n>\n>I note that gitlab.com also returns the \"service\" line for v2 (I don't\n>know anything about their implementation, but I would not be at all\n>surprised if they also use a custom HTTP endpoint; apache+http-backend\n>is not very flexible or scalable).\n> \ngitee.com returns the \"version 1\" line for v1, so died for invalid server response\nand it returns the \"version 2\" line for v2, which is expected.\n\n>Anyway, that's all just an interesting side note. The client is happy\n>with either form (though it might be nice if we had tests for the \"#\n>service\" form; I suspect our tests don't cover that because they are all\n>using http-backend).\n>\n>Getting back to the patch at hand, if there is a server saying \"version\n>1\" without a \"service\" line, then I think that is a bug in that server.\n> \nIf the problem is on the server side, then, is this patch worth continuing?\n\nThanks!\n\n>-Peff"},{"id":"420443","messageId":"YGGukSxGVnTeXae0@coredump.intra.peff.net","threadId":"55376","inReplyTo":"68765f14907111eb8e180024e87935e7@oschina.cn","subject":"Re: Re: [PATCH 2/2] remote-curl.c: handle v1 when check_smart_http","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-03-29T10:40:17Z","receivedAt":"2021-03-29T10:41:06Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 29, 2021 at 05:30:29PM +0800, lilinchao@oschina.cn wrote:\n\n> >Anyway, that's all just an interesting side note. The client is happy\n> >with either form (though it might be nice if we had tests for the \"#\n> >service\" form; I suspect our tests don't cover that because they are all\n> >using http-backend).\n> >\n> >Getting back to the patch at hand, if there is a server saying \"version\n> >1\" without a \"service\" line, then I think that is a bug in that server.\n> > \n> If the problem is on the server side, then, is this patch worth continuing?\n\nIMHO, no. I think the response from gitee.com is violating the protocol\nspec. It would be nice to fix, but in practice it isn't all that\nimportant because somebody would have to manually set protocol.version=1\nto see the problem.\n\n-Peff\n"}]}