{"thread":{"id":"36492","subject":"GIT, libcurl and GSS-Negotiate","startedAt":"2014-04-24T17:17:36Z","lastAt":"2014-05-17T06:51:02Z","messageCount":8,"participants":["Ivo Bellin Salarin","brian m. carlson","Carlos Martín Nieto","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"239575","messageId":"CAPc4eF__gWMy=E-8tdpMn_irA4m7mYF3=cwN6JeAqJsdPshNLw@mail.gmail.com","threadId":"36492","inReplyTo":null,"subject":"GIT, libcurl and GSS-Negotiate","fromName":"Ivo Bellin Salarin","fromEmail":"ivo.bellinsalarin@gmail.com","sentAt":"2014-04-24T17:17:36Z","receivedAt":"2014-04-24T17:17:36Z","isPatch":false,"sender":{"key":"ivo.bellinsalarin@gmail.com","avatar":null},"body":"Hello,\n\nI'm having problems while trying to authenticate against a TFS hosted\nrepository.\n\nI experience the same problem in git for windows and in git for linux\n(both versions are 1.9.2).\n\nThe problem is described on a [github msysgit/git\nissue](https://github.com/msysgit/git/issues/171)\n\nTo shortly resume it, the problem is that:\n* when the authentication method (WWW-Authenticate) is Negotiate AND\n* when the server proposes a NTLMSSP_CHALLENGE in response of the\nclient's NTLMSSP_NEGOTIATE,\n=> libcurl yields an \"Authentication problem. Ignoring this.\\n\"\nAnd the communication is closed.\n\nAt this point, in a normal communication, the client should send a\nNTLMSSP_AUTH containing a Kerberos ticket.\n\nHaving seen the libcurl source code, I think we're passing through the\nlines  from 776 to 780 of\n[http.c](https://github.com/bagder/curl/blob/2e57c7e0fcfb9214b2a9dfa8b3da258ded013b8a/lib/http.c).\nSome guy, on the github issue page, has suggested that this could be\nrelated to an update of libcurl, when git was at its 1.8.2 version.\n\nI'm not debugging libcurl, and I can't reproduce this problem @home.\nSo, has somebody already experienced the same problem? Is there a\nsolution?\n\nMany thanks in advance,\nIvo\n-- \nhttp://www.nilleb.com\n"},{"id":"239766","messageId":"20140426174718.GC238861@vauxhall.crustytoothpaste.net","threadId":"36492","inReplyTo":"CAPc4eF__gWMy=E-8tdpMn_irA4m7mYF3=cwN6JeAqJsdPshNLw@mail.gmail.com","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-04-26T17:47:18Z","receivedAt":"2014-04-26T17:47:18Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Thu, Apr 24, 2014 at 07:17:36PM +0200, Ivo Bellin Salarin wrote:\n> To shortly resume it, the problem is that:\n> * when the authentication method (WWW-Authenticate) is Negotiate AND\n> * when the server proposes a NTLMSSP_CHALLENGE in response of the\n> client's NTLMSSP_NEGOTIATE,\n> => libcurl yields an \"Authentication problem. Ignoring this.\\n\"\n> And the communication is closed.\n> \n> At this point, in a normal communication, the client should send a\n> NTLMSSP_AUTH containing a Kerberos ticket.\n> \n> Having seen the libcurl source code, I think we're passing through the\n> lines  from 776 to 780 of\n> [http.c](https://github.com/bagder/curl/blob/2e57c7e0fcfb9214b2a9dfa8b3da258ded013b8a/lib/http.c).\n> Some guy, on the github issue page, has suggested that this could be\n> related to an update of libcurl, when git was at its 1.8.2 version.\n> \n> I'm not debugging libcurl, and I can't reproduce this problem @home.\n> So, has somebody already experienced the same problem? Is there a\n> solution?\n\nI'm personally using Git with GSS-Negotiate (and MIT Kerberos 5) and it\ndoes seem to work correctly for me.  For large pushes, your server (and\nany intermediate proxies) will need to support 100 Continue properly, as\nthere's simply no other way to make it work.\n\nWhat version of curl are you using (and what distro if you didn't\ncompile it yourself)?  Also, can you post output of an attempt to push\nwith GIT_CURL_VERBOSE=1?\n\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"},{"id":"240737","messageId":"CAPc4eF-aT47aEPmmPPkPRfntTNdNp=c4+OK_CPdq_7YB6rxDug@mail.gmail.com","threadId":"36492","inReplyTo":"20140426174718.GC238861@vauxhall.crustytoothpaste.net","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"Ivo Bellin Salarin","fromEmail":"ivo.bellinsalarin@gmail.com","sentAt":"2014-05-05T10:21:33Z","receivedAt":"2014-05-05T10:21:33Z","isPatch":false,"sender":{"key":"ivo.bellinsalarin@gmail.com","avatar":null},"body":"Well, I'm on Windows.\nusing `git version 1.9.2.msysgit.0`.\n\nYou can find all the exchanges, recorded with wireshark, of the\nfollowing usecases:\n* git vanilla (not working),\n* VisualStudio2013 with libgit (working)\n* curl (--ntlm, working)\n* curl (--negotiate, not working)\n\nThey're available on\n[github](https://github.com/nilleb/my-documents/tree/master/msysgit%23git%2C%20issue-171).\n\n\n\nOn Sat, Apr 26, 2014 at 7:47 PM, brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> On Thu, Apr 24, 2014 at 07:17:36PM +0200, Ivo Bellin Salarin wrote:\n>> To shortly resume it, the problem is that:\n>> * when the authentication method (WWW-Authenticate) is Negotiate AND\n>> * when the server proposes a NTLMSSP_CHALLENGE in response of the\n>> client's NTLMSSP_NEGOTIATE,\n>> => libcurl yields an \"Authentication problem. Ignoring this.\\n\"\n>> And the communication is closed.\n>>\n>> At this point, in a normal communication, the client should send a\n>> NTLMSSP_AUTH containing a Kerberos ticket.\n>>\n>> Having seen the libcurl source code, I think we're passing through the\n>> lines  from 776 to 780 of\n>> [http.c](https://github.com/bagder/curl/blob/2e57c7e0fcfb9214b2a9dfa8b3da258ded013b8a/lib/http.c).\n>> Some guy, on the github issue page, has suggested that this could be\n>> related to an update of libcurl, when git was at its 1.8.2 version.\n>>\n>> I'm not debugging libcurl, and I can't reproduce this problem @home.\n>> So, has somebody already experienced the same problem? Is there a\n>> solution?\n>\n> I'm personally using Git with GSS-Negotiate (and MIT Kerberos 5) and it\n> does seem to work correctly for me.  For large pushes, your server (and\n> any intermediate proxies) will need to support 100 Continue properly, as\n> there's simply no other way to make it work.\n>\n> What version of curl are you using (and what distro if you didn't\n> compile it yourself)?  Also, can you post output of an attempt to push\n> with GIT_CURL_VERBOSE=1?\n>\n> --\n> brian m. carlson / brian with sandals: Houston, Texas, US\n> +1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\n> OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n\n\n\n-- \nhttp://www.nilleb.com\n"},{"id":"241252","messageId":"20140510210132.GD45511@vauxhall.crustytoothpaste.net","threadId":"36492","inReplyTo":"CAPc4eF-aT47aEPmmPPkPRfntTNdNp=c4+OK_CPdq_7YB6rxDug@mail.gmail.com","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-05-10T21:01:32Z","receivedAt":"2014-05-10T21:01:32Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Mon, May 05, 2014 at 12:21:33PM +0200, Ivo Bellin Salarin wrote:\n> Well, I'm on Windows.\n> using `git version 1.9.2.msysgit.0`.\n> \n> You can find all the exchanges, recorded with wireshark, of the\n> following usecases:\n> * git vanilla (not working),\n> * VisualStudio2013 with libgit (working)\n> * curl (--ntlm, working)\n> * curl (--negotiate, not working)\n\nOkay, so what it looks like is that for some reason, the server and\nlibcurl refuse to connect with Negotiate authentication.  git uses\nCURLAUTH_ANY, and libcurl picks the best choice: Negotiate.  The\ndifference between your setup and mine is that I'm using Negotiate with\nKerberos 5, and you're using Negotiate with NTLM.\n\nWhat it looks like is happening is that git is offering Negotiate data,\nand then your server is responding with a 401 Unauthorized.  libgit2\n(presumably using WinHTTP) continues in this case, retrying with a\nlonger set of credential containing more data, but git gives up.\n\nBoth responses comply with RFC 2616, by my reading.  I guess there are a\ncouple of choices here:\n\n* Make your web server happy with the data that it gets passed\n  initially.\n* Make git understand that it really needs to try again with different\n  credentials in this case (how to do that is unknown).\n* Provide some way of forcing git to use a particular authentication\n  protocol.\n\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"},{"id":"241300","messageId":"1399917719.2595.5.camel@centaur.cmartin.tk","threadId":"36492","inReplyTo":"20140510210132.GD45511@vauxhall.crustytoothpaste.net","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"Carlos Martín Nieto","fromEmail":"cmn@elego.de","sentAt":"2014-05-12T18:01:59Z","receivedAt":"2014-05-12T18:01:59Z","isPatch":false,"sender":{"key":"cmn@elego.de","avatar":"https://avatars.githubusercontent.com/u/335443?v=4"},"body":"On Sat, 2014-05-10 at 21:01 +0000, brian m. carlson wrote:\n> On Mon, May 05, 2014 at 12:21:33PM +0200, Ivo Bellin Salarin wrote:\n> > Well, I'm on Windows.\n> > using `git version 1.9.2.msysgit.0`.\n> > \n> > You can find all the exchanges, recorded with wireshark, of the\n> > following usecases:\n> > * git vanilla (not working),\n> > * VisualStudio2013 with libgit (working)\n> > * curl (--ntlm, working)\n> > * curl (--negotiate, not working)\n> \n> Okay, so what it looks like is that for some reason, the server and\n> libcurl refuse to connect with Negotiate authentication.  git uses\n> CURLAUTH_ANY, and libcurl picks the best choice: Negotiate.  The\n> difference between your setup and mine is that I'm using Negotiate with\n> Kerberos 5, and you're using Negotiate with NTLM.\n> \n> What it looks like is happening is that git is offering Negotiate data,\n> and then your server is responding with a 401 Unauthorized.  libgit2\n> (presumably using WinHTTP) continues in this case, retrying with a\n> longer set of credential containing more data, but git gives up.\n\nWhile libgit2 does use WinHTTP by default on Windows, Visual Studio\noverrides this and uses their own HTTP transport (which makes the .NET\nstack to handle it) because of the way the prefer to do things, with\njust the one persistent connection to TFS.\n\nBut details aside, the code Visual Studio uses to do authentication has\nnothing to do with any of the others.\n\n   cmn\n"},{"id":"241314","messageId":"20140512202153.GB2329@sigill.intra.peff.net","threadId":"36492","inReplyTo":"20140510210132.GD45511@vauxhall.crustytoothpaste.net","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-05-12T20:21:53Z","receivedAt":"2014-05-12T20:21:53Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, May 10, 2014 at 09:01:32PM +0000, brian m. carlson wrote:\n\n> What it looks like is happening is that git is offering Negotiate data,\n> and then your server is responding with a 401 Unauthorized.  libgit2\n> (presumably using WinHTTP) continues in this case, retrying with a\n> longer set of credential containing more data, but git gives up.\n> \n> Both responses comply with RFC 2616, by my reading.  I guess there are a\n> couple of choices here:\n> \n> * Make your web server happy with the data that it gets passed\n>   initially.\n> * Make git understand that it really needs to try again with different\n>   credentials in this case (how to do that is unknown).\n\nIt should be pretty straightforward to loop again; http_request_reauth\njust needs to turn into a for-loop on getting HTTP_REAUTH, rather than a\nstatic two-tries (I even had a patch for this a while ago, but the\nfunction has changed a bit in the interim).\n\nThe tricky part is figuring out when to return HTTP_NOAUTH (\"do not try\nagain, we failed\") versus HTTP_REAUTH (\"get credentials and try again\")\nin handle_curl_result. Right now the decision is based on \"did we have a\nusername and password for this request?\" I'm not clear on what extra\nbits would be needed to decide to continue in the case you guys are\ndiscussing.\n\n> * Provide some way of forcing git to use a particular authentication\n>   protocol.\n\nYeah, we just set CURLAUTH_ANY now, but it would be fairly trivial to\nadd \"http.authtype\" and \"http.proxyauthtype\" to map to CURLOPT_HTTPAUTH\nand CURLOPT_PROXYAUTH.\n\n-Peff\n"},{"id":"242005","messageId":"20140516223410.GB180798@vauxhall.crustytoothpaste.net","threadId":"36492","inReplyTo":"20140512202153.GB2329@sigill.intra.peff.net","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2014-05-16T22:34:10Z","receivedAt":"2014-05-16T22:34:10Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Mon, May 12, 2014 at 04:21:53PM -0400, Jeff King wrote:\n> On Sat, May 10, 2014 at 09:01:32PM +0000, brian m. carlson wrote:\n> > * Make git understand that it really needs to try again with different\n> >   credentials in this case (how to do that is unknown).\n> \n> It should be pretty straightforward to loop again; http_request_reauth\n> just needs to turn into a for-loop on getting HTTP_REAUTH, rather than a\n> static two-tries (I even had a patch for this a while ago, but the\n> function has changed a bit in the interim).\n> \n> The tricky part is figuring out when to return HTTP_NOAUTH (\"do not try\n> again, we failed\") versus HTTP_REAUTH (\"get credentials and try again\")\n> in handle_curl_result. Right now the decision is based on \"did we have a\n> username and password for this request?\" I'm not clear on what extra\n> bits would be needed to decide to continue in the case you guys are\n> discussing.\n\nI'm honestly not sure, either.  That's why I said, \"how to do that is\nunknown\".\n\nHowever, if you base64-decode the two Negotiate replies in the\nsuccessful attempt with WinHTTP and pass it through od -tc, you'll see\nthat the second reply contains some form of user ID that the first one\ndoes not.  The curl binary sends an identical reply for the first pass,\nbut then gives up and does not try a second pass.  I don't know if\nlibcurl is able to provide the data required in the second pass.\n\nAll of this is way outside my knowledge, since my Kerberos/GSSAPI\nNegotiate requests look very different than the NTLM ones.\n\n> > * Provide some way of forcing git to use a particular authentication\n> >   protocol.\n> \n> Yeah, we just set CURLAUTH_ANY now, but it would be fairly trivial to\n> add \"http.authtype\" and \"http.proxyauthtype\" to map to CURLOPT_HTTPAUTH\n> and CURLOPT_PROXYAUTH.\n\nThis might be the easiest option.\n\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187\n"},{"id":"242033","messageId":"20140517065102.GB13003@sigill.intra.peff.net","threadId":"36492","inReplyTo":"20140516223410.GB180798@vauxhall.crustytoothpaste.net","subject":"Re: GIT, libcurl and GSS-Negotiate","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2014-05-17T06:51:02Z","receivedAt":"2014-05-17T06:51:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, May 16, 2014 at 10:34:10PM +0000, brian m. carlson wrote:\n\n> > The tricky part is figuring out when to return HTTP_NOAUTH (\"do not try\n> > again, we failed\") versus HTTP_REAUTH (\"get credentials and try again\")\n> > in handle_curl_result. Right now the decision is based on \"did we have a\n> > username and password for this request?\" I'm not clear on what extra\n> > bits would be needed to decide to continue in the case you guys are\n> > discussing.\n> \n> I'm honestly not sure, either.  That's why I said, \"how to do that is\n> unknown\".\n> \n> However, if you base64-decode the two Negotiate replies in the\n> successful attempt with WinHTTP and pass it through od -tc, you'll see\n> that the second reply contains some form of user ID that the first one\n> does not.  The curl binary sends an identical reply for the first pass,\n> but then gives up and does not try a second pass.  I don't know if\n> libcurl is able to provide the data required in the second pass.\n> \n> All of this is way outside my knowledge, since my Kerberos/GSSAPI\n> Negotiate requests look very different than the NTLM ones.\n\nMine too. I think a good place to start would be somebody who has a\nsetup to replicate the problem dumping the curl data (either from\nGIT_CURL_VERBOSE, or instrumenting git with some curl_easy_getinfo\ncalls), and seeing what is interesting.\n\n> > Yeah, we just set CURLAUTH_ANY now, but it would be fairly trivial to\n> > add \"http.authtype\" and \"http.proxyauthtype\" to map to CURLOPT_HTTPAUTH\n> > and CURLOPT_PROXYAUTH.\n> \n> This might be the easiest option.\n\nI can help with working up a patch, but I don't have any meaningful way\nto test it.\n\nThe patch below might help somebody get started. I don't know if\nCURLAUTH_ONLY would be useful to be able to set, too.\n\ndiff --git a/http.c b/http.c\nindex 94e1afd..ba56f7e 100644\n--- a/http.c\n+++ b/http.c\n@@ -51,6 +51,9 @@ struct credential http_auth = CREDENTIAL_INIT;\n static int http_proactive_auth;\n static const char *user_agent;\n \n+static long curl_http_authtype;\n+static long curl_proxy_authtype;\n+\n #if LIBCURL_VERSION_NUM >= 0x071700\n /* Use CURLOPT_KEYPASSWD as is */\n #elif LIBCURL_VERSION_NUM >= 0x070903\n@@ -143,6 +146,37 @@ static void process_curl_messages(void)\n }\n #endif\n \n+static int parse_auth_type(const char *var, const char *value, long *type)\n+{\n+\tstatic struct {\n+\t\tconst char *name;\n+\t\tlong value;\n+\t} types[] = {\n+\t\t { \"basic\", CURLAUTH_BASIC },\n+\t\t { \"digest\", CURLAUTH_DIGEST },\n+#if CURL_VERSION >= 0x071303\n+\t\t { \"digest-ie\", CURLAUTH_DIGEST_IE },\n+#endif\n+\t\t { \"negotiate\", CURLAUTH_GSSNEGOTIATE },\n+#if CURL_VERSION >= 0x071600\n+\t\t { \"ntlm-wb\", CURLAUTH_NTLM_WB },\n+#endif\n+\t\t { \"ntlm\", CURLAUTH_NTLM }\n+\t};\n+\tint i;\n+\n+\tif (!value)\n+\t\treturn config_error_nonbool(var);\n+\n+\tfor (i = 0; i < ARRAY_SIZE(types); i++)\n+\t\tif (!strcmp(value, types[i].name))\n+\t\t\t*type |= types[i].value;\n+\n+\tif (i == ARRAY_SIZE(types))\n+\t\treturn error(\"unknown auth type for '%s': %s\", var, value);\n+\treturn 0;\n+}\n+\n static int http_options(const char *var, const char *value, void *cb)\n {\n \tif (!strcmp(\"http.sslverify\", var)) {\n@@ -216,6 +250,11 @@ static int http_options(const char *var, const char *value, void *cb)\n \tif (!strcmp(\"http.useragent\", var))\n \t\treturn git_config_string(&user_agent, var, value);\n \n+\tif (!strcmp(\"http.authtype\", var))\n+\t\treturn parse_auth_type(var, value, &curl_http_authtype);\n+\tif (!strcmp(\"http.proxyauthtype\", var))\n+\t\treturn parse_auth_type(var, value, &curl_proxy_authtype);\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -296,6 +335,17 @@ static void set_curl_keepalive(CURL *c)\n }\n #endif\n \n+static void set_curl_authtype(CURL *c, CURLoption option, long value)\n+{\n+\tif (value)\n+\t\tcurl_easy_setopt(c, option, value);\n+\telse {\n+#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY\n+\t\tcurl_easy_setopt(c, option, CURLAUTH_ANY);\n+#endif\n+\t}\n+}\n+\n static CURL *get_curl_handle(void)\n {\n \tCURL *result = curl_easy_init();\n@@ -313,8 +363,13 @@ static CURL *get_curl_handle(void)\n #if LIBCURL_VERSION_NUM >= 0x070907\n \tcurl_easy_setopt(result, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);\n #endif\n-#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY\n-\tcurl_easy_setopt(result, CURLOPT_HTTPAUTH, CURLAUTH_ANY);\n+\n+#if LIBCURL_VERSION_NUM >= 0x070a06\n+\tset_curl_authtype(result, CURLOPT_HTTPAUTH, curl_http_authtype);\n+#endif\n+\n+#if LIBCURL_VERSION_NUM >= 0x070a07\n+\tset_curl_authtype(result, CURLOPT_PROXYAUTH, curl_proxy_authtype);\n #endif\n \n \tif (http_proactive_auth)\n"}]}