{"thread":{"id":"41261","subject":"[PATCHv2 1/2] http: allow selection of proxy authentication method","startedAt":"2016-01-26T13:02:47Z","lastAt":"2016-01-26T18:59:33Z","messageCount":3,"participants":["Elia Pinto","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"276790","messageId":"1453813368-8990-1-git-send-email-gitter.spiros@gmail.com","threadId":"41261","inReplyTo":null,"subject":"[PATCHv2 1/2] http: allow selection of proxy authentication method","fromName":"Elia Pinto","fromEmail":"gitter.spiros@gmail.com","sentAt":"2016-01-26T13:02:47Z","receivedAt":"2016-01-26T13:02:47Z","isPatch":false,"sender":{"key":"gitter.spiros@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158490?v=4"},"body":"From: Knut Franke \n\nCURLAUTH_ANY does not work with proxies which answer unauthenticated requests\nwith a 307 redirect to an error page instead of a 407 listing supported\nauthentication methods. Therefore, allow the authentication method to be set\nusing the environment variable GIT_HTTP_PROXY_AUTHMETHOD or configuration\nvariables http.proxyAuthmethod and remote..proxyAuthmethod (in analogy\nto http.proxy and remote..proxy).\n\nThe following values are supported:\n\n* anyauth (default)\n* basic\n* digest\n* negotiate\n* ntlm\n\nSigned-off-by: Knut Franke \nSigned-off-by: Elia Pinto \nHelped-by: Junio C Hamano \nHelped-by: Eric Sunshine \nHelped-by: Elia Pinto \nSigned-off-by: Junio C Hamano \n---\nThis is the second version of the patch,\nbased on the rebase of kf/http-proxy-auth-methods on origin/next ($gmane/283734).\n\nI have tested that the fix introduced by the commit 6d7afe07\nstill works properly.\n(http://stackoverflow.com/questions/15227130/using-a-socks-proxy-with-git-for-the-http-transport#15228479)\n\nI do not know if it is correct that I add my signed off, if not free to delete it.\n\n Documentation/config.txt | 26 ++++++++++++++\n Documentation/technical/api-remote.txt | 4 +++\n http.c | 65 ++++++++++++++++++++++++++++++++--\n remote.c | 3 ++\n remote.h | 1 +\n 5 files changed, 96 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 877cbc8..0a4d41e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1614,6 +1614,27 @@ http.proxy::\n \t`curl(1)`). This can be overridden on a per-remote basis; see\n \tremote..proxy\n \n+http.proxyAuthMethod::\n+\tSet the method with which to authenticate against the HTTP proxy. This\n+\tonly takes effect if the configured proxy string contains a user name part\n+\t(i.e. is of the form 'user@host' or 'user@host:port'). This can be\n+\toverridden on a per-remote basis; see `remote..proxyAuthMethod`.\n+\tBoth can be overridden by the 'GIT_HTTP_PROXY_AUTHMETHOD' environment\n+\tvariable. Possible values are:\n++\n+--\n+* `anyauth` - Automatically pick a suitable authentication method. It is\n+ assumed that the proxy answers an unauthenticated request with a 407\n+ status code and one or more Proxy-authenticate headers with supported\n+ authentication methods. This is the default.\n+* `basic` - HTTP Basic authentication\n+* `digest` - HTTP Digest authentication; this prevents the password from being\n+ transmitted to the proxy in clear text\n+* `negotiate` - GSS-Negotiate authentication (compare the --negotiate option\n+ of `curl(1)`)\n+* `ntlm` - NTLM authentication (compare the --ntlm option of `curl(1)`)\n+--\n+\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\n@@ -2423,6 +2444,11 @@ remote..proxy::\n \tthe proxy to use for that remote. Set to the empty string to\n \tdisable proxying for that remote.\n \n+remote..proxyAuthMethod::\n+\tFor remotes that require curl (http, https and ftp), the method to use for\n+\tauthenticating against the proxy in use (probably set in\n+\t`remote..proxy`). See `http.proxyAuthMethod`.\n+\n remote..fetch::\n \tThe default set of \"refspec\" for linkgit:git-fetch[1]. See\n \tlinkgit:git-fetch[1].\ndiff --git a/Documentation/technical/api-remote.txt b/Documentation/technical/api-remote.txt\nindex 2cfdd22..f10941b 100644\n--- a/Documentation/technical/api-remote.txt\n+++ b/Documentation/technical/api-remote.txt\n@@ -51,6 +51,10 @@ struct remote\n \n \tThe proxy to use for curl (http, https, ftp, etc.) URLs.\n \n+`http_proxy_authmethod`::\n+\n+\tThe method used for authenticating against `http_proxy`.\n+\n struct remotes can be found by name with remote_get(), and iterated\n through with for_each_remote(). remote_get(NULL) will return the\n default remote, given the current branch and configuration.\ndiff --git a/http.c b/http.c\nindex 0da9e66..f46bfc4 100644\n--- a/http.c\n+++ b/http.c\n@@ -62,6 +62,24 @@ static long curl_low_speed_limit = -1;\n static long curl_low_speed_time = -1;\n static int curl_ftp_no_epsv;\n static const char *curl_http_proxy;\n+static const char *http_proxy_authmethod;\n+static struct {\n+\tconst char *name;\n+\tlong curlauth_param;\n+} proxy_authmethods[] = {\n+\t{ \"basic\", CURLAUTH_BASIC },\n+\t{ \"digest\", CURLAUTH_DIGEST },\n+\t{ \"negotiate\", CURLAUTH_GSSNEGOTIATE },\n+\t{ \"ntlm\", CURLAUTH_NTLM },\n+#ifdef LIBCURL_CAN_HANDLE_AUTH_ANY\n+\t{ \"anyauth\", CURLAUTH_ANY },\n+#endif\n+\t/*\n+\t * CURLAUTH_DIGEST_IE has no corresponding command-line option in\n+\t * curl(1) and is not included in CURLAUTH_ANY, so we leave it out\n+\t * here, too\n+\t */\n+};\n static const char *curl_cookie_file;\n static int curl_save_cookies;\n struct credential http_auth = CREDENTIAL_INIT;\n@@ -256,6 +274,9 @@ static int http_options(const char *var, const char *value, void *cb)\n \tif (!strcmp(\"http.proxy\", var))\n \t\treturn git_config_string(&curl_http_proxy, var, value);\n \n+\tif (!strcmp(\"http.proxyauthmethod\", var))\n+\t\treturn git_config_string(&http_proxy_authmethod, var, value);\n+\n \tif (!strcmp(\"http.cookiefile\", var))\n \t\treturn git_config_string(&curl_cookie_file, var, value);\n \tif (!strcmp(\"http.savecookies\", var)) {\n@@ -304,6 +325,40 @@ static void init_curl_http_auth(CURL *result)\n #endif\n }\n \n+/* *var must be free-able */\n+static void var_override(const char **var, char *value)\n+{\n+\tif (value) {\n+\t\tfree((void *)*var);\n+\t\t*var = xstrdup(value);\n+\t}\n+}\n+\n+static void init_curl_proxy_auth(CURL *result)\n+{\n+\tvar_override(&http_proxy_authmethod, getenv(\"GIT_HTTP_PROXY_AUTHMETHOD\"));\n+\n+#if LIBCURL_VERSION_NUM >= 0x070a07 /* CURLOPT_PROXYAUTH and CURLAUTH_ANY */\n+\tif (http_proxy_authmethod) {\n+\t\tint i;\n+\t\tfor (i = 0; i < ARRAY_SIZE(proxy_authmethods); i++) {\n+\t\t\tif (!strcmp(http_proxy_authmethod, proxy_authmethods[i].name)) {\n+\t\t\t\tcurl_easy_setopt(result, CURLOPT_PROXYAUTH,\n+\t\t\t\t\t\tproxy_authmethods[i].curlauth_param);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t\tif (i == ARRAY_SIZE(proxy_authmethods)) {\n+\t\t\twarning(\"unsupported proxy authentication method %s: using anyauth\",\n+\t\t\t\t\thttp_proxy_authmethod);\n+\t\t\tcurl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);\n+\t\t}\n+\t}\n+\telse\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);\n+#endif\n+}\n+\n static int has_cert_password(void)\n {\n \tif (ssl_cert == NULL || ssl_cert_password_required != 1)\n@@ -476,9 +531,7 @@ static CURL *get_curl_handle(void)\n \t\t\t\tCURLOPT_PROXYTYPE, CURLPROXY_SOCKS4);\n #endif\n \t}\n-#if LIBCURL_VERSION_NUM >= 0x070a07\n-\tcurl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);\n-#endif\n+\tinit_curl_proxy_auth(result);\n \n \tset_curl_keepalive(result);\n \n@@ -519,6 +572,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote && remote->http_proxy)\n \t\tcurl_http_proxy = xstrdup(remote->http_proxy);\n \n+\tif (remote)\n+\t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n+\n \tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n \tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n \n@@ -617,6 +673,9 @@ void http_cleanup(void)\n \t\tcurl_http_proxy = NULL;\n \t}\n \n+\tfree((void *)http_proxy_authmethod);\n+\thttp_proxy_authmethod = NULL;\n+\n \tif (cert_auth.password != NULL) {\n \t\tmemset(cert_auth.password, 0, strlen(cert_auth.password));\n \t\tfree(cert_auth.password);\ndiff --git a/remote.c b/remote.c\nindex 9d34b5a..0bb5a69 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -428,6 +428,9 @@ static int handle_config(const char *key, const char *value, void *cb)\n \t} else if (!strcmp(subkey, \".proxy\")) {\n \t\treturn git_config_string((const char **)&remote->http_proxy,\n \t\t\t\t\t key, value);\n+\t} else if (!strcmp(subkey, \".proxyauthmethod\")) {\n+\t\treturn git_config_string((const char **)&remote->http_proxy_authmethod,\n+\t\t\t\t\t key, value);\n \t} else if (!strcmp(subkey, \".vcs\")) {\n \t\treturn git_config_string(&remote->foreign_vcs, key, value);\n \t}\ndiff --git a/remote.h b/remote.h\nindex 4a039ba..4fd7a0f 100644\n--- a/remote.h\n+++ b/remote.h\n@@ -54,6 +54,7 @@ struct remote {\n \t * for curl remotes only\n \t */\n \tchar *http_proxy;\n+\tchar *http_proxy_authmethod;\n };\n \n struct remote *remote_get(const char *name);\n-- \n2.7.0.77.gc864936\n"},{"id":"276791","messageId":"1453813368-8990-2-git-send-email-gitter.spiros@gmail.com","threadId":"41261","inReplyTo":"1453813368-8990-1-git-send-email-gitter.spiros@gmail.com","subject":"[PATCHv2 2/2] http: use credential API to handle proxy authentication","fromName":"Elia Pinto","fromEmail":"gitter.spiros@gmail.com","sentAt":"2016-01-26T13:02:48Z","receivedAt":"2016-01-26T13:02:48Z","isPatch":false,"sender":{"key":"gitter.spiros@gmail.com","avatar":"https://avatars.githubusercontent.com/u/158490?v=4"},"body":"From: Knut Franke \n\nCurrently, the only way to pass proxy credentials to curl is by including them\nin the proxy URL. Usually, this means they will end up on disk unencrypted, one\nway or another (by inclusion in ~/.gitconfig, shell profile or history). Since\nproxy authentication often uses a domain user, credentials can be security\nsensitive; therefore, a safer way of passing credentials is desirable.\n\nIf the configured proxy contains a username but not a password, query the\ncredential API for one. Also, make sure we approve/reject proxy credentials\nproperly.\n\nFor consistency reasons, add parsing of http_proxy/https_proxy/all_proxy\nenvironment variables, which would otherwise be evaluated as a fallback by curl.\nWithout this, we would have different semantics for git configuration and\nenvironment variables.\n\nHelped-by: Junio C Hamano \nHelped-by: Eric Sunshine \nHelped-by: Elia Pinto \nSigned-off-by: Knut Franke \nSigned-off-by: Elia Pinto \nSigned-off-by: Junio C Hamano \n---\nThis is the second version of the patch,\nbased on the rebase of kf/http-proxy-auth-methods on origin/next ($gmane/283734).\n\nI have tested that the fix introduced by the commit 6d7afe07 \nstill works properly. \n(http://stackoverflow.com/questions/15227130/using-a-socks-proxy-with-git-for-the-http-transport#15228479)\n\nI do not know if it is correct that I add my signed off, if not free to delete it.\n\n Documentation/config.txt | 10 +++++--\n http.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++\n http.h | 1 +\n 3 files changed, 85 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 0a4d41e..02bcde6 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1610,9 +1610,13 @@ help.htmlPath::\n \n http.proxy::\n \tOverride the HTTP proxy, normally configured using the 'http_proxy',\n-\t'https_proxy', and 'all_proxy' environment variables (see\n-\t`curl(1)`). This can be overridden on a per-remote basis; see\n-\tremote..proxy\n+\t'https_proxy', and 'all_proxy' environment variables (see `curl(1)`). In\n+\taddition to the syntax understood by curl, it is possible to specify a\n+\tproxy string with a user name but no password, in which case git will\n+\tattempt to acquire one in the same way it does for other credentials. See\n+\tlinkgit:gitcredentials[7] for more information. The syntax thus is\n+\t'[protocol://][user[:password]@]proxyhost[:port]'. This can be overridden\n+\ton a per-remote basis; see remote..proxy\n \n http.proxyAuthMethod::\n \tSet the method with which to authenticate against the HTTP proxy. This\ndiff --git a/http.c b/http.c\nindex f46bfc4..dfc53c1 100644\n--- a/http.c\n+++ b/http.c\n@@ -80,6 +80,8 @@ static struct {\n \t * here, too\n \t */\n };\n+static struct credential proxy_auth = CREDENTIAL_INIT;\n+static const char *curl_proxyuserpwd;\n static const char *curl_cookie_file;\n static int curl_save_cookies;\n struct credential http_auth = CREDENTIAL_INIT;\n@@ -177,6 +179,9 @@ static void finish_active_slot(struct active_request_slot *slot)\n #else\n \t\tslot->results->auth_avail = 0;\n #endif\n+\n+\t\tcurl_easy_getinfo(slot->curl, CURLINFO_HTTP_CONNECTCODE,\n+\t\t\t&slot->results->http_connectcode);\n \t}\n \n \t/* Run callback if appropriate */\n@@ -334,8 +339,32 @@ static void var_override(const char **var, char *value)\n \t}\n }\n \n+static void set_proxyauth_name_password(CURL *result)\n+{\n+#if LIBCURL_VERSION_NUM >= 0x071301\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXYUSERNAME,\n+\t\t\tproxy_auth.username);\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXYPASSWORD,\n+\t\t\tproxy_auth.password);\n+#else\n+\t\tstruct strbuf s = STRBUF_INIT;\n+\n+\t\tstrbuf_addstr_urlencode(&s, proxy_auth.username, 1);\n+\t\tstrbuf_addch(&s, ':');\n+\t\tstrbuf_addstr_urlencode(&s, proxy_auth.password, 1);\n+\t\tcurl_proxyuserpwd = strbuf_detach(&s, NULL);\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXYUSERPWD, curl_proxyuserpwd);\n+#endif\n+}\n+\n static void init_curl_proxy_auth(CURL *result)\n {\n+\tif (proxy_auth.username) {\n+\t\tif (!proxy_auth.password)\n+\t\t\tcredential_fill(&proxy_auth);\n+\t\tset_proxyauth_name_password(result);\n+\t}\n+\n \tvar_override(&http_proxy_authmethod, getenv(\"GIT_HTTP_PROXY_AUTHMETHOD\"));\n \n #if LIBCURL_VERSION_NUM >= 0x070a07 /* CURLOPT_PROXYAUTH and CURLAUTH_ANY */\n@@ -517,6 +546,31 @@ static CURL *get_curl_handle(void)\n \t\tcurl_easy_setopt(result, CURLOPT_USE_SSL, CURLUSESSL_TRY);\n #endif\n \n+\t/*\n+\t * CURL also examines these variables as a fallback; but we need to query\n+\t * them here in order to decide whether to prompt for missing password (cf.\n+\t * init_curl_proxy_auth()).\n+\t *\n+\t * Unlike many other common environment variables, these are historically\n+\t * lowercase only. It appears that CURL did not know this and implemented\n+\t * only uppercase variants, which was later corrected to take both - with\n+\t * the exception of http_proxy, which is lowercase only also in CURL. As\n+\t * the lowercase versions are the historical quasi-standard, they take\n+\t * precedence here, as in CURL.\n+\t */\n+\tif (!curl_http_proxy) {\n+\t\tif (!strcmp(http_auth.protocol, \"https\")) {\n+\t\t\tvar_override(&curl_http_proxy, getenv(\"HTTPS_PROXY\"));\n+\t\t\tvar_override(&curl_http_proxy, getenv(\"https_proxy\"));\n+\t\t} else {\n+\t\t\tvar_override(&curl_http_proxy, getenv(\"http_proxy\"));\n+\t\t}\n+\t\tif (!curl_http_proxy) {\n+\t\t\tvar_override(&curl_http_proxy, getenv(\"ALL_PROXY\"));\n+\t\t\tvar_override(&curl_http_proxy, getenv(\"all_proxy\"));\n+\t\t}\n+\t}\n+\n \tif (curl_http_proxy) {\n \t\tcurl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);\n #if LIBCURL_VERSION_NUM >= 0x071800\n@@ -530,6 +584,16 @@ static CURL *get_curl_handle(void)\n \t\t\tcurl_easy_setopt(result,\n \t\t\t\tCURLOPT_PROXYTYPE, CURLPROXY_SOCKS4);\n #endif\n+\t\tif (strstr(curl_http_proxy, \"://\"))\n+\t\t\tcredential_from_url(&proxy_auth, curl_http_proxy);\n+\t\telse {\n+\t\t\tstruct strbuf url = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&url, \"http://%s\", curl_http_proxy);\n+\t\t\tcredential_from_url(&proxy_auth, url.buf);\n+\t\t\tstrbuf_release(&url);\n+\t\t}\n+\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXY, proxy_auth.host);\n \t}\n \tinit_curl_proxy_auth(result);\n \n@@ -673,6 +737,15 @@ void http_cleanup(void)\n \t\tcurl_http_proxy = NULL;\n \t}\n \n+\tif (proxy_auth.password) {\n+\t\tmemset(proxy_auth.password, 0, strlen(proxy_auth.password));\n+\t\tfree(proxy_auth.password);\n+\t\tproxy_auth.password = NULL;\n+\t}\n+\n+\tfree((void *)curl_proxyuserpwd);\n+\tcurl_proxyuserpwd = NULL;\n+\n \tfree((void *)http_proxy_authmethod);\n \thttp_proxy_authmethod = NULL;\n \n@@ -1005,6 +1078,8 @@ static int handle_curl_result(struct slot_results *results)\n \n \tif (results->curl_result == CURLE_OK) {\n \t\tcredential_approve(&http_auth);\n+\t\tif (proxy_auth.password)\n+\t\t\tcredential_approve(&proxy_auth);\n \t\treturn HTTP_OK;\n \t} else if (missing_target(results))\n \t\treturn HTTP_MISSING_TARGET;\n@@ -1019,6 +1094,8 @@ static int handle_curl_result(struct slot_results *results)\n \t\t\treturn HTTP_REAUTH;\n \t\t}\n \t} else {\n+\t\tif (results->http_connectcode == 407)\n+\t\t\tcredential_reject(&proxy_auth);\n #if LIBCURL_VERSION_NUM >= 0x070c00\n \t\tif (!curl_errorstr[0])\n \t\t\tstrlcpy(curl_errorstr,\ndiff --git a/http.h b/http.h\nindex 4f97b60..f83cfa6 100644\n--- a/http.h\n+++ b/http.h\n@@ -54,6 +54,7 @@ struct slot_results {\n \tCURLcode curl_result;\n \tlong http_code;\n \tlong auth_avail;\n+\tlong http_connectcode;\n };\n \n struct active_request_slot {\n-- \n2.7.0.77.gc864936\n"},{"id":"276826","messageId":"xmqqfuxkcgkq.fsf@gitster.mtv.corp.google.com","threadId":"41261","inReplyTo":"1453813368-8990-2-git-send-email-gitter.spiros@gmail.com","subject":"Re: [PATCHv2 2/2] http: use credential API to handle proxy authentication","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-26T18:59:33Z","receivedAt":"2016-01-26T18:59:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thanks, will queue.\n"}]}