{"thread":{"id":"42138","subject":"[PATCH] http: Support sending custom HTTP headers","startedAt":"2016-04-25T13:13:08Z","lastAt":"2016-05-16T13:35:33Z","messageCount":119,"participants":["Johannes Schindelin","Shawn Pearce","Jeff King","Junio C Hamano","Jacob Keller","Stefan Beller","Lars Schneider","Torsten Bögershausen"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"284396","messageId":"abe253758829795c285c2036196ebe7edd9bab34.1461589951.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":null,"subject":"[PATCH] http: Support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-25T13:13:08Z","receivedAt":"2016-04-25T13:13:08Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To make communication for `git fetch`, `git ls-remote` and friends extra\nsecure, we introduce a way to send custom HTTP headers with all\nrequests.\n\nThis allows us, for example, to send an extra token that the server\ntests for. The server could use this token e.g. to ensure that only\ncertain operations or refs are allowed, or allow the token to be used\nonly once.\n\nThis feature can be used like this:\n\n\tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nSigned-off-by: Johannes Schindelin \nPublished-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v1\n---\n http-push.c | 10 +++++-----\n http.c | 28 +++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n 4 files changed, 33 insertions(+), 10 deletions(-)\n\ndiff --git a/http-push.c b/http-push.c\nindex bd60668..04eef17 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_get_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_get_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_get_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_get_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n@@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_get_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\ndiff --git a/http.c b/http.c\nindex 4304b80..02d7147 100644\n--- a/http.c\n+++ b/http.c\n@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n \n static struct curl_slist *pragma_header;\n static struct curl_slist *no_pragma_header;\n+static struct curl_slist *extra_http_headers;\n \n static struct active_request_slot *active_queue_head;\n \n@@ -323,6 +324,12 @@ static int http_options(const char *var, const char *value, void *cb)\n #endif\n \t}\n \n+\tif (!strcmp(\"http.extraheader\", var)) {\n+\t\textra_http_headers =\n+\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\treturn 0;\n+\t}\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -678,8 +685,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n-\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n+\tpragma_header = curl_slist_append(http_get_default_headers(),\n+\t\t\"Pragma: no-cache\");\n+\tno_pragma_header = curl_slist_append(http_get_default_headers(),\n+\t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n \t{\n@@ -765,6 +774,9 @@ void http_cleanup(void)\n #endif\n \tcurl_global_cleanup();\n \n+\tcurl_slist_free_all(extra_http_headers);\n+\textra_http_headers = NULL;\n+\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n@@ -1163,6 +1175,16 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n+struct curl_slist *http_get_default_headers()\n+{\n+\tstruct curl_slist *headers = NULL, *h;\n+\n+\tfor (h = extra_http_headers; h; h = h->next)\n+\t\theaders = curl_slist_append(headers, h->data);\n+\n+\treturn headers;\n+}\n+\n static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n {\n \tchar *ptr;\n@@ -1380,7 +1402,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_get_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\ndiff --git a/http.h b/http.h\nindex 4ef4bbd..b0927de 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,6 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n+extern struct curl_slist *http_get_default_headers();\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 15e48e2..86ba787 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_get_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_get_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\n-- \n2.8.1.306.gff998f2\n"},{"id":"284430","messageId":"CAJo=hJucHFFxDeChwWNy7KQ2mbn21WBsJ5eG_S3W4R5z5CGqfg@mail.gmail.com","threadId":"42138","inReplyTo":"abe253758829795c285c2036196ebe7edd9bab34.1461589951.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2016-04-25T15:53:01Z","receivedAt":"2016-04-25T15:53:01Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Mon, Apr 25, 2016 at 6:13 AM, Johannes Schindelin\n wrote:\n> To make communication for `git fetch`, `git ls-remote` and friends extra\n> secure, we introduce a way to send custom HTTP headers with all\n> requests.\n\nHmm. Its not Apr 1 2016. So I guess you are serious. :)\n\n> This allows us, for example, to send an extra token that the server\n> tests for. The server could use this token e.g. to ensure that only\n> certain operations or refs are allowed, or allow the token to be used\n> only once.\n>\n> This feature can be used like this:\n>\n> git -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nIts not very secure to be adding secure data to the command line, e.g.\non Linux you can see that data in /proc.\n"},{"id":"284448","messageId":"20160425170326.GA1551@sigill.intra.peff.net","threadId":"42138","inReplyTo":"abe253758829795c285c2036196ebe7edd9bab34.1461589951.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-25T17:03:26Z","receivedAt":"2016-04-25T17:03:26Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Apr 25, 2016 at 03:13:08PM +0200, Johannes Schindelin wrote:\n\n> diff --git a/http.c b/http.c\n> index 4304b80..02d7147 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n> \n> static struct curl_slist *pragma_header;\n> static struct curl_slist *no_pragma_header;\n> +static struct curl_slist *extra_http_headers;\n> \n> static struct active_request_slot *active_queue_head;\n> \n> @@ -323,6 +324,12 @@ static int http_options(const char *var, const char *value, void *cb)\n> #endif\n> \t}\n> \n> +\tif (!strcmp(\"http.extraheader\", var)) {\n> +\t\textra_http_headers =\n> +\t\t\tcurl_slist_append(extra_http_headers, value);\n> +\t\treturn 0;\n> +\t}\n\nI wondered if this would trigger for \"http.*.extraheader\", too. And it\nshould, as that is all handled in the caller of http_options. Good.\n\n> @@ -678,8 +685,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n> \tif (remote)\n> \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n> \n> -\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n> -\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n> +\tpragma_header = curl_slist_append(http_get_default_headers(),\n> +\t\t\"Pragma: no-cache\");\n> +\tno_pragma_header = curl_slist_append(http_get_default_headers(),\n> +\t\t\"Pragma:\");\n\nThis looked wrong to me at first, because we are appending to the\ndefault header list in each case. But the secret sauce is that calling\nhttp_get_default_headers() actually creates a _new_ list that is a copy\nof the default headers (and the caller can do what they will with it,\nand must free it).\n\nI think that's really the only sane way to do it because of curl's\ninterfaces. But maybe it is worth a comment either here, or along with\nhttp_get_default_headers(), or both.\n\n-Peff\n"},{"id":"284466","messageId":"xmqq7fflleau.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"abe253758829795c285c2036196ebe7edd9bab34.1461589951.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-25T18:43:37Z","receivedAt":"2016-04-25T18:43:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> To make communication for `git fetch`, `git ls-remote` and friends extra\n> secure, we introduce a way to send custom HTTP headers with all\n> requests.\n\nI think an ability to send custom headers may be a good addition and\nhave no problem with it, but I tend to agree with Shawn that its log\nmessage that advertises it as if it has anything to do with security\nis probably a bad idea in both ways (i.e. it isn't very secure, and\nthe usefulness of the feature is not limited to security).\n\n> This allows us, for example, to send an extra token that the server\n> tests for. The server could use this token e.g. to ensure that only\n> certain operations or refs are allowed, or allow the token to be used\n> only once.\n>\n> This feature can be used like this:\n>\n> \tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n>\n> Signed-off-by: Johannes Schindelin \n\n\n> Published-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v1\n\nMove this after \"---\".\n\n> ---\n\nThis obviously needs documentation updates and tests, no?\n\n> http-push.c | 10 +++++-----\n> http.c | 28 +++++++++++++++++++++++++---\n> http.h | 1 +\n> remote-curl.c | 4 ++--\n> 4 files changed, 33 insertions(+), 10 deletions(-)\n>\n> diff --git a/http-push.c b/http-push.c\n> index bd60668..04eef17 100644\n> --- a/http-push.c\n> +++ b/http-push.c\n> @@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n> static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n> {\n> \tstruct strbuf buf = STRBUF_INIT;\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_get_default_headers();\n> \n> \tif (options & DAV_HEADER_IF) {\n> \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n> @@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n> static void start_move(struct transfer_request *request)\n> {\n> \tstruct active_request_slot *slot;\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_get_default_headers();\n> \n> \tslot = get_active_slot();\n> \tslot->callback_func = process_response;\n> @@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n> \tchar *ep;\n> \tchar timeout_header[25];\n> \tstruct remote_lock *lock = NULL;\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_get_default_headers();\n> \tstruct xml_ctx ctx;\n> \tchar *escaped;\n> \n> @@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n> \tstruct slot_results results;\n> \tstruct strbuf in_buffer = STRBUF_INIT;\n> \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_get_default_headers();\n> \tstruct xml_ctx ctx;\n> \tstruct remote_ls_ctx ls;\n> \n> @@ -1204,7 +1204,7 @@ static int locking_available(void)\n> \tstruct slot_results results;\n> \tstruct strbuf in_buffer = STRBUF_INIT;\n> \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_get_default_headers();\n> \tstruct xml_ctx ctx;\n> \tint lock_flags = 0;\n> \tchar *escaped;\n> diff --git a/http.c b/http.c\n> index 4304b80..02d7147 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n> \n> static struct curl_slist *pragma_header;\n> static struct curl_slist *no_pragma_header;\n> +static struct curl_slist *extra_http_headers;\n> \n> static struct active_request_slot *active_queue_head;\n> \n> @@ -323,6 +324,12 @@ static int http_options(const char *var, const char *value, void *cb)\n> #endif\n> \t}\n> \n> +\tif (!strcmp(\"http.extraheader\", var)) {\n> +\t\textra_http_headers =\n> +\t\t\tcurl_slist_append(extra_http_headers, value);\n> +\t\treturn 0;\n> +\t}\n> +\n> \t/* Fall back on the default ones */\n> \treturn git_default_config(var, value, cb);\n> }\n> @@ -678,8 +685,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n> \tif (remote)\n> \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n> \n> -\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n> -\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n> +\tpragma_header = curl_slist_append(http_get_default_headers(),\n> +\t\t\"Pragma: no-cache\");\n> +\tno_pragma_header = curl_slist_append(http_get_default_headers(),\n> +\t\t\"Pragma:\");\n> \n> #ifdef USE_CURL_MULTI\n> \t{\n> @@ -765,6 +774,9 @@ void http_cleanup(void)\n> #endif\n> \tcurl_global_cleanup();\n> \n> +\tcurl_slist_free_all(extra_http_headers);\n> +\textra_http_headers = NULL;\n> +\n> \tcurl_slist_free_all(pragma_header);\n> \tpragma_header = NULL;\n> \n> @@ -1163,6 +1175,16 @@ int run_one_slot(struct active_request_slot *slot,\n> \treturn handle_curl_result(results);\n> }\n> \n> +struct curl_slist *http_get_default_headers()\n> +{\n> +\tstruct curl_slist *headers = NULL, *h;\n> +\n> +\tfor (h = extra_http_headers; h; h = h->next)\n> +\t\theaders = curl_slist_append(headers, h->data);\n> +\n> +\treturn headers;\n> +}\n> +\n> static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n> {\n> \tchar *ptr;\n> @@ -1380,7 +1402,7 @@ static int http_request(const char *url,\n> {\n> \tstruct active_request_slot *slot;\n> \tstruct slot_results results;\n> -\tstruct curl_slist *headers = NULL;\n> +\tstruct curl_slist *headers = http_get_default_headers();\n> \tstruct strbuf buf = STRBUF_INIT;\n> \tconst char *accept_language;\n> \tint ret;\n> diff --git a/http.h b/http.h\n> index 4ef4bbd..b0927de 100644\n> --- a/http.h\n> +++ b/http.h\n> @@ -106,6 +106,7 @@ extern void step_active_slots(void);\n> extern void http_init(struct remote *remote, const char *url,\n> \t\t int proactive_auth);\n> extern void http_cleanup(void);\n> +extern struct curl_slist *http_get_default_headers();\n> \n> extern long int git_curl_ipresolve;\n> extern int active_requests;\n> diff --git a/remote-curl.c b/remote-curl.c\n> index 15e48e2..86ba787 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n> static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n> {\n> \tstruct active_request_slot *slot;\n> -\tstruct curl_slist *headers = NULL;\n> +\tstruct curl_slist *headers = http_get_default_headers();\n> \tstruct strbuf buf = STRBUF_INIT;\n> \tint err;\n> \n> @@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n> static int post_rpc(struct rpc_state *rpc)\n> {\n> \tstruct active_request_slot *slot;\n> -\tstruct curl_slist *headers = NULL;\n> +\tstruct curl_slist *headers = http_get_default_headers();\n> \tint use_gzip = rpc->gzip_request;\n> \tchar *gzip_body = NULL;\n> \tsize_t gzip_size = 0;\n"},{"id":"284526","messageId":"alpine.DEB.2.20.1604260851390.2896@virtualbox","threadId":"42138","inReplyTo":"xmqq7fflleau.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-26T15:33:33Z","receivedAt":"2016-04-26T15:33:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Mon, 25 Apr 2016, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > To make communication for `git fetch`, `git ls-remote` and friends\n> > extra secure, we introduce a way to send custom HTTP headers with all\n> > requests.\n> \n> I think an ability to send custom headers may be a good addition and\n> have no problem with it, but I tend to agree with Shawn that its log\n> message that advertises it as if it has anything to do with security is\n> probably a bad idea in both ways (i.e. it isn't very secure, and the\n> usefulness of the feature is not limited to security).\n\nYou know, it never occurred to me that anybody could even *think* that I\nwas talking about the security of the client setup.\n\nYou see, it is much easier to read $HOME/.netrc than /proc/, especially if\nyou are looking outside of Linux, where the proc filesystem does not even\nexist. And it is almost as easy to query the credential helper for a\nplain text password as looking at $HOME/.netrc.\n\nSo I took it for granted that everybody knows that they have to keep their\nown computer safe.\n\nInstead, I was thinking of server side security (with the clear\nexpectation that the users will keep their client setups secure).\n\nI will rephrase the commit message to describe the actual use case I have\nhere: build agents need temporary access to private repositories, and I'd\nlike to do that via sort of One-Time-Passwords, sent as additional HTTP\nheaders (via HTTPS, I should not need to point out, but now feel I have to\nspell out).\n\n> > Published-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v1\n> \n> Move this after \"---\".\n\nWhoops. That is what I intended, but overlooked. Will fix.\n\n> This obviously needs documentation updates and tests, no?\n\nDocumentation, yes. I have that already, but somehow it slipped out of the\npatch.\n\nTesting the headers? I dunno, do we have tests for that already? I thought\nwe did not: it requires an HTTP server (so that the headers are actually\nsent) that we can force to check the header...\n\nSo I see we have some tests that use Apache, and one that uses our own\nhttp-backend. But is there already anything that logs HTTP requests? I did\nnot think so, please correct me if I am wrong.\n\nCiao,\nDscho\n"},{"id":"284527","messageId":"alpine.DEB.2.20.1604261733410.2896@virtualbox","threadId":"42138","inReplyTo":"20160425170326.GA1551@sigill.intra.peff.net","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-26T15:37:32Z","receivedAt":"2016-04-26T15:37:32Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Mon, 25 Apr 2016, Jeff King wrote:\n\n> On Mon, Apr 25, 2016 at 03:13:08PM +0200, Johannes Schindelin wrote:\n> \n> > diff --git a/http.c b/http.c\n> > index 4304b80..02d7147 100644\n> > --- a/http.c\n> > +++ b/http.c\n> > @@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n> > \n> > static struct curl_slist *pragma_header;\n> > static struct curl_slist *no_pragma_header;\n> > +static struct curl_slist *extra_http_headers;\n> > \n> > static struct active_request_slot *active_queue_head;\n> > \n> > @@ -323,6 +324,12 @@ static int http_options(const char *var, const char *value, void *cb)\n> > #endif\n> > \t}\n> > \n> > +\tif (!strcmp(\"http.extraheader\", var)) {\n> > +\t\textra_http_headers =\n> > +\t\t\tcurl_slist_append(extra_http_headers, value);\n> > +\t\treturn 0;\n> > +\t}\n> \n> I wondered if this would trigger for \"http.*.extraheader\", too. And it\n> should, as that is all handled in the caller of http_options. Good.\n\nYes, I was surprised about that, too, but all the other http.* settings\nare handled via the urlmatch mechanism (which rewrites the matching\nhttp..* settings).\n\n> > @@ -678,8 +685,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n> > \tif (remote)\n> > \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n> > \n> > -\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n> > -\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n> > +\tpragma_header = curl_slist_append(http_get_default_headers(),\n> > +\t\t\"Pragma: no-cache\");\n> > +\tno_pragma_header = curl_slist_append(http_get_default_headers(),\n> > +\t\t\"Pragma:\");\n> \n> This looked wrong to me at first, because we are appending to the\n> default header list in each case. But the secret sauce is that calling\n> http_get_default_headers() actually creates a _new_ list that is a copy\n> of the default headers (and the caller can do what they will with it,\n> and must free it).\n> \n> I think that's really the only sane way to do it because of curl's\n> interfaces. But maybe it is worth a comment either here, or along with\n> http_get_default_headers(), or both.\n\nI chose to rename it to http_copy_default_headers(); That should make it\neasier to understand.\n\nCiao,\nDscho\n"},{"id":"284528","messageId":"921e007f445476b27325c12a9e92fdd169a073b7.1461685158.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"abe253758829795c285c2036196ebe7edd9bab34.1461589951.git.johannes.schindelin@gmx.de","subject":"[PATCH v2] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-26T15:40:30Z","receivedAt":"2016-04-26T15:40:30Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"We introduce a way to send custom HTTP headers with all requests.\n\nThis allows us, for example, to send an extra token from build agents\nfor temporary access to private repositories. (This is the use case that\ntriggered this patch.)\n\nThis feature can be used like this:\n\n\tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nAs `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` overrides previous\ncalls' headers (instead of appending the headers, as this unsuspecting\ndeveloper thought initially), we piggyback onto the `Pragma:` setting by\ndefault, and introduce the global helper `http_copy_default_headers()`\nto help functions that want to specify HTTP headers themselves.\n\nSigned-off-by: Johannes Schindelin \n---\n\nPublished-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v2\n\n Documentation/config.txt | 6 ++++++\n http-push.c | 10 +++++-----\n http.c | 28 +++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n 5 files changed, 39 insertions(+), 10 deletions(-)\n\nInterdiff vs v1:\n\n diff --git a/Documentation/config.txt b/Documentation/config.txt\n index 42d2b50..37b9af7 100644\n --- a/Documentation/config.txt\n +++ b/Documentation/config.txt\n @@ -1655,6 +1655,12 @@ http.emptyAuth::\n \ta username in the URL, as libcurl normally requires a username for\n \tauthentication.\n \n +http.extraHeader::\n +\tPass an additional HTTP header when communicating with a server. If\n +\tmore than one such entry exists, all of them are added as extra headers.\n +\tThis feature is useful e.g. to increase security, or to allow\n +\ttime-limited access based on expiring tokens.\n +\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\n diff --git a/http-push.c b/http-push.c\n index 04eef17..ae2b7f1 100644\n --- a/http-push.c\n +++ b/http-push.c\n @@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n -\tstruct curl_slist *dav_headers = http_get_default_headers();\n +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n @@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n -\tstruct curl_slist *dav_headers = http_get_default_headers();\n +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n @@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n -\tstruct curl_slist *dav_headers = http_get_default_headers();\n +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n @@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n -\tstruct curl_slist *dav_headers = http_get_default_headers();\n +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n @@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n -\tstruct curl_slist *dav_headers = http_get_default_headers();\n +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\n diff --git a/http.c b/http.c\n index 02d7147..3d662bb 100644\n --- a/http.c\n +++ b/http.c\n @@ -685,9 +685,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n -\tpragma_header = curl_slist_append(http_get_default_headers(),\n +\tpragma_header = curl_slist_append(http_copy_default_headers(),\n \t\t\"Pragma: no-cache\");\n -\tno_pragma_header = curl_slist_append(http_get_default_headers(),\n +\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n \t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n @@ -1175,7 +1175,7 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n -struct curl_slist *http_get_default_headers()\n +struct curl_slist *http_copy_default_headers()\n {\n \tstruct curl_slist *headers = NULL, *h;\n \n @@ -1402,7 +1402,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n -\tstruct curl_slist *headers = http_get_default_headers();\n +\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\n diff --git a/http.h b/http.h\n index b0927de..5f13695 100644\n --- a/http.h\n +++ b/http.h\n @@ -106,7 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n -extern struct curl_slist *http_get_default_headers();\n +extern struct curl_slist *http_copy_default_headers();\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\n diff --git a/remote-curl.c b/remote-curl.c\n index 86ba787..672b382 100644\n --- a/remote-curl.c\n +++ b/remote-curl.c\n @@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n -\tstruct curl_slist *headers = http_get_default_headers();\n +\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n @@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n -\tstruct curl_slist *headers = http_get_default_headers();\n +\tstruct curl_slist *headers = http_copy_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\n\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 42d2b50..37b9af7 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1655,6 +1655,12 @@ http.emptyAuth::\n \ta username in the URL, as libcurl normally requires a username for\n \tauthentication.\n \n+http.extraHeader::\n+\tPass an additional HTTP header when communicating with a server. If\n+\tmore than one such entry exists, all of them are added as extra headers.\n+\tThis feature is useful e.g. to increase security, or to allow\n+\ttime-limited access based on expiring tokens.\n+\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\ndiff --git a/http-push.c b/http-push.c\nindex bd60668..ae2b7f1 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n@@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\ndiff --git a/http.c b/http.c\nindex 4304b80..3d662bb 100644\n--- a/http.c\n+++ b/http.c\n@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n \n static struct curl_slist *pragma_header;\n static struct curl_slist *no_pragma_header;\n+static struct curl_slist *extra_http_headers;\n \n static struct active_request_slot *active_queue_head;\n \n@@ -323,6 +324,12 @@ static int http_options(const char *var, const char *value, void *cb)\n #endif\n \t}\n \n+\tif (!strcmp(\"http.extraheader\", var)) {\n+\t\textra_http_headers =\n+\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\treturn 0;\n+\t}\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -678,8 +685,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n-\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n+\tpragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma: no-cache\");\n+\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n \t{\n@@ -765,6 +774,9 @@ void http_cleanup(void)\n #endif\n \tcurl_global_cleanup();\n \n+\tcurl_slist_free_all(extra_http_headers);\n+\textra_http_headers = NULL;\n+\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n@@ -1163,6 +1175,16 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n+struct curl_slist *http_copy_default_headers()\n+{\n+\tstruct curl_slist *headers = NULL, *h;\n+\n+\tfor (h = extra_http_headers; h; h = h->next)\n+\t\theaders = curl_slist_append(headers, h->data);\n+\n+\treturn headers;\n+}\n+\n static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n {\n \tchar *ptr;\n@@ -1380,7 +1402,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\ndiff --git a/http.h b/http.h\nindex 4ef4bbd..5f13695 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,6 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n+extern struct curl_slist *http_copy_default_headers();\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 15e48e2..672b382 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\n-- \n2.8.1.306.gff998f2\n"},{"id":"284533","messageId":"xmqqeg9sibl7.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604260851390.2896@virtualbox","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-26T16:22:44Z","receivedAt":"2016-04-26T16:22:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Testing the headers? I dunno, do we have tests for that already? I thought\n> we did not: it requires an HTTP server (so that the headers are actually\n> sent) that we can force to check the header...\n>\n> So I see we have some tests that use Apache, and one that uses our own\n> http-backend. But is there already anything that logs HTTP requests? I did\n> not think so, please correct me if I am wrong.\n\nI suspect that no codepath in the current system has cared about\nwhat http headers are sent; auth stuff might have but even then I\nwould imagine that a test for auth would observe the end result\n(i.e. it would ask \"did the server accept or reject us?\") not the\nmechanism (i.e. it would not ask \"did we correctly send an\nAuthorization header?\").\n\nSo I wouldn't be surprised if this topic is the first one that cares\nexactly what headers are sent out (eh, rather, \"we told Git to send\nthis and that header, do they appear at the server end?\"), in which\ncase it is very likely that we do not have any existing test that\ncan be imitated for that purpose X-<.\n\nIn other words, the answer to \"Do we already have a test so that I\ncan mimick it instead of thinking of a way to test this?\" would\nprobably be \"No\".\n\nDo we care about this feature deeply enough to devise a mechanism\nto prevent it from getting broken by careless others in the future?\n"},{"id":"284540","messageId":"20160426165726.GA6572@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604261733410.2896@virtualbox","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-26T16:57:26Z","receivedAt":"2016-04-26T16:57:26Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 26, 2016 at 05:37:32PM +0200, Johannes Schindelin wrote:\n\n> > I think that's really the only sane way to do it because of curl's\n> > interfaces. But maybe it is worth a comment either here, or along with\n> > http_get_default_headers(), or both.\n> \n> I chose to rename it to http_copy_default_headers(); That should make it\n> easier to understand.\n\nThanks, I think that makes it clearer.\n\n-Peff\n"},{"id":"284541","messageId":"xmqq8u00i9pp.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"921e007f445476b27325c12a9e92fdd169a073b7.1461685158.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-26T17:03:14Z","receivedAt":"2016-04-26T17:03:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> We introduce a way to send custom HTTP headers with all requests.\n>\n> This allows us, for example, to send an extra token from build agents\n> for temporary access to private repositories. (This is the use case that\n> triggered this patch.)\n>\n> This feature can be used like this:\n>\n> \tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n>\n> As `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` overrides previous\n> calls' headers (instead of appending the headers, as this unsuspecting\n> developer thought initially), we piggyback onto the `Pragma:` setting by\n> default, and introduce the global helper `http_copy_default_headers()`\n> to help functions that want to specify HTTP headers themselves.\n\nMy reading stuttered at \"we piggyback onto the `Pragma:` setting by\ndefault\", which made me stop and wonder if a description about a\nknob that changes this behaviour and makes us piggyback onto\nsomething else follows.\n\nI guess \"by default\" you meant that the majority of codepaths set\nthe headers using no_pragma_header or pragma_header variables, and\nby [no_]pragma_header to mean more than just about \"Pragma:\" by\nadding the extra headers to them, you did not have to touch them.\nOther codepaths that do not use these two variables but start from\nNULL you made them start from these extra headers.\n\nWhich makes sense.\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 42d2b50..37b9af7 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -1655,6 +1655,12 @@ http.emptyAuth::\n> \ta username in the URL, as libcurl normally requires a username for\n> \tauthentication.\n> \n> +http.extraHeader::\n> +\tPass an additional HTTP header when communicating with a server. If\n> +\tmore than one such entry exists, all of them are added as extra headers.\n> +\tThis feature is useful e.g. to increase security, or to allow\n> +\ttime-limited access based on expiring tokens.\n> +\n\nI think one-time/short-lived use case does not want to have this in\na configuration file, and instead want to do the command line thing\nyou illustrated in the proposed log message. I however wonder if\nthere are other use cases where having this in $GIT_DIR/config for\nrepeated use is useful. If there is, not being able to override a\nconfigured value per invocation would become a problem.\n\nPeff, what do you think? I vaguely recollect that you did a hack to\none variable that declares \"an empty value means discard accumulated\nvalues so far\" or something like that, and this variable deserves a\nmechanism like that, too.\n\n> diff --git a/http.c b/http.c\n> index 4304b80..3d662bb 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -1163,6 +1175,16 @@ int run_one_slot(struct active_request_slot *slot,\n> \treturn handle_curl_result(results);\n> }\n> \n> +struct curl_slist *http_copy_default_headers()\n\nstruct curl_slist *http_copy_default_headers(void)\n\n> +{\n> +\tstruct curl_slist *headers = NULL, *h;\n> +\n> +\tfor (h = extra_http_headers; h; h = h->next)\n> +\t\theaders = curl_slist_append(headers, h->data);\n> +\n> +\treturn headers;\n> +}\n> +\n> diff --git a/http.h b/http.h\n> index 4ef4bbd..5f13695 100644\n> --- a/http.h\n> +++ b/http.h\n> @@ -106,6 +106,7 @@ extern void step_active_slots(void);\n> extern void http_init(struct remote *remote, const char *url,\n> \t\t int proactive_auth);\n> extern void http_cleanup(void);\n> +extern struct curl_slist *http_copy_default_headers();\n\nextern struct curl_slist *http_copy_default_headers(void);\n"},{"id":"284543","messageId":"20160426171238.GA7609@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqq8u00i9pp.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-26T17:12:38Z","receivedAt":"2016-04-26T17:12:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 26, 2016 at 10:03:14AM -0700, Junio C Hamano wrote:\n\n> > +http.extraHeader::\n> > +\tPass an additional HTTP header when communicating with a server. If\n> > +\tmore than one such entry exists, all of them are added as extra headers.\n> > +\tThis feature is useful e.g. to increase security, or to allow\n> > +\ttime-limited access based on expiring tokens.\n> > +\n> \n> I think one-time/short-lived use case does not want to have this in\n> a configuration file, and instead want to do the command line thing\n> you illustrated in the proposed log message. I however wonder if\n> there are other use cases where having this in $GIT_DIR/config for\n> repeated use is useful. If there is, not being able to override a\n> configured value per invocation would become a problem.\n> \n> Peff, what do you think? I vaguely recollect that you did a hack to\n> one variable that declares \"an empty value means discard accumulated\n> values so far\" or something like that, and this variable deserves a\n> mechanism like that, too.\n\nYes, it was for credential.helper. I think the _implementation_ is a\nhack (because each callback has to handle it individually), but the\nuser-facing part is reasonably elegant, given the constraint of not\nintroducing new syntax.\n\nIn this case it would be something like:\n\ndiff --git a/http.c b/http.c\nindex 3d662bb..a7a4be5 100644\n--- a/http.c\n+++ b/http.c\n@@ -325,8 +325,13 @@ static int http_options(const char *var, const char *value, void *cb)\n \t}\n \n \tif (!strcmp(\"http.extraheader\", var)) {\n-\t\textra_http_headers =\n-\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\tif (*value)\n+\t\t\textra_http_headers =\n+\t\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\telse {\n+\t\t\tcurl_slist_free_all(extra_http_headers);\n+\t\t\textra_http_headers = NULL;\n+\t\t}\n \t\treturn 0;\n \t}\n \n\nBut I think this block (even before my patch) also needs to handle the\ncase where \"value\" is NULL (presumably by complaining with\nconfig_error_nonbool).\n\n-Peff\n"},{"id":"284544","messageId":"xmqq4maoi8x8.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160426171238.GA7609@sigill.intra.peff.net","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-26T17:20:19Z","receivedAt":"2016-04-26T17:20:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> But I think this block (even before my patch) also needs to handle the\n> case where \"value\" is NULL (presumably by complaining with\n> config_error_nonbool).\n\nOK, so squashes found to be necessary so far amounts to the attached\npatch. I still haven't figured out the best way to rephrase the \"by\ndefault\" in the proposed log message that made me stutter while\nreading it, though.\n\n\n http.c | 13 ++++++++++---\n http.h | 2 +-\n 2 files changed, 11 insertions(+), 4 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 6c4d2ed..aae9944 100644\n--- a/http.c\n+++ b/http.c\n@@ -325,8 +325,15 @@ static int http_options(const char *var, const char *value, void *cb)\n \t}\n \n \tif (!strcmp(\"http.extraheader\", var)) {\n-\t\textra_http_headers =\n-\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\tif (!value) {\n+\t\t\treturn config_error_nonbool(var);\n+\t\t} else if (!*value) {\n+\t\t\tcurl_slist_free_all(extra_http_headers);\n+\t\t\textra_http_headers = NULL;\n+\t\t} else {\n+\t\t\textra_http_headers =\n+\t\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\t}\n \t\treturn 0;\n \t}\n \n@@ -1172,7 +1179,7 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n-struct curl_slist *http_copy_default_headers()\n+struct curl_slist *http_copy_default_headers(void)\n {\n \tstruct curl_slist *headers = NULL, *h;\n \ndiff --git a/http.h b/http.h\nindex 5f13695..36f558b 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,7 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n-extern struct curl_slist *http_copy_default_headers();\n+extern struct curl_slist *http_copy_default_headers(void);\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\n"},{"id":"284546","messageId":"20160426173853.GB7609@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604260851390.2896@virtualbox","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-26T17:38:53Z","receivedAt":"2016-04-26T17:38:53Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 26, 2016 at 05:33:33PM +0200, Johannes Schindelin wrote:\n\n> Testing the headers? I dunno, do we have tests for that already? I thought\n> we did not: it requires an HTTP server (so that the headers are actually\n> sent) that we can force to check the header...\n> \n> So I see we have some tests that use Apache, and one that uses our own\n> http-backend. But is there already anything that logs HTTP requests? I did\n> not think so, please correct me if I am wrong.\n\nYou can ask apache to check for specific headers. Like this:\n\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 9317ba0..de5a8fe 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -102,6 +102,12 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \tHeader set Set-Cookie name=value\n \n+\n+\tRequire expr %{HTTP:x-magic-one} == 'abra'\n+\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n+\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n+\tSetEnv GIT_HTTP_EXPORT_ALL\n+\n ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error/ error.sh/\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex 58207d8..e44fe72 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n \ttest_line_count = 100000 tags\n '\n \n+test_expect_success 'custom http headers' '\n+\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+'\n+\n stop_httpd\n test_done\n"},{"id":"284549","messageId":"20160426174418.GC7609@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqq4maoi8x8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-26T17:44:19Z","receivedAt":"2016-04-26T17:44:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Apr 26, 2016 at 10:20:19AM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > But I think this block (even before my patch) also needs to handle the\n> > case where \"value\" is NULL (presumably by complaining with\n> > config_error_nonbool).\n> \n> OK, so squashes found to be necessary so far amounts to the attached\n> patch. I still haven't figured out the best way to rephrase the \"by\n> default\" in the proposed log message that made me stutter while\n> reading it, though.\n\nI think that part is just trying to explain the implementation hackery.\nMaybe something like:\n\n Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\n a single list, overriding any previous call. This means we have to\n collect _all_ of the headers we want to use into a single list, and\n feed it to curl in one shot. Since we already unconditionally set a\n \"pragma\" header when initializing the curl handles, we can add our new\n headers to that list.\n\n For callers which override the default header list (like probe_rpc),\n we provide `http_copy_default_headers()` so they can do the same\n trick.\n\n-Peff\n"},{"id":"284571","messageId":"xmqqzisgfaxs.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"921e007f445476b27325c12a9e92fdd169a073b7.1461685158.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-26T19:05:03Z","receivedAt":"2016-04-26T19:05:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 42d2b50..37b9af7 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -1655,6 +1655,12 @@ http.emptyAuth::\n> \ta username in the URL, as libcurl normally requires a username for\n> \tauthentication.\n> \n> +http.extraHeader::\n> +\tPass an additional HTTP header when communicating with a server. If\n> +\tmore than one such entry exists, all of them are added as extra headers.\n> +\tThis feature is useful e.g. to increase security, or to allow\n> +\ttime-limited access based on expiring tokens.\n\nI am unsure about the last two lines. Is it necessary to have them?\n"},{"id":"284628","messageId":"alpine.DEB.2.20.1604270807250.2896@virtualbox","threadId":"42138","inReplyTo":"20160426174418.GC7609@sigill.intra.peff.net","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-27T06:08:09Z","receivedAt":"2016-04-27T06:08:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Tue, 26 Apr 2016, Jeff King wrote:\n\n> On Tue, Apr 26, 2016 at 10:20:19AM -0700, Junio C Hamano wrote:\n> \n> > Jeff King writes:\n> > \n> > > But I think this block (even before my patch) also needs to handle the\n> > > case where \"value\" is NULL (presumably by complaining with\n> > > config_error_nonbool).\n> > \n> > OK, so squashes found to be necessary so far amounts to the attached\n> > patch. I still haven't figured out the best way to rephrase the \"by\n> > default\" in the proposed log message that made me stutter while\n> > reading it, though.\n> \n> I think that part is just trying to explain the implementation hackery.\n> Maybe something like:\n> \n> Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\n> a single list, overriding any previous call. This means we have to\n> collect _all_ of the headers we want to use into a single list, and\n> feed it to curl in one shot. Since we already unconditionally set a\n> \"pragma\" header when initializing the curl handles, we can add our new\n> headers to that list.\n> \n> For callers which override the default header list (like probe_rpc),\n> we provide `http_copy_default_headers()` so they can do the same\n> trick.\n\nThank you very much for this. I replaced that part of the commit message\nwith your version.\n\nCiao,\nJohannes\n"},{"id":"284629","messageId":"d087647975742a803735c838d0b970d962d75048.1461738362.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"921e007f445476b27325c12a9e92fdd169a073b7.1461685158.git.johannes.schindelin@gmx.de","subject":"[PATCH v3] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-27T06:29:10Z","receivedAt":"2016-04-27T06:29:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"We introduce a way to send custom HTTP headers with all requests.\n\nThis allows us, for example, to send an extra token from build agents\nfor temporary access to private repositories. (This is the use case that\ntriggered this patch.)\n\nThis feature can be used like this:\n\n\tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nNote that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\na single list, overriding any previous call. This means we have to\ncollect _all_ of the headers we want to use into a single list, and\nfeed it to cURL in one shot. Since we already unconditionally set a\n\"pragma\" header when initializing the curl handles, we can add our new\nheaders to that list.\n\nFor callers which override the default header list (like probe_rpc),\nwe provide `http_copy_default_headers()` so they can do the same\ntrick.\n\nBig thanks to Jeff King and Junio Hamano for their outstanding help and\npatient reviews.\n\nSigned-off-by: Johannes Schindelin \n---\n\nChanges since v2:\n\t- now using Peff's much improved wording in the commit message\n\t- skipped the last two lines from the documentation of the feature\n\t- fixed function declaration/definition by using `(void)`\n\t- we now allow resetting extra headers with an empty value\n\t- added a test\n\nPublished-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v3\n Documentation/config.txt | 6 ++++++\n http-push.c | 10 +++++-----\n http.c | 35 ++++++++++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n t/t5550-http-fetch-dumb.sh | 8 ++++++++\n 6 files changed, 54 insertions(+), 10 deletions(-)\nInterdiff vs v2:\n\n diff --git a/Documentation/config.txt b/Documentation/config.txt\n index 37b9af7..c7bbe98 100644\n --- a/Documentation/config.txt\n +++ b/Documentation/config.txt\n @@ -1657,9 +1657,9 @@ http.emptyAuth::\n \n http.extraHeader::\n \tPass an additional HTTP header when communicating with a server. If\n -\tmore than one such entry exists, all of them are added as extra headers.\n -\tThis feature is useful e.g. to increase security, or to allow\n -\ttime-limited access based on expiring tokens.\n +\tmore than one such entry exists, all of them are added as extra\n +\theaders. To allow overriding the settings inherited from the system\n +\tconfig, an empty value will reset the extra headers to the empty list.\n \n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n diff --git a/http.c b/http.c\n index 3d662bb..985b995 100644\n --- a/http.c\n +++ b/http.c\n @@ -325,8 +325,15 @@ static int http_options(const char *var, const char *value, void *cb)\n \t}\n \n \tif (!strcmp(\"http.extraheader\", var)) {\n -\t\textra_http_headers =\n -\t\t\tcurl_slist_append(extra_http_headers, value);\n +\t\tif (!value) {\n +\t\t\treturn config_error_nonbool(var);\n +\t\t} else if (!*value) {\n +\t\t\tcurl_slist_free_all(extra_http_headers);\n +\t\t\textra_http_headers = NULL;\n +\t\t} else {\n +\t\t\textra_http_headers =\n +\t\t\t\tcurl_slist_append(extra_http_headers, value);\n +\t\t}\n \t\treturn 0;\n \t}\n \n @@ -1175,7 +1182,7 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n -struct curl_slist *http_copy_default_headers()\n +struct curl_slist *http_copy_default_headers(void)\n {\n \tstruct curl_slist *headers = NULL, *h;\n \n diff --git a/http.h b/http.h\n index 5f13695..36f558b 100644\n --- a/http.h\n +++ b/http.h\n @@ -106,7 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n -extern struct curl_slist *http_copy_default_headers();\n +extern struct curl_slist *http_copy_default_headers(void);\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\n diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\n index 48e2ab6..96425b1 100755\n --- a/t/t5550-http-fetch-dumb.sh\n +++ b/t/t5550-http-fetch-dumb.sh\n @@ -267,5 +267,13 @@ test_expect_success 'git client does not send an empty Accept-Language' '\n \t! grep \"^Accept-Language:\" stderr\n '\n \n +test_expect_success 'extra HTTP headers are sent' '\n +\tGIT_CURL_VERBOSE=1 \\\n +\tgit -c http.extraheader=\"Hello: World\" \\\n +\t\tls-remote \"$HTTPD_URL/dumb/repo.git\" >out 2>err &&\n +\ttest_i18ngrep \"Hello: World\" err >hello.txt &&\n +\ttest_line_count = 2 hello.txt\n +'\n +\n stop_httpd\n test_done\n\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 42d2b50..c7bbe98 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1655,6 +1655,12 @@ http.emptyAuth::\n \ta username in the URL, as libcurl normally requires a username for\n \tauthentication.\n \n+http.extraHeader::\n+\tPass an additional HTTP header when communicating with a server. If\n+\tmore than one such entry exists, all of them are added as extra\n+\theaders. To allow overriding the settings inherited from the system\n+\tconfig, an empty value will reset the extra headers to the empty list.\n+\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\ndiff --git a/http-push.c b/http-push.c\nindex bd60668..ae2b7f1 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n@@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\ndiff --git a/http.c b/http.c\nindex 4304b80..985b995 100644\n--- a/http.c\n+++ b/http.c\n@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n \n static struct curl_slist *pragma_header;\n static struct curl_slist *no_pragma_header;\n+static struct curl_slist *extra_http_headers;\n \n static struct active_request_slot *active_queue_head;\n \n@@ -323,6 +324,19 @@ static int http_options(const char *var, const char *value, void *cb)\n #endif\n \t}\n \n+\tif (!strcmp(\"http.extraheader\", var)) {\n+\t\tif (!value) {\n+\t\t\treturn config_error_nonbool(var);\n+\t\t} else if (!*value) {\n+\t\t\tcurl_slist_free_all(extra_http_headers);\n+\t\t\textra_http_headers = NULL;\n+\t\t} else {\n+\t\t\textra_http_headers =\n+\t\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\t}\n+\t\treturn 0;\n+\t}\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -678,8 +692,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n-\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n+\tpragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma: no-cache\");\n+\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n \t{\n@@ -765,6 +781,9 @@ void http_cleanup(void)\n #endif\n \tcurl_global_cleanup();\n \n+\tcurl_slist_free_all(extra_http_headers);\n+\textra_http_headers = NULL;\n+\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n@@ -1163,6 +1182,16 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n+struct curl_slist *http_copy_default_headers(void)\n+{\n+\tstruct curl_slist *headers = NULL, *h;\n+\n+\tfor (h = extra_http_headers; h; h = h->next)\n+\t\theaders = curl_slist_append(headers, h->data);\n+\n+\treturn headers;\n+}\n+\n static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n {\n \tchar *ptr;\n@@ -1380,7 +1409,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\ndiff --git a/http.h b/http.h\nindex 4ef4bbd..36f558b 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,6 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n+extern struct curl_slist *http_copy_default_headers(void);\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 15e48e2..672b382 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\ndiff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\nindex 48e2ab6..96425b1 100755\n--- a/t/t5550-http-fetch-dumb.sh\n+++ b/t/t5550-http-fetch-dumb.sh\n@@ -267,5 +267,13 @@ test_expect_success 'git client does not send an empty Accept-Language' '\n \t! grep \"^Accept-Language:\" stderr\n '\n \n+test_expect_success 'extra HTTP headers are sent' '\n+\tGIT_CURL_VERBOSE=1 \\\n+\tgit -c http.extraheader=\"Hello: World\" \\\n+\t\tls-remote \"$HTTPD_URL/dumb/repo.git\" >out 2>err &&\n+\ttest_i18ngrep \"Hello: World\" err >hello.txt &&\n+\ttest_line_count = 2 hello.txt\n+'\n+\n stop_httpd\n test_done\n-- \n2.8.1.306.gff998f2\n"},{"id":"284630","messageId":"alpine.DEB.2.20.1604270829180.2896@virtualbox","threadId":"42138","inReplyTo":"xmqq4maoi8x8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v2] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-27T06:29:35Z","receivedAt":"2016-04-27T06:29:35Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Tue, 26 Apr 2016, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > But I think this block (even before my patch) also needs to handle the\n> > case where \"value\" is NULL (presumably by complaining with\n> > config_error_nonbool).\n> \n> OK, so squashes found to be necessary so far amounts to the attached\n> patch.\n\nThanks.\nDscho\n"},{"id":"284631","messageId":"alpine.DEB.2.20.1604270830220.2896@virtualbox","threadId":"42138","inReplyTo":"20160426173853.GB7609@sigill.intra.peff.net","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-27T06:31:50Z","receivedAt":"2016-04-27T06:31:50Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Tue, 26 Apr 2016, Jeff King wrote:\n\n> On Tue, Apr 26, 2016 at 05:33:33PM +0200, Johannes Schindelin wrote:\n> \n> > Testing the headers? I dunno, do we have tests for that already? I thought\n> > we did not: it requires an HTTP server (so that the headers are actually\n> > sent) that we can force to check the header...\n> > \n> > So I see we have some tests that use Apache, and one that uses our own\n> > http-backend. But is there already anything that logs HTTP requests? I did\n> > not think so, please correct me if I am wrong.\n> \n> You can ask apache to check for specific headers. Like this:\n> \n> diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n> index 9317ba0..de5a8fe 100644\n> --- a/t/lib-httpd/apache.conf\n> +++ b/t/lib-httpd/apache.conf\n> @@ -102,6 +102,12 @@ Alias /auth/dumb/ www/auth/dumb/\n> \tSetEnv GIT_HTTP_EXPORT_ALL\n> \tHeader set Set-Cookie name=value\n> \n> +\n> +\tRequire expr %{HTTP:x-magic-one} == 'abra'\n> +\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n> +\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n> +\tSetEnv GIT_HTTP_EXPORT_ALL\n> +\n> ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n> ScriptAlias /broken_smart/ broken-smart-http.sh/\n> ScriptAlias /error/ error.sh/\n> diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n> index 58207d8..e44fe72 100755\n> --- a/t/t5551-http-fetch-smart.sh\n> +++ b/t/t5551-http-fetch-smart.sh\n> @@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n> \ttest_line_count = 100000 tags\n> '\n> \n> +test_expect_success 'custom http headers' '\n> +\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> +\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n> +'\n> +\n> stop_httpd\n> test_done\n\nThat's pretty easy.\n\nAfter sleeping over the issue, I realized, though, that the test can use\nthe very same method *I* used to verify that the headers are sent: using\nGIT_CURL_VERBOSE.\n\nI hope you do not mind that I used this method instead.\n\nThanks,\nDscho\n"},{"id":"284636","messageId":"20160427075219.GA32535@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604270830220.2896@virtualbox","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-27T07:52:19Z","receivedAt":"2016-04-27T07:52:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 27, 2016 at 08:31:50AM +0200, Johannes Schindelin wrote:\n\n> > +test_expect_success 'custom http headers' '\n> > +\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> > +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> > +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> > +\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n> > +'\n> > +\n> > stop_httpd\n> > test_done\n> \n> That's pretty easy.\n> \n> After sleeping over the issue, I realized, though, that the test can use\n> the very same method *I* used to verify that the headers are sent: using\n> GIT_CURL_VERBOSE.\n> \n> I hope you do not mind that I used this method instead.\n\nTBH, I think mine is a little more robust, but I don't overly care\neither way.\n\n-Peff\n"},{"id":"284640","messageId":"alpine.DEB.2.20.1604271355350.2896@virtualbox","threadId":"42138","inReplyTo":"20160427075219.GA32535@sigill.intra.peff.net","subject":"Re: [PATCH] http: Support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-27T11:56:47Z","receivedAt":"2016-04-27T11:56:47Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Wed, 27 Apr 2016, Jeff King wrote:\n\n> On Wed, Apr 27, 2016 at 08:31:50AM +0200, Johannes Schindelin wrote:\n> \n> > > +test_expect_success 'custom http headers' '\n> > > +\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> > > +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> > > +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> > > +\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n> > > +'\n> > > +\n> > > stop_httpd\n> > > test_done\n> > \n> > That's pretty easy.\n> > \n> > After sleeping over the issue, I realized, though, that the test can use\n> > the very same method *I* used to verify that the headers are sent: using\n> > GIT_CURL_VERBOSE.\n> > \n> > I hope you do not mind that I used this method instead.\n> \n> TBH, I think mine is a little more robust, but I don't overly care\n> either way.\n\nYes, you are correct, I only test that the header is sent twice, assuming\nthat exactly two HTTP requests are sent. Your test verifies that whatever\nrequest is sent *requires* the header.\n\nWill send out v4 with your test instead of mine in a few moments.\n\nThanks,\nDscho\n"},{"id":"284643","messageId":"3b71deffa5d07cf73a793773cc3d60ff611843fa.1461759454.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"d087647975742a803735c838d0b970d962d75048.1461738362.git.johannes.schindelin@gmx.de","subject":"[PATCH v4] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-27T12:20:37Z","receivedAt":"2016-04-27T12:20:37Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"We introduce a way to send custom HTTP headers with all requests.\n\nThis allows us, for example, to send an extra token from build agents\nfor temporary access to private repositories. (This is the use case that\ntriggered this patch.)\n\nThis feature can be used like this:\n\n\tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nNote that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\na single list, overriding any previous call. This means we have to\ncollect _all_ of the headers we want to use into a single list, and\nfeed it to cURL in one shot. Since we already unconditionally set a\n\"pragma\" header when initializing the curl handles, we can add our new\nheaders to that list.\n\nFor callers which override the default header list (like probe_rpc),\nwe provide `http_copy_default_headers()` so they can do the same\ntrick.\n\nBig thanks to Jeff King and Junio Hamano for their outstanding help and\npatient reviews.\n\nSigned-off-by: Johannes Schindelin \n---\nPublished-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v4\n\nThe only change vs v3 is that I replaced my flimsical test by Peff's (with\n*one* change: I realized that we need to group the Require statements in a\n block when I tried to verify that the test fails when I\nmodify the first header).\n\n Documentation/config.txt | 6 ++++++\n http-push.c | 10 +++++-----\n http.c | 35 ++++++++++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n t/lib-httpd/apache.conf | 8 ++++++++\n t/t5551-http-fetch-smart.sh | 7 +++++++\n 7 files changed, 61 insertions(+), 10 deletions(-)\nInterdiff vs v3:\n\n diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n index 9317ba0..b8ed96f 100644\n --- a/t/lib-httpd/apache.conf\n +++ b/t/lib-httpd/apache.conf\n @@ -102,6 +102,14 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \tHeader set Set-Cookie name=value\n \n +\n +\t\n +\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n +\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n +\t\n +\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n +\tSetEnv GIT_HTTP_EXPORT_ALL\n +\n ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error/ error.sh/\n diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh\n index 96425b1..48e2ab6 100755\n --- a/t/t5550-http-fetch-dumb.sh\n +++ b/t/t5550-http-fetch-dumb.sh\n @@ -267,13 +267,5 @@ test_expect_success 'git client does not send an empty Accept-Language' '\n \t! grep \"^Accept-Language:\" stderr\n '\n \n -test_expect_success 'extra HTTP headers are sent' '\n -\tGIT_CURL_VERBOSE=1 \\\n -\tgit -c http.extraheader=\"Hello: World\" \\\n -\t\tls-remote \"$HTTPD_URL/dumb/repo.git\" >out 2>err &&\n -\ttest_i18ngrep \"Hello: World\" err >hello.txt &&\n -\ttest_line_count = 2 hello.txt\n -'\n -\n stop_httpd\n test_done\n diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n index 58207d8..e44fe72 100755\n --- a/t/t5551-http-fetch-smart.sh\n +++ b/t/t5551-http-fetch-smart.sh\n @@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n \ttest_line_count = 100000 tags\n '\n \n +test_expect_success 'custom http headers' '\n +\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n +\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n +'\n +\n stop_httpd\n test_done\n\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 42d2b50..c7bbe98 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1655,6 +1655,12 @@ http.emptyAuth::\n \ta username in the URL, as libcurl normally requires a username for\n \tauthentication.\n \n+http.extraHeader::\n+\tPass an additional HTTP header when communicating with a server. If\n+\tmore than one such entry exists, all of them are added as extra\n+\theaders. To allow overriding the settings inherited from the system\n+\tconfig, an empty value will reset the extra headers to the empty list.\n+\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\ndiff --git a/http-push.c b/http-push.c\nindex bd60668..ae2b7f1 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n@@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\ndiff --git a/http.c b/http.c\nindex 4304b80..985b995 100644\n--- a/http.c\n+++ b/http.c\n@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n \n static struct curl_slist *pragma_header;\n static struct curl_slist *no_pragma_header;\n+static struct curl_slist *extra_http_headers;\n \n static struct active_request_slot *active_queue_head;\n \n@@ -323,6 +324,19 @@ static int http_options(const char *var, const char *value, void *cb)\n #endif\n \t}\n \n+\tif (!strcmp(\"http.extraheader\", var)) {\n+\t\tif (!value) {\n+\t\t\treturn config_error_nonbool(var);\n+\t\t} else if (!*value) {\n+\t\t\tcurl_slist_free_all(extra_http_headers);\n+\t\t\textra_http_headers = NULL;\n+\t\t} else {\n+\t\t\textra_http_headers =\n+\t\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\t}\n+\t\treturn 0;\n+\t}\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -678,8 +692,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n-\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n+\tpragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma: no-cache\");\n+\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n \t{\n@@ -765,6 +781,9 @@ void http_cleanup(void)\n #endif\n \tcurl_global_cleanup();\n \n+\tcurl_slist_free_all(extra_http_headers);\n+\textra_http_headers = NULL;\n+\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n@@ -1163,6 +1182,16 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n+struct curl_slist *http_copy_default_headers(void)\n+{\n+\tstruct curl_slist *headers = NULL, *h;\n+\n+\tfor (h = extra_http_headers; h; h = h->next)\n+\t\theaders = curl_slist_append(headers, h->data);\n+\n+\treturn headers;\n+}\n+\n static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n {\n \tchar *ptr;\n@@ -1380,7 +1409,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\ndiff --git a/http.h b/http.h\nindex 4ef4bbd..36f558b 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,6 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n+extern struct curl_slist *http_copy_default_headers(void);\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 15e48e2..672b382 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 9317ba0..b8ed96f 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -102,6 +102,14 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \tHeader set Set-Cookie name=value\n \n+\n+\t\n+\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n+\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n+\t\n+\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n+\tSetEnv GIT_HTTP_EXPORT_ALL\n+\n ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error/ error.sh/\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex 58207d8..e44fe72 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n \ttest_line_count = 100000 tags\n '\n \n+test_expect_success 'custom http headers' '\n+\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+'\n+\n stop_httpd\n test_done\n-- \n2.8.1.306.gff998f2\n"},{"id":"284710","messageId":"20160427193055.GA6727@sigill.intra.peff.net","threadId":"42138","inReplyTo":"3b71deffa5d07cf73a793773cc3d60ff611843fa.1461759454.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v4] http: support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-27T19:30:55Z","receivedAt":"2016-04-27T19:30:55Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Apr 27, 2016 at 02:20:37PM +0200, Johannes Schindelin wrote:\n\n> The only change vs v3 is that I replaced my flimsical test by Peff's (with\n> *one* change: I realized that we need to group the Require statements in a\n> block when I tried to verify that the test fails when I\n> modify the first header).\n\nWhoops, I didn't actually test that case. Thanks for catching (as you\nmight guess, I wanted to make sure we handle multiple values correctly).\n\n> Documentation/config.txt | 6 ++++++\n> http-push.c | 10 +++++-----\n> http.c | 35 ++++++++++++++++++++++++++++++++---\n> http.h | 1 +\n> remote-curl.c | 4 ++--\n> t/lib-httpd/apache.conf | 8 ++++++++\n> t/t5551-http-fetch-smart.sh | 7 +++++++\n> 7 files changed, 61 insertions(+), 10 deletions(-)\n\nThis version looks good to me.\n\n-Peff\n"},{"id":"284747","messageId":"xmqq4mam932j.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160427193055.GA6727@sigill.intra.peff.net","subject":"Re: [PATCH v4] http: support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-27T21:03:48Z","receivedAt":"2016-04-27T21:03:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Wed, Apr 27, 2016 at 02:20:37PM +0200, Johannes Schindelin wrote:\n>\n>> The only change vs v3 is that I replaced my flimsical test by Peff's (with\n>> *one* change: I realized that we need to group the Require statements in a\n>> block when I tried to verify that the test fails when I\n>> modify the first header).\n>\n> Whoops, I didn't actually test that case. Thanks for catching (as you\n> might guess, I wanted to make sure we handle multiple values correctly).\n>\n>> Documentation/config.txt | 6 ++++++\n>> http-push.c | 10 +++++-----\n>> http.c | 35 ++++++++++++++++++++++++++++++++---\n>> http.h | 1 +\n>> remote-curl.c | 4 ++--\n>> t/lib-httpd/apache.conf | 8 ++++++++\n>> t/t5551-http-fetch-smart.sh | 7 +++++++\n>> 7 files changed, 61 insertions(+), 10 deletions(-)\n>\n> This version looks good to me.\n>\n> -Peff\n\nThanks, both.\n"},{"id":"284782","messageId":"cover.1461837783.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"3b71deffa5d07cf73a793773cc3d60ff611843fa.1461759454.git.johannes.schindelin@gmx.de","subject":"[PATCH v5 0/2] Add support for sending additional HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-28T10:03:34Z","receivedAt":"2016-04-28T10:03:34Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"My use case is an army of build agents that need only limited and\nselective access to otherwise private repositories.\n\nI apologize for sending out v5 after v4 was already acknowledged: my\ninitial testing was on simple repositories and I forgot that my build\nagents need to handle submodules, too.\n\nSo here goes v5, the only change being the addition of the second patch\nthat adds support for passing the extra headers to git-submodule through\nthe command-line.\n\n\nJohannes Schindelin (2):\n http: support sending custom HTTP headers\n submodule: pass on http.extraheader config settings\n\n Documentation/config.txt | 6 ++++++\n builtin/submodule--helper.c | 4 +++-\n http-push.c | 10 +++++-----\n http.c | 35 ++++++++++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n t/lib-httpd/apache.conf | 8 ++++++++\n t/t5551-http-fetch-smart.sh | 16 ++++++++++++++++\n 8 files changed, 73 insertions(+), 11 deletions(-)\n\nInterdiff vs v4:\n\n diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n index 3bd6883..b338f93 100644\n --- a/builtin/submodule--helper.c\n +++ b/builtin/submodule--helper.c\n @@ -127,7 +127,9 @@ static int module_name(int argc, const char **argv, const char *prefix)\n */\n static int submodule_config_ok(const char *var)\n {\n -\tif (starts_with(var, \"credential.\"))\n +\tif (starts_with(var, \"credential.\") ||\n +\t\t\t(starts_with(var, \"http.\") &&\n +\t\t\t ends_with(var, \".extraheader\")))\n \t\treturn 1;\n \treturn 0;\n }\n diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n index e44fe72..1794168 100755\n --- a/t/t5551-http-fetch-smart.sh\n +++ b/t/t5551-http-fetch-smart.sh\n @@ -286,7 +286,16 @@ test_expect_success 'custom http headers' '\n \ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n -\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n +\t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n +\tgit update-index --add --cacheinfo 160000,$(git rev-parse HEAD),sub &&\n +\tgit config -f .gitmodules submodule.sub.path sub &&\n +\tgit config -f .gitmodules submodule.sub.url \\\n +\t\t\"$HTTPD_URL/smart_headers/repo.git\" &&\n +\tgit submodule init sub &&\n +\ttest_must_fail git submodule update sub &&\n +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n +\t\tsubmodule update sub\n '\n \n stop_httpd\n\n-- \n2.8.1.306.gff998f2\n"},{"id":"284784","messageId":"3b71deffa5d07cf73a793773cc3d60ff611843fa.1461837783.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1461837783.git.johannes.schindelin@gmx.de","subject":"[PATCH v5 1/2] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-28T10:03:40Z","receivedAt":"2016-04-28T10:03:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"We introduce a way to send custom HTTP headers with all requests.\n\nThis allows us, for example, to send an extra token from build agents\nfor temporary access to private repositories. (This is the use case that\ntriggered this patch.)\n\nThis feature can be used like this:\n\n\tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nNote that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\na single list, overriding any previous call. This means we have to\ncollect _all_ of the headers we want to use into a single list, and\nfeed it to cURL in one shot. Since we already unconditionally set a\n\"pragma\" header when initializing the curl handles, we can add our new\nheaders to that list.\n\nFor callers which override the default header list (like probe_rpc),\nwe provide `http_copy_default_headers()` so they can do the same\ntrick.\n\nBig thanks to Jeff King and Junio Hamano for their outstanding help and\npatient reviews.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/config.txt | 6 ++++++\n http-push.c | 10 +++++-----\n http.c | 35 ++++++++++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n t/lib-httpd/apache.conf | 8 ++++++++\n t/t5551-http-fetch-smart.sh | 7 +++++++\n 7 files changed, 61 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 42d2b50..c7bbe98 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1655,6 +1655,12 @@ http.emptyAuth::\n \ta username in the URL, as libcurl normally requires a username for\n \tauthentication.\n \n+http.extraHeader::\n+\tPass an additional HTTP header when communicating with a server. If\n+\tmore than one such entry exists, all of them are added as extra\n+\theaders. To allow overriding the settings inherited from the system\n+\tconfig, an empty value will reset the extra headers to the empty list.\n+\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\ndiff --git a/http-push.c b/http-push.c\nindex bd60668..ae2b7f1 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n@@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\ndiff --git a/http.c b/http.c\nindex 4304b80..985b995 100644\n--- a/http.c\n+++ b/http.c\n@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n \n static struct curl_slist *pragma_header;\n static struct curl_slist *no_pragma_header;\n+static struct curl_slist *extra_http_headers;\n \n static struct active_request_slot *active_queue_head;\n \n@@ -323,6 +324,19 @@ static int http_options(const char *var, const char *value, void *cb)\n #endif\n \t}\n \n+\tif (!strcmp(\"http.extraheader\", var)) {\n+\t\tif (!value) {\n+\t\t\treturn config_error_nonbool(var);\n+\t\t} else if (!*value) {\n+\t\t\tcurl_slist_free_all(extra_http_headers);\n+\t\t\textra_http_headers = NULL;\n+\t\t} else {\n+\t\t\textra_http_headers =\n+\t\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\t}\n+\t\treturn 0;\n+\t}\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -678,8 +692,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n-\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n+\tpragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma: no-cache\");\n+\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n \t{\n@@ -765,6 +781,9 @@ void http_cleanup(void)\n #endif\n \tcurl_global_cleanup();\n \n+\tcurl_slist_free_all(extra_http_headers);\n+\textra_http_headers = NULL;\n+\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n@@ -1163,6 +1182,16 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n+struct curl_slist *http_copy_default_headers(void)\n+{\n+\tstruct curl_slist *headers = NULL, *h;\n+\n+\tfor (h = extra_http_headers; h; h = h->next)\n+\t\theaders = curl_slist_append(headers, h->data);\n+\n+\treturn headers;\n+}\n+\n static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n {\n \tchar *ptr;\n@@ -1380,7 +1409,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\ndiff --git a/http.h b/http.h\nindex 4ef4bbd..36f558b 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,6 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n+extern struct curl_slist *http_copy_default_headers(void);\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 15e48e2..672b382 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 9317ba0..b8ed96f 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -102,6 +102,14 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \tHeader set Set-Cookie name=value\n \n+\n+\t\n+\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n+\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n+\t\n+\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n+\tSetEnv GIT_HTTP_EXPORT_ALL\n+\n ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error/ error.sh/\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex 58207d8..e44fe72 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n \ttest_line_count = 100000 tags\n '\n \n+test_expect_success 'custom http headers' '\n+\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+'\n+\n stop_httpd\n test_done\n-- \n2.8.1.306.gff998f2\n"},{"id":"284783","messageId":"89d0024450b0e6e9997ad9e3d681248bde1bafc0.1461837783.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1461837783.git.johannes.schindelin@gmx.de","subject":"[PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-28T10:03:47Z","receivedAt":"2016-04-28T10:03:47Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To support this developer's use case of allowing build agents token-based\naccess to private repositories, we introduced the http.extraheader\nfeature, allowing extra HTTP headers to be sent along with every HTTP\nrequest.\n\nThis patch allows us to configure these extra HTTP headers for use with\n`git submodule update`, too. It requires somewhat special handling:\nsubmodules do not share the parent project's config. It would be\nincorrect to simply reuse that specific part of the parent's config.\nInstead, the config option needs to be specified on the command-line or\nin ~/.gitconfig or friends.\n\nExample: git -c http.extraheader=\"Secret: Sauce\" submodule update --init\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/submodule--helper.c | 4 +++-\n t/t5551-http-fetch-smart.sh | 11 ++++++++++-\n 2 files changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 3bd6883..b338f93 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -127,7 +127,9 @@ static int module_name(int argc, const char **argv, const char *prefix)\n */\n static int submodule_config_ok(const char *var)\n {\n-\tif (starts_with(var, \"credential.\"))\n+\tif (starts_with(var, \"credential.\") ||\n+\t\t\t(starts_with(var, \"http.\") &&\n+\t\t\t ends_with(var, \".extraheader\")))\n \t\treturn 1;\n \treturn 0;\n }\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex e44fe72..1794168 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -286,7 +286,16 @@ test_expect_success 'custom http headers' '\n \ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n-\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit update-index --add --cacheinfo 160000,$(git rev-parse HEAD),sub &&\n+\tgit config -f .gitmodules submodule.sub.path sub &&\n+\tgit config -f .gitmodules submodule.sub.url \\\n+\t\t\"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit submodule init sub &&\n+\ttest_must_fail git submodule update sub &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t\tsubmodule update sub\n '\n \n stop_httpd\n-- \n2.8.1.306.gff998f2\n"},{"id":"284787","messageId":"20160428112912.GB11522@sigill.intra.peff.net","threadId":"42138","inReplyTo":"89d0024450b0e6e9997ad9e3d681248bde1bafc0.1461837783.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T11:29:13Z","receivedAt":"2016-04-28T11:29:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 12:03:47PM +0200, Johannes Schindelin wrote:\n\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index 3bd6883..b338f93 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -127,7 +127,9 @@ static int module_name(int argc, const char **argv, const char *prefix)\n> */\n> static int submodule_config_ok(const char *var)\n> {\n> -\tif (starts_with(var, \"credential.\"))\n> +\tif (starts_with(var, \"credential.\") ||\n> +\t\t\t(starts_with(var, \"http.\") &&\n> +\t\t\t ends_with(var, \".extraheader\")))\n> \t\treturn 1;\n> \treturn 0;\n> }\n\nShould we consider just white-listing all of \"http.*\"?\n\nThat would help other cases which have come up, like:\n\n http://thread.gmane.org/gmane.comp.version-control.git/264840\n\nwhich wants to turn off http.sslverify. That would mean it turns off for\nevery submodule, too, but if you want to be choosy about your http\nvariables, you should be using the \"http.$URL.sslverify\" form, to only\naffect specific servers (whether they are in submodules or not).\n\n-Peff\n"},{"id":"284794","messageId":"alpine.DEB.2.20.1604281405540.2896@virtualbox","threadId":"42138","inReplyTo":"20160428112912.GB11522@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-28T12:19:37Z","receivedAt":"2016-04-28T12:19:37Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nCc:ing Jacob, the author of the CONFIG_DATA_ENVIRONMENT sanitizing code.\n\nOn Thu, 28 Apr 2016, Jeff King wrote:\n\n> On Thu, Apr 28, 2016 at 12:03:47PM +0200, Johannes Schindelin wrote:\n> \n> > diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> > index 3bd6883..b338f93 100644\n> > --- a/builtin/submodule--helper.c\n> > +++ b/builtin/submodule--helper.c\n> > @@ -127,7 +127,9 @@ static int module_name(int argc, const char **argv, const char *prefix)\n> > */\n> > static int submodule_config_ok(const char *var)\n> > {\n> > -\tif (starts_with(var, \"credential.\"))\n> > +\tif (starts_with(var, \"credential.\") ||\n> > +\t\t\t(starts_with(var, \"http.\") &&\n> > +\t\t\t ends_with(var, \".extraheader\")))\n> > \t\treturn 1;\n> > \treturn 0;\n> > }\n> \n> Should we consider just white-listing all of \"http.*\"?\n> \n> That would help other cases which have come up, like:\n> \n> http://thread.gmane.org/gmane.comp.version-control.git/264840\n> \n> which wants to turn off http.sslverify. That would mean it turns off for\n> every submodule, too, but if you want to be choosy about your http\n> variables, you should be using the \"http.$URL.sslverify\" form, to only\n> affect specific servers (whether they are in submodules or not).\n\nI considered that, and thought that it might be dangerous, what with me\nnot vetting carefully which http.* variables are safe to pass on to the\nsubmodules' update and which are not.\n\nSo I had a look now, and the most prominent potential problem is the\nhttp.cookieFile setting, which could be reused all of a sudden if we\nmade my patch more general.\n\nBut then, we are talking about the code that filters what gets passed via\nthe *command-line*. And to be quite honest, I am not sure that we should\nactually filter out *any* of these settings.\n\nThe commit message that introduced this particular filtering has this\nrationale to let only credential.* through:\n\n GIT_CONFIG_PARAMETERS is special, and we actually do want to\n preserve these settings. However, we do not want to preserve all\n configuration as many things should be left specific to the parent\n project.\n\n Add a git submodule--helper function, sanitize-config, which shall be\n used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs\n except a small subset that are known to be safe and necessary.\n\nDunno. I tried to err on the side of caution... But this sounds maybe a\nbit *too* cautious?\n\nJacob, Junio?\n\nCiao,\nDscho\n"},{"id":"284805","messageId":"20160428134953.GB25364@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604281405540.2896@virtualbox","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T13:49:53Z","receivedAt":"2016-04-28T13:49:53Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 02:19:37PM +0200, Johannes Schindelin wrote:\n\n> > Should we consider just white-listing all of \"http.*\"?\n> > \n> > That would help other cases which have come up, like:\n> > \n> > http://thread.gmane.org/gmane.comp.version-control.git/264840\n> > \n> > which wants to turn off http.sslverify. That would mean it turns off for\n> > every submodule, too, but if you want to be choosy about your http\n> > variables, you should be using the \"http.$URL.sslverify\" form, to only\n> > affect specific servers (whether they are in submodules or not).\n> \n> I considered that, and thought that it might be dangerous, what with me\n> not vetting carefully which http.* variables are safe to pass on to the\n> submodules' update and which are not.\n> \n> So I had a look now, and the most prominent potential problem is the\n> http.cookieFile setting, which could be reused all of a sudden if we\n> made my patch more general.\n> \n> But then, we are talking about the code that filters what gets passed via\n> the *command-line*. And to be quite honest, I am not sure that we should\n> actually filter out *any* of these settings.\n\nThe intent of the whitelist (from my recollection of the discussion) is\nto filter out config that must be repo-specific. E.g., core.worktree or\ncore.bare should definitely _not_ be passed to a submodule.\n\nI don't know if there are others. We started with a whitelist because it\nwas the smallest and safest change away from the status quo. A blacklist\nwould also work, with the risk that we might let through nonsense in\nsome cases (but only if the user triggers us to do so).\n\n> The commit message that introduced this particular filtering has this\n> rationale to let only credential.* through:\n> \n> GIT_CONFIG_PARAMETERS is special, and we actually do want to\n> preserve these settings. However, we do not want to preserve all\n> configuration as many things should be left specific to the parent\n> project.\n> \n> Add a git submodule--helper function, sanitize-config, which shall be\n> used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs\n> except a small subset that are known to be safe and necessary.\n> \n> Dunno. I tried to err on the side of caution... But this sounds maybe a\n> bit *too* cautious?\n\nSo if we all agree that the sanitizing is really about preventing\nrepo-specific variables from leaking, and not any kind of security\nboundary, I think we should generally be pretty liberal in whitelisting\nthings.\n\nI can certainly come up with a pathological case where using it as a\nsecurity boundary may have some practical use, but in general I think it\nis mostly getting in the way of what users are trying to do.\n\n-Peff\n"},{"id":"284806","messageId":"20160428135317.GC25364@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604281405540.2896@virtualbox","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T13:53:18Z","receivedAt":"2016-04-28T13:53:18Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 02:19:37PM +0200, Johannes Schindelin wrote:\n\n> > Should we consider just white-listing all of \"http.*\"?\n> > \n> > That would help other cases which have come up, like:\n> > \n> > http://thread.gmane.org/gmane.comp.version-control.git/264840\n> > \n> > which wants to turn off http.sslverify. That would mean it turns off for\n> > every submodule, too, but if you want to be choosy about your http\n> > variables, you should be using the \"http.$URL.sslverify\" form, to only\n> > affect specific servers (whether they are in submodules or not).\n> \n> I considered that, and thought that it might be dangerous, what with me\n> not vetting carefully which http.* variables are safe to pass on to the\n> submodules' update and which are not.\n\nBTW, just in case you or anybody else ends up playing around with this\nand finds your tests do not work as expected: the config pass-through\nfeature is somewhat broken for anything except cloning. I just posted\nsome fixes in:\n\n http://thread.gmane.org/gmane.comp.version-control.git/292466/focus=292875\n\n-Peff\n"},{"id":"284828","messageId":"CA+P7+xq-_D2Mszyjd11CyYLiKBBh9A2e1exaZQVmWz1qVKv7ug@mail.gmail.com","threadId":"42138","inReplyTo":"20160428134953.GB25364@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2016-04-28T15:37:10Z","receivedAt":"2016-04-28T15:37:10Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Apr 28, 2016 at 6:49 AM, Jeff King wrote:\n> On Thu, Apr 28, 2016 at 02:19:37PM +0200, Johannes Schindelin wrote:\n>\n>> > Should we consider just white-listing all of \"http.*\"?\n>> >\n>> > That would help other cases which have come up, like:\n>> >\n>> > http://thread.gmane.org/gmane.comp.version-control.git/264840\n>> >\n>> > which wants to turn off http.sslverify. That would mean it turns off for\n>> > every submodule, too, but if you want to be choosy about your http\n>> > variables, you should be using the \"http.$URL.sslverify\" form, to only\n>> > affect specific servers (whether they are in submodules or not).\n>>\n>> I considered that, and thought that it might be dangerous, what with me\n>> not vetting carefully which http.* variables are safe to pass on to the\n>> submodules' update and which are not.\n>>\n>> So I had a look now, and the most prominent potential problem is the\n>> http.cookieFile setting, which could be reused all of a sudden if we\n>> made my patch more general.\n>>\n>> But then, we are talking about the code that filters what gets passed via\n>> the *command-line*. And to be quite honest, I am not sure that we should\n>> actually filter out *any* of these settings.\n>\n> The intent of the whitelist (from my recollection of the discussion) is\n> to filter out config that must be repo-specific. E.g., core.worktree or\n> core.bare should definitely _not_ be passed to a submodule.\n>\n> I don't know if there are others. We started with a whitelist because it\n> was the smallest and safest change away from the status quo. A blacklist\n> would also work, with the risk that we might let through nonsense in\n> some cases (but only if the user triggers us to do so).\n>\n>> The commit message that introduced this particular filtering has this\n>> rationale to let only credential.* through:\n>>\n>> GIT_CONFIG_PARAMETERS is special, and we actually do want to\n>> preserve these settings. However, we do not want to preserve all\n>> configuration as many things should be left specific to the parent\n>> project.\n>>\n>> Add a git submodule--helper function, sanitize-config, which shall be\n>> used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs\n>> except a small subset that are known to be safe and necessary.\n>>\n>> Dunno. I tried to err on the side of caution... But this sounds maybe a\n>> bit *too* cautious?\n>\n> So if we all agree that the sanitizing is really about preventing\n> repo-specific variables from leaking, and not any kind of security\n> boundary, I think we should generally be pretty liberal in whitelisting\n> things.\n>\n> I can certainly come up with a pathological case where using it as a\n> security boundary may have some practical use, but in general I think it\n> is mostly getting in the way of what users are trying to do.\n>\n> -Peff\n\nI think I prefer a blacklist approach, since it reduces the need for\nfuture changes, since most cases will either not put config on the\nenvironment or (based on feedback on the mailing list and bug reports)\nthe user will believe it should be applied.\n\nA black list which only removed configurations we know are harmful\nwould be easier to maintain but risks new additions forgetting to do\nso. A whitelist means we only fix things as they come up but also\nmeans we aren't \"breaking\" anything that works today, where as a\nblacklist could break something that works today.\n\nThanks,\nJake\n"},{"id":"284830","messageId":"20160428153902.GF31063@sigill.intra.peff.net","threadId":"42138","inReplyTo":"CA+P7+xq-_D2Mszyjd11CyYLiKBBh9A2e1exaZQVmWz1qVKv7ug@mail.gmail.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T15:39:02Z","receivedAt":"2016-04-28T15:39:02Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 08:37:10AM -0700, Jacob Keller wrote:\n\n> I think I prefer a blacklist approach, since it reduces the need for\n> future changes, since most cases will either not put config on the\n> environment or (based on feedback on the mailing list and bug reports)\n> the user will believe it should be applied.\n> \n> A black list which only removed configurations we know are harmful\n> would be easier to maintain but risks new additions forgetting to do\n> so. A whitelist means we only fix things as they come up but also\n> means we aren't \"breaking\" anything that works today, where as a\n> blacklist could break something that works today.\n\nI think the key thing with a blacklist is somebody has to go to the work\nto audit the existing keys.\n\n-Peff\n"},{"id":"284834","messageId":"CAGZ79kZFLTARQ25h4u4SGgNn=Q4TQi-kxFLN3sQvOmejsRmAWA@mail.gmail.com","threadId":"42138","inReplyTo":"20160428153902.GF31063@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-28T16:09:44Z","receivedAt":"2016-04-28T16:09:44Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Apr 28, 2016 at 8:39 AM, Jeff King wrote:\n> On Thu, Apr 28, 2016 at 08:37:10AM -0700, Jacob Keller wrote:\n>\n>> I think I prefer a blacklist approach, since it reduces the need for\n>> future changes, since most cases will either not put config on the\n>> environment or (based on feedback on the mailing list and bug reports)\n>> the user will believe it should be applied.\n>>\n>> A black list which only removed configurations we know are harmful\n>> would be easier to maintain but risks new additions forgetting to do\n>> so. A whitelist means we only fix things as they come up but also\n>> means we aren't \"breaking\" anything that works today, where as a\n>> blacklist could break something that works today.\n>\n> I think the key thing with a blacklist is somebody has to go to the work\n> to audit the existing keys.\n\nWould it be sufficient to wait until someone screams at the mailing list\nfor some key to be blacklisted? (I mean in the short term that would be\nof less quality, but relying on the larger community would result in a better\nend result? So your going through is just a jump start this process of\nlistening to the community?)\n\nThanks,\nStefan\n\n>\n> -Peff\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"284841","messageId":"20160428165031.GA31421@sigill.intra.peff.net","threadId":"42138","inReplyTo":"CAGZ79kZFLTARQ25h4u4SGgNn=Q4TQi-kxFLN3sQvOmejsRmAWA@mail.gmail.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T16:50:31Z","receivedAt":"2016-04-28T16:50:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 09:09:44AM -0700, Stefan Beller wrote:\n\n> > I think the key thing with a blacklist is somebody has to go to the work\n> > to audit the existing keys.\n> \n> Would it be sufficient to wait until someone screams at the mailing list\n> for some key to be blacklisted? (I mean in the short term that would be\n> of less quality, but relying on the larger community would result in a better\n> end result? So your going through is just a jump start this process of\n> listening to the community?)\n\nYeah, I think ultimately we will rely on the community. But I would feel\na lot more comfortable if somebody made at least a single pass.\n\nI'll be curious what Junio says, too. I generally defer to him on how\nconservative we want to be in cases like this.\n\n-Peff\n"},{"id":"284857","messageId":"xmqq1t5p5z8v.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160428165031.GA31421@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T19:06:56Z","receivedAt":"2016-04-28T19:06:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Thu, Apr 28, 2016 at 09:09:44AM -0700, Stefan Beller wrote:\n>\n>> > I think the key thing with a blacklist is somebody has to go to the work\n>> > to audit the existing keys.\n>> \n>> Would it be sufficient to wait until someone screams at the mailing list\n>> for some key to be blacklisted? (I mean in the short term that would be\n>> of less quality, but relying on the larger community would result in a better\n>> end result? So your going through is just a jump start this process of\n>> listening to the community?)\n>\n> Yeah, I think ultimately we will rely on the community. But I would feel\n> a lot more comfortable if somebody made at least a single pass.\n>\n> I'll be curious what Junio says, too. I generally defer to him on how\n> conservative we want to be in cases like this.\n\nStarting from an empty whitelist and waiting for people to scream\nwith valid use cases would automatically give us the single pass to\nidentify the set of essential ones that users must be able to pass,\nno?\n\nOf course, the screamed proposal to add something to whitelist must\nbe vetted (i.e. \"yeah, we can see passing X in _your_ usecase might\nbe useful, but here are downsides (e.g. security implications) of\nallowing X in other usecases\"). And we might even find that it is\ninsufficient safety to allow/disallow per variable name during that\ndiscussion, in which case choice between whitelist and blacklist\nbecomes moot.\n"},{"id":"284858","messageId":"20160428191038.GA10574@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqq1t5p5z8v.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T19:10:38Z","receivedAt":"2016-04-28T19:10:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 12:06:56PM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > On Thu, Apr 28, 2016 at 09:09:44AM -0700, Stefan Beller wrote:\n> >\n> >> > I think the key thing with a blacklist is somebody has to go to the work\n> >> > to audit the existing keys.\n> >> \n> >> Would it be sufficient to wait until someone screams at the mailing list\n> >> for some key to be blacklisted? (I mean in the short term that would be\n> >> of less quality, but relying on the larger community would result in a better\n> >> end result? So your going through is just a jump start this process of\n> >> listening to the community?)\n> >\n> > Yeah, I think ultimately we will rely on the community. But I would feel\n> > a lot more comfortable if somebody made at least a single pass.\n> >\n> > I'll be curious what Junio says, too. I generally defer to him on how\n> > conservative we want to be in cases like this.\n> \n> Starting from an empty whitelist and waiting for people to scream\n> with valid use cases would automatically give us the single pass to\n> identify the set of essential ones that users must be able to pass,\n> no?\n\nIt's definitely sufficient, it's just annoying if a user shows up every\nweek and says \"I want X.Y\", and then somebody else shows up a week later\nand says \"I want X.Z\".\n\nAre we serving any purpose in vetting each one (and if so, what)?\n\n-Peff\n"},{"id":"284861","messageId":"xmqqwpnh4joq.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160428191038.GA10574@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T19:28:21Z","receivedAt":"2016-04-28T19:28:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> It's definitely sufficient, it's just annoying if a user shows up every\n> week and says \"I want X.Y\", and then somebody else shows up a week later\n> and says \"I want X.Z\".\n>\n> Are we serving any purpose in vetting each one (and if so, what)?\n\nPersonally I do not think we would need to filter _anything_ if we\ncan tell that the user directly said\n\n\tgit -c var1=val1 -c var2=val2 $cmd ...\n\nand \"git $cmd\" ended up needing to spawn another \"git\" subcommand,\npossibly in some other repository (i.e. \"$cmd\" in this case is\nlikely to be \"submodule\", but in principle it does not have to be).\nIf the user somehow gives variables like core.worktree that are\ninappropriate to be applied across repositories, that's user's\nproblem, i.e. \"don't do it then if it hurts\".\n\nIf we are doing any filtering, however, it is always hard, if not\nimpossible, to take away what we originally granted, even by\nmistake, for any reason, even for correctness or for security, in a\nlater release.\n\nWe probably could sidestep it by introducing an end-user\nconfigurable \"whitelist\" somewhere.\n"},{"id":"284862","messageId":"CAGZ79kYoRP=rkfaL+rLapmvouUdPxXGBr-KWOLhL94bYB1B2-w@mail.gmail.com","threadId":"42138","inReplyTo":"xmqqwpnh4joq.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-28T19:34:31Z","receivedAt":"2016-04-28T19:34:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Apr 28, 2016 at 12:28 PM, Junio C Hamano wrote:\n> Jeff King writes:\n>\n>> It's definitely sufficient, it's just annoying if a user shows up every\n>> week and says \"I want X.Y\", and then somebody else shows up a week later\n>> and says \"I want X.Z\".\n>>\n>> Are we serving any purpose in vetting each one (and if so, what)?\n>\n> Personally I do not think we would need to filter _anything_ if we\n> can tell that the user directly said\n>\n> git -c var1=val1 -c var2=val2 $cmd ...\n>\n> and \"git $cmd\" ended up needing to spawn another \"git\" subcommand,\n> possibly in some other repository (i.e. \"$cmd\" in this case is\n> likely to be \"submodule\", but in principle it does not have to be).\n> If the user somehow gives variables like core.worktree that are\n> inappropriate to be applied across repositories, that's user's\n> problem, i.e. \"don't do it then if it hurts\".\n\nSo when going with that philosophy, the user might be missing\nswitches like\n\n -c-for-this-repo-only core.worktree=... -c\nsubmodule.worktree=align-relative-to-parent\n\ni.e. when shifting the responsibility to the user, we need to have\nswitches to pass options to the repository or a subset of submodules?\n\n>\n> If we are doing any filtering, however, it is always hard, if not\n> impossible, to take away what we originally granted, even by\n> mistake, for any reason, even for correctness or for security, in a\n> later release.\n>\n> We probably could sidestep it by introducing an end-user\n> configurable \"whitelist\" somewhere.\n>\n>\n"},{"id":"284863","messageId":"xmqqshy54j2h.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"89d0024450b0e6e9997ad9e3d681248bde1bafc0.1461837783.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T19:41:42Z","receivedAt":"2016-04-28T19:41:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> - if (starts_with(var, \"credential.\"))\n> + if (starts_with(var, \"credential.\") ||\n> + (starts_with(var, \"http.\") &&\n> + ends_with(var, \".extraheader\")))\n\nI know you are fond of indenting with HT without aligning things,\nbut this is going too far in the quest of making the code\nunreadable.\n\n if (starts_with(var, \"credential.\") ||\n (starts_with(var, \"http.\") && ends_with(var, \".extraheader\")))\n\nwould make iteasier to see what are the top-level items (there are two)\nand how they are related (just one of them needs to be satisfied).\n\nAssuming that we will discover more variables that can be safely\npassed, I'd rather see the above written like this, though:\n\n if (starts_with(var, \"credential.\"))\n return 1;\n if (starts_with(var, \"http.\") && ends_with(var, \".extraheader\"))\n return 1;\n\n return 0;\n\nOr even something along this line:\n\n struct whitelist {\n const char *prefix;\n const char *suffix;\n } whitelist[] = {\n { \"credential.\", NULL },\n { \"http.\", \".extraheader\" },\n };\n\n for (i = 0; i < ARRAY_SIZE(whitelist); i++) {\n struct whitelist *w = &whitelist[i];\n if ((!w->prefix || starts_with(var, w->prefix)) &&\n (!w->suffix || ends_with(var, w->suffix)))\n return 1;\n }\n return 0;\n"},{"id":"284864","messageId":"xmqqoa8t4il8.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"CAGZ79kYoRP=rkfaL+rLapmvouUdPxXGBr-KWOLhL94bYB1B2-w@mail.gmail.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T19:52:03Z","receivedAt":"2016-04-28T19:52:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> So when going with that philosophy, the user might be missing\n> switches like\n>\n> -c-for-this-repo-only core.worktree=... -c\n> submodule.worktree=align-relative-to-parent\n>\n> i.e. when shifting the responsibility to the user, we need to have\n> switches to pass options to the repository or a subset of submodules?\n\nI think that is an excellent illlustration of the issue by an\nexample of what we shouldn't do ;-)\n\n\"git\" is not always about submodules, so \"-c-but-not-for-submodules\"\noption does not belong to \"git\" wrapper.\n\nUsers use \"git -c\" and hope to affect what happens in submodules,\nonly because \"git submodule\" support is still immature and does not\nhave options to do that. You certainly smell a linkage between\n\"pass options to a selected subset of submodules\" and your recent\n\"give labels to submodules so that they can be named with *group\nsyntax\" topic, no?\n"},{"id":"284865","messageId":"xmqqk2jh4iif.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"xmqqoa8t4il8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T19:53:44Z","receivedAt":"2016-04-28T19:53:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Users use \"git -c\" and hope to affect what happens in submodules,\n> only because \"git submodule\" support is still immature and does not\n> have options to do that. You certainly smell a linkage between\n> \"pass options to a selected subset of submodules\" and your recent\n> \"give labels to submodules so that they can be named with *group\n> syntax\" topic, no?\n\nBy \"options to do that\", I meant \"options to specify 'these\nconfigurations are passed down to part of this command that operates\non these submodules'\".\n"},{"id":"284866","messageId":"CAGZ79kY1TB9sZpZcihiC5rPmbf8qD9KsAZsWDcon+UnHG7R3jw@mail.gmail.com","threadId":"42138","inReplyTo":"xmqqoa8t4il8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-28T20:01:09Z","receivedAt":"2016-04-28T20:01:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Apr 28, 2016 at 12:52 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> So when going with that philosophy, the user might be missing\n>> switches like\n>>\n>> -c-for-this-repo-only core.worktree=... -c\n>> submodule.worktree=align-relative-to-parent\n>>\n>> i.e. when shifting the responsibility to the user, we need to have\n>> switches to pass options to the repository or a subset of submodules?\n>\n> I think that is an excellent illlustration of the issue by an\n> example of what we shouldn't do ;-)\n\nSo rather have a white / black list?\n\nCould we have a pre curated list with the list easily changed by the user?\nSo instead of screaming at the mailing list they can work around easily,\nand eventually someone fixes the \"stupid\" default?\n\nMaybe a user would want to do\n\n git config --global submodule-credentials-filter-white-list \"host.*\"\n\ninstead? (Naming intentionally bad)\n\n>\n> \"git\" is not always about submodules, so \"-c-but-not-for-submodules\"\n> option does not belong to \"git\" wrapper.\n>\n> Users use \"git -c\" and hope to affect what happens in submodules,\n> only because \"git submodule\" support is still immature and does not\n> have options to do that. You certainly smell a linkage between\n> \"pass options to a selected subset of submodules\" and your recent\n> \"give labels to submodules so that they can be named with *group\n> syntax\" topic, no?\n\nI thought about that for a split second, but no. I mean it in the\nmost general way possible.\n\nIt doesn't even have to be a submodule related thing at all.\n\nI can imagine that `git gc` could learn to walk nested repos\n(not submodules, just repos on disk inside the work tree).\nAnd for that use case we'd maybe want to have a setting\nto pack the nested repos more aggressively than the toplevel repo.\n\n(Not sure if there would be a use case or such a thing, but it is the\nfirst I came up with)\n"},{"id":"284873","messageId":"20160428210026.GA12268@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqqwpnh4joq.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T21:00:27Z","receivedAt":"2016-04-28T21:00:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 12:28:21PM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > It's definitely sufficient, it's just annoying if a user shows up every\n> > week and says \"I want X.Y\", and then somebody else shows up a week later\n> > and says \"I want X.Z\".\n> >\n> > Are we serving any purpose in vetting each one (and if so, what)?\n> \n> Personally I do not think we would need to filter _anything_ if we\n> can tell that the user directly said\n> \n> \tgit -c var1=val1 -c var2=val2 $cmd ...\n> \n> and \"git $cmd\" ended up needing to spawn another \"git\" subcommand,\n> possibly in some other repository (i.e. \"$cmd\" in this case is\n> likely to be \"submodule\", but in principle it does not have to be).\n> If the user somehow gives variables like core.worktree that are\n> inappropriate to be applied across repositories, that's user's\n> problem, i.e. \"don't do it then if it hurts\".\n\nRight, we are talking about that direct case here. And any time our\nfilter heuristic lets something through, it is probably \"if it hurts\ndon't do it\" as the worst case.\n\nSo I think the only two cases worth filtering are:\n\n 1. Ones where we _know_ that the config is nonsense to pass along,\n _and_ where a user might conceivably make use of the\n just-the-top-level version of it (core.worktree\n comes to mind, though of course they are probably better served by\n \"--work-tree\" in such a case).\n\n 2. An option where we think there may be some security implication.\n Setting \"http.sslverify\" to false does have some security\n implications (\"oops, I only meant to turn off verification for the\n root repo, and I got MiTM-attacked for the submodules!\"). But it's\n so obscure and unlikely that I think the benefit outweighs it.\n\n And I can't think of any other cases whose security implications\n aren't similarly unlikely. But I haven't carefully gone down the\n list (and as I said, I'd be hesitant to support a blacklist until\n _somebody_ takes the time to do so).\n\n> If we are doing any filtering, however, it is always hard, if not\n> impossible, to take away what we originally granted, even by\n> mistake, for any reason, even for correctness or for security, in a\n> later release.\n\nYep, agreed.\n\nI am OK staying with a whitelist. But I think we should be fairly\nlenient in whitelisting hierarchies that people have a use for, and\nwhich do not violate (1) or (2) above.\n\n> We probably could sidestep it by introducing an end-user\n> configurable \"whitelist\" somewhere.\n\nUgh. Please no. I do not want to have to think about explaining to\nsomebody that they can accomplish what they want with submodules, but\nonly by pre-configuring their ~/.gitconfig to allow certain keys so that\nthey can pass the appropriate config on the command line.\n\n-Peff\n"},{"id":"284874","messageId":"20160428210342.GB12268@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqqoa8t4il8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T21:03:42Z","receivedAt":"2016-04-28T21:03:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 12:52:03PM -0700, Junio C Hamano wrote:\n\n> \"git\" is not always about submodules, so \"-c-but-not-for-submodules\"\n> option does not belong to \"git\" wrapper.\n> \n> Users use \"git -c\" and hope to affect what happens in submodules,\n> only because \"git submodule\" support is still immature and does not\n> have options to do that. You certainly smell a linkage between\n> \"pass options to a selected subset of submodules\" and your recent\n> \"give labels to submodules so that they can be named with *group\n> syntax\" topic, no?\n\nKeep in mind that submodule interactions may be triggered from other\nnon-submodule commands. So \"git fetch\", for instance, may end up caring\nabout whether you pass \"http.*\" or \"credential.*\" down to the\nsubmodules. I do not think \"fetch\" should grow submodule-specific\noptions, so that pretty much leaves \"git\" options as the only place\nleft.\n\n-Peff\n"},{"id":"284876","messageId":"CAGZ79kZF4UeNuwBoA-Vw0OFHin4=8GTWPK-GT+5fsEuCFe7Now@mail.gmail.com","threadId":"42138","inReplyTo":"20160428210026.GA12268@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-28T21:08:43Z","receivedAt":"2016-04-28T21:08:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Apr 28, 2016 at 2:00 PM, Jeff King wrote:\n> On Thu, Apr 28, 2016 at 12:28:21PM -0700, Junio C Hamano wrote:\n>\n>> Jeff King writes:\n>>\n>> > It's definitely sufficient, it's just annoying if a user shows up every\n>> > week and says \"I want X.Y\", and then somebody else shows up a week later\n>> > and says \"I want X.Z\".\n>> >\n>> > Are we serving any purpose in vetting each one (and if so, what)?\n>>\n>> Personally I do not think we would need to filter _anything_ if we\n>> can tell that the user directly said\n>>\n>> git -c var1=val1 -c var2=val2 $cmd ...\n>>\n>> and \"git $cmd\" ended up needing to spawn another \"git\" subcommand,\n>> possibly in some other repository (i.e. \"$cmd\" in this case is\n>> likely to be \"submodule\", but in principle it does not have to be).\n>> If the user somehow gives variables like core.worktree that are\n>> inappropriate to be applied across repositories, that's user's\n>> problem, i.e. \"don't do it then if it hurts\".\n>\n> Right, we are talking about that direct case here. And any time our\n> filter heuristic lets something through, it is probably \"if it hurts\n> don't do it\" as the worst case.\n>\n> So I think the only two cases worth filtering are:\n>\n> 1. Ones where we _know_ that the config is nonsense to pass along,\n> _and_ where a user might conceivably make use of the\n> just-the-top-level version of it (core.worktree\n> comes to mind, though of course they are probably better served by\n> \"--work-tree\" in such a case).\n\nMy gut reaction to this:\nIn this specific case I would rather error out, as you never want to have\ncore.worktree to point at the same dir for all of the repo and submodules.\n\nThinking about it further, I am not so sure any more.\n(What if you have multiple submodules tracking the same project\nand you want to see each submodule version with the one worktree you point to?\nHighly unlikely edge case, but it voids the /never/ assumption of my\ngut reaction)\n\n\n>\n> 2. An option where we think there may be some security implication.\n> Setting \"http.sslverify\" to false does have some security\n> implications (\"oops, I only meant to turn off verification for the\n> root repo, and I got MiTM-attacked for the submodules!\"). But it's\n> so obscure and unlikely that I think the benefit outweighs it.\n>\n> And I can't think of any other cases whose security implications\n> aren't similarly unlikely. But I haven't carefully gone down the\n> list (and as I said, I'd be hesitant to support a blacklist until\n> _somebody_ takes the time to do so).\n\nI think in this case, it is more likely for the user to say:\n\n I know my ssl is so borked, so I want all traffic with no ssl\n(including submodules).\n\n>\n>> If we are doing any filtering, however, it is always hard, if not\n>> impossible, to take away what we originally granted, even by\n>> mistake, for any reason, even for correctness or for security, in a\n>> later release.\n>\n> Yep, agreed.\n>\n> I am OK staying with a whitelist. But I think we should be fairly\n> lenient in whitelisting hierarchies that people have a use for, and\n> which do not violate (1) or (2) above.\n>\n>> We probably could sidestep it by introducing an end-user\n>> configurable \"whitelist\" somewhere.\n>\n> Ugh. Please no. I do not want to have to think about explaining to\n> somebody that they can accomplish what they want with submodules, but\n> only by pre-configuring their ~/.gitconfig to allow certain keys so that\n> they can pass the appropriate config on the command line.\n\nI view the whitelist more like an \"emergency knob to turn, because the\ndevelopers did it wrong and I want it now\". the general case should be\ncovered by a mechanism we provide?\n\nThanks,\nStefan\n\n>\n> -Peff\n"},{"id":"284878","messageId":"CAGZ79kY8s9pV_Lb8D0OYCrarDyC6v2U0VkaJSJBnviditYK9aA@mail.gmail.com","threadId":"42138","inReplyTo":"20160428210342.GB12268@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-28T21:12:37Z","receivedAt":"2016-04-28T21:12:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Apr 28, 2016 at 2:03 PM, Jeff King wrote:\n> On Thu, Apr 28, 2016 at 12:52:03PM -0700, Junio C Hamano wrote:\n>\n>> \"git\" is not always about submodules, so \"-c-but-not-for-submodules\"\n>> option does not belong to \"git\" wrapper.\n>>\n>> Users use \"git -c\" and hope to affect what happens in submodules,\n>> only because \"git submodule\" support is still immature and does not\n>> have options to do that. You certainly smell a linkage between\n>> \"pass options to a selected subset of submodules\" and your recent\n>> \"give labels to submodules so that they can be named with *group\n>> syntax\" topic, no?\n>\n> Keep in mind that submodule interactions may be triggered from other\n> non-submodule commands. So \"git fetch\", for instance, may end up caring\n> about whether you pass \"http.*\" or \"credential.*\" down to the\n> submodules.\n\nOr clone. I recently sent origin/sb/clone-shallow-passthru\nwhich adds more options to a non-submodule command.\nI think a command line option there is better than a \"git -c $OPTION\"\n\n> I do not think \"fetch\" should grow submodule-specific\n> options, so that pretty much leaves \"git\" options as the only place\n> left.\n\nI would rather see it in fetch as in the generic Git?\n\nStefan\n\n>\n> -Peff\n"},{"id":"284879","messageId":"20160428212011.GC12268@sigill.intra.peff.net","threadId":"42138","inReplyTo":"CAGZ79kZF4UeNuwBoA-Vw0OFHin4=8GTWPK-GT+5fsEuCFe7Now@mail.gmail.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-28T21:20:11Z","receivedAt":"2016-04-28T21:20:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 02:08:43PM -0700, Stefan Beller wrote:\n\n> > 1. Ones where we _know_ that the config is nonsense to pass along,\n> > _and_ where a user might conceivably make use of the\n> > just-the-top-level version of it (core.worktree\n> > comes to mind, though of course they are probably better served by\n> > \"--work-tree\" in such a case).\n> \n> My gut reaction to this:\n> In this specific case I would rather error out, as you never want to have\n> core.worktree to point at the same dir for all of the repo and submodules.\n\nBut then you're erroring out on a case that currently works today (we\napply core.worktree to the root repo, and ignore it for the others),\nwhich I think is worth avoiding.\n\n> Thinking about it further, I am not so sure any more.\n> (What if you have multiple submodules tracking the same project\n> and you want to see each submodule version with the one worktree you point to?\n> Highly unlikely edge case, but it voids the /never/ assumption of my\n> gut reaction)\n\nI think this falls in my \"nonsense\" category. Especially when there are\nother ways to handle it (e.g., by asking \"git submodule foreach\" to look\nat each directory).\n\n> I view the whitelist more like an \"emergency knob to turn, because the\n> developers did it wrong and I want it now\". the general case should be\n> covered by a mechanism we provide?\n\nI think the emergency knob is \"visit each submodule individually if you\nhave want to do something clever in each one\" (or \"ask git not to\nrecurse into submodules\" for the opposite effect). The use cases I have\nseen discussed are:\n\n 1. Convenience. People expect to set some global-ish config like\n credential.helper, and have it applied uniformly.\n\n 2. Inserting config into awkward parts of the call chain. E.g., I\n think Lars is mostly interested in speeding up clones where the\n submodules have a lot of slow clean/smudge filters. So he wants to\n disable \"filter.*\" _just_ for the clone, but there's no easy way to\n intercept the clone step of each submodule, stop, and the run the\n individual checkout with \"git -c filter.foo.smudge=\". I'm sure\n there's probably a way to hack it with plumbing, but it fails in\n \"convenience\".\n\n-Peff\n"},{"id":"284880","messageId":"xmqqwpnh2w0t.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160428210342.GB12268@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T22:44:50Z","receivedAt":"2016-04-28T22:44:50Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Keep in mind that submodule interactions may be triggered from other\n> non-submodule commands. So \"git fetch\", for instance, may end up caring\n> about whether you pass \"http.*\" or \"credential.*\" down to the\n> submodules.\n\n> I do not think \"fetch\" should grow submodule-specific\n> options,...\n\nThe updated \"git fetch\" needs to grow submodule-specific options to\nat least either enable or disable \"recurse into submodules\", and\nthat is true even if the default behaviour in the future were to\nrecurse into submodules in a top-level project repository that has\nsubmodules (i.e. you must have \"git fetch --no-recurse-submodules\"\noption). \"Please use these configuration when you do recurse into\nthem\" options are very much submodule specific in the same way.\n\nIf anything, with Stefan's \"submodule groups\" thing, I would expect\nmore commands (like \"git diff\") to become aware of the possibility\nof descending into submodules (and even \"selected subset of\nsubmodules\"), and they need command line options to tell which ones\nto descend into. \"Here is the set of submodules I want you to\ndescend into\" and \"By the way, I want you to use these settings\nwhile working in them\" would go naturally hand in hand, I would\nimagine, so I strongly disagree with the statement \"fetch should not\ngrow submodule specific options\".\n\nOf course, we can stop teaching --recurse-submodules to non \"git\nsubmodule\" commands and concentrate on improving \"git submodule\" as\nthe end-user facing command, or cover usecases to work on subset of\nsubmodules with \"git submodule foreach\". But I do not think that is\nwhat you are advocating for.\n"},{"id":"284881","messageId":"xmqqshy52vwb.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"CAGZ79kY1TB9sZpZcihiC5rPmbf8qD9KsAZsWDcon+UnHG7R3jw@mail.gmail.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-28T22:47:32Z","receivedAt":"2016-04-28T22:47:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> It doesn't even have to be a submodule related thing at all.\n>\n> I can imagine that `git gc` could learn to walk nested repos\n> (not submodules, just repos on disk inside the work tree).\n> And for that use case we'd maybe want to have a setting\n> to pack the nested repos more aggressively than the toplevel repo.\n>\n> (Not sure if there would be a use case or such a thing, but it is the\n> first I came up with)\n\nI do not think we are in the business of mucking with \"dump of\nunrelated collections of repositories, whose inter-relationship the\nuser did not tell us anything about\".\n\nI however can foresee a value in \"git gc --recurse-submodules\", and\nI could see users may want to gc different submodules in different\nways (which again brings us back to your \"submodule group\" thing).\n"},{"id":"284911","messageId":"alpine.DEB.2.20.1604291417580.9313@virtualbox","threadId":"42138","inReplyTo":"20160428210026.GA12268@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-29T12:29:25Z","receivedAt":"2016-04-29T12:29:25Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Thu, 28 Apr 2016, Jeff King wrote:\n\n> On Thu, Apr 28, 2016 at 12:28:21PM -0700, Junio C Hamano wrote:\n> \n> > Jeff King writes:\n> > \n> > > It's definitely sufficient, it's just annoying if a user shows up\n> > > every week and says \"I want X.Y\", and then somebody else shows up a\n> > > week later and says \"I want X.Z\".\n> > >\n> > > Are we serving any purpose in vetting each one (and if so, what)?\n> > \n> > Personally I do not think we would need to filter _anything_ if we can\n> > tell that the user directly said\n> > \n> > \tgit -c var1=val1 -c var2=val2 $cmd ...\n> > \n> > and \"git $cmd\" ended up needing to spawn another \"git\" subcommand,\n> > possibly in some other repository (i.e. \"$cmd\" in this case is likely\n> > to be \"submodule\", but in principle it does not have to be). If the\n> > user somehow gives variables like core.worktree that are inappropriate\n> > to be applied across repositories, that's user's problem, i.e. \"don't\n> > do it then if it hurts\".\n> \n> Right, we are talking about that direct case here. And any time our\n> filter heuristic lets something through, it is probably \"if it hurts\n> don't do it\" as the worst case.\n\nThe more I think about it, I actually think that we do the user a *really*\ngreat disservice by filtering the CONFIG_DATA_ENVIRONMENT. If I call\n\n\tgit -c ... $cmd\n\nand that configuration is *not* picked up, it is much worse than letting\nusers shoot themselves in their own feet by specifying config settings\nthat are *prone* to wreak havoc.\n\n> So I think the only two cases worth filtering are:\n> \n> 1. Ones where we _know_ that the config is nonsense to pass along,\n> _and_ where a user might conceivably make use of the\n> just-the-top-level version of it (core.worktree\n> comes to mind, though of course they are probably better served by\n> \"--work-tree\" in such a case).\n> 2. An option where we think there may be some security implication.\n> Setting \"http.sslverify\" to false does have some security\n> implications (\"oops, I only meant to turn off verification for the\n> root repo, and I got MiTM-attacked for the submodules!\"). But it's\n> so obscure and unlikely that I think the benefit outweighs it.\n\nI can see that happening when somebody calls an alias with `git -c ...`\nand that alias performs actions in the top-level project as well as all\nsubmodules.\n\nBut. Do we really have to be \"Big Daddy\" for users who do that?\n\n> I am OK staying with a whitelist.\n\nMe, too. But I am even more in favor of abandoning this \"we know what is\ngood for you\" approach, i.e. that whitelist that filters\nCONFIG_DATA_ENVIRONMENT.\n\nCiao,\nDscho\n"},{"id":"284913","messageId":"alpine.DEB.2.20.1604291429560.9313@virtualbox","threadId":"42138","inReplyTo":"xmqqshy54j2h.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-29T12:35:07Z","receivedAt":"2016-04-29T12:35:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Thu, 28 Apr 2016, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > - if (starts_with(var, \"credential.\"))\n> > + if (starts_with(var, \"credential.\") ||\n> > + (starts_with(var, \"http.\") &&\n> > + ends_with(var, \".extraheader\")))\n> \n> I know you are fond of indenting with HT without aligning things,\n> but this is going too far in the quest of making the code\n> unreadable.\n\nHah. I am actually not fond of anything there, but I go with the default\nin my vim when selecting lines and pressing the '=' key...\n\nIf you know off-hand how to teach my vim to use your preferred indenting,\nI'll gladly just brow-beat it into submission.\n\n> if (starts_with(var, \"credential.\") ||\n> (starts_with(var, \"http.\") && ends_with(var, \".extraheader\")))\n> \n> would make iteasier to see what are the top-level items (there are two)\n> and how they are related (just one of them needs to be satisfied).\n\nFine by me!\n\n> Assuming that we will discover more variables that can be safely\n> passed, I'd rather see the above written like this, though:\n> \n> if (starts_with(var, \"credential.\"))\n> return 1;\n> if (starts_with(var, \"http.\") && ends_with(var, \".extraheader\"))\n> return 1;\n> \n> return 0;\n> \n> Or even something along this line:\n> \n> struct whitelist {\n> const char *prefix;\n> const char *suffix;\n> } whitelist[] = {\n> { \"credential.\", NULL },\n> { \"http.\", \".extraheader\" },\n> };\n> \n> for (i = 0; i < ARRAY_SIZE(whitelist); i++) {\n> struct whitelist *w = &whitelist[i];\n> if ((!w->prefix || starts_with(var, w->prefix)) &&\n> (!w->suffix || ends_with(var, w->suffix)))\n> return 1;\n> }\n> return 0;\n\nIff. Iff we go with a white-list.\n\nHowever, I think you did a really good job arguing that the\nCONFIG_DATA_ENVIRONMENT filtering is, in fact, overzealous.\n\nJust let me know what to go with, and I'll update the patch accordingly.\n\nCiao,\nDscho\n"},{"id":"284914","messageId":"alpine.DEB.2.20.1604291443561.9313@virtualbox","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604291429560.9313@virtualbox","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-29T12:48:44Z","receivedAt":"2016-04-29T12:48:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Fri, 29 Apr 2016, Johannes Schindelin wrote:\n\n> On Thu, 28 Apr 2016, Junio C Hamano wrote:\n> \n> > Johannes Schindelin writes:\n> > \n> > > - if (starts_with(var, \"credential.\"))\n> > > + if (starts_with(var, \"credential.\") ||\n> > > + (starts_with(var, \"http.\") &&\n> > > + ends_with(var, \".extraheader\")))\n> > \n> > I know you are fond of indenting with HT without aligning things,\n> > but this is going too far in the quest of making the code\n> > unreadable.\n> \n> Hah. I am actually not fond of anything there, but I go with the default\n> in my vim when selecting lines and pressing the '=' key...\n> \n> If you know off-hand how to teach my vim to use your preferred indenting,\n> I'll gladly just brow-beat it into submission.\n\nFor the record, I think the default in vim is to indent two tabs after an\nunclosed parenthesis in an `if` line (the \"(2s\" just below\nhttp://vimdoc.sourceforge.net/htmldoc/indent.html#javascript-indenting in\nthe default cinoptions). The \"(0\" setting fixes this, I think, so I forced\nmy vim to use\n\n\tcinoptions=>s,e0,n0,f0,{0,}0,^0,L-1,:s,=s,l0,b0,gs,hs,ps,ts,is,+s,c3,C0,/0,(0,us,U0,w0,W0,m0,j0,J0,)20,*70,#0\n\nfrom now on.\n\nCiao,\nDscho\n"},{"id":"284917","messageId":"20160429131029.GA30931@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604291443561.9313@virtualbox","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-29T13:10:30Z","receivedAt":"2016-04-29T13:10:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Apr 29, 2016 at 02:48:44PM +0200, Johannes Schindelin wrote:\n\n> > If you know off-hand how to teach my vim to use your preferred indenting,\n> > I'll gladly just brow-beat it into submission.\n> \n> For the record, I think the default in vim is to indent two tabs after an\n> unclosed parenthesis in an `if` line (the \"(2s\" just below\n> http://vimdoc.sourceforge.net/htmldoc/indent.html#javascript-indenting in\n> the default cinoptions). The \"(0\" setting fixes this, I think, so I forced\n> my vim to use\n> \n> \tcinoptions=>s,e0,n0,f0,{0,}0,^0,L-1,:s,=s,l0,b0,gs,hs,ps,ts,is,+s,c3,C0,/0,(0,us,U0,w0,W0,m0,j0,J0,)20,*70,#0\n> \n> from now on.\n\nYeah, \"(0\" is the only non-default cinoption I use for git. Other than\nthat, everything fits our guidelines (I guess I explicitly make sure\nshiftwidth and tabstops are all at 8, but I think those are the\ndefaults, too).\n\n-Peff\n"},{"id":"284918","messageId":"20160429132617.GB30931@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1604291417580.9313@virtualbox","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-29T13:26:18Z","receivedAt":"2016-04-29T13:26:18Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Apr 29, 2016 at 02:29:25PM +0200, Johannes Schindelin wrote:\n\n> The more I think about it, I actually think that we do the user a *really*\n> great disservice by filtering the CONFIG_DATA_ENVIRONMENT. If I call\n> \n> \tgit -c ... $cmd\n> \n> and that configuration is *not* picked up, it is much worse than letting\n> users shoot themselves in their own feet by specifying config settings\n> that are *prone* to wreak havoc.\n\nThat's a good point. For every \"whoops, I didn't mean for this to kick\nin for the submodule!\" there is an equal and opposite \"whoops, this\nneeded to kick in for the submodule!\" case (e.g., instead of\nover-reaching turning http.sslverify off, you might be trying to turn it\n_on_ and fail to apply it in all cases).\n\nAnd making the rule \"-c applies to all sub-processes, period\" is much\nsimpler to explain.\n\n(Though of course it is still not entirely true. We clear the config\n when accessing another repository as a remote for fetching or pushing.\n But a good chunk of that is simply for consistency of all remotes, as\n we do not pass the environment across TCP sessions).\n\n> > So I think the only two cases worth filtering are:\n> > \n> > 1. Ones where we _know_ that the config is nonsense to pass along,\n> > _and_ where a user might conceivably make use of the\n> > just-the-top-level version of it (core.worktree\n> > comes to mind, though of course they are probably better served by\n> > \"--work-tree\" in such a case).\n> > 2. An option where we think there may be some security implication.\n> > Setting \"http.sslverify\" to false does have some security\n> > implications (\"oops, I only meant to turn off verification for the\n> > root repo, and I got MiTM-attacked for the submodules!\"). But it's\n> > so obscure and unlikely that I think the benefit outweighs it.\n> \n> I can see that happening when somebody calls an alias with `git -c ...`\n> and that alias performs actions in the top-level project as well as all\n> submodules.\n> \n> But. Do we really have to be \"Big Daddy\" for users who do that?\n\nI have more sympathy for cases that full under (1) above, just because\nthey currently work _now_. So it's possible somebody is currently doing\na thing that makes sense under the current rules, but will involve\nfoot-shooting under the new version.\n\n> > I am OK staying with a whitelist.\n> \n> Me, too. But I am even more in favor of abandoning this \"we know what is\n> good for you\" approach, i.e. that whitelist that filters\n> CONFIG_DATA_ENVIRONMENT.\n\nI could live with that, too. I've added Jens to the cc; his preference\nfor not blindly sharing config to submodules is part of what influenced\nme in the original discussion.\n\n-Peff\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/264840\n\n I think that's the right link, which I dug out of my\n <20160219043019.GA14764@sigill.intra.peff.net> from a few months\n ago. Gmane seems to be down.\n"},{"id":"284919","messageId":"20160429133527.GC30931@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqqwpnh2w0t.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-04-29T13:35:27Z","receivedAt":"2016-04-29T13:35:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 28, 2016 at 03:44:50PM -0700, Junio C Hamano wrote:\n\n> > I do not think \"fetch\" should grow submodule-specific\n> > options,...\n> \n> The updated \"git fetch\" needs to grow submodule-specific options to\n> at least either enable or disable \"recurse into submodules\", and\n> that is true even if the default behaviour in the future were to\n> recurse into submodules in a top-level project repository that has\n> submodules (i.e. you must have \"git fetch --no-recurse-submodules\"\n> option). \"Please use these configuration when you do recurse into\n> them\" options are very much submodule specific in the same way.\n\nI suppose so. I actually think it might be useful / convenient to have a\nglobal \"do not recurse\" environment variable (and possibly a matching\n\"git --recurse\" command-line flag to set it). Otherwise we end up with\nannoying propagation problems for command-line options when scripts call\nother programs, etc.\n\nIt's one of the reasons I think \"-c\" is so useful. Because it Just Works\nfor the whole environment of the command you are invoking, without\nhaving to worry about whether it's \"git fetch\" itself, or \"git remote\"\ncalling \"git fetch\", or a script calling \"git fetch\", or whatever.\n\nOf course there are downsides, too. If you're a script who really\n_doesn't_ want to propagate that option to your children, you have to\ntake an explicit step to prevent it. But I think it helps more often\nthan hurts.\n\nI think there are other options that would benefit, too. For example,\nthe way we handle \"--progress\", \"--verbose\", and \"--quiet\" is a bit of a\nmess. All of our programs understand these concepts to some degree, and\nthere have been numerous bugs involve in propagating their values to\nsub-programs. Having \"$GIT_VERBOSITY_LEVEL\" in the environment would\nprobably make this simpler and more consistent.\n\nBut you'll note that this email isn't accompanied by any patches. This\nis philosophical, and I'm not sure it's worth the effort to transition\nthings like verbosity at this point. So you may take it with the\nappropriate grain of salt. And especially I don't actually care all\nthat much about submodules myself, so I am fine with whatever people who\n_are_ actively using and developing them decide.\n\n-Peff\n"},{"id":"284944","messageId":"alpine.DEB.2.20.1604291756290.9313@virtualbox","threadId":"42138","inReplyTo":"20160429131029.GA30931@sigill.intra.peff.net","subject":"Re: [PATCH v5 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-04-29T15:56:40Z","receivedAt":"2016-04-29T15:56:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Fri, 29 Apr 2016, Jeff King wrote:\n\n> On Fri, Apr 29, 2016 at 02:48:44PM +0200, Johannes Schindelin wrote:\n> \n> > > If you know off-hand how to teach my vim to use your preferred indenting,\n> > > I'll gladly just brow-beat it into submission.\n> > \n> > For the record, I think the default in vim is to indent two tabs after an\n> > unclosed parenthesis in an `if` line (the \"(2s\" just below\n> > http://vimdoc.sourceforge.net/htmldoc/indent.html#javascript-indenting in\n> > the default cinoptions). The \"(0\" setting fixes this, I think, so I forced\n> > my vim to use\n> > \n> > \tcinoptions=>s,e0,n0,f0,{0,}0,^0,L-1,:s,=s,l0,b0,gs,hs,ps,ts,is,+s,c3,C0,/0,(0,us,U0,w0,W0,m0,j0,J0,)20,*70,#0\n> > \n> > from now on.\n> \n> Yeah, \"(0\" is the only non-default cinoption I use for git. Other than\n> that, everything fits our guidelines (I guess I explicitly make sure\n> shiftwidth and tabstops are all at 8, but I think those are the\n> defaults, too).\n\nThanks for the confirmation!\n\nCiao,\nDscho\n"},{"id":"285418","messageId":"cover.1462342213.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1461837783.git.johannes.schindelin@gmx.de","subject":"[PATCH v6 0/2] Add support for sending additional HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-04T06:14:07Z","receivedAt":"2016-05-04T06:14:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"My use case is an army of build agents that need only limited and\nselective access to otherwise private repositories.\n\nv6 supports submodules better by allowing\n\n\tgit -c http.extraheader submodule update\n\nto work as one would expect intuitively.\n\nJohannes Schindelin (2):\n http: support sending custom HTTP headers\n submodule: pass on http.extraheader config settings\n\n Documentation/config.txt | 6 ++++++\n builtin/submodule--helper.c | 3 ++-\n http-push.c | 10 +++++-----\n http.c | 35 ++++++++++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n t/lib-httpd/apache.conf | 8 ++++++++\n t/t5551-http-fetch-smart.sh | 16 ++++++++++++++++\n 8 files changed, 72 insertions(+), 11 deletions(-)\n\nInterdiff vs v5:\n\n diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n index b338f93..789e081 100644\n --- a/builtin/submodule--helper.c\n +++ b/builtin/submodule--helper.c\n @@ -128,8 +128,7 @@ static int module_name(int argc, const char **argv, const char *prefix)\n static int submodule_config_ok(const char *var)\n {\n \tif (starts_with(var, \"credential.\") ||\n -\t\t\t(starts_with(var, \"http.\") &&\n -\t\t\t ends_with(var, \".extraheader\")))\n +\t (starts_with(var, \"http.\") && ends_with(var, \".extraheader\")))\n \t\treturn 1;\n \treturn 0;\n }\n\n-- \n2.8.1.306.gff998f2\n"},{"id":"285419","messageId":"ff7edf51fe8baf57f6f4cab170602154e5b1e73e.1462342213.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462342213.git.johannes.schindelin@gmx.de","subject":"[PATCH v6 1/2] http: support sending custom HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-04T06:14:13Z","receivedAt":"2016-05-04T06:14:13Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"We introduce a way to send custom HTTP headers with all requests.\n\nThis allows us, for example, to send an extra token from build agents\nfor temporary access to private repositories. (This is the use case that\ntriggered this patch.)\n\nThis feature can be used like this:\n\n\tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n\nNote that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\na single list, overriding any previous call. This means we have to\ncollect _all_ of the headers we want to use into a single list, and\nfeed it to cURL in one shot. Since we already unconditionally set a\n\"pragma\" header when initializing the curl handles, we can add our new\nheaders to that list.\n\nFor callers which override the default header list (like probe_rpc),\nwe provide `http_copy_default_headers()` so they can do the same\ntrick.\n\nBig thanks to Jeff King and Junio Hamano for their outstanding help and\npatient reviews.\n\nSigned-off-by: Johannes Schindelin \n---\n Documentation/config.txt | 6 ++++++\n http-push.c | 10 +++++-----\n http.c | 35 ++++++++++++++++++++++++++++++++---\n http.h | 1 +\n remote-curl.c | 4 ++--\n t/lib-httpd/apache.conf | 8 ++++++++\n t/t5551-http-fetch-smart.sh | 7 +++++++\n 7 files changed, 61 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 42d2b50..c7bbe98 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1655,6 +1655,12 @@ http.emptyAuth::\n \ta username in the URL, as libcurl normally requires a username for\n \tauthentication.\n \n+http.extraHeader::\n+\tPass an additional HTTP header when communicating with a server. If\n+\tmore than one such entry exists, all of them are added as extra\n+\theaders. To allow overriding the settings inherited from the system\n+\tconfig, an empty value will reset the extra headers to the empty list.\n+\n http.cookieFile::\n \tFile containing previously stored cookie lines which should be used\n \tin the Git http session, if they match the server. The file format\ndiff --git a/http-push.c b/http-push.c\nindex bd60668..ae2b7f1 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tif (options & DAV_HEADER_IF) {\n \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n static void start_move(struct transfer_request *request)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \n \tslot = get_active_slot();\n \tslot->callback_func = process_response;\n@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n \tchar *ep;\n \tchar timeout_header[25];\n \tstruct remote_lock *lock = NULL;\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tchar *escaped;\n \n@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tstruct remote_ls_ctx ls;\n \n@@ -1204,7 +1204,7 @@ static int locking_available(void)\n \tstruct slot_results results;\n \tstruct strbuf in_buffer = STRBUF_INIT;\n \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n-\tstruct curl_slist *dav_headers = NULL;\n+\tstruct curl_slist *dav_headers = http_copy_default_headers();\n \tstruct xml_ctx ctx;\n \tint lock_flags = 0;\n \tchar *escaped;\ndiff --git a/http.c b/http.c\nindex 4304b80..985b995 100644\n--- a/http.c\n+++ b/http.c\n@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n \n static struct curl_slist *pragma_header;\n static struct curl_slist *no_pragma_header;\n+static struct curl_slist *extra_http_headers;\n \n static struct active_request_slot *active_queue_head;\n \n@@ -323,6 +324,19 @@ static int http_options(const char *var, const char *value, void *cb)\n #endif\n \t}\n \n+\tif (!strcmp(\"http.extraheader\", var)) {\n+\t\tif (!value) {\n+\t\t\treturn config_error_nonbool(var);\n+\t\t} else if (!*value) {\n+\t\t\tcurl_slist_free_all(extra_http_headers);\n+\t\t\textra_http_headers = NULL;\n+\t\t} else {\n+\t\t\textra_http_headers =\n+\t\t\t\tcurl_slist_append(extra_http_headers, value);\n+\t\t}\n+\t\treturn 0;\n+\t}\n+\n \t/* Fall back on the default ones */\n \treturn git_default_config(var, value, cb);\n }\n@@ -678,8 +692,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \tif (remote)\n \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n \n-\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n+\tpragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma: no-cache\");\n+\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n+\t\t\"Pragma:\");\n \n #ifdef USE_CURL_MULTI\n \t{\n@@ -765,6 +781,9 @@ void http_cleanup(void)\n #endif\n \tcurl_global_cleanup();\n \n+\tcurl_slist_free_all(extra_http_headers);\n+\textra_http_headers = NULL;\n+\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n@@ -1163,6 +1182,16 @@ int run_one_slot(struct active_request_slot *slot,\n \treturn handle_curl_result(results);\n }\n \n+struct curl_slist *http_copy_default_headers(void)\n+{\n+\tstruct curl_slist *headers = NULL, *h;\n+\n+\tfor (h = extra_http_headers; h; h = h->next)\n+\t\theaders = curl_slist_append(headers, h->data);\n+\n+\treturn headers;\n+}\n+\n static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n {\n \tchar *ptr;\n@@ -1380,7 +1409,7 @@ static int http_request(const char *url,\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tconst char *accept_language;\n \tint ret;\ndiff --git a/http.h b/http.h\nindex 4ef4bbd..36f558b 100644\n--- a/http.h\n+++ b/http.h\n@@ -106,6 +106,7 @@ extern void step_active_slots(void);\n extern void http_init(struct remote *remote, const char *url,\n \t\t int proactive_auth);\n extern void http_cleanup(void);\n+extern struct curl_slist *http_copy_default_headers(void);\n \n extern long int git_curl_ipresolve;\n extern int active_requests;\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 15e48e2..672b382 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tstruct strbuf buf = STRBUF_INIT;\n \tint err;\n \n@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n static int post_rpc(struct rpc_state *rpc)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = NULL;\n+\tstruct curl_slist *headers = http_copy_default_headers();\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 9317ba0..b8ed96f 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -102,6 +102,14 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \tHeader set Set-Cookie name=value\n \n+\n+\t\n+\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n+\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n+\t\n+\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n+\tSetEnv GIT_HTTP_EXPORT_ALL\n+\n ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error/ error.sh/\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex 58207d8..e44fe72 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n \ttest_line_count = 100000 tags\n '\n \n+test_expect_success 'custom http headers' '\n+\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+'\n+\n stop_httpd\n test_done\n-- \n2.8.1.306.gff998f2\n"},{"id":"285420","messageId":"df040cfd84b354ca206a5cb1430a26ae8afa5a45.1462342213.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462342213.git.johannes.schindelin@gmx.de","subject":"[PATCH v6 2/2] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-04T06:14:20Z","receivedAt":"2016-05-04T06:14:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To support this developer's use case of allowing build agents token-based\naccess to private repositories, we introduced the http.extraheader\nfeature, allowing extra HTTP headers to be sent along with every HTTP\nrequest.\n\nThis patch allows us to configure these extra HTTP headers for use with\n`git submodule update`, too. It requires somewhat special handling:\nsubmodules do not share the parent project's config. It would be\nincorrect to simply reuse that specific part of the parent's config.\nInstead, the config option needs to be specified on the command-line or\nin ~/.gitconfig or friends.\n\nExample: git -c http.extraheader=\"Secret: Sauce\" submodule update --init\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/submodule--helper.c | 3 ++-\n t/t5551-http-fetch-smart.sh | 11 ++++++++++-\n 2 files changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 3bd6883..789e081 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -127,7 +127,8 @@ static int module_name(int argc, const char **argv, const char *prefix)\n */\n static int submodule_config_ok(const char *var)\n {\n-\tif (starts_with(var, \"credential.\"))\n+\tif (starts_with(var, \"credential.\") ||\n+\t (starts_with(var, \"http.\") && ends_with(var, \".extraheader\")))\n \t\treturn 1;\n \treturn 0;\n }\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex e44fe72..1794168 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -286,7 +286,16 @@ test_expect_success 'custom http headers' '\n \ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n-\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit update-index --add --cacheinfo 160000,$(git rev-parse HEAD),sub &&\n+\tgit config -f .gitmodules submodule.sub.path sub &&\n+\tgit config -f .gitmodules submodule.sub.url \\\n+\t\t\"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit submodule init sub &&\n+\ttest_must_fail git submodule update sub &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t\tsubmodule update sub\n '\n \n stop_httpd\n-- \n2.8.1.306.gff998f2\n"},{"id":"285421","messageId":"20160504062618.GA9849@sigill.intra.peff.net","threadId":"42138","inReplyTo":"cover.1462342213.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v6 0/2] Add support for sending additional HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-04T06:26:19Z","receivedAt":"2016-05-04T06:26:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 04, 2016 at 08:14:07AM +0200, Johannes Schindelin wrote:\n\n> My use case is an army of build agents that need only limited and\n> selective access to otherwise private repositories.\n> \n> v6 supports submodules better by allowing\n> \n> \tgit -c http.extraheader submodule update\n> \n> to work as one would expect intuitively.\n> \n> Johannes Schindelin (2):\n> http: support sending custom HTTP headers\n\nI think this one is in \"next\", with \"will merge to master\" in the latest\nWhat's Cooking. So fortunately it does not have any changes in this\nversion of the series. :)\n\n> submodule: pass on http.extraheader config settings\n\nIMHO this should come on top of jk/submodule-config-sanitize-fix (I was\nsurprised at first that your test worked at all, but that is because it\nis using \"clone\", which is the one code path that works).\n\nBut I think we are waiting on going one of two paths:\n\n 1. drop sanitizing entirely\n\n 2. fix sanitizing and add more variables to it\n\nIf we go the route of (2), then we'd want my fix topic and this patch.\nAnd if not, then we don't need any of it (just a patch dropping the\nfiltering, which AFAIK nobody has written yet).\n\n-Peff\n"},{"id":"285427","messageId":"xmqq7ffajmvz.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160504062618.GA9849@sigill.intra.peff.net","subject":"Re: [PATCH v6 0/2] Add support for sending additional HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-04T07:36:00Z","receivedAt":"2016-05-04T07:36:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n>> submodule: pass on http.extraheader config settings\n>\n> IMHO this should come on top of jk/submodule-config-sanitize-fix (I was\n> surprised at first that your test worked at all, but that is because it\n> is using \"clone\", which is the one code path that works).\n\nYes.\n\n> But I think we are waiting on going one of two paths:\n>\n> 1. drop sanitizing entirely\n>\n> 2. fix sanitizing and add more variables to it\n>\n> If we go the route of (2), then we'd want my fix topic and this patch.\n> And if not, then we don't need any of it (just a patch dropping the\n> filtering, which AFAIK nobody has written yet).\n\nDoubly yes. That is why I didn't pick up 2/2 in the previous round\nand also jk/submodule-config-sanitize-fix is not in 'next' for the\nsame reason.\n\nI agree with you that we have not yet reached concensus on which one\nof the two we would want to take. I was sort of surprised to see\n2/2 sent again, after seeing that Dscho sounded strongly in favor of\nnot filtering the passed configuration variables, which would make\nthe patch unnecessary.\n"},{"id":"285430","messageId":"20160504074559.GA3077@sigill.intra.peff.net","threadId":"42138","inReplyTo":"20160504062618.GA9849@sigill.intra.peff.net","subject":"Re: [PATCH v6 0/2] Add support for sending additional HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-04T07:45:59Z","receivedAt":"2016-05-04T07:45:59Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 04, 2016 at 02:26:18AM -0400, Jeff King wrote:\n\n> > submodule: pass on http.extraheader config settings\n> \n> IMHO this should come on top of jk/submodule-config-sanitize-fix (I was\n> surprised at first that your test worked at all, but that is because it\n> is using \"clone\", which is the one code path that works).\n> \n> But I think we are waiting on going one of two paths:\n> \n> 1. drop sanitizing entirely\n> \n> 2. fix sanitizing and add more variables to it\n> \n> If we go the route of (2), then we'd want my fix topic and this patch.\n> And if not, then we don't need any of it (just a patch dropping the\n> filtering, which AFAIK nobody has written yet).\n\nActually, I think this last bit is not quite true. If we want to go back\nto \"nothing gets passed to submodules\", we can drop all of my patches,\nbut I don't think anybody wants to do that.\n\nBut if we want \"everything gets passed to submodules\", then we do need\nsomething like my patch series, because every use of local_repo_env\nneeds to be come \"local_repo_env excluding GIT_CONFIG_PARAMETERS\". I\ndon't think we want to simply drop that variable from local_repo_env\n(which would also mean that it would be propagated to a local\ngit-upload-pack, for example, along with any third-party scripts that\nuse rev-parse --local-env-vars).\n\nSo I think we'd actually want my series as a preliminary fix, followed\nby dropping the whitelist entirely on top of that, and then probably\nsimplifying the shell sanitize_submodule_env() on top of that (it would\nbe correct without the whitelist, but you can also trivially implement\nit without having to call submodule--helper at all).\n\n-Peff\n"},{"id":"285431","messageId":"20160504080047.GA2436@sigill.intra.peff.net","threadId":"42138","inReplyTo":"20160504074559.GA3077@sigill.intra.peff.net","subject":"[PATCH] submodule: stop sanitizing config options","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-04T08:00:47Z","receivedAt":"2016-05-04T08:00:47Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"[+cc Stefan and Jacob since this is really resuming that earlier thread]\n\nOn Wed, May 04, 2016 at 03:45:59AM -0400, Jeff King wrote:\n\n> On Wed, May 04, 2016 at 02:26:18AM -0400, Jeff King wrote:\n> \n> > > submodule: pass on http.extraheader config settings\n> > \n> > IMHO this should come on top of jk/submodule-config-sanitize-fix (I was\n> > surprised at first that your test worked at all, but that is because it\n> > is using \"clone\", which is the one code path that works).\n> > \n> > But I think we are waiting on going one of two paths:\n> > \n> > 1. drop sanitizing entirely\n> > \n> > 2. fix sanitizing and add more variables to it\n> > \n> > If we go the route of (2), then we'd want my fix topic and this patch.\n> > And if not, then we don't need any of it (just a patch dropping the\n> > filtering, which AFAIK nobody has written yet).\n> \n> Actually, I think this last bit is not quite true. If we want to go back\n> to \"nothing gets passed to submodules\", we can drop all of my patches,\n> but I don't think anybody wants to do that.\n> \n> But if we want \"everything gets passed to submodules\", then we do need\n> something like my patch series, because every use of local_repo_env\n> needs to be come \"local_repo_env excluding GIT_CONFIG_PARAMETERS\". I\n> don't think we want to simply drop that variable from local_repo_env\n> (which would also mean that it would be propagated to a local\n> git-upload-pack, for example, along with any third-party scripts that\n> use rev-parse --local-env-vars).\n> \n> So I think we'd actually want my series as a preliminary fix, followed\n> by dropping the whitelist entirely on top of that, and then probably\n> simplifying the shell sanitize_submodule_env() on top of that (it would\n> be correct without the whitelist, but you can also trivially implement\n> it without having to call submodule--helper at all).\n\nI think we'd actually do it all in one, and that patch looks something\nlike the one below (on top of jk/submodule-config-sanitize-fix).\n\nI don't feel that strongly about going either direction with this, but I\nfigure it doesn't hurt to make the patch so we know what the actual\noption looks like.\n\n-- >8 --\nSubject: [PATCH] submodule: stop sanitizing config options\n\nThe point of having a whitelist of command-line config\noptions to pass to submodules was two-fold:\n\n 1. It prevented obvious nonsense like using core.worktree\n for multiple repos.\n\n 2. It could prevent surprise when the user did not mean\n for the options to leak to the submodules (e.g.,\n http.sslverify=false).\n\nFor case 1, the answer is mostly \"if it hurts, don't do\nthat\". For case 2, we can note that any such example has a\nmatching inverted surprise (e.g., a user who meant\nhttp.sslverify=true to apply everywhere, but it didn't).\n\nSo this whitelist is probably not giving us any benefit, and\nis already creating a hassle as people propose things to put\non it. Let's just drop it entirely.\n\nNote that we still need to keep a special code path for\n\"prepare the submodule environment\", because we still have\nto take care to pass through $GIT_CONFIG_PARAMETERS (and\nblock the rest of the repo-specific environment variables).\n\nWe can do this easily from within the submodule shell\nscript, which lets us drop the submodule--helper option\nentirely (and it's OK to do so because as a \"--\" program, it\nis entirely a private implementation detail).\n\nSigned-off-by: Jeff King \n---\n builtin/submodule--helper.c | 17 -----------------\n git-submodule.sh | 4 ++--\n submodule.c | 40 +---------------------------------------\n t/t7412-submodule--helper.sh | 26 --------------------------\n 4 files changed, 3 insertions(+), 84 deletions(-)\n delete mode 100755 t/t7412-submodule--helper.sh\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex de3ad5b..48cfc48 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -246,22 +246,6 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \treturn 0;\n }\n \n-static int module_sanitize_config(int argc, const char **argv, const char *prefix)\n-{\n-\tstruct strbuf sanitized_config = STRBUF_INIT;\n-\n-\tif (argc > 1)\n-\t\tusage(_(\"git submodule--helper sanitize-config\"));\n-\n-\tgit_config_from_parameters(sanitize_submodule_config, &sanitized_config);\n-\tif (sanitized_config.len)\n-\t\tprintf(\"%s\\n\", sanitized_config.buf);\n-\n-\tstrbuf_release(&sanitized_config);\n-\n-\treturn 0;\n-}\n-\n struct submodule_update_clone {\n \t/* index into 'list', the list of submodules to look into for cloning */\n \tint current;\n@@ -522,7 +506,6 @@ static struct cmd_struct commands[] = {\n \t{\"list\", module_list},\n \t{\"name\", module_name},\n \t{\"clone\", module_clone},\n-\t{\"sanitize-config\", module_sanitize_config},\n \t{\"update-clone\", update_clone}\n };\n \ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3a40d4b..c9d53e1 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -197,9 +197,9 @@ isnumber()\n # of the settings from GIT_CONFIG_PARAMETERS.\n sanitize_submodule_env()\n {\n-\tsanitized_config=$(git submodule--helper sanitize-config)\n+\tsave_config=$GIT_CONFIG_PARAMETERS\n \tclear_local_git_env\n-\tGIT_CONFIG_PARAMETERS=$sanitized_config\n+\tGIT_CONFIG_PARAMETERS=$save_config\n \texport GIT_CONFIG_PARAMETERS\n }\n \ndiff --git a/submodule.c b/submodule.c\nindex 4e76b98..072ea82 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1131,50 +1131,12 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-/*\n- * Rules to sanitize configuration variables that are Ok to be passed into\n- * submodule operations from the parent project using \"-c\". Should only\n- * include keys which are both (a) safe and (b) necessary for proper\n- * operation.\n- */\n-static int submodule_config_ok(const char *var)\n-{\n-\tif (starts_with(var, \"credential.\"))\n-\t\treturn 1;\n-\treturn 0;\n-}\n-\n-int sanitize_submodule_config(const char *var, const char *value, void *data)\n-{\n-\tstruct strbuf *out = data;\n-\n-\tif (submodule_config_ok(var)) {\n-\t\tif (out->len)\n-\t\t\tstrbuf_addch(out, ' ');\n-\n-\t\tif (value)\n-\t\t\tsq_quotef(out, \"%s=%s\", var, value);\n-\t\telse\n-\t\t\tsq_quote_buf(out, var);\n-\t}\n-\n-\treturn 0;\n-}\n-\n void prepare_submodule_repo_env(struct argv_array *out)\n {\n \tconst char * const *var;\n \n \tfor (var = local_repo_env; *var; var++) {\n-\t\tif (!strcmp(*var, CONFIG_DATA_ENVIRONMENT)) {\n-\t\t\tstruct strbuf sanitized_config = STRBUF_INIT;\n-\t\t\tgit_config_from_parameters(sanitize_submodule_config,\n-\t\t\t\t\t\t &sanitized_config);\n-\t\t\targv_array_pushf(out, \"%s=%s\", *var, sanitized_config.buf);\n-\t\t\tstrbuf_release(&sanitized_config);\n-\t\t} else {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n \t\t\targv_array_push(out, *var);\n-\t\t}\n \t}\n-\n }\ndiff --git a/t/t7412-submodule--helper.sh b/t/t7412-submodule--helper.sh\ndeleted file mode 100755\nindex 149d428..0000000\n--- a/t/t7412-submodule--helper.sh\n+++ /dev/null\n@@ -1,26 +0,0 @@\n-#!/bin/sh\n-#\n-# Copyright (c) 2016 Jacob Keller\n-#\n-\n-test_description='Basic plumbing support of submodule--helper\n-\n-This test verifies the submodule--helper plumbing command used to implement\n-git-submodule.\n-'\n-\n-. ./test-lib.sh\n-\n-test_expect_success 'sanitize-config clears configuration' '\n-\tgit -c user.name=\"Some User\" submodule--helper sanitize-config >actual &&\n-\ttest_must_be_empty actual\n-'\n-\n-sq=\"'\"\n-test_expect_success 'sanitize-config keeps credential.helper' '\n-\tgit -c credential.helper=helper submodule--helper sanitize-config >actual &&\n-\techo \"${sq}credential.helper=helper${sq}\" >expect &&\n-\ttest_cmp expect actual\n-'\n-\n-test_done\n-- \n2.8.2.600.g439cdc9\n"},{"id":"285435","messageId":"xmqqwpnai6em.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160504080047.GA2436@sigill.intra.peff.net","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-04T08:17:21Z","receivedAt":"2016-05-04T08:17:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> [+cc Stefan and Jacob since this is really resuming that earlier thread]\n> ...\n>> \n>> So I think we'd actually want my series as a preliminary fix, followed\n>> by dropping the whitelist entirely on top of that, and then probably\n>> simplifying the shell sanitize_submodule_env() on top of that (it would\n>> be correct without the whitelist, but you can also trivially implement\n>> it without having to call submodule--helper at all).\n>\n> I think we'd actually do it all in one, and that patch looks something\n> like the one below (on top of jk/submodule-config-sanitize-fix).\n>\n> I don't feel that strongly about going either direction with this, but I\n> figure it doesn't hurt to make the patch so we know what the actual\n> option looks like.\n\nI do not feel strongly either, but I suspect \"we do not filter\"\nwould be a lot easier to explain than \"we pass these selected\nthings, each with such and such justification why it has to be\npassed down\".\n\nNice code reduction is very attractive, too, but that is secondary.\n\n> -- >8 --\n> Subject: [PATCH] submodule: stop sanitizing config options\n>\n> The point of having a whitelist of command-line config\n> options to pass to submodules was two-fold:\n>\n> 1. It prevented obvious nonsense like using core.worktree\n> for multiple repos.\n>\n> 2. It could prevent surprise when the user did not mean\n> for the options to leak to the submodules (e.g.,\n> http.sslverify=false).\n>\n> For case 1, the answer is mostly \"if it hurts, don't do\n> that\". For case 2, we can note that any such example has a\n> matching inverted surprise (e.g., a user who meant\n> http.sslverify=true to apply everywhere, but it didn't).\n>\n> So this whitelist is probably not giving us any benefit, and\n> is already creating a hassle as people propose things to put\n> on it. Let's just drop it entirely.\n>\n> Note that we still need to keep a special code path for\n> \"prepare the submodule environment\", because we still have\n> to take care to pass through $GIT_CONFIG_PARAMETERS (and\n> block the rest of the repo-specific environment variables).\n>\n> We can do this easily from within the submodule shell\n> script, which lets us drop the submodule--helper option\n> entirely (and it's OK to do so because as a \"--\" program, it\n> is entirely a private implementation detail).\n>\n> Signed-off-by: Jeff King \n> ---\n> builtin/submodule--helper.c | 17 -----------------\n> git-submodule.sh | 4 ++--\n> submodule.c | 40 +---------------------------------------\n> t/t7412-submodule--helper.sh | 26 --------------------------\n> 4 files changed, 3 insertions(+), 84 deletions(-)\n> delete mode 100755 t/t7412-submodule--helper.sh\n>\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index de3ad5b..48cfc48 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -246,22 +246,6 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n> \treturn 0;\n> }\n> \n> -static int module_sanitize_config(int argc, const char **argv, const char *prefix)\n> -{\n> -\tstruct strbuf sanitized_config = STRBUF_INIT;\n> -\n> -\tif (argc > 1)\n> -\t\tusage(_(\"git submodule--helper sanitize-config\"));\n> -\n> -\tgit_config_from_parameters(sanitize_submodule_config, &sanitized_config);\n> -\tif (sanitized_config.len)\n> -\t\tprintf(\"%s\\n\", sanitized_config.buf);\n> -\n> -\tstrbuf_release(&sanitized_config);\n> -\n> -\treturn 0;\n> -}\n> -\n> struct submodule_update_clone {\n> \t/* index into 'list', the list of submodules to look into for cloning */\n> \tint current;\n> @@ -522,7 +506,6 @@ static struct cmd_struct commands[] = {\n> \t{\"list\", module_list},\n> \t{\"name\", module_name},\n> \t{\"clone\", module_clone},\n> -\t{\"sanitize-config\", module_sanitize_config},\n> \t{\"update-clone\", update_clone}\n> };\n> \n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 3a40d4b..c9d53e1 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -197,9 +197,9 @@ isnumber()\n> # of the settings from GIT_CONFIG_PARAMETERS.\n> sanitize_submodule_env()\n> {\n> -\tsanitized_config=$(git submodule--helper sanitize-config)\n> +\tsave_config=$GIT_CONFIG_PARAMETERS\n> \tclear_local_git_env\n> -\tGIT_CONFIG_PARAMETERS=$sanitized_config\n> +\tGIT_CONFIG_PARAMETERS=$save_config\n> \texport GIT_CONFIG_PARAMETERS\n> }\n> \n> diff --git a/submodule.c b/submodule.c\n> index 4e76b98..072ea82 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1131,50 +1131,12 @@ int parallel_submodules(void)\n> \treturn parallel_jobs;\n> }\n> \n> -/*\n> - * Rules to sanitize configuration variables that are Ok to be passed into\n> - * submodule operations from the parent project using \"-c\". Should only\n> - * include keys which are both (a) safe and (b) necessary for proper\n> - * operation.\n> - */\n> -static int submodule_config_ok(const char *var)\n> -{\n> -\tif (starts_with(var, \"credential.\"))\n> -\t\treturn 1;\n> -\treturn 0;\n> -}\n> -\n> -int sanitize_submodule_config(const char *var, const char *value, void *data)\n> -{\n> -\tstruct strbuf *out = data;\n> -\n> -\tif (submodule_config_ok(var)) {\n> -\t\tif (out->len)\n> -\t\t\tstrbuf_addch(out, ' ');\n> -\n> -\t\tif (value)\n> -\t\t\tsq_quotef(out, \"%s=%s\", var, value);\n> -\t\telse\n> -\t\t\tsq_quote_buf(out, var);\n> -\t}\n> -\n> -\treturn 0;\n> -}\n> -\n> void prepare_submodule_repo_env(struct argv_array *out)\n> {\n> \tconst char * const *var;\n> \n> \tfor (var = local_repo_env; *var; var++) {\n> -\t\tif (!strcmp(*var, CONFIG_DATA_ENVIRONMENT)) {\n> -\t\t\tstruct strbuf sanitized_config = STRBUF_INIT;\n> -\t\t\tgit_config_from_parameters(sanitize_submodule_config,\n> -\t\t\t\t\t\t &sanitized_config);\n> -\t\t\targv_array_pushf(out, \"%s=%s\", *var, sanitized_config.buf);\n> -\t\t\tstrbuf_release(&sanitized_config);\n> -\t\t} else {\n> +\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n> \t\t\targv_array_push(out, *var);\n> -\t\t}\n> \t}\n> -\n> }\n> diff --git a/t/t7412-submodule--helper.sh b/t/t7412-submodule--helper.sh\n> deleted file mode 100755\n> index 149d428..0000000\n> --- a/t/t7412-submodule--helper.sh\n> +++ /dev/null\n> @@ -1,26 +0,0 @@\n> -#!/bin/sh\n> -#\n> -# Copyright (c) 2016 Jacob Keller\n> -#\n> -\n> -test_description='Basic plumbing support of submodule--helper\n> -\n> -This test verifies the submodule--helper plumbing command used to implement\n> -git-submodule.\n> -'\n> -\n> -. ./test-lib.sh\n> -\n> -test_expect_success 'sanitize-config clears configuration' '\n> -\tgit -c user.name=\"Some User\" submodule--helper sanitize-config >actual &&\n> -\ttest_must_be_empty actual\n> -'\n> -\n> -sq=\"'\"\n> -test_expect_success 'sanitize-config keeps credential.helper' '\n> -\tgit -c credential.helper=helper submodule--helper sanitize-config >actual &&\n> -\techo \"${sq}credential.helper=helper${sq}\" >expect &&\n> -\ttest_cmp expect actual\n> -'\n> -\n> -test_done\n"},{"id":"285456","messageId":"alpine.DEB.2.20.1605041316160.9313@virtualbox","threadId":"42138","inReplyTo":"xmqq7ffajmvz.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v6 0/2] Add support for sending additional HTTP headers","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-04T11:20:34Z","receivedAt":"2016-05-04T11:20:34Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 4 May 2016, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> >> submodule: pass on http.extraheader config settings\n> >\n> > IMHO this should come on top of jk/submodule-config-sanitize-fix (I was\n> > surprised at first that your test worked at all, but that is because it\n> > is using \"clone\", which is the one code path that works).\n> \n> Yes.\n\nOkay.\n\n> > But I think we are waiting on going one of two paths:\n> >\n> > 1. drop sanitizing entirely\n> >\n> > 2. fix sanitizing and add more variables to it\n> >\n> > If we go the route of (2), then we'd want my fix topic and this patch.\n> > And if not, then we don't need any of it (just a patch dropping the\n> > filtering, which AFAIK nobody has written yet).\n> \n> Doubly yes. That is why I didn't pick up 2/2 in the previous round\n> and also jk/submodule-config-sanitize-fix is not in 'next' for the\n> same reason.\n\nOkay. It was not clear to me that the indentation was not the reason it\nwas ignored.\n\n> I agree with you that we have not yet reached concensus on which one\n> of the two we would want to take. I was sort of surprised to see\n> 2/2 sent again, after seeing that Dscho sounded strongly in favor of\n> not filtering the passed configuration variables, which would make\n> the patch unnecessary.\n\nHah, my opinion matters after all.\n\nYes, I am in favor of not filtering any config settings passed through the\ncommand-line.\n\nBut traditionally, I did not always get what I strongly preferred, and in\nthis case it is really important to me that support for `git -c\nhttp.extraheader=... submodule ...` be part of an official Git version\nsoon.\n\nI just *really* would hate for, say, 2.9.0 to be released with `git -c\nhttp.extraheader=Hello:\\ Junio submodule update --init` not working as\nintended.\n\nCiao,\nDscho\n"},{"id":"285457","messageId":"alpine.DEB.2.20.1605041324030.9313@virtualbox","threadId":"42138","inReplyTo":"xmqqwpnai6em.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-04T11:25:20Z","receivedAt":"2016-05-04T11:25:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 4 May 2016, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > [+cc Stefan and Jacob since this is really resuming that earlier thread]\n> > ...\n> >> \n> >> So I think we'd actually want my series as a preliminary fix, followed\n> >> by dropping the whitelist entirely on top of that, and then probably\n> >> simplifying the shell sanitize_submodule_env() on top of that (it would\n> >> be correct without the whitelist, but you can also trivially implement\n> >> it without having to call submodule--helper at all).\n> >\n> > I think we'd actually do it all in one, and that patch looks something\n> > like the one below (on top of jk/submodule-config-sanitize-fix).\n> >\n> > I don't feel that strongly about going either direction with this, but I\n> > figure it doesn't hurt to make the patch so we know what the actual\n> > option looks like.\n> \n> I do not feel strongly either, but I suspect \"we do not filter\"\n> would be a lot easier to explain than \"we pass these selected\n> things, each with such and such justification why it has to be\n> passed down\".\n\nI really like the simplicity of the rationale.\n\n> Nice code reduction is very attractive, too, but that is secondary.\n\nMe, too. It just makes things simpler.\n\nCiao,\nDscho\n"},{"id":"285479","messageId":"CAGZ79kZUbhhjwV83_FQKfJR45K6QsZUN_-HJVJmefZUbJEDYYQ@mail.gmail.com","threadId":"42138","inReplyTo":"20160504080047.GA2436@sigill.intra.peff.net","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-05-04T17:58:26Z","receivedAt":"2016-05-04T17:58:26Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, May 4, 2016 at 1:00 AM, Jeff King wrote:\n> [+cc Stefan and Jacob since this is really resuming that earlier thread]\n>\n> On Wed, May 04, 2016 at 03:45:59AM -0400, Jeff King wrote:\n>\n>> On Wed, May 04, 2016 at 02:26:18AM -0400, Jeff King wrote:\n>>\n>> > > submodule: pass on http.extraheader config settings\n>> >\n>> > IMHO this should come on top of jk/submodule-config-sanitize-fix (I was\n>> > surprised at first that your test worked at all, but that is because it\n>> > is using \"clone\", which is the one code path that works).\n>> >\n>> > But I think we are waiting on going one of two paths:\n>> >\n>> > 1. drop sanitizing entirely\n>> >\n>> > 2. fix sanitizing and add more variables to it\n>> >\n>> > If we go the route of (2), then we'd want my fix topic and this patch.\n>> > And if not, then we don't need any of it (just a patch dropping the\n>> > filtering, which AFAIK nobody has written yet).\n>>\n>> Actually, I think this last bit is not quite true. If we want to go back\n>> to \"nothing gets passed to submodules\", we can drop all of my patches,\n>> but I don't think anybody wants to do that.\n>>\n>> But if we want \"everything gets passed to submodules\", then we do need\n>> something like my patch series, because every use of local_repo_env\n>> needs to be come \"local_repo_env excluding GIT_CONFIG_PARAMETERS\". I\n>> don't think we want to simply drop that variable from local_repo_env\n>> (which would also mean that it would be propagated to a local\n>> git-upload-pack, for example, along with any third-party scripts that\n>> use rev-parse --local-env-vars).\n>>\n>> So I think we'd actually want my series as a preliminary fix, followed\n>> by dropping the whitelist entirely on top of that, and then probably\n>> simplifying the shell sanitize_submodule_env() on top of that (it would\n>> be correct without the whitelist, but you can also trivially implement\n>> it without having to call submodule--helper at all).\n>\n> I think we'd actually do it all in one, and that patch looks something\n> like the one below (on top of jk/submodule-config-sanitize-fix).\n>\n> I don't feel that strongly about going either direction with this, but I\n> figure it doesn't hurt to make the patch so we know what the actual\n> option looks like.\n>\n> -- >8 --\n> Subject: [PATCH] submodule: stop sanitizing config options\n>\n> The point of having a whitelist of command-line config\n> options to pass to submodules was two-fold:\n>\n> 1. It prevented obvious nonsense like using core.worktree\n> for multiple repos.\n>\n> 2. It could prevent surprise when the user did not mean\n> for the options to leak to the submodules (e.g.,\n> http.sslverify=false).\n>\n> For case 1, the answer is mostly \"if it hurts, don't do\n> that\". For case 2, we can note that any such example has a\n> matching inverted surprise (e.g., a user who meant\n> http.sslverify=true to apply everywhere, but it didn't).\n>\n> So this whitelist is probably not giving us any benefit, and\n> is already creating a hassle as people propose things to put\n> on it. Let's just drop it entirely.\n\nJust to recap:\nBefore jk/submodule-config-sanitize-fix (jk/submodule-c-credential actually)\nwe passed nothing down to the commands operating on submodules.\n\nThen we decided to pass on some of it based on a curated list.\n\nCurating the list is too hard, so we pass on everything now, because\nit is easy to maintain and easy to explain. And when the user is hurt,\nthey're holding it wrong?\n\n>\n> Note that we still need to keep a special code path for\n> \"prepare the submodule environment\", because we still have\n> to take care to pass through $GIT_CONFIG_PARAMETERS (and\n> block the rest of the repo-specific environment variables).\n\nSo when running `git -c foo=bar command --recurse-submodules`\nthe `-c` parsing calls git_config_push_parameter, which\nexports that string `foo=baz` into the environment variable\nGIT_CONFIG_PARAMETERS.\n\nWhen the submodule command is called, sanitize_submodule_env\njust wipes all the Git related configurations except those in\nGIT_CONFIG_PARAMETERS as they are set again after\nclear_local_git_env wiped it.\n\nI wonder about the implementation detail, if we rather want to introduce\na `git rev-parse --repo-only-local-env-vars` which is\n`git rev-parse --local-env-vars` without the GIT_CONFIG_PARAMETERS.\nsuch that clear_local_git_env does the right thing and we don't\nhave to have 2 functions for it (i.e. clear_local_git_env and\nsanitize_submodule_env, which is the newer not as strict version of it)\n\nBut as Jeff once put it, rev-parse is already a messy kitchensink,\nso adding another option there may also not the right way?\n\n>\n> We can do this easily from within the submodule shell\n> script, which lets us drop the submodule--helper option\n> entirely (and it's OK to do so because as a \"--\" program, it\n> is entirely a private implementation detail).\n\nYeah that -- program may change any time in no backwards\ncompatible way.\n\nDo we want to add documentation for the new behavior though?\n Before: pass not -c arguments to submodules\n Now: Pass all the -c arguments to submodules\n\n>\n> Signed-off-by: Jeff King \n> ---\n> builtin/submodule--helper.c | 17 -----------------\n> git-submodule.sh | 4 ++--\n> submodule.c | 40 +---------------------------------------\n> t/t7412-submodule--helper.sh | 26 --------------------------\n> 4 files changed, 3 insertions(+), 84 deletions(-)\n> delete mode 100755 t/t7412-submodule--helper.sh\n>\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index de3ad5b..48cfc48 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -246,22 +246,6 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n> return 0;\n> }\n>\n> -static int module_sanitize_config(int argc, const char **argv, const char *prefix)\n> -{\n> - struct strbuf sanitized_config = STRBUF_INIT;\n> -\n> - if (argc > 1)\n> - usage(_(\"git submodule--helper sanitize-config\"));\n> -\n> - git_config_from_parameters(sanitize_submodule_config, &sanitized_config);\n> - if (sanitized_config.len)\n> - printf(\"%s\\n\", sanitized_config.buf);\n> -\n> - strbuf_release(&sanitized_config);\n> -\n> - return 0;\n> -}\n> -\n> struct submodule_update_clone {\n> /* index into 'list', the list of submodules to look into for cloning */\n> int current;\n> @@ -522,7 +506,6 @@ static struct cmd_struct commands[] = {\n> {\"list\", module_list},\n> {\"name\", module_name},\n> {\"clone\", module_clone},\n> - {\"sanitize-config\", module_sanitize_config},\n> {\"update-clone\", update_clone}\n> };\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 3a40d4b..c9d53e1 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -197,9 +197,9 @@ isnumber()\n> # of the settings from GIT_CONFIG_PARAMETERS.\n> sanitize_submodule_env()\n> {\n> - sanitized_config=$(git submodule--helper sanitize-config)\n> + save_config=$GIT_CONFIG_PARAMETERS\n> clear_local_git_env\n> - GIT_CONFIG_PARAMETERS=$sanitized_config\n> + GIT_CONFIG_PARAMETERS=$save_config\n> export GIT_CONFIG_PARAMETERS\n> }\n>\n> diff --git a/submodule.c b/submodule.c\n> index 4e76b98..072ea82 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1131,50 +1131,12 @@ int parallel_submodules(void)\n> return parallel_jobs;\n> }\n>\n> -/*\n> - * Rules to sanitize configuration variables that are Ok to be passed into\n> - * submodule operations from the parent project using \"-c\". Should only\n> - * include keys which are both (a) safe and (b) necessary for proper\n> - * operation.\n> - */\n> -static int submodule_config_ok(const char *var)\n> -{\n> - if (starts_with(var, \"credential.\"))\n> - return 1;\n> - return 0;\n> -}\n> -\n> -int sanitize_submodule_config(const char *var, const char *value, void *data)\n> -{\n> - struct strbuf *out = data;\n> -\n> - if (submodule_config_ok(var)) {\n> - if (out->len)\n> - strbuf_addch(out, ' ');\n> -\n> - if (value)\n> - sq_quotef(out, \"%s=%s\", var, value);\n> - else\n> - sq_quote_buf(out, var);\n> - }\n> -\n> - return 0;\n> -}\n> -\n> void prepare_submodule_repo_env(struct argv_array *out)\n> {\n> const char * const *var;\n>\n> for (var = local_repo_env; *var; var++) {\n> - if (!strcmp(*var, CONFIG_DATA_ENVIRONMENT)) {\n> - struct strbuf sanitized_config = STRBUF_INIT;\n> - git_config_from_parameters(sanitize_submodule_config,\n> - &sanitized_config);\n> - argv_array_pushf(out, \"%s=%s\", *var, sanitized_config.buf);\n> - strbuf_release(&sanitized_config);\n> - } else {\n> + if (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n> argv_array_push(out, *var);\n> - }\n> }\n> -\n> }\n> diff --git a/t/t7412-submodule--helper.sh b/t/t7412-submodule--helper.sh\n> deleted file mode 100755\n> index 149d428..0000000\n> --- a/t/t7412-submodule--helper.sh\n> +++ /dev/null\n> @@ -1,26 +0,0 @@\n> -#!/bin/sh\n> -#\n> -# Copyright (c) 2016 Jacob Keller\n> -#\n> -\n> -test_description='Basic plumbing support of submodule--helper\n> -\n> -This test verifies the submodule--helper plumbing command used to implement\n> -git-submodule.\n> -'\n> -\n> -. ./test-lib.sh\n> -\n> -test_expect_success 'sanitize-config clears configuration' '\n> - git -c user.name=\"Some User\" submodule--helper sanitize-config >actual &&\n> - test_must_be_empty actual\n> -'\n> -\n> -sq=\"'\"\n> -test_expect_success 'sanitize-config keeps credential.helper' '\n> - git -c credential.helper=helper submodule--helper sanitize-config >actual &&\n> - echo \"${sq}credential.helper=helper${sq}\" >expect &&\n> - test_cmp expect actual\n> -'\n> -\n> -test_done\n> --\n> 2.8.2.600.g439cdc9\n>\n"},{"id":"285482","messageId":"xmqq4madisw2.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1605041316160.9313@virtualbox","subject":"Re: [PATCH v6 0/2] Add support for sending additional HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-04T18:23:57Z","receivedAt":"2016-05-04T18:23:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Hi Junio,\n>\n> On Wed, 4 May 2016, Junio C Hamano wrote:\n>\n>> Jeff King writes:\n>> \n>> >> submodule: pass on http.extraheader config settings\n>> >\n>> > IMHO this should come on top of jk/submodule-config-sanitize-fix (I was\n>> > surprised at first that your test worked at all, but that is because it\n>> > is using \"clone\", which is the one code path that works).\n>> \n>> Yes.\n>\n> Okay.\n>\n>> > But I think we are waiting on going one of two paths:\n>> >\n>> > 1. drop sanitizing entirely\n>> >\n>> > 2. fix sanitizing and add more variables to it\n>> >\n>> > If we go the route of (2), then we'd want my fix topic and this patch.\n>> > And if not, then we don't need any of it (just a patch dropping the\n>> > filtering, which AFAIK nobody has written yet).\n>> \n>> Doubly yes. That is why I didn't pick up 2/2 in the previous round\n>> and also jk/submodule-config-sanitize-fix is not in 'next' for the\n>> same reason.\n>\n> Okay. It was not clear to me that the indentation was not the reason it\n> was ignored.\n\nIt wasn't even ignored. I looked at it carefully, noticed that it\ncontradicts with what you said in a different message, and made a\nconcious decision to wait.\n\n>> I agree with you that we have not yet reached concensus on which one\n>> of the two we would want to take. I was sort of surprised to see\n>> 2/2 sent again, after seeing that Dscho sounded strongly in favor of\n>> not filtering the passed configuration variables, which would make\n>> the patch unnecessary.\n>\n> Hah, my opinion matters after all.\n\nWhen your proposal ends up getting rejected, it is not because your\nopinion does not matter. In any case, for this one, the reason I\ndecided to wait until the \"filter or not filter, and if filter, use\nwhitelist or blacklist\" discussion settles does not depend whether I\nhappen to agreed with your preference (which is \"not to filter\").\n\nThat is, there is a difference between \"I will not apply this ever,\nbecause I know the outcome of the the other discussion already and\nit will make this patch unnecessary\", and \"I cannot decide to apply\nthis yet, because this may be needed if the other discussion goes in\na certain way but this may turn out to be unnecessary if it goes in\nanother way.\" And the topic branch having only the first one is\nbecause this case was (and I think still is) the latter.\n"},{"id":"285487","messageId":"xmqqzis5hdek.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160504080047.GA2436@sigill.intra.peff.net","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-04T18:43:47Z","receivedAt":"2016-05-04T18:43:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 3a40d4b..c9d53e1 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -197,9 +197,9 @@ isnumber()\n> # of the settings from GIT_CONFIG_PARAMETERS.\n> sanitize_submodule_env()\n> {\n> -\tsanitized_config=$(git submodule--helper sanitize-config)\n> +\tsave_config=$GIT_CONFIG_PARAMETERS\n> \tclear_local_git_env\n> -\tGIT_CONFIG_PARAMETERS=$sanitized_config\n> +\tGIT_CONFIG_PARAMETERS=$save_config\n> \texport GIT_CONFIG_PARAMETERS\n> }\n\nThis does \"clear the obviously per-repository stuff, but add back in\nanything that came from -c\". If it is easy to do \"add anything that\ncame from -c, and then clear the obviously per-repository stuff\", we\ndon't even have to say \"exporting core.worktree down may hurt; do\nnot do it then\", which may be the best of both worlds?\n\nOr have we decided that even sharing core.worktree may have a valid\nuse case and it is better not to filter them?\n"},{"id":"285489","messageId":"20160504190423.GB21259@sigill.intra.peff.net","threadId":"42138","inReplyTo":"CAGZ79kZUbhhjwV83_FQKfJR45K6QsZUN_-HJVJmefZUbJEDYYQ@mail.gmail.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-04T19:04:24Z","receivedAt":"2016-05-04T19:04:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 04, 2016 at 10:58:26AM -0700, Stefan Beller wrote:\n\n> > So this whitelist is probably not giving us any benefit, and\n> > is already creating a hassle as people propose things to put\n> > on it. Let's just drop it entirely.\n> \n> Just to recap:\n> Before jk/submodule-config-sanitize-fix (jk/submodule-c-credential actually)\n> we passed nothing down to the commands operating on submodules.\n> \n> Then we decided to pass on some of it based on a curated list.\n> \n> Curating the list is too hard, so we pass on everything now, because\n> it is easy to maintain and easy to explain. And when the user is hurt,\n> they're holding it wrong?\n\nYes, I think that sums it up (the last paragraph is what's under\ndiscussion, so it's up for debate).\n\n> > Note that we still need to keep a special code path for\n> > \"prepare the submodule environment\", because we still have\n> > to take care to pass through $GIT_CONFIG_PARAMETERS (and\n> > block the rest of the repo-specific environment variables).\n> \n> So when running `git -c foo=bar command --recurse-submodules`\n> the `-c` parsing calls git_config_push_parameter, which\n> exports that string `foo=baz` into the environment variable\n> GIT_CONFIG_PARAMETERS.\n> \n> When the submodule command is called, sanitize_submodule_env\n> just wipes all the Git related configurations except those in\n> GIT_CONFIG_PARAMETERS as they are set again after\n> clear_local_git_env wiped it.\n\nRight.\n\n> I wonder about the implementation detail, if we rather want to introduce\n> a `git rev-parse --repo-only-local-env-vars` which is\n> `git rev-parse --local-env-vars` without the GIT_CONFIG_PARAMETERS.\n> such that clear_local_git_env does the right thing and we don't\n> have to have 2 functions for it (i.e. clear_local_git_env and\n> sanitize_submodule_env, which is the newer not as strict version of it)\n\nI don't think that really buys you much. The policy is technically\nduplicated in prepare_submodule_repo_env() in submodule.c and\nsanitize_submodule_env in submodule.h. But without the whitelist, it's\nsuch a simple policy that I'm not sure it's worth the boilerplate of\nhaving the shell function call into the C code.\n\nIf we did want to go that route, the more appropriate interface would\nprobably be:\n\n git submodule--helper sanitize-env\n\nor something. That avoids making it a public interface, and makes it\nclear that we are invoking the submodule rules.\n\n> Do we want to add documentation for the new behavior though?\n> Before: pass not -c arguments to submodules\n> Now: Pass all the -c arguments to submodules\n\nI don't think there was any documentation for the _old_ behavior, and\ncertainly jk/submodule-c-credential didn't add any. But it probably is\nworth document, maybe as part of \"-c\"? Care to roll a patch on top?\n\n-Peff\n"},{"id":"285492","messageId":"20160504190935.GC21259@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqqzis5hdek.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-04T19:09:35Z","receivedAt":"2016-05-04T19:09:35Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 04, 2016 at 11:43:47AM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > diff --git a/git-submodule.sh b/git-submodule.sh\n> > index 3a40d4b..c9d53e1 100755\n> > --- a/git-submodule.sh\n> > +++ b/git-submodule.sh\n> > @@ -197,9 +197,9 @@ isnumber()\n> > # of the settings from GIT_CONFIG_PARAMETERS.\n> > sanitize_submodule_env()\n> > {\n> > -\tsanitized_config=$(git submodule--helper sanitize-config)\n> > +\tsave_config=$GIT_CONFIG_PARAMETERS\n> > \tclear_local_git_env\n> > -\tGIT_CONFIG_PARAMETERS=$sanitized_config\n> > +\tGIT_CONFIG_PARAMETERS=$save_config\n> > \texport GIT_CONFIG_PARAMETERS\n> > }\n> \n> This does \"clear the obviously per-repository stuff, but add back in\n> anything that came from -c\". If it is easy to do \"add anything that\n> came from -c, and then clear the obviously per-repository stuff\", we\n> don't even have to say \"exporting core.worktree down may hurt; do\n> not do it then\", which may be the best of both worlds?\n\nIt's not easy at this level. clear_local_git_env() is just clearing\nwhole variables like $GIT_WORK_TREE. But if you say \"-c core.worktree\",\nthat is still live in $GIT_CONFIG_PARAMETERS, and gets fed to\nsub-processes on each invocation. Nobody ever removes individual content\nfrom $GIT_CONFIG_PARAMETERS once something is in it.\n\n> Or have we decided that even sharing core.worktree may have a valid\n> use case and it is better not to filter them?\n\nI do not personally consider it a valid use case. The sentiment just\nseemed to be that it was better to make the rule simple and easy to\nexplain (\"everything gets passed through; if it hurts, don't do it\")\nthan to keep sanitizing.\n\nIf you want \"pass through everything except core.worktree\", then that is\nthe blacklist approach (which is easy to do if we scrap this patch and\njust teak submodule_config_ok() from a whitelist into a blacklist).\n\n-Peff\n"},{"id":"285535","messageId":"CAGZ79kaUiVLuXvpLPKuZZ55zbQXA3Wt7WP3a_65gBW2Cj-gMoQ@mail.gmail.com","threadId":"42138","inReplyTo":"20160504080047.GA2436@sigill.intra.peff.net","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-05-04T22:53:26Z","receivedAt":"2016-05-04T22:53:26Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"> I don't think there was any documentation for the _old_ behavior, and\n> certainly jk/submodule-c-credential didn't add any. But it probably is\n> worth document, maybe as part of \"-c\"? Care to roll a patch on top?\n\nSure.\n\n>\n> I think we'd actually do it all in one, and that patch looks something\n> like the one below (on top of jk/submodule-config-sanitize-fix).\n\n $ git checkout origin/jk/submodule-config-sanitize-fix\n $ git am p\nApplying: submodule: stop sanitizing config options\nerror: patch failed: builtin/submodule--helper.c:246\nerror: builtin/submodule--helper.c: patch does not apply\nerror: patch failed: submodule.c:1131\nerror: submodule.c: patch does not apply\nPatch failed at 0001 submodule: stop sanitizing config options\n\nSo if you want some documentation on top of that, where would I base it on?\n"},{"id":"285552","messageId":"20160505012219.GA15090@sigill.intra.peff.net","threadId":"42138","inReplyTo":"CAGZ79kaUiVLuXvpLPKuZZ55zbQXA3Wt7WP3a_65gBW2Cj-gMoQ@mail.gmail.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-05T01:22:19Z","receivedAt":"2016-05-05T01:22:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 04, 2016 at 03:53:26PM -0700, Stefan Beller wrote:\n\n> > I think we'd actually do it all in one, and that patch looks something\n> > like the one below (on top of jk/submodule-config-sanitize-fix).\n> \n> $ git checkout origin/jk/submodule-config-sanitize-fix\n> $ git am p\n> Applying: submodule: stop sanitizing config options\n> error: patch failed: builtin/submodule--helper.c:246\n> error: builtin/submodule--helper.c: patch does not apply\n> error: patch failed: submodule.c:1131\n> error: submodule.c: patch does not apply\n> Patch failed at 0001 submodule: stop sanitizing config options\n> \n> So if you want some documentation on top of that, where would I base it on?\n\nI build the patches for jk/submodule-config-sanitize-fix on top of\nmaster as of the other day, and then built this most recent patch on top\nof that.\n\nLooks like Junio applied them directly on the tip of\njk/submodule-c-credential, and had to wiggle the code in submodule.c,\nwhich conflicted with the parallel-process stuff that was merged in\nbetween. Since the new patch updates that code, it will likewise run\ninto conflicts.\n\nI don't think there's a strict right answer here; if the original buggy\nsubmodule-c-credential code had been released, we would definitely want\nto build off of it for \"maint\" releases. But it wasn't, so master is\n\"just as good\" in a sense. But I think Junio makes it a habit to apply\nfixes as far back as the introduced bug, even when it's not going to\nmaint.\n\nSo since that's what published, it makes sense to build on that. Here's\na version of my patch that should apply for you (no semantic changes,\njust differences in the surrounding context):\n\n-- >8 --\nSubject: [PATCH] submodule: stop sanitizing config options\n\nThe point of having a whitelist of command-line config\noptions to pass to submodules was two-fold:\n\n 1. It prevented obvious nonsense like using core.worktree\n for multiple repos.\n\n 2. It could prevent surprise when the user did not mean\n for the options to leak to the submodules (e.g.,\n http.sslverify=false).\n\nFor case 1, the answer is mostly \"if it hurts, don't do\nthat\". For case 2, we can note that any such example has a\nmatching inverted surprise (e.g., a user who meant\nhttp.sslverify=true to apply everywhere, but it didn't).\n\nSo this whitelist is probably not giving us any benefit, and\nis already creating a hassle as people propose things to put\non it. Let's just drop it entirely.\n\nNote that we still need to keep a special code path for\n\"prepare the submodule environment\", because we still have\nto take care to pass through $GIT_CONFIG_PARAMETERS (and\nblock the rest of the repo-specific environment variables).\n\nWe can do this easily from within the submodule shell\nscript, which lets us drop the submodule--helper option\nentirely (and it's OK to do so because as a \"--\" program, it\nis entirely a private implementation detail).\n\nSigned-off-by: Jeff King \n---\n builtin/submodule--helper.c | 17 -----------------\n git-submodule.sh | 4 ++--\n submodule.c | 39 +--------------------------------------\n t/t7412-submodule--helper.sh | 26 --------------------------\n 4 files changed, 3 insertions(+), 83 deletions(-)\n delete mode 100755 t/t7412-submodule--helper.sh\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 16d6432..89250f0 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -260,22 +260,6 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \treturn 0;\n }\n \n-static int module_sanitize_config(int argc, const char **argv, const char *prefix)\n-{\n-\tstruct strbuf sanitized_config = STRBUF_INIT;\n-\n-\tif (argc > 1)\n-\t\tusage(_(\"git submodule--helper sanitize-config\"));\n-\n-\tgit_config_from_parameters(sanitize_submodule_config, &sanitized_config);\n-\tif (sanitized_config.len)\n-\t\tprintf(\"%s\\n\", sanitized_config.buf);\n-\n-\tstrbuf_release(&sanitized_config);\n-\n-\treturn 0;\n-}\n-\n struct cmd_struct {\n \tconst char *cmd;\n \tint (*fn)(int, const char **, const char *);\n@@ -285,7 +269,6 @@ static struct cmd_struct commands[] = {\n \t{\"list\", module_list},\n \t{\"name\", module_name},\n \t{\"clone\", module_clone},\n-\t{\"sanitize-config\", module_sanitize_config},\n };\n \n int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 91f5856..b1c056c 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -197,9 +197,9 @@ isnumber()\n # of the settings from GIT_CONFIG_PARAMETERS.\n sanitize_submodule_env()\n {\n-\tsanitized_config=$(git submodule--helper sanitize-config)\n+\tsave_config=$GIT_CONFIG_PARAMETERS\n \tclear_local_git_env\n-\tGIT_CONFIG_PARAMETERS=$sanitized_config\n+\tGIT_CONFIG_PARAMETERS=$save_config\n \texport GIT_CONFIG_PARAMETERS\n }\n \ndiff --git a/submodule.c b/submodule.c\nindex c18ab9b..d598881 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1098,50 +1098,13 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n \tstrbuf_release(&rel_path);\n \tfree((void *)real_work_tree);\n }\n-/*\n- * Rules to sanitize configuration variables that are Ok to be passed into\n- * submodule operations from the parent project using \"-c\". Should only\n- * include keys which are both (a) safe and (b) necessary for proper\n- * operation.\n- */\n-static int submodule_config_ok(const char *var)\n-{\n-\tif (starts_with(var, \"credential.\"))\n-\t\treturn 1;\n-\treturn 0;\n-}\n-\n-int sanitize_submodule_config(const char *var, const char *value, void *data)\n-{\n-\tstruct strbuf *out = data;\n-\n-\tif (submodule_config_ok(var)) {\n-\t\tif (out->len)\n-\t\t\tstrbuf_addch(out, ' ');\n-\n-\t\tif (value)\n-\t\t\tsq_quotef(out, \"%s=%s\", var, value);\n-\t\telse\n-\t\t\tsq_quote_buf(out, var);\n-\t}\n-\n-\treturn 0;\n-}\n \n void prepare_submodule_repo_env(struct argv_array *out)\n {\n \tconst char * const *var;\n \n \tfor (var = local_repo_env; *var; var++) {\n-\t\tif (!strcmp(*var, CONFIG_DATA_ENVIRONMENT)) {\n-\t\t\tstruct strbuf sanitized_config = STRBUF_INIT;\n-\t\t\tgit_config_from_parameters(sanitize_submodule_config,\n-\t\t\t\t\t\t &sanitized_config);\n-\t\t\targv_array_pushf(out, \"%s=%s\", *var, sanitized_config.buf);\n-\t\t\tstrbuf_release(&sanitized_config);\n-\t\t} else {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n \t\t\targv_array_push(out, *var);\n-\t\t}\n \t}\n-\n }\ndiff --git a/t/t7412-submodule--helper.sh b/t/t7412-submodule--helper.sh\ndeleted file mode 100755\nindex 149d428..0000000\n--- a/t/t7412-submodule--helper.sh\n+++ /dev/null\n@@ -1,26 +0,0 @@\n-#!/bin/sh\n-#\n-# Copyright (c) 2016 Jacob Keller\n-#\n-\n-test_description='Basic plumbing support of submodule--helper\n-\n-This test verifies the submodule--helper plumbing command used to implement\n-git-submodule.\n-'\n-\n-. ./test-lib.sh\n-\n-test_expect_success 'sanitize-config clears configuration' '\n-\tgit -c user.name=\"Some User\" submodule--helper sanitize-config >actual &&\n-\ttest_must_be_empty actual\n-'\n-\n-sq=\"'\"\n-test_expect_success 'sanitize-config keeps credential.helper' '\n-\tgit -c credential.helper=helper submodule--helper sanitize-config >actual &&\n-\techo \"${sq}credential.helper=helper${sq}\" >expect &&\n-\ttest_cmp expect actual\n-'\n-\n-test_done\n-- \n2.8.2.600.g439cdc9\n"},{"id":"285574","messageId":"xmqq60uscufw.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160505012219.GA15090@sigill.intra.peff.net","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-05T16:59:15Z","receivedAt":"2016-05-05T16:59:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Here's a version of my patch that should apply for you (no\n> semantic changes, just differences in the surrounding context):\n\nSorry for the trouble.\n\nI queued jk/submodule-c-credential to be mergeable to 'maint', as it\ncould be argued two ways. We can say that not propagating -c down\nis a bug and the series is a bugfix. Or it is merely a known\nlimitation and the series is a new feature. I was leaning towards\nthe former, but I was also willing to declare that is a known bug\nthat will left unfixed in the maintenance track.\n\nIf I knew the 5 patches on jk/submodule-config-sanitize-fix was what\nwe would eventually agree to be the right change, I would have\napplied them directly on top of jk/submodule-c-credential instead of\nforking a separate branch for them. That way, either the \"bugfix\"\nwould all go to 'maint', or the \"feature\" wouldn't, and we do not\nhave to worry about making a mistake to merge only half-done state\n(i.e. jk/submodule-c-credential that passes only the credential.*\nconfiguration) that nobody would want to 'maint'.\n\nBut when I picked up jk/submodule-config-sanitize-fix, I didn't have\nenough time to think things through or carefully read the discussion\nto convince myself that we already had a list concensus, so I queued\nit separately only to make sure I won't lose track of it--I can come\nback to it later that way.\n\nIt probably is a good time to merge jk/submodule-config-sanitize-fix\ninto jk/submodule-c-credential (i.e. a mere fast-forward), remove\nthat \"-fix\" branch, and apply this patch directly on top of the\nresulting jk/submodule-c-credential. That will make the whole thing\na 13-patch series, consisting of:\n\n 7 patches up to the current jk/submodule-c-credential\n d1f8849 git_config_push_parameter: handle empty GIT_CONFIG_PARAMETERS\n 14111fc git: submodule honor -c credential.* from command line\n e70986d quote: implement sq_quotef()\n 7dad263 submodule: fix segmentation fault in submodule--helper clone\n 717416c submodule: fix submodule--helper clone usage\n 08e0970 submodule: check argc count for git submodule--helper clone\n d10e3b4 submodule: don't pass empty string arguments to submodule--helper clone\n \n 5 patches up to the current jk/submodule-config-sanitize-fix\n c12e865 submodule: use prepare_submodule_repo_env consistently\n 4638728 submodule--helper: move config-sanitizing to submodule.c\n 860cba6 submodule: export sanitized GIT_CONFIG_PARAMETERS\n 455d22c t5550: break submodule config test into multiple sub-tests\n 1149ee2 t5550: fix typo in $HTTPD_URL\n\n 1 patch (this one)\n 4e6706a submodule: stop sanitizing config options\n\nwhose early 7 patches have already been merged to 'master' (and none\nhas been merged to 'maint').\n\n> -- >8 --\n> Subject: [PATCH] submodule: stop sanitizing config options\n>\n> The point of having a whitelist of command-line config\n> options to pass to submodules was two-fold:\n>\n> 1. It prevented obvious nonsense like using core.worktree\n> for multiple repos.\n>\n> 2. It could prevent surprise when the user did not mean\n> for the options to leak to the submodules (e.g.,\n> http.sslverify=false).\n>\n> For case 1, the answer is mostly \"if it hurts, don't do\n> that\". For case 2, we can note that any such example has a\n> matching inverted surprise (e.g., a user who meant\n> http.sslverify=true to apply everywhere, but it didn't).\n>\n> So this whitelist is probably not giving us any benefit, and\n> is already creating a hassle as people propose things to put\n> on it. Let's just drop it entirely.\n>\n> Note that we still need to keep a special code path for\n> \"prepare the submodule environment\", because we still have\n> to take care to pass through $GIT_CONFIG_PARAMETERS (and\n> block the rest of the repo-specific environment variables).\n>\n> We can do this easily from within the submodule shell\n> script, which lets us drop the submodule--helper option\n> entirely (and it's OK to do so because as a \"--\" program, it\n> is entirely a private implementation detail).\n>\n> Signed-off-by: Jeff King \n> ---\n> builtin/submodule--helper.c | 17 -----------------\n> git-submodule.sh | 4 ++--\n> submodule.c | 39 +--------------------------------------\n> t/t7412-submodule--helper.sh | 26 --------------------------\n> 4 files changed, 3 insertions(+), 83 deletions(-)\n> delete mode 100755 t/t7412-submodule--helper.sh\n>\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index 16d6432..89250f0 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -260,22 +260,6 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n> \treturn 0;\n> }\n> \n> -static int module_sanitize_config(int argc, const char **argv, const char *prefix)\n> -{\n> -\tstruct strbuf sanitized_config = STRBUF_INIT;\n> -\n> -\tif (argc > 1)\n> -\t\tusage(_(\"git submodule--helper sanitize-config\"));\n> -\n> -\tgit_config_from_parameters(sanitize_submodule_config, &sanitized_config);\n> -\tif (sanitized_config.len)\n> -\t\tprintf(\"%s\\n\", sanitized_config.buf);\n> -\n> -\tstrbuf_release(&sanitized_config);\n> -\n> -\treturn 0;\n> -}\n> -\n> struct cmd_struct {\n> \tconst char *cmd;\n> \tint (*fn)(int, const char **, const char *);\n> @@ -285,7 +269,6 @@ static struct cmd_struct commands[] = {\n> \t{\"list\", module_list},\n> \t{\"name\", module_name},\n> \t{\"clone\", module_clone},\n> -\t{\"sanitize-config\", module_sanitize_config},\n> };\n> \n> int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 91f5856..b1c056c 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -197,9 +197,9 @@ isnumber()\n> # of the settings from GIT_CONFIG_PARAMETERS.\n> sanitize_submodule_env()\n> {\n> -\tsanitized_config=$(git submodule--helper sanitize-config)\n> +\tsave_config=$GIT_CONFIG_PARAMETERS\n> \tclear_local_git_env\n> -\tGIT_CONFIG_PARAMETERS=$sanitized_config\n> +\tGIT_CONFIG_PARAMETERS=$save_config\n> \texport GIT_CONFIG_PARAMETERS\n> }\n> \n> diff --git a/submodule.c b/submodule.c\n> index c18ab9b..d598881 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1098,50 +1098,13 @@ void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)\n> \tstrbuf_release(&rel_path);\n> \tfree((void *)real_work_tree);\n> }\n> -/*\n> - * Rules to sanitize configuration variables that are Ok to be passed into\n> - * submodule operations from the parent project using \"-c\". Should only\n> - * include keys which are both (a) safe and (b) necessary for proper\n> - * operation.\n> - */\n> -static int submodule_config_ok(const char *var)\n> -{\n> -\tif (starts_with(var, \"credential.\"))\n> -\t\treturn 1;\n> -\treturn 0;\n> -}\n> -\n> -int sanitize_submodule_config(const char *var, const char *value, void *data)\n> -{\n> -\tstruct strbuf *out = data;\n> -\n> -\tif (submodule_config_ok(var)) {\n> -\t\tif (out->len)\n> -\t\t\tstrbuf_addch(out, ' ');\n> -\n> -\t\tif (value)\n> -\t\t\tsq_quotef(out, \"%s=%s\", var, value);\n> -\t\telse\n> -\t\t\tsq_quote_buf(out, var);\n> -\t}\n> -\n> -\treturn 0;\n> -}\n> \n> void prepare_submodule_repo_env(struct argv_array *out)\n> {\n> \tconst char * const *var;\n> \n> \tfor (var = local_repo_env; *var; var++) {\n> -\t\tif (!strcmp(*var, CONFIG_DATA_ENVIRONMENT)) {\n> -\t\t\tstruct strbuf sanitized_config = STRBUF_INIT;\n> -\t\t\tgit_config_from_parameters(sanitize_submodule_config,\n> -\t\t\t\t\t\t &sanitized_config);\n> -\t\t\targv_array_pushf(out, \"%s=%s\", *var, sanitized_config.buf);\n> -\t\t\tstrbuf_release(&sanitized_config);\n> -\t\t} else {\n> +\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n> \t\t\targv_array_push(out, *var);\n> -\t\t}\n> \t}\n> -\n> }\n> diff --git a/t/t7412-submodule--helper.sh b/t/t7412-submodule--helper.sh\n> deleted file mode 100755\n> index 149d428..0000000\n> --- a/t/t7412-submodule--helper.sh\n> +++ /dev/null\n> @@ -1,26 +0,0 @@\n> -#!/bin/sh\n> -#\n> -# Copyright (c) 2016 Jacob Keller\n> -#\n> -\n> -test_description='Basic plumbing support of submodule--helper\n> -\n> -This test verifies the submodule--helper plumbing command used to implement\n> -git-submodule.\n> -'\n> -\n> -. ./test-lib.sh\n> -\n> -test_expect_success 'sanitize-config clears configuration' '\n> -\tgit -c user.name=\"Some User\" submodule--helper sanitize-config >actual &&\n> -\ttest_must_be_empty actual\n> -'\n> -\n> -sq=\"'\"\n> -test_expect_success 'sanitize-config keeps credential.helper' '\n> -\tgit -c credential.helper=helper submodule--helper sanitize-config >actual &&\n> -\techo \"${sq}credential.helper=helper${sq}\" >expect &&\n> -\ttest_cmp expect actual\n> -'\n> -\n> -test_done\n"},{"id":"285584","messageId":"67F2E414-657C-4C6C-9746-9B3026737C66@gmail.com","threadId":"42138","inReplyTo":"ff7edf51fe8baf57f6f4cab170602154e5b1e73e.1462342213.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v6 1/2] http: support sending custom HTTP headers","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-05-05T19:10:21Z","receivedAt":"2016-05-05T19:10:21Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 04 May 2016, at 08:14, Johannes Schindelin wrote:\n\n> We introduce a way to send custom HTTP headers with all requests.\n> \n> This allows us, for example, to send an extra token from build agents\n> for temporary access to private repositories. (This is the use case that\n> triggered this patch.)\n> \n> This feature can be used like this:\n> \n> \tgit -c http.extraheader='Secret: sssh!' fetch $URL $REF\n> \n> Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only\n> a single list, overriding any previous call. This means we have to\n> collect _all_ of the headers we want to use into a single list, and\n> feed it to cURL in one shot. Since we already unconditionally set a\n> \"pragma\" header when initializing the curl handles, we can add our new\n> headers to that list.\n> \n> For callers which override the default header list (like probe_rpc),\n> we provide `http_copy_default_headers()` so they can do the same\n> trick.\n> \n> Big thanks to Jeff King and Junio Hamano for their outstanding help and\n> patient reviews.\n> \n> Signed-off-by: Johannes Schindelin \n> ---\n> Documentation/config.txt | 6 ++++++\n> http-push.c | 10 +++++-----\n> http.c | 35 ++++++++++++++++++++++++++++++++---\n> http.h | 1 +\n> remote-curl.c | 4 ++--\n> t/lib-httpd/apache.conf | 8 ++++++++\n> t/t5551-http-fetch-smart.sh | 7 +++++++\n> 7 files changed, 61 insertions(+), 10 deletions(-)\n> \n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 42d2b50..c7bbe98 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -1655,6 +1655,12 @@ http.emptyAuth::\n> \ta username in the URL, as libcurl normally requires a username for\n> \tauthentication.\n> \n> +http.extraHeader::\n> +\tPass an additional HTTP header when communicating with a server. If\n> +\tmore than one such entry exists, all of them are added as extra\n> +\theaders. To allow overriding the settings inherited from the system\n> +\tconfig, an empty value will reset the extra headers to the empty list.\n> +\n> http.cookieFile::\n> \tFile containing previously stored cookie lines which should be used\n> \tin the Git http session, if they match the server. The file format\n> diff --git a/http-push.c b/http-push.c\n> index bd60668..ae2b7f1 100644\n> --- a/http-push.c\n> +++ b/http-push.c\n> @@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,\n> static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)\n> {\n> \tstruct strbuf buf = STRBUF_INIT;\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n> \n> \tif (options & DAV_HEADER_IF) {\n> \t\tstrbuf_addf(&buf, \"If: (<%s>)\", lock->token);\n> @@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)\n> static void start_move(struct transfer_request *request)\n> {\n> \tstruct active_request_slot *slot;\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n> \n> \tslot = get_active_slot();\n> \tslot->callback_func = process_response;\n> @@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)\n> \tchar *ep;\n> \tchar timeout_header[25];\n> \tstruct remote_lock *lock = NULL;\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n> \tstruct xml_ctx ctx;\n> \tchar *escaped;\n> \n> @@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,\n> \tstruct slot_results results;\n> \tstruct strbuf in_buffer = STRBUF_INIT;\n> \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n> \tstruct xml_ctx ctx;\n> \tstruct remote_ls_ctx ls;\n> \n> @@ -1204,7 +1204,7 @@ static int locking_available(void)\n> \tstruct slot_results results;\n> \tstruct strbuf in_buffer = STRBUF_INIT;\n> \tstruct buffer out_buffer = { STRBUF_INIT, 0 };\n> -\tstruct curl_slist *dav_headers = NULL;\n> +\tstruct curl_slist *dav_headers = http_copy_default_headers();\n> \tstruct xml_ctx ctx;\n> \tint lock_flags = 0;\n> \tchar *escaped;\n> diff --git a/http.c b/http.c\n> index 4304b80..985b995 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;\n> \n> static struct curl_slist *pragma_header;\n> static struct curl_slist *no_pragma_header;\n> +static struct curl_slist *extra_http_headers;\n> \n> static struct active_request_slot *active_queue_head;\n> \n> @@ -323,6 +324,19 @@ static int http_options(const char *var, const char *value, void *cb)\n> #endif\n> \t}\n> \n> +\tif (!strcmp(\"http.extraheader\", var)) {\n> +\t\tif (!value) {\n> +\t\t\treturn config_error_nonbool(var);\n> +\t\t} else if (!*value) {\n> +\t\t\tcurl_slist_free_all(extra_http_headers);\n> +\t\t\textra_http_headers = NULL;\n> +\t\t} else {\n> +\t\t\textra_http_headers =\n> +\t\t\t\tcurl_slist_append(extra_http_headers, value);\n> +\t\t}\n> +\t\treturn 0;\n> +\t}\n> +\n> \t/* Fall back on the default ones */\n> \treturn git_default_config(var, value, cb);\n> }\n> @@ -678,8 +692,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n> \tif (remote)\n> \t\tvar_override(&http_proxy_authmethod, remote->http_proxy_authmethod);\n> \n> -\tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\n> -\tno_pragma_header = curl_slist_append(no_pragma_header, \"Pragma:\");\n> +\tpragma_header = curl_slist_append(http_copy_default_headers(),\n> +\t\t\"Pragma: no-cache\");\n> +\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n> +\t\t\"Pragma:\");\n> \n> #ifdef USE_CURL_MULTI\n> \t{\n> @@ -765,6 +781,9 @@ void http_cleanup(void)\n> #endif\n> \tcurl_global_cleanup();\n> \n> +\tcurl_slist_free_all(extra_http_headers);\n> +\textra_http_headers = NULL;\n> +\n> \tcurl_slist_free_all(pragma_header);\n> \tpragma_header = NULL;\n> \n> @@ -1163,6 +1182,16 @@ int run_one_slot(struct active_request_slot *slot,\n> \treturn handle_curl_result(results);\n> }\n> \n> +struct curl_slist *http_copy_default_headers(void)\n> +{\n> +\tstruct curl_slist *headers = NULL, *h;\n> +\n> +\tfor (h = extra_http_headers; h; h = h->next)\n> +\t\theaders = curl_slist_append(headers, h->data);\n> +\n> +\treturn headers;\n> +}\n> +\n> static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)\n> {\n> \tchar *ptr;\n> @@ -1380,7 +1409,7 @@ static int http_request(const char *url,\n> {\n> \tstruct active_request_slot *slot;\n> \tstruct slot_results results;\n> -\tstruct curl_slist *headers = NULL;\n> +\tstruct curl_slist *headers = http_copy_default_headers();\n> \tstruct strbuf buf = STRBUF_INIT;\n> \tconst char *accept_language;\n> \tint ret;\n> diff --git a/http.h b/http.h\n> index 4ef4bbd..36f558b 100644\n> --- a/http.h\n> +++ b/http.h\n> @@ -106,6 +106,7 @@ extern void step_active_slots(void);\n> extern void http_init(struct remote *remote, const char *url,\n> \t\t int proactive_auth);\n> extern void http_cleanup(void);\n> +extern struct curl_slist *http_copy_default_headers(void);\n> \n> extern long int git_curl_ipresolve;\n> extern int active_requests;\n> diff --git a/remote-curl.c b/remote-curl.c\n> index 15e48e2..672b382 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,\n> static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n> {\n> \tstruct active_request_slot *slot;\n> -\tstruct curl_slist *headers = NULL;\n> +\tstruct curl_slist *headers = http_copy_default_headers();\n> \tstruct strbuf buf = STRBUF_INIT;\n> \tint err;\n> \n> @@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n> static int post_rpc(struct rpc_state *rpc)\n> {\n> \tstruct active_request_slot *slot;\n> -\tstruct curl_slist *headers = NULL;\n> +\tstruct curl_slist *headers = http_copy_default_headers();\n> \tint use_gzip = rpc->gzip_request;\n> \tchar *gzip_body = NULL;\n> \tsize_t gzip_size = 0;\n> diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n> index 9317ba0..b8ed96f 100644\n> --- a/t/lib-httpd/apache.conf\n> +++ b/t/lib-httpd/apache.conf\n> @@ -102,6 +102,14 @@ Alias /auth/dumb/ www/auth/dumb/\n> \tSetEnv GIT_HTTP_EXPORT_ALL\n> \tHeader set Set-Cookie name=value\n> \n> +\n> +\t\n> +\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n> +\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n> +\t\n\nI think \"\" depends on mod_authz_core which is only\navailable in Apache HTTPD 2.3 or later [1].\n\nRight now the test only checks if Apache version greater 2\nis installed. Should we guard this test with a special version\ncheck? Or do you see a way to check the magic values without\n\"\"?\n\nI only noticed this because I enabled these tests on Travis-CI\nand the Travis Linux box comes with Apache 2.2.22 installed...\n\n- Lars\n\n[1] https://httpd.apache.org/docs/trunk/mod/mod_authz_core.html\n\n> +\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n> +\tSetEnv GIT_HTTP_EXPORT_ALL\n> +\n> ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n> ScriptAlias /broken_smart/ broken-smart-http.sh/\n> ScriptAlias /error/ error.sh/\n> diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n> index 58207d8..e44fe72 100755\n> --- a/t/t5551-http-fetch-smart.sh\n> +++ b/t/t5551-http-fetch-smart.sh\n> @@ -282,5 +282,12 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n> \ttest_line_count = 100000 tags\n> '\n> \n> +test_expect_success 'custom http headers' '\n> +\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> +\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n> +'\n> +\n> stop_httpd\n> test_done\n> -- \n> 2.8.1.306.gff998f2\n> \n> \n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"285591","messageId":"xmqq37pwb8dx.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"67F2E414-657C-4C6C-9746-9B3026737C66@gmail.com","subject":"Re: [PATCH v6 1/2] http: support sending custom HTTP headers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-05T19:40:58Z","receivedAt":"2016-05-05T19:40:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n[administrivia: next time please snip the 224 lines you are not\nresponding to when commenting on only five lines].\n\n>> +\n>> +\t\n>> +\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n>> +\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n>> +\t\n>\n> I think \"\" depends on mod_authz_core which is only\n> available in Apache HTTPD 2.3 or later [1].\n>\n> Right now the test only checks if Apache version greater 2\n> is installed. Should we guard this test with a special version\n> check? Or do you see a way to check the magic values without\n> \"\"?\n>\n> I only noticed this because I enabled these tests on Travis-CI\n> and the Travis Linux box comes with Apache 2.2.22 installed...\n>\n> - Lars\n>\n> [1] https://httpd.apache.org/docs/trunk/mod/mod_authz_core.html\n\n\n2.2 is 12 years old, but the last update is last summer, which is\nstill fairly recent. It would be really nice if we can accomodate\n2.2.x series.\n\nIt is only test, so it probably is OK to skip it in the worst case,\nbut on the other hand, the point of having CI is so that we can\ncatch bugs that would manifest only in tests that normal people\nwould not run regularly, so...\n\nThanks. Real-world value of having Travis canary is seen here.\n"},{"id":"285595","messageId":"20160505200327.GC9162@sigill.intra.peff.net","threadId":"42138","inReplyTo":"67F2E414-657C-4C6C-9746-9B3026737C66@gmail.com","subject":"Re: [PATCH v6 1/2] http: support sending custom HTTP headers","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-05T20:03:27Z","receivedAt":"2016-05-05T20:03:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 05, 2016 at 09:10:21PM +0200, Lars Schneider wrote:\n\n> > +\n> > +\t\n> > +\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n> > +\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n> > +\t\n> \n> I think \"\" depends on mod_authz_core which is only\n> available in Apache HTTPD 2.3 or later [1].\n> \n> Right now the test only checks if Apache version greater 2\n> is installed. Should we guard this test with a special version\n> check? Or do you see a way to check the magic values without\n> \"\"?\n\nI think you can get rid of RequireAll with:\n\n Require expr %{HTTP:x-magic-one} == 'abra' && %{HTTP:x-magic-two} == 'cadabra'\n\nBut I am also not sure that \"expr\" existed in Apache 2.2.\n\nI think the older way of checking headers was to do some trickery with\nRewriteCond; I tried briefly to make that work when I wrote the test,\nbut never did (but I'm far from an expert in Apache; the only reason I\nhave touched it in the last 15 years is for Git's test suite).\n\nThe nuclear option would be to put a shell script between Apache and\ngit-http-backend that checks for those headers (I suspect we'd still\nneed some magic in the Apache config to pass the headers out in the\nenvironment, though).\n\n-Peff\n"},{"id":"285597","messageId":"20160505201416.GD9162@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqq60uscufw.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-05T20:14:16Z","receivedAt":"2016-05-05T20:14:16Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 05, 2016 at 09:59:15AM -0700, Junio C Hamano wrote:\n\n> Sorry for the trouble.\n> \n> I queued jk/submodule-c-credential to be mergeable to 'maint', as it\n> could be argued two ways. We can say that not propagating -c down\n> is a bug and the series is a bugfix. Or it is merely a known\n> limitation and the series is a new feature. I was leaning towards\n> the former, but I was also willing to declare that is a known bug\n> that will left unfixed in the maintenance track.\n\nAh, OK, that makes perfect sense.\n\n> It probably is a good time to merge jk/submodule-config-sanitize-fix\n> into jk/submodule-c-credential (i.e. a mere fast-forward), remove\n> that \"-fix\" branch, and apply this patch directly on top of the\n> resulting jk/submodule-c-credential. That will make the whole thing\n> a 13-patch series, consisting of:\n> \n> 7 patches up to the current jk/submodule-c-credential\n> d1f8849 git_config_push_parameter: handle empty GIT_CONFIG_PARAMETERS\n> 14111fc git: submodule honor -c credential.* from command line\n> e70986d quote: implement sq_quotef()\n> 7dad263 submodule: fix segmentation fault in submodule--helper clone\n> 717416c submodule: fix submodule--helper clone usage\n> 08e0970 submodule: check argc count for git submodule--helper clone\n> d10e3b4 submodule: don't pass empty string arguments to submodule--helper clone\n> \n> 5 patches up to the current jk/submodule-config-sanitize-fix\n> c12e865 submodule: use prepare_submodule_repo_env consistently\n> 4638728 submodule--helper: move config-sanitizing to submodule.c\n> 860cba6 submodule: export sanitized GIT_CONFIG_PARAMETERS\n> 455d22c t5550: break submodule config test into multiple sub-tests\n> 1149ee2 t5550: fix typo in $HTTPD_URL\n> \n> 1 patch (this one)\n> 4e6706a submodule: stop sanitizing config options\n\nThat sounds reasonable. Note that the later patches drop the only caller\nof the newly-introduced sq_quotef(). So we could revert e70986d\n(omitting it from the series doesn't make sense, as it would leave a\nbroken state in the middle). I am also fine with leaving it. It seems\nlike a potentially useful addition.\n\nI had originally thought after the final one that we could further clean\nup by turning prepare_submodule_repo_env() into a static function. But\nwe can't; it gets called in one spot from submodule--helper. However,\nwhile looking at it, I did notice that we probably want to squash this\ninto the final patch (since sanitize_submodule_config went away\ncompletely):\n\ndiff --git a/submodule.h b/submodule.h\nindex 48690b1..869d259 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -43,19 +43,10 @@ int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_nam\n int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);\n void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n \n-/*\n- * This function is intended as a callback for use with\n- * git_config_from_parameters(). It ignores any config options which\n- * are not suitable for passing along to a submodule, and accumulates the rest\n- * in \"data\", which must be a pointer to a strbuf. The end result can\n- * be put into $GIT_CONFIG_PARAMETERS for passing to a sub-process.\n- */\n-int sanitize_submodule_config(const char *var, const char *value, void *data);\n-\n /*\n * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n * a submodule by clearing any repo-specific envirionment variables, but\n- * retaining any config approved by sanitize_submodule_config().\n+ * retaining any config in the environment.\n */\n void prepare_submodule_repo_env(struct argv_array *out);\n \n\n-Peff\n"},{"id":"285649","messageId":"xmqqtwic6pxs.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160505201416.GD9162@sigill.intra.peff.net","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-05T23:33:03Z","receivedAt":"2016-05-05T23:33:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> I had originally thought after the final one that we could further clean\n> up by turning prepare_submodule_repo_env() into a static function. But\n> we can't; it gets called in one spot from submodule--helper. However,\n> while looking at it, I did notice that we probably want to squash this\n> into the final patch (since sanitize_submodule_config went away\n> completely):\n>\n> diff --git a/submodule.h b/submodule.h\n> index 48690b1..869d259 100644\n> --- a/submodule.h\n> +++ b/submodule.h\n> @@ -43,19 +43,10 @@ int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_nam\n> int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);\n> void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n> \n> -/*\n> - * This function is intended as a callback for use with\n> - * git_config_from_parameters(). It ignores any config options which\n> - * are not suitable for passing along to a submodule, and accumulates the rest\n> - * in \"data\", which must be a pointer to a strbuf. The end result can\n> - * be put into $GIT_CONFIG_PARAMETERS for passing to a sub-process.\n> - */\n> -int sanitize_submodule_config(const char *var, const char *value, void *data);\n> -\n> /*\n> * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n> * a submodule by clearing any repo-specific envirionment variables, but\n> - * retaining any config approved by sanitize_submodule_config().\n> + * retaining any config in the environment.\n> */\n> void prepare_submodule_repo_env(struct argv_array *out);\n> \n>\n> -Peff\n\nHmph, Stefan, do you want to keep this (if you want to resurrect the\nfunction in some future, for example)?\n"},{"id":"285652","messageId":"CAGZ79kZ4wOQrZETB4UVVRO6oecF-qqjzb9hP6-JenNxFNfjiLw@mail.gmail.com","threadId":"42138","inReplyTo":"xmqqtwic6pxs.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-05-06T00:23:51Z","receivedAt":"2016-05-06T00:23:51Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, May 5, 2016 at 4:33 PM, Junio C Hamano wrote:\n> Jeff King writes:\n>\n>> I had originally thought after the final one that we could further clean\n>> up by turning prepare_submodule_repo_env() into a static function. But\n>> we can't; it gets called in one spot from submodule--helper. However,\n>> while looking at it, I did notice that we probably want to squash this\n>> into the final patch (since sanitize_submodule_config went away\n>> completely):\n>>\n>> diff --git a/submodule.h b/submodule.h\n>> index 48690b1..869d259 100644\n>> --- a/submodule.h\n>> +++ b/submodule.h\n>> @@ -43,19 +43,10 @@ int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_nam\n>> int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);\n>> void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n>>\n>> -/*\n>> - * This function is intended as a callback for use with\n>> - * git_config_from_parameters(). It ignores any config options which\n>> - * are not suitable for passing along to a submodule, and accumulates the rest\n>> - * in \"data\", which must be a pointer to a strbuf. The end result can\n>> - * be put into $GIT_CONFIG_PARAMETERS for passing to a sub-process.\n>> - */\n>> -int sanitize_submodule_config(const char *var, const char *value, void *data);\n>> -\n>> /*\n>> * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n>> * a submodule by clearing any repo-specific envirionment variables, but\n>> - * retaining any config approved by sanitize_submodule_config().\n>> + * retaining any config in the environment.\n>> */\n>> void prepare_submodule_repo_env(struct argv_array *out);\n>>\n>>\n>> -Peff\n>\n> Hmph, Stefan, do you want to keep this (if you want to resurrect the\n> function in some future, for example)?\n\nIMHO it is easier to revert or rewrite than to carry unused code?\nUnused code is not tested and untested code is broken code.\nAnd relying on broken code in the future will guarantee bugs.\n(Sure new code may also have bugs, but I just think that bugs\nin newly written code can be fixed easier)\n\nI would prefer to get rid of it, i.e. taking that patch.\n\nThanks,\nStefan\n"},{"id":"285655","messageId":"20160506010053.GA15474@sigill.intra.peff.net","threadId":"42138","inReplyTo":"CAGZ79kZ4wOQrZETB4UVVRO6oecF-qqjzb9hP6-JenNxFNfjiLw@mail.gmail.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-06T01:00:54Z","receivedAt":"2016-05-06T01:00:54Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 05, 2016 at 05:23:51PM -0700, Stefan Beller wrote:\n\n> >> -/*\n> >> - * This function is intended as a callback for use with\n> >> - * git_config_from_parameters(). It ignores any config options which\n> >> - * are not suitable for passing along to a submodule, and accumulates the rest\n> >> - * in \"data\", which must be a pointer to a strbuf. The end result can\n> >> - * be put into $GIT_CONFIG_PARAMETERS for passing to a sub-process.\n> >> - */\n> >> -int sanitize_submodule_config(const char *var, const char *value, void *data);\n> >> -\n> >> /*\n> >> * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n> >> * a submodule by clearing any repo-specific envirionment variables, but\n> >> - * retaining any config approved by sanitize_submodule_config().\n> >> + * retaining any config in the environment.\n> >> */\n> >> void prepare_submodule_repo_env(struct argv_array *out);\n> >>\n> >>\n> >> -Peff\n> >\n> > Hmph, Stefan, do you want to keep this (if you want to resurrect the\n> > function in some future, for example)?\n> \n> IMHO it is easier to revert or rewrite than to carry unused code?\n> Unused code is not tested and untested code is broken code.\n> And relying on broken code in the future will guarantee bugs.\n> (Sure new code may also have bugs, but I just think that bugs\n> in newly written code can be fixed easier)\n> \n> I would prefer to get rid of it, i.e. taking that patch.\n\nKeep in mind this squash is just dropping the _declaration_. The code\nitself was dropped in the earlier patch. (And I agree there isn't a good\nreason to keep it when we can easily \"git revert\" later).\n\n-Peff\n"},{"id":"285775","messageId":"xmqqmvo32c5n.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"CAGZ79kZ4wOQrZETB4UVVRO6oecF-qqjzb9hP6-JenNxFNfjiLw@mail.gmail.com","subject":"Re: [PATCH] submodule: stop sanitizing config options","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-06T19:56:36Z","receivedAt":"2016-05-06T19:56:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n>> Hmph, Stefan, do you want to keep this (if you want to resurrect the\n>> function in some future, for example)?\n>\n> IMHO it is easier to revert or rewrite than to carry unused code?\n> Unused code is not tested and untested code is broken code.\n> And relying on broken code in the future will guarantee bugs.\n> (Sure new code may also have bugs, but I just think that bugs\n> in newly written code can be fixed easier)\n>\n> I would prefer to get rid of it, i.e. taking that patch.\n\nOK.\n"},{"id":"285907","messageId":"cover.1462774709.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462342213.git.johannes.schindelin@gmx.de","subject":"[PATCH v7 0/3] Add support for sending additional HTTP headers (part 2)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T06:18:46Z","receivedAt":"2016-05-09T06:18:46Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"My use case is an army of build agents that need only limited and\nselective access to otherwise private repositories.\n\nThe first part already made it into `master`, this is the remainder.\n\nThis iteration still has the specific patch to make `git -c\nhttp.extraHeader=... submodule update` work; I plan to keep only the\ntest (and adjust the commit message) as soon as Peff's patch is applied\nthat skips -c ... sanitizing for submodules.\n\n\nJohannes Schindelin (3):\n tests: Adjust the configuration for Apache 2.2\n t5551: make the test for extra HTTP headers more robust\n submodule: pass on http.extraheader config settings\n\n builtin/submodule--helper.c | 3 ++-\n t/lib-httpd/apache.conf | 12 ++++++++----\n t/t5551-http-fetch-smart.sh | 14 ++++++++++++--\n 3 files changed, 22 insertions(+), 7 deletions(-)\n\nPublished-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v7\nInterdiff vs v6:\n\n diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n index b8ed96f..29b34bb 100644\n --- a/t/lib-httpd/apache.conf\n +++ b/t/lib-httpd/apache.conf\n @@ -103,10 +103,6 @@ Alias /auth/dumb/ www/auth/dumb/\n \tHeader set Set-Cookie name=value\n \n \n -\t\n -\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n -\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n -\t\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \n @@ -136,6 +132,14 @@ RewriteRule ^/ftp-redir/(.*)$ ftp://localhost:1000/$1 [R=302]\n RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302]\n RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n \n +# Apache 2.2 does not understand , so we use RewriteCond.\n +# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n +# ugly trick to fail *unless* the two headers are present.\n +RewriteCond %{HTTP:x-magic-one} =abra\n +RewriteCond %{HTTP:x-magic-two} =cadabra\n +RewriteRule ^/smart_headers/.* - [L]\n +RewriteRule ^/smart_headers/.* - [F]\n +\n \n LoadModule ssl_module modules/mod_ssl.so\n \n diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n index 1794168..2f375eb 100755\n --- a/t/t5551-http-fetch-smart.sh\n +++ b/t/t5551-http-fetch-smart.sh\n @@ -283,7 +283,8 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n '\n \n test_expect_success 'custom http headers' '\n -\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n +\ttest_must_fail git -c http.extraheader=\"x-magic-two: cadabra\" \\\n +\t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n \t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n\n-- \n2.8.2.463.g99156ee\n"},{"id":"285908","messageId":"4a15c4e6c35cfb425da568d87e8b20b984e5325c.1462774709.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462774709.git.johannes.schindelin@gmx.de","subject":"[PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T06:18:52Z","receivedAt":"2016-05-09T06:18:52Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Lars Schneider noticed that the configuration introduced to test the extra\nHTTP headers cannot be used with Apache 2.2 (which is still actively\nmaintained, as pointed out by Junio Hamano).\n\nTo let the tests pass with Apache 2.2 again, let's substitute the\noffending and `expr` by using old school RewriteCond\nstatements.\n\nSigned-off-by: Johannes Schindelin \n---\n t/lib-httpd/apache.conf | 12 ++++++++----\n 1 file changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex b8ed96f..29b34bb 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -103,10 +103,6 @@ Alias /auth/dumb/ www/auth/dumb/\n \tHeader set Set-Cookie name=value\n \n \n-\t\n-\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n-\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n-\t\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \n@@ -136,6 +132,14 @@ RewriteRule ^/ftp-redir/(.*)$ ftp://localhost:1000/$1 [R=302]\n RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302]\n RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n \n+# Apache 2.2 does not understand , so we use RewriteCond.\n+# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n+# ugly trick to fail *unless* the two headers are present.\n+RewriteCond %{HTTP:x-magic-one} =abra\n+RewriteCond %{HTTP:x-magic-two} =cadabra\n+RewriteRule ^/smart_headers/.* - [L]\n+RewriteRule ^/smart_headers/.* - [F]\n+\n \n LoadModule ssl_module modules/mod_ssl.so\n \n-- \n2.8.2.463.g99156ee\n"},{"id":"285909","messageId":"f29dac327aeac23677aec955f5b46a7a4702abfe.1462774709.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462774709.git.johannes.schindelin@gmx.de","subject":"[PATCH v7 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T06:19:00Z","receivedAt":"2016-05-09T06:19:00Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To test that extra HTTP headers are passed correctly, t5551 verifies that\na fetch succeeds when two required headers are passed, and that the fetch\ndoes not succeed when those headers are not passed.\n\nHowever, this test would also succeed if the configuration required only\none header. As Apache's configuration is notoriously tricky (this\ndeveloper frequently requires StackOverflow's help to understand Apache's\ndocumentation), especially when still supporting the 2.2 line, let's just\nreally make sure that the test verifies what we want it to verify.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t5551-http-fetch-smart.sh | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex e44fe72..43b257e 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -283,7 +283,8 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n '\n \n test_expect_success 'custom http headers' '\n-\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\ttest_must_fail git -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n \t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n-- \n2.8.2.463.g99156ee\n"},{"id":"285906","messageId":"554ebb7aa9c4a9974181ff932f1695aca3b42abf.1462774709.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462774709.git.johannes.schindelin@gmx.de","subject":"[PATCH v7 3/3] submodule: pass on http.extraheader config settings","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T06:19:09Z","receivedAt":"2016-05-09T06:19:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To support this developer's use case of allowing build agents token-based\naccess to private repositories, we introduced the http.extraheader\nfeature, allowing extra HTTP headers to be sent along with every HTTP\nrequest.\n\nThis patch allows us to configure these extra HTTP headers for use with\n`git submodule update`, too. It requires somewhat special handling:\nsubmodules do not share the parent project's config. It would be\nincorrect to simply reuse that specific part of the parent's config.\nInstead, the config option needs to be specified on the command-line or\nin ~/.gitconfig or friends.\n\nExample: git -c http.extraheader=\"Secret: Sauce\" submodule update --init\n\nSigned-off-by: Johannes Schindelin \n---\n builtin/submodule--helper.c | 3 ++-\n t/t5551-http-fetch-smart.sh | 11 ++++++++++-\n 2 files changed, 12 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 3bd6883..789e081 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -127,7 +127,8 @@ static int module_name(int argc, const char **argv, const char *prefix)\n */\n static int submodule_config_ok(const char *var)\n {\n-\tif (starts_with(var, \"credential.\"))\n+\tif (starts_with(var, \"credential.\") ||\n+\t (starts_with(var, \"http.\") && ends_with(var, \".extraheader\")))\n \t\treturn 1;\n \treturn 0;\n }\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex 43b257e..2f375eb 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -287,7 +287,16 @@ test_expect_success 'custom http headers' '\n \t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n-\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit update-index --add --cacheinfo 160000,$(git rev-parse HEAD),sub &&\n+\tgit config -f .gitmodules submodule.sub.path sub &&\n+\tgit config -f .gitmodules submodule.sub.url \\\n+\t\t\"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit submodule init sub &&\n+\ttest_must_fail git submodule update sub &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t\tsubmodule update sub\n '\n \n stop_httpd\n-- \n2.8.2.463.g99156ee\n"},{"id":"285911","messageId":"BC0E627C-416C-41AA-9D09-68634845BC3D@gmail.com","threadId":"42138","inReplyTo":"f29dac327aeac23677aec955f5b46a7a4702abfe.1462774709.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v7 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-05-09T07:56:29Z","receivedAt":"2016-05-09T07:56:29Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 09 May 2016, at 08:19, Johannes Schindelin wrote:\n\n> To test that extra HTTP headers are passed correctly, t5551 verifies that\n> a fetch succeeds when two required headers are passed, and that the fetch\n> does not succeed when those headers are not passed.\n> \n> However, this test would also succeed if the configuration required only\n> one header. As Apache's configuration is notoriously tricky (this\n> developer frequently requires StackOverflow's help to understand Apache's\n> documentation), especially when still supporting the 2.2 line, let's just\n> really make sure that the test verifies what we want it to verify.\n\nHaha. Me, too :-) After I wasn't able to find a working solution myself\nI posted a question on ServerFault [1] ... I will test your solution\ntomorrow!\n\nCheers,\nLars\n\n[1] https://serverfault.com/questions/775515/requireall-require-expr-equivalent-in-apache-2-2-22-to-check-for-headers\n"},{"id":"285912","messageId":"20160509080315.GA14383@sigill.intra.peff.net","threadId":"42138","inReplyTo":"4a15c4e6c35cfb425da568d87e8b20b984e5325c.1462774709.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-09T08:03:16Z","receivedAt":"2016-05-09T08:03:16Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 09, 2016 at 08:18:52AM +0200, Johannes Schindelin wrote:\n\n> +# Apache 2.2 does not understand , so we use RewriteCond.\n> +# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n> +# ugly trick to fail *unless* the two headers are present.\n> +RewriteCond %{HTTP:x-magic-one} =abra\n> +RewriteCond %{HTTP:x-magic-two} =cadabra\n> +RewriteRule ^/smart_headers/.* - [L]\n> +RewriteRule ^/smart_headers/.* - [F]\n> +\n\nThanks, this is the magic that eluded me earlier. I had to look up the\nflags, so for any observers in the same boat, this works because:\n\n - the '[L]' flag says \"stop doing any more rewrite rules\"; it triggers\n only when the RewriteConds above match\n\n - the '[F]' flag says \"return 403 Forbidden\"; it triggers always,\n because after a RewriteRule, all RewriteConds are reset\n\nI'm sure that is all apparent to somebody who is familiar with Apache\nconfig, but I think that does not include most people on this project. I\ndunno if it is worth a comment here or in the commit message.\n\n-Peff\n"},{"id":"285913","messageId":"20160509080544.GB14383@sigill.intra.peff.net","threadId":"42138","inReplyTo":"f29dac327aeac23677aec955f5b46a7a4702abfe.1462774709.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v7 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-09T08:05:44Z","receivedAt":"2016-05-09T08:05:44Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 09, 2016 at 08:19:00AM +0200, Johannes Schindelin wrote:\n\n> To test that extra HTTP headers are passed correctly, t5551 verifies that\n> a fetch succeeds when two required headers are passed, and that the fetch\n> does not succeed when those headers are not passed.\n> \n> However, this test would also succeed if the configuration required only\n> one header. As Apache's configuration is notoriously tricky (this\n> developer frequently requires StackOverflow's help to understand Apache's\n> documentation), especially when still supporting the 2.2 line, let's just\n> really make sure that the test verifies what we want it to verify.\n\nAgreed, this makes sense.\n\n> test_expect_success 'custom http headers' '\n> -\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> +\ttest_must_fail git -c http.extraheader=\"x-magic-two: cadabra\" \\\n> +\t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> \t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n\nThis loses the 0-header check, but I don't think that is particularly\ninteresting to us (I had originally wanted to double-check that our\napache config worked at all in the absence of this feature, but I think\nit is OK for the 1-header case to cover this; if our code is so buggy we\naccidentally send 0 headers in the first command, we'll catch that,\ntoo).\n\nSo looks good to me.\n\n-Peff\n"},{"id":"285914","messageId":"alpine.DEB.2.20.1605091011420.4092@virtualbox","threadId":"42138","inReplyTo":"20160509080544.GB14383@sigill.intra.peff.net","subject":"Re: [PATCH v7 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T08:13:51Z","receivedAt":"2016-05-09T08:13:51Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\ndo you sleep at all?\n\nOn Mon, 9 May 2016, Jeff King wrote:\n\n> On Mon, May 09, 2016 at 08:19:00AM +0200, Johannes Schindelin wrote:\n> \n> > test_expect_success 'custom http headers' '\n> > -\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> > +\ttest_must_fail git -c http.extraheader=\"x-magic-two: cadabra\" \\\n> > +\t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> > \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> > \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> > \t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n> \n> This loses the 0-header check, but I don't think that is particularly\n> interesting to us (I had originally wanted to double-check that our\n> apache config worked at all in the absence of this feature, but I think\n> it is OK for the 1-header case to cover this; if our code is so buggy we\n> accidentally send 0 headers in the first command, we'll catch that,\n> too).\n\nYeah, a faulty Apache config will unfortunately *skip* the entire test, as\nhttpd refuses to start.\n\nAnd I also considered testing 0-header and 1st header only. But as you\nknow, Git's test suite takes already 3.5h in a moderately sized Windows\nVM, so I am really reluctant to add overly extensive tests.\n\n> So looks good to me.\n\nThanks,\nDscho\n"},{"id":"285915","messageId":"20160509082024.GA14299@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1605091011420.4092@virtualbox","subject":"Re: [PATCH v7 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-09T08:20:24Z","receivedAt":"2016-05-09T08:20:24Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 09, 2016 at 10:13:51AM +0200, Johannes Schindelin wrote:\n\n> do you sleep at all?\n\nActually, I just woke up. Nothing like some Git ML to get the blood\npumping in the morning.\n\n> Yeah, a faulty Apache config will unfortunately *skip* the entire test, as\n> httpd refuses to start.\n\nIf you care about this, you can set GIT_TEST_HTTPD=true. The default is\n\"auto\", which will skip whenever apache setup fails. But with \"true\", it\nwill show a hard error (so at least your \"make test\" will report the\nfailure).\n\n-Peff\n"},{"id":"285917","messageId":"alpine.DEB.2.20.1605091557050.4092@virtualbox","threadId":"42138","inReplyTo":"20160509080315.GA14383@sigill.intra.peff.net","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T14:03:48Z","receivedAt":"2016-05-09T14:03:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Mon, 9 May 2016, Jeff King wrote:\n\n> On Mon, May 09, 2016 at 08:18:52AM +0200, Johannes Schindelin wrote:\n> \n> > +# Apache 2.2 does not understand , so we use RewriteCond.\n> > +# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n> > +# ugly trick to fail *unless* the two headers are present.\n> > +RewriteCond %{HTTP:x-magic-one} =abra\n> > +RewriteCond %{HTTP:x-magic-two} =cadabra\n> > +RewriteRule ^/smart_headers/.* - [L]\n> > +RewriteRule ^/smart_headers/.* - [F]\n> > +\n> \n> Thanks, this is the magic that eluded me earlier. I had to look up the\n> flags, so for any observers in the same boat, this works because:\n> \n> - the '[L]' flag says \"stop doing any more rewrite rules\"; it triggers\n> only when the RewriteConds above match\n> \n> - the '[F]' flag says \"return 403 Forbidden\"; it triggers always,\n> because after a RewriteRule, all RewriteConds are reset\n> \n> I'm sure that is all apparent to somebody who is familiar with Apache\n> config, but I think that does not include most people on this project. I\n> dunno if it is worth a comment here or in the commit message.\n\nOh, you're absolutely correct, I should have described this better. It\ntook me quite a couple of iterations to get it right, after all.\n\nHow about this:\n\n\tAs RewriteCond does not allow testing for *non*-matches, we simply\n\tmatch the desired case first and let it pass by marking the\n\tRewriteRule as '[L]' (\"last rule, do not process any other\n\tmatching RewriteRules after this\"), and then have another\n\tRewriteRule that matches all other cases and lets them fail via\n\t'[F]' (\"fail\").\n\nGood enough?\n\nCiao,\nDscho\n"},{"id":"285919","messageId":"20160509142711.GA9552@sigill.intra.peff.net","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1605091557050.4092@virtualbox","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-09T14:27:11Z","receivedAt":"2016-05-09T14:27:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 09, 2016 at 04:03:48PM +0200, Johannes Schindelin wrote:\n\n> How about this:\n> \n> \tAs RewriteCond does not allow testing for *non*-matches, we simply\n> \tmatch the desired case first and let it pass by marking the\n> \tRewriteRule as '[L]' (\"last rule, do not process any other\n> \tmatching RewriteRules after this\"), and then have another\n> \tRewriteRule that matches all other cases and lets them fail via\n> \t'[F]' (\"fail\").\n> \n> Good enough?\n\nYep, I think that explains it. Thanks.\n\n-Peff\n"},{"id":"285922","messageId":"alpine.DEB.2.20.1605091710310.4092@virtualbox","threadId":"42138","inReplyTo":"20160509142711.GA9552@sigill.intra.peff.net","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-09T15:11:44Z","receivedAt":"2016-05-09T15:11:44Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Mon, 9 May 2016, Jeff King wrote:\n\n> On Mon, May 09, 2016 at 04:03:48PM +0200, Johannes Schindelin wrote:\n> \n> > How about this:\n> > \n> > \tAs RewriteCond does not allow testing for *non*-matches, we simply\n> > \tmatch the desired case first and let it pass by marking the\n> > \tRewriteRule as '[L]' (\"last rule, do not process any other\n> > \tmatching RewriteRules after this\"), and then have another\n> > \tRewriteRule that matches all other cases and lets them fail via\n> > \t'[F]' (\"fail\").\n> > \n> > Good enough?\n> \n> Yep, I think that explains it. Thanks.\n\nOkay, I already force-pushed my extra-http-header branch and the next\niteration will sport this paragraph.\n\nHopefully your patch to remove the -c ... sanitizing makes it to `master`\nsoon, then I can submit my next iteration.\n\nCiao,\nDscho\n"},{"id":"285932","messageId":"xmqqeg9bxkrs.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"4a15c4e6c35cfb425da568d87e8b20b984e5325c.1462774709.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-09T16:23:51Z","receivedAt":"2016-05-09T16:23:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> +# Apache 2.2 does not understand , so we use RewriteCond.\n> +# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n> +# ugly trick to fail *unless* the two headers are present.\n> +RewriteCond %{HTTP:x-magic-one} =abra\n> +RewriteCond %{HTTP:x-magic-two} =cadabra\n> +RewriteRule ^/smart_headers/.* - [L]\n> +RewriteRule ^/smart_headers/.* - [F]\n\nClever. Thanks. Will queue.\n"},{"id":"285936","messageId":"xmqqwpn3w5c7.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1605091710310.4092@virtualbox","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-09T16:42:32Z","receivedAt":"2016-05-09T16:42:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Okay, I already force-pushed my extra-http-header branch and the next\n> iteration will sport this paragraph.\n\nThe new explanation is well written and can and should also replace\nthe comment before the implementation in the configuration file to\nhelp readers.\n\nTo be honest, I do not quite understand why you call it \"ugly hack\"\nat all. It is saying \"The client is sending a satisfactory request\nwhen these two headers are there; otherwise the request is not\ngood\", which sounds quite natural way to express what is being\ntested. The rejection being part of \"Rewrite\" may be clever, but I\ndo not see it as ugly.\n\nAnd it matches the spirit of the implementation for 2.3+ that uses\n quite well--you just do not need to say \"Fail\"\nyourself over there, as that is implied.\n\n> Hopefully your patch to remove the -c ... sanitizing makes it to `master`\n> soon, then I can submit my next iteration.\n\nOr we can just merge that \"do not sanitize\" branch in, and then\nqueue the \"next iteration\" which I'd assume would only be the test\naddition?\n\nThanks.\n\n-- >8 --\nFrom: Johannes Schindelin \nDate: Mon, 9 May 2016 07:59:16 +0200\nSubject: [PATCH] tests: adjust the configuration for Apache 2.2\n\nLars Schneider noticed that the configuration introduced to test the\nextra HTTP headers cannot be used with Apache 2.2 (which is still\nactively maintained, as pointed out by Junio Hamano).\n\nTo let the tests pass with Apache 2.2 again, let's substitute the\noffending and `expr` by using old school RewriteCond\nstatements.\n\nAs RewriteCond does not allow testing for *non*-matches, we simply match\nthe desired case first and let it pass by marking the RewriteRule as\n'[L]' (\"last rule, do not process any other matching RewriteRules after\nthis\"), and then have another RewriteRule that matches all other cases\nand lets them fail via '[F]' (\"fail\").\n\nSigned-off-by: Johannes Schindelin \nSigned-off-by: Junio C Hamano \n---\n t/lib-httpd/apache.conf | 16 ++++++++++++----\n 1 file changed, 12 insertions(+), 4 deletions(-)\n\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 838770c..2dcbb00 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -102,10 +102,6 @@ Alias /auth/dumb/ www/auth/dumb/\n \tHeader set Set-Cookie name=value\n \n \n-\t\n-\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n-\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n-\t\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \n@@ -135,6 +131,18 @@ RewriteRule ^/ftp-redir/(.*)$ ftp://localhost:1000/$1 [R=302]\n RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302]\n RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n \n+# Apache 2.2 does not understand , so we use RewriteCond.\n+# And as RewriteCond does not allow testing for non-matches, we match\n+# the desired case first (one has abra, two has cadabra), and let it\n+# pass by marking the RewriteRule as [L], \"last rule, do not process\n+# any other matching RewriteRules after this\"), and then have another\n+# RewriteRule that matches all other cases and lets them fail via '[F]',\n+# \"fail the request\".\n+RewriteCond %{HTTP:x-magic-one} =abra\n+RewriteCond %{HTTP:x-magic-two} =cadabra\n+RewriteRule ^/smart_headers/.* - [L]\n+RewriteRule ^/smart_headers/.* - [F]\n+\n \n LoadModule ssl_module modules/mod_ssl.so\n \n-- \n2.8.2-557-gee41d5e\n"},{"id":"285938","messageId":"20160509165139.GA10048@sigill.intra.peff.net","threadId":"42138","inReplyTo":"xmqqwpn3w5c7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-09T16:51:39Z","receivedAt":"2016-05-09T16:51:39Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 09, 2016 at 09:42:32AM -0700, Junio C Hamano wrote:\n\n> > Hopefully your patch to remove the -c ... sanitizing makes it to `master`\n> > soon, then I can submit my next iteration.\n> \n> Or we can just merge that \"do not sanitize\" branch in, and then\n> queue the \"next iteration\" which I'd assume would only be the test\n> addition?\n\nI think we'd also want the change to the test script to make sure that\nit fails with only a single header (Dscho's patch 2).\n\n-Peff\n"},{"id":"285948","messageId":"xmqqzirzuo1t.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"20160509165139.GA10048@sigill.intra.peff.net","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-09T17:41:18Z","receivedAt":"2016-05-09T17:41:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Mon, May 09, 2016 at 09:42:32AM -0700, Junio C Hamano wrote:\n>\n>> > Hopefully your patch to remove the -c ... sanitizing makes it to `master`\n>> > soon, then I can submit my next iteration.\n>> \n>> Or we can just merge that \"do not sanitize\" branch in, and then\n>> queue the \"next iteration\" which I'd assume would only be the test\n>> addition?\n>\n> I think we'd also want the change to the test script to make sure that\n> it fails with only a single header (Dscho's patch 2).\n\nI think so, too.\n"},{"id":"286023","messageId":"C5C883AD-E684-4D2F-811B-A479DE5E41AB@gmail.com","threadId":"42138","inReplyTo":"4a15c4e6c35cfb425da568d87e8b20b984e5325c.1462774709.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2016-05-10T06:37:11Z","receivedAt":"2016-05-10T06:37:11Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 09 May 2016, at 08:18, Johannes Schindelin wrote:\n> \n> Lars Schneider noticed that the configuration introduced to test the extra\n> HTTP headers cannot be used with Apache 2.2 (which is still actively\n> maintained, as pointed out by Junio Hamano).\n> \n> To let the tests pass with Apache 2.2 again, let's substitute the\n> offending and `expr` by using old school RewriteCond\n> statements.\n\nAll Apache 2.2 tests run nicely on Travis CI with Ubuntu and OSX using\nthis patch series:\nhttps://travis-ci.org/larsxschneider/git/builds/128955548\n\nThanks,\nLars\n\n> \n> Signed-off-by: Johannes Schindelin \n> ---\n> t/lib-httpd/apache.conf | 12 ++++++++----\n> 1 file changed, 8 insertions(+), 4 deletions(-)\n> \n> diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n> index b8ed96f..29b34bb 100644\n> --- a/t/lib-httpd/apache.conf\n> +++ b/t/lib-httpd/apache.conf\n> @@ -103,10 +103,6 @@ Alias /auth/dumb/ www/auth/dumb/\n> \tHeader set Set-Cookie name=value\n> \n> \n> -\t\n> -\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n> -\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n> -\t\n> \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n> \tSetEnv GIT_HTTP_EXPORT_ALL\n> \n> @@ -136,6 +132,14 @@ RewriteRule ^/ftp-redir/(.*)$ ftp://localhost:1000/$1 [R=302]\n> RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302]\n> RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n> \n> +# Apache 2.2 does not understand , so we use RewriteCond.\n> +# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n> +# ugly trick to fail *unless* the two headers are present.\n> +RewriteCond %{HTTP:x-magic-one} =abra\n> +RewriteCond %{HTTP:x-magic-two} =cadabra\n> +RewriteRule ^/smart_headers/.* - [L]\n> +RewriteRule ^/smart_headers/.* - [F]\n> +\n> \n> LoadModule ssl_module modules/mod_ssl.so\n> \n> -- \n> 2.8.2.463.g99156ee\n> \n> \n"},{"id":"286025","messageId":"alpine.DEB.2.20.1605100832350.4092@virtualbox","threadId":"42138","inReplyTo":"xmqqwpn3w5c7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-10T06:53:07Z","receivedAt":"2016-05-10T06:53:07Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Mon, 9 May 2016, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > Okay, I already force-pushed my extra-http-header branch and the next\n> > iteration will sport this paragraph.\n> \n> The new explanation is well written and can and should also replace the\n> comment before the implementation in the configuration file to help\n> readers.\n\nI picked your commit and force-pushed my branch; it will be part of the\nnext iteration.\n\n> To be honest, I do not quite understand why you call it \"ugly hack\"\n> at all.\n\nWell, it is convoluted. I would have preferred to say \"if this condition\nis not met or that condition is not met, fail\". Instead I had to say \"If`\nthese two conditions are met, proceed as before. Otherwise, fail.\"\n\nAnd of course its ugliness increased in my mind because I had to go\nthrough so many iterations until it finally worked. Not really\nstraight-forward a solution.\n\n> > Hopefully your patch to remove the -c ... sanitizing makes it to\n> > `master` soon, then I can submit my next iteration.\n> \n> Or we can just merge that \"do not sanitize\" branch in, and then queue\n> the \"next iteration\" which I'd assume would only be the test addition?\n\nI'll prepare something.\n\nCiao,\nDscho\n"},{"id":"286027","messageId":"cover.1462863934.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462774709.git.johannes.schindelin@gmx.de","subject":"[PATCH v8 0/3] Add support for sending additional HTTP headers (part 2)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-10T07:08:15Z","receivedAt":"2016-05-10T07:08:15Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"My use case is an army of build agents that need only limited and\nselective access to otherwise private repositories.\n\nThe first part already made it into `master`, this is the remainder.\n\nThis iteration is based on 'jk/submodule-c-credential' and therefore\nconverted the original config-sanitizing patch into a test-only patch.\nThis iteration also replaces the \"ugly\" comment with the explanation\npreferred by Junio.\n\n\nJohannes Schindelin (3):\n tests: adjust the configuration for Apache 2.2\n t5551: make the test for extra HTTP headers more robust\n submodule: ensure that -c http.extraheader is heeded\n\n t/lib-httpd/apache.conf | 16 ++++++++++++----\n t/t5551-http-fetch-smart.sh | 14 ++++++++++++--\n 2 files changed, 24 insertions(+), 6 deletions(-)\n\nPublished-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v8\n-- \nInterdiff vs v7:\n\n diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n [no longer applies; skipped]\n diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n index 29b34bb..018a83a 100644\n --- a/t/lib-httpd/apache.conf\n +++ b/t/lib-httpd/apache.conf\n @@ -133,8 +133,12 @@ RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302\n RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n \n # Apache 2.2 does not understand , so we use RewriteCond.\n -# And as RewriteCond unfortunately lacks \"not equal\" matching, we use this\n -# ugly trick to fail *unless* the two headers are present.\n +# And as RewriteCond does not allow testing for non-matches, we match\n +# the desired case first (one has abra, two has cadabra), and let it\n +# pass by marking the RewriteRule as [L], \"last rule, do not process\n +# any other matching RewriteRules after this\"), and then have another\n +# RewriteRule that matches all other cases and lets them fail via '[F]',\n +# \"fail the request\".\n RewriteCond %{HTTP:x-magic-one} =abra\n RewriteCond %{HTTP:x-magic-two} =cadabra\n RewriteRule ^/smart_headers/.* - [L]\n\n2.8.2.463.g99156ee\n"},{"id":"286026","messageId":"10a0848e281bf8595c78d4e9f9a770fea9bc6ba8.1462863934.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462863934.git.johannes.schindelin@gmx.de","subject":"[PATCH v8 1/3] tests: adjust the configuration for Apache 2.2","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-10T07:08:27Z","receivedAt":"2016-05-10T07:08:27Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Lars Schneider noticed that the configuration introduced to test the\nextra HTTP headers cannot be used with Apache 2.2 (which is still\nactively maintained, as pointed out by Junio Hamano).\n\nTo let the tests pass with Apache 2.2 again, let's substitute the\noffending and `expr` by using old school RewriteCond\nstatements.\n\nAs RewriteCond does not allow testing for *non*-matches, we simply match\nthe desired case first and let it pass by marking the RewriteRule as\n'[L]' (\"last rule, do not process any other matching RewriteRules after\nthis\"), and then have another RewriteRule that matches all other cases\nand lets them fail via '[F]' (\"fail\").\n\nSigned-off-by: Johannes Schindelin \nSigned-off-by: Junio C Hamano \n---\n t/lib-httpd/apache.conf | 16 ++++++++++++----\n 1 file changed, 12 insertions(+), 4 deletions(-)\n\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex b8ed96f..018a83a 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -103,10 +103,6 @@ Alias /auth/dumb/ www/auth/dumb/\n \tHeader set Set-Cookie name=value\n \n \n-\t\n-\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n-\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n-\t\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n \n@@ -136,6 +132,18 @@ RewriteRule ^/ftp-redir/(.*)$ ftp://localhost:1000/$1 [R=302]\n RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302]\n RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n \n+# Apache 2.2 does not understand , so we use RewriteCond.\n+# And as RewriteCond does not allow testing for non-matches, we match\n+# the desired case first (one has abra, two has cadabra), and let it\n+# pass by marking the RewriteRule as [L], \"last rule, do not process\n+# any other matching RewriteRules after this\"), and then have another\n+# RewriteRule that matches all other cases and lets them fail via '[F]',\n+# \"fail the request\".\n+RewriteCond %{HTTP:x-magic-one} =abra\n+RewriteCond %{HTTP:x-magic-two} =cadabra\n+RewriteRule ^/smart_headers/.* - [L]\n+RewriteRule ^/smart_headers/.* - [F]\n+\n \n LoadModule ssl_module modules/mod_ssl.so\n \n-- \n2.8.2.463.g99156ee\n"},{"id":"286028","messageId":"ff8cbab7e62211b13835e520d402fbd89b90849e.1462863934.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462863934.git.johannes.schindelin@gmx.de","subject":"[PATCH v8 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-10T07:08:38Z","receivedAt":"2016-05-10T07:08:38Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To test that extra HTTP headers are passed correctly, t5551 verifies that\na fetch succeeds when two required headers are passed, and that the fetch\ndoes not succeed when those headers are not passed.\n\nHowever, this test would also succeed if the configuration required only\none header. As Apache's configuration is notoriously tricky (this\ndeveloper frequently requires StackOverflow's help to understand Apache's\ndocumentation), especially when still supporting the 2.2 line, let's just\nreally make sure that the test verifies what we want it to verify.\n\nSigned-off-by: Johannes Schindelin \n---\n t/t5551-http-fetch-smart.sh | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex e44fe72..43b257e 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -283,7 +283,8 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n '\n \n test_expect_success 'custom http headers' '\n-\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\ttest_must_fail git -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n \t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n-- \n2.8.2.463.g99156ee\n"},{"id":"286029","messageId":"1b3d7bde2b92da7722ae46a2f76ccedf532ce35e.1462863934.git.johannes.schindelin@gmx.de","threadId":"42138","inReplyTo":"cover.1462863934.git.johannes.schindelin@gmx.de","subject":"[PATCH v8 3/3] submodule: ensure that -c http.extraheader is heeded","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-10T07:08:56Z","receivedAt":"2016-05-10T07:08:56Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"To support this developer's use case of allowing build agents token-based\naccess to private repositories, we introduced the http.extraheader\nfeature, allowing extra HTTP headers to be sent along with every HTTP\nrequest.\n\nThis patch verifies that we can configure these extra HTTP headers via the\ncommand-line for use with `git submodule update`, too. Example: git -c\nhttp.extraheader=\"Secret: Sauce\" submodule update --init\n\nSigned-off-by: Johannes Schindelin \n---\n t/t5551-http-fetch-smart.sh | 11 ++++++++++-\n 1 file changed, 10 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex 43b257e..2f375eb 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -287,7 +287,16 @@ test_expect_success 'custom http headers' '\n \t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n-\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n+\t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit update-index --add --cacheinfo 160000,$(git rev-parse HEAD),sub &&\n+\tgit config -f .gitmodules submodule.sub.path sub &&\n+\tgit config -f .gitmodules submodule.sub.url \\\n+\t\t\"$HTTPD_URL/smart_headers/repo.git\" &&\n+\tgit submodule init sub &&\n+\ttest_must_fail git submodule update sub &&\n+\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n+\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n+\t\tsubmodule update sub\n '\n \n stop_httpd\n-- \n2.8.2.463.g99156ee\n"},{"id":"286030","messageId":"xmqqbn4es7uz.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"alpine.DEB.2.20.1605100832350.4092@virtualbox","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-10T07:13:56Z","receivedAt":"2016-05-10T07:13:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> To be honest, I do not quite understand why you call it \"ugly hack\"\n>> at all.\n>\n> Well, it is convoluted. I would have preferred to say \"if this condition\n> is not met or that condition is not met, fail\". Instead I had to say \"If`\n> these two conditions are met, proceed as before. Otherwise, fail.\"\n>\n> And of course its ugliness increased in my mind because I had to go\n> through so many iterations until it finally worked. Not really\n> straight-forward a solution.\n\nTo my eyes without your battle scar, it felt more natural to say \"If\nsomebody says he is user $U and gives a password $P, let him in;\nkeep everybody else out\" when configuring a server than saying \"Fail\nanybody whose user name is not $U or who says his password is not\n$P\". I can certainly understand if somebody who has spent a lot of\neffort to make things _fail_ finds the latter more natural.\n"},{"id":"286031","messageId":"xmqq7ff2s7ud.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"C5C883AD-E684-4D2F-811B-A479DE5E41AB@gmail.com","subject":"Re: [PATCH v7 1/3] tests: Adjust the configuration for Apache 2.2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-10T07:14:18Z","receivedAt":"2016-05-10T07:14:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n>> On 09 May 2016, at 08:18, Johannes Schindelin wrote:\n>> \n>> Lars Schneider noticed that the configuration introduced to test the extra\n>> HTTP headers cannot be used with Apache 2.2 (which is still actively\n>> maintained, as pointed out by Junio Hamano).\n>> \n>> To let the tests pass with Apache 2.2 again, let's substitute the\n>> offending and `expr` by using old school RewriteCond\n>> statements.\n>\n> All Apache 2.2 tests run nicely on Travis CI with Ubuntu and OSX using\n> this patch series:\n> https://travis-ci.org/larsxschneider/git/builds/128955548\n>\n> Thanks,\n> Lars\n\nThanks for a report.\n"},{"id":"286083","messageId":"xmqqd1otrfae.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"10a0848e281bf8595c78d4e9f9a770fea9bc6ba8.1462863934.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v8 1/3] tests: adjust the configuration for Apache 2.2","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-10T17:31:05Z","receivedAt":"2016-05-10T17:31:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> Lars Schneider noticed that the configuration introduced to test the\n> extra HTTP headers cannot be used with Apache 2.2 (which is still\n> actively maintained, as pointed out by Junio Hamano).\n>\n> To let the tests pass with Apache 2.2 again, let's substitute the\n> offending and `expr` by using old school RewriteCond\n> statements.\n>\n> As RewriteCond does not allow testing for *non*-matches, we simply match\n> the desired case first and let it pass by marking the RewriteRule as\n> '[L]' (\"last rule, do not process any other matching RewriteRules after\n> this\"), and then have another RewriteRule that matches all other cases\n> and lets them fail via '[F]' (\"fail\").\n>\n> Signed-off-by: Johannes Schindelin \n> Signed-off-by: Junio C Hamano \n> ---\n\nI applied this and compared what was queued from the previous round.\nIt turns out that it is the same, except that I amended it with\n\"Tested-by:\" from Lars', so I'll skip this and nothing is lost ;-)\n\nThanks for being thorough (the above is not a suggestion to omit\nunchanged ones next time--quite the contrary, being able to verify\nby comparing is a good thing).\n\n> t/lib-httpd/apache.conf | 16 ++++++++++++----\n> 1 file changed, 12 insertions(+), 4 deletions(-)\n>\n> diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\n> index b8ed96f..018a83a 100644\n> --- a/t/lib-httpd/apache.conf\n> +++ b/t/lib-httpd/apache.conf\n> @@ -103,10 +103,6 @@ Alias /auth/dumb/ www/auth/dumb/\n> \tHeader set Set-Cookie name=value\n> \n> \n> -\t\n> -\t\tRequire expr %{HTTP:x-magic-one} == 'abra'\n> -\t\tRequire expr %{HTTP:x-magic-two} == 'cadabra'\n> -\t\n> \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n> \tSetEnv GIT_HTTP_EXPORT_ALL\n> \n> @@ -136,6 +132,18 @@ RewriteRule ^/ftp-redir/(.*)$ ftp://localhost:1000/$1 [R=302]\n> RewriteRule ^/loop-redir/x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-x-(.*) /$1 [R=302]\n> RewriteRule ^/loop-redir/(.*)$ /loop-redir/x-$1 [R=302]\n> \n> +# Apache 2.2 does not understand , so we use RewriteCond.\n> +# And as RewriteCond does not allow testing for non-matches, we match\n> +# the desired case first (one has abra, two has cadabra), and let it\n> +# pass by marking the RewriteRule as [L], \"last rule, do not process\n> +# any other matching RewriteRules after this\"), and then have another\n> +# RewriteRule that matches all other cases and lets them fail via '[F]',\n> +# \"fail the request\".\n> +RewriteCond %{HTTP:x-magic-one} =abra\n> +RewriteCond %{HTTP:x-magic-two} =cadabra\n> +RewriteRule ^/smart_headers/.* - [L]\n> +RewriteRule ^/smart_headers/.* - [F]\n> +\n> \n> LoadModule ssl_module modules/mod_ssl.so\n"},{"id":"286085","messageId":"xmqq4ma5rf58.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"ff8cbab7e62211b13835e520d402fbd89b90849e.1462863934.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v8 2/3] t5551: make the test for extra HTTP headers more robust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-10T17:34:11Z","receivedAt":"2016-05-10T17:34:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> To test that extra HTTP headers are passed correctly, t5551 verifies that\n> a fetch succeeds when two required headers are passed, and that the fetch\n> does not succeed when those headers are not passed.\n>\n> However, this test would also succeed if the configuration required only\n> one header. As Apache's configuration is notoriously tricky (this\n> developer frequently requires StackOverflow's help to understand Apache's\n> documentation), especially when still supporting the 2.2 line, let's just\n> really make sure that the test verifies what we want it to verify.\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n\nMatches the previous one I queued with Reviewed-by: from Peff; good.\n\n> t/t5551-http-fetch-smart.sh | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n>\n> diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n> index e44fe72..43b257e 100755\n> --- a/t/t5551-http-fetch-smart.sh\n> +++ b/t/t5551-http-fetch-smart.sh\n> @@ -283,7 +283,8 @@ test_expect_success EXPENSIVE 'http can handle enormous ref negotiation' '\n> '\n> \n> test_expect_success 'custom http headers' '\n> -\ttest_must_fail git fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> +\ttest_must_fail git -c http.extraheader=\"x-magic-two: cadabra\" \\\n> +\t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> \t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n"},{"id":"286086","messageId":"xmqqzirxq0ec.fsf@gitster.mtv.corp.google.com","threadId":"42138","inReplyTo":"1b3d7bde2b92da7722ae46a2f76ccedf532ce35e.1462863934.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH v8 3/3] submodule: ensure that -c http.extraheader is heeded","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-10T17:38:03Z","receivedAt":"2016-05-10T17:38:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> To support this developer's use case of allowing build agents token-based\n> access to private repositories, we introduced the http.extraheader\n> feature, allowing extra HTTP headers to be sent along with every HTTP\n> request.\n>\n> This patch verifies that we can configure these extra HTTP headers via the\n> command-line for use with `git submodule update`, too. Example: git -c\n> http.extraheader=\"Secret: Sauce\" submodule update --init\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n\nApplying this directly on top of the other two fails, and when\nmerged with jk/submodule-c-credential, the test passes.\n\nWhich is exactly what we expect to see. Nice.\n\nI'll merge jk/submodule-c-credential into js/http-custom-headers\nthat already has 1 & 2, and then apply this. An alternative would\nbe to hold this and wait until both jk/submodule-c-credential and\njs/http-custom-headers with 1 & 2 graduates and then apply this,\nwhich is a lot inferior option.\n\nThanks.\n\n> t/t5551-http-fetch-smart.sh | 11 ++++++++++-\n> 1 file changed, 10 insertions(+), 1 deletion(-)\n>\n> diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\n> index 43b257e..2f375eb 100755\n> --- a/t/t5551-http-fetch-smart.sh\n> +++ b/t/t5551-http-fetch-smart.sh\n> @@ -287,7 +287,16 @@ test_expect_success 'custom http headers' '\n> \t\tfetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> \tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> \t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> -\t fetch \"$HTTPD_URL/smart_headers/repo.git\"\n> +\t fetch \"$HTTPD_URL/smart_headers/repo.git\" &&\n> +\tgit update-index --add --cacheinfo 160000,$(git rev-parse HEAD),sub &&\n> +\tgit config -f .gitmodules submodule.sub.path sub &&\n> +\tgit config -f .gitmodules submodule.sub.url \\\n> +\t\t\"$HTTPD_URL/smart_headers/repo.git\" &&\n> +\tgit submodule init sub &&\n> +\ttest_must_fail git submodule update sub &&\n> +\tgit -c http.extraheader=\"x-magic-one: abra\" \\\n> +\t -c http.extraheader=\"x-magic-two: cadabra\" \\\n> +\t\tsubmodule update sub\n> '\n> \n> stop_httpd\n"},{"id":"286154","messageId":"alpine.DEB.2.20.1605110856100.4092@virtualbox","threadId":"42138","inReplyTo":"xmqqzirxq0ec.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v8 3/3] submodule: ensure that -c http.extraheader is heeded","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-11T06:57:02Z","receivedAt":"2016-05-11T06:57:02Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Tue, 10 May 2016, Junio C Hamano wrote:\n\n> Johannes Schindelin writes:\n> \n> > To support this developer's use case of allowing build agents token-based\n> > access to private repositories, we introduced the http.extraheader\n> > feature, allowing extra HTTP headers to be sent along with every HTTP\n> > request.\n> >\n> > This patch verifies that we can configure these extra HTTP headers via the\n> > command-line for use with `git submodule update`, too. Example: git -c\n> > http.extraheader=\"Secret: Sauce\" submodule update --init\n> >\n> > Signed-off-by: Johannes Schindelin \n> > ---\n> \n> Applying this directly on top of the other two fails, and when\n> merged with jk/submodule-c-credential, the test passes.\n> \n> Which is exactly what we expect to see. Nice.\n\nGood.\n\n> I'll merge jk/submodule-c-credential into js/http-custom-headers\n> that already has 1 & 2, and then apply this. An alternative would\n> be to hold this and wait until both jk/submodule-c-credential and\n> js/http-custom-headers with 1 & 2 graduates and then apply this,\n> which is a lot inferior option.\n\nAs long as it ends up in 2.9.0, I am happy with whatever path these\npatches take ;-)\n\nThanks,\nDscho\n"},{"id":"286275","messageId":"db56fae5-799a-29af-3a0f-a7b5c671063a@web.de","threadId":"42138","inReplyTo":"ff8cbab7e62211b13835e520d402fbd89b90849e.1462863934.git.johannes.schindelin@gmx.de","subject":"t5551 hangs ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-11T17:13:56Z","receivedAt":"2016-05-11T17:13:56Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 10.05.16 09:08, Johannes Schindelin wrote:\n- I'm not sure, if this is the right thread to report on -\n\nIt seems as if t5551 is hanging ?\nThis is the last line from the log:\nok 25 - large fetch-pack requests can be split across POSTs\n\nI have 7 such processes running:\n/trash directory.t5551-http-fetch-smart/httpd -f\n/Users/tb/projects/git/git.pu/t/lib-httpd/apache.conf -DDarwin -c Listen\n127.0.0.1:5551 -k start\n\nThis happens both under Mac OS X and Debian.\n\nDoes anybody have the same hanging ?\n"},{"id":"286277","messageId":"20160511173130.GA29731@sigill.intra.peff.net","threadId":"42138","inReplyTo":"db56fae5-799a-29af-3a0f-a7b5c671063a@web.de","subject":"Re: t5551 hangs ?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-11T17:31:31Z","receivedAt":"2016-05-11T17:31:31Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 11, 2016 at 07:13:56PM +0200, Torsten Bögershausen wrote:\n\n> On 10.05.16 09:08, Johannes Schindelin wrote:\n> - I'm not sure, if this is the right thread to report on -\n> \n> It seems as if t5551 is hanging ?\n> This is the last line from the log:\n> ok 25 - large fetch-pack requests can be split across POSTs\n\nAre you running the tests with \"--long\" or GIT_TEST_LONG in the\nenvironment? The next line should show it skipping test 26 unless one of\nthose is set.\n\nIf you are, can you confirm that it's actually hanging, and not just\nslow? On my system, test 26 takes about a minute to run (which is why we\ndon't do it by default).\n\n> I have 7 such processes running:\n> /trash directory.t5551-http-fetch-smart/httpd -f\n> /Users/tb/projects/git/git.pu/t/lib-httpd/apache.conf -DDarwin -c Listen\n> 127.0.0.1:5551 -k start\n\nThat's normal while the test is running; apache pre-forks a bunch of\nworker threads.\n\n-Peff\n"},{"id":"286283","messageId":"5f285a5f-f66a-ed35-ecf9-0fece19ee5ca@web.de","threadId":"42138","inReplyTo":"20160511173130.GA29731@sigill.intra.peff.net","subject":"Re: t5551 hangs ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-11T20:03:45Z","receivedAt":"2016-05-11T20:03:45Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 11.05.16 19:31, Jeff King wrote:\n> On Wed, May 11, 2016 at 07:13:56PM +0200, Torsten Bögershausen wrote:\n> \n>> On 10.05.16 09:08, Johannes Schindelin wrote:\n>> - I'm not sure, if this is the right thread to report on -\n>>\n>> It seems as if t5551 is hanging ?\n>> This is the last line from the log:\n>> ok 25 - large fetch-pack requests can be split across POSTs\n> \n> Are you running the tests with \"--long\" or GIT_TEST_LONG in the\n> environment? The next line should show it skipping test 26 unless one of\n> those is set.\nYes\n> \n> If you are, can you confirm that it's actually hanging, and not just\n> slow? On my system, test 26 takes about a minute to run (which is why we\n> don't do it by default).\nNearly sure. After 10 minutes, the test was still running.\n\nYesterday another machine was running even longer.\n\nAny tips, how to debug, are welcome.\n"},{"id":"286304","messageId":"20160512031628.GA2741@sigill.intra.peff.net","threadId":"42138","inReplyTo":"5f285a5f-f66a-ed35-ecf9-0fece19ee5ca@web.de","subject":"Re: t5551 hangs ?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-12T03:16:28Z","receivedAt":"2016-05-12T03:16:28Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 11, 2016 at 10:03:45PM +0200, Torsten Bögershausen wrote:\n\n> > If you are, can you confirm that it's actually hanging, and not just\n> > slow? On my system, test 26 takes about a minute to run (which is why we\n> > don't do it by default).\n> Nearly sure. After 10 minutes, the test was still running.\n> \n> Yesterday another machine was running even longer.\n> \n> Any tips, how to debug, are welcome.\n\nTry running with \"-x\" to see what the test is doing. It will probably be\nin:\n\n + git -C too-many-refs fetch -q --tags\n\nafter a while. Check \"ps\" to see if you have a fetch-pack sub-process\nrunning. It should be writing \"have\" lines and reading lots of ACK\nlines, which you can check via strace.\n\nIf it's blocked on read() or write(), then it's probably some kind of\nI/O deadlock.\n\n-Peff\n"},{"id":"286315","messageId":"c0f2ae7b-0a67-af75-86ca-258a58b053bd@web.de","threadId":"42138","inReplyTo":"20160512031628.GA2741@sigill.intra.peff.net","subject":"Re: t5551 hangs ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-12T06:21:10Z","receivedAt":"2016-05-12T06:21:10Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 12.05.16 05:16, Jeff King wrote:\n> On Wed, May 11, 2016 at 10:03:45PM +0200, Torsten Bögershausen wrote:\n> \n>>> If you are, can you confirm that it's actually hanging, and not just\n>>> slow? On my system, test 26 takes about a minute to run (which is why we\n>>> don't do it by default).\n>> Nearly sure. After 10 minutes, the test was still running.\n>>\n>> Yesterday another machine was running even longer.\n>>\n>> Any tips, how to debug, are welcome.\n> \n> Try running with \"-x\" to see what the test is doing. It will probably be\n> in:\n> \n> + git -C too-many-refs fetch -q --tags\n> \n> after a while. Check \"ps\" to see if you have a fetch-pack sub-process\n> running. It should be writing \"have\" lines and reading lots of ACK\n> lines, which you can check via strace.\n> \n> If it's blocked on read() or write(), then it's probably some kind of\n> I/O deadlock.\n> \n> -Peff\n\nThis is the last log that I see:\n---------------------------------------------------------------------\npack_report: getpagesize() = 4096\npack_report: core.packedGitWindowSize = 1073741824\npack_report: core.packedGitLimit = 8589934592\npack_report: pack_used_ctr = 96001\npack_report: pack_mmap_calls = 48002\npack_report: pack_open_windows = 2 / 2\npack_report: pack_mapped = 6605494 / 6605494\n---------------------------------------------------------------------\n\n+++ perl -e 'print \"bla\" x 30'\n+++ command /usr/bin/perl -e 'print \"bla\" x 30'\n+++ /usr/bin/perl -e 'print \"bla\" x 30'\n++ tag=blablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla\n++ sed -e 's|^:\\([^ ]*\\) \\(.*\\)$|\\2 refs/tags/blablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla-\\1|'\n++ git -C too-many-refs fetch -q --tags\n\nAnd this may be the processes :\n(Not sure, probaly need to reboot & clean ?)\n/bin/sh ./t5551-http-fetch-smart.sh -x\n73459 ttys010 0:21.45 /Users/tb/projects/git/git.pu/git -C too-many-refs fetch -q --tags\n73460 ttys010 0:00.40 git-remote-http origin http://127.0.0.1:5551/smart/repo.git\n\n ps | grep fetch\n73540 ttys006 0:00.00 grep fetch\n73025 ttys010 0:00.14 /bin/sh ./t5551-http-fetch-smart.sh -x\n73459 ttys010 3:40.70 /Users/tb/projects/git/git.pu/git -C too-many-refs fetch -q --tags\n\n\nBeside that, reverting the last 2 commits on 5551 doesn't seem to help:\n Revert \"t5551: make the test for extra HTTP headers more robust\"\n Revert \"submodule: ensure that -c http.extraheader is heeded\"\n"},{"id":"286316","messageId":"20160512064038.GA32271@sigill.intra.peff.net","threadId":"42138","inReplyTo":"c0f2ae7b-0a67-af75-86ca-258a58b053bd@web.de","subject":"Re: t5551 hangs ?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-12T06:40:39Z","receivedAt":"2016-05-12T06:40:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 12, 2016 at 08:21:10AM +0200, Torsten Bögershausen wrote:\n\n> This is the last log that I see:\n> [...]\n> ++ git -C too-many-refs fetch -q --tags\n\nNot surprising.\n\n> And this may be the processes :\n> (Not sure, probaly need to reboot & clean ?)\n\nIf you're killing the hung test with \"^C\", you shouldn't need to; that\ntries to clean up any processes and shut down apache.\n\n> /bin/sh ./t5551-http-fetch-smart.sh -x\n> 73459 ttys010 0:21.45 /Users/tb/projects/git/git.pu/git -C too-many-refs fetch -q --tags\n> 73460 ttys010 0:00.40 git-remote-http origin http://127.0.0.1:5551/smart/repo.git\n> \n> ps | grep fetch\n> 73540 ttys006 0:00.00 grep fetch\n> 73025 ttys010 0:00.14 /bin/sh ./t5551-http-fetch-smart.sh -x\n> 73459 ttys010 3:40.70 /Users/tb/projects/git/git.pu/git -C too-many-refs fetch -q --tags\n\nI'm surprised not to see fetch-pack in that list. And to see so much CPU\ngoing to fetch itself. But perhaps you are simply at a different stage\nin the test. 3:40 of CPU time is a lot (the whole thing runs in under a\nminute on my machine).\n\nHmm. Switching to \"pu\" seems to make things slow on my machine, too, and\nthe time all goes to fetch. So perhaps there is some recent regression\nthere. It should be bisectable.\n\n-Peff\n"},{"id":"286318","messageId":"20160512072939.GA10922@sigill.intra.peff.net","threadId":"42138","inReplyTo":"20160512064038.GA32271@sigill.intra.peff.net","subject":"Re: t5551 hangs ?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-12T07:29:39Z","receivedAt":"2016-05-12T07:29:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, May 12, 2016 at 02:40:39AM -0400, Jeff King wrote:\n\n> Hmm. Switching to \"pu\" seems to make things slow on my machine, too, and\n> the time all goes to fetch. So perhaps there is some recent regression\n> there. It should be bisectable.\n\nIt's 66d33af21bd1e398973414435af43d06f2e2099c. I don't think it's\nhanging, but it is _really_ slow. I'll reply to that patch separately\nwith a report.\n\n-Peff\n"},{"id":"286658","messageId":"alpine.DEB.2.20.1605161531090.3303@virtualbox","threadId":"42138","inReplyTo":"34DE0A16-F0B2-4379-8E02-5235D34FDD76@gmail.com","subject":"mail-patch-series.sh, was Re: [PATCH v7 0/3] Add support for sending additional HTTP headers (part 2)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-05-16T13:35:33Z","receivedAt":"2016-05-16T13:35:33Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Lars,\n\nOn Tue, 10 May 2016, Lars Schneider wrote:\n\n> > On 09 May 2016, at 08:18, Johannes Schindelin\n> > wrote:\n> > \n> > [...]\n> > \n> > Published-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v7\n> > Interdiff vs v6:\n> \n> Published-As and Interdiff are really neat. I assume you have some kind\n> of script to create all this? If yes, are you willing to share it?\n\nThanks for prodding me. I had planned to clean up my script and make it\npublic for a long time already, as I think that submitting code to the Git\nproject is quite a little bit too tedious, even for old timers like me, in\nparticular when compared to the ease of opening Pull Requests a la GitHub.\n\nSo now I cleaned up my script a bit and published it here:\n\n\thttps://github.com/dscho/mail-patch-series\n\nIronically, I welcome Pull Requests for this script ;-)\n\nCiao,\nDscho\n"}]}