git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3] http: support sending custom HTTP headers

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Apr 27, 2016, 06:29 UTC
Message-ID
<d087647975742a803735c838d0b970d962d75048.1461738362.git.johannes.schindelin@gmx.de>
In-Reply-To
<921e007f445476b27325c12a9e92fdd169a073b7.1461685158.git.johannes.schindelin@gmx.de>
We introduce a way to send custom HTTP headers with all requests.

This allows us, for example, to send an extra token from build agents for temporary access to private repositories. (This is the use case that triggered this patch.)

This feature can be used like this:
	git -c http.extraheader='Secret: sssh!' fetch $URL $REF

Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only a single list, overriding any previous call. This means we have to collect _all_ of the headers we want to use into a single list, and feed it to cURL in one shot. Since we already unconditionally set a "pragma" header when initializing the curl handles, we can add our new headers to that list.

For callers which override the default header list (like probe_rpc), we provide `http_copy_default_headers()` so they can do the same trick.

Big thanks to Jeff King and Junio Hamano for their outstanding help and patient reviews.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Changes since v2:
	- now using Peff's much improved wording in the commit message
	- skipped the last two lines from the documentation of the feature
	- fixed function declaration/definition by using `(void)`
	- we now allow resetting extra headers with an empty value
	- added a test
Published-As: https://github.com/dscho/git/releases/tag/extra-http-headers-v3
 Documentation/config.txt   |  6 ++++++
 http-push.c                | 10 +++++-----
 http.c                     | 35 ++++++++++++++++++++++++++++++++---
 http.h                     |  1 +
 remote-curl.c              |  4 ++--
 t/t5550-http-fetch-dumb.sh |  8 ++++++++
 6 files changed, 54 insertions(+), 10 deletions(-)
Interdiff vs v2:
 diff --git a/Documentation/config.txt b/Documentation/config.txt
 index 37b9af7..c7bbe98 100644
 --- a/Documentation/config.txt
 +++ b/Documentation/config.txt
 @@ -1657,9 +1657,9 @@ http.emptyAuth::
  
  http.extraHeader::
  	Pass an additional HTTP header when communicating with a server.  If
 -	more than one such entry exists, all of them are added as extra headers.
 -	This feature is useful e.g. to increase security, or to allow
 -	time-limited access based on expiring tokens.
 +	more than one such entry exists, all of them are added as extra
 +	headers.  To allow overriding the settings inherited from the system
 +	config, an empty value will reset the extra headers to the empty list.
  
  http.cookieFile::
  	File containing previously stored cookie lines which should be used
 diff --git a/http.c b/http.c
 index 3d662bb..985b995 100644
 --- a/http.c
 +++ b/http.c
 @@ -325,8 +325,15 @@ static int http_options(const char *var, const char *value, void *cb)
  	}
  
  	if (!strcmp("http.extraheader", var)) {
 -		extra_http_headers =
 -			curl_slist_append(extra_http_headers, value);
 +		if (!value) {
 +			return config_error_nonbool(var);
 +		} else if (!*value) {
 +			curl_slist_free_all(extra_http_headers);
 +			extra_http_headers = NULL;
 +		} else {
 +			extra_http_headers =
 +				curl_slist_append(extra_http_headers, value);
 +		}
  		return 0;
  	}
  
 @@ -1175,7 +1182,7 @@ int run_one_slot(struct active_request_slot *slot,
  	return handle_curl_result(results);
  }
  
 -struct curl_slist *http_copy_default_headers()
 +struct curl_slist *http_copy_default_headers(void)
  {
  	struct curl_slist *headers = NULL, *h;
  
 diff --git a/http.h b/http.h
 index 5f13695..36f558b 100644
 --- a/http.h
 +++ b/http.h
 @@ -106,7 +106,7 @@ extern void step_active_slots(void);
  extern void http_init(struct remote *remote, const char *url,
  		      int proactive_auth);
  extern void http_cleanup(void);
 -extern struct curl_slist *http_copy_default_headers();
 +extern struct curl_slist *http_copy_default_headers(void);
  
  extern long int git_curl_ipresolve;
  extern int active_requests;
 diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
 index 48e2ab6..96425b1 100755
 --- a/t/t5550-http-fetch-dumb.sh
 +++ b/t/t5550-http-fetch-dumb.sh
 @@ -267,5 +267,13 @@ test_expect_success 'git client does not send an empty Accept-Language' '
  	! grep "^Accept-Language:" stderr
  '
  
 +test_expect_success 'extra HTTP headers are sent' '
 +	GIT_CURL_VERBOSE=1 \
 +	git -c http.extraheader="Hello: World" \
 +		ls-remote "$HTTPD_URL/dumb/repo.git" >out 2>err &&
 +	test_i18ngrep "Hello: World" err >hello.txt &&
 +	test_line_count = 2 hello.txt
 +'
 +
  stop_httpd
  test_done
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 42d2b50..c7bbe98 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1655,6 +1655,12 @@ http.emptyAuth::
 	a username in the URL, as libcurl normally requires a username for
 	authentication.
 
+http.extraHeader::
+	Pass an additional HTTP header when communicating with a server.  If
+	more than one such entry exists, all of them are added as extra
+	headers.  To allow overriding the settings inherited from the system
+	config, an empty value will reset the extra headers to the empty list.
+
 http.cookieFile::
 	File containing previously stored cookie lines which should be used
 	in the Git http session, if they match the server. The file format
diff --git a/http-push.c b/http-push.c
index bd60668..ae2b7f1 100644
--- a/http-push.c
+++ b/http-push.c
@@ -211,7 +211,7 @@ static void curl_setup_http(CURL *curl, const char *url,
 static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options)
 {
 	struct strbuf buf = STRBUF_INIT;
-	struct curl_slist *dav_headers = NULL;
+	struct curl_slist *dav_headers = http_copy_default_headers();
 
 	if (options & DAV_HEADER_IF) {
 		strbuf_addf(&buf, "If: (<%s>)", lock->token);
@@ -417,7 +417,7 @@ static void start_put(struct transfer_request *request)
 static void start_move(struct transfer_request *request)
 {
 	struct active_request_slot *slot;
-	struct curl_slist *dav_headers = NULL;
+	struct curl_slist *dav_headers = http_copy_default_headers();
 
 	slot = get_active_slot();
 	slot->callback_func = process_response;
@@ -845,7 +845,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout)
 	char *ep;
 	char timeout_header[25];
 	struct remote_lock *lock = NULL;
-	struct curl_slist *dav_headers = NULL;
+	struct curl_slist *dav_headers = http_copy_default_headers();
 	struct xml_ctx ctx;
 	char *escaped;
 
@@ -1126,7 +1126,7 @@ static void remote_ls(const char *path, int flags,
 	struct slot_results results;
 	struct strbuf in_buffer = STRBUF_INIT;
 	struct buffer out_buffer = { STRBUF_INIT, 0 };
-	struct curl_slist *dav_headers = NULL;
+	struct curl_slist *dav_headers = http_copy_default_headers();
 	struct xml_ctx ctx;
 	struct remote_ls_ctx ls;
 
@@ -1204,7 +1204,7 @@ static int locking_available(void)
 	struct slot_results results;
 	struct strbuf in_buffer = STRBUF_INIT;
 	struct buffer out_buffer = { STRBUF_INIT, 0 };
-	struct curl_slist *dav_headers = NULL;
+	struct curl_slist *dav_headers = http_copy_default_headers();
 	struct xml_ctx ctx;
 	int lock_flags = 0;
 	char *escaped;
diff --git a/http.c b/http.c
index 4304b80..985b995 100644
--- a/http.c
+++ b/http.c
@@ -114,6 +114,7 @@ static unsigned long http_auth_methods = CURLAUTH_ANY;
 
 static struct curl_slist *pragma_header;
 static struct curl_slist *no_pragma_header;
+static struct curl_slist *extra_http_headers;
 
 static struct active_request_slot *active_queue_head;
 
@@ -323,6 +324,19 @@ static int http_options(const char *var, const char *value, void *cb)
 #endif
 	}
 
+	if (!strcmp("http.extraheader", var)) {
+		if (!value) {
+			return config_error_nonbool(var);
+		} else if (!*value) {
+			curl_slist_free_all(extra_http_headers);
+			extra_http_headers = NULL;
+		} else {
+			extra_http_headers =
+				curl_slist_append(extra_http_headers, value);
+		}
+		return 0;
+	}
+
 	/* Fall back on the default ones */
 	return git_default_config(var, value, cb);
 }
@@ -678,8 +692,10 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	if (remote)
 		var_override(&http_proxy_authmethod, remote->http_proxy_authmethod);
 
-	pragma_header = curl_slist_append(pragma_header, "Pragma: no-cache");
-	no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
+	pragma_header = curl_slist_append(http_copy_default_headers(),
+		"Pragma: no-cache");
+	no_pragma_header = curl_slist_append(http_copy_default_headers(),
+		"Pragma:");
 
 #ifdef USE_CURL_MULTI
 	{
@@ -765,6 +781,9 @@ void http_cleanup(void)
 #endif
 	curl_global_cleanup();
 
+	curl_slist_free_all(extra_http_headers);
+	extra_http_headers = NULL;
+
 	curl_slist_free_all(pragma_header);
 	pragma_header = NULL;
 
@@ -1163,6 +1182,16 @@ int run_one_slot(struct active_request_slot *slot,
 	return handle_curl_result(results);
 }
 
+struct curl_slist *http_copy_default_headers(void)
+{
+	struct curl_slist *headers = NULL, *h;
+
+	for (h = extra_http_headers; h; h = h->next)
+		headers = curl_slist_append(headers, h->data);
+
+	return headers;
+}
+
 static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf)
 {
 	char *ptr;
@@ -1380,7 +1409,7 @@ static int http_request(const char *url,
 {
 	struct active_request_slot *slot;
 	struct slot_results results;
-	struct curl_slist *headers = NULL;
+	struct curl_slist *headers = http_copy_default_headers();
 	struct strbuf buf = STRBUF_INIT;
 	const char *accept_language;
 	int ret;
diff --git a/http.h b/http.h
index 4ef4bbd..36f558b 100644
--- a/http.h
+++ b/http.h
@@ -106,6 +106,7 @@ extern void step_active_slots(void);
 extern void http_init(struct remote *remote, const char *url,
 		      int proactive_auth);
 extern void http_cleanup(void);
+extern struct curl_slist *http_copy_default_headers(void);
 
 extern long int git_curl_ipresolve;
 extern int active_requests;
diff --git a/remote-curl.c b/remote-curl.c
index 15e48e2..672b382 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -474,7 +474,7 @@ static int run_slot(struct active_request_slot *slot,
 static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)
 {
 	struct active_request_slot *slot;
-	struct curl_slist *headers = NULL;
+	struct curl_slist *headers = http_copy_default_headers();
 	struct strbuf buf = STRBUF_INIT;
 	int err;
 
@@ -503,7 +503,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)
 static int post_rpc(struct rpc_state *rpc)
 {
 	struct active_request_slot *slot;
-	struct curl_slist *headers = NULL;
+	struct curl_slist *headers = http_copy_default_headers();
 	int use_gzip = rpc->gzip_request;
 	char *gzip_body = NULL;
 	size_t gzip_size = 0;
diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
index 48e2ab6..96425b1 100755
--- a/t/t5550-http-fetch-dumb.sh
+++ b/t/t5550-http-fetch-dumb.sh
@@ -267,5 +267,13 @@ test_expect_success 'git client does not send an empty Accept-Language' '
 	! grep "^Accept-Language:" stderr
 '
 
+test_expect_success 'extra HTTP headers are sent' '
+	GIT_CURL_VERBOSE=1 \
+	git -c http.extraheader="Hello: World" \
+		ls-remote "$HTTPD_URL/dumb/repo.git" >out 2>err &&
+	test_i18ngrep "Hello: World" err >hello.txt &&
+	test_line_count = 2 hello.txt
+'
+
 stop_httpd
 test_done

-- 
2.8.1.306.gff998f2
Previous: Junio C HamanoNext: Johannes Schindelin
Message 21 of 119 in “http: Support sending custom HTTP headers”
  1. http: Support sending custom HTTP headersJohannes Schindelin, Apr 25, 2016
  2. Shawn PearceApr 25, 2016
  3. Jeff KingApr 25, 2016
  4. Johannes SchindelinApr 26, 2016
  5. Jeff KingApr 26, 2016
  6. Junio C HamanoApr 25, 2016
  7. Johannes SchindelinApr 26, 2016
  8. Junio C HamanoApr 26, 2016
  9. Jeff KingApr 26, 2016
  10. Johannes SchindelinApr 27, 2016
  11. Jeff KingApr 27, 2016
  12. Johannes SchindelinApr 27, 2016
  13. http: support sending custom HTTP headersJohannes Schindelin, Apr 26, 2016
  14. Junio C HamanoApr 26, 2016
  15. Jeff KingApr 26, 2016
  16. Junio C HamanoApr 26, 2016
  17. Jeff KingApr 26, 2016
  18. Johannes SchindelinApr 27, 2016
  19. Johannes SchindelinApr 27, 2016
  20. Junio C HamanoApr 26, 2016
  21. http: support sending custom HTTP headersJohannes Schindelin, Apr 27, 2016
  22. http: support sending custom HTTP headersJohannes Schindelin, Apr 27, 2016
  23. Jeff KingApr 27, 2016
  24. Junio C HamanoApr 27, 2016
  25. 0/2 Add support for sending additional HTTP headersJohannes Schindelin, Apr 28, 2016
  26. 1/2 http: support sending custom HTTP headersJohannes Schindelin, Apr 28, 2016
  27. 2/2 submodule: pass on http.extraheader config settingsJohannes Schindelin, Apr 28, 2016
  28. Jeff KingApr 28, 2016
  29. Johannes SchindelinApr 28, 2016
  30. Jeff KingApr 28, 2016
  31. Jacob KellerApr 28, 2016
  32. Jeff KingApr 28, 2016
  33. Stefan BellerApr 28, 2016
  34. Jeff KingApr 28, 2016
  35. Junio C HamanoApr 28, 2016
  36. Jeff KingApr 28, 2016
  37. Junio C HamanoApr 28, 2016
  38. Stefan BellerApr 28, 2016
  39. Junio C HamanoApr 28, 2016
  40. Junio C HamanoApr 28, 2016
  41. Stefan BellerApr 28, 2016
  42. Junio C HamanoApr 28, 2016
  43. Jeff KingApr 28, 2016
  44. Stefan BellerApr 28, 2016
  45. Junio C HamanoApr 28, 2016
  46. Jeff KingApr 29, 2016
  47. Jeff KingApr 28, 2016
  48. Stefan BellerApr 28, 2016
  49. Jeff KingApr 28, 2016
  50. Johannes SchindelinApr 29, 2016
  51. Jeff KingApr 29, 2016
  52. Jeff KingApr 28, 2016
  53. Junio C HamanoApr 28, 2016
  54. Johannes SchindelinApr 29, 2016
  55. Johannes SchindelinApr 29, 2016
  56. Jeff KingApr 29, 2016
  57. Johannes SchindelinApr 29, 2016
  58. 0/2 Add support for sending additional HTTP headersJohannes Schindelin, May 4, 2016
  59. 1/2 http: support sending custom HTTP headersJohannes Schindelin, May 4, 2016
  60. Lars SchneiderMay 5, 2016
  61. Junio C HamanoMay 5, 2016
  62. Jeff KingMay 5, 2016
  63. 2/2 submodule: pass on http.extraheader config settingsJohannes Schindelin, May 4, 2016
  64. Jeff KingMay 4, 2016
  65. Junio C HamanoMay 4, 2016
  66. Johannes SchindelinMay 4, 2016
  67. Junio C HamanoMay 4, 2016
  68. Jeff KingMay 4, 2016
  69. submodule: stop sanitizing config optionsJeff King, May 4, 2016
  70. Junio C HamanoMay 4, 2016
  71. Johannes SchindelinMay 4, 2016
  72. Stefan BellerMay 4, 2016
  73. Jeff KingMay 4, 2016
  74. Junio C HamanoMay 4, 2016
  75. Jeff KingMay 4, 2016
  76. Stefan BellerMay 4, 2016
  77. Jeff KingMay 5, 2016
  78. Junio C HamanoMay 5, 2016
  79. Jeff KingMay 5, 2016
  80. Junio C HamanoMay 5, 2016
  81. Stefan BellerMay 6, 2016
  82. Jeff KingMay 6, 2016
  83. Junio C HamanoMay 6, 2016
  84. 0/3 Add support for sending additional HTTP headers (part 2)Johannes Schindelin, May 9, 2016
  85. 1/3 tests: Adjust the configuration for Apache 2.2Johannes Schindelin, May 9, 2016
  86. Jeff KingMay 9, 2016
  87. Johannes SchindelinMay 9, 2016
  88. Jeff KingMay 9, 2016
  89. Johannes SchindelinMay 9, 2016
  90. Junio C HamanoMay 9, 2016
  91. Jeff KingMay 9, 2016
  92. Junio C HamanoMay 9, 2016
  93. Johannes SchindelinMay 10, 2016
  94. Junio C HamanoMay 10, 2016
  95. Junio C HamanoMay 9, 2016
  96. Lars SchneiderMay 10, 2016
  97. Junio C HamanoMay 10, 2016
  98. 2/3 t5551: make the test for extra HTTP headers more robustJohannes Schindelin, May 9, 2016
  99. Lars SchneiderMay 9, 2016
  100. Jeff KingMay 9, 2016
  101. Johannes SchindelinMay 9, 2016
  102. Jeff KingMay 9, 2016
  103. 3/3 submodule: pass on http.extraheader config settingsJohannes Schindelin, May 9, 2016
  104. 0/3 Add support for sending additional HTTP headers (part 2)Johannes Schindelin, May 10, 2016
  105. 1/3 tests: adjust the configuration for Apache 2.2Johannes Schindelin, May 10, 2016
  106. Junio C HamanoMay 10, 2016
  107. 2/3 t5551: make the test for extra HTTP headers more robustJohannes Schindelin, May 10, 2016
  108. Junio C HamanoMay 10, 2016
  109. t5551 hangs ?Torsten Bögershausen, May 11, 2016
  110. Jeff KingMay 11, 2016
  111. Torsten BögershausenMay 11, 2016
  112. Jeff KingMay 12, 2016
  113. Torsten BögershausenMay 12, 2016
  114. Jeff KingMay 12, 2016
  115. Jeff KingMay 12, 2016
  116. 3/3 submodule: ensure that -c http.extraheader is heededJohannes Schindelin, May 10, 2016
  117. Junio C HamanoMay 10, 2016
  118. Johannes SchindelinMay 11, 2016
  119. mail-patch-series.sh, was Re: [PATCH v7 0/3] Add support for sending additional HTTP headers (part 2)Johannes Schindelin, May 16, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.