threads / patch / 10985

patchAllow HTTP proxy to be overridden in config

Subject: [PATCH] Allow HTTP proxy to be overridden in config

## tl;dr

9 messages between Nov 23, 2007 and Dec 4, 2007. Diffs are folded; open one to read it.

replies: 8people: 4as markdown or json

Sam Vilain· Nov 23, 2007, 00:07 UTC · lore
The http_proxy / HTTPS_PROXY variables used by curl to control
proxying may not be suitable for git.  Allow the user to override them
in the configuration file.
---
  In particular, privoxy will block directories called /ad/ ... d'oh!
 Documentation/config.txt |    4 ++++
 http.c                   |   11 +++++++++++
 2 files changed, 15 insertions(+), 0 deletions(-)
Show changes to 2 files +15 −0

Documentation/config.txt, http.c

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 7ee97df..859a7f3 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -515,6 +515,10 @@ specified as 'gitcvs.<access_method>.<varname>' (where 'access_method'
 is one of "ext" and "pserver") to make them apply only for the given
 access method.
 
+http.proxy::
+	Override the HTTP proxy, normally configured using the 'http_proxy'
+	environment variable (see gitlink:curl[1]).
+
 http.sslVerify::
 	Whether to verify the SSL certificate when fetching or pushing
 	over HTTPS. Can be overridden by the 'GIT_SSL_NO_VERIFY' environment
diff --git a/http.c b/http.c
index c6fb8ac..8f60d89 100644
--- a/http.c
+++ b/http.c
@@ -24,6 +24,7 @@ char *ssl_cainfo = NULL;
 long curl_low_speed_limit = -1;
 long curl_low_speed_time = -1;
 int curl_ftp_no_epsv = 0;
+char *curl_http_proxy = NULL;
 
 struct curl_slist *pragma_header;
 
@@ -160,6 +161,13 @@ static int http_options(const char *var, const char *value)
 		curl_ftp_no_epsv = git_config_bool(var, value);
 		return 0;
 	}
+	if (!strcmp("http.proxy", var)) {
+		if (curl_http_proxy == NULL) {
+			curl_http_proxy = xmalloc(strlen(value)+1);
+			strcpy(curl_http_proxy, value);
+		}
+		return 0;
+	}
 
 	/* Fall back on the default ones */
 	return git_default_config(var, value);
@@ -205,6 +213,9 @@ static CURL* get_curl_handle(void)
 	if (curl_ftp_no_epsv)
 		curl_easy_setopt(result, CURLOPT_FTP_USE_EPSV, 0);
 
+	if (curl_http_proxy)
+		curl_easy_setopt(result, CURLOPT_PROXY, curl_http_proxy);
+
 	return result;
 }
 
-- 
1.5.3.5
Junio C Hamano· Nov 23, 2007, 00:28 UTC · re: Sam Vilain · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Sam Vilain <sam.vilain@catalyst.net.nz> writes:
Show 5 quoted lines
> The http_proxy / HTTPS_PROXY variables used by curl to control
> proxying may not be suitable for git.  Allow the user to override them
> in the configuration file.
> ---
>   In particular, privoxy will block directories called /ad/ ... d'oh!
Show 12 quoted lines
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 7ee97df..859a7f3 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -515,6 +515,10 @@ specified as 'gitcvs.<access_method>.<varname>' (where 'access_method'
>  is one of "ext" and "pserver") to make them apply only for the given
>  access method.
>  
> +http.proxy::
> +	Override the HTTP proxy, normally configured using the 'http_proxy'
> +	environment variable (see gitlink:curl[1]).
> +

This may work around the issue you cited, but it makes me wonder if it is a road to insanity. Does the curl library expect that (1) each and every HTTP talking application that uses the library offer this kind of knob for its users to tweak, and (2) users set the knob for each and every one of such application?

I would say if privoxy cannot be tweaked to allow /ad/ in chosen context (e.g. /ad/ in general is rejected but /objects/ad/ is Ok), that is what needs to be fixed.

Or it would be the use of such a broken proxy by the user. That can be fixed and much easily.

Sam Vilain· Nov 23, 2007, 03:35 UTC · re: Junio C Hamano · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Junio C Hamano wrote:
Show 15 quoted lines
>> The http_proxy / HTTPS_PROXY variables used by curl to control
>> proxying may not be suitable for git.  Allow the user to override them
>> in the configuration file.
>> ---
>>   In particular, privoxy will block directories called /ad/ ... d'oh!
>> +++ b/Documentation/config.txt
>> +http.proxy::
>> +	Override the HTTP proxy, normally configured using the 'http_proxy'
>> +	environment variable (see gitlink:curl[1]).
> 
> This may work around the issue you cited, but it makes me wonder
> if it is a road to insanity.  Does the curl library expect that
> (1) each and every HTTP talking application that uses the
> library offer this kind of knob for its users to tweak, and (2)
> users set the knob for each and every one of such application?

This is true. However I still think that it is a useful feature for many users, with few side effects. If nothing else the bit on the man page will prompt them to think, "oh, I should set that in the environment".

Show 5 quoted lines
> I would say if privoxy cannot be tweaked to allow /ad/ in chosen
> context (e.g. /ad/ in general is rejected but /objects/ad/ is
> Ok), that is what needs to be fixed.
> Or it would be the use of such a broken proxy by the user.  That
> can be fixed and much easily.

Yes - but consider the dilemma of the user. They've apt-get installed this privoxy thing and figured out how to set their applications to use it. Now, it doesn't work and they think it is the proxy in the way, and they've no idea that they might be able to reconfigure it. This way they can tell git to bypass it.

I don't know, I see your point and pretty much agree with it. It just seems like something that might come in handy (as well as be another vector for something you need to check when you get HTTP fetch issues - so maybe a command-line option would be better).

Actually something that would really have helped is more documentation of the various GIT_CURL_* environment variables available for debugging.

-- 
Sam Vilain, Chief Yak Shaver, Catalyst IT (NZ) Ltd.
phone: +64 4 499 2267        PGP ID: 0x66B25843
Junio C Hamano· Dec 1, 2007, 02:37 UTC · re: Sam Vilain · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Sam Vilain <sam.vilain@catalyst.net.nz> writes:
Show 40 quoted lines
> Junio C Hamano wrote:
>>> The http_proxy / HTTPS_PROXY variables used by curl to control
>>> proxying may not be suitable for git.  Allow the user to override them
>>> in the configuration file.
>>> ---
>>>   In particular, privoxy will block directories called /ad/ ... d'oh!
>>> +++ b/Documentation/config.txt
>>> +http.proxy::
>>> +	Override the HTTP proxy, normally configured using the 'http_proxy'
>>> +	environment variable (see gitlink:curl[1]).
>> 
>> This may work around the issue you cited, but it makes me wonder
>> if it is a road to insanity.  Does the curl library expect that
>> (1) each and every HTTP talking application that uses the
>> library offer this kind of knob for its users to tweak, and (2)
>> users set the knob for each and every one of such application?
>
> This is true.  However I still think that it is a useful feature for
> many users, with few side effects.  If nothing else the bit on the man
> page will prompt them to think, "oh, I should set that in the environment".
>
>> I would say if privoxy cannot be tweaked to allow /ad/ in chosen
>> context (e.g. /ad/ in general is rejected but /objects/ad/ is
>> Ok), that is what needs to be fixed.
>> Or it would be the use of such a broken proxy by the user.  That
>> can be fixed and much easily.
>
> Yes - but consider the dilemma of the user.  They've apt-get installed
> this privoxy thing and figured out how to set their applications to use
> it.  Now, it doesn't work and they think it is the proxy in the way, and
> they've no idea that they might be able to reconfigure it.  This way
> they can tell git to bypass it.
>
> I don't know, I see your point and pretty much agree with it.  It just
> seems like something that might come in handy (as well as be another
> vector for something you need to check when you get HTTP fetch issues -
> so maybe a command-line option would be better).
>
> Actually something that would really have helped is more documentation
> of the various GIT_CURL_* environment variables available for debugging.

Having thought about this a bit more, I changed my mind. From the beginning, I did not mind adding a dozen or so lines if the change helps the user. I was not _fundamentally_ opposed to your patch.

However.

There may be other environment variables that the users may want to set differently when running git from the settings they use in their interactive sessions. We have precedentsto support such situations in the form of git-specific environment variables (e.g. GIT_EDITOR and GIT_PAGER). These might have been a road to insanity already, and what we should have done may be to introduce multivalued core.environment variables in $HOME/.gitconfig, like so:

	[core.environment]
        	EDITOR = vim
		PAGER = less

without introducing GIT_foo environment variables. We can then change the git potty[*1*] start-up sequence to add them to the enviornment. But this is a bit like water under the bridge now.

While that approach could work for environment variables that apply globally to any and all git sessions the user may run, I suspect it would not work well for things like http_proxy. If we really wanted to help users, I think it should be tied to which remote we are going to, just like we made git-send-email to use different mailpath depending on which project you are communicating with.

In that sense, I think http.proxy configuration variable does not go far enough, even though it might be a step in the right direction. Perhaps use your configuration variable http.proxy (or "core.environment") to define the global default, with remote.$name.httpproxy to override it?

[Footnote]

*1* git(7) calls "git" itself as "git potty". Is this word still used? I also notice that Andreas's name is misspelled there.

Andreas Ericsson· Dec 3, 2007, 09:09 UTC · re: Junio C Hamano · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Junio C Hamano wrote:
Show 5 quoted lines
> 
> [Footnote]
> 
> *1* git(7) calls "git" itself as "git potty".  Is this word still used?
> I also notice that Andreas's name is misspelled there.

AFAIR I sort of invented that to disambiguate git-the-program from git-the-project while working on writing the program in C. I haven't heard/seen it a lot since then, except on a few occasions where it helped serve that same purpose.

I also misspelled my own name in that patch. Perhaps it's time I corrected that.

---%<---%<---%<---
From: Andreas Ericsson <ae@op5.se>
Subject: [PATCH] Correct typo in Documentation/git.txt
Signed-off-by: Andreas Ericsson <ae@op5.se>
---
Show changes to Documentation/git.txt +1 −1
diff --git a/Documentation/git.txt b/Documentation/git.txt
index 9ff4659..6c98ab2 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -536,7 +536,7 @@ Authors
 -------
 * git's founding father is Linus Torvalds <torvalds@osdl.org>.
 * The current git nurse is Junio C Hamano <gitster@pobox.com>.
-* The git potty was written by Andres Ericsson <ae@op5.se>.
+* The git potty was written by Andreas Ericsson <ae@op5.se>.
 * General upbringing is handled by the git-list <git@vger.kernel.org>.
 
 Documentation
---%<---%<---%<---
-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
Johannes Schindelin· Dec 3, 2007, 11:13 UTC · re: Andreas Ericsson · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Hi,
On Mon, 3 Dec 2007, Andreas Ericsson wrote:
Show 12 quoted lines
> Junio C Hamano wrote:
> > 
> > [Footnote]
> > 
> > *1* git(7) calls "git" itself as "git potty".  Is this word still used?
> > I also notice that Andreas's name is misspelled there.
> 
> 
> AFAIR I sort of invented that to disambiguate git-the-program from 
> git-the-project while working on writing the program in C. I haven't 
> heard/seen it a lot since then, except on a few occasions where it 
> helped serve that same purpose.
I always spell it "git wrapper".

Ciao, Dscho

Sam Vilain· Dec 3, 2007, 21:48 UTC · re: Junio C Hamano · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Junio C Hamano wrote:
> In that sense, I think http.proxy configuration variable does not go far
> enough, even though it might be a step in the right direction.  Perhaps
> use your configuration variable http.proxy (or "core.environment") to
> define the global default, with remote.$name.httpproxy to override it?
Sure, why not.
Subject: [PATCH] Add remote.<name>.proxy

As well as allowing a default proxy option, allow it to be set per-remote.

Signed-off-by: Sam Vilain <sam.vilain@catalyst.net.nz>
---
 Documentation/config.txt |    8 +++++++-
 remote.c                 |    2 ++
 remote.h                 |    5 +++++
 transport.c              |    3 +++
 4 files changed, 17 insertions(+), 1 deletions(-)
Show changes to 4 files +17 −1

Documentation/config.txt, remote.c, remote.h, transport.c

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 7e6c02a..831df58 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -545,7 +545,8 @@ access method.
 
 http.proxy::
 	Override the HTTP proxy, normally configured using the 'http_proxy'
-	environment variable (see gitlink:curl[1]).
+	environment variable (see gitlink:curl[1]).  This can be overridden
+	on a per-remote basis; see remote.<name>.proxy
 
 http.sslVerify::
 	Whether to verify the SSL certificate when fetching or pushing
@@ -695,6 +696,11 @@ remote.<name>.url::
 	The URL of a remote repository.  See gitlink:git-fetch[1] or
 	gitlink:git-push[1].
 
+remote.<name>.proxy::
+	For remotes that require curl (http, https and ftp), the URL to
+	the proxy to use for that remote.  Set to the empty string to
+	disable proxying for that remote.
+
 remote.<name>.fetch::
 	The default set of "refspec" for gitlink:git-fetch[1]. See
 	gitlink:git-fetch[1].
diff --git a/remote.c b/remote.c
index bec2ba1..94df314 100644
--- a/remote.c
+++ b/remote.c
@@ -278,6 +278,8 @@ static int handle_config(const char *key, const char *value)
 	} else if (!strcmp(subkey, ".tagopt")) {
 		if (!strcmp(value, "--no-tags"))
 			remote->fetch_tags = -1;
+	} else if (!strcmp(subkey, ".proxy")) {
+		remote->fetch_refspec = xstrdup(value);
 	}
 	return 0;
 }
diff --git a/remote.h b/remote.h
index 878b4ec..2268558 100644
--- a/remote.h
+++ b/remote.h
@@ -25,6 +25,11 @@ struct remote {
 
 	const char *receivepack;
 	const char *uploadpack;
+
+	/*
+	 * for curl remotes only
+	 */
+	const char *proxy;
 };
 
 struct remote *remote_get(const char *name);
diff --git a/transport.c b/transport.c
index 43b9e7c..c62ec88 100644
--- a/transport.c
+++ b/transport.c
@@ -463,6 +463,9 @@ static struct ref *get_refs_via_curl(const struct transport *transport)
 	curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
 	curl_easy_setopt(slot->curl, CURLOPT_URL, refs_url);
 	curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
+	if (transport->remote->proxy) {
+		curl_easy_setopt(slot->curl, CURLOPT_PROXY, transport->remote->proxy);
+	}
 	if (start_active_slot(slot)) {
 		run_active_slot(slot);
 		if (results.curl_result != CURLE_OK) {
-- 
1.5.3.5
Junio C Hamano· Dec 4, 2007, 08:49 UTC · re: Sam Vilain · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Sam Vilain <sam.vilain@catalyst.net.nz> writes:
> Subject: [PATCH] Add remote.<name>.proxy
>
> As well as allowing a default proxy option, allow it to be set
> per-remote.
Thanks.

I was very tempted to rename them to remote.*.httpproxy, but ended up keeping the name of the variable. We might want to allow tunnelling the git native transport over the named proxy when remote.*.url is native transport.

Sam Vilain· Dec 4, 2007, 09:22 UTC · re: Junio C Hamano · lore

Re: [PATCH] Allow HTTP proxy to be overridden in config

Junio C Hamano wrote:
Show 10 quoted lines
>> Subject: [PATCH] Add remote.<name>.proxy
>> As well as allowing a default proxy option, allow it to be set
>> per-remote.
> 
> Thanks.
> 
> I was very tempted to rename them to remote.*.httpproxy, but ended up
> keeping the name of the variable.  We might want to allow tunnelling the
> git native transport over the named proxy when remote.*.url is native
> transport.

Also, it controls proxy for all of the curl-based git transports, not just http.

Sam.

← back to recent threads