threads / discuss / 4165

Fwd: [OT] Re: Git via a proxy server?

Subject: Fwd: [OT] Re: Git via a proxy server?

## tl;dr

7 messages between May 16, 2006 and May 18, 2006.

replies: 6people: 3as markdown or json

Sam Song· May 16, 2006, 12:13 UTC · lore
Hello,
Petr Vandrovec <petr@vmware.com> wrote:
Show 37 quoted lines
> Date:	Mon, 27 Feb 2006 00:35:00 +0100
> From:	Petr Vandrovec <petr@vmware.com>
> To:	Sergey Vlasov <vsu@altlinux.ru>
> CC:	"Salyzyn, Mark" <mark_salyzyn@adaptec.com>,
> 	Kernel Mailing List 
> <linux-kernel@vger.kernel.org>
> > [snip]
> > I have successfully used transconnect
> > (http://sourceforge.net/projects/transconnect) for
> > tunnelling git
> > protocol through a HTTP proxy (squid in my case)
> > supporting the CONNECT method.
> >
> > [snip] 
> > Note: most HTTP proxy servers allow CONNECT method
> > to a very limited range of ports, and 
> > administrators will need to enable the git port
> > (9418) explicitly.
> > 
> I know I'm coming kinda late, but I'm using:
> 
> export GIT_PROXY_COMMAND=/usr/local/bin/proxy-cmd.sh
> 
> and proxy-cmd.sh is just single-line command glued
> from what I found available in /bin:
> 
> #! /bin/bash
> 
> (echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | socket
> 
> proxy.ourcompany.com 3128 | (read a; read a; cat )
> 
> Replace socket's arguments 'proxy.ourcompany.com
> 3128' with your http proxy.  Fortunately our proxy
> does not see anything wrong with git's port.
> 		Best regards,
> 			Petr Vandrovec

With above usage on GIT_PROXY_COMMAND, I still have problem on connection with remote git repository.

I also tried setting http_proxy directly but the same result. It's first usage of git in our network. Well, need I enable the git port 9418 at proxy server? Or did I miss sth or what?

I use git-2006-05-14.tar.gz package on FC3. 
Method I : Use GIT_PROXY_COMMAND

[root@sam u-boot]# git clone \ git://www.denx.de/git/u-boot.git u-boot-denx.git

fatal: exec failed fetch-pack from 'git://www.denx.de/git/u-boot.git' failed.

[root@sam u-boot]# git clone \ http://parisc-linux.org/git/linux-2.6.git/ parisc-2.6

Cannot get remote repository information. Perhaps git-update-server-info needs to be run there? [root@sam u-boot]#

/usr/local/bin/proxy-cmd.sh 
#! /bin/bash

(echo "CONNECT $1:$2 HTTP/1.0";echo;cat) | socket <um> <pwd> 192.168.40.99 80 | (read a;read a;cat)

Method II : Use http_proxy directly

[root@sam u-boot]# export \ http_proxy="http://<username>:<pwd>@192.168.40.99:80" [root@sam u-boot]# git clone \ http://parisc-linux.org/git/linux-2.6.git/ parisc-2.6

Cannot get remote repository information. Perhaps git-update-server-info needs to be run there?

[root@sam u-boot]# git clone \ git://www.denx.de/git/u-boot.git u-boot-denx.git fatal: unable to connect a socket (Connection timed out) fetch-pack from 'git://www.denx.de/git/u-boot.git' failed. [root@sam u-boot]#

Thanks in advance,
Sam

P.S. I forward this thread from LKML for better discussion. Hope Petr wouldn't mind.

__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

Petr Vandrovec· May 16, 2006, 13:11 UTC · re: Sam Song · lore

Re: Fwd: [OT] Re: Git via a proxy server?

Sam Song wrote:
Show 52 quoted lines
> Hello,
> 
> Petr Vandrovec <petr@vmware.com> wrote:
> 
>>Date:	Mon, 27 Feb 2006 00:35:00 +0100
>>From:	Petr Vandrovec <petr@vmware.com>
>>To:	Sergey Vlasov <vsu@altlinux.ru>
>>CC:	"Salyzyn, Mark" <mark_salyzyn@adaptec.com>,
>>	Kernel Mailing List 
>><linux-kernel@vger.kernel.org>
>>
>>>[snip]
>>>I have successfully used transconnect
>>>(http://sourceforge.net/projects/transconnect) for
>>>tunnelling git
>>>protocol through a HTTP proxy (squid in my case)
>>>supporting the CONNECT method.
>>>
>>>[snip] 
>>>Note: most HTTP proxy servers allow CONNECT method
>>>to a very limited range of ports, and 
>>>administrators will need to enable the git port
>>>(9418) explicitly.
>>>
>>
>>I know I'm coming kinda late, but I'm using:
>>
>>export GIT_PROXY_COMMAND=/usr/local/bin/proxy-cmd.sh
>>
>>and proxy-cmd.sh is just single-line command glued
>>from what I found available in /bin:
>>
>>#! /bin/bash
>>
>>(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | socket
>>
>>proxy.ourcompany.com 3128 | (read a; read a; cat )
>>
>>Replace socket's arguments 'proxy.ourcompany.com
>>3128' with your http proxy.  Fortunately our proxy
>>does not see anything wrong with git's port.
>>		Best regards,
>>			Petr Vandrovec
> 
> 
> With above usage on GIT_PROXY_COMMAND, I still have
> problem on connection with remote git repository.
> 
> I also tried setting http_proxy directly but the same
> result. It's first usage of git in our network. Well, 
> need I enable the git port 9418 at proxy server? Or 
> did I miss sth or what?
Yes.  Try running 'socket 192.168.40.99 80', and type
CONNECT 204.152.191.37:9418 HTTP/1.0
Proxy-Authorization: Basic <yoursecret,f.e.wget -d should reveal this to you>
<empty line>

You should get back user readable diagnostics what went wrong. Yes, your admin must allow CONNECT method for target port 9418.

Show 10 quoted lines
> I use git-2006-05-14.tar.gz package on FC3. 
> 
> Method I : Use GIT_PROXY_COMMAND
> 
> [root@sam u-boot]# git clone \
> git://www.denx.de/git/u-boot.git u-boot-denx.git
> 
> fatal: exec failed
> fetch-pack from 'git://www.denx.de/git/u-boot.git' 
> failed.

Is $GIT_PROXY_COMMAND executable? (just in case...) Try 'strace -f git clone ...', it should tell you what's going on.

> #! /bin/bash
> 
> (echo "CONNECT $1:$2 HTTP/1.0";echo;cat) | socket
> <um> <pwd> 192.168.40.99 80 | (read a;read a;cat)

What is '<um>' and '<pwd>' ? socket just connects somewhere, so if you are supposed to use <username>:<pwd> to connect to your proxy, you must add Proxy-Authorization header yourself:

(echo "CONNECT $1:$2 HTTP/1.0";
  echo "Proxy-Authorization: Basic <base64encoded um:pwd>";
  echo;
  cat ) | socket 192.168.40.99 80 | (read a; read a; cat)

Best to test this is to start 'socket 192.168.40.99 80' from command line and then type these two lines above, plus one empty line. You should get back '200 OK', empty line, and then you can start communicating using git protocol - if you can do that...

Show 6 quoted lines
> Method II : Use http_proxy directly
> 
> [root@sam u-boot]# export \
> http_proxy="http://<username>:<pwd>@192.168.40.99:80"
> [root@sam u-boot]# git clone \ 
> http://parisc-linux.org/git/linux-2.6.git/ parisc-2.6
As far as I can tell, http_proxy is ignored (Debian's git 1.3.2-1/cogito 0.17.2-1).
								Petr
Sam Song· May 17, 2006, 03:56 UTC · re: Petr Vandrovec · lore

Re: Fwd: [OT] Re: Git via a proxy server?

Petr Vandrovec <petr@vmware.com> wrote:
Show 7 quoted lines
> Best to test this is to start 'socket 192.168.40.99
> 80' from command line and 
> then type these two lines above, plus one empty
> line.  You should get back '200 
> OK', empty line, and then you can start
> communicating using git protocol - if 
> you can do that...

I cannot run "socket" and "CONNECT" on Fedora Core 3. It simply told me that no such command. How could I do this task in my case?

> As far as I can tell, http_proxy is ignored
> (Debian's git 1.3.2-1/cogito 0.17.2-1).

Seems you tried proxy-cmd.sh on Debian. Which distribution did you use?

Thanks a lot,
Sam

__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

Jan-Benedict Glaw· May 17, 2006, 08:38 UTC · re: Sam Song · lore

Re: Fwd: [OT] Re: Git via a proxy server?

On Tue, 2006-05-16 20:56:39 -0700, Sam Song <samlinuxkernel@yahoo.com> wrote:
Show 12 quoted lines
> Petr Vandrovec <petr@vmware.com> wrote:
> > Best to test this is to start 'socket 192.168.40.99
> > 80' from command line and 
> > then type these two lines above, plus one empty
> > line.  You should get back '200 
> > OK', empty line, and then you can start
> > communicating using git protocol - if 
> > you can do that...
> 
> I cannot run "socket" and "CONNECT" on Fedora Core 3.
> It simply told me that no such command. How could I 
> do this task in my case?

Well, install some package to have `socket' available? Debian calls the packet `socket', too, so I guess Fedora may have something similar.

MfG, JBG
-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
Petr Vandrovec· May 17, 2006, 10:54 UTC · re: Jan-Benedict Glaw · lore

Re: Fwd: [OT] Re: Git via a proxy server?

Jan-Benedict Glaw wrote:
Show 20 quoted lines
> On Tue, 2006-05-16 20:56:39 -0700, Sam Song <samlinuxkernel@yahoo.com> wrote:
> 
>>Petr Vandrovec <petr@vmware.com> wrote:
>>
>>>Best to test this is to start 'socket 192.168.40.99
>>>80' from command line and 
>>>then type these two lines above, plus one empty
>>>line.  You should get back '200 
>>>OK', empty line, and then you can start
>>>communicating using git protocol - if 
>>>you can do that...
>>
>>I cannot run "socket" and "CONNECT" on Fedora Core 3.
>>It simply told me that no such command. How could I 
>>do this task in my case?
> 
> 
> Well, install some package to have `socket' available? Debian calls
> the packet `socket', too, so I guess Fedora may have something
> similar.
Surprisingly they do not...  You should be able to replace 'socket' with 
'netcat' - and I believe that netcat/nc package is available for Fedora.  For 
this purpose they have same command line & behavior.
							Petr
Sam Song· May 18, 2006, 03:44 UTC · re: Petr Vandrovec · lore

Re: Fwd: [OT] Re: Git via a proxy server?

Petr Vandrovec <petr@vmware.com> wrote:
Show 11 quoted lines
> Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> > Well, install some package to have `socket'
> > available? Debian calls
> > the packet `socket', too, so I guess Fedora may
> > have something similar.
> 
> Surprisingly they do not...  You should be able to
> replace 'socket' with 
> 'netcat' - and I believe that netcat/nc package is
> available for Fedora.  For 
> this purpose they have same command line & behavior.

Ummm, I am trying on that. nc is avaiable for Fedora. But what could be the replacement for CONNECT in Fedora? :-)

Thanks for your kind support,
Sam

__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

Jan-Benedict Glaw· May 18, 2006, 08:31 UTC · re: Sam Song · lore

Re: Fwd: [OT] Re: Git via a proxy server?

On Wed, 2006-05-17 20:44:28 -0700, Sam Song <samlinuxkernel@yahoo.com> wrote:
Show 16 quoted lines
> Petr Vandrovec <petr@vmware.com> wrote:
> > Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> > > Well, install some package to have `socket'
> > > available? Debian calls
> > > the packet `socket', too, so I guess Fedora may
> > > have something similar.
> > 
> > Surprisingly they do not...  You should be able to
> > replace 'socket' with 
> > 'netcat' - and I believe that netcat/nc package is
> > available for Fedora.  For 
> > this purpose they have same command line & behavior.
> 
> Ummm, I am trying on that. nc is avaiable for Fedora.
> But what could be the replacement for CONNECT in
> Fedora? :-)
Erm, you haven't understood what you're doing there, have you?

With the GIT_PROXY_COMMAND helper, you're expected to create a clean tunnel which in turn git can use to transfer its data.

You've only got some limited internet connectivity via a HTTP proxy available, so you need to use this. This means:

  * The proxy administrator needs to allos outgoing connections for
    the CONNECT method with git's TCP port.
  * You need to have some minimalistic program to initially speak HTTP
    with the proxy and later on just stream the raw git protocol
    through the link.
  * You may or may not need to strip anything that came into the git
    stream by accident because you tunnled it through a HTTP proxy. A
    reply message from the proxy server is an example for this.

So this little script (using "CONNECT" and netcat or socket) does the first part: it talks in the language HTTP with the proxy server. It may be enough to just use CONNECT, but you may need to speak some more lines, eg. for proxy authorization.

The first `cat' in there is just for pushing the git protocol though the HTTP proxy connection later on (hopefully after the proxy was made to accept the the CONNECT request.) Once the proxy accepted it, it'll send you a HTTP/200 message (or something like that) and an empty line. This is what the two reads are for; the next `cat' simply again transfers all the rest (the git protocol).

To draw the line, there's not _one_ solution to HTTP proxy tunneling, there are many, and you'll need to design one that fits your network. It should be quite simple, given that you've got nice tools like `strace' and `tcpdump', which will help you to understand how the proxy reacts and so on.

MfG, JBG
-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

← back to recent threads