# [PATCH] Fix problem with authentification on http repository. 18 messages from 2007-09-29 to 2008-07-10. Participants: jean.guyader@linkea.org, Johannes Schindelin, Junio C Hamano, Mike Hommey, Daniel Stenberg, Jeff King, Andreas Ericsson. Thread: https://gitlist.dev/t/10082 ## jean.guyader@linkea.org, 2007-09-29 22:26 Subject: [PATCH] Fix problem with authentification on http repository. Message-ID: <11911047823308-git-send-email-jean.guyader@linkea.org> URL: https://gitlist.dev/e/11911047823308-git-send-email-jean.guyader%40linkea.org ``` From: Jean Guyader Curl uses the option -u user:passwd and not the user:password given in the url. The solution was to extract user:password from the url and set the option. Here the regex used : sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g' --- git-clone.sh | 3 ++- git-fetch.sh | 8 ++++---- git-ls-remote.sh | 3 ++- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/git-clone.sh b/git-clone.sh index 5e582fe..57206ac 100755 --- a/git-clone.sh +++ b/git-clone.sh @@ -34,7 +34,8 @@ fi http_fetch () { # $1 = Remote, $2 = Local - curl -nsfL $curl_extra_args "$1" >"$2" || + curl_userpw=`echo "$1" | sed -re 's-http://([^:]*):([^@]+)@.*-\1:\2-g'` + curl -u "$curl_userpw" -nsfL $curl_extra_args "$1" >"$2" || case $? in 126|127) exit ;; *) return $? ;; diff --git a/git-fetch.sh b/git-fetch.sh index e44af2c..0b7d055 100755 --- a/git-fetch.sh +++ b/git-fetch.sh @@ -268,13 +268,13 @@ fetch_per_ref () { "`git config --bool http.noEPSV`" = true ]; then noepsv_opt="--disable-epsv" fi - - # Find $remote_name from ls-remote output. + pretty_remote=$(echo "$remote" | sed -re 's-http://([^:]+):([^@]+)@-http://\1:******@-g') + # Find $remote_name from ls-remote output. head=$(echo "$ls_remote_result" | \ git fetch--tool -s pick-rref "$remote_name" "-") expr "z$head" : "z$_x40\$" >/dev/null || - die "No such ref $remote_name at $remote" - echo >&2 "Fetching $remote_name from $remote using $proto" + die "No such ref $remote_name at $pretty_remote" + echo >&2 "Fetching $remote_name from $pretty_remote using $proto" case "$quiet" in '') v=-v ;; *) v= ;; esac git-http-fetch $v -a "$head" "$remote" || exit ;; diff --git a/git-ls-remote.sh b/git-ls-remote.sh index d56cf92..d6e9906 100755 --- a/git-ls-remote.sh +++ b/git-ls-remote.sh @@ -61,7 +61,8 @@ http://* | https://* | ftp://* ) "`git config --bool http.noEPSV`" = true ]; then curl_extra_args="${curl_extra_args} --disable-epsv" fi - curl -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" || + curl_userpw=`echo "$peek_repo" | sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'` + curl -u "$curl_userpw" -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" || echo "failed slurping" ;; -- 1.5.2.4 ``` ## Johannes Schindelin, 2007-09-29 23:10 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0709300009580.28395%40racer.site In-Reply-To: <11911047823308-git-send-email-jean.guyader@linkea.org> ``` Hi, On Sun, 30 Sep 2007, jean.guyader@linkea.org wrote: > git-clone.sh | 3 ++- > git-fetch.sh | 8 ++++---- > git-ls-remote.sh | 3 ++- git-fetch is already a builtin in "next", and -clone and -ls-remote are likely to follow suit. Ciao, Dscho ``` ## Junio C Hamano, 2007-09-29 23:38 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <7v8x6pjb4c.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7v8x6pjb4c.fsf%40gitster.siamese.dyndns.org In-Reply-To: <11911047823308-git-send-email-jean.guyader@linkea.org> ``` jean.guyader@linkea.org writes: > From: Jean Guyader > > Curl uses the option -u user:passwd and not the user:password > given in the url. > The solution was to extract user:password from the url and set > the option. > > Here the regex used : > sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g' This is more like "allowing embedded authentication credentials in URL for http transport". We never supported URLs with embedded credentials (see Documentation/urls.txt), partly because nobody asked for it, but more importantly because giving -n to curl to have it read from user's .netrc is generally much more preferred approach. ``` ## Johannes Schindelin, 2007-09-29 23:40 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0709300039430.28395%40racer.site In-Reply-To: <7v8x6pjb4c.fsf@gitster.siamese.dyndns.org> ``` Hi, On Sat, 29 Sep 2007, Junio C Hamano wrote: > jean.guyader@linkea.org writes: > > > From: Jean Guyader > > > > Curl uses the option -u user:passwd and not the user:password > > given in the url. > > The solution was to extract user:password from the url and set > > the option. > > > > Here the regex used : > > sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g' > > This is more like "allowing embedded authentication credentials > in URL for http transport". > > We never supported URLs with embedded credentials (see > Documentation/urls.txt), partly because nobody asked for it, but > more importantly because giving -n to curl to have it read from > user's .netrc is generally much more preferred approach. To elaborate on that: if you fetch from somewhere, your url, username and password can be read from the output of "ps ax | grep http" very easily. Ciao, Dscho ``` ## Junio C Hamano, 2007-09-30 00:02 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <7v4phdja01.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7v4phdja01.fsf%40gitster.siamese.dyndns.org In-Reply-To: ``` Johannes Schindelin writes: > On Sat, 29 Sep 2007, Junio C Hamano wrote: > >> We never supported URLs with embedded credentials (see >> Documentation/urls.txt), partly because nobody asked for it, but >> more importantly because giving -n to curl to have it read from >> user's .netrc is generally much more preferred approach. > > To elaborate on that: if you fetch from somewhere, your url, username and > password can be read from the output of "ps ax | grep http" very easily. Actually Documentation/howto/setup-git-server-over-http.txt talks about http://user@server/path/ format. How well does this work in practice? If it does, we should update Documentation/urls.txt to allow optional user@ there like... --- Documentation/urls.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/urls.txt b/Documentation/urls.txt index e67f914..c9aab86 100644 --- a/Documentation/urls.txt +++ b/Documentation/urls.txt @@ -6,8 +6,8 @@ to name the remote repository: =============================================================== - rsync://host.xz/path/to/repo.git/ -- http://host.xz/path/to/repo.git/ -- https://host.xz/path/to/repo.git/ +- http://{startsb}user@{endsb}host.xz/path/to/repo.git/ +- https://{startsb}user@{endsb}host.xz/path/to/repo.git/ - git://host.xz/path/to/repo.git/ - git://host.xz/~user/path/to/repo.git/ - ssh://{startsb}user@{endsb}host.xz{startsb}:port{endsb}/path/to/repo.git/ ``` ## Junio C Hamano, 2008-07-09 05:44 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <7v4p6zmx0l.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7v4p6zmx0l.fsf%40gitster.siamese.dyndns.org In-Reply-To: <7v4phdja01.fsf@gitster.siamese.dyndns.org> ``` Johannes Schindelin writes: > On Sat, 29 Sep 2007, Junio C Hamano wrote: > >> We never supported URLs with embedded credentials (see >> Documentation/urls.txt), partly because nobody asked for it, but >> more importantly because giving -n to curl to have it read from >> user's .netrc is generally much more preferred approach. > > To elaborate on that: if you fetch from somewhere, your url, username and > password can be read from the output of "ps ax | grep http" very easily. Actually Documentation/howto/setup-git-server-over-http.txt talks about http://user@server/path/ format. How well does this work in practice? If it does, we should update Documentation/urls.txt to allow optional user@ there like... --- Documentation/urls.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/urls.txt b/Documentation/urls.txt index e67f914..c9aab86 100644 --- a/Documentation/urls.txt +++ b/Documentation/urls.txt @@ -6,8 +6,8 @@ to name the remote repository: =============================================================== - rsync://host.xz/path/to/repo.git/ -- http://host.xz/path/to/repo.git/ -- https://host.xz/path/to/repo.git/ +- http://{startsb}user@{endsb}host.xz/path/to/repo.git/ +- https://{startsb}user@{endsb}host.xz/path/to/repo.git/ - git://host.xz/path/to/repo.git/ - git://host.xz/~user/path/to/repo.git/ - ssh://{startsb}user@{endsb}host.xz{startsb}:port{endsb}/path/to/repo.git/ ``` ## Mike Hommey, 2008-07-09 05:51 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <20080709055150.GA28482@glandium.org> URL: https://gitlist.dev/e/20080709055150.GA28482%40glandium.org In-Reply-To: <7v4p6zmx0l.fsf@gitster.siamese.dyndns.org> ``` On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote: > Johannes Schindelin writes: > > > On Sat, 29 Sep 2007, Junio C Hamano wrote: > > > >> We never supported URLs with embedded credentials (see > >> Documentation/urls.txt), partly because nobody asked for it, but > >> more importantly because giving -n to curl to have it read from > >> user's .netrc is generally much more preferred approach. > > > > To elaborate on that: if you fetch from somewhere, your url, username and > > password can be read from the output of "ps ax | grep http" very easily. > > Actually Documentation/howto/setup-git-server-over-http.txt > talks about http://user@server/path/ format. How well does this > work in practice? If it does, we should update Documentation/urls.txt > to allow optional user@ there like... AFAIK, there is nothing to deal with asking a password to the user in git in this case, so that doesn't work. Mike ``` ## Junio C Hamano, 2008-07-09 06:28 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <7vlk0blgg0.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vlk0blgg0.fsf%40gitster.siamese.dyndns.org In-Reply-To: <20080709055150.GA28482@glandium.org> ``` Mike Hommey writes: > On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote: >> Johannes Schindelin writes: >> >> > On Sat, 29 Sep 2007, Junio C Hamano wrote: >> > >> >> We never supported URLs with embedded credentials (see >> >> Documentation/urls.txt), partly because nobody asked for it, but >> >> more importantly because giving -n to curl to have it read from >> >> user's .netrc is generally much more preferred approach. >> > >> > To elaborate on that: if you fetch from somewhere, your url, username and >> > password can be read from the output of "ps ax | grep http" very easily. >> >> Actually Documentation/howto/setup-git-server-over-http.txt >> talks about http://user@server/path/ format. How well does this >> work in practice? If it does, we should update Documentation/urls.txt >> to allow optional user@ there like... > > AFAIK, there is nothing to deal with asking a password to the user in git > in this case, so that doesn't work. Then perhaps the howto needs updating? ``` ## Mike Hommey, 2008-07-09 07:53 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <20080709075345.GA22022@glandium.org> URL: https://gitlist.dev/e/20080709075345.GA22022%40glandium.org In-Reply-To: <7vlk0blgg0.fsf@gitster.siamese.dyndns.org> ``` On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano wrote: > Mike Hommey writes: > > > On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote: > >> Johannes Schindelin writes: > >> > >> > On Sat, 29 Sep 2007, Junio C Hamano wrote: > >> > > >> >> We never supported URLs with embedded credentials (see > >> >> Documentation/urls.txt), partly because nobody asked for it, but > >> >> more importantly because giving -n to curl to have it read from > >> >> user's .netrc is generally much more preferred approach. > >> > > >> > To elaborate on that: if you fetch from somewhere, your url, username and > >> > password can be read from the output of "ps ax | grep http" very easily. > >> > >> Actually Documentation/howto/setup-git-server-over-http.txt > >> talks about http://user@server/path/ format. How well does this > >> work in practice? If it does, we should update Documentation/urls.txt > >> to allow optional user@ there like... > > > > AFAIK, there is nothing to deal with asking a password to the user in git > > in this case, so that doesn't work. > > Then perhaps the howto needs updating? Actually, the howto tells to setup a ~/.netrc file, which should make the http://user@server/path/ format work. I don't know if dropping user@ would still work, that depends how libcurl works with it. Mike ``` ## Daniel Stenberg, 2008-07-09 08:04 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: URL: https://gitlist.dev/e/alpine.LRH.1.10.0807090955520.5588%40yvahk3.pbagnpgbe.fr In-Reply-To: <20080709075345.GA22022@glandium.org> ``` On Wed, 9 Jul 2008, Mike Hommey wrote: > Actually, the howto tells to setup a ~/.netrc file, which should make > the http://user@server/path/ format work. I don't know if dropping user@ > would still work, that depends how libcurl works with it. git has the netrc parser set to "optional", which makes libcurl use the login name given in the URL (if present) and search for the associated password in the .netrc file. Typical users probably only have one user per host name in the .netrc file and then the name part in the URL provides no particular extra goodness but can just as well be excluded. -- / daniel.haxx.se ``` ## Junio C Hamano, 2008-07-09 08:18 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <7vhcazlbch.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vhcazlbch.fsf%40gitster.siamese.dyndns.org In-Reply-To: <20080709075345.GA22022@glandium.org> ``` Mike Hommey writes: > On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano wrote: > ... >> >> Actually Documentation/howto/setup-git-server-over-http.txt >> >> talks about http://user@server/path/ format. How well does this >> >> work in practice? If it does, we should update Documentation/urls.txt >> >> to allow optional user@ there like... >> > >> > AFAIK, there is nothing to deal with asking a password to the user in git >> > in this case, so that doesn't work. >> >> Then perhaps the howto needs updating? > > Actually, the howto tells to setup a ~/.netrc file, which should make > the http://user@server/path/ format work. I don't know if dropping user@ > would still work, that depends how libcurl works with it. Ok, so the documentation update I suggested earlier may actually be valid, right? ``` ## Mike Hommey, 2008-07-09 08:56 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <20080709085651.GA23453@glandium.org> URL: https://gitlist.dev/e/20080709085651.GA23453%40glandium.org In-Reply-To: <7vhcazlbch.fsf@gitster.siamese.dyndns.org> ``` On Wed, Jul 09, 2008 at 01:18:22AM -0700, Junio C Hamano wrote: > Mike Hommey writes: > > > On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano wrote: > > ... > >> >> Actually Documentation/howto/setup-git-server-over-http.txt > >> >> talks about http://user@server/path/ format. How well does this > >> >> work in practice? If it does, we should update Documentation/urls.txt > >> >> to allow optional user@ there like... > >> > > >> > AFAIK, there is nothing to deal with asking a password to the user in git > >> > in this case, so that doesn't work. > >> > >> Then perhaps the howto needs updating? > > > > Actually, the howto tells to setup a ~/.netrc file, which should make > > the http://user@server/path/ format work. I don't know if dropping user@ > > would still work, that depends how libcurl works with it. > > Ok, so the documentation update I suggested earlier may actually be > valid, right? The problem is that these urls won't work out of the box if the user doesn't setup a .netrc file. I'd say it would be better not to add these forms in Documentation/urls.txt until git prompts for the password when it is missing. Note that http://user:pass@server/path/ /should/ work (but that would need validation), though not a good idea on command line. Mike ``` ## Johannes Schindelin, 2008-07-09 11:43 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0807091341230.5277%40eeepc-johanness In-Reply-To: <20080709085651.GA23453@glandium.org> ``` Hi, On Wed, 9 Jul 2008, Mike Hommey wrote: > Note that http://user:pass@server/path/ /should/ work (but that would > need validation), though not a good idea on command line. Well, now that the programs using URLs are all builtins, we can actually do something about it. We can edit out the "user[:pass]@" part out of argv, which usually means that "ps" will not see it anymore. Ciao, Dscho ``` ## Jeff King, 2008-07-10 07:24 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <20080710072420.GA5167@sigill.intra.peff.net> URL: https://gitlist.dev/e/20080710072420.GA5167%40sigill.intra.peff.net In-Reply-To: ``` On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote: > > Note that http://user:pass@server/path/ /should/ work (but that would > > need validation), though not a good idea on command line. > > Well, now that the programs using URLs are all builtins, we can actually > do something about it. We can edit out the "user[:pass]@" part out of > argv, which usually means that "ps" will not see it anymore. Wouldn't there still be a race condition for publicly broadcasting your password via ps? -Peff ``` ## Junio C Hamano, 2008-07-10 07:32 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <7vk5fuqjn8.fsf@gitster.siamese.dyndns.org> URL: https://gitlist.dev/e/7vk5fuqjn8.fsf%40gitster.siamese.dyndns.org In-Reply-To: <20080710072420.GA5167@sigill.intra.peff.net> ``` Jeff King writes: > On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote: > >> > Note that http://user:pass@server/path/ /should/ work (but that would >> > need validation), though not a good idea on command line. >> >> Well, now that the programs using URLs are all builtins, we can actually >> do something about it. We can edit out the "user[:pass]@" part out of >> argv, which usually means that "ps" will not see it anymore. > > Wouldn't there still be a race condition for publicly broadcasting your > password via ps? In addition, I think on some Unices overwriting argv[0] does not have the effect Dscho is aiming for. I thought we do discourage [:pass] part in our documentation. ``` ## Jeff King, 2008-07-10 08:47 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <20080710084728.GA6199@sigill.intra.peff.net> URL: https://gitlist.dev/e/20080710084728.GA6199%40sigill.intra.peff.net In-Reply-To: <7vk5fuqjn8.fsf@gitster.siamese.dyndns.org> ``` On Thu, Jul 10, 2008 at 12:32:27AM -0700, Junio C Hamano wrote: > > Wouldn't there still be a race condition for publicly broadcasting your > > password via ps? > > In addition, I think on some Unices overwriting argv[0] does not have the > effect Dscho is aiming for. I thought so, too, but I couldn't find any mention of it online. For curiosity's sake, do you happen to know offhand any platform where this doesn't work (of course, Dscho is really proposing overwriting argv[i], but I don't know if that makes a difference to the portability)? -Peff ``` ## Johannes Schindelin, 2008-07-10 10:51 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: URL: https://gitlist.dev/e/alpine.DEB.1.00.0807101248360.3135%40eeepc-johanness In-Reply-To: <20080710072420.GA5167@sigill.intra.peff.net> ``` Hi, On Thu, 10 Jul 2008, Jeff King wrote: > On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote: > > > > Note that http://user:pass@server/path/ /should/ work (but that > > > would need validation), though not a good idea on command line. > > > > Well, now that the programs using URLs are all builtins, we can > > actually do something about it. We can edit out the "user[:pass]@" > > part out of argv, which usually means that "ps" will not see it > > anymore. > > Wouldn't there still be a race condition for publicly broadcasting your > password via ps? Yes. For a brief amount of time, but yes. BTW I thought I remembered one program xxx'ing out the password via argv, but unfortunately I do not remember which one it was. Anyway, was just an idea. Ciao, Dscho ``` ## Andreas Ericsson, 2008-07-10 11:30 Subject: Re: [PATCH] Fix problem with authentification on http repository. Message-ID: <4875F2E7.1010209@op5.se> URL: https://gitlist.dev/e/4875F2E7.1010209%40op5.se In-Reply-To: ``` Johannes Schindelin wrote: > Hi, > > On Thu, 10 Jul 2008, Jeff King wrote: > >> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote: >> >>>> Note that http://user:pass@server/path/ /should/ work (but that >>>> would need validation), though not a good idea on command line. >>> Well, now that the programs using URLs are all builtins, we can >>> actually do something about it. We can edit out the "user[:pass]@" >>> part out of argv, which usually means that "ps" will not see it >>> anymore. >> Wouldn't there still be a race condition for publicly broadcasting your >> password via ps? > > Yes. For a brief amount of time, but yes. > > BTW I thought I remembered one program xxx'ing out the password via argv, > but unfortunately I do not remember which one it was. > The mysql client does (or did) it, although perhaps only on systems that support it. -- Andreas Ericsson andreas.ericsson@op5.se OP5 AB www.op5.se Tel: +46 8-230225 Fax: +46 8-230231 ```