threads / patch / 10082

patchFix problem with authentification on http repository.

Subject: [PATCH] Fix problem with authentification on http repository.

## tl;dr

18 messages between Sep 29, 2007 and Jul 10, 2008. Diffs are folded; open one to read it.

replies: 17people: 7as markdown or json

jean.guyader@linkea.org· Sep 29, 2007, 22:26 UTC · lore
From: Jean Guyader <jean.guyader@gmail.com>

Curl uses the option -u user:passwd and not the user:password given in the url. The solution was to extract user:password from the url and set the option.

Here the regex used :
        sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'
---
 git-clone.sh     |    3 ++-
 git-fetch.sh     |    8 ++++----
 git-ls-remote.sh |    3 ++-
 3 files changed, 8 insertions(+), 6 deletions(-)
Show changes to 3 files +8 −6

git-clone.sh, git-fetch.sh, git-ls-remote.sh

diff --git a/git-clone.sh b/git-clone.sh
index 5e582fe..57206ac 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -34,7 +34,8 @@ fi
 
 http_fetch () {
 	# $1 = Remote, $2 = Local
-	curl -nsfL $curl_extra_args "$1" >"$2" ||
+        curl_userpw=`echo "$1" | sed -re 's-http://([^:]*):([^@]+)@.*-\1:\2-g'`
+	curl -u "$curl_userpw" -nsfL $curl_extra_args "$1" >"$2" ||
 		case $? in
 		126|127) exit ;;
 		*)	 return $? ;;
diff --git a/git-fetch.sh b/git-fetch.sh
index e44af2c..0b7d055 100755
--- a/git-fetch.sh
+++ b/git-fetch.sh
@@ -268,13 +268,13 @@ fetch_per_ref () {
 		"`git config --bool http.noEPSV`" = true ]; then
 	      noepsv_opt="--disable-epsv"
 	  fi
-
-	  # Find $remote_name from ls-remote output.
+          pretty_remote=$(echo "$remote" | sed -re 's-http://([^:]+):([^@]+)@-http://\1:******@-g')
+          # Find $remote_name from ls-remote output.
 	  head=$(echo "$ls_remote_result" | \
 		git fetch--tool -s pick-rref "$remote_name" "-")
 	  expr "z$head" : "z$_x40\$" >/dev/null ||
-		die "No such ref $remote_name at $remote"
-	  echo >&2 "Fetching $remote_name from $remote using $proto"
+		die "No such ref $remote_name at $pretty_remote"
+	  echo >&2 "Fetching $remote_name from $pretty_remote using $proto"
 	  case "$quiet" in '') v=-v ;; *) v= ;; esac
 	  git-http-fetch $v -a "$head" "$remote" || exit
 	  ;;
diff --git a/git-ls-remote.sh b/git-ls-remote.sh
index d56cf92..d6e9906 100755
--- a/git-ls-remote.sh
+++ b/git-ls-remote.sh
@@ -61,7 +61,8 @@ http://* | https://* | ftp://* )
 		"`git config --bool http.noEPSV`" = true ]; then
 		curl_extra_args="${curl_extra_args} --disable-epsv"
 	fi
-	curl -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" ||
+        curl_userpw=`echo "$peek_repo" | sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'`
+	curl -u "$curl_userpw" -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" ||
 		echo "failed	slurping"
 	;;
 
-- 
1.5.2.4
Johannes Schindelin· Sep 29, 2007, 23:10 UTC · re: jean.guyader@linkea.org · lore

Re: [PATCH] Fix problem with authentification on http repository.

Hi,
On Sun, 30 Sep 2007, jean.guyader@linkea.org wrote:
>  git-clone.sh     |    3 ++-
>  git-fetch.sh     |    8 ++++----
>  git-ls-remote.sh |    3 ++-

git-fetch is already a builtin in "next", and -clone and -ls-remote are likely to follow suit.

Ciao, Dscho

Junio C Hamano· Sep 29, 2007, 23:38 UTC · re: jean.guyader@linkea.org · lore

Re: [PATCH] Fix problem with authentification on http repository.

jean.guyader@linkea.org writes:
Show 9 quoted lines
> From: Jean Guyader <jean.guyader@gmail.com>
>
> Curl uses the option -u user:passwd and not the user:password
> given in the url.
> The solution was to extract user:password from the url and set
> the option.
>
> Here the regex used :
>         sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'

This is more like "allowing embedded authentication credentials in URL for http transport".

We never supported URLs with embedded credentials (see Documentation/urls.txt), partly because nobody asked for it, but more importantly because giving -n to curl to have it read from user's .netrc is generally much more preferred approach.

Johannes Schindelin· Sep 29, 2007, 23:40 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix problem with authentification on http repository.

Hi,
On Sat, 29 Sep 2007, Junio C Hamano wrote:
Show 19 quoted lines
> jean.guyader@linkea.org writes:
> 
> > From: Jean Guyader <jean.guyader@gmail.com>
> >
> > Curl uses the option -u user:passwd and not the user:password
> > given in the url.
> > The solution was to extract user:password from the url and set
> > the option.
> >
> > Here the regex used :
> >         sed -re 's-.*http://([^:]*):([^@]+)@.*-\1:\2-g'
> 
> This is more like "allowing embedded authentication credentials
> in URL for http transport".
> 
> We never supported URLs with embedded credentials (see
> Documentation/urls.txt), partly because nobody asked for it, but
> more importantly because giving -n to curl to have it read from
> user's .netrc is generally much more preferred approach.

To elaborate on that: if you fetch from somewhere, your url, username and password can be read from the output of "ps ax | grep http" very easily.

Ciao, Dscho

Junio C Hamano· Sep 30, 2007, 00:02 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Fix problem with authentification on http repository.

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 9 quoted lines
> On Sat, 29 Sep 2007, Junio C Hamano wrote:
>
>> We never supported URLs with embedded credentials (see
>> Documentation/urls.txt), partly because nobody asked for it, but
>> more importantly because giving -n to curl to have it read from
>> user's .netrc is generally much more preferred approach.
>
> To elaborate on that: if you fetch from somewhere, your url, username and 
> password can be read from the output of "ps ax | grep http" very easily.

Actually Documentation/howto/setup-git-server-over-http.txt talks about http://user@server/path/ format. How well does this work in practice? If it does, we should update Documentation/urls.txt to allow optional user@ there like...

---
 Documentation/urls.txt |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
Show changes to Documentation/urls.txt +2 −2
diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index e67f914..c9aab86 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -6,8 +6,8 @@ to name the remote repository:
 
 ===============================================================
 - rsync://host.xz/path/to/repo.git/
-- http://host.xz/path/to/repo.git/
-- https://host.xz/path/to/repo.git/
+- http://{startsb}user@{endsb}host.xz/path/to/repo.git/
+- https://{startsb}user@{endsb}host.xz/path/to/repo.git/
 - git://host.xz/path/to/repo.git/
 - git://host.xz/~user/path/to/repo.git/
 - ssh://{startsb}user@{endsb}host.xz{startsb}:port{endsb}/path/to/repo.git/
Junio C Hamano· Jul 9, 2008, 05:44 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix problem with authentification on http repository.

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 9 quoted lines
> On Sat, 29 Sep 2007, Junio C Hamano wrote:
>
>> We never supported URLs with embedded credentials (see
>> Documentation/urls.txt), partly because nobody asked for it, but
>> more importantly because giving -n to curl to have it read from
>> user's .netrc is generally much more preferred approach.
>
> To elaborate on that: if you fetch from somewhere, your url, username and 
> password can be read from the output of "ps ax | grep http" very easily.

Actually Documentation/howto/setup-git-server-over-http.txt talks about http://user@server/path/ format. How well does this work in practice? If it does, we should update Documentation/urls.txt to allow optional user@ there like...

---
 Documentation/urls.txt |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
Show changes to Documentation/urls.txt +2 −2
diff --git a/Documentation/urls.txt b/Documentation/urls.txt
index e67f914..c9aab86 100644
--- a/Documentation/urls.txt
+++ b/Documentation/urls.txt
@@ -6,8 +6,8 @@ to name the remote repository:
 
 ===============================================================
 - rsync://host.xz/path/to/repo.git/
-- http://host.xz/path/to/repo.git/
-- https://host.xz/path/to/repo.git/
+- http://{startsb}user@{endsb}host.xz/path/to/repo.git/
+- https://{startsb}user@{endsb}host.xz/path/to/repo.git/
 - git://host.xz/path/to/repo.git/
 - git://host.xz/~user/path/to/repo.git/
 - ssh://{startsb}user@{endsb}host.xz{startsb}:port{endsb}/path/to/repo.git/
Mike Hommey· Jul 9, 2008, 05:51 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix problem with authentification on http repository.

On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote:
Show 16 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Sat, 29 Sep 2007, Junio C Hamano wrote:
> >
> >> We never supported URLs with embedded credentials (see
> >> Documentation/urls.txt), partly because nobody asked for it, but
> >> more importantly because giving -n to curl to have it read from
> >> user's .netrc is generally much more preferred approach.
> >
> > To elaborate on that: if you fetch from somewhere, your url, username and 
> > password can be read from the output of "ps ax | grep http" very easily.
> 
> Actually Documentation/howto/setup-git-server-over-http.txt
> talks about http://user@server/path/ format.  How well does this
> work in practice?  If it does, we should update Documentation/urls.txt
> to allow optional user@ there like...

AFAIK, there is nothing to deal with asking a password to the user in git in this case, so that doesn't work.

Mike
Junio C Hamano· Jul 9, 2008, 06:28 UTC · re: Mike Hommey · lore

Re: [PATCH] Fix problem with authentification on http repository.

Mike Hommey <mh@glandium.org> writes:
Show 20 quoted lines
> On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote:
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > On Sat, 29 Sep 2007, Junio C Hamano wrote:
>> >
>> >> We never supported URLs with embedded credentials (see
>> >> Documentation/urls.txt), partly because nobody asked for it, but
>> >> more importantly because giving -n to curl to have it read from
>> >> user's .netrc is generally much more preferred approach.
>> >
>> > To elaborate on that: if you fetch from somewhere, your url, username and 
>> > password can be read from the output of "ps ax | grep http" very easily.
>> 
>> Actually Documentation/howto/setup-git-server-over-http.txt
>> talks about http://user@server/path/ format.  How well does this
>> work in practice?  If it does, we should update Documentation/urls.txt
>> to allow optional user@ there like...
>
> AFAIK, there is nothing to deal with asking a password to the user in git
> in this case, so that doesn't work.
Then perhaps the howto needs updating?
Mike Hommey· Jul 9, 2008, 07:53 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix problem with authentification on http repository.

On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
Show 24 quoted lines
> Mike Hommey <mh@glandium.org> writes:
> 
> > On Tue, Jul 08, 2008 at 10:44:58PM -0700, Junio C Hamano wrote:
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> 
> >> > On Sat, 29 Sep 2007, Junio C Hamano wrote:
> >> >
> >> >> We never supported URLs with embedded credentials (see
> >> >> Documentation/urls.txt), partly because nobody asked for it, but
> >> >> more importantly because giving -n to curl to have it read from
> >> >> user's .netrc is generally much more preferred approach.
> >> >
> >> > To elaborate on that: if you fetch from somewhere, your url, username and 
> >> > password can be read from the output of "ps ax | grep http" very easily.
> >> 
> >> Actually Documentation/howto/setup-git-server-over-http.txt
> >> talks about http://user@server/path/ format.  How well does this
> >> work in practice?  If it does, we should update Documentation/urls.txt
> >> to allow optional user@ there like...
> >
> > AFAIK, there is nothing to deal with asking a password to the user in git
> > in this case, so that doesn't work.
> 
> Then perhaps the howto needs updating?

Actually, the howto tells to setup a ~/.netrc file, which should make the http://user@server/path/ format work. I don't know if dropping user@ would still work, that depends how libcurl works with it.

Mike
Daniel Stenberg· Jul 9, 2008, 08:04 UTC · re: Mike Hommey · lore

Re: [PATCH] Fix problem with authentification on http repository.

On Wed, 9 Jul 2008, Mike Hommey wrote:
> Actually, the howto tells to setup a ~/.netrc file, which should make
> the http://user@server/path/ format work. I don't know if dropping user@
> would still work, that depends how libcurl works with it.

git has the netrc parser set to "optional", which makes libcurl use the login name given in the URL (if present) and search for the associated password in the .netrc file.

Typical users probably only have one user per host name in the .netrc file and then the name part in the URL provides no particular extra goodness but can just as well be excluded.

-- 
  / daniel.haxx.se
Junio C Hamano· Jul 9, 2008, 08:18 UTC · re: Mike Hommey · lore

Re: [PATCH] Fix problem with authentification on http repository.

Mike Hommey <mh@glandium.org> writes:
Show 15 quoted lines
> On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> >> Actually Documentation/howto/setup-git-server-over-http.txt
>> >> talks about http://user@server/path/ format.  How well does this
>> >> work in practice?  If it does, we should update Documentation/urls.txt
>> >> to allow optional user@ there like...
>> >
>> > AFAIK, there is nothing to deal with asking a password to the user in git
>> > in this case, so that doesn't work.
>> 
>> Then perhaps the howto needs updating?
>
> Actually, the howto tells to setup a ~/.netrc file, which should make
> the http://user@server/path/ format work. I don't know if dropping user@
> would still work, that depends how libcurl works with it.

Ok, so the documentation update I suggested earlier may actually be valid, right?

Mike Hommey· Jul 9, 2008, 08:56 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix problem with authentification on http repository.

On Wed, Jul 09, 2008 at 01:18:22AM -0700, Junio C Hamano <gitster@pobox.com> wrote:
Show 20 quoted lines
> Mike Hommey <mh@glandium.org> writes:
> 
> > On Tue, Jul 08, 2008 at 11:28:15PM -0700, Junio C Hamano <gitster@pobox.com> wrote:
> > ...
> >> >> Actually Documentation/howto/setup-git-server-over-http.txt
> >> >> talks about http://user@server/path/ format.  How well does this
> >> >> work in practice?  If it does, we should update Documentation/urls.txt
> >> >> to allow optional user@ there like...
> >> >
> >> > AFAIK, there is nothing to deal with asking a password to the user in git
> >> > in this case, so that doesn't work.
> >> 
> >> Then perhaps the howto needs updating?
> >
> > Actually, the howto tells to setup a ~/.netrc file, which should make
> > the http://user@server/path/ format work. I don't know if dropping user@
> > would still work, that depends how libcurl works with it.
> 
> Ok, so the documentation update I suggested earlier may actually be
> valid, right?

The problem is that these urls won't work out of the box if the user doesn't setup a .netrc file. I'd say it would be better not to add these forms in Documentation/urls.txt until git prompts for the password when it is missing.

Note that http://user:pass@server/path/ /should/ work (but that would need validation), though not a good idea on command line.

Mike
Johannes Schindelin· Jul 9, 2008, 11:43 UTC · re: Mike Hommey · lore

Re: [PATCH] Fix problem with authentification on http repository.

Hi,
On Wed, 9 Jul 2008, Mike Hommey wrote:
> Note that http://user:pass@server/path/ /should/ work (but that would 
> need validation), though not a good idea on command line.

Well, now that the programs using URLs are all builtins, we can actually do something about it. We can edit out the "user[:pass]@" part out of argv, which usually means that "ps" will not see it anymore.

Ciao, Dscho

Jeff King· Jul 10, 2008, 07:24 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Fix problem with authentification on http repository.

On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
Show 6 quoted lines
> > Note that http://user:pass@server/path/ /should/ work (but that would 
> > need validation), though not a good idea on command line.
> 
> Well, now that the programs using URLs are all builtins, we can actually 
> do something about it.  We can edit out the "user[:pass]@" part out of 
> argv, which usually means that "ps" will not see it anymore.

Wouldn't there still be a race condition for publicly broadcasting your password via ps?

-Peff
Junio C Hamano· Jul 10, 2008, 07:32 UTC · re: Jeff King · lore

Re: [PATCH] Fix problem with authentification on http repository.

Jeff King <peff@peff.net> writes:
Show 11 quoted lines
> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
>
>> > Note that http://user:pass@server/path/ /should/ work (but that would 
>> > need validation), though not a good idea on command line.
>> 
>> Well, now that the programs using URLs are all builtins, we can actually 
>> do something about it.  We can edit out the "user[:pass]@" part out of 
>> argv, which usually means that "ps" will not see it anymore.
>
> Wouldn't there still be a race condition for publicly broadcasting your
> password via ps?

In addition, I think on some Unices overwriting argv[0] does not have the effect Dscho is aiming for.

I thought we do discourage [:pass] part in our documentation.
Jeff King· Jul 10, 2008, 08:47 UTC · re: Junio C Hamano · lore

Re: [PATCH] Fix problem with authentification on http repository.

On Thu, Jul 10, 2008 at 12:32:27AM -0700, Junio C Hamano wrote:
Show 5 quoted lines
> > Wouldn't there still be a race condition for publicly broadcasting your
> > password via ps?
> 
> In addition, I think on some Unices overwriting argv[0] does not have the
> effect Dscho is aiming for.

I thought so, too, but I couldn't find any mention of it online. For curiosity's sake, do you happen to know offhand any platform where this doesn't work (of course, Dscho is really proposing overwriting argv[i], but I don't know if that makes a difference to the portability)?

-Peff
Johannes Schindelin· Jul 10, 2008, 10:51 UTC · re: Jeff King · lore

Re: [PATCH] Fix problem with authentification on http repository.

Hi,
On Thu, 10 Jul 2008, Jeff King wrote:
Show 12 quoted lines
> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
> 
> > > Note that http://user:pass@server/path/ /should/ work (but that 
> > > would need validation), though not a good idea on command line.
> > 
> > Well, now that the programs using URLs are all builtins, we can 
> > actually do something about it.  We can edit out the "user[:pass]@" 
> > part out of argv, which usually means that "ps" will not see it 
> > anymore.
> 
> Wouldn't there still be a race condition for publicly broadcasting your 
> password via ps?
Yes.  For a brief amount of time, but yes.

BTW I thought I remembered one program xxx'ing out the password via argv, but unfortunately I do not remember which one it was.

Anyway, was just an idea.

Ciao, Dscho

Andreas Ericsson· Jul 10, 2008, 11:30 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Fix problem with authentification on http repository.

Johannes Schindelin wrote:
Show 20 quoted lines
> Hi,
> 
> On Thu, 10 Jul 2008, Jeff King wrote:
> 
>> On Wed, Jul 09, 2008 at 01:43:07PM +0200, Johannes Schindelin wrote:
>>
>>>> Note that http://user:pass@server/path/ /should/ work (but that 
>>>> would need validation), though not a good idea on command line.
>>> Well, now that the programs using URLs are all builtins, we can 
>>> actually do something about it.  We can edit out the "user[:pass]@" 
>>> part out of argv, which usually means that "ps" will not see it 
>>> anymore.
>> Wouldn't there still be a race condition for publicly broadcasting your 
>> password via ps?
> 
> Yes.  For a brief amount of time, but yes.
> 
> BTW I thought I remembered one program xxx'ing out the password via argv, 
> but unfortunately I do not remember which one it was.
> 

The mysql client does (or did) it, although perhaps only on systems that support it.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

← back to recent threads