threads / discuss / 18006

git with https and client cert asks for password repeatedly

Subject: git with https and client cert asks for password repeatedly

## tl;dr

4 messages between Feb 25, 2009 and Feb 26, 2009.

replies: 3people: 4as markdown or json

Mark Lodato· Feb 25, 2009, 03:11 UTC · lore

First off, I am fairly new to git, so let me apologize in advance if I suggest anything stupid.

When fetching or pushing over https:// with a client certificate (http.sslCert / http.sslKey), git asks for a password for every single requested file. For example, here I push three commits with a couple changed files each:

> git push origin master
Enter PEM pass phrase:
Enter PEM pass phrase:
Fetching remote heads...
  refs/
  refs/tags/
  refs/heads/
updating 'refs/heads/master'
  from 1df865db590b4a7d4991c13053437ac90b2780e4
  to   05e856a6a5ce9b05a5a7d10cb5d10010467eea72
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
    sending 12 objects
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
Enter PEM pass phrase:
    done
Updating remote server info

To make matters worse, when you try to CTRL-C from the "Enter PEM pass phrase" prompt, it just re-prompts you! If you want to see this in action, set up a webdav server on https://localhost with a copy of git.git and try cloning it with a password-protected client certificate.

This problem makes client-side certificates unusable with git. A possible workaround is to leave the key unencrypted, but this is usually unacceptable for security reasons. Ideally, I would just type my password once per invocation and git would remember it. (This is how svn works.)

I think the root problem is that git creates a completely new http(s) connection for every request, rather than using one persistent connection. Using a persistent connection would theoretically speed up the transfers, in addition to fixing the password prompt issue. I'm pretty sure that calling `curl_easy_cleanup()' after every request is causing this behavior; I don't think this is necessary.

I tried fixing this myself, but the http/curl code is pretty
confusing.  Just wondering - why is HTTP_MULTI required for http-push?
 I saw a thread from Jan '08 about this, but it never said *why*
HTTP_MULTI is required, only that the push doesn't work without it.
It doesn't appear to me that git uses concurrent connections in any
useful way, so I don't know why having a single connection would not
work.

Finally, is there interest in refactoring the http code to make it a little cleaner? That is, make a wrapper library around curl so that you can just call GET or POST or whatever and not worry about how to invoke curl?

-- Mark Lodato

Daniel Stenberg· Feb 25, 2009, 08:24 UTC · re: Mark Lodato · lore

Re: git with https and client cert asks for password repeatedly

On Tue, 24 Feb 2009, Mark Lodato wrote:
Show 6 quoted lines
> I think the root problem is that git creates a completely new http(s) 
> connection for every request, rather than using one persistent connection. 
> Using a persistent connection would theoretically speed up the transfers, in 
> addition to fixing the password prompt issue. I'm pretty sure that calling 
> `curl_easy_cleanup()' after every request is causing this behavior; I don't 
> think this is necessary.

This may or may not be true depending on several circumstances. You didn't mention what libcurl version nor did you say if you build git to use the multi interface. But assuming that you don't use multi or that the libcurl is fairly old, your conclusion is correct.

-- 
  / daniel.haxx.se
Josef Wolf· Feb 25, 2009, 21:42 UTC · re: Mark Lodato · lore

Re: git with https and client cert asks for password repeatedly

On Tue, Feb 24, 2009 at 10:11:40PM -0500, Mark Lodato wrote:
> When fetching or pushing over https:// with a client certificate
> (http.sslCert / http.sslKey), git asks for a password for every single
> requested file.  For example, here I push three commits with a couple
> changed files each:
[ ... ]
> This problem makes client-side certificates unusable with git.  A
> possible workaround is to leave the key unencrypted, but this is
> usually unacceptable for security reasons.

For the (pretty much standard) Basic authentication over https, it is even worse: you have to put your password in plain text into .netrc :-( Check out http://marc.info/?l=git&m=122426078301793&w=2 if you have a couple of minutes left.

> Ideally, I would just type
> my password once per invocation and git would remember it.  (This is
> how svn works.)
Yeah, that would be great!
Show 9 quoted lines
> I think the root problem is that git creates a completely new http(s)
> connection for every request, rather than using one persistent
> connection.  Using a persistent connection would theoretically speed
> up the transfers, in addition to fixing the password prompt issue.
> I'm pretty sure that calling `curl_easy_cleanup()' after every request
> is causing this behavior; I don't think this is necessary.
> 
> I tried fixing this myself, but the http/curl code is pretty
> confusing.  Just wondering - why is HTTP_MULTI required for http-push?

The integration of curl into git looks somewhat strange to me, also. It seems to implement some sort of request-queue on top of curl-multi and falls back to re-implement some curl-multi functionality in the case when only curl-easy is available. Or something like that. I have not looked too much into the details, and have already forgotten what I saw there.

Instead, I looked into curl-lib to implement some sort of credential caching. The idea was to implement something like get_basic_credentials() in the LWP::UserAgent perl module.

Once such a retrieval callback/cache would be implemented in curl, it would be pretty easy to make use of it from git without messing/breaking too much with the existing code.

So I went to the curl-library list. Please feel free to read the discussion on http://curl.haxx.se/mail/lib-2008-10/index.html#286

Check out http://curl.haxx.se/mail/lib-2008-11/index.html#38 for a first draft of the change I suggested.

Unfortunately, I got stuck implementing the regression tests for this change. Then I got short on time to proceed the work on this topic. Check out http://curl.haxx.se/mail/lib-2008-11/index.html#51 for the last state of affairs I posted. Once the regression tests are implemented, implementing the remaining functionality would be pretty much straight forward. But currently, I have no clue when I will get around to continue working on that.

> Finally, is there interest in refactoring the http code to make it a
> little cleaner?  That is, make a wrapper library around curl so that
> you can just call GET or POST or whatever and not worry about how to
> invoke curl?

As I mentioned above, I figured that implementing the callback method in curl-lib would probably be easier than understanding the usage of curl within git. So I'd say refactoring would be an improvement. But I bet the git gurus will flame me for saying that =:-S

Johannes Schindelin· Feb 26, 2009, 10:17 UTC · re: Josef Wolf · lore

Re: git with https and client cert asks for password repeatedly

Hi,

[Josef, it is extremely annoying that you culled the Cc: list; I do not know if Mark will get this mail, and my time is too scarce to undo your mistake.]

On Wed, 25 Feb 2009, Josef Wolf wrote:
Show 11 quoted lines
> On Tue, Feb 24, 2009 at 10:11:40PM -0500, Mark Lodato wrote:
> 
> > Finally, is there interest in refactoring the http code to make it a 
> > little cleaner?  That is, make a wrapper library around curl so that 
> > you can just call GET or POST or whatever and not worry about how to 
> > invoke curl?
> 
> As I mentioned above, I figured that implementing the callback method in 
> curl-lib would probably be easier than understanding the usage of curl 
> within git.  So I'd say refactoring would be an improvement.  But I bet 
> the git gurus will flame me for saying that =:-S

To the contrary. I have been waiting for over one year for Mike Hommey to submit his refactoring patches.

A very quick list search would find you his patch series, I am sure.

Status quo: the series needs polishing, and we need http-fetch tests (if I saw it correctly, Peff started something like this).

Ciao, Dscho

← back to recent threads