threads / discuss / 48708

OAuth2 support in git?

Subject: OAuth2 support in git?

## tl;dr

12 messages between Jun 14, 2018 and Jun 19, 2018.

replies: 11people: 6as markdown or json

Christian Halstrick· Jun 14, 2018, 08:09 UTC · lore

Can I use native git as client to contact a git server which does authentication with OAuth2 Client Credentials Grant [1]?

Background: We are running gerrit based git servers [2] in a cloud
environment. That environment supports OAuth2 authorization for the
apps running in the cloud. The idea is that clients (e.g. jenkins
jobs) talking git over http with such git servers should be able to
use OAuth2 tokens to authenticate clone/fetch requests. We would have
to adapt gerrit source code for token handling/validation but I am
asking here about the client side.

I know that other git server environments like github support that on client side by allowing tokens to be used as usernames in a BASIC authentication flow. We could do the same but I am asking whether there is also a way to transport tokens in a standard conform "Authorization: Bearer ..." Header field.

[1] https://tools.ietf.org/html/rfc6749#section-4.4 [2] https://www.gerritcodereview.com/

brian m. carlson· Jun 14, 2018, 10:13 UTC · re: Christian Halstrick · lore

Re: OAuth2 support in git?

On Thu, Jun 14, 2018 at 10:09:39AM +0200, Christian Halstrick wrote:
Show 16 quoted lines
> Can I use native git as client to contact a git server which does
> authentication with OAuth2 Client Credentials Grant [1]?
> 
> Background: We are running gerrit based git servers [2] in a cloud
> environment. That environment supports OAuth2 authorization for the
> apps running in the cloud. The idea is that clients (e.g. jenkins
> jobs) talking git over http with such git servers should be able to
> use OAuth2 tokens to authenticate clone/fetch requests. We would have
> to adapt gerrit source code for token handling/validation but I am
> asking here about the client side.
> 
> I know that other git server environments like github support that on
> client side by allowing tokens to be used as usernames in a BASIC
> authentication flow. We could do the same but I am asking whether
> there is also a way to transport tokens in a standard conform
> "Authorization: Bearer ..." Header field.

There isn't any support for Bearer authentication in Git. For HTTP, we use libcurl, which doesn't provide this natively. While it could in theory be added, it would require some reworking of the auth code.

You are, of course, welcome to send a patch.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Jeff King· Jun 14, 2018, 15:15 UTC · re: brian m. carlson · lore

Re: OAuth2 support in git?

On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
Show 11 quoted lines
> > I know that other git server environments like github support that on
> > client side by allowing tokens to be used as usernames in a BASIC
> > authentication flow. We could do the same but I am asking whether
> > there is also a way to transport tokens in a standard conform
> > "Authorization: Bearer ..." Header field.
> 
> There isn't any support for Bearer authentication in Git.  For HTTP, we
> use libcurl, which doesn't provide this natively.  While it could in
> theory be added, it would require some reworking of the auth code.
> 
> You are, of course, welcome to send a patch.

If it's just a custom Authorization header, we should be able to support it with existing curl versions without _too_ much effort.

I think there are probably two possible directions:
 1. add a special "bearer" command line option, etc, as a string
 2. add a boolean option to send the existing "password" field as a
    "bearer" header

I suspect (2) would fit in with the existing code better, as the special case would mostly be limited to the manner in which we feed the credential to curl. And you could probably just set a config option for "this url's auth will be oauth2", and use the existing mechanisms for providing the password.

We'd maybe also want to allow credential helpers to say "by the way, this password should be treated as a bearer token", for cases where you might sometimes use oauth2 and sometimes a real password.

-Peff
Randall S. Becker· Jun 14, 2018, 20:46 UTC · re: Jeff King · lore

RE: OAuth2 support in git?

On June 14, 2018 11:15 AM, Jeff King wrote:
Show 32 quoted lines
> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
> 
> > > I know that other git server environments like github support that
> > > on client side by allowing tokens to be used as usernames in a BASIC
> > > authentication flow. We could do the same but I am asking whether
> > > there is also a way to transport tokens in a standard conform
> > > "Authorization: Bearer ..." Header field.
> >
> > There isn't any support for Bearer authentication in Git.  For HTTP,
> > we use libcurl, which doesn't provide this natively.  While it could
> > in theory be added, it would require some reworking of the auth code.
> >
> > You are, of course, welcome to send a patch.
> 
> If it's just a custom Authorization header, we should be able to support it
> with existing curl versions without _too_ much effort.
> 
> I think there are probably two possible directions:
> 
>  1. add a special "bearer" command line option, etc, as a string
> 
>  2. add a boolean option to send the existing "password" field as a
>     "bearer" header
> 
> I suspect (2) would fit in with the existing code better, as the special case
> would mostly be limited to the manner in which we feed the credential to
> curl. And you could probably just set a config option for "this url's auth will
> be oauth2", and use the existing mechanisms for providing the password.
> 
> We'd maybe also want to allow credential helpers to say "by the way, this
> password should be treated as a bearer token", for cases where you might
> sometimes use oauth2 and sometimes a real password.
Be aware that there are 4 (ish) flavours of OAuth2 the last time I checked. It is important to know which one (or all) to implement. The embedded form is probably the easiest to comprehend - and the least implemented from my research. More common OAuth2 instances use a third-man website to hold session keys and authorization. That may be problematic for a whole bunch of us who do not play in that world.

Cheers, Randall

-- Brief whoami:
  NonStop developer since approximately NonStop(211288444200000000)
  UNIX developer since approximately 421664400
-- In my real life, I talk too much.
Jeff King· Jun 14, 2018, 21:01 UTC · re: Randall S. Becker · lore

Re: OAuth2 support in git?

On Thu, Jun 14, 2018 at 04:46:10PM -0400, Randall S. Becker wrote:
Show 15 quoted lines
> > I suspect (2) would fit in with the existing code better, as the special case
> > would mostly be limited to the manner in which we feed the credential to
> > curl. And you could probably just set a config option for "this url's auth will
> > be oauth2", and use the existing mechanisms for providing the password.
> > 
> > We'd maybe also want to allow credential helpers to say "by the way, this
> > password should be treated as a bearer token", for cases where you might
> > sometimes use oauth2 and sometimes a real password.
> 
> Be aware that there are 4 (ish) flavours of OAuth2 the last time I
> checked. It is important to know which one (or all) to implement. The
> embedded form is probably the easiest to comprehend - and the least
> implemented from my research. More common OAuth2 instances use a
> third-man website to hold session keys and authorization. That may be
> problematic for a whole bunch of us who do not play in that world.

I think Git's usage would be limited to "how do I present this token for my requests". I don't think we'd ever recognize an oauth redirect and try to fulfill it ourselves. We'd rely on getting a 401 and punting all those bits to a credential helper to do the heavy lifting.

I say that not knowing much about oauth2, of course, so maybe there would be complications with that approach (I do know there are multiple ways you can present a token, but we'd support whichever ones people are interested in enough to show up and provide a patch for).

-Peff
brian m. carlson· Jun 14, 2018, 22:20 UTC · re: Jeff King · lore

Re: OAuth2 support in git?

On Thu, Jun 14, 2018 at 11:15:07AM -0400, Jeff King wrote:
Show 9 quoted lines
> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
> > There isn't any support for Bearer authentication in Git.  For HTTP, we
> > use libcurl, which doesn't provide this natively.  While it could in
> > theory be added, it would require some reworking of the auth code.
> > 
> > You are, of course, welcome to send a patch.
> 
> If it's just a custom Authorization header, we should be able to support
> it with existing curl versions without _too_ much effort.

It shouldn't be too difficult, but we have some fallback among various authentication types that would need reworking.

Show 12 quoted lines
> I think there are probably two possible directions:
> 
>  1. add a special "bearer" command line option, etc, as a string
> 
>  2. add a boolean option to send the existing "password" field as a
>     "bearer" header
> 
> I suspect (2) would fit in with the existing code better, as the special
> case would mostly be limited to the manner in which we feed the
> credential to curl. And you could probably just set a config option for
> "this url's auth will be oauth2", and use the existing mechanisms for
> providing the password.
I agree option (2) would be better.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Johannes Schindelin· Jun 17, 2018, 11:37 UTC · re: Jeff King · lore

Re: OAuth2 support in git?

Hi Peff,
On Thu, 14 Jun 2018, Jeff King wrote:
Show 16 quoted lines
> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
> 
> > > I know that other git server environments like github support that on
> > > client side by allowing tokens to be used as usernames in a BASIC
> > > authentication flow. We could do the same but I am asking whether
> > > there is also a way to transport tokens in a standard conform
> > > "Authorization: Bearer ..." Header field.
> > 
> > There isn't any support for Bearer authentication in Git.  For HTTP, we
> > use libcurl, which doesn't provide this natively.  While it could in
> > theory be added, it would require some reworking of the auth code.
> > 
> > You are, of course, welcome to send a patch.
> 
> If it's just a custom Authorization header, we should be able to support
> it with existing curl versions without _too_ much effort.
Indeed. Because it is already implemented:
	git -c http.extraheader="Authorization: Bearer ..." ...
To make this a *little* safer, you can use http.<URL>.extraheader.

Ciao, Dscho

Jeff King· Jun 18, 2018, 04:17 UTC · re: Johannes Schindelin · lore

Re: OAuth2 support in git?

On Sun, Jun 17, 2018 at 01:37:24PM +0200, Johannes Schindelin wrote:
Show 8 quoted lines
> > If it's just a custom Authorization header, we should be able to support
> > it with existing curl versions without _too_ much effort.
> 
> Indeed. Because it is already implemented:
> 
> 	git -c http.extraheader="Authorization: Bearer ..." ...
> 
> To make this a *little* safer, you can use http.<URL>.extraheader.
Yeah, that will work for some cases. A few places it might not:
 - some people may want to provide this only in response to a 401
 - some tokens may need to be refreshed, which would require interacting
   with a credential helper to do the rest of the oauth conversation
 - there's no good way to hide your token in secure storage (versus
   sticking it on the command-line or in a config file).
-Peff
Junio C Hamano· Jun 18, 2018, 15:53 UTC · re: Jeff King · lore

Re: OAuth2 support in git?

Jeff King <peff@peff.net> writes:
Show 20 quoted lines
> On Sun, Jun 17, 2018 at 01:37:24PM +0200, Johannes Schindelin wrote:
>
>> > If it's just a custom Authorization header, we should be able to support
>> > it with existing curl versions without _too_ much effort.
>> 
>> Indeed. Because it is already implemented:
>> 
>> 	git -c http.extraheader="Authorization: Bearer ..." ...
>> 
>> To make this a *little* safer, you can use http.<URL>.extraheader.
>
> Yeah, that will work for some cases. A few places it might not:
>
>  - some people may want to provide this only in response to a 401
>
>  - some tokens may need to be refreshed, which would require interacting
>    with a credential helper to do the rest of the oauth conversation
>
>  - there's no good way to hide your token in secure storage (versus
>    sticking it on the command-line or in a config file).

And all of these three are what you get for free by building on the credential helper framework, after extending it a bit so that the filled credential structure can tell the http code to show it to the other side as a bearer token, not a password or password hash. The helper is asked to supply the auth material only after 401, which covers both the first and the second points, and then keeping the auth material in-core (e.g. cache--daemon) would be more secure which covers the third point. Am I following you correctly?

Thanks.
Jeff King· Jun 18, 2018, 21:26 UTC · re: Junio C Hamano · lore

Re: OAuth2 support in git?

On Mon, Jun 18, 2018 at 08:53:27AM -0700, Junio C Hamano wrote:
Show 18 quoted lines
> > Yeah, that will work for some cases. A few places it might not:
> >
> >  - some people may want to provide this only in response to a 401
> >
> >  - some tokens may need to be refreshed, which would require interacting
> >    with a credential helper to do the rest of the oauth conversation
> >
> >  - there's no good way to hide your token in secure storage (versus
> >    sticking it on the command-line or in a config file).
> 
> And all of these three are what you get for free by building on the
> credential helper framework, after extending it a bit so that the
> filled credential structure can tell the http code to show it to the
> other side as a bearer token, not a password or password hash.  The
> helper is asked to supply the auth material only after 401, which
> covers both the first and the second points, and then keeping the
> auth material in-core (e.g. cache--daemon) would be more secure
> which covers the third point.  Am I following you correctly?
Yes, exactly.

Even if the credential protocol itself doesn't learn about this feature, even a config option for "treat password as token to send via bearer" would help. The "how" of sending the token isn't secret, just the token itself. So everything else can just pretend it's a password (it's a little funny because I think there isn't a matching username, but you could probably get by with an empty one).

That's all just off the top of my head without digging back into the code, nor running any experiments, of course. There may be some gotchas. :)

-Peff
Christian Halstrick· Jun 19, 2018, 12:36 UTC · re: Jeff King · lore

Re: OAuth2 support in git?

What is not clear to me is how we can make use of the servers initial response in order control which credential helper to call and how to transport the credentials.

Imagine we try to clone over http. The initial request sent to the server may not contain a "Authorization: ..." header and the server responds with Unauthorized. But the server response contains hints like a "WWW-Authenticate: Basic realm=..." line or a "WWW-Authenticate: Bearer realm=..." line which helps choosing the authentication scheme used next. Maybe the server even responds with both lines telling I would accept BASIC or BEARER.

I can imagine that we want libcurl to deal with that decisions. But even then. How do we make sure the our credential helpers can act return either user/password or bearer tokens based on the server response? If credential helper would have access to the servers response (or only relevant parts of it?) it could decide whether to feel responsible for that server or not and what data to return.

And if credential helper could optionally give metadata about the kind credential they offer (e.g. "I return user/password" or "I return a bearer token") then core code could know where to transport this data. E.g. in a "Authorization: Basic ..." or a "Authorization: Bearer ..." field.

Ciao
  Chris
Jeff King· Jun 19, 2018, 16:45 UTC · re: Christian Halstrick · lore

Re: OAuth2 support in git?

On Tue, Jun 19, 2018 at 02:36:50PM +0200, Christian Halstrick wrote:
> What is not clear to me is how we can make use of the servers initial
> response in order control which credential helper to call and how to
> transport the credentials.

I don't think we'd ever decide _which_ credential helper to call; we always call all of them, in order, and then quit when we have sufficient credentials to continue.

But potentially we could feed some extra information to each helper and let it decide what to do.

Show 7 quoted lines
> Imagine we try to clone over http. The initial request sent to the
> server may not contain a "Authorization: ..." header and the server
> responds with Unauthorized.  But the server response contains hints
> like a "WWW-Authenticate: Basic realm=..." line or a
> "WWW-Authenticate: Bearer realm=..." line which helps choosing the
> authentication scheme used next. Maybe the server even responds with
> both lines telling I would accept BASIC or BEARER.

So for this example, yeah, I think it might make sense to feed the credential helper extra context like "authtype=basic" or similar. Most helpers would ignore it, but smart ones could make a decision based on it.

And then the response could contain a similar "authtype" key in the response.

Show 12 quoted lines
> I can imagine that we want libcurl to deal with that decisions. But
> even then. How do we make sure the our credential helpers can act
> return either user/password or bearer tokens based on the server
> response? If credential helper would have access to the servers
> response (or only relevant parts of it?) it could decide whether to
> feel responsible for that server or not and what data to return.
> 
> And if credential helper could optionally give metadata about the kind
> credential they offer (e.g. "I return user/password" or "I return a
> bearer token") then core code could know where to transport this data.
> E.g. in a "Authorization: Basic ..." or a "Authorization: Bearer ..."
> field.

Yep, I think that all matches my general line of thinking. It would help if we had some concrete cases. In particular, it's unclear to me if:

  1. A config option to say "treat password as a bearer token" would be
     enough.
  2. We'd need the credential helper to say "I'm giving you a token"
     versus "I'm giving you a password".
  3. We might need _both_ (1) and (2), because some servers would be
     fine with (1) and it lets them Just Work with credential helpers
     that are unaware of bearer tokens in the first place.

I suspect the answer is (3), but I'd probably delay working on (2) until I saw a situation that really needed it. :)

But I think we're on the same page, so if you're looking into or developing more concrete cases, those answers should become more clear.

-Peff

← back to recent threads