{"thread":{"id":"48708","subject":"OAuth2 support in git?","startedAt":"2018-06-14T08:10:05Z","lastAt":"2018-06-19T16:45:46Z","messageCount":12,"participants":["Christian Halstrick","brian m. carlson","Jeff King","Randall S. Becker","Johannes Schindelin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"350143","messageId":"CAENte7iUYcLX1ym1rdiYT2L8yLSWforf8kUvfHKLvhi_GhKQvg@mail.gmail.com","threadId":"48708","inReplyTo":null,"subject":"OAuth2 support in git?","fromName":"Christian Halstrick","fromEmail":"christian.halstrick@gmail.com","sentAt":"2018-06-14T08:09:39Z","receivedAt":"2018-06-14T08:10:05Z","isPatch":false,"sender":{"key":"christian.halstrick@gmail.com","avatar":"https://gravatar.com/avatar/3598bf518644c7dc32d4dcd8e0554b6a313e12103011862402c83ae2854203ce?d=mp&s=160"},"body":"Can I use native git as client to contact a git server which does\nauthentication with OAuth2 Client Credentials Grant [1]?\n\nBackground: We are running gerrit based git servers [2] in a cloud\nenvironment. That environment supports OAuth2 authorization for the\napps running in the cloud. The idea is that clients (e.g. jenkins\njobs) talking git over http with such git servers should be able to\nuse OAuth2 tokens to authenticate clone/fetch requests. We would have\nto adapt gerrit source code for token handling/validation but I am\nasking here about the client side.\n\nI know that other git server environments like github support that on\nclient side by allowing tokens to be used as usernames in a BASIC\nauthentication flow. We could do the same but I am asking whether\nthere is also a way to transport tokens in a standard conform\n\"Authorization: Bearer ...\" Header field.\n\n[1] https://tools.ietf.org/html/rfc6749#section-4.4\n[2] https://www.gerritcodereview.com/\n"},{"id":"350145","messageId":"20180614101342.GO38834@genre.crustytoothpaste.net","threadId":"48708","inReplyTo":"CAENte7iUYcLX1ym1rdiYT2L8yLSWforf8kUvfHKLvhi_GhKQvg@mail.gmail.com","subject":"Re: OAuth2 support in git?","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-06-14T10:13:42Z","receivedAt":"2018-06-14T10:13:54Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Thu, Jun 14, 2018 at 10:09:39AM +0200, Christian Halstrick wrote:\n> Can I use native git as client to contact a git server which does\n> authentication with OAuth2 Client Credentials Grant [1]?\n> \n> Background: We are running gerrit based git servers [2] in a cloud\n> environment. That environment supports OAuth2 authorization for the\n> apps running in the cloud. The idea is that clients (e.g. jenkins\n> jobs) talking git over http with such git servers should be able to\n> use OAuth2 tokens to authenticate clone/fetch requests. We would have\n> to adapt gerrit source code for token handling/validation but I am\n> asking here about the client side.\n> \n> I know that other git server environments like github support that on\n> client side by allowing tokens to be used as usernames in a BASIC\n> authentication flow. We could do the same but I am asking whether\n> there is also a way to transport tokens in a standard conform\n> \"Authorization: Bearer ...\" Header field.\n\nThere isn't any support for Bearer authentication in Git.  For HTTP, we\nuse libcurl, which doesn't provide this natively.  While it could in\ntheory be added, it would require some reworking of the auth code.\n\nYou are, of course, welcome to send a patch.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"350154","messageId":"20180614151507.GA6933@sigill.intra.peff.net","threadId":"48708","inReplyTo":"20180614101342.GO38834@genre.crustytoothpaste.net","subject":"Re: OAuth2 support in git?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-06-14T15:15:07Z","receivedAt":"2018-06-14T15:15:12Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:\n\n> > I know that other git server environments like github support that on\n> > client side by allowing tokens to be used as usernames in a BASIC\n> > authentication flow. We could do the same but I am asking whether\n> > there is also a way to transport tokens in a standard conform\n> > \"Authorization: Bearer ...\" Header field.\n> \n> There isn't any support for Bearer authentication in Git.  For HTTP, we\n> use libcurl, which doesn't provide this natively.  While it could in\n> theory be added, it would require some reworking of the auth code.\n> \n> You are, of course, welcome to send a patch.\n\nIf it's just a custom Authorization header, we should be able to support\nit with existing curl versions without _too_ much effort.\n\nI think there are probably two possible directions:\n\n 1. add a special \"bearer\" command line option, etc, as a string\n\n 2. add a boolean option to send the existing \"password\" field as a\n    \"bearer\" header\n\nI suspect (2) would fit in with the existing code better, as the special\ncase would mostly be limited to the manner in which we feed the\ncredential to curl. And you could probably just set a config option for\n\"this url's auth will be oauth2\", and use the existing mechanisms for\nproviding the password.\n\nWe'd maybe also want to allow credential helpers to say \"by the way,\nthis password should be treated as a bearer token\", for cases where you\nmight sometimes use oauth2 and sometimes a real password.\n\n-Peff\n"},{"id":"350202","messageId":"003c01d40420$bd522990$37f67cb0$@nexbridge.com","threadId":"48708","inReplyTo":"20180614151507.GA6933@sigill.intra.peff.net","subject":"RE: OAuth2 support in git?","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2018-06-14T20:46:10Z","receivedAt":"2018-06-14T20:46:33Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On June 14, 2018 11:15 AM, Jeff King wrote:\n> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:\n> \n> > > I know that other git server environments like github support that\n> > > on client side by allowing tokens to be used as usernames in a BASIC\n> > > authentication flow. We could do the same but I am asking whether\n> > > there is also a way to transport tokens in a standard conform\n> > > \"Authorization: Bearer ...\" Header field.\n> >\n> > There isn't any support for Bearer authentication in Git.  For HTTP,\n> > we use libcurl, which doesn't provide this natively.  While it could\n> > in theory be added, it would require some reworking of the auth code.\n> >\n> > You are, of course, welcome to send a patch.\n> \n> If it's just a custom Authorization header, we should be able to support it\n> with existing curl versions without _too_ much effort.\n> \n> I think there are probably two possible directions:\n> \n>  1. add a special \"bearer\" command line option, etc, as a string\n> \n>  2. add a boolean option to send the existing \"password\" field as a\n>     \"bearer\" header\n> \n> I suspect (2) would fit in with the existing code better, as the special case\n> would mostly be limited to the manner in which we feed the credential to\n> curl. And you could probably just set a config option for \"this url's auth will\n> be oauth2\", and use the existing mechanisms for providing the password.\n> \n> We'd maybe also want to allow credential helpers to say \"by the way, this\n> password should be treated as a bearer token\", for cases where you might\n> sometimes use oauth2 and sometimes a real password.\n\nBe aware that there are 4 (ish) flavours of OAuth2 the last time I checked. It is important to know which one (or all) to implement. The embedded form is probably the easiest to comprehend - and the least implemented from my research. More common OAuth2 instances use a third-man website to hold session keys and authorization. That may be problematic for a whole bunch of us who do not play in that world.\n\nCheers,\nRandall\n\n-- Brief whoami:\n  NonStop developer since approximately NonStop(211288444200000000)\n  UNIX developer since approximately 421664400\n-- In my real life, I talk too much.\n\n\n\n"},{"id":"350204","messageId":"20180614210132.GA12460@sigill.intra.peff.net","threadId":"48708","inReplyTo":"003c01d40420$bd522990$37f67cb0$@nexbridge.com","subject":"Re: OAuth2 support in git?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-06-14T21:01:33Z","receivedAt":"2018-06-14T21:01:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jun 14, 2018 at 04:46:10PM -0400, Randall S. Becker wrote:\n\n> > I suspect (2) would fit in with the existing code better, as the special case\n> > would mostly be limited to the manner in which we feed the credential to\n> > curl. And you could probably just set a config option for \"this url's auth will\n> > be oauth2\", and use the existing mechanisms for providing the password.\n> > \n> > We'd maybe also want to allow credential helpers to say \"by the way, this\n> > password should be treated as a bearer token\", for cases where you might\n> > sometimes use oauth2 and sometimes a real password.\n> \n> Be aware that there are 4 (ish) flavours of OAuth2 the last time I\n> checked. It is important to know which one (or all) to implement. The\n> embedded form is probably the easiest to comprehend - and the least\n> implemented from my research. More common OAuth2 instances use a\n> third-man website to hold session keys and authorization. That may be\n> problematic for a whole bunch of us who do not play in that world.\n\nI think Git's usage would be limited to \"how do I present this token for\nmy requests\". I don't think we'd ever recognize an oauth redirect and\ntry to fulfill it ourselves.  We'd rely on getting a 401 and punting all\nthose bits to a credential helper to do the heavy lifting.\n\nI say that not knowing much about oauth2, of course, so maybe there\nwould be complications with that approach (I do know there are multiple\nways you can present a token, but we'd support whichever ones people are\ninterested in enough to show up and provide a patch for).\n\n-Peff\n"},{"id":"350210","messageId":"20180614222000.GA622873@genre.crustytoothpaste.net","threadId":"48708","inReplyTo":"20180614151507.GA6933@sigill.intra.peff.net","subject":"Re: OAuth2 support in git?","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-06-14T22:20:00Z","receivedAt":"2018-06-14T22:20:09Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Thu, Jun 14, 2018 at 11:15:07AM -0400, Jeff King wrote:\n> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:\n> > There isn't any support for Bearer authentication in Git.  For HTTP, we\n> > use libcurl, which doesn't provide this natively.  While it could in\n> > theory be added, it would require some reworking of the auth code.\n> > \n> > You are, of course, welcome to send a patch.\n> \n> If it's just a custom Authorization header, we should be able to support\n> it with existing curl versions without _too_ much effort.\n\nIt shouldn't be too difficult, but we have some fallback among various\nauthentication types that would need reworking.\n\n> I think there are probably two possible directions:\n> \n>  1. add a special \"bearer\" command line option, etc, as a string\n> \n>  2. add a boolean option to send the existing \"password\" field as a\n>     \"bearer\" header\n> \n> I suspect (2) would fit in with the existing code better, as the special\n> case would mostly be limited to the manner in which we feed the\n> credential to curl. And you could probably just set a config option for\n> \"this url's auth will be oauth2\", and use the existing mechanisms for\n> providing the password.\n\nI agree option (2) would be better.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"350354","messageId":"nycvar.QRO.7.76.6.1806171335480.77@tvgsbejvaqbjf.bet","threadId":"48708","inReplyTo":"20180614151507.GA6933@sigill.intra.peff.net","subject":"Re: OAuth2 support in git?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2018-06-17T11:37:24Z","receivedAt":"2018-06-17T11:37:39Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Thu, 14 Jun 2018, Jeff King wrote:\n\n> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:\n> \n> > > I know that other git server environments like github support that on\n> > > client side by allowing tokens to be used as usernames in a BASIC\n> > > authentication flow. We could do the same but I am asking whether\n> > > there is also a way to transport tokens in a standard conform\n> > > \"Authorization: Bearer ...\" Header field.\n> > \n> > There isn't any support for Bearer authentication in Git.  For HTTP, we\n> > use libcurl, which doesn't provide this natively.  While it could in\n> > theory be added, it would require some reworking of the auth code.\n> > \n> > You are, of course, welcome to send a patch.\n> \n> If it's just a custom Authorization header, we should be able to support\n> it with existing curl versions without _too_ much effort.\n\nIndeed. Because it is already implemented:\n\n\tgit -c http.extraheader=\"Authorization: Bearer ...\" ...\n\nTo make this a *little* safer, you can use http.<URL>.extraheader.\n\nCiao,\nDscho\n"},{"id":"350370","messageId":"20180618041713.GA31125@sigill.intra.peff.net","threadId":"48708","inReplyTo":"nycvar.QRO.7.76.6.1806171335480.77@tvgsbejvaqbjf.bet","subject":"Re: OAuth2 support in git?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-06-18T04:17:14Z","receivedAt":"2018-06-18T04:17:19Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Jun 17, 2018 at 01:37:24PM +0200, Johannes Schindelin wrote:\n\n> > If it's just a custom Authorization header, we should be able to support\n> > it with existing curl versions without _too_ much effort.\n> \n> Indeed. Because it is already implemented:\n> \n> \tgit -c http.extraheader=\"Authorization: Bearer ...\" ...\n> \n> To make this a *little* safer, you can use http.<URL>.extraheader.\n\nYeah, that will work for some cases. A few places it might not:\n\n - some people may want to provide this only in response to a 401\n\n - some tokens may need to be refreshed, which would require interacting\n   with a credential helper to do the rest of the oauth conversation\n\n - there's no good way to hide your token in secure storage (versus\n   sticking it on the command-line or in a config file).\n\n-Peff\n"},{"id":"350390","messageId":"xmqqo9g8xf9k.fsf@gitster-ct.c.googlers.com","threadId":"48708","inReplyTo":"20180618041713.GA31125@sigill.intra.peff.net","subject":"Re: OAuth2 support in git?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-06-18T15:53:27Z","receivedAt":"2018-06-18T15:53:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Sun, Jun 17, 2018 at 01:37:24PM +0200, Johannes Schindelin wrote:\n>\n>> > If it's just a custom Authorization header, we should be able to support\n>> > it with existing curl versions without _too_ much effort.\n>> \n>> Indeed. Because it is already implemented:\n>> \n>> \tgit -c http.extraheader=\"Authorization: Bearer ...\" ...\n>> \n>> To make this a *little* safer, you can use http.<URL>.extraheader.\n>\n> Yeah, that will work for some cases. A few places it might not:\n>\n>  - some people may want to provide this only in response to a 401\n>\n>  - some tokens may need to be refreshed, which would require interacting\n>    with a credential helper to do the rest of the oauth conversation\n>\n>  - there's no good way to hide your token in secure storage (versus\n>    sticking it on the command-line or in a config file).\n\nAnd all of these three are what you get for free by building on the\ncredential helper framework, after extending it a bit so that the\nfilled credential structure can tell the http code to show it to the\nother side as a bearer token, not a password or password hash.  The\nhelper is asked to supply the auth material only after 401, which\ncovers both the first and the second points, and then keeping the\nauth material in-core (e.g. cache--daemon) would be more secure\nwhich covers the third point.  Am I following you correctly?\n\nThanks.\n\n"},{"id":"350426","messageId":"20180618212614.GA2504@sigill.intra.peff.net","threadId":"48708","inReplyTo":"xmqqo9g8xf9k.fsf@gitster-ct.c.googlers.com","subject":"Re: OAuth2 support in git?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-06-18T21:26:14Z","receivedAt":"2018-06-18T21:26:19Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 18, 2018 at 08:53:27AM -0700, Junio C Hamano wrote:\n\n> > Yeah, that will work for some cases. A few places it might not:\n> >\n> >  - some people may want to provide this only in response to a 401\n> >\n> >  - some tokens may need to be refreshed, which would require interacting\n> >    with a credential helper to do the rest of the oauth conversation\n> >\n> >  - there's no good way to hide your token in secure storage (versus\n> >    sticking it on the command-line or in a config file).\n> \n> And all of these three are what you get for free by building on the\n> credential helper framework, after extending it a bit so that the\n> filled credential structure can tell the http code to show it to the\n> other side as a bearer token, not a password or password hash.  The\n> helper is asked to supply the auth material only after 401, which\n> covers both the first and the second points, and then keeping the\n> auth material in-core (e.g. cache--daemon) would be more secure\n> which covers the third point.  Am I following you correctly?\n\nYes, exactly.\n\nEven if the credential protocol itself doesn't learn about this feature,\neven a config option for \"treat password as token to send via bearer\"\nwould help. The \"how\" of sending the token isn't secret, just the token\nitself. So everything else can just pretend it's a password (it's a\nlittle funny because I think there isn't a matching username, but you\ncould probably get by with an empty one).\n\nThat's all just off the top of my head without digging back into the\ncode, nor running any experiments, of course. There may be some gotchas. :)\n\n-Peff\n"},{"id":"350469","messageId":"CAENte7hzJw5VW2JFLV1Pj5v4u52=xL-dvhcfRACYa2eUvQnAVA@mail.gmail.com","threadId":"48708","inReplyTo":"20180618212614.GA2504@sigill.intra.peff.net","subject":"Re: OAuth2 support in git?","fromName":"Christian Halstrick","fromEmail":"christian.halstrick@gmail.com","sentAt":"2018-06-19T12:36:50Z","receivedAt":"2018-06-19T12:37:15Z","isPatch":false,"sender":{"key":"christian.halstrick@gmail.com","avatar":"https://gravatar.com/avatar/3598bf518644c7dc32d4dcd8e0554b6a313e12103011862402c83ae2854203ce?d=mp&s=160"},"body":"What is not clear to me is how we can make use of the servers initial\nresponse in\norder control which credential helper to call and how to transport the\ncredentials.\n\nImagine we try to clone over http. The initial request sent to the server\nmay not contain a \"Authorization: ...\" header and the server responds\nwith Unauthorized.\nBut the server response contains hints like a \"WWW-Authenticate: Basic\nrealm=...\" line\nor a \"WWW-Authenticate: Bearer realm=...\" line which helps choosing the\nauthentication scheme used next. Maybe the server even responds with both lines\ntelling I would accept BASIC or BEARER.\n\nI can imagine that we want libcurl to deal with that decisions. But\neven then. How\ndo we make sure the our credential helpers can act return either user/password\nor bearer tokens based on the server response? If credential helper\nwould have access\nto the servers response (or only relevant parts of it?) it could\ndecide whether to\nfeel responsible for that server or not and what data to return.\n\nAnd if credential helper could optionally give metadata about the kind\ncredential they offer\n(e.g. \"I return user/password\" or \"I return a bearer token\") then core\ncode could know\nwhere to transport this data. E.g. in a \"Authorization: Basic ...\" or\na \"Authorization: Bearer ...\"\nfield.\n\nCiao\n  Chris\n"},{"id":"350496","messageId":"20180619164540.GA22697@sigill.intra.peff.net","threadId":"48708","inReplyTo":"CAENte7hzJw5VW2JFLV1Pj5v4u52=xL-dvhcfRACYa2eUvQnAVA@mail.gmail.com","subject":"Re: OAuth2 support in git?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-06-19T16:45:41Z","receivedAt":"2018-06-19T16:45:46Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 19, 2018 at 02:36:50PM +0200, Christian Halstrick wrote:\n\n> What is not clear to me is how we can make use of the servers initial\n> response in order control which credential helper to call and how to\n> transport the credentials.\n\nI don't think we'd ever decide _which_ credential helper to call; we\nalways call all of them, in order, and then quit when we have sufficient\ncredentials to continue.\n\nBut potentially we could feed some extra information to each helper and\nlet it decide what to do.\n\n> Imagine we try to clone over http. The initial request sent to the\n> server may not contain a \"Authorization: ...\" header and the server\n> responds with Unauthorized.  But the server response contains hints\n> like a \"WWW-Authenticate: Basic realm=...\" line or a\n> \"WWW-Authenticate: Bearer realm=...\" line which helps choosing the\n> authentication scheme used next. Maybe the server even responds with\n> both lines telling I would accept BASIC or BEARER.\n\nSo for this example, yeah, I think it might make sense to feed the\ncredential helper extra context like \"authtype=basic\" or similar. Most\nhelpers would ignore it, but smart ones could make a decision based on\nit.\n\nAnd then the response could contain a similar \"authtype\" key in the\nresponse.\n\n> I can imagine that we want libcurl to deal with that decisions. But\n> even then. How do we make sure the our credential helpers can act\n> return either user/password or bearer tokens based on the server\n> response? If credential helper would have access to the servers\n> response (or only relevant parts of it?) it could decide whether to\n> feel responsible for that server or not and what data to return.\n> \n> And if credential helper could optionally give metadata about the kind\n> credential they offer (e.g. \"I return user/password\" or \"I return a\n> bearer token\") then core code could know where to transport this data.\n> E.g. in a \"Authorization: Basic ...\" or a \"Authorization: Bearer ...\"\n> field.\n\nYep, I think that all matches my general line of thinking. It would help\nif we had some concrete cases. In particular, it's unclear to me if:\n\n  1. A config option to say \"treat password as a bearer token\" would be\n     enough.\n\n  2. We'd need the credential helper to say \"I'm giving you a token\"\n     versus \"I'm giving you a password\".\n\n  3. We might need _both_ (1) and (2), because some servers would be\n     fine with (1) and it lets them Just Work with credential helpers\n     that are unaware of bearer tokens in the first place.\n\nI suspect the answer is (3), but I'd probably delay working on (2) until\nI saw a situation that really needed it. :)\n\nBut I think we're on the same page, so if you're looking into or\ndeveloping more concrete cases, those answers should become more clear.\n\n-Peff\n"}]}