{"thread":{"id":"59852","subject":"Remembering which http remotes require authentication","startedAt":"2023-06-12T07:13:57Z","lastAt":"2023-06-21T10:10:46Z","messageCount":5,"participants":["M Hickford","brian m. carlson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"478253","messageId":"CAGJzqsnkH_dtv-CGsSFJ8K0kXKBf1+QQqy=MEw-Vq9XaEqr=AQ@mail.gmail.com","threadId":"59852","inReplyTo":null,"subject":"Remembering which http remotes require authentication","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2023-06-12T06:47:19Z","receivedAt":"2023-06-12T07:13:57Z","isPatch":false,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"Most http remotes require authentication to push. Private repositories\nalso require authentication to fetch.\n\nIf I understand correctly, http_request_reauth [1] first makes an\nunauthenticated http request. If this fails with error 401\nUnauthorized, then Git retries with authentication. This avoids\npotentially expensive `credential fill` unless necessary. (`credential\nfill` may be fast, slow or even require user input)\n\nCould Git remember which remotes require authentication for which\noperations? On high-latency networks such as 3G or satellite\nconnections [2], skipping a doomed unauthenticated request could\npotentially halve the time taken for some commands.\n\nA complication: if a repository changed from private to public, Git\nwould continue to authenticate when no longer necessary. A solution\ncould be to remember to skip the unauthenticated request for a limited\namount of time, or a maximum number of requests, before rechecking.\n\n[1] https://github.com/git/git/blob/fe86abd7511a9a6862d5706c6fa1d9b57a63ba09/http.c#L2123-L2169\n[2] https://hpbn.co/primer-on-latency-and-bandwidth/\n"},{"id":"478306","messageId":"ZIeMcyloOiyAJ8ak@tapette.crustytoothpaste.net","threadId":"59852","inReplyTo":"CAGJzqsnkH_dtv-CGsSFJ8K0kXKBf1+QQqy=MEw-Vq9XaEqr=AQ@mail.gmail.com","subject":"Re: Remembering which http remotes require authentication","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2023-06-12T21:21:55Z","receivedAt":"2023-06-12T21:25:27Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2023-06-12 at 06:47:19, M Hickford wrote:\n> Most http remotes require authentication to push. Private repositories\n> also require authentication to fetch.\n> \n> If I understand correctly, http_request_reauth [1] first makes an\n> unauthenticated http request. If this fails with error 401\n> Unauthorized, then Git retries with authentication. This avoids\n> potentially expensive `credential fill` unless necessary. (`credential\n> fill` may be fast, slow or even require user input)\n> \n> Could Git remember which remotes require authentication for which\n> operations? On high-latency networks such as 3G or satellite\n> connections [2], skipping a doomed unauthenticated request could\n> potentially halve the time taken for some commands.\n> \n> A complication: if a repository changed from private to public, Git\n> would continue to authenticate when no longer necessary. A solution\n> could be to remember to skip the unauthenticated request for a limited\n> amount of time, or a maximum number of requests, before rechecking.\n\nWhat I would rather see instead is a way to indicate to Git that it\nshould always authenticate via a config option (probably something like\n`http.*.alwaysAuth`). I think this would work for your use case, but it\nwould also work for things like giving authenticated users higher rate\nlimits for public repositories.\n\nI intend to add this feature at some point relatively soon, but I need\nto sneak in a feature first to specify the authentication scheme as part\nof the credential protocol, since we won't have it as part of the HTTP\n401 response.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"478313","messageId":"xmqqh6rc72k8.fsf@gitster.g","threadId":"59852","inReplyTo":"CAGJzqsnkH_dtv-CGsSFJ8K0kXKBf1+QQqy=MEw-Vq9XaEqr=AQ@mail.gmail.com","subject":"Re: Remembering which http remotes require authentication","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-06-12T22:12:23Z","receivedAt":"2023-06-12T22:12:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"M Hickford writes:\n\n> Could Git remember which remotes require authentication for which\n> operations?\n\nInteresting idea.\n\nCredential helpers may know which URL needed authentication before,\nbut that is not per operation. I wonder if it is sufficient as the\nzeroth approximation to always try authenticated access if helpers\nhave any auth material to the URL regardless of the operation?\n\nThere needs a way to force unautheticated access if we were to do\nanything of that sort, of course.\n\n\n\n"},{"id":"478611","messageId":"CAGJzqsnU9HOCWOOaMdUcNzSDWg6wUqK0bhNvtNAv82yeCkHwgQ@mail.gmail.com","threadId":"59852","inReplyTo":"xmqqh6rc72k8.fsf@gitster.g","subject":"Re: Remembering which http remotes require authentication","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2023-06-21T06:30:24Z","receivedAt":"2023-06-21T06:31:06Z","isPatch":false,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Mon, 12 Jun 2023 at 23:12, Junio C Hamano wrote:\n>\n> M Hickford writes:\n>\n> > Could Git remember which remotes require authentication for which\n> > operations?\n>\n> Interesting idea.\n>\n> Credential helpers may know which URL needed authentication before,\n> but that is not per operation. I wonder if it is sufficient as the\n> zeroth approximation to always try authenticated access if helpers\n> have any auth material to the URL regardless of the operation?\n\nThe simplicity appeals. A challenge is that `helper get` may be slow\nor even block requiring user interaction (to unlock a keyring or\ncomplete OAuth). A solution could be to introduce a new action\n`get-existing` or `fast-get` to the helper protocol. Storage helpers\nwould handle this similarly to `get` (unless a keyring is locked).\nCredential-generating helpers would refrain from generating fresh\ncredentials.\n\nHowever as Brian pointed out, Git doesn't know which authentication\nscheme to use until it sees the WWW-Authenticate header in the\nresponse from the initial unauthenticated request, so I think\nconfig or state would be necessary.\n\n\n\n>\n> There needs a way to force unautheticated access if we were to do\n> anything of that sort, of course.\n>\n>\n>\n"},{"id":"478618","messageId":"ZJLMSz6jp+rBwhTA@tapette.crustytoothpaste.net","threadId":"59852","inReplyTo":"CAGJzqsnMnJqed8ztiCAS9oDXNMmT73=8caXXsDKeYmxevsQMjg@mail.gmail.com","subject":"Re: Remembering which http remotes require authentication","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2023-06-21T10:09:15Z","receivedAt":"2023-06-21T10:10:46Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2023-06-21 at 06:30:33, M Hickford wrote:\n> On Mon, 12 Jun 2023 at 22:22, brian m. carlson\n> wrote:\n> >\n> > What I would rather see instead is a way to indicate to Git that it\n> > should always authenticate via a config option (probably something like\n> > `http.*.alwaysAuth`). I think this would work for your use case, but it\n> > would also work for things like giving authenticated users higher rate\n> > limits for public repositories.\n> \n> Interesting. Do you know any such hosts that rate limit unauthenticated\n> users? Do their users prefer SSH which always authenticates?\n\nAt GitHub, we do this with the API, and because Git LFS uses Git's\ncredentials and is technically implemented as part of the API, it\napplies there as well. I don't know that we have any intention of doing\nit for Git operations themselves, but the reason we typically prefer\nauthentication is that if there's a problem, it's much easier and a\nbetter experience to contact user XYZ and ask them to be more gentle\nthan to block their IP.\n\nThis is particularly useful for internal users at GitHub (like the\nautomatic dependency updating service, Dependabot), where we'd like to\naccount for who's performing lots of operations in terms of rate\nlimiting, and the alternative is that people use http.extraHeader or a\ntoken in the URL, neither of which we want to encourage. I'd much\nrather rate-limit an internal service which is using too many resources\nthan impact unrelated users, but if they're all anonymous, that's hard.\n\n> > I intend to add this feature at some point relatively soon, but I need\n> > to sneak in a feature first to specify the authentication scheme as part\n> > of the credential protocol, since we won't have it as part of the HTTP\n> > 401 response.\n> \n> Would it suffice to ask the user to specify the authentication scheme\n> in the configuration? eg. `http.alwaysAuth=basic` or\n> `http.alwaysAuth=digest`\n\nThat will probably work for most cases, but we may need to allow either\na scheme or a full WWW-Authenticate header, since in some cases there\nmay be parameters that are needed.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"}]}