# Remembering which http remotes require authentication 5 messages from 2023-06-12 to 2023-06-21. Participants: M Hickford, brian m. carlson, Junio C Hamano. Thread: https://gitlist.dev/t/59852 ## M Hickford, 2023-06-12 06:47 Subject: Remembering which http remotes require authentication Message-ID: URL: https://gitlist.dev/e/CAGJzqsnkH_dtv-CGsSFJ8K0kXKBf1%2BQQqy%3DMEw-Vq9XaEqr%3DAQ%40mail.gmail.com ``` Most http remotes require authentication to push. Private repositories also require authentication to fetch. If I understand correctly, http_request_reauth [1] first makes an unauthenticated http request. If this fails with error 401 Unauthorized, then Git retries with authentication. This avoids potentially expensive `credential fill` unless necessary. (`credential fill` may be fast, slow or even require user input) Could Git remember which remotes require authentication for which operations? On high-latency networks such as 3G or satellite connections [2], skipping a doomed unauthenticated request could potentially halve the time taken for some commands. A complication: if a repository changed from private to public, Git would continue to authenticate when no longer necessary. A solution could be to remember to skip the unauthenticated request for a limited amount of time, or a maximum number of requests, before rechecking. [1] https://github.com/git/git/blob/fe86abd7511a9a6862d5706c6fa1d9b57a63ba09/http.c#L2123-L2169 [2] https://hpbn.co/primer-on-latency-and-bandwidth/ ``` ## brian m. carlson, 2023-06-12 21:21 Subject: Re: Remembering which http remotes require authentication Message-ID: URL: https://gitlist.dev/e/ZIeMcyloOiyAJ8ak%40tapette.crustytoothpaste.net In-Reply-To: ``` On 2023-06-12 at 06:47:19, M Hickford wrote: > Most http remotes require authentication to push. Private repositories > also require authentication to fetch. > > If I understand correctly, http_request_reauth [1] first makes an > unauthenticated http request. If this fails with error 401 > Unauthorized, then Git retries with authentication. This avoids > potentially expensive `credential fill` unless necessary. (`credential > fill` may be fast, slow or even require user input) > > Could Git remember which remotes require authentication for which > operations? On high-latency networks such as 3G or satellite > connections [2], skipping a doomed unauthenticated request could > potentially halve the time taken for some commands. > > A complication: if a repository changed from private to public, Git > would continue to authenticate when no longer necessary. A solution > could be to remember to skip the unauthenticated request for a limited > amount of time, or a maximum number of requests, before rechecking. What I would rather see instead is a way to indicate to Git that it should always authenticate via a config option (probably something like `http.*.alwaysAuth`). I think this would work for your use case, but it would also work for things like giving authenticated users higher rate limits for public repositories. I intend to add this feature at some point relatively soon, but I need to sneak in a feature first to specify the authentication scheme as part of the credential protocol, since we won't have it as part of the HTTP 401 response. -- brian m. carlson (he/him or they/them) Toronto, Ontario, CA ``` ## Junio C Hamano, 2023-06-12 22:12 Subject: Re: Remembering which http remotes require authentication Message-ID: URL: https://gitlist.dev/e/xmqqh6rc72k8.fsf%40gitster.g In-Reply-To: ``` M Hickford writes: > Could Git remember which remotes require authentication for which > operations? Interesting idea. Credential helpers may know which URL needed authentication before, but that is not per operation. I wonder if it is sufficient as the zeroth approximation to always try authenticated access if helpers have any auth material to the URL regardless of the operation? There needs a way to force unautheticated access if we were to do anything of that sort, of course. ``` ## M Hickford, 2023-06-21 06:30 Subject: Re: Remembering which http remotes require authentication Message-ID: URL: https://gitlist.dev/e/CAGJzqsnU9HOCWOOaMdUcNzSDWg6wUqK0bhNvtNAv82yeCkHwgQ%40mail.gmail.com In-Reply-To: ``` On Mon, 12 Jun 2023 at 23:12, Junio C Hamano wrote: > > M Hickford writes: > > > Could Git remember which remotes require authentication for which > > operations? > > Interesting idea. > > Credential helpers may know which URL needed authentication before, > but that is not per operation. I wonder if it is sufficient as the > zeroth approximation to always try authenticated access if helpers > have any auth material to the URL regardless of the operation? The simplicity appeals. A challenge is that `helper get` may be slow or even block requiring user interaction (to unlock a keyring or complete OAuth). A solution could be to introduce a new action `get-existing` or `fast-get` to the helper protocol. Storage helpers would handle this similarly to `get` (unless a keyring is locked). Credential-generating helpers would refrain from generating fresh credentials. However as Brian pointed out, Git doesn't know which authentication scheme to use until it sees the WWW-Authenticate header in the response from the initial unauthenticated request, so I think config or state would be necessary. > > There needs a way to force unautheticated access if we were to do > anything of that sort, of course. > > > ``` ## brian m. carlson, 2023-06-21 10:09 Subject: Re: Remembering which http remotes require authentication Message-ID: URL: https://gitlist.dev/e/ZJLMSz6jp%2BrBwhTA%40tapette.crustytoothpaste.net In-Reply-To: ``` On 2023-06-21 at 06:30:33, M Hickford wrote: > On Mon, 12 Jun 2023 at 22:22, brian m. carlson > wrote: > > > > What I would rather see instead is a way to indicate to Git that it > > should always authenticate via a config option (probably something like > > `http.*.alwaysAuth`). I think this would work for your use case, but it > > would also work for things like giving authenticated users higher rate > > limits for public repositories. > > Interesting. Do you know any such hosts that rate limit unauthenticated > users? Do their users prefer SSH which always authenticates? At GitHub, we do this with the API, and because Git LFS uses Git's credentials and is technically implemented as part of the API, it applies there as well. I don't know that we have any intention of doing it for Git operations themselves, but the reason we typically prefer authentication is that if there's a problem, it's much easier and a better experience to contact user XYZ and ask them to be more gentle than to block their IP. This is particularly useful for internal users at GitHub (like the automatic dependency updating service, Dependabot), where we'd like to account for who's performing lots of operations in terms of rate limiting, and the alternative is that people use http.extraHeader or a token in the URL, neither of which we want to encourage. I'd much rather rate-limit an internal service which is using too many resources than impact unrelated users, but if they're all anonymous, that's hard. > > I intend to add this feature at some point relatively soon, but I need > > to sneak in a feature first to specify the authentication scheme as part > > of the credential protocol, since we won't have it as part of the HTTP > > 401 response. > > Would it suffice to ask the user to specify the authentication scheme > in the configuration? eg. `http.alwaysAuth=basic` or > `http.alwaysAuth=digest` That will probably work for most cases, but we may need to allow either a scheme or a full WWW-Authenticate header, since in some cases there may be parameters that are needed. -- brian m. carlson (he/him or they/them) Toronto, Ontario, CA ```