Remembering which http remotes require authentication

Most http remotes require authentication to push. Private repositories also require authentication to fetch.

If I understand correctly, http_request_reauth [1] first makes an unauthenticated http request. If this fails with error 401 Unauthorized, then Git retries with authentication. This avoids potentially expensive `credential fill` unless necessary. (`credential fill` may be fast, slow or even require user input)

Could Git remember which remotes require authentication for which operations? On high-latency networks such as 3G or satellite connections [2], skipping a doomed unauthenticated request could potentially halve the time taken for some commands.

A complication: if a repository changed from private to public, Git would continue to authenticate when no longer necessary. A solution could be to remember to skip the unauthenticated request for a limited amount of time, or a maximum number of requests, before rechecking.

[1] https://github.com/git/git/blob/fe86abd7511a9a6862d5706c6fa1d9b57a63ba09/http.c#L2123-L2169 [2] https://hpbn.co/primer-on-latency-and-bandwidth/

On 2023-06-12 at 06:47:19, M Hickford wrote:

What I would rather see instead is a way to indicate to Git that it should always authenticate via a config option (probably something like `http.*.alwaysAuth`). I think this would work for your use case, but it would also work for things like giving authenticated users higher rate limits for public repositories.

I intend to add this feature at some point relatively soon, but I need to sneak in a feature first to specify the authentication scheme as part of the credential protocol, since we won't have it as part of the HTTP 401 response.

M Hickford <mirth.hickford@gmail.com> writes:

Interesting idea.

Credential helpers may know which URL needed authentication before, but that is not per operation. I wonder if it is sufficient as the zeroth approximation to always try authenticated access if helpers have any auth material to the URL regardless of the operation?

There needs a way to force unautheticated access if we were to do anything of that sort, of course.

On Mon, 12 Jun 2023 at 23:12, Junio C Hamano <gitster@pobox.com> wrote:

The simplicity appeals. A challenge is that `helper get` may be slow or even block requiring user interaction (to unlock a keyring or complete OAuth). A solution could be to introduce a new action `get-existing` or `fast-get` to the helper protocol. Storage helpers would handle this similarly to `get` (unless a keyring is locked). Credential-generating helpers would refrain from generating fresh credentials.

However as Brian pointed out, Git doesn't know which authentication scheme to use until it sees the WWW-Authenticate header in the response from the initial unauthenticated request, so I think config or state would be necessary.

On 2023-06-21 at 06:30:33, M Hickford wrote:

At GitHub, we do this with the API, and because Git LFS uses Git's credentials and is technically implemented as part of the API, it applies there as well. I don't know that we have any intention of doing it for Git operations themselves, but the reason we typically prefer authentication is that if there's a problem, it's much easier and a better experience to contact user XYZ and ask them to be more gentle than to block their IP.

This is particularly useful for internal users at GitHub (like the automatic dependency updating service, Dependabot), where we'd like to account for who's performing lots of operations in terms of rate limiting, and the alternative is that people use http.extraHeader or a token in the URL, neither of which we want to encourage. I'd much rather rate-limit an internal service which is using too many resources than impact unrelated users, but if they're all anonymous, that's hard.

That will probably work for most cases, but we may need to allow either a scheme or a full WWW-Authenticate header, since in some cases there may be parameters that are needed.