transfer.credentialsInUrl should warn about personal access tokens in user field #leftoverbits

Assuming config key transfer.credentialsInUrl is set to "warn", Git warns about "plaintext credentials" if the user includes a password in the remote URL. https://git-scm.com/docs/git-config#Documentation/git-config.txt-transfercredentialsInUrl . This is implemented in remote.c

    $ git clone https://tim:hunter2@example.com/example.git
    warning: URL 'https://tim:<redacted>@example.com/example.git' uses plaintext credentials

It would be neat to warn similarly if the user includes a personal access token in the *user* field of the remote URL:

    git clone https://<pat>@github.com/...

This is a popular practice according to StackOverflow https://stackoverflow.com/a/70320541/284795 (800k views).

GitHub personal access tokens are easily recognised by their prefixes "ghp_" and "github_pat_" https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/about-authentication-to-github#githubs-token-formats 

#leftoverbits

M Hickford <mirth.hickford@gmail.com> writes:

Curious. I do not think we have *any* code to special case such a "token", so to Git itself, https://<pat>@github.com/... should look as it (assuming <pat> does not have any colon in it) is trying to access the site with <pat> as the username.

How do we _know_ that this request with <pat> do not need a password? I ask because I'd prefer not to see us hardcoding any hosting-site specific heuristics in the code, and these users apparently are doing fine without any such hardcoding. If we can reuse the mechanism that is letting them do so when deciding if we should warn, it would be great.

Are users expected to configure their credential helpers know that it is a <pat> (perhaps with authtype=Bearer)?

Sorry for not giving any answer and piling more questions on top.

On 2025-01-10 at 21:32:44, Junio C Hamano wrote:

There are also other prefixes as well.

That is accurate. GitHub has accepted a token in the username for some time (at least since 2019, and probably longer) for reasons which are unknown to me. From a security perspective, I would recommend users not do so, since many tools log the username and not the password[0], but it's almost certainly too late to change it now. I can mention it, but even if there was a decision to drop that, it would require a long deprecation period.

I don't think in general we can know whether a credential is just a plain username or a token without trying to guess based on the content. For instance, before `http.emptyAuth`, it was common if one was using Kerberos to put one's username in the URL because that triggered libcurl to do authentication, whereas it would not if no credentials were specified. I still have that configured, and I bet a lot of others do as well.

It's also common for people with both work and personal accounts on a site to put the username in the URL so that the correct credentials are looked up in the credential helper. And all of that is fine and secure since there are no actual secrets in the username in those cases.

So there are lots of legitimate reasons to place only a username there, and we'd only be able to know if it's actually a token by hard-coding patterns. I would recommend that we not do that, since I can't guarantee that the list of patterns won't expand in the future (it already has before), and there are still some older hex-only patterns which may be in use and which are much less obvious.

No, this is Basic auth. It's just that GitHub will look at either the username or password field for the token.

[0] The two systems at GitHub which handle these credentials (the Git service and the Git auth service) are designed to recognize tokens in the username and obscure them instead of logging them. Pretty much no other tools on the Internet do that, though.

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

Yes, I do strongly object us to keep a hardcoded list that can go stale (or be stale from the beginning).

What I was wondering is that because we are in control of the Git end of the credential subsystem (even if the user may be using a third-party credential helper), we

 - can notice that the URL has embedded single thing (which could be
   username, but which could be a token);

 - can also notice that we asked the credential-helper, or
   keyboard-interactive, and obtained a password (or not).

When the former is true and the latter is false, it is an indication that for that site with the username-or-token, there wasn't anything necessary to authenticate and the access was authorized. Which is what the original poster wants us to warn against.

And if we can do so, we do not need any list of patterns, right?

On 2025-01-10 at 22:51:01, Junio C Hamano wrote:

No, I don't think that will work in the general case. Here's why. If I do `git push https://bmc@git.crustytoothpaste.net/git/bmc/xyzzy.git`, that uses Kerberos (Negotiate). There's a username there to make libcurl enable auth, but it's never used and a credential helper is never invoked, so case 1 is true and case 2 is false.

Now, we _could_ do that only for Basic auth, which would catch the GitHub case. However, it's _also_ possible to use TLS client certificate auth (and I think Bitbucket does support that) and use the username only for choosing the account (because, say, your work account uses a client certificate and your personal account uses something else). There might be Basic auth sent (say, if you'd set `http.proactiveAuth`), but the server would ignore it since you were already authenticated via the TLS cert. That would also make case 1 be true and case 2 be false.

Perhaps that latter case is not worth worrying about, but it is a possibility and I'm sure some people will hit it. Maybe with a config option for the advice that's okay, though.

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

Thanks, so in short, if we really want to do this "warning", it has to look at the "username" string and "guess" that it is something the user does not want to have as a value to remote.*.url field X-<.

So "should warn" in the title is not really #leftoverbits (i.e. we clearly know it is a good idea, we just didn't have bandwidth to do it, anybody is welcome to fill the void) at this point.

Thanks.

On January 10, 2025 7:45 PM, Junio C Hamano wrote:

If the use case is like mine, where someone wants to use something similar to a PAT for creating a Pull Request, then GitHub, BitBucket, GitLab, and AzureGit all behave in a very similar way. The PAT structure is different, but there is conceptual equivalence.

--Randall

at the

to have

know it is a

fill the

On 2025-01-10 22:06, brian m. carlson wrote:

Thanks Brian and Junio for the discussion. I understand your aversion to introducing GitHub-specific PAT detection logic.

The better solution would be for GitHub to stop accepting PAT in the username field. Hopefully that will happen one day. It's undocumented, so its usage ought to diminish.

To speed that along, I've edited the offending StackOverflow answer https://stackoverflow.com/a/70320541/284795

On January 10, 2025 4:33 PM, Junio C Hamano wrote:

This is why I implemented my own credential layer - sorry, cannot share, but can describe if asked (maybe at a Git Merge? 😉 ). Git should not need to know about GitHub specific access methods for HTTPS, where PAT is required for git push. There is no password for PAT use, which makes it highly insecure if persisted on disk - I do not know why GitHub went down that path, I guess they did not want any potential CVEs associated with SSH-like passphrases. PAT became a real annoyance for my community, so I had to do something quick and adding my own credential manager seemed expedient (and works pretty well - but if someone wants to do this, they must be very careful managing access to the cached PAT's memory use). The primary use case in my situation is not actually git push, but the creation of Pull Requests on GitHub from scripts or a CLI since NonStop has no GUI adapter to access their website.

My email rsbecker@nexbridge.com is being blocked by gmail, so I cannot get to Derek. Anyone know why? Have I been a bad boy somehow? Resending via another account.

On January 10, 2025 4:33 PM, Junio C Hamano wrote:

This is why I implemented my own credential layer - sorry, cannot share, but can describe if asked (maybe at a Git Merge? 😉 ). Git should not need to know about GitHub specific access methods for HTTPS, where PAT is required for git push. There is no password for PAT use, which makes it highly insecure if persisted on disk - I do not know why GitHub went down that path, I guess they did not want any potential CVEs associated with SSH-like passphrases. PAT became a real annoyance for my community, so I had to do something quick and adding my own credential manager seemed expedient (and works pretty well - but if someone wants to do this, they must be very careful managing access to the cached PAT's memory use). The primary use case in my situation is not actually git push, but the creation of Pull Requests on GitHub from scripts or a CLI since NonStop has no GUI adapter to access their website.

Randall Becker <randall.becker@nexbridge.ca> writes:

It is not you, but him.

commit 6e5457d8c7332c279e1482c442922f954af5f8f2
Author: Derrick Stolee <stolee@gmail.com>
Date:   Thu Oct 12 17:30:33 2023 +0000

    mailmap: change primary address for Derrick Stolee
    
    The previous primary address is no longer valid.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>