Re: git clone with basic auth in url directly returns authentication failure after 401 received under some git versions

From: Jeff King <peff@peff.net>
Date: Aug 20, 2022, 08:44 UTC
Message-ID: <YwCe6ONEaeIj4SO/@coredump.intra.peff.net>
In-Reply-To: <CADmGLV32OAg6HU+n1UsP2Fq-MjcyUsFFF=q0_jZCB0JEop5VUg@mail.gmail.com>

On Sat, Aug 20, 2022 at 10:51:16AM +0800, 王小建 wrote:

Show 9 quoted lines

> What's different between what you expected and what actually happened?
> 
> When I use git v2.36.2  (docker image is alpine/git:v2.36.2) to clone
> with basic auth in url, when receiving the 401, it directly returns
> authentication failure, even recv head has www-authenticate: Basic
> realm=Restricted,
> and no request is send again. I think it should send request with
> authorization: Basic header after receive 401.
> And use git v2.34.2 (docker image is alpine/git:v2.34.1) to clone it works well.

I think the problem here is not the difference in Git versions, but rather in libcurl versions. I can reproduce your problem using the docker containers. But if I build locally, using the same version of curl, then I see the issue with both git versions.

The problem is how curl handles cross-protocol redirects. From Git's perspective, we hand the credentials to libcurl, and ask it to fetch the requested URL, including following redirects. If it comes back with a 401, then we assume our credentials were bad.

But what changed in curl is that it will now discard credentials during a redirect. And in your example, there's a redirect from http to https (uninteresting bits snipped from the output):

> Info: Connected to xxx.xxx (xxx.xxx.xxx.xxx) port 80 (#0)
> Send header: GET /xxx/xxx/info/refs?service=git-upload-pack HTTP/1.1
> Recv header: HTTP/1.1 302 Found
> Recv header: Location: https://xxx.xxx/xxx/xxx/info/refs?service=git-upload-pack

In the older version, after the redirect we see a 401 and curl (not git) resends with the stored credentials.

But in the newer version, we see this right after the redirect:

> Info: Connection #0 to host xxx.xxx left intact
> Info: Clear auth, redirects to port from 80 to 443

So it is dropping the credential that Git gave it.

The curl change seems to be from 620ea2141 (transfer: redirects to other protocols or ports clear auth, 2022-04-25). The goal is to avoid leaking credentials between ports: https://curl.se/docs/CVE-2022-27774.html

So that makes sense, though I wonder if curl ought to make an exception for moving from 80 to 443 and http to https?

I don't think there's otherwise much Git can do here. We thought we gave curl a username and password, but they weren't ultimately used. But Git won't reissue the request, because it assumes the auth was rejected.

I guess we can ask curl if it saw a redirect, and assume if so that the auth was cleared. That feels a bit hacky. And it's subverting curl's attempt not to leak the credentials. In general, I'd like to defer as much as possible to curl's ideas of how to handle things, because they're much better at implementing http best practices than we are. :)

Another option is to allow the user to set CURLOPT_UNRESTRICTED_AUTH, but that seems like a bad idea for the same reason.

Hopefully that explains what's going on. The short answer for your case is: use an https url directly, and it should work. But there's an open question of whether curl ought to handle this limited redirect case more gracefully.

-Peff