git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] http: preserve wwwauth_headers across redirects

From
Matthew John Cheetham <mjcheetham@outlook.com>
Date
Nov 4, 2026, 09:34 UTC
Message-ID
<VI0PR03MB11634E7F7FE122AF61085D7D8C0682@VI0PR03MB11634.eurprd03.prod.outlook.com>
In-Reply-To
<20261104062423.GA1763458@coredump.intra.peff.net>
On 04/11/2026 06:24, Jeff King wrote:
Show 20 quoted lines
> On Wed, Aug 19, 2026 at 08:21:04PM -0700, Aaron Plattner wrote:
> >> When cURL follows a redirect, it calls the CURLOPT_HEADERFUNCTION for
>> each header received including ones from a redirect. http_request() sets
>> fwrite_wwwauth() as the header function, which will record the wwwauth[]
>> entries for the last step in the redirection chain.
>>
>> However, when http_request_recoverable() sees that cURL followed a
>> redirect, it attempts to update the credentials for the request from the
>> new URL using credential_from_url(). The first thing that does is call
>> credential_clear(), which clears everything including wwwauth_headers.
>>
>> If the new URL should use a credential helper rather than credentials
>> embedded in the URL, this loses the list of authentication methods that
>> the server provided in the redirect.
> > OK, the problem description makes sense. I think the root cause here is
> that wwwauth_headers should not really be in the credential struct in
> the first place. They are a property of the HTTP request, and we get
> them _back_ from the server; we don't send them. They are stuffed into
> the credential struct because we pass them to helpers, but they're
> otherwise unlike the rest of the struct in that respect.

Yeah, this was one of the awkward parts of passing WWW-Authenticate headers through to credential helpers in the original implementation.

http.c has per-request slots, but much of its authentication and configuration state is global. The credential struct already carried helper configuration and interaction state, such as the helpers list, use_http_path, approved and quit, so it was a convenient place to carry the challenges to helpers too. That does leave URL-derived and response-derived state with different lifetimes in the same object, though.

I've wondered whether a clearer separation between shared HTTP client state and per-request state would make this easier to reason about. That would also need to account for callers like remote-curl.c and http-push.c, which configure curl handles through the lower-level slot API exposed by http.h.

The current split can make ownership and retries harder to follow, and I've certainly contributed some of that complexity myself! ;-)

I've made a few exploratory attempts in that direction: a reusable client with no global state and explicit per-request context and policies for authentication and retries. Nothing ready to propose yet - it's a messy space!

That's a broader cleanup, though, rather than a prerequisite for this fix.

> So one possible path forward would be to pull them out of the struct,
> though I think that does create new headaches. We'd need to pass them in
> to credential_fill(), for example, so it can feed them to the helpers.

Right, I remember thinking the other alternative would be to expand the credential_fill() function to take headers. Other than the churn, I also didn't do this because WWW-Authenticate headers are an HTTP only construct, and credential_fill is used for other things like SMTP creds and the password to certificates.

You raised the smell in the original series about adding wwwauth_headers and header_is_last_match to the credential struct: https://lore.kernel.org/git/Y9JWnQeEV0weV4yu@coredump.intra.peff.net

Your follow-up clarified that the main concern was the internal HTTP parsing flag. Moving it out would require some extra plumbing across curl callbacks, so we left it as-is at the time: https://lore.kernel.org/git/Y+TxRDatM9Iirwwu@coredump.intra.peff.net

Show 23 quoted lines
> So it may be the past of least resistance to just hack around this reset
> problem and preserve them (i.e., your patch).
> >> The WWW-Authenticate challenge is not derived from the URL; it is
>> populated from the server's response, and after a redirect it describes
>> how to authenticate to the redirect target and it needs to survive the
>> URL update so that credential helpers can know which authentication
>> methods are allowed.
>>
>> Add a new credential_update_url() that wraps credential_from_url() and
>> preserves wwwauth_headers specifically. Use SWAP() to avoid having to
>> copy the whole strbuf.
> > So my biggest question/fear would be: are we creating a security problem
> wherein we use auth information from the original server with the
> redirected-to server (the avoidance of which is the whole point of
> clearing the credential struct in the first place).
> > And I think the answer is "no". These wwwauth headers are coming from
> the server back to us. And as described in 6b8dda9a4f (http: read HTTP
> WWW-Authenticate response headers, 2023-02-27), we get the headers from
> _all_ servers we talk to (the original and the redirect target). So we
> have to deal with this problem already, and do so by resetting the list
> when we see a new HTTP request start. So by definition, the wwwauth
> headers we get back are the ones from the redirected-to server. And so
> preserving them as we move to that new URL is the correct thing.

Right. fwrite_wwwauth() clears the challenge list whenever it sees a new HTTP status line. After libcurl follows the redirects, the list therefore belongs to the final response, not an earlier response in the chain. Preserving it while updating the credential context to match that response's URL is the intended behaviour.

Show 36 quoted lines
> I see that the credential_update_url() wrapper is new in this iteration
> of the patch. I actually think it is a slight step in the wrong
> direction. We know that retaining wwwauth_headers is safe here only
> because we know we the updated URL we are feeding matches the response
> from that server. Any other hypothetical callers of the new function
> would also need to consider that (and if they get it wrong, it fails in
> the insecure direction).
> > That said, I'd highly doubt we'd ever get another caller either way (nor
> add any other data to the credential that is treated this way). So I am
> OK with it, but have a slight preference for the original inline
> treatment from the v1 patch.
> >> +void credential_update_url(struct credential *c, const char *url)
>> +{
>> +	struct strvec wwwauth_headers = STRVEC_INIT;
>> +
>> +	/*
>> +	 * credential_from_url() clears the whole credential. Preserve the
>> +	 * WWW-Authenticate list, which is derived from the server's original
>> +	 * response rather than from the URL and is required to authenticate to
>> +	 * the new URL.
>> +	 */
>> +	SWAP(wwwauth_headers, c->wwwauth_headers);
>> +	credential_from_url(c, url);
>> +	SWAP(c->wwwauth_headers, wwwauth_headers);
>> +	strvec_clear(&wwwauth_headers);
>> +}
> > OK. This strvec_clear() at the end is almost certainly a noop. We
> swapped in the value set by credential_from_url(), which will always be
> blank. But it probably does not hurt. OTOH, if that assumption is wrong,
> we are throwing away useful bits that we pulled from the URL, without
> even knowing what they are. So maybe:
> >    if (wwwauth_headers.nr)
> 	BUG("credential_from_url put something in the wwwauth_headers?");
> > would better describe what's going on.
> > I'm adding Matthew Cheetham to the cc, as he was the original author of
> these wwwauth bits.
I appreciate the cc! :-)
> -Peff

Thanks, Matthew

Previous: Jeff KingNext: Michal Suchánek
Message 8 of 9 in “http: preserve wwwauth_headers across redirects”
  1. http: preserve wwwauth_headers across redirectsAaron Plattner, Aug 20, 2026
  2. Junio C HamanoAug 20, 2026
  3. Junio C HamanoAug 25, 2026
  4. Junio C HamanoNov 2, 2026
  5. Jeff KingNov 4, 2026
  6. Junio C HamanoNov 4, 2026
  7. Jeff KingNov 4, 2026
  8. Matthew John CheethamNov 4, 2026
  9. Michal SuchánekNov 4, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.