{"thread":{"id":"64788","subject":"[PATCH v2] remote-curl: Use auth for probe_rpc() requests too","startedAt":"2026-01-13T03:20:20Z","lastAt":"2026-01-14T17:41:23Z","messageCount":9,"participants":["Aaron Plattner","Patrick Steinhardt","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"533695","messageId":"20260113031929.3746753-1-aplattner@nvidia.com","threadId":"64788","inReplyTo":null,"subject":"[PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Aaron Plattner","fromEmail":"aplattner@nvidia.com","sentAt":"2026-01-13T03:19:28Z","receivedAt":"2026-01-13T03:20:20Z","isPatch":true,"sender":{"key":"aplattner@nvidia.com","avatar":"https://avatars.githubusercontent.com/u/343551?v=4"},"body":"If a large request requires post_rpc() to call probe_rpc(), the latter\ndoes not use the authorization credentials used for other requests. If\nthis fails with an HTTP 401 error and http_auth.multistage isn't set,\nthen the whole request just fails.\n\nFor example, using git-credential-msal [1], the following attempt to clone a\nlarge repository fails partway through because the initial request to download\nthe commit history and promisor packs succeeds, but the\nsubsequent request to download the blobs needed to construct the working\ntree fails with a 401 error and the checkout fails.\n\n(lines removed for brevity)\n\n  git clone --filter=blob:none https://secure-server.example/repo\n  11:03:26.855369 git.c:502               trace: built-in: git clone --filter=blob:none https://secure-server.example/repo\n  Cloning into 'sw'...\n  warning: templates not found in /home/aaron/share/git-core/templates\n  11:03:26.857169 run-command.c:673       trace: run_command: git remote-https origin https://secure-server.example/repo\n  11:03:27.012104 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n  11:03:27.049243 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n  11:03:27.049270 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n  11:03:27.053786 run-command.c:673       trace: run_command: 'git credential-msal get'\n  11:03:27.952830 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n  11:03:27.952849 http.c:849              => Send header: Authorization: Bearer <redacted>\n  11:03:27.995419 http.c:849              <= Recv header: HTTP/1.1 200 OK\n  11:03:28.230039 http.c:890              == Info: Reusing existing https: connection with host secure-server.example\n  11:03:28.230208 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n  11:03:28.230216 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request\n  11:03:28.230221 http.c:849              => Send header: Authorization: Bearer <redacted>\n  11:03:28.269085 http.c:849              <= Recv header: HTTP/1.1 200 OK\n  11:03:28.684163 http.c:890              == Info: Reusing existing https: connection with host secure-server.example\n  11:03:28.684379 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n  11:03:28.684391 http.c:849              => Send header: Accept: application/x-git-upload-pack-result\n  11:03:28.684393 http.c:849              => Send header: Authorization: Bearer <redacted>\n  11:03:28.869546 run-command.c:673       trace: run_command: git index-pack --stdin --fix-thin '--keep=fetch-pack 43856 on dgx-spark' --promisor\n  11:06:39.861237 run-command.c:673       trace: run_command: git -c fetch.negotiationAlgorithm=noop fetch origin --no-tags --no-write-fetch-head --recurse-submodules=no --filter=blob:none --stdin\n  11:06:39.865981 run-command.c:673       trace: run_command: git remote-https origin https://secure-server.example/repo\n  11:06:39.868039 run-command.c:673       trace: run_command: git-remote-https origin https://secure-server.example/repo\n  11:07:30.412575 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n  11:07:30.456285 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n  11:07:30.456318 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n  11:07:30.456439 run-command.c:673       trace: run_command: 'git credential-cache get'\n  11:07:30.461266 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n  11:07:30.461282 http.c:849              => Send header: Authorization: Bearer <redacted>\n  11:07:30.501628 http.c:849              <= Recv header: HTTP/1.1 200 OK\n  11:07:34.725262 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n  11:07:34.725279 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request\n  11:07:34.761407 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n  11:07:34.761443 http.c:890              == Info: Bearer authentication problem, ignoring.\n  11:07:34.761453 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n  11:07:34.761509 http.c:890              == Info: The requested URL returned error: 401\n  11:07:34.761530 http.c:890              == Info: closing connection #0\n  11:07:34.761913 run-command.c:673       trace: run_command: 'git credential-cache erase'\n  11:07:34.761927 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-cache erase' 'git credential-cache erase'\n  11:07:34.768069 git.c:502               trace: built-in: git credential-cache erase\n  11:07:34.768690 run-command.c:673       trace: run_command: 'git credential-msal erase'\n  11:07:34.768713 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-msal erase' 'git credential-msal erase'\n  11:07:34.772742 git.c:808               trace: exec: git-credential-msal erase\n  11:07:34.772783 run-command.c:673       trace: run_command: git-credential-msal erase\n  11:07:34.772819 run-command.c:765       trace: start_command: /usr/bin/git-credential-msal erase\n  error: RPC failed; HTTP 401 curl 22 The requested URL returned error: 401\n  fatal: unable to write request to remote: Broken pipe\n  fatal: could not fetch c4fff0229c9be06ecf576356a4d39a8a755b8d81 from promisor remote\n  warning: Clone succeeded, but checkout failed.\n  You can inspect what was checked out with 'git status'\n  and retry with 'git restore --source=HEAD :/'\n\nIn this case, the HTTP_REAUTH retry logic is not used because the\ncredential helper didn't set the 'continue' flag, so\nhttp_auth.multistage is false and handle_curl_result() fails with\nHTTP_NOAUTH instead.\n\nFix the immediate problem by including the authorization headers in the\nprobe_rpc() request as well.\n\nAdd a test for this scenario:\n\n 1. Create a repository with two thousand refs.\n 2. Clone that into the web root used by t5563-simple-http-auth.sh.\n 3. Configure http.postBuffer to be very small in order to trigger the\n    probe_rpc() path that fails.\n 4. Clone using a valid Bearer token.\n\n[1] https://github.com/Binary-Eater/git-credential-msal\n\nSigned-off-by: Aaron Plattner <aplattner@nvidia.com>\nTested-by: Lucas De Marchi <demarchi@kernel.org>\n---\nv2: Update commit description to clarify why HTTP_REAUTH doesn't help here, and\nadd a test to verify the fix.\n\n remote-curl.c               |  2 ++\n t/t5563-simple-http-auth.sh | 46 +++++++++++++++++++++++++++++++++++++\n 2 files changed, 48 insertions(+)\n\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 69f919454a..1d0ae72521 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -877,6 +877,8 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n \theaders = curl_slist_append(headers, rpc->hdr_content_type);\n \theaders = curl_slist_append(headers, rpc->hdr_accept);\n \n+\theaders = http_append_auth_header(&http_auth, headers);\n+\n \tcurl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0L);\n \tcurl_easy_setopt(slot->curl, CURLOPT_POST, 1L);\n \tcurl_easy_setopt(slot->curl, CURLOPT_URL, rpc->service_url);\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex c1febbae9d..adc210cdd3 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -63,6 +63,26 @@ test_expect_success 'setup repository' '\n \tgit push --mirror \"$HTTPD_DOCUMENT_ROOT_PATH/repo.git\"\n '\n \n+test_expect_success 'setup large repository for probe_rpc testing' '\n+\trm -rf large.git &&\n+\tgit init large.git &&\n+\t(\n+\t\tcd large.git &&\n+\t\tgit config set maintenance.auto false &&\n+\t\tgit commit --allow-empty --message \"initial\" &&\n+\t\t# Create many refs to trigger probe_rpc, which is called when\n+\t\t# the request body is larger than http.postBuffer.\n+\t\t#\n+\t\t# In the test later, http.postBuffer is set to 70000. Each\n+\t\t# \"want\" line is ~45 bytes, so we need at least 70000/45 = ~1600\n+\t\t# refs\n+\t\tprintf \"create refs/heads/branch-%d @\\n\" $(test_seq 2000) |\n+\t\t    git update-ref --stdin\n+\t) &&\n+\tgit clone --bare large.git \"$HTTPD_DOCUMENT_ROOT_PATH/large.git\" &&\n+\trm -rf large.git\n+'\n+\n test_expect_success 'access using basic auth' '\n \ttest_when_finished \"per_test_cleanup\" &&\n \n@@ -605,6 +625,32 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \tEOF\n '\n \n+test_expect_success 'clone with bearer auth and probe_rpc' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\t# Bearer token\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tid=1 creds=Bearer YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\n+\tEOF\n+\n+\t# Set a small buffer to force probe_rpc to be called\n+\t# Must be > LARGE_PACKET_MAX (65520)\n+\ttest_config_global http.postBuffer 70000 &&\n+\ttest_config_global credential.helper test-helper &&\n+\tgit clone \"$HTTPD_URL/custom_auth/large.git\" partial-auth-clone 2>clone-error\n+'\n+\n test_expect_success 'access using three-legged auth' '\n \ttest_when_finished \"per_test_cleanup\" &&\n \n-- \n2.52.0\n\n"},{"id":"533705","messageId":"aWXq3Lrzx9NGnwGb@pks.im","threadId":"64788","inReplyTo":"20260113031929.3746753-1-aplattner@nvidia.com","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-01-13T06:49:00Z","receivedAt":"2026-01-13T06:49:11Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Jan 12, 2026 at 07:19:28PM -0800, Aaron Plattner wrote:\n> If a large request requires post_rpc() to call probe_rpc(), the latter\n> does not use the authorization credentials used for other requests. If\n> this fails with an HTTP 401 error and http_auth.multistage isn't set,\n> then the whole request just fails.\n> \n> For example, using git-credential-msal [1], the following attempt to clone a\n> large repository fails partway through because the initial request to download\n> the commit history and promisor packs succeeds, but the\n> subsequent request to download the blobs needed to construct the working\n> tree fails with a 401 error and the checkout fails.\n> \n> (lines removed for brevity)\n> \n>   git clone --filter=blob:none https://secure-server.example/repo\n>   11:03:26.855369 git.c:502               trace: built-in: git clone --filter=blob:none https://secure-server.example/repo\n>   Cloning into 'sw'...\n>   warning: templates not found in /home/aaron/share/git-core/templates\n>   11:03:26.857169 run-command.c:673       trace: run_command: git remote-https origin https://secure-server.example/repo\n>   11:03:27.012104 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n>   11:03:27.049243 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n>   11:03:27.049270 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n>   11:03:27.053786 run-command.c:673       trace: run_command: 'git credential-msal get'\n>   11:03:27.952830 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n>   11:03:27.952849 http.c:849              => Send header: Authorization: Bearer <redacted>\n>   11:03:27.995419 http.c:849              <= Recv header: HTTP/1.1 200 OK\n>   11:03:28.230039 http.c:890              == Info: Reusing existing https: connection with host secure-server.example\n>   11:03:28.230208 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n>   11:03:28.230216 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request\n>   11:03:28.230221 http.c:849              => Send header: Authorization: Bearer <redacted>\n>   11:03:28.269085 http.c:849              <= Recv header: HTTP/1.1 200 OK\n>   11:03:28.684163 http.c:890              == Info: Reusing existing https: connection with host secure-server.example\n>   11:03:28.684379 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n>   11:03:28.684391 http.c:849              => Send header: Accept: application/x-git-upload-pack-result\n>   11:03:28.684393 http.c:849              => Send header: Authorization: Bearer <redacted>\n>   11:03:28.869546 run-command.c:673       trace: run_command: git index-pack --stdin --fix-thin '--keep=fetch-pack 43856 on dgx-spark' --promisor\n>   11:06:39.861237 run-command.c:673       trace: run_command: git -c fetch.negotiationAlgorithm=noop fetch origin --no-tags --no-write-fetch-head --recurse-submodules=no --filter=blob:none --stdin\n>   11:06:39.865981 run-command.c:673       trace: run_command: git remote-https origin https://secure-server.example/repo\n>   11:06:39.868039 run-command.c:673       trace: run_command: git-remote-https origin https://secure-server.example/repo\n>   11:07:30.412575 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n>   11:07:30.456285 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n>   11:07:30.456318 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n>   11:07:30.456439 run-command.c:673       trace: run_command: 'git credential-cache get'\n>   11:07:30.461266 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1\n>   11:07:30.461282 http.c:849              => Send header: Authorization: Bearer <redacted>\n>   11:07:30.501628 http.c:849              <= Recv header: HTTP/1.1 200 OK\n>   11:07:34.725262 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n>   11:07:34.725279 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request\n>   11:07:34.761407 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n>   11:07:34.761443 http.c:890              == Info: Bearer authentication problem, ignoring.\n>   11:07:34.761453 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n>   11:07:34.761509 http.c:890              == Info: The requested URL returned error: 401\n>   11:07:34.761530 http.c:890              == Info: closing connection #0\n>   11:07:34.761913 run-command.c:673       trace: run_command: 'git credential-cache erase'\n>   11:07:34.761927 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-cache erase' 'git credential-cache erase'\n>   11:07:34.768069 git.c:502               trace: built-in: git credential-cache erase\n>   11:07:34.768690 run-command.c:673       trace: run_command: 'git credential-msal erase'\n>   11:07:34.768713 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-msal erase' 'git credential-msal erase'\n>   11:07:34.772742 git.c:808               trace: exec: git-credential-msal erase\n>   11:07:34.772783 run-command.c:673       trace: run_command: git-credential-msal erase\n>   11:07:34.772819 run-command.c:765       trace: start_command: /usr/bin/git-credential-msal erase\n>   error: RPC failed; HTTP 401 curl 22 The requested URL returned error: 401\n>   fatal: unable to write request to remote: Broken pipe\n>   fatal: could not fetch c4fff0229c9be06ecf576356a4d39a8a755b8d81 from promisor remote\n>   warning: Clone succeeded, but checkout failed.\n>   You can inspect what was checked out with 'git status'\n>   and retry with 'git restore --source=HEAD :/'\n> \n> In this case, the HTTP_REAUTH retry logic is not used because the\n> credential helper didn't set the 'continue' flag, so\n> http_auth.multistage is false and handle_curl_result() fails with\n> HTTP_NOAUTH instead.\n\nNice, this is a helpful little detail.\n\n> diff --git a/remote-curl.c b/remote-curl.c\n> index 69f919454a..1d0ae72521 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -877,6 +877,8 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n>  \theaders = curl_slist_append(headers, rpc->hdr_content_type);\n>  \theaders = curl_slist_append(headers, rpc->hdr_accept);\n>  \n> +\theaders = http_append_auth_header(&http_auth, headers);\n> +\n>  \tcurl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0L);\n>  \tcurl_easy_setopt(slot->curl, CURLOPT_POST, 1L);\n>  \tcurl_easy_setopt(slot->curl, CURLOPT_URL, rpc->service_url);\n> diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\n> index c1febbae9d..adc210cdd3 100755\n> --- a/t/t5563-simple-http-auth.sh\n> +++ b/t/t5563-simple-http-auth.sh\n> @@ -63,6 +63,26 @@ test_expect_success 'setup repository' '\n>  \tgit push --mirror \"$HTTPD_DOCUMENT_ROOT_PATH/repo.git\"\n>  '\n>  \n> +test_expect_success 'setup large repository for probe_rpc testing' '\n> +\trm -rf large.git &&\n\nNo previous test creates this directory, so it shouldn't ever exist.\n\n> +\tgit init large.git &&\n> +\t(\n> +\t\tcd large.git &&\n> +\t\tgit config set maintenance.auto false &&\n> +\t\tgit commit --allow-empty --message \"initial\" &&\n> +\t\t# Create many refs to trigger probe_rpc, which is called when\n> +\t\t# the request body is larger than http.postBuffer.\n> +\t\t#\n> +\t\t# In the test later, http.postBuffer is set to 70000. Each\n> +\t\t# \"want\" line is ~45 bytes, so we need at least 70000/45 = ~1600\n> +\t\t# refs\n> +\t\tprintf \"create refs/heads/branch-%d @\\n\" $(test_seq 2000) |\n> +\t\t    git update-ref --stdin\n> +\t) &&\n> +\tgit clone --bare large.git \"$HTTPD_DOCUMENT_ROOT_PATH/large.git\" &&\n> +\trm -rf large.git\n\nYou can use \"test_when_finished rm -rf large.git\" before creating the\nrepo to ensure it's always getting pruned at the end of the test.\n\n> +'\n> +\n>  test_expect_success 'access using basic auth' '\n>  \ttest_when_finished \"per_test_cleanup\" &&\n>  \n\nI think we can simply inline this setup into the below test. We only use\nit in that specific test anyway.\n\n> @@ -605,6 +625,32 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n>  \tEOF\n>  '\n>  \n> +test_expect_success 'clone with bearer auth and probe_rpc' '\n> +\ttest_when_finished \"per_test_cleanup\" &&\n> +\n> +\tset_credential_reply get <<-EOF &&\n> +\tcapability[]=authtype\n> +\tauthtype=Bearer\n> +\tcredential=YS1naXQtdG9rZW4=\n> +\tEOF\n> +\n> +\t# Bearer token\n> +\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n> +\tid=1 creds=Bearer YS1naXQtdG9rZW4=\n> +\tEOF\n> +\n> +\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n> +\tid=1 status=200\n> +\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\n> +\tEOF\n> +\n> +\t# Set a small buffer to force probe_rpc to be called\n> +\t# Must be > LARGE_PACKET_MAX (65520)\n> +\ttest_config_global http.postBuffer 70000 &&\n> +\ttest_config_global credential.helper test-helper &&\n> +\tgit clone \"$HTTPD_URL/custom_auth/large.git\" partial-auth-clone 2>clone-error\n> +'\n> +\n\nYup, I've verified that this test fails when the bug fix is reverted.\nGood.\n\nYou might want to reroll to address my two nits around the test, but\neven without that reroll I'm already happy with this v2. Thanks!\n\nPatrick\n"},{"id":"533738","messageId":"xmqqfr89lkve.fsf@gitster.g","threadId":"64788","inReplyTo":"20260113031929.3746753-1-aplattner@nvidia.com","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-01-13T13:09:41Z","receivedAt":"2026-01-13T13:09:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Aaron Plattner <aplattner@nvidia.com> writes:\n\n> Subject: Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too\n\nMicronit.  \"Use\" -> \"use\" (see \"git shortlog --no-merges -200\nmaster\" and notice that the usual \"the first word of the sentence is\nupcased\" rule does not typically apply to the word after \"<area>: \"\nprefix).\n\n> If a large request requires post_rpc() to call probe_rpc(), the latter\n> does not use the authorization credentials used for other requests. If\n> this fails with an HTTP 401 error and http_auth.multistage isn't set,\n> then the whole request just fails.\n\n> For example, using git-credential-msal [1], the following attempt to clone a\n> large repository fails partway through because the initial request to download\n> the commit history and promisor packs succeeds, but the\n> subsequent request to download the blobs needed to construct the working\n> tree fails with a 401 error and the checkout fails.\n>\n> (lines removed for brevity)\n>\n>   git clone --filter=blob:none https://secure-server.example/repo\n>   ...\n>   11:07:34.725262 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n>   11:07:34.725279 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request\n>   11:07:34.761407 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n>   11:07:34.761443 http.c:890              == Info: Bearer authentication problem, ignoring.\n>   11:07:34.761453 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n>   11:07:34.761509 http.c:890              == Info: The requested URL returned error: 401\n>   11:07:34.761530 http.c:890              == Info: closing connection #0\n\nExcellent illustration here.\n\n>   11:07:34.761913 run-command.c:673       trace: run_command: 'git credential-cache erase'\n>   11:07:34.761927 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-cache erase' 'git credential-cache erase'\n>   11:07:34.768069 git.c:502               trace: built-in: git credential-cache erase\n>   11:07:34.768690 run-command.c:673       trace: run_command: 'git credential-msal erase'\n>   11:07:34.768713 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-msal erase' 'git credential-msal erase'\n>   11:07:34.772742 git.c:808               trace: exec: git-credential-msal erase\n>   11:07:34.772783 run-command.c:673       trace: run_command: git-credential-msal erase\n>   11:07:34.772819 run-command.c:765       trace: start_command: /usr/bin/git-credential-msal erase\n>   error: RPC failed; HTTP 401 curl 22 The requested URL returned error: 401\n>   fatal: unable to write request to remote: Broken pipe\n>   fatal: could not fetch c4fff0229c9be06ecf576356a4d39a8a755b8d81 from promisor remote\n>   warning: Clone succeeded, but checkout failed.\n>   You can inspect what was checked out with 'git status'\n>   and retry with 'git restore --source=HEAD :/'\n>\n> In this case, the HTTP_REAUTH retry logic is not used because the\n> credential helper didn't set the 'continue' flag, so\n> http_auth.multistage is false and handle_curl_result() fails with\n> HTTP_NOAUTH instead.\n>\n> Fix the immediate problem by including the authorization headers in the\n> probe_rpc() request as well.\n\nGreat.\n\n> diff --git a/remote-curl.c b/remote-curl.c\n> index 69f919454a..1d0ae72521 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -877,6 +877,8 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n>  \theaders = curl_slist_append(headers, rpc->hdr_content_type);\n>  \theaders = curl_slist_append(headers, rpc->hdr_accept);\n>  \n> +\theaders = http_append_auth_header(&http_auth, headers);\n> +\n\nIs http_auth headers so different from the content-type and accept\nin the larger picture to warrant the blank line before this new\ncall?  If not, you probably would want to have these three\nassignments to \"headers\" that accumulates the header lines together\nin a single block of three lines without any blank line in between.\n\n>  \tcurl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0L);\n>  \tcurl_easy_setopt(slot->curl, CURLOPT_POST, 1L);\n>  \tcurl_easy_setopt(slot->curl, CURLOPT_URL, rpc->service_url);\n> diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\n> index c1febbae9d..adc210cdd3 100755\n> --- a/t/t5563-simple-http-auth.sh\n> +++ b/t/t5563-simple-http-auth.sh\n> @@ -63,6 +63,26 @@ test_expect_success 'setup repository' '\n>  \tgit push --mirror \"$HTTPD_DOCUMENT_ROOT_PATH/repo.git\"\n>  '\n>  \n> +test_expect_success 'setup large repository for probe_rpc testing' '\n> +\trm -rf large.git &&\n> +\tgit init large.git &&\n> +\t(\n> +\t\tcd large.git &&\n> +\t\tgit config set maintenance.auto false &&\n> +\t\tgit commit --allow-empty --message \"initial\" &&\n> +\t\t# Create many refs to trigger probe_rpc, which is called when\n> +\t\t# the request body is larger than http.postBuffer.\n> +\t\t#\n> +\t\t# In the test later, http.postBuffer is set to 70000. Each\n> +\t\t# \"want\" line is ~45 bytes, so we need at least 70000/45 = ~1600\n> +\t\t# refs\n> +\t\tprintf \"create refs/heads/branch-%d @\\n\" $(test_seq 2000) |\n> +\t\t    git update-ref --stdin\n> +\t) &&\n\nHopefully, $(test_seq 2000) would not bust $(sysconf ARG_MAX), which\ncould be as low as 4KB, on any system we care about.  If not, of\ncourse we could\n\n\ttest_seq 2000 |\n\txargs printf \"create ...\\n\" |\n\tgit update-ref --stdin\n\nwhich probably is not all that more expensive than what you wrote above.\n\nOther than that, looking great.  Thanks for working on this.\n"},{"id":"533785","messageId":"a919f4cf-8355-43dd-a552-df99325e4cc6@nvidia.com","threadId":"64788","inReplyTo":"xmqqfr89lkve.fsf@gitster.g","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Aaron Plattner","fromEmail":"aplattner@nvidia.com","sentAt":"2026-01-14T01:06:20Z","receivedAt":"2026-01-14T01:06:39Z","isPatch":true,"sender":{"key":"aplattner@nvidia.com","avatar":"https://avatars.githubusercontent.com/u/343551?v=4"},"body":"On 1/13/26 5:09 AM, Junio C Hamano wrote:\n> Aaron Plattner <aplattner@nvidia.com> writes:\n> \n>> Subject: Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too\n> \n> Micronit.  \"Use\" -> \"use\" (see \"git shortlog --no-merges -200\n> master\" and notice that the usual \"the first word of the sentence is\n> upcased\" rule does not typically apply to the word after \"<area>: \"\n> prefix).\n\nThanks, sorry for missing that.\n\n>> If a large request requires post_rpc() to call probe_rpc(), the latter\n>> does not use the authorization credentials used for other requests. If\n>> this fails with an HTTP 401 error and http_auth.multistage isn't set,\n>> then the whole request just fails.\n> \n>> For example, using git-credential-msal [1], the following attempt to clone a\n>> large repository fails partway through because the initial request to download\n>> the commit history and promisor packs succeeds, but the\n>> subsequent request to download the blobs needed to construct the working\n>> tree fails with a 401 error and the checkout fails.\n>>\n>> (lines removed for brevity)\n>>\n>>    git clone --filter=blob:none https://secure-server.example/repo\n>>    ...\n>>    11:07:34.725262 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1\n>>    11:07:34.725279 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request\n>>    11:07:34.761407 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized\n>>    11:07:34.761443 http.c:890              == Info: Bearer authentication problem, ignoring.\n>>    11:07:34.761453 http.c:849              <= Recv header: WWW-Authenticate: Bearer error=\"invalid_request\", error_description=\"No bearer token found in the request\", msal-tenant-id=\"<tenant>\", msal-client-id=\"<client>\"\n>>    11:07:34.761509 http.c:890              == Info: The requested URL returned error: 401\n>>    11:07:34.761530 http.c:890              == Info: closing connection #0\n> \n> Excellent illustration here.\n> \n>>    11:07:34.761913 run-command.c:673       trace: run_command: 'git credential-cache erase'\n>>    11:07:34.761927 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-cache erase' 'git credential-cache erase'\n>>    11:07:34.768069 git.c:502               trace: built-in: git credential-cache erase\n>>    11:07:34.768690 run-command.c:673       trace: run_command: 'git credential-msal erase'\n>>    11:07:34.768713 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-msal erase' 'git credential-msal erase'\n>>    11:07:34.772742 git.c:808               trace: exec: git-credential-msal erase\n>>    11:07:34.772783 run-command.c:673       trace: run_command: git-credential-msal erase\n>>    11:07:34.772819 run-command.c:765       trace: start_command: /usr/bin/git-credential-msal erase\n>>    error: RPC failed; HTTP 401 curl 22 The requested URL returned error: 401\n>>    fatal: unable to write request to remote: Broken pipe\n>>    fatal: could not fetch c4fff0229c9be06ecf576356a4d39a8a755b8d81 from promisor remote\n>>    warning: Clone succeeded, but checkout failed.\n>>    You can inspect what was checked out with 'git status'\n>>    and retry with 'git restore --source=HEAD :/'\n>>\n>> In this case, the HTTP_REAUTH retry logic is not used because the\n>> credential helper didn't set the 'continue' flag, so\n>> http_auth.multistage is false and handle_curl_result() fails with\n>> HTTP_NOAUTH instead.\n>>\n>> Fix the immediate problem by including the authorization headers in the\n>> probe_rpc() request as well.\n> \n> Great.\n> \n>> diff --git a/remote-curl.c b/remote-curl.c\n>> index 69f919454a..1d0ae72521 100644\n>> --- a/remote-curl.c\n>> +++ b/remote-curl.c\n>> @@ -877,6 +877,8 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)\n>>   \theaders = curl_slist_append(headers, rpc->hdr_content_type);\n>>   \theaders = curl_slist_append(headers, rpc->hdr_accept);\n>>   \n>> +\theaders = http_append_auth_header(&http_auth, headers);\n>> +\n> \n> Is http_auth headers so different from the content-type and accept\n> in the larger picture to warrant the blank line before this new\n> call?  If not, you probably would want to have these three\n> assignments to \"headers\" that accumulates the header lines together\n> in a single block of three lines without any blank line in between.\n\nNo, that's fair.\n\n>>   \tcurl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0L);\n>>   \tcurl_easy_setopt(slot->curl, CURLOPT_POST, 1L);\n>>   \tcurl_easy_setopt(slot->curl, CURLOPT_URL, rpc->service_url);\n>> diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\n>> index c1febbae9d..adc210cdd3 100755\n>> --- a/t/t5563-simple-http-auth.sh\n>> +++ b/t/t5563-simple-http-auth.sh\n>> @@ -63,6 +63,26 @@ test_expect_success 'setup repository' '\n>>   \tgit push --mirror \"$HTTPD_DOCUMENT_ROOT_PATH/repo.git\"\n>>   '\n>>   \n>> +test_expect_success 'setup large repository for probe_rpc testing' '\n>> +\trm -rf large.git &&\n>> +\tgit init large.git &&\n>> +\t(\n>> +\t\tcd large.git &&\n>> +\t\tgit config set maintenance.auto false &&\n>> +\t\tgit commit --allow-empty --message \"initial\" &&\n>> +\t\t# Create many refs to trigger probe_rpc, which is called when\n>> +\t\t# the request body is larger than http.postBuffer.\n>> +\t\t#\n>> +\t\t# In the test later, http.postBuffer is set to 70000. Each\n>> +\t\t# \"want\" line is ~45 bytes, so we need at least 70000/45 = ~1600\n>> +\t\t# refs\n>> +\t\tprintf \"create refs/heads/branch-%d @\\n\" $(test_seq 2000) |\n>> +\t\t    git update-ref --stdin\n>> +\t) &&\n> \n> Hopefully, $(test_seq 2000) would not bust $(sysconf ARG_MAX), which\n> could be as low as 4KB, on any system we care about.  If not, of\n> course we could\n> \n> \ttest_seq 2000 |\n> \txargs printf \"create ...\\n\" |\n> \tgit update-ref --stdin\n> \n> which probably is not all that more expensive than what you wrote above.\n\nThat's a good call. I tried this\n\n\ttest_seq 2000 |\n\txargs printf \"create refs/heads/branch-%d @\\n\" |\n\tgit update-ref --stdin\n\nand verified it produces the same results, as does the same plus passing \n\"-n 10\" to xargs.\n\n> Other than that, looking great.  Thanks for working on this.\n\nThanks Junio, I'll send a v4 with these updates.\n\n-- Aaron\n"},{"id":"533788","messageId":"20260114022057.GA858110@coredump.intra.peff.net","threadId":"64788","inReplyTo":"a919f4cf-8355-43dd-a552-df99325e4cc6@nvidia.com","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-01-14T02:20:57Z","receivedAt":"2026-01-14T02:20:59Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jan 13, 2026 at 05:06:20PM -0800, Aaron Plattner wrote:\n\n> > Hopefully, $(test_seq 2000) would not bust $(sysconf ARG_MAX), which\n> > could be as low as 4KB, on any system we care about.  If not, of\n> > course we could\n> > \n> > \ttest_seq 2000 |\n> > \txargs printf \"create ...\\n\" |\n> > \tgit update-ref --stdin\n> > \n> > which probably is not all that more expensive than what you wrote above.\n> \n> That's a good call. I tried this\n> \n> \ttest_seq 2000 |\n> \txargs printf \"create refs/heads/branch-%d @\\n\" |\n> \tgit update-ref --stdin\n> \n> and verified it produces the same results, as does the same plus passing \"-n\n> 10\" to xargs.\n\ntest_seq can take a format parameter these days, so just:\n\n  test_seq -f \"create refs/heads/branch-%d @\" |\n  git update-ref --stdin\n\nis enough, and saves some processes.\n\n-Peff\n"},{"id":"533824","messageId":"xmqqikd4gu7r.fsf@gitster.g","threadId":"64788","inReplyTo":"20260114022057.GA858110@coredump.intra.peff.net","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-01-14T14:11:20Z","receivedAt":"2026-01-14T14:11:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> test_seq can take a format parameter these days, so just:\n>\n>   test_seq -f \"create refs/heads/branch-%d @\" |\n>   git update-ref --stdin\n>\n> is enough, and saves some processes.\n\nAhh, I forgot about that one.  Good suggestion.\n\nThanks.\n"},{"id":"533840","messageId":"613b47af-5269-44e3-87f5-d29fba9d73b3@nvidia.com","threadId":"64788","inReplyTo":"xmqqikd4gu7r.fsf@gitster.g","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Aaron Plattner","fromEmail":"aplattner@nvidia.com","sentAt":"2026-01-14T16:33:43Z","receivedAt":"2026-01-14T16:34:52Z","isPatch":true,"sender":{"key":"aplattner@nvidia.com","avatar":"https://avatars.githubusercontent.com/u/343551?v=4"},"body":"On 1/14/26 6:11 AM, Junio C Hamano wrote:\n> Jeff King <peff@peff.net> writes:\n> \n>> test_seq can take a format parameter these days, so just:\n>>\n>>    test_seq -f \"create refs/heads/branch-%d @\" |\n>>    git update-ref --stdin\n>>\n>> is enough, and saves some processes.\n> \n> Ahh, I forgot about that one.  Good suggestion.\n> \n> Thanks.\n\nOoh, neat. I guess I copied the wrong example. I verified that this \nworks too, so I can send a v5 for that.\n\nIs it worth changing up the other cases of this pattern, mostly in \npack-refs-tests.sh? E.g.,\n\n\t# Create 15 loose references.\n\tprintf \"create refs/heads/loose-%d HEAD\\n\" $(test_seq 15) >stdin &&\n\tgit update-ref --stdin <stdin &&\n\n[...]\n\n\t# Create 99 packed refs. This should cause the heuristic\n\t# to require more than the minimum amount of loose refs.\n\ttest_seq 99 |\n\twhile read i\n\tdo\n\t\tprintf \"create refs/heads/packed-%d HEAD\\n\" $i || return 1\n\tdone >stdin &&\n\tgit update-ref --stdin <stdin &&\n\nI can put together a patch for those.\n\n-- Aaron\n"},{"id":"533847","messageId":"20260114173055.GD885771@coredump.intra.peff.net","threadId":"64788","inReplyTo":"613b47af-5269-44e3-87f5-d29fba9d73b3@nvidia.com","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-01-14T17:30:55Z","receivedAt":"2026-01-14T17:30:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jan 14, 2026 at 08:33:43AM -0800, Aaron Plattner wrote:\n\n> Ooh, neat. I guess I copied the wrong example. I verified that this works\n> too, so I can send a v5 for that.\n> \n> Is it worth changing up the other cases of this pattern, mostly in\n> pack-refs-tests.sh? E.g.,\n> \n> \t# Create 15 loose references.\n> \tprintf \"create refs/heads/loose-%d HEAD\\n\" $(test_seq 15) >stdin &&\n> \tgit update-ref --stdin <stdin &&\n> \n> [...]\n> \n> \t# Create 99 packed refs. This should cause the heuristic\n> \t# to require more than the minimum amount of loose refs.\n> \ttest_seq 99 |\n> \twhile read i\n> \tdo\n> \t\tprintf \"create refs/heads/packed-%d HEAD\\n\" $i || return 1\n> \tdone >stdin &&\n> \tgit update-ref --stdin <stdin &&\n> \n> I can put together a patch for those.\n\nYeah, I think they are worth updating. I looked for spots to convert\nwhen I added the feature in b32c7ec02f (test-lib: teach test_seq the -f\noption, 2025-06-23). But I missed those ones.\n\nI think I grepped for \"for i in $(test_seq ...)\", but the use of the\nwhile loop and the $()-substitution meant I didn't see them. Examining\nevery test_seq call didn't seem worthwhile, as there are hundreds. ;)\n\nGrepping for \"test_seq .* |\" does yield a few more (e.g., p5551), but\nmost are false positives. Grepping for \"printf.*$(test_seq\" gets some\nmore.\n\nI don't know that we need to exhaustively find all cases. ;) In the\nfirst case above, it does save us a subshell. In the second case I think\nit saves us a subshell _and_ the result is much easier to read (because\nit avoids the loop and return). So IMHO it's a nice improvement, but\nthere's diminishing returns to investigating every test_seq call.\n\n-Peff\n"},{"id":"533852","messageId":"xmqqcy3cf5xa.fsf@gitster.g","threadId":"64788","inReplyTo":"613b47af-5269-44e3-87f5-d29fba9d73b3@nvidia.com","subject":"Re: [PATCH v2] remote-curl: Use auth for probe_rpc() requests too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-01-14T17:41:21Z","receivedAt":"2026-01-14T17:41:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Aaron Plattner <aplattner@nvidia.com> writes:\n\n> Is it worth changing up the other cases of this pattern, mostly in \n> pack-refs-tests.sh? E.g.,\n>\n> \t# Create 15 loose references.\n> \tprintf \"create refs/heads/loose-%d HEAD\\n\" $(test_seq 15) >stdin &&\n> \tgit update-ref --stdin <stdin &&\n>\n> [...]\n>\n> \t# Create 99 packed refs. This should cause the heuristic\n> \t# to require more than the minimum amount of loose refs.\n> \ttest_seq 99 |\n> \twhile read i\n> \tdo\n> \t\tprintf \"create refs/heads/packed-%d HEAD\\n\" $i || return 1\n> \tdone >stdin &&\n> \tgit update-ref --stdin <stdin &&\n>\n> I can put together a patch for those.\n\nI am fairly sure that these existing ones were written way before\nthe feature in test_seq to use the format parameter got popular.  A\nseparate patch to clean them up would be a good addition but of\ncourse should be outside the current topic ;-)\n\nThanks.\n"}]}