{"thread":{"id":"61518","subject":"[PATCH] http: display the response body on POST failure","startedAt":"2024-05-20T21:09:31Z","lastAt":"2024-05-23T09:29:59Z","messageCount":3,"participants":["Boris Mbarga via GitGitGadget","brian m. carlson","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"495120","messageId":"pull.1722.git.git.1716239367046.gitgitgadget@gmail.com","threadId":"61518","inReplyTo":null,"subject":"[PATCH] http: display the response body on POST failure","fromName":"Boris Mbarga via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2024-05-20T21:09:26Z","receivedAt":"2024-05-20T21:09:31Z","isPatch":true,"sender":{"key":"name:Boris Mbarga","avatar":null},"body":"From: elhmn <elhmn@github.com>\n\nWhen Git sends a GET request and receives an HTTP code indicating\nfailure (and that failure does not indicate an authentication problem),\nit shows the body of the response, i.e. the error message.\nThe same is not true for POST requests. However, it would be good to show\nthose error messages e.g. in the case of \"429 Too many requests\", because\nthe user might otherwise be left puzzled about the reason why their clone\ndid not work.\n\nThis patch aligns the way POST requests are handled with the GET request\nhandling.\n\nSigned-off-by: elhmn <elhmn@github.com>\n---\n    http: display response body on POST failure\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1722%2Felhmn%2Fdisplay-rpc-post-err-message-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1722/elhmn/display-rpc-post-err-message-v1\nPull-Request: https://github.com/git/git/pull/1722\n\n http.c                               |  1 +\n http.h                               |  1 +\n remote-curl.c                        | 11 ++++++++++-\n t/lib-httpd.sh                       |  1 +\n t/lib-httpd/apache.conf              |  8 ++++++++\n t/lib-httpd/wrap-git-http-backend.sh |  7 +++++++\n t/t5551-http-fetch-smart.sh          |  5 +++++\n 7 files changed, 33 insertions(+), 1 deletion(-)\n create mode 100755 t/lib-httpd/wrap-git-http-backend.sh\n\ndiff --git a/http.c b/http.c\nindex 3d80bd6116e..2017e909054 100644\n--- a/http.c\n+++ b/http.c\n@@ -1419,6 +1419,7 @@ struct active_request_slot *get_active_slot(void)\n \t\tnewslot->curl = NULL;\n \t\tnewslot->in_use = 0;\n \t\tnewslot->next = NULL;\n+\t\tnewslot->errstr = NULL;\n \n \t\tslot = active_queue_head;\n \t\tif (!slot) {\ndiff --git a/http.h b/http.h\nindex 3af19a8bf53..cb542c62933 100644\n--- a/http.h\n+++ b/http.h\n@@ -30,6 +30,7 @@ struct active_request_slot {\n \tvoid *callback_data;\n \tvoid (*callback_func)(void *data);\n \tstruct active_request_slot *next;\n+\tstruct strbuf *errstr;\n };\n \n struct buffer {\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 0b6d7815fdd..9b2a41b2451 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -804,8 +804,11 @@ static size_t rpc_in(char *ptr, size_t eltsize,\n \tif (curl_easy_getinfo(data->slot->curl, CURLINFO_RESPONSE_CODE,\n \t\t\t      &response_code) != CURLE_OK)\n \t\treturn size;\n-\tif (response_code >= 300)\n+\tif (response_code >= 300) {\n+\t\tstrbuf_reset(data->slot->errstr);\n+\t\tstrbuf_add(data->slot->errstr, ptr, size);\n \t\treturn size;\n+\t}\n \tif (size)\n \t\tdata->rpc->any_written = 1;\n \tif (data->check_pktline)\n@@ -837,6 +840,8 @@ static int run_slot(struct active_request_slot *slot,\n \t\t\t\tstrbuf_addch(&msg, ' ');\n \t\t\t\tstrbuf_addstr(&msg, curl_errorstr);\n \t\t\t}\n+\t\t\tif (slot->errstr && slot->errstr->len)\n+\t\t\t\terror(_(\"%s\"), slot->errstr->buf);\n \t\t}\n \t\terror(_(\"RPC failed; %s\"), msg.buf);\n \t\tstrbuf_release(&msg);\n@@ -896,6 +901,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \tint err, large_request = 0;\n \tint needs_100_continue = 0;\n \tstruct rpc_in_data rpc_in_data;\n+\tstruct strbuf errstr = STRBUF_INIT;\n \n \t/* Try to load the entire request, if we can fit it into the\n \t * allocated buffer space we can use HTTP/1.0 and avoid the\n@@ -1030,6 +1036,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \n \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);\n \tcurl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, rpc_in);\n+\tslot->errstr = &errstr;\n \trpc_in_data.rpc = rpc;\n \trpc_in_data.slot = slot;\n \trpc_in_data.check_pktline = stateless_connect;\n@@ -1040,6 +1047,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \n \trpc->any_written = 0;\n \terr = run_slot(slot, NULL);\n+\tslot->errstr = NULL;\n \tif (err == HTTP_REAUTH && !large_request) {\n \t\tcredential_fill(&http_auth);\n \t\tgoto retry;\n@@ -1060,6 +1068,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \n \tcurl_slist_free_all(headers);\n \tfree(gzip_body);\n+\tstrbuf_release(&errstr);\n \treturn err;\n }\n \ndiff --git a/t/lib-httpd.sh b/t/lib-httpd.sh\nindex d83bafeab32..8f5b6357176 100644\n--- a/t/lib-httpd.sh\n+++ b/t/lib-httpd.sh\n@@ -164,6 +164,7 @@ prepare_httpd() {\n \tinstall_script error-no-report.sh\n \tinstall_script broken-smart-http.sh\n \tinstall_script error-smart-http.sh\n+\tinstall_script wrap-git-http-backend.sh\n \tinstall_script error.sh\n \tinstall_script apply-one-time-perl.sh\n \tinstall_script nph-custom-auth.sh\ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 022276a6b9a..0ca58f54d72 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -108,6 +108,10 @@ SetEnv PERL_PATH ${PERL_PATH}\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n </LocationMatch>\n+<LocationMatch /smart_fail_on_post_with_body/>\n+\tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n+\tSetEnv GIT_HTTP_EXPORT_ALL\n+</LocationMatch>\n <LocationMatch /smart_noexport/>\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n </LocationMatch>\n@@ -155,6 +159,7 @@ ScriptAlias /smart/incomplete_length/git-upload-pack incomplete-length-upload-pa\n ScriptAlias /smart/incomplete_body/git-upload-pack incomplete-body-upload-pack-v2-http.sh/\n ScriptAlias /smart/no_report/git-receive-pack error-no-report.sh/\n ScriptAliasMatch /error_git_upload_pack/(.*)/git-upload-pack error.sh/\n+ScriptAliasMatch /smart_fail_on_post_with_body/(.*) wrap-git-http-backend.sh/$1\n ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error_smart/ error-smart-http.sh/\n@@ -182,6 +187,9 @@ ScriptAliasMatch /custom_auth/(.*) nph-custom-auth.sh/$1\n <Files error.sh>\n   Options ExecCGI\n </Files>\n+<Files wrap-git-http-backend.sh>\n+  Options ExecCGI\n+</Files>\n <Files apply-one-time-perl.sh>\n \tOptions ExecCGI\n </Files>\ndiff --git a/t/lib-httpd/wrap-git-http-backend.sh b/t/lib-httpd/wrap-git-http-backend.sh\nnew file mode 100755\nindex 00000000000..d1fc540330e\n--- /dev/null\n+++ b/t/lib-httpd/wrap-git-http-backend.sh\n@@ -0,0 +1,7 @@\n+if [ \"$REQUEST_METHOD\" = \"GET\" ]; then\n+\t\"$GIT_EXEC_PATH\"/git-http-backend \"$@\"\n+elif [ \"$REQUEST_METHOD\" = \"POST\" ]; then\n+\tprintf \"Status: 429 Too Many Requests\\n\"\n+\techo\n+\tprintf \"Too many requests\"\n+fi\ndiff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh\nindex a623a1058cd..badf37b4d68 100755\n--- a/t/t5551-http-fetch-smart.sh\n+++ b/t/t5551-http-fetch-smart.sh\n@@ -302,6 +302,11 @@ test_expect_success 'dumb clone via http-backend respects namespace' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'smart http should display an error message on POST request failure' '\n+\ttest_must_fail git clone --single-branch $HTTPD_URL/smart_fail_on_post_with_body/repo.git 2>actual &&\n+\tgrep -q \"error: Too many requests\" actual\n+'\n+\n test_expect_success 'cookies stored in http.cookiefile when http.savecookies set' '\n \tcat >cookies.txt <<-\\EOF &&\n \t127.0.0.1\tFALSE\t/smart_cookies/\tFALSE\t0\tothername\tothervalue\n\nbase-commit: 786a3e4b8d754d2b14b1208b98eeb0a554ef19a8\n-- \ngitgitgadget\n"},{"id":"495255","messageId":"Zk0wBltFytCGjUFv@tapette.crustytoothpaste.net","threadId":"61518","inReplyTo":"pull.1722.git.git.1716239367046.gitgitgadget@gmail.com","subject":"Re: [PATCH] http: display the response body on POST failure","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-05-21T23:36:38Z","receivedAt":"2024-05-21T23:36:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-05-20 at 21:09:26, Boris Mbarga via GitGitGadget wrote:\n> From: elhmn <elhmn@github.com>\n> \n> When Git sends a GET request and receives an HTTP code indicating\n> failure (and that failure does not indicate an authentication problem),\n> it shows the body of the response, i.e. the error message.\n> The same is not true for POST requests. However, it would be good to show\n> those error messages e.g. in the case of \"429 Too many requests\", because\n> the user might otherwise be left puzzled about the reason why their clone\n> did not work.\n\nI think this is a good idea (and not just because I talked about it with\nBoris).  If a user gets an error response, we want them to be able to\nreceive a helpful response from the server telling them what might have\ngone wrong.  While many of us know full well what a 429 response means,\nit isn't immediately obvious to many users (judging from my experience\non Stack Overflow), and we might still want to tell users why they got a\n429 (too many requests from their IP, network, token, application or\nintegration, etc.).\n\n> This patch aligns the way POST requests are handled with the GET request\n> handling.\n> \n> Signed-off-by: elhmn <elhmn@github.com>\n\nI believe the policy is to put your real name here and in the commit. We\nknow what it is already from the headers.  You can probably do `git -c\nuser.name=\"Boris Mbarga\" commit --amend --reset-author --signoff` to fix\nthat.\n\n> diff --git a/http.c b/http.c\n> index 3d80bd6116e..2017e909054 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -1419,6 +1419,7 @@ struct active_request_slot *get_active_slot(void)\n>  \t\tnewslot->curl = NULL;\n>  \t\tnewslot->in_use = 0;\n>  \t\tnewslot->next = NULL;\n> +\t\tnewslot->errstr = NULL;\n>  \n>  \t\tslot = active_queue_head;\n>  \t\tif (!slot) {\n> diff --git a/http.h b/http.h\n> index 3af19a8bf53..cb542c62933 100644\n> --- a/http.h\n> +++ b/http.h\n> @@ -30,6 +30,7 @@ struct active_request_slot {\n>  \tvoid *callback_data;\n>  \tvoid (*callback_func)(void *data);\n>  \tstruct active_request_slot *next;\n> +\tstruct strbuf *errstr;\n>  };\n>  \n>  struct buffer {\n> diff --git a/remote-curl.c b/remote-curl.c\n> index 0b6d7815fdd..9b2a41b2451 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -804,8 +804,11 @@ static size_t rpc_in(char *ptr, size_t eltsize,\n>  \tif (curl_easy_getinfo(data->slot->curl, CURLINFO_RESPONSE_CODE,\n>  \t\t\t      &response_code) != CURLE_OK)\n>  \t\treturn size;\n> -\tif (response_code >= 300)\n> +\tif (response_code >= 300) {\n> +\t\tstrbuf_reset(data->slot->errstr);\n> +\t\tstrbuf_add(data->slot->errstr, ptr, size);\n>  \t\treturn size;\n> +\t}\n>  \tif (size)\n>  \t\tdata->rpc->any_written = 1;\n>  \tif (data->check_pktline)\n> @@ -837,6 +840,8 @@ static int run_slot(struct active_request_slot *slot,\n>  \t\t\t\tstrbuf_addch(&msg, ' ');\n>  \t\t\t\tstrbuf_addstr(&msg, curl_errorstr);\n>  \t\t\t}\n> +\t\t\tif (slot->errstr && slot->errstr->len)\n> +\t\t\t\terror(_(\"%s\"), slot->errstr->buf);\n>  \t\t}\n>  \t\terror(_(\"RPC failed; %s\"), msg.buf);\n\nI think we should print the \"RPC failed\" message as well, and first.\nThat message, even if not super helpful, contains the status code and\nsome other information, which might be more helpful than what the remote\nserver actually said.\n\nWe might also want to print \"remote:\" in front of the error string, so\nthat the user knows it comes from the server instead of Git.  That can\nbe helpful when searching for it online.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"495370","messageId":"20240523092957.GF1306938@coredump.intra.peff.net","threadId":"61518","inReplyTo":"pull.1722.git.git.1716239367046.gitgitgadget@gmail.com","subject":"Re: [PATCH] http: display the response body on POST failure","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-05-23T09:29:57Z","receivedAt":"2024-05-23T09:29:59Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 20, 2024 at 09:09:26PM +0000, Boris Mbarga via GitGitGadget wrote:\n\n> From: elhmn <elhmn@github.com>\n> \n> When Git sends a GET request and receives an HTTP code indicating\n> failure (and that failure does not indicate an authentication problem),\n> it shows the body of the response, i.e. the error message.\n\nHmm, do we always do so?  Long ago, I implemented that for the initial\nHTTP connection in 426e70d4a1 (remote-curl: show server content on http\nerrors, 2013-04-05). That disables CURLOPT_FAILONERROR for that request,\nand then shows the result even if we saw an error.\n\nAfter that, we turned off CURLOPT_FAILONERROR for all requests in\ne6cf87b12d (http: enable keep_error for HTTP requests, 2019-01-10), with\nthe rationale that we'd show them with GIT_CURL_VERBOSE (and presumably\nGIT_TRACE_CURL, too). But do we actually write them out in most cases?\n\nI'm not opposed to doing so, but just trying to understand what the\nimplications might be (and whether we are really bringing POSTs in line,\nor if this is a new area).\n\nIn particular...\n\n> @@ -837,6 +840,8 @@ static int run_slot(struct active_request_slot *slot,\n>  \t\t\t\tstrbuf_addch(&msg, ' ');\n>  \t\t\t\tstrbuf_addstr(&msg, curl_errorstr);\n>  \t\t\t}\n> +\t\t\tif (slot->errstr && slot->errstr->len)\n> +\t\t\t\terror(_(\"%s\"), slot->errstr->buf);\n>  \t\t}\n>  \t\terror(_(\"RPC failed; %s\"), msg.buf);\n>  \t\tstrbuf_release(&msg);\n\nIf I understand correctly, slot->errstr is just the raw body content\nreturned by the request. That _might_ be something human-readable, but\nit might not. For that initial connection, we have show_http_message(),\nwhich shows only messages that come back as text/plain, does some light\ncleanup, and shows the error with the \"remote:\" prefix. We'd want to use\nthat here, I'd think?\n\n-Peff\n"}]}