{"thread":{"id":"61189","subject":"[PATCH 12/13] strvec: implement swapping two strvecs","startedAt":"2024-03-24T01:13:12Z","lastAt":"2024-04-17T00:03:12Z","messageCount":66,"participants":["brian m. carlson","Junio C Hamano","M Hickford","Patrick Steinhardt","Jeff King","mirth hickford","Calvin Wan","Jackson Toeniskoetter"],"isPatch":true,"patchVersion":1,"patchTotal":13},"messages":[{"id":"491325","messageId":"20240324011301.1553072-10-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 09/13] credential: enable state capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:57Z","receivedAt":"2024-03-24T01:13:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we've implemented the state capability, let's send it along by\ndefault when filling credentials so we can make use of it.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 1 +\n t/t5563-simple-http-auth.sh | 10 ++++++++++\n 2 files changed, 11 insertions(+)\n\ndiff --git a/credential.c b/credential.c\nindex 0cd7dd2a00..0ca7c12895 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -38,6 +38,7 @@ void credential_clear(struct credential *c)\n static void credential_set_all_capabilities(struct credential *c)\n {\n \tc->capa_authtype.request_initial = 1;\n+\tc->capa_state.request_initial = 1;\n }\n \n int credential_match(const struct credential *want,\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex b3ed0d9fc2..b098cd0fdf 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -75,6 +75,7 @@ test_expect_success 'access using basic auth' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -111,6 +112,7 @@ test_expect_success 'access using basic auth via authtype' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -147,6 +149,7 @@ test_expect_success 'access using basic auth invalid credentials' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -185,6 +188,7 @@ test_expect_success 'access using basic auth with extra challenges' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -224,6 +228,7 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=foobar param1=\"value1\" param2=\"value2\"\n@@ -268,6 +273,7 @@ test_expect_success 'access using basic auth with wwwauth header continuations'\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -314,6 +320,7 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -356,6 +363,7 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -397,6 +405,7 @@ test_expect_success 'access using bearer auth' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -440,6 +449,7 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n"},{"id":"491326","messageId":"20240324011301.1553072-9-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 08/13] credential: add an argument to keep state","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:56Z","receivedAt":"2024-03-24T01:13:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Until now, our credential code has mostly deal with usernames and\npasswords and we've let libcurl deal with the variant of authentication\nto be used. However, now that we have the credential value, the\ncredential helper can take control of the authentication, so the value\nprovided might be something that's generated, such as a Digest hash\nvalue.\n\nIn such a case, it would be helpful for a credential helper that gets an\nerase or store command to be able to keep track of an identifier for the\noriginal secret that went into the computation. Furthermore, some types\nof authentication, such as NTLM and Kerberos, actually need two round\ntrips to authenticate, which will require that the credential helper\nkeep some state.\n\nIn order to allow for these use cases and others, allow storing state in\na field called \"state[]\". This value is passed back to the credential\nhelper that created it, which avoids confusion caused by parsing values\nfrom different helpers.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 29 ++++++++++++++++++-----------\n credential.c | 20 +++++++++++++++++---\n credential.h | 7 +++++++\n t/t0300-credentials.sh | 29 +++++++++++++++++++++++++++++\n 4 files changed, 71 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex f3ed3a82fa..ef30c89c00 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -196,6 +196,15 @@ provided on input.\n This value should not be sent unless the appropriate capability (see below) is\n provided on input.\n \n+`state[]`::\n+\tThis value provides an opaque state that will be passed back to this helper\n+\tif it is called again. Each different credential helper may specify this\n+\tonce. The value should include a prefix unique to the credential helper and\n+\tshould ignore values that don't match its prefix.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\n@@ -208,18 +217,16 @@ they appear in the HTTP response. This attribute is 'one-way' from Git\n to pass additional information to credential helpers.\n \n `capability[]`::\n-\tThis signals that the caller supports the capability in question.\n-\tThis can be used to provide better, more specific data as part of the\n-\tprotocol.\n+ This signals that Git, or the helper, as appropriate, supports the\n+\tcapability in question. This can be used to provide better, more specific\n+\tdata as part of the protocol.\n +\n-The only capability currently supported is `authtype`, which indicates that the\n-`authtype` and `credential` values are understood. It is not obligatory to use\n-these values in such a case, but they should not be provided without this\n-capability.\n-+\n-Callers of `git credential` and credential helpers should emit the\n-capabilities they support unconditionally, and Git will gracefully\n-handle passing them on.\n+There are two currently supported capabilities. The first is `authtype`, which\n+indicates that the `authtype` and `credential` values are understood. The\n+second is `state`, which indicates that the `state[]` value is understood.\n+\n+It is not obligatory to use the additional features just because the capability\n+is supported, but they should not be provided without this capability.\n \n Unrecognised attributes and capabilities are silently discarded.\n \ndiff --git a/credential.c b/credential.c\nindex f2a26b8672..0cd7dd2a00 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -30,6 +30,7 @@ void credential_clear(struct credential *c)\n \tfree(c->authtype);\n \tstring_list_clear(&c->helpers, 0);\n \tstrvec_clear(&c->wwwauth_headers);\n+\tstrvec_clear(&c->state_headers);\n \n \tcredential_init(c);\n }\n@@ -286,8 +287,13 @@ int credential_read(struct credential *c, FILE *fp, int op_type)\n \t\t\tc->path = xstrdup(value);\n \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n \t\t\tstrvec_push(&c->wwwauth_headers, value);\n-\t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n-\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n+\t\t} else if (!strcmp(key, \"state[]\")) {\n+\t\t\tstrvec_push(&c->state_headers, value);\n+\t\t} else if (!strcmp(key, \"capability[]\")) {\n+\t\t\tif (!strcmp(value, \"authtype\"))\n+\t\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n+\t\t\telse if (!strcmp(value, \"state\"))\n+\t\t\t\tcredential_set_capability(&c->capa_state, op_type);\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n@@ -329,8 +335,12 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,\n \n void credential_write(const struct credential *c, FILE *fp, int op_type)\n {\n-\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n+\tif (credential_has_capability(&c->capa_authtype, op_type))\n \t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n+\tif (credential_has_capability(&c->capa_state, op_type))\n+\t\tcredential_write_item(fp, \"capability[]\", \"state\", 0);\n+\n+\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n \t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n \t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n \t}\n@@ -347,6 +357,10 @@ void credential_write(const struct credential *c, FILE *fp, int op_type)\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n+\tif (credential_has_capability(&c->capa_state, op_type)) {\n+\t\tfor (size_t i = 0; i < c->state_headers.nr; i++)\n+\t\t\tcredential_write_item(fp, \"state[]\", c->state_headers.v[i], 0);\n+\t}\n }\n \n static int run_credential_helper(struct credential *c,\ndiff --git a/credential.h b/credential.h\nindex 2051d04c5a..e2021455fe 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -142,6 +142,11 @@ struct credential {\n \t */\n \tstruct strvec wwwauth_headers;\n \n+\t/**\n+\t * A `strvec` of state headers from credential helpers.\n+\t */\n+\tstruct strvec state_headers;\n+\n \t/**\n \t * Internal use only. Keeps track of if we previously matched against a\n \t * WWW-Authenticate header line in order to re-fold future continuation\n@@ -156,6 +161,7 @@ struct credential {\n \t\t username_from_proto:1;\n \n \tstruct credential_capability capa_authtype;\n+\tstruct credential_capability capa_state;\n \n \tchar *username;\n \tchar *password;\n@@ -177,6 +183,7 @@ struct credential {\n \t.helpers = STRING_LIST_INIT_DUP, \\\n \t.password_expiry_utc = TIME_MAX, \\\n \t.wwwauth_headers = STRVEC_INIT, \\\n+\t.state_headers = STRVEC_INIT, \\\n }\n \n /* Initialize a credential structure, setting all fields to empty. */\ndiff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\nindex 8477108b28..aa56e0dc84 100755\n--- a/t/t0300-credentials.sh\n+++ b/t/t0300-credentials.sh\n@@ -46,9 +46,12 @@ test_expect_success 'setup helper scripts' '\n \tcredential=$1; shift\n \t. ./dump\n \techo capability[]=authtype\n+\techo capability[]=state\n \ttest -z \"${capability##*authtype*}\" || return\n \ttest -z \"$authtype\" || echo authtype=$authtype\n \ttest -z \"$credential\" || echo credential=$credential\n+\ttest -z \"${capability##*state*}\" || return\n+\techo state[]=verbatim-cred:foo\n \tEOF\n \n \twrite_script git-credential-verbatim-with-expiry <<-\\EOF &&\n@@ -99,6 +102,29 @@ test_expect_success 'credential_fill invokes helper with credential' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill invokes helper with credential and state' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Bearer\n+\tcredential=token\n+\tprotocol=http\n+\thost=example.com\n+\tstate[]=verbatim-cred:foo\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: capability[]=authtype\n+\tverbatim-cred: capability[]=state\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n \n test_expect_success 'credential_fill invokes multiple helpers' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n@@ -122,6 +148,7 @@ test_expect_success 'credential_fill invokes multiple helpers' '\n test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=example.com\n \t--\n@@ -132,10 +159,12 @@ test_expect_success 'credential_fill response does not get capabilities when hel\n \t--\n \tuseless: get\n \tuseless: capability[]=authtype\n+\tuseless: capability[]=state\n \tuseless: protocol=http\n \tuseless: host=example.com\n \tverbatim: get\n \tverbatim: capability[]=authtype\n+\tverbatim: capability[]=state\n \tverbatim: protocol=http\n \tverbatim: host=example.com\n \tEOF\n"},{"id":"491328","messageId":"20240324011301.1553072-11-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 10/13] docs: set a limit on credential line length","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:58Z","receivedAt":"2024-03-24T01:13:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We recently introduced a way for credential helpers to add arbitrary\nstate as part of the protocol. Set some limits on line length to avoid\nhelpers passing extremely large amounts of data. While Git doesn't have\na fixed parsing length, there are other tools which support this\nprotocol and it's kind to allow them to use a reasonable fixed-size\nbuffer for parsing. In addition, we would like to be moderate in our\nmemory usage and imposing reasonable limits is helpful for that purpose.\n\nIn the event a credential helper is incapable of storing its serialized\nstate in 64 KiB, it can feel free to serialize it on disk and store a\nreference instead.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 4 +++-\n 1 file changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex ef30c89c00..6b7e017066 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -111,7 +111,9 @@ attribute per line. Each attribute is specified by a key-value pair,\n separated by an `=` (equals) sign, followed by a newline.\n \n The key may contain any bytes except `=`, newline, or NUL. The value may\n-contain any bytes except newline or NUL.\n+contain any bytes except newline or NUL. A line, including the trailing\n+newline, may not exceed 65535 bytes in order to allow implementations to\n+parse efficiently.\n \n Attributes with keys that end with C-style array brackets `[]` can have\n multiple values. Each instance of a multi-valued attribute forms an\n"},{"id":"491330","messageId":"20240324011301.1553072-6-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 05/13] credential: gate new fields on capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:53Z","receivedAt":"2024-03-24T01:13:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We support the new credential and authtype fields, but we lack a way to\nindicate to a credential helper that we'd like them to be used. Without\nsome sort of indication, the credential helper doesn't know if it should\ntry to provide us a username and password, or a pre-encoded credential.\nFor example, the helper might prefer a more restricted Bearer token if\npre-encoded credentials are possible, but might have to fall back to\nmore general username and password if not.\n\nLet's provide a simple way to indicate whether Git (or, for that matter,\nthe helper) is capable of understanding the authtype and credential\nfields. We send this capability when we generate a request, and the\nother side may reply to indicate to us that it does, too.\n\nFor now, don't enable sending capabilities for the HTTP code. In a\nfuture commit, we'll introduce appropriate handling for that code,\nwhich requires more in-depth work.\n\nThe logic for determining whether a capability is supported may seem\ncomplex, but it is not. At each stage, we emit the capability to the\nfollowing stage if all preceding stages have declared it. Thus, if the\ncaller to git credential fill didn't declare it, then we won't send it\nto the helper, and if fill's caller did send but the helper doesn't\nunderstand it, then we won't send it on in the response. If we're an\ninternal user, then we know about all capabilities and will request\nthem.\n\nSigned-off-by: brian m. carlson \n---\n builtin/credential-cache--daemon.c | 2 +-\n builtin/credential-store.c | 2 +-\n builtin/credential.c | 6 +-\n credential.c | 58 ++++++++++++++--\n credential.h | 30 +++++++-\n http.c | 10 +--\n imap-send.c | 2 +-\n remote-curl.c | 4 +-\n t/t0300-credentials.sh | 107 ++++++++++++++++++++++++++++-\n 9 files changed, 197 insertions(+), 24 deletions(-)\n\ndiff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c\nindex 3a6a750a8e..ccbcf99ac1 100644\n--- a/builtin/credential-cache--daemon.c\n+++ b/builtin/credential-cache--daemon.c\n@@ -115,7 +115,7 @@ static int read_request(FILE *fh, struct credential *c,\n \t\treturn error(\"client sent bogus timeout line: %s\", item.buf);\n \t*timeout = atoi(p);\n \n-\tif (credential_read(c, fh) < 0)\n+\tif (credential_read(c, fh, CREDENTIAL_OP_HELPER) < 0)\n \t\treturn -1;\n \treturn 0;\n }\ndiff --git a/builtin/credential-store.c b/builtin/credential-store.c\nindex 4a492411bb..494c809332 100644\n--- a/builtin/credential-store.c\n+++ b/builtin/credential-store.c\n@@ -205,7 +205,7 @@ int cmd_credential_store(int argc, const char **argv, const char *prefix)\n \tif (!fns.nr)\n \t\tdie(\"unable to set up default path; use --file\");\n \n-\tif (credential_read(&c, stdin) < 0)\n+\tif (credential_read(&c, stdin, CREDENTIAL_OP_HELPER) < 0)\n \t\tdie(\"unable to read credential\");\n \n \tif (!strcmp(op, \"get\"))\ndiff --git a/builtin/credential.c b/builtin/credential.c\nindex 7010752987..5123dabcf1 100644\n--- a/builtin/credential.c\n+++ b/builtin/credential.c\n@@ -17,12 +17,12 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n \t\tusage(usage_msg);\n \top = argv[1];\n \n-\tif (credential_read(&c, stdin) < 0)\n+\tif (credential_read(&c, stdin, CREDENTIAL_OP_INITIAL) < 0)\n \t\tdie(\"unable to read credential from stdin\");\n \n \tif (!strcmp(op, \"fill\")) {\n-\t\tcredential_fill(&c);\n-\t\tcredential_write(&c, stdout);\n+\t\tcredential_fill(&c, 0);\n+\t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n \t} else if (!strcmp(op, \"approve\")) {\n \t\tcredential_approve(&c);\n \t} else if (!strcmp(op, \"reject\")) {\ndiff --git a/credential.c b/credential.c\nindex c521822e5a..f2a26b8672 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -34,6 +34,11 @@ void credential_clear(struct credential *c)\n \tcredential_init(c);\n }\n \n+static void credential_set_all_capabilities(struct credential *c)\n+{\n+\tc->capa_authtype.request_initial = 1;\n+}\n+\n int credential_match(const struct credential *want,\n \t\t const struct credential *have, int match_password)\n {\n@@ -210,7 +215,39 @@ static void credential_getpass(struct credential *c)\n \t\t\t\t\t\t PROMPT_ASKPASS);\n }\n \n-int credential_read(struct credential *c, FILE *fp)\n+static void credential_set_capability(struct credential_capability *capa, int op_type)\n+{\n+\tswitch (op_type) {\n+\tcase CREDENTIAL_OP_INITIAL:\n+\t\tcapa->request_initial = 1;\n+\t\tbreak;\n+\tcase CREDENTIAL_OP_HELPER:\n+\t\tcapa->request_helper = 1;\n+\t\tbreak;\n+\tcase CREDENTIAL_OP_RESPONSE:\n+\t\tcapa->response = 1;\n+\t\tbreak;\n+\t}\n+}\n+\n+static int credential_has_capability(const struct credential_capability *capa, int op_type)\n+{\n+\t/*\n+\t * We're checking here if each previous step indicated that we had the\n+\t * capability. If it did, then we want to pass it along; conversely, if\n+\t * it did not, we don't want to report that to our caller.\n+\t */\n+\tswitch (op_type) {\n+\tcase CREDENTIAL_OP_HELPER:\n+\t\treturn capa->request_initial;\n+\tcase CREDENTIAL_OP_RESPONSE:\n+\t\treturn capa->request_initial && capa->request_helper;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+}\n+\n+int credential_read(struct credential *c, FILE *fp, int op_type)\n {\n \tstruct strbuf line = STRBUF_INIT;\n \n@@ -249,6 +286,8 @@ int credential_read(struct credential *c, FILE *fp)\n \t\t\tc->path = xstrdup(value);\n \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n \t\t\tstrvec_push(&c->wwwauth_headers, value);\n+\t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n+\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n@@ -288,14 +327,18 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,\n \tfprintf(fp, \"%s=%s\\n\", key, value);\n }\n \n-void credential_write(const struct credential *c, FILE *fp)\n+void credential_write(const struct credential *c, FILE *fp, int op_type)\n {\n+\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n+\t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n+\t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n+\t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n+\t}\n \tcredential_write_item(fp, \"protocol\", c->protocol, 1);\n \tcredential_write_item(fp, \"host\", c->host, 1);\n \tcredential_write_item(fp, \"path\", c->path, 0);\n \tcredential_write_item(fp, \"username\", c->username, 0);\n \tcredential_write_item(fp, \"password\", c->password, 0);\n-\tcredential_write_item(fp, \"credential\", c->credential, 0);\n \tcredential_write_item(fp, \"oauth_refresh_token\", c->oauth_refresh_token, 0);\n \tif (c->password_expiry_utc != TIME_MAX) {\n \t\tchar *s = xstrfmt(\"%\"PRItime, c->password_expiry_utc);\n@@ -304,7 +347,6 @@ void credential_write(const struct credential *c, FILE *fp)\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n-\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n }\n \n static int run_credential_helper(struct credential *c,\n@@ -327,14 +369,14 @@ static int run_credential_helper(struct credential *c,\n \n \tfp = xfdopen(helper.in, \"w\");\n \tsigchain_push(SIGPIPE, SIG_IGN);\n-\tcredential_write(c, fp);\n+\tcredential_write(c, fp, want_output ? CREDENTIAL_OP_HELPER : CREDENTIAL_OP_RESPONSE);\n \tfclose(fp);\n \tsigchain_pop(SIGPIPE);\n \n \tif (want_output) {\n \t\tint r;\n \t\tfp = xfdopen(helper.out, \"r\");\n-\t\tr = credential_read(c, fp);\n+\t\tr = credential_read(c, fp, CREDENTIAL_OP_HELPER);\n \t\tfclose(fp);\n \t\tif (r < 0) {\n \t\t\tfinish_command(&helper);\n@@ -367,7 +409,7 @@ static int credential_do(struct credential *c, const char *helper,\n \treturn r;\n }\n \n-void credential_fill(struct credential *c)\n+void credential_fill(struct credential *c, int all_capabilities)\n {\n \tint i;\n \n@@ -375,6 +417,8 @@ void credential_fill(struct credential *c)\n \t\treturn;\n \n \tcredential_apply_config(c);\n+\tif (all_capabilities)\n+\t\tcredential_set_all_capabilities(c);\n \n \tfor (i = 0; i < c->helpers.nr; i++) {\n \t\tcredential_do(c, c->helpers.items[i].string, \"get\");\ndiff --git a/credential.h b/credential.h\nindex 9db892cf4d..2051d04c5a 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -93,6 +93,25 @@\n * -----------------------------------------------------------------------\n */\n \n+/*\n+ * These values define the kind of operation we're performing and the\n+ * capabilities at each stage. The first is either an external request (via git\n+ * credential fill) or an internal request (e.g., via the HTTP) code. The\n+ * second is the call to the credential helper, and the third is the response\n+ * we're providing.\n+ *\n+ * At each stage, we will emit the capability only if the previous stage\n+ * supported it.\n+ */\n+#define CREDENTIAL_OP_INITIAL 1\n+#define CREDENTIAL_OP_HELPER 2\n+#define CREDENTIAL_OP_RESPONSE 3\n+\n+struct credential_capability {\n+\tunsigned request_initial:1,\n+\t\t request_helper:1,\n+\t\t response:1;\n+};\n \n /**\n * This struct represents a single username/password combination\n@@ -136,6 +155,8 @@ struct credential {\n \t\t use_http_path:1,\n \t\t username_from_proto:1;\n \n+\tstruct credential_capability capa_authtype;\n+\n \tchar *username;\n \tchar *password;\n \tchar *credential;\n@@ -174,8 +195,11 @@ void credential_clear(struct credential *);\n * returns, the username and password fields of the credential are\n * guaranteed to be non-NULL. If an error occurs, the function will\n * die().\n+ *\n+ * If all_capabilities is set, this is an internal user that is prepared\n+ * to deal with all known capabilities, and we should advertise that fact.\n */\n-void credential_fill(struct credential *);\n+void credential_fill(struct credential *, int all_capabilities);\n \n /**\n * Inform the credential subsystem that the provided credentials\n@@ -198,8 +222,8 @@ void credential_approve(struct credential *);\n */\n void credential_reject(struct credential *);\n \n-int credential_read(struct credential *, FILE *);\n-void credential_write(const struct credential *, FILE *);\n+int credential_read(struct credential *, FILE *, int);\n+void credential_write(const struct credential *, FILE *, int);\n \n /*\n * Parse a url into a credential struct, replacing any existing contents.\ndiff --git a/http.c b/http.c\nindex 1c2200da77..4f5df6fb14 100644\n--- a/http.c\n+++ b/http.c\n@@ -569,7 +569,7 @@ static void init_curl_http_auth(CURL *result)\n \t\treturn;\n \t}\n \n-\tcredential_fill(&http_auth);\n+\tcredential_fill(&http_auth, 0);\n \n \tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n \tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n@@ -596,7 +596,7 @@ static void init_curl_proxy_auth(CURL *result)\n {\n \tif (proxy_auth.username) {\n \t\tif (!proxy_auth.password)\n-\t\t\tcredential_fill(&proxy_auth);\n+\t\t\tcredential_fill(&proxy_auth, 0);\n \t\tset_proxyauth_name_password(result);\n \t}\n \n@@ -630,7 +630,7 @@ static int has_cert_password(void)\n \t\tcert_auth.host = xstrdup(\"\");\n \t\tcert_auth.username = xstrdup(\"\");\n \t\tcert_auth.path = xstrdup(ssl_cert);\n-\t\tcredential_fill(&cert_auth);\n+\t\tcredential_fill(&cert_auth, 0);\n \t}\n \treturn 1;\n }\n@@ -645,7 +645,7 @@ static int has_proxy_cert_password(void)\n \t\tproxy_cert_auth.host = xstrdup(\"\");\n \t\tproxy_cert_auth.username = xstrdup(\"\");\n \t\tproxy_cert_auth.path = xstrdup(http_proxy_ssl_cert);\n-\t\tcredential_fill(&proxy_cert_auth);\n+\t\tcredential_fill(&proxy_cert_auth, 0);\n \t}\n \treturn 1;\n }\n@@ -2191,7 +2191,7 @@ static int http_request_reauth(const char *url,\n \t\tBUG(\"Unknown http_request target\");\n \t}\n \n-\tcredential_fill(&http_auth);\n+\tcredential_fill(&http_auth, 0);\n \n \treturn http_request(url, result, target, options);\n }\ndiff --git a/imap-send.c b/imap-send.c\nindex f2e1947e63..8c89e866b6 100644\n--- a/imap-send.c\n+++ b/imap-send.c\n@@ -944,7 +944,7 @@ static void server_fill_credential(struct imap_server_conf *srvc, struct credent\n \tcred->username = xstrdup_or_null(srvc->user);\n \tcred->password = xstrdup_or_null(srvc->pass);\n \n-\tcredential_fill(cred);\n+\tcredential_fill(cred, 1);\n \n \tif (!srvc->user)\n \t\tsrvc->user = xstrdup(cred->username);\ndiff --git a/remote-curl.c b/remote-curl.c\nindex e37eaa17b7..f96bda2431 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -926,7 +926,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \t\tdo {\n \t\t\terr = probe_rpc(rpc, &results);\n \t\t\tif (err == HTTP_REAUTH)\n-\t\t\t\tcredential_fill(&http_auth);\n+\t\t\t\tcredential_fill(&http_auth, 0);\n \t\t} while (err == HTTP_REAUTH);\n \t\tif (err != HTTP_OK)\n \t\t\treturn -1;\n@@ -1044,7 +1044,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \trpc->any_written = 0;\n \terr = run_slot(slot, NULL);\n \tif (err == HTTP_REAUTH && !large_request) {\n-\t\tcredential_fill(&http_auth);\n+\t\tcredential_fill(&http_auth, 0);\n \t\tcurl_slist_free_all(headers);\n \t\tgoto retry;\n \t}\ndiff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\nindex 400f6bdbca..8477108b28 100755\n--- a/t/t0300-credentials.sh\n+++ b/t/t0300-credentials.sh\n@@ -12,7 +12,13 @@ test_expect_success 'setup helper scripts' '\n \tIFS==\n \twhile read key value; do\n \t\techo >&2 \"$whoami: $key=$value\"\n-\t\teval \"$key=$value\"\n+\t\tif test -z \"${key%%*\\[\\]}\"\n+\t\tthen\n+\t\t\tkey=${key%%\\[\\]}\n+\t\t\teval \"$key=\\\"\\$$key $value\\\"\"\n+\t\telse\n+\t\t\teval \"$key=$value\"\n+\t\tfi\n \tdone\n \tIFS=$OIFS\n \tEOF\n@@ -35,6 +41,16 @@ test_expect_success 'setup helper scripts' '\n \ttest -z \"$pass\" || echo password=$pass\n \tEOF\n \n+\twrite_script git-credential-verbatim-cred <<-\\EOF &&\n+\tauthtype=$1; shift\n+\tcredential=$1; shift\n+\t. ./dump\n+\techo capability[]=authtype\n+\ttest -z \"${capability##*authtype*}\" || return\n+\ttest -z \"$authtype\" || echo authtype=$authtype\n+\ttest -z \"$credential\" || echo credential=$credential\n+\tEOF\n+\n \twrite_script git-credential-verbatim-with-expiry <<-\\EOF &&\n \tuser=$1; shift\n \tpass=$1; shift\n@@ -64,6 +80,26 @@ test_expect_success 'credential_fill invokes helper' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill invokes helper with credential' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=token\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: capability[]=authtype\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n+\n test_expect_success 'credential_fill invokes multiple helpers' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n \tprotocol=http\n@@ -83,6 +119,42 @@ test_expect_success 'credential_fill invokes multiple helpers' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '\n+\tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tprotocol=http\n+\thost=example.com\n+\tusername=foo\n+\tpassword=bar\n+\t--\n+\tuseless: get\n+\tuseless: capability[]=authtype\n+\tuseless: protocol=http\n+\tuseless: host=example.com\n+\tverbatim: get\n+\tverbatim: capability[]=authtype\n+\tverbatim: protocol=http\n+\tverbatim: host=example.com\n+\tEOF\n+'\n+\n+test_expect_success 'credential_fill response does not get capabilities when caller is incapable' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n test_expect_success 'credential_fill stops when we get a full response' '\n \tcheck fill \"verbatim one two\" \"verbatim three four\" <<-\\EOF\n \tprotocol=http\n@@ -99,6 +171,25 @@ test_expect_success 'credential_fill stops when we get a full response' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill thinks a credential is a full response' '\n+\tcheck fill \"verbatim-cred Bearer token\" \"verbatim three four\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=token\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: capability[]=authtype\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n test_expect_success 'credential_fill continues through partial response' '\n \tcheck fill \"verbatim one \\\"\\\"\" \"verbatim two three\" <<-\\EOF\n \tprotocol=http\n@@ -175,6 +266,20 @@ test_expect_success 'credential_fill passes along metadata' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill produces no credential without capability' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n test_expect_success 'credential_approve calls all helpers' '\n \tcheck approve useless \"verbatim one two\" <<-\\EOF\n \tprotocol=http\n"},{"id":"491331","messageId":"20240324011301.1553072-8-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 07/13] http: add support for authtype and credential","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:55Z","receivedAt":"2024-03-24T01:13:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we have the credential helper code set up to handle arbitrary\nauthentications schemes, let's add support for this in the HTTP code,\nwhere we really want to use it. If we're using this new functionality,\ndon't set a username and password, and instead set a header wherever\nwe'd normally do so, including for proxy authentication.\n\nSince we can now handle this case, ask the credential helper to enable\nthe appropriate capabilities.\n\nFinally, if we're using the authtype value, set \"Expect: 100-continue\".\nAny type of authentication that requires multiple rounds (such as NTLM\nor Kerberos) requires a 100 Continue (if we're larger than\nhttp.postBuffer) because otherwise we send the pack data before we're\nauthenticated, the push gets a 401 response, and we can't rewind the\nstream. We don't know for certain what other custom schemes might\nrequire this, the HTTP/1.1 standard has required handling this since\n1999, the broken HTTP server for which we disabled this (Google's) is\nnow fixed and has been for some time, and libcurl has a 1-second\nfallback in case the HTTP server is still broken. In addition, it is\nnot unreasonable to require compliance with a 25-year old standard to\nuse new Git features. For all of these reasons, do so here.\n\nSigned-off-by: brian m. carlson \n---\n http.c | 48 ++++++++++---\n http.h | 3 +\n remote-curl.c | 4 +-\n t/t5563-simple-http-auth.sh | 133 ++++++++++++++++++++++++++++++++++++\n 4 files changed, 176 insertions(+), 12 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 4f5df6fb14..f98c520924 100644\n--- a/http.c\n+++ b/http.c\n@@ -561,18 +561,34 @@ static int curl_empty_auth_enabled(void)\n \treturn 0;\n }\n \n+struct curl_slist *http_append_auth_header(const struct credential *c,\n+\t\t\t\t\t struct curl_slist *headers)\n+{\n+\tif (c->authtype && c->credential) {\n+\t\tstruct strbuf auth = STRBUF_INIT;\n+\t\tstrbuf_addf(&auth, \"Authorization: %s %s\",\n+\t\t\t c->authtype, c->credential);\n+\t\theaders = curl_slist_append(headers, auth.buf);\n+\t\tstrbuf_release(&auth);\n+\t}\n+\treturn headers;\n+}\n+\n static void init_curl_http_auth(CURL *result)\n {\n-\tif (!http_auth.username || !*http_auth.username) {\n+\tif ((!http_auth.username || !*http_auth.username) &&\n+\t (!http_auth.credential || !*http_auth.credential)) {\n \t\tif (curl_empty_auth_enabled())\n \t\t\tcurl_easy_setopt(result, CURLOPT_USERPWD, \":\");\n \t\treturn;\n \t}\n \n-\tcredential_fill(&http_auth, 0);\n+\tcredential_fill(&http_auth, 1);\n \n-\tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n-\tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n+\tif (http_auth.password) {\n+\t\tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n+\t\tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n+\t}\n }\n \n /* *var must be free-able */\n@@ -586,17 +602,22 @@ static void var_override(const char **var, char *value)\n \n static void set_proxyauth_name_password(CURL *result)\n {\n+\tif (proxy_auth.password) {\n \t\tcurl_easy_setopt(result, CURLOPT_PROXYUSERNAME,\n \t\t\tproxy_auth.username);\n \t\tcurl_easy_setopt(result, CURLOPT_PROXYPASSWORD,\n \t\t\tproxy_auth.password);\n+\t} else if (proxy_auth.authtype && proxy_auth.credential) {\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXYHEADER,\n+\t\t\t\t http_append_auth_header(&proxy_auth, NULL));\n+\t}\n }\n \n static void init_curl_proxy_auth(CURL *result)\n {\n \tif (proxy_auth.username) {\n-\t\tif (!proxy_auth.password)\n-\t\t\tcredential_fill(&proxy_auth, 0);\n+\t\tif (!proxy_auth.password && !proxy_auth.credential)\n+\t\t\tcredential_fill(&proxy_auth, 1);\n \t\tset_proxyauth_name_password(result);\n \t}\n \n@@ -1469,7 +1490,7 @@ struct active_request_slot *get_active_slot(void)\n \n \tcurl_easy_setopt(slot->curl, CURLOPT_IPRESOLVE, git_curl_ipresolve);\n \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, http_auth_methods);\n-\tif (http_auth.password || curl_empty_auth_enabled())\n+\tif (http_auth.password || http_auth.credential || curl_empty_auth_enabled())\n \t\tinit_curl_http_auth(slot->curl);\n \n \treturn slot;\n@@ -1758,7 +1779,8 @@ static int handle_curl_result(struct slot_results *results)\n \t} else if (missing_target(results))\n \t\treturn HTTP_MISSING_TARGET;\n \telse if (results->http_code == 401) {\n-\t\tif (http_auth.username && http_auth.password) {\n+\t\tif ((http_auth.username && http_auth.password) ||\\\n+\t\t (http_auth.authtype && http_auth.credential)) {\n \t\t\tcredential_reject(&http_auth);\n \t\t\treturn HTTP_NOAUTH;\n \t\t} else {\n@@ -2066,11 +2088,15 @@ static int http_request(const char *url,\n \t/* Add additional headers here */\n \tif (options && options->extra_headers) {\n \t\tconst struct string_list_item *item;\n-\t\tfor_each_string_list_item(item, options->extra_headers) {\n-\t\t\theaders = curl_slist_append(headers, item->string);\n+\t\tif (options && options->extra_headers) {\n+\t\t\tfor_each_string_list_item(item, options->extra_headers) {\n+\t\t\t\theaders = curl_slist_append(headers, item->string);\n+\t\t\t}\n \t\t}\n \t}\n \n+\theaders = http_append_auth_header(&http_auth, headers);\n+\n \tcurl_easy_setopt(slot->curl, CURLOPT_URL, url);\n \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);\n \tcurl_easy_setopt(slot->curl, CURLOPT_ENCODING, \"\");\n@@ -2191,7 +2217,7 @@ static int http_request_reauth(const char *url,\n \t\tBUG(\"Unknown http_request target\");\n \t}\n \n-\tcredential_fill(&http_auth, 0);\n+\tcredential_fill(&http_auth, 1);\n \n \treturn http_request(url, result, target, options);\n }\ndiff --git a/http.h b/http.h\nindex c5f8cc4620..a516ca4a9a 100644\n--- a/http.h\n+++ b/http.h\n@@ -175,6 +175,9 @@ int http_get_file(const char *url, const char *filename,\n \n int http_fetch_ref(const char *base, struct ref *ref);\n \n+struct curl_slist *http_append_auth_header(const struct credential *c,\n+\t\t\t\t\t struct curl_slist *headers);\n+\n /* Helpers for fetching packs */\n int http_get_info_packs(const char *base_url,\n \t\t\tstruct packed_git **packs_head);\ndiff --git a/remote-curl.c b/remote-curl.c\nindex f96bda2431..1c5416812a 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -931,7 +931,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \t\tif (err != HTTP_OK)\n \t\t\treturn -1;\n \n-\t\tif (results.auth_avail & CURLAUTH_GSSNEGOTIATE)\n+\t\tif (results.auth_avail & CURLAUTH_GSSNEGOTIATE || http_auth.authtype)\n \t\t\tneeds_100_continue = 1;\n \t}\n \n@@ -942,6 +942,8 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \theaders = curl_slist_append(headers, needs_100_continue ?\n \t\t\"Expect: 100-continue\" : \"Expect:\");\n \n+\theaders = http_append_auth_header(&http_auth, headers);\n+\n \t/* Add Accept-Language header */\n \tif (rpc->hdr_accept_language)\n \t\theaders = curl_slist_append(headers, rpc->hdr_accept_language);\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex ab8a721ccc..b3ed0d9fc2 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -74,6 +74,7 @@ test_expect_success 'access using basic auth' '\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -87,6 +88,43 @@ test_expect_success 'access using basic auth' '\n \tEOF\n '\n \n+test_expect_success 'access using basic auth via authtype' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Basic\n+\tcredential=YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tEOF\n+\n+\t# Basic base64(alice:secret-passwd)\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tEOF\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\tGIT_CURL_VERBOSE=1 git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+\n+\texpect_credential_query store <<-EOF\n+\tcapability[]=authtype\n+\tauthtype=Basic\n+\tcredential=YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\tEOF\n+'\n+\n test_expect_success 'access using basic auth invalid credentials' '\n \ttest_when_finished \"per_test_cleanup\" &&\n \n@@ -108,6 +146,7 @@ test_expect_success 'access using basic auth invalid credentials' '\n \ttest_must_fail git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -145,6 +184,7 @@ test_expect_success 'access using basic auth with extra challenges' '\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -183,6 +223,7 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=foobar param1=\"value1\" param2=\"value2\"\n@@ -226,6 +267,7 @@ test_expect_success 'access using basic auth with wwwauth header continuations'\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -271,6 +313,7 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -312,6 +355,7 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -326,4 +370,93 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \tEOF\n '\n \n+test_expect_success 'access using bearer auth' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\t# Basic base64(a-git-token)\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tBearer YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+\n+\texpect_credential_query store <<-EOF\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=YS1naXQtdG9rZW4=\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\tEOF\n+'\n+\n+test_expect_success 'access using bearer auth with invalid credentials' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=incorrect-token\n+\tEOF\n+\n+\t# Basic base64(a-git-token)\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tBearer YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\ttest_must_fail git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+\n+\texpect_credential_query erase <<-EOF\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=incorrect-token\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+'\n+\n test_done\n"},{"id":"491332","messageId":"20240324011301.1553072-7-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 06/13] docs: indicate new credential protocol fields","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:54Z","receivedAt":"2024-03-24T01:13:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we have new fields (authtype and credential), let's document\nthem for users and credential helper implementers.\n\nIndicate specifically what common values of authtype are and what values\nare allowed. Note that, while common, digest and NTLM authentication\nare insecure because they require unsalted, uniterated password hashes\nto be stored.\n\nTell users that they can continue to use a username and password even if\nthe new capability is supported.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 34 +++++++++++++++++++++++++++++++-\n 1 file changed, 33 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex 918a0aa42b..f3ed3a82fa 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -178,6 +178,24 @@ empty string.\n Components which are missing from the URL (e.g., there is no\n username in the example above) will be left unset.\n \n+`authtype`::\n+\tThis indicates that the authentication scheme in question should be used.\n+\tCommon values for HTTP and HTTPS include `basic`, `digest`, and `ntlm`,\n+\talthough the latter two are insecure and should not be used. If `credential`\n+\tis used, this may be set to an arbitrary string suitable for the protocol in\n+\tquestion (usually HTTP).\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n+`credential`::\n+\tThe pre-encoded credential, suitable for the protocol in question (usually\n+\tHTTP). If this key is sent, `authtype` is mandatory, and `username` and\n+\t`password` are not used.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\n@@ -189,7 +207,21 @@ attribute 'wwwauth[]', where the order of the attributes is the same as\n they appear in the HTTP response. This attribute is 'one-way' from Git\n to pass additional information to credential helpers.\n \n-Unrecognised attributes are silently discarded.\n+`capability[]`::\n+\tThis signals that the caller supports the capability in question.\n+\tThis can be used to provide better, more specific data as part of the\n+\tprotocol.\n++\n+The only capability currently supported is `authtype`, which indicates that the\n+`authtype` and `credential` values are understood. It is not obligatory to use\n+these values in such a case, but they should not be provided without this\n+capability.\n++\n+Callers of `git credential` and credential helpers should emit the\n+capabilities they support unconditionally, and Git will gracefully\n+handle passing them on.\n+\n+Unrecognised attributes and capabilities are silently discarded.\n \n GIT\n ---\n"},{"id":"491324","messageId":"20240324011301.1553072-13-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 12/13] strvec: implement swapping two strvecs","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:13:00Z","receivedAt":"2024-03-24T01:13:13Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"In a future commit, we'll want the ability to efficiently swap the\ncontents of two strvec instances without needing to copy any data.\nSince a strvec is simply a pointer and two sizes, swapping them is as\nsimply as copying the two pointers and sizes, so let's do that.\n\nWe use a temporary here simply because C doesn't provide a standard\nswapping function, unlike C++ and Rust, but a good optimizing compiler\nwill recognize this syntax and handle it appropriately using an\noptimization pass.\n\nSigned-off-by: brian m. carlson \n---\n strvec.c | 7 +++++++\n strvec.h | 5 +++++\n 2 files changed, 12 insertions(+)\n\ndiff --git a/strvec.c b/strvec.c\nindex 178f4f3748..93006f1e63 100644\n--- a/strvec.c\n+++ b/strvec.c\n@@ -106,3 +106,10 @@ const char **strvec_detach(struct strvec *array)\n \t\treturn ret;\n \t}\n }\n+\n+void strvec_swap(struct strvec *a, struct strvec *b)\n+{\n+\tstruct strvec t = *a;\n+\t*a = *b;\n+\t*b = t;\n+}\ndiff --git a/strvec.h b/strvec.h\nindex 4715d3e51f..5b762532bb 100644\n--- a/strvec.h\n+++ b/strvec.h\n@@ -88,4 +88,9 @@ void strvec_clear(struct strvec *);\n */\n const char **strvec_detach(struct strvec *);\n \n+/**\n+ * Swap the contents of two `strvec` structs without allocating.\n+ */\n+void strvec_swap(struct strvec *, struct strvec *);\n+\n #endif /* STRVEC_H */\n"},{"id":"491327","messageId":"20240324011301.1553072-12-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 11/13] t5563: refactor for multi-stage authentication","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:59Z","receivedAt":"2024-03-24T01:13:13Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Some HTTP authentication schemes, such as NTLM- and Kerberos-based\noptions, require more than one round trip to authenticate. Currently,\nthese can only be supported in libcurl, since Git does not have support\nfor this in the credential helper protocol.\n\nHowever, in a future commit, we'll add support for this functionality\ninto the credential helper protocol and Git itself. Because we don't\nreally want to implement either NTLM or Kerberos, both of which are\ncomplex protocols, we'll want to test this using a fake credential\nauthentication scheme. In order to do so, update t5563 and its backend\nto allow us to accept multiple sets of credentials and respond with\ndifferent behavior in each case.\n\nSince we can now provide any number of possible status codes, provide a\nnon-specific reason phrase so we don't have to generate a more specific\none based on the response. The reason phrase is mandatory according to\nthe status-line production in RFC 7230, but clients SHOULD ignore it,\nand curl does (except to print it).\n\nEach entry in the authorization and challenge fields contains an ID,\nwhich indicates a corresponding credential and response. If the\nresponse is a 200 status, then we continue to execute git-http-backend.\nOtherwise, we print the corresponding status and response. If no ID is\nmatched, we use the default response with a status of 401.\n\nNote that there is an implicit order to the parameters. The ID is\nalways first and the creds or response value is always last, and\ntherefore may contain spaces, equals signs, or other arbitrary data.\n\nSigned-off-by: brian m. carlson \n---\n t/lib-httpd/nph-custom-auth.sh | 17 ++++--\n t/t5563-simple-http-auth.sh | 96 +++++++++++++++++++---------------\n 2 files changed, 66 insertions(+), 47 deletions(-)\n\ndiff --git a/t/lib-httpd/nph-custom-auth.sh b/t/lib-httpd/nph-custom-auth.sh\nindex f5345e775e..d408d2caad 100644\n--- a/t/lib-httpd/nph-custom-auth.sh\n+++ b/t/lib-httpd/nph-custom-auth.sh\n@@ -19,21 +19,30 @@ CHALLENGE_FILE=custom-auth.challenge\n #\n \n if test -n \"$HTTP_AUTHORIZATION\" && \\\n-\tgrep -Fqsx \"${HTTP_AUTHORIZATION}\" \"$VALID_CREDS_FILE\"\n+\tgrep -Fqs \"creds=${HTTP_AUTHORIZATION}\" \"$VALID_CREDS_FILE\"\n then\n+\tidno=$(grep -F \"creds=${HTTP_AUTHORIZATION}\" \"$VALID_CREDS_FILE\" | sed -e 's/^id=\\([a-z0-9-][a-z0-9-]*\\) .*$/\\1/')\n+\tstatus=$(sed -ne \"s/^id=$idno.*status=\\\\([0-9][0-9][0-9]\\\\).*\\$/\\\\1/p\" \"$CHALLENGE_FILE\" | head -n1)\n \t# Note that although git-http-backend returns a status line, it\n \t# does so using a CGI 'Status' header. Because this script is an\n \t# No Parsed Headers (NPH) script, we must return a real HTTP\n \t# status line.\n \t# This is only a test script, so we don't bother to check for\n \t# the actual status from git-http-backend and always return 200.\n-\techo 'HTTP/1.1 200 OK'\n-\texec \"$GIT_EXEC_PATH\"/git-http-backend\n+\techo \"HTTP/1.1 $status Nonspecific Reason Phrase\"\n+\tif test \"$status\" -eq 200\n+\tthen\n+\t\texec \"$GIT_EXEC_PATH\"/git-http-backend\n+\telse\n+\t\tsed -ne \"s/^id=$idno.*response=//p\" \"$CHALLENGE_FILE\"\n+\t\techo\n+\t\texit\n+\tfi\n fi\n \n echo 'HTTP/1.1 401 Authorization Required'\n if test -f \"$CHALLENGE_FILE\"\n then\n-\tcat \"$CHALLENGE_FILE\"\n+\tsed -ne 's/^id=default.*response=//p' \"$CHALLENGE_FILE\"\n fi\n echo\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex b098cd0fdf..515185ae00 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -63,11 +63,12 @@ test_expect_success 'access using basic auth' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -100,11 +101,12 @@ test_expect_success 'access using basic auth via authtype' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -137,11 +139,12 @@ test_expect_success 'access using basic auth invalid credentials' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -174,13 +177,14 @@ test_expect_success 'access using basic auth with extra challenges' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -214,13 +218,14 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\twww-authenticate: foobar param1=\"value1\" param2=\"value2\"\n-\tWWW-AUTHENTICATE: BEARER authorize_uri=\"id.example.com\" p=1 q=0\n-\tWwW-aUtHeNtIcAtE: baSiC realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=www-authenticate: foobar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-AUTHENTICATE: BEARER authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WwW-aUtHeNtIcAtE: baSiC realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -254,18 +259,19 @@ test_expect_success 'access using basic auth with wwwauth header continuations'\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \t# Note that leading and trailing whitespace is important to correctly\n \t# simulate a continuation/folded header.\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\"\n-\t param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\n-\t p=1\n-\t q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\"\n+\tid=default response= param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\n+\tid=default response= p=1\n+\tid=default response= q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -299,21 +305,22 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \t# Note that leading and trailing whitespace is important to correctly\n \t# simulate a continuation/folded header.\n-\tprintf \"WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >\"$CHALLENGE\" &&\n-\tprintf \" \\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" param2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"WWW-Authenticate: Bearer authorize_uri=\\\"id.example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" p=1\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" \\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" q=0\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"WWW-Authenticate: Basic realm=\\\"example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=1 status=200\\n\" >\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= \\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= param2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: Bearer authorize_uri=\\\"id.example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= p=1\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= \\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= q=0\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: Basic realm=\\\"example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n \n \ttest_config_global credential.helper test-helper &&\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n@@ -346,17 +353,18 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \t# Note that leading and trailing whitespace is important to correctly\n \t# simulate a continuation/folded header.\n-\tprintf \"WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >\"$CHALLENGE\" &&\n-\tprintf \" \\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"\\tparam2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"WWW-Authenticate: Basic realm=\\\"example.com\\\"\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=1 status=200\\n\" >\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= \\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=\\tparam2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: Basic realm=\\\"example.com\\\"\" >>\"$CHALLENGE\" &&\n \n \ttest_config_global credential.helper test-helper &&\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n@@ -389,15 +397,16 @@ test_expect_success 'access using bearer auth' '\n \n \t# Basic base64(a-git-token)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBearer YS1naXQtdG9rZW4=\n+\tid=1 creds=Bearer YS1naXQtdG9rZW4=\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -433,15 +442,16 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \n \t# Basic base64(a-git-token)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBearer YS1naXQtdG9rZW4=\n+\tid=1 creds=Bearer YS1naXQtdG9rZW4=\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n"},{"id":"491329","messageId":"20240324011301.1553072-14-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 13/13] credential: add support for multistage credential rounds","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:13:01Z","receivedAt":"2024-03-24T01:13:13Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Over HTTP, NTLM and Kerberos require two rounds of authentication on the\nclient side. It's possible that there are custom authentication schemes\nthat also implement this same approach. Since these are tricky schemes\nto implement and the HTTP library in use may not always handle them\ngracefully on all systems, it would be helpful to allow the credential\nhelper to implement them instead for increased portability and\nrobustness.\n\nTo allow this to happen, add a boolean flag, continue, that indicates\nthat instead of failing when we get a 401, we should retry another round\nof authentication. However, this necessitates some changes in our\ncurrent credential code so that we can make this work.\n\nKeep the state[] headers between iterations, but only use them to send\nto the helper and only consider the new ones we read from the credential\nhelper to be valid on subsequent iterations. That avoids us passing\nstale data when we finally approve or reject the credential. Similarly,\nclear the multistage and wwwauth[] values appropriately so that we\ndon't pass stale data or think we're trying a multiround response when\nwe're not. Remove the credential values so that we can actually fill a\nsecond time with new responses.\n\nLimit the number of iterations of reauthentication we do to 3. This\nmeans that if there's a problem, we'll terminate with an error message\ninstead of retrying indefinitely and not informing the user (and\npossibly conducting a DoS on the server).\n\nIn our tests, handle creating multiple response output files from our\nhelper so we can verify that each of the messages sent is correct.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 16 +++++-\n builtin/credential.c | 1 +\n credential.c | 32 ++++++++++--\n credential.h | 27 +++++++++-\n http.c | 59 +++++++++++----------\n t/t5563-simple-http-auth.sh | 89 ++++++++++++++++++++++++++++++--\n 6 files changed, 187 insertions(+), 37 deletions(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex 6b7e017066..160dee5c6a 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -207,6 +207,19 @@ provided on input.\n This value should not be sent unless the appropriate capability (see below) is\n provided on input.\n \n+`continue`::\n+\tThis is a boolean value, which, if enabled, indicates that this\n+\tauthentication is a non-final part of a multistage authentication step. This\n+\tis common in protocols such as NTLM and Kerberos, where two rounds of client\n+\tauthentication are required, and setting this flag allows the credential\n+\thelper to implement the multistage authentication step. This flag should\n+\tonly be sent if a further stage is required; that is, if another round of\n+\tauthentication is expected.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input. This attribute is 'one-way' from a credential helper to\n+pass information to Git (or other programs invoking `git credential`).\n+\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\n@@ -225,7 +238,8 @@ to pass additional information to credential helpers.\n +\n There are two currently supported capabilities. The first is `authtype`, which\n indicates that the `authtype` and `credential` values are understood. The\n-second is `state`, which indicates that the `state[]` value is understood.\n+second is `state`, which indicates that the `state[]` and `continue` values are\n+understood.\n \n It is not obligatory to use the additional features just because the capability\n is supported, but they should not be provided without this capability.\ndiff --git a/builtin/credential.c b/builtin/credential.c\nindex 5123dabcf1..f14d1b5ed6 100644\n--- a/builtin/credential.c\n+++ b/builtin/credential.c\n@@ -22,6 +22,7 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n \n \tif (!strcmp(op, \"fill\")) {\n \t\tcredential_fill(&c, 0);\n+\t\tcredential_next_state(&c);\n \t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n \t} else if (!strcmp(op, \"approve\")) {\n \t\tcredential_approve(&c);\ndiff --git a/credential.c b/credential.c\nindex 0ca7c12895..9a08efe113 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -31,10 +31,23 @@ void credential_clear(struct credential *c)\n \tstring_list_clear(&c->helpers, 0);\n \tstrvec_clear(&c->wwwauth_headers);\n \tstrvec_clear(&c->state_headers);\n+\tstrvec_clear(&c->state_headers_to_send);\n \n \tcredential_init(c);\n }\n \n+void credential_next_state(struct credential *c)\n+{\n+\tstrvec_clear(&c->state_headers_to_send);\n+\tstrvec_swap(&c->state_headers, &c->state_headers_to_send);\n+}\n+\n+void credential_clear_secrets(struct credential *c)\n+{\n+\tFREE_AND_NULL(c->password);\n+\tFREE_AND_NULL(c->credential);\n+}\n+\n static void credential_set_all_capabilities(struct credential *c)\n {\n \tc->capa_authtype.request_initial = 1;\n@@ -295,6 +308,8 @@ int credential_read(struct credential *c, FILE *fp, int op_type)\n \t\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n \t\t\telse if (!strcmp(value, \"state\"))\n \t\t\t\tcredential_set_capability(&c->capa_state, op_type);\n+\t\t} else if (!strcmp(key, \"continue\")) {\n+\t\t\tc->multistage = !!git_config_bool(\"continue\", value);\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n@@ -359,8 +374,10 @@ void credential_write(const struct credential *c, FILE *fp, int op_type)\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n \tif (credential_has_capability(&c->capa_state, op_type)) {\n-\t\tfor (size_t i = 0; i < c->state_headers.nr; i++)\n-\t\t\tcredential_write_item(fp, \"state[]\", c->state_headers.v[i], 0);\n+\t\tif (c->multistage)\n+\t\t\tcredential_write_item(fp, \"continue\", \"1\", 0);\n+\t\tfor (size_t i = 0; i < c->state_headers_to_send.nr; i++)\n+\t\t\tcredential_write_item(fp, \"state[]\", c->state_headers_to_send.v[i], 0);\n \t}\n }\n \n@@ -431,6 +448,9 @@ void credential_fill(struct credential *c, int all_capabilities)\n \tif ((c->username && c->password) || c->credential)\n \t\treturn;\n \n+\tcredential_next_state(c);\n+\tc->multistage = 0;\n+\n \tcredential_apply_config(c);\n \tif (all_capabilities)\n \t\tcredential_set_all_capabilities(c);\n@@ -443,8 +463,10 @@ void credential_fill(struct credential *c, int all_capabilities)\n \t\t\t/* Reset expiry to maintain consistency */\n \t\t\tc->password_expiry_utc = TIME_MAX;\n \t\t}\n-\t\tif ((c->username && c->password) || c->credential)\n+\t\tif ((c->username && c->password) || c->credential) {\n+\t\t\tstrvec_clear(&c->wwwauth_headers);\n \t\t\treturn;\n+\t\t}\n \t\tif (c->quit)\n \t\t\tdie(\"credential helper '%s' told us to quit\",\n \t\t\t c->helpers.items[i].string);\n@@ -464,6 +486,8 @@ void credential_approve(struct credential *c)\n \tif (((!c->username || !c->password) && !c->credential) || c->password_expiry_utc < time(NULL))\n \t\treturn;\n \n+\tcredential_next_state(c);\n+\n \tcredential_apply_config(c);\n \n \tfor (i = 0; i < c->helpers.nr; i++)\n@@ -475,6 +499,8 @@ void credential_reject(struct credential *c)\n {\n \tint i;\n \n+\tcredential_next_state(c);\n+\n \tcredential_apply_config(c);\n \n \tfor (i = 0; i < c->helpers.nr; i++)\ndiff --git a/credential.h b/credential.h\nindex e2021455fe..adb1fc370a 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -143,10 +143,15 @@ struct credential {\n \tstruct strvec wwwauth_headers;\n \n \t/**\n-\t * A `strvec` of state headers from credential helpers.\n+\t * A `strvec` of state headers received from credential helpers.\n \t */\n \tstruct strvec state_headers;\n \n+\t/**\n+\t * A `strvec` of state headers to send to credential helpers.\n+\t */\n+\tstruct strvec state_headers_to_send;\n+\n \t/**\n \t * Internal use only. Keeps track of if we previously matched against a\n \t * WWW-Authenticate header line in order to re-fold future continuation\n@@ -156,6 +161,7 @@ struct credential {\n \n \tunsigned approved:1,\n \t\t configured:1,\n+\t\t multistage: 1,\n \t\t quit:1,\n \t\t use_http_path:1,\n \t\t username_from_proto:1;\n@@ -184,6 +190,7 @@ struct credential {\n \t.password_expiry_utc = TIME_MAX, \\\n \t.wwwauth_headers = STRVEC_INIT, \\\n \t.state_headers = STRVEC_INIT, \\\n+\t.state_headers_to_send = STRVEC_INIT, \\\n }\n \n /* Initialize a credential structure, setting all fields to empty. */\n@@ -229,6 +236,24 @@ void credential_approve(struct credential *);\n */\n void credential_reject(struct credential *);\n \n+/**\n+ * Clear the secrets in this credential, but leave other data intact.\n+ *\n+ * This is useful for resetting credentials in preparation for a subsequent\n+ * stage of filling.\n+ */\n+void credential_clear_secrets(struct credential *c);\n+\n+/**\n+ * Prepares the credential for the next iteration of the helper protocol by\n+ * updating the state headers to send with the ones read by the last iteration\n+ * of the protocol.\n+ *\n+ * Except for internal callers, this should be called exactly once between\n+ * reading credentials with `credential_fill` and writing them.\n+ */\n+void credential_next_state(struct credential *c);\n+\n int credential_read(struct credential *, FILE *, int);\n void credential_write(const struct credential *, FILE *, int);\n \ndiff --git a/http.c b/http.c\nindex f98c520924..9d373c6460 100644\n--- a/http.c\n+++ b/http.c\n@@ -1781,6 +1781,10 @@ static int handle_curl_result(struct slot_results *results)\n \telse if (results->http_code == 401) {\n \t\tif ((http_auth.username && http_auth.password) ||\\\n \t\t (http_auth.authtype && http_auth.credential)) {\n+\t\t\tif (http_auth.multistage) {\n+\t\t\t\tcredential_clear_secrets(&http_auth);\n+\t\t\t\treturn HTTP_REAUTH;\n+\t\t\t}\n \t\t\tcredential_reject(&http_auth);\n \t\t\treturn HTTP_NOAUTH;\n \t\t} else {\n@@ -2178,6 +2182,7 @@ static int http_request_reauth(const char *url,\n \t\t\t void *result, int target,\n \t\t\t struct http_get_options *options)\n {\n+\tint i = 3;\n \tint ret = http_request(url, result, target, options);\n \n \tif (ret != HTTP_OK && ret != HTTP_REAUTH)\n@@ -2191,35 +2196,35 @@ static int http_request_reauth(const char *url,\n \t\t}\n \t}\n \n-\tif (ret != HTTP_REAUTH)\n-\t\treturn ret;\n+\twhile (ret == HTTP_REAUTH && --i) {\n+\t\t/*\n+\t\t * The previous request may have put cruft into our output stream; we\n+\t\t * should clear it out before making our next request.\n+\t\t */\n+\t\tswitch (target) {\n+\t\tcase HTTP_REQUEST_STRBUF:\n+\t\t\tstrbuf_reset(result);\n+\t\t\tbreak;\n+\t\tcase HTTP_REQUEST_FILE:\n+\t\t\tif (fflush(result)) {\n+\t\t\t\terror_errno(\"unable to flush a file\");\n+\t\t\t\treturn HTTP_START_FAILED;\n+\t\t\t}\n+\t\t\trewind(result);\n+\t\t\tif (ftruncate(fileno(result), 0) < 0) {\n+\t\t\t\terror_errno(\"unable to truncate a file\");\n+\t\t\t\treturn HTTP_START_FAILED;\n+\t\t\t}\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tBUG(\"Unknown http_request target\");\n+\t\t}\n \n-\t/*\n-\t * The previous request may have put cruft into our output stream; we\n-\t * should clear it out before making our next request.\n-\t */\n-\tswitch (target) {\n-\tcase HTTP_REQUEST_STRBUF:\n-\t\tstrbuf_reset(result);\n-\t\tbreak;\n-\tcase HTTP_REQUEST_FILE:\n-\t\tif (fflush(result)) {\n-\t\t\terror_errno(\"unable to flush a file\");\n-\t\t\treturn HTTP_START_FAILED;\n-\t\t}\n-\t\trewind(result);\n-\t\tif (ftruncate(fileno(result), 0) < 0) {\n-\t\t\terror_errno(\"unable to truncate a file\");\n-\t\t\treturn HTTP_START_FAILED;\n-\t\t}\n-\t\tbreak;\n-\tdefault:\n-\t\tBUG(\"Unknown http_request target\");\n+\t\tcredential_fill(&http_auth, 1);\n+\n+\t\tret = http_request(url, result, target, options);\n \t}\n-\n-\tcredential_fill(&http_auth, 1);\n-\n-\treturn http_request(url, result, target, options);\n+\treturn ret;\n }\n \n int http_get_strbuf(const char *url,\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex 515185ae00..5d5caa3f58 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -21,9 +21,17 @@ test_expect_success 'setup_credential_helper' '\n \tCREDENTIAL_HELPER=\"$TRASH_DIRECTORY/bin/git-credential-test-helper\" &&\n \twrite_script \"$CREDENTIAL_HELPER\" <<-\\EOF\n \tcmd=$1\n-\tteefile=$cmd-query.cred\n+\tteefile=$cmd-query-temp.cred\n \tcatfile=$cmd-reply.cred\n \tsed -n -e \"/^$/q\" -e \"p\" >>$teefile\n+\tstate=$(sed -ne \"s/^state\\[\\]=helper://p\" \"$teefile\")\n+\tif test -z \"$state\"\n+\tthen\n+\t\tmv \"$teefile\" \"$cmd-query.cred\"\n+\telse\n+\t\tmv \"$teefile\" \"$cmd-query-$state.cred\"\n+\t\tcatfile=\"$cmd-reply-$state.cred\"\n+\tfi\n \tif test \"$cmd\" = \"get\"\n \tthen\n \t\tcat $catfile\n@@ -32,13 +40,15 @@ test_expect_success 'setup_credential_helper' '\n '\n \n set_credential_reply () {\n-\tcat >\"$TRASH_DIRECTORY/$1-reply.cred\"\n+\tlocal suffix=\"$(test -n \"$2\" && echo \"-$2\")\"\n+\tcat >\"$TRASH_DIRECTORY/$1-reply$suffix.cred\"\n }\n \n expect_credential_query () {\n-\tcat >\"$TRASH_DIRECTORY/$1-expect.cred\" &&\n-\ttest_cmp \"$TRASH_DIRECTORY/$1-expect.cred\" \\\n-\t\t \"$TRASH_DIRECTORY/$1-query.cred\"\n+\tlocal suffix=\"$(test -n \"$2\" && echo \"-$2\")\"\n+\tcat >\"$TRASH_DIRECTORY/$1-expect$suffix.cred\" &&\n+\ttest_cmp \"$TRASH_DIRECTORY/$1-expect$suffix.cred\" \\\n+\t\t \"$TRASH_DIRECTORY/$1-query$suffix.cred\"\n }\n \n per_test_cleanup () {\n@@ -479,4 +489,73 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \tEOF\n '\n \n+test_expect_success 'access using three-legged auth' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tcredential=YS1naXQtdG9rZW4=\n+\tstate[]=helper:foobar\n+\tcontinue=1\n+\tEOF\n+\n+\tset_credential_reply get foobar <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tcredential=YW5vdGhlci10b2tlbg==\n+\tstate[]=helper:bazquux\n+\tEOF\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tid=1 creds=Multistage YS1naXQtdG9rZW4=\n+\tid=2 creds=Multistage YW5vdGhlci10b2tlbg==\n+\tEOF\n+\n+\tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tid=1 status=401 response=WWW-Authenticate: Multistage challenge=\"456\"\n+\tid=1 status=401 response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=2 status=200\n+\tid=default response=WWW-Authenticate: Multistage challenge=\"123\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=Multistage challenge=\"123\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tEOF\n+\n+\texpect_credential_query get foobar <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=Multistage challenge=\"456\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tstate[]=helper:foobar\n+\tEOF\n+\n+\texpect_credential_query store bazquux <<-EOF\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tcredential=YW5vdGhlci10b2tlbg==\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\tstate[]=helper:bazquux\n+\tEOF\n+'\n+\n test_done\n"},{"id":"491333","messageId":"20240324011301.1553072-3-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 02/13] remote-curl: reset headers on new request","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:50Z","receivedAt":"2024-03-24T01:20:10Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When we retry a post_rpc request, we currently reuse the same headers as\nbefore. In the future, we'd like to be able to modify them based on the\nresult we get back, so let's reset them on each retry so we can avoid\nsending potentially duplicate headers if the values change.\n\nSigned-off-by: brian m. carlson \n---\n remote-curl.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 1161dc7fed..e37eaa17b7 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -893,7 +893,7 @@ static curl_off_t xcurl_off_t(size_t len)\n static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_received)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = http_copy_default_headers();\n+\tstruct curl_slist *headers = NULL;\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\n@@ -935,6 +935,8 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \t\t\tneeds_100_continue = 1;\n \t}\n \n+retry:\n+\theaders = http_copy_default_headers();\n \theaders = curl_slist_append(headers, rpc->hdr_content_type);\n \theaders = curl_slist_append(headers, rpc->hdr_accept);\n \theaders = curl_slist_append(headers, needs_100_continue ?\n@@ -948,7 +950,6 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \tif (rpc->protocol_header)\n \t\theaders = curl_slist_append(headers, rpc->protocol_header);\n \n-retry:\n \tslot = get_active_slot();\n \n \tcurl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0);\n@@ -1044,6 +1045,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \terr = run_slot(slot, NULL);\n \tif (err == HTTP_REAUTH && !large_request) {\n \t\tcredential_fill(&http_auth);\n+\t\tcurl_slist_free_all(headers);\n \t\tgoto retry;\n \t}\n \tif (err != HTTP_OK)\n"},{"id":"491334","messageId":"20240324011301.1553072-1-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":null,"subject":"[PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:48Z","receivedAt":"2024-03-24T01:20:10Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Right now, HTTP authentication in Git is mostly limited to approaches\nthat require a username and password or are Kerberos (GSSAPI). In\naddition, we effectively require that libcurl (or, for other software,\nsuch as Git LFS, using the credential helper, that HTTP library) knows\nhow to implement the authentication scheme.\n\nHowever, this poses two sets of problems. First, some sites, such as\nAzure DevOps, want to use Bearer authentication, which we don't support.\nThis is implemented using `http.extraHeader`, which is not a secure way\nto store credentials, since our credential helper protocol does not\nsupport this functionality.\n\nIn addition, other tools using the credential helper protocol do not\nsupport the variety of authentication mechanisms that Git does.\nSpecifically, making NTLM function in a useful way on Windows is\nnontrivial and requires extensive integration and testing with C code,\nand because of this difficulty and the fact that NTLM uses cryptography\nknown to be insecure since 1995, there is often little interest in\nimplementing this support outside of libcurl. However, it would be\nhelpful if people who want to use it can still use it.\n\nThis series introduces new functionality to the credential helper\nprotocol that allows helpers to produce credentials for arbitrary HTTP\nauthentication schemes using the `authtype` and `credential`[0] fields.\nThis allows a suitable credential helper to send Bearer credentials or\nany other standard or custom authentication scheme. (It may be able to\nbe extended to other functionality in the future, such as\ngit-send-email, to implement custom SASL functionality, and due care has\nbeen taken to make the protocol adequately flexible for that purpose.)\n\nIn addition, the protocol is also expanded to include per-helper state\nand multi-legged authentication (the former is effectively required for\nthe latter). The per-helper state can be useful to help credential\nhelpers identify where the credential is stored, or any other\ninformation necessary. Because NTLM and Negotiate (Kerberos/wrapped\nNTLM) require two rounds of authentication, the multi-legged\nauthentication support along with per-helper state allows the helper to\nsupport these authentication methods without Git or other clients having\nto be aware of how they work. (This would also be useful for SASL, as\nmentioned above.)\n\nThis series introduces a capability mechanism to announce this\nfunctionality, which allows a helper to provide a username and password\non older versions of Git while supporting more advanced functionality on\nnewer versions. (This is especially important on Azure DevOps, where\nNTLM uses a username and password but Basic or Bearer can use a personal\naccess token.) It is also designed such that extremely simple\ncredential helpers, such as the shell one-liner in the Git FAQ that\nreads from the environment, don't accidentally claim to support\nfunctionality they don't offer.\n\nIn addition, there is documentation for the expanded protocol, although\nnone of the built-in helpers have been updated (that will be a future\nseries for those for which it's possible).\n\nMy personal interest here is getting credentials out of config files\nwith `http.extraHeader` (which a future series will produce a warning\nfor) and also allowing Git LFS to support Digest and NTLM with a\nsuitable credential helper. Git LFS used to support NTLM using custom\ncode (because the Go standard library does not), but it was found to be\nbroken in lots of ways on Windows, and nobody with a Windows system\nwanted to fix it or support it, so we removed it. However, there are\nstill some people who do want to use it, so allowing them to use a\ncustom credential helper they maintain themselves seems like the best\nway forward. Despite the advantages of this series for Azure DevOps, I\nhave no personal or professional stake in their product; my only\ninterest is the general one in whether their users can securely store\ncredentials. I believe the changes here are of general advantage to the\nGit userbase in a variety of ways such that the goal of this series\nshould be uncontroversial.\n\nFeedback on any portion of this series is of course welcome.\n\n[0] A name different from `password` was explicitly chosen to avoid\nconfusion from less capable protocol helpers so that they don't\naccidentally send invalid data. This does have the downside that\ncredential helpers must learn a new field to not log, but that should be\ngenerally easy to fix in most cases.\n\nbrian m. carlson (13):\n credential: add an authtype field\n remote-curl: reset headers on new request\n http: use new headers for each object request\n credential: add a field for pre-encoded credentials\n credential: gate new fields on capability\n docs: indicate new credential protocol fields\n http: add support for authtype and credential\n credential: add an argument to keep state\n credential: enable state capability\n docs: set a limit on credential line length\n t5563: refactor for multi-stage authentication\n strvec: implement swapping two strvecs\n credential: add support for multistage credential rounds\n\n Documentation/git-credential.txt | 59 +++++-\n builtin/credential-cache--daemon.c | 2 +-\n builtin/credential-store.c | 2 +-\n builtin/credential.c | 7 +-\n credential.c | 114 ++++++++++-\n credential.h | 69 ++++++-\n http.c | 128 +++++++-----\n http.h | 5 +\n imap-send.c | 2 +-\n remote-curl.c | 14 +-\n strvec.c | 7 +\n strvec.h | 5 +\n t/lib-httpd/nph-custom-auth.sh | 17 +-\n t/t0300-credentials.sh | 136 ++++++++++++-\n t/t5563-simple-http-auth.sh | 308 +++++++++++++++++++++++++----\n 15 files changed, 760 insertions(+), 115 deletions(-)\n\n"},{"id":"491335","messageId":"20240324011301.1553072-4-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 03/13] http: use new headers for each object request","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:51Z","receivedAt":"2024-03-24T01:20:10Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Currently we create one set of headers for all object requests and reuse\nit. However, we'll need to adjust the headers for authentication\npurposes in the future, so let's create a new set for each request so\nthat we can adjust them if the authentication changes.\n\nNote that the cost of allocation here is tiny compared to the fact that\nwe're making a network call, not to mention probably a full TLS\nconnection, so this shouldn't have a significant impact on performance.\nMoreover, nobody who cares about performance is using the dumb HTTP\nprotocol anyway, since it often makes huge numbers of requests compared\nto the smart protocol.\n\nSigned-off-by: brian m. carlson \n---\n http.c | 19 +++++++++++--------\n http.h | 2 ++\n 2 files changed, 13 insertions(+), 8 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex e73b136e58..1c2200da77 100644\n--- a/http.c\n+++ b/http.c\n@@ -128,7 +128,6 @@ static unsigned long empty_auth_useless =\n \t| CURLAUTH_DIGEST;\n \n static struct curl_slist *pragma_header;\n-static struct curl_slist *no_pragma_header;\n static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;\n \n static struct curl_slist *host_resolutions;\n@@ -299,6 +298,11 @@ size_t fwrite_null(char *ptr UNUSED, size_t eltsize UNUSED, size_t nmemb,\n \treturn nmemb;\n }\n \n+static struct curl_slist *object_request_headers(void)\n+{\n+\treturn curl_slist_append(http_copy_default_headers(), \"Pragma:\");\n+}\n+\n static void closedown_active_slot(struct active_request_slot *slot)\n {\n \tactive_requests--;\n@@ -1275,8 +1279,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \n \tpragma_header = curl_slist_append(http_copy_default_headers(),\n \t\t\"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n-\t\t\"Pragma:\");\n \n \t{\n \t\tchar *http_max_requests = getenv(\"GIT_HTTP_MAX_REQUESTS\");\n@@ -1360,8 +1362,6 @@ void http_cleanup(void)\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n-\tcurl_slist_free_all(no_pragma_header);\n-\tno_pragma_header = NULL;\n \n \tcurl_slist_free_all(host_resolutions);\n \thost_resolutions = NULL;\n@@ -2370,6 +2370,7 @@ void release_http_pack_request(struct http_pack_request *preq)\n \t}\n \tpreq->slot = NULL;\n \tstrbuf_release(&preq->tmpfile);\n+\tcurl_slist_free_all(preq->headers);\n \tfree(preq->url);\n \tfree(preq);\n }\n@@ -2454,11 +2455,11 @@ struct http_pack_request *new_direct_http_pack_request(\n \t}\n \n \tpreq->slot = get_active_slot();\n+\tpreq->headers = object_request_headers();\n \tcurl_easy_setopt(preq->slot->curl, CURLOPT_WRITEDATA, preq->packfile);\n \tcurl_easy_setopt(preq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite);\n \tcurl_easy_setopt(preq->slot->curl, CURLOPT_URL, preq->url);\n-\tcurl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER,\n-\t\tno_pragma_header);\n+\tcurl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER, preq->headers);\n \n \t/*\n \t * If there is data present from a previous transfer attempt,\n@@ -2624,13 +2625,14 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \t}\n \n \tfreq->slot = get_active_slot();\n+\tfreq->headers = object_request_headers();\n \n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_WRITEDATA, freq);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_FAILONERROR, 0);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite_sha1_file);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_ERRORBUFFER, freq->errorstr);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_URL, freq->url);\n-\tcurl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, no_pragma_header);\n+\tcurl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, freq->headers);\n \n \t/*\n \t * If we have successfully processed data from a previous fetch\n@@ -2718,5 +2720,6 @@ void release_http_object_request(struct http_object_request *freq)\n \t\trelease_active_slot(freq->slot);\n \t\tfreq->slot = NULL;\n \t}\n+\tcurl_slist_free_all(freq->headers);\n \tstrbuf_release(&freq->tmpfile);\n }\ndiff --git a/http.h b/http.h\nindex 3af19a8bf5..c5f8cc4620 100644\n--- a/http.h\n+++ b/http.h\n@@ -196,6 +196,7 @@ struct http_pack_request {\n \tFILE *packfile;\n \tstruct strbuf tmpfile;\n \tstruct active_request_slot *slot;\n+\tstruct curl_slist *headers;\n };\n \n struct http_pack_request *new_http_pack_request(\n@@ -229,6 +230,7 @@ struct http_object_request {\n \tint zret;\n \tint rename;\n \tstruct active_request_slot *slot;\n+\tstruct curl_slist *headers;\n };\n \n struct http_object_request *new_http_object_request(\n"},{"id":"491336","messageId":"20240324011301.1553072-5-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 04/13] credential: add a field for pre-encoded credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:52Z","receivedAt":"2024-03-24T01:20:10Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"At the moment, our credential code wants to find a username and password\nfor access, which, for HTTP, it will pass to libcurl to encode and\nprocess. However, many users want to use authentication schemes that\nlibcurl doesn't support, such as Bearer authentication. In these\nschemes, the secret is not a username and password pair, but some sort\nof token that meets the production for authentication data in the RFC.\n\nIn fact, in general, it's useful to allow our credential helper to have\nknowledge about what specifically to put in the protocol header. Thus,\nadd a field, credential, which contains data that's preencoded to be\nsuitable for the protocol in question. If we have such data, we need\nneither a username nor a password, so make that adjustment as well.\n\nIt is in theory possible to reuse the password field for this. However,\nif we do so, we must know whether the credential helper supports our new\nscheme before sending it data, which necessitates some sort of\ncapability inquiry, because otherwise an uninformed credential helper\nwould store our preencoded data as a password, which would fail the next\ntime we attempted to connect to the remote server. This design is\nsubstantially simpler, and we can hint to the credential helper that we\nsupport this approach with a simple new field instead of needing to\nquery it first.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 14 ++++++++++----\n credential.h | 1 +\n 2 files changed, 11 insertions(+), 4 deletions(-)\n\ndiff --git a/credential.c b/credential.c\nindex 3dec433df5..c521822e5a 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -25,6 +25,7 @@ void credential_clear(struct credential *c)\n \tfree(c->path);\n \tfree(c->username);\n \tfree(c->password);\n+\tfree(c->credential);\n \tfree(c->oauth_refresh_token);\n \tfree(c->authtype);\n \tstring_list_clear(&c->helpers, 0);\n@@ -234,6 +235,9 @@ int credential_read(struct credential *c, FILE *fp)\n \t\t} else if (!strcmp(key, \"password\")) {\n \t\t\tfree(c->password);\n \t\t\tc->password = xstrdup(value);\n+\t\t} else if (!strcmp(key, \"credential\")) {\n+\t\t\tfree(c->credential);\n+\t\t\tc->credential = xstrdup(value);\n \t\t} else if (!strcmp(key, \"protocol\")) {\n \t\t\tfree(c->protocol);\n \t\t\tc->protocol = xstrdup(value);\n@@ -291,6 +295,7 @@ void credential_write(const struct credential *c, FILE *fp)\n \tcredential_write_item(fp, \"path\", c->path, 0);\n \tcredential_write_item(fp, \"username\", c->username, 0);\n \tcredential_write_item(fp, \"password\", c->password, 0);\n+\tcredential_write_item(fp, \"credential\", c->credential, 0);\n \tcredential_write_item(fp, \"oauth_refresh_token\", c->oauth_refresh_token, 0);\n \tif (c->password_expiry_utc != TIME_MAX) {\n \t\tchar *s = xstrfmt(\"%\"PRItime, c->password_expiry_utc);\n@@ -366,7 +371,7 @@ void credential_fill(struct credential *c)\n {\n \tint i;\n \n-\tif (c->username && c->password)\n+\tif ((c->username && c->password) || c->credential)\n \t\treturn;\n \n \tcredential_apply_config(c);\n@@ -379,7 +384,7 @@ void credential_fill(struct credential *c)\n \t\t\t/* Reset expiry to maintain consistency */\n \t\t\tc->password_expiry_utc = TIME_MAX;\n \t\t}\n-\t\tif (c->username && c->password)\n+\t\tif ((c->username && c->password) || c->credential)\n \t\t\treturn;\n \t\tif (c->quit)\n \t\t\tdie(\"credential helper '%s' told us to quit\",\n@@ -387,7 +392,7 @@ void credential_fill(struct credential *c)\n \t}\n \n \tcredential_getpass(c);\n-\tif (!c->username && !c->password)\n+\tif (!c->username && !c->password && !c->credential)\n \t\tdie(\"unable to get password from user\");\n }\n \n@@ -397,7 +402,7 @@ void credential_approve(struct credential *c)\n \n \tif (c->approved)\n \t\treturn;\n-\tif (!c->username || !c->password || c->password_expiry_utc < time(NULL))\n+\tif (((!c->username || !c->password) && !c->credential) || c->password_expiry_utc < time(NULL))\n \t\treturn;\n \n \tcredential_apply_config(c);\n@@ -418,6 +423,7 @@ void credential_reject(struct credential *c)\n \n \tFREE_AND_NULL(c->username);\n \tFREE_AND_NULL(c->password);\n+\tFREE_AND_NULL(c->credential);\n \tFREE_AND_NULL(c->oauth_refresh_token);\n \tc->password_expiry_utc = TIME_MAX;\n \tc->approved = 0;\ndiff --git a/credential.h b/credential.h\nindex dc96ca0318..9db892cf4d 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -138,6 +138,7 @@ struct credential {\n \n \tchar *username;\n \tchar *password;\n+\tchar *credential;\n \tchar *protocol;\n \tchar *host;\n \tchar *path;\n"},{"id":"491337","messageId":"20240324011301.1553072-2-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH 01/13] credential: add an authtype field","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T01:12:49Z","receivedAt":"2024-03-24T01:20:10Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When Git makes an HTTP request, it can negotiate the type of\nauthentication to use with the server provided the authentication scheme\nis one of a few well-known types (Basic, Digest, NTLM, or Negotiate).\nHowever, some servers wish to use other types of authentication, such as\nthe Bearer type from OAuth2. Since libcurl doesn't natively support\nthis type, it isn't possible to use it, and the user is forced to\nspecify the Authorization header using the http.extraheader setting.\n\nHowever, storing a plaintext token in the repository configuration is\nnot very secure, especially if a repository can be shared by multiple\nparties. We already have support for many types of secure credential\nstorage by using credential helpers, so let's teach credential helpers\nhow to produce credentials for an arbitrary scheme.\n\nIf the credential helper specifies an authtype field, then it specifies\nan authentication scheme (e.g., Bearer) and the password field specifies\nthe raw authentication token, with any encoding already specified. We\nreuse the password field for this because some credential helpers store\nthe metadata without encryption even though the password is encrypted,\nand we'd like to avoid insecure storage if an older version of the\ncredential helper gets ahold of the data.\n\nThe username is not used in this case, but it is still preserved for the\npurpose of finding the right credential if the user has multiple\naccounts.\n\nIf the authtype field is not specified, then the password behaves as\nnormal and it is passed along with the username to libcurl.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 5 +++++\n credential.h | 6 ++++++\n 2 files changed, 11 insertions(+)\n\ndiff --git a/credential.c b/credential.c\nindex 18098bd35e..3dec433df5 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -26,6 +26,7 @@ void credential_clear(struct credential *c)\n \tfree(c->username);\n \tfree(c->password);\n \tfree(c->oauth_refresh_token);\n+\tfree(c->authtype);\n \tstring_list_clear(&c->helpers, 0);\n \tstrvec_clear(&c->wwwauth_headers);\n \n@@ -252,6 +253,9 @@ int credential_read(struct credential *c, FILE *fp)\n \t\t} else if (!strcmp(key, \"oauth_refresh_token\")) {\n \t\t\tfree(c->oauth_refresh_token);\n \t\t\tc->oauth_refresh_token = xstrdup(value);\n+\t\t} else if (!strcmp(key, \"authtype\")) {\n+\t\t\tfree(c->authtype);\n+\t\t\tc->authtype = xstrdup(value);\n \t\t} else if (!strcmp(key, \"url\")) {\n \t\t\tcredential_from_url(c, value);\n \t\t} else if (!strcmp(key, \"quit\")) {\n@@ -295,6 +299,7 @@ void credential_write(const struct credential *c, FILE *fp)\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n+\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n }\n \n static int run_credential_helper(struct credential *c,\ndiff --git a/credential.h b/credential.h\nindex acc41adf54..dc96ca0318 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -143,6 +143,12 @@ struct credential {\n \tchar *path;\n \tchar *oauth_refresh_token;\n \ttimestamp_t password_expiry_utc;\n+\n+\t/**\n+\t * The authorization scheme to use. If this is NULL, libcurl is free to\n+\t * negotiate any scheme it likes.\n+\t */\n+\tchar *authtype;\n };\n \n #define CREDENTIAL_INIT { \\\n"},{"id":"491340","messageId":"xmqqedc0v029.fsf@gitster.g","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-24T02:24:46Z","receivedAt":"2024-03-24T02:24:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> ... and because of this difficulty and the fact that NTLM uses cryptography\n> known to be insecure since 1995, there is often little interest in\n> implementing this support outside of libcurl. However, it would be\n> helpful if people who want to use it can still use it.\n\nThis position was a bit surprising to me to come from you, but\nperhaps I am mixing up my recollection of your past work on this\nproject with somebody else's? I somehow expected to hear something\nmore like \"if a less secure thing is cumbersome to implement, let it\nbe, as that is better for the world\". But I am OK to add less secure\nthing as long as it is an opt-in \"easy way out\".\n\nEverything else I read in the cover letter made sense to me. I just\nwanted to say that the above part was a bit surprising.\n\nThanks.\n"},{"id":"491354","messageId":"ZgBE37Xs3TQEF2c-@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"xmqqedc0v029.fsf@gitster.g","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-24T15:21:03Z","receivedAt":"2024-03-24T15:21:05Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-24 at 02:24:46, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > ... and because of this difficulty and the fact that NTLM uses cryptography\n> > known to be insecure since 1995, there is often little interest in\n> > implementing this support outside of libcurl. However, it would be\n> > helpful if people who want to use it can still use it.\n> \n> This position was a bit surprising to me to come from you, but\n> perhaps I am mixing up my recollection of your past work on this\n> project with somebody else's? I somehow expected to hear something\n> more like \"if a less secure thing is cumbersome to implement, let it\n> be, as that is better for the world\". But I am OK to add less secure\n> thing as long as it is an opt-in \"easy way out\".\n> \n> Everything else I read in the cover letter made sense to me. I just\n> wanted to say that the above part was a bit surprising.\n\nI do firmly feel that NTLM should go gentle into that good night.\n\nHowever, we've seen a lot of user questions on Git LFS from users who\nwant to use NTLM or Digest, which we don't support and probably will not\nfor security and maintainability reasons, but which their corporate\nenvironment imprudently requires. Thus, my goal is to make it _possible_\nfor people to implement this, but it to make it their responsibility (or\nthat of a suitable open source project) to do so instead of asking me to\nmaintain it. That, I think, is a fair and equitable tradeoff for this\nsituation.\n\nAlso, right now, Windows users sometimes choose to use the older v2.13.3\nversion of Git LFS which, due to a Go standard library bug, has an\narbitrary code execution vulnerability, but which did support NTLM.\nThus, it would also be better for security to have people on a suitably\npatched version of Git LFS with an external NTLM helper.\n\nThis series would also, in an approach that's better for security, allow\npeople to use better Kerberos mechanisms than what Windows supports\nnatively, or to use an AWS HMAC-v4-style request signing (using\nHMAC-SHA-256) if they want to do so, both of which would be a win for\nsecurity.\n\nI could have put all of this into the cover letter, but I felt it was\npretty long and didn't want to sell this as an advantage only for Git\nLFS when I think it provides general benefit for Git users. I know\nthe policy of the project is not to prioritize any one external\nproject, and I try to be sensitive to that here.\n\nI don't think this constitutes a marked change in my historical\n\"let's-remove-all-the-obsolete-cryptography-and-obsolete-operating-systems\"\napproach, but I see how it might look that way at first glance.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491360","messageId":"xmqqzfunsj59.fsf@gitster.g","threadId":"61189","inReplyTo":"ZgBE37Xs3TQEF2c-@tapette.crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-24T16:13:06Z","receivedAt":"2024-03-24T16:13:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n>> be, as that is better for the world\". But I am OK to add less secure\n>> thing as long as it is an opt-in \"easy way out\".\n> \n> ... Thus, my goal is to make it _possible_\n> for people to implement this, but it to make it their responsibility (or\n> that of a suitable open source project) to do so instead of asking me to\n> maintain it. That, I think, is a fair and equitable tradeoff for this\n> situation.\n\nYup, I think we are in agreement. I have no problem with giving\nthem a long rope ;-)\n"},{"id":"491543","messageId":"20240325231609.284-1-mirth.hickford@gmail.com","threadId":"61189","inReplyTo":"20240324011301.1553072-7-sandals@crustytoothpaste.net","subject":"Re: [PATCH 06/13] docs: indicate new credential protocol fields","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-03-25T23:16:09Z","receivedAt":"2024-03-25T23:16:20Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"> Tell users that they can continue to use a username and password even if\n> the new capability is supported.\n> \n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/git-credential.txt | 34 +++++++++++++++++++++++++++++++-\n> 1 file changed, 33 insertions(+), 1 deletion(-)\n> \n> diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\n> index 918a0aa42b..f3ed3a82fa 100644\n> --- a/Documentation/git-credential.txt\n> +++ b/Documentation/git-credential.txt\n> @@ -178,6 +178,24 @@ empty string.\n> Components which are missing from the URL (e.g., there is no\n> username in the example above) will be left unset.\n> \n> +`authtype`::\n> +\tThis indicates that the authentication scheme in question should be used.\n> +\tCommon values for HTTP and HTTPS include `basic`, `digest`, and `ntlm`,\n> +\talthough the latter two are insecure and should not be used. If `credential`\n> +\tis used, this may be set to an arbitrary string suitable for the protocol in\n> +\tquestion (usually HTTP).\n\nHow about adding 'bearer' to this list? Popular hosts Bitbucket https://bitbucket.org and Gitea/Forgejo (such as https://codeberg.org) support Bearer auth with OAuth tokens.\n\n> ++\n> +This value should not be sent unless the appropriate capability (see below) is\n> +provided on input.\n> +\n> +`credential`::\n> +\tThe pre-encoded credential, suitable for the protocol in question (usually\n> +\tHTTP). If this key is sent, `authtype` is mandatory, and `username` and\n> +\t`password` are not used.\n\nA credential protocol attribute named 'credential' is confusing. How about 'authorization' since it determines the HTTP Authorization header? This detail is surely worth mentioning too.\n\n> ++\n> +This value should not be sent unless the appropriate capability (see below) is\n> +provided on input.\n> +\n> `wwwauth[]`::\n> \n> \tWhen an HTTP response is received by Git that includes one or more\n> @@ -189,7 +207,21 @@ attribute 'wwwauth[]', where the order of the attributes is the same as\n> they appear in the HTTP response. This attribute is 'one-way' from Git\n> to pass additional information to credential helpers.\n> \n> -Unrecognised attributes are silently discarded.\n> +`capability[]`::\n> +\tThis signals that the caller supports the capability in question.\n>\n>\n> \n> +\tThis can be used to provide better, more specific data as part of the\n> +\tprotocol.\n> ++\n> +The only capability currently supported is `authtype`, which indicates that the\n> +`authtype` and `credential` values are understood. It is not obligatory to use\n> +these values in such a case, but they should not be provided without this\n> +capability.\n>\n> ++\n> +Callers of `git credential` and credential helpers should emit the\n> +capabilities they support unconditionally, and Git will gracefully\n> +handle passing them on.\n> +> +Unrecognised attributes and capabilities are silently discarded.\n"},{"id":"491552","messageId":"ZgIKqJaE79-Erzif@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"20240325231609.284-1-mirth.hickford@gmail.com","subject":"Re: [PATCH 06/13] docs: indicate new credential protocol fields","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-25T23:37:12Z","receivedAt":"2024-03-25T23:37:21Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-25 at 23:16:09, M Hickford wrote:\n> > +`authtype`::\n> > +\tThis indicates that the authentication scheme in question should be used.\n> > +\tCommon values for HTTP and HTTPS include `basic`, `digest`, and `ntlm`,\n> > +\talthough the latter two are insecure and should not be used. If `credential`\n> > +\tis used, this may be set to an arbitrary string suitable for the protocol in\n> > +\tquestion (usually HTTP).\n> \n> How about adding 'bearer' to this list? Popular hosts Bitbucket\n> https://bitbucket.org and Gitea/Forgejo (such as https://codeberg.org)\n> support Bearer auth with OAuth tokens.\n\nSure, I can do that.\n\n> > ++\n> > +This value should not be sent unless the appropriate capability (see below) is\n> > +provided on input.\n> > +\n> > +`credential`::\n> > +\tThe pre-encoded credential, suitable for the protocol in question (usually\n> > +\tHTTP). If this key is sent, `authtype` is mandatory, and `username` and\n> > +\t`password` are not used.\n> \n> A credential protocol attribute named 'credential' is confusing. How\n> about 'authorization' since it determines the HTTP Authorization\n> header? This detail is surely worth mentioning too.\n\nI don't want this to be very specific to HTTP, so I don't think that's a\ngreat name. As I mentioned in the cover letter, I might well extend\nthis to IMAP and SMTP for our mail handling in the future, and that name\nwouldn't work well there.\n\nI named it `credential` because, well, it's the credential that's used\nin the protocol. I feel like saying that the field represents \"the\nauthorization\" sounds unnatural. It's not wrong, per se, but it sounds\nconfusing.\n\nI'm open to other ideas if you or others have them, but between these\ntwo, I think I'd prefer to stick with `credential`.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491646","messageId":"ZgPSl0hWjH5B45Rt@tanuki","threadId":"61189","inReplyTo":"20240324011301.1553072-4-sandals@crustytoothpaste.net","subject":"Re: [PATCH 03/13] http: use new headers for each object request","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-03-27T08:02:31Z","receivedAt":"2024-03-27T08:02:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Mar 24, 2024 at 01:12:51AM +0000, brian m. carlson wrote:\n> Currently we create one set of headers for all object requests and reuse\n> it. However, we'll need to adjust the headers for authentication\n> purposes in the future, so let's create a new set for each request so\n> that we can adjust them if the authentication changes.\n> \n> Note that the cost of allocation here is tiny compared to the fact that\n> we're making a network call, not to mention probably a full TLS\n> connection, so this shouldn't have a significant impact on performance.\n> Moreover, nobody who cares about performance is using the dumb HTTP\n> protocol anyway, since it often makes huge numbers of requests compared\n> to the smart protocol.\n> \n> Signed-off-by: brian m. carlson \n> ---\n> http.c | 19 +++++++++++--------\n> http.h | 2 ++\n> 2 files changed, 13 insertions(+), 8 deletions(-)\n> \n> diff --git a/http.c b/http.c\n> index e73b136e58..1c2200da77 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -128,7 +128,6 @@ static unsigned long empty_auth_useless =\n> \t| CURLAUTH_DIGEST;\n> \n> static struct curl_slist *pragma_header;\n> -static struct curl_slist *no_pragma_header;\n> static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;\n> \n> static struct curl_slist *host_resolutions;\n\nNice to see that this also allows us to get rid of one more global\nvariable.\n\n> @@ -299,6 +298,11 @@ size_t fwrite_null(char *ptr UNUSED, size_t eltsize UNUSED, size_t nmemb,\n> \treturn nmemb;\n> }\n> \n> +static struct curl_slist *object_request_headers(void)\n> +{\n> +\treturn curl_slist_append(http_copy_default_headers(), \"Pragma:\");\n> +}\n> +\n> static void closedown_active_slot(struct active_request_slot *slot)\n> {\n> \tactive_requests--;\n> @@ -1275,8 +1279,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n> \n> \tpragma_header = curl_slist_append(http_copy_default_headers(),\n> \t\t\"Pragma: no-cache\");\n> -\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n> -\t\t\"Pragma:\");\n> \n> \t{\n> \t\tchar *http_max_requests = getenv(\"GIT_HTTP_MAX_REQUESTS\");\n> @@ -1360,8 +1362,6 @@ void http_cleanup(void)\n> \tcurl_slist_free_all(pragma_header);\n> \tpragma_header = NULL;\n> \n> -\tcurl_slist_free_all(no_pragma_header);\n> -\tno_pragma_header = NULL;\n> \n\nNit: this leaves behind two consecutive empty lines.\n\nPatrick\n\n> \tcurl_slist_free_all(host_resolutions);\n> \thost_resolutions = NULL;\n> @@ -2370,6 +2370,7 @@ void release_http_pack_request(struct http_pack_request *preq)\n> \t}\n> \tpreq->slot = NULL;\n> \tstrbuf_release(&preq->tmpfile);\n> +\tcurl_slist_free_all(preq->headers);\n> \tfree(preq->url);\n> \tfree(preq);\n> }\n> @@ -2454,11 +2455,11 @@ struct http_pack_request *new_direct_http_pack_request(\n> \t}\n> \n> \tpreq->slot = get_active_slot();\n> +\tpreq->headers = object_request_headers();\n> \tcurl_easy_setopt(preq->slot->curl, CURLOPT_WRITEDATA, preq->packfile);\n> \tcurl_easy_setopt(preq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite);\n> \tcurl_easy_setopt(preq->slot->curl, CURLOPT_URL, preq->url);\n> -\tcurl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER,\n> -\t\tno_pragma_header);\n> +\tcurl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER, preq->headers);\n> \n> \t/*\n> \t * If there is data present from a previous transfer attempt,\n> @@ -2624,13 +2625,14 @@ struct http_object_request *new_http_object_request(const char *base_url,\n> \t}\n> \n> \tfreq->slot = get_active_slot();\n> +\tfreq->headers = object_request_headers();\n> \n> \tcurl_easy_setopt(freq->slot->curl, CURLOPT_WRITEDATA, freq);\n> \tcurl_easy_setopt(freq->slot->curl, CURLOPT_FAILONERROR, 0);\n> \tcurl_easy_setopt(freq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite_sha1_file);\n> \tcurl_easy_setopt(freq->slot->curl, CURLOPT_ERRORBUFFER, freq->errorstr);\n> \tcurl_easy_setopt(freq->slot->curl, CURLOPT_URL, freq->url);\n> -\tcurl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, no_pragma_header);\n> +\tcurl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, freq->headers);\n> \n> \t/*\n> \t * If we have successfully processed data from a previous fetch\n> @@ -2718,5 +2720,6 @@ void release_http_object_request(struct http_object_request *freq)\n> \t\trelease_active_slot(freq->slot);\n> \t\tfreq->slot = NULL;\n> \t}\n> +\tcurl_slist_free_all(freq->headers);\n> \tstrbuf_release(&freq->tmpfile);\n> }\n> diff --git a/http.h b/http.h\n> index 3af19a8bf5..c5f8cc4620 100644\n> --- a/http.h\n> +++ b/http.h\n> @@ -196,6 +196,7 @@ struct http_pack_request {\n> \tFILE *packfile;\n> \tstruct strbuf tmpfile;\n> \tstruct active_request_slot *slot;\n> +\tstruct curl_slist *headers;\n> };\n> \n> struct http_pack_request *new_http_pack_request(\n> @@ -229,6 +230,7 @@ struct http_object_request {\n> \tint zret;\n> \tint rename;\n> \tstruct active_request_slot *slot;\n> +\tstruct curl_slist *headers;\n> };\n> \n> struct http_object_request *new_http_object_request(\n> \n"},{"id":"491647","messageId":"ZgPSn_D8luXQyZGf@tanuki","threadId":"61189","inReplyTo":"20240324011301.1553072-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-03-27T08:02:39Z","receivedAt":"2024-03-27T08:02:43Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Mar 24, 2024 at 01:12:53AM +0000, brian m. carlson wrote:\n> We support the new credential and authtype fields, but we lack a way to\n> indicate to a credential helper that we'd like them to be used. Without\n> some sort of indication, the credential helper doesn't know if it should\n> try to provide us a username and password, or a pre-encoded credential.\n> For example, the helper might prefer a more restricted Bearer token if\n> pre-encoded credentials are possible, but might have to fall back to\n> more general username and password if not.\n> \n> Let's provide a simple way to indicate whether Git (or, for that matter,\n> the helper) is capable of understanding the authtype and credential\n> fields. We send this capability when we generate a request, and the\n> other side may reply to indicate to us that it does, too.\n> \n> For now, don't enable sending capabilities for the HTTP code. In a\n> future commit, we'll introduce appropriate handling for that code,\n> which requires more in-depth work.\n> \n> The logic for determining whether a capability is supported may seem\n> complex, but it is not. At each stage, we emit the capability to the\n> following stage if all preceding stages have declared it. Thus, if the\n> caller to git credential fill didn't declare it, then we won't send it\n> to the helper, and if fill's caller did send but the helper doesn't\n> understand it, then we won't send it on in the response. If we're an\n> internal user, then we know about all capabilities and will request\n> them.\n> \n> Signed-off-by: brian m. carlson \n> ---\n> builtin/credential-cache--daemon.c | 2 +-\n> builtin/credential-store.c | 2 +-\n> builtin/credential.c | 6 +-\n> credential.c | 58 ++++++++++++++--\n> credential.h | 30 +++++++-\n> http.c | 10 +--\n> imap-send.c | 2 +-\n> remote-curl.c | 4 +-\n> t/t0300-credentials.sh | 107 ++++++++++++++++++++++++++++-\n> 9 files changed, 197 insertions(+), 24 deletions(-)\n> \n> diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c\n> index 3a6a750a8e..ccbcf99ac1 100644\n> --- a/builtin/credential-cache--daemon.c\n> +++ b/builtin/credential-cache--daemon.c\n> @@ -115,7 +115,7 @@ static int read_request(FILE *fh, struct credential *c,\n> \t\treturn error(\"client sent bogus timeout line: %s\", item.buf);\n> \t*timeout = atoi(p);\n> \n> -\tif (credential_read(c, fh) < 0)\n> +\tif (credential_read(c, fh, CREDENTIAL_OP_HELPER) < 0)\n> \t\treturn -1;\n> \treturn 0;\n> }\n> diff --git a/builtin/credential-store.c b/builtin/credential-store.c\n> index 4a492411bb..494c809332 100644\n> --- a/builtin/credential-store.c\n> +++ b/builtin/credential-store.c\n> @@ -205,7 +205,7 @@ int cmd_credential_store(int argc, const char **argv, const char *prefix)\n> \tif (!fns.nr)\n> \t\tdie(\"unable to set up default path; use --file\");\n> \n> -\tif (credential_read(&c, stdin) < 0)\n> +\tif (credential_read(&c, stdin, CREDENTIAL_OP_HELPER) < 0)\n> \t\tdie(\"unable to read credential\");\n> \n> \tif (!strcmp(op, \"get\"))\n> diff --git a/builtin/credential.c b/builtin/credential.c\n> index 7010752987..5123dabcf1 100644\n> --- a/builtin/credential.c\n> +++ b/builtin/credential.c\n> @@ -17,12 +17,12 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n> \t\tusage(usage_msg);\n> \top = argv[1];\n> \n> -\tif (credential_read(&c, stdin) < 0)\n> +\tif (credential_read(&c, stdin, CREDENTIAL_OP_INITIAL) < 0)\n> \t\tdie(\"unable to read credential from stdin\");\n> \n> \tif (!strcmp(op, \"fill\")) {\n> -\t\tcredential_fill(&c);\n> -\t\tcredential_write(&c, stdout);\n> +\t\tcredential_fill(&c, 0);\n> +\t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n> \t} else if (!strcmp(op, \"approve\")) {\n> \t\tcredential_approve(&c);\n> \t} else if (!strcmp(op, \"reject\")) {\n> diff --git a/credential.c b/credential.c\n> index c521822e5a..f2a26b8672 100644\n> --- a/credential.c\n> +++ b/credential.c\n> @@ -34,6 +34,11 @@ void credential_clear(struct credential *c)\n> \tcredential_init(c);\n> }\n> \n> +static void credential_set_all_capabilities(struct credential *c)\n> +{\n> +\tc->capa_authtype.request_initial = 1;\n> +}\n> +\n> int credential_match(const struct credential *want,\n> \t\t const struct credential *have, int match_password)\n> {\n> @@ -210,7 +215,39 @@ static void credential_getpass(struct credential *c)\n> \t\t\t\t\t\t PROMPT_ASKPASS);\n> }\n> \n> -int credential_read(struct credential *c, FILE *fp)\n> +static void credential_set_capability(struct credential_capability *capa, int op_type)\n> +{\n> +\tswitch (op_type) {\n> +\tcase CREDENTIAL_OP_INITIAL:\n> +\t\tcapa->request_initial = 1;\n> +\t\tbreak;\n> +\tcase CREDENTIAL_OP_HELPER:\n> +\t\tcapa->request_helper = 1;\n> +\t\tbreak;\n> +\tcase CREDENTIAL_OP_RESPONSE:\n> +\t\tcapa->response = 1;\n> +\t\tbreak;\n> +\t}\n> +}\n> +\n> +static int credential_has_capability(const struct credential_capability *capa, int op_type)\n> +{\n> +\t/*\n> +\t * We're checking here if each previous step indicated that we had the\n> +\t * capability. If it did, then we want to pass it along; conversely, if\n> +\t * it did not, we don't want to report that to our caller.\n> +\t */\n> +\tswitch (op_type) {\n> +\tcase CREDENTIAL_OP_HELPER:\n> +\t\treturn capa->request_initial;\n> +\tcase CREDENTIAL_OP_RESPONSE:\n> +\t\treturn capa->request_initial && capa->request_helper;\n> +\tdefault:\n> +\t\treturn 0;\n> +\t}\n> +}\n\nI think I'm missing the bigger picture here, so please bear with me.\n\nWhat you provide here is simply an `op_type` that indicates the phase we\nare currently in and thus allows us to check whether all of the\npreceding phases had the capability set. But to me it seems like a phase\nand the actual capability should be different things. So why is it that\nthe capability seems to be a mere boolean value instead of something\nlike a bitfield indicating whether a specific capability is supported or\nnot? Or is all of this infra really only to support a single capability,\nnamely the credential capability?\n\nI'm mostly coming from the angle of how capabilities work with remote\nhelpers. When asked, the helper will announce a set of capabilities that\nit supports, e.g. \"capabilities stateless-connect\". So from thereon the\nclient of the helper knows that it can assume \"stateless-connect\" to be\nunderstood by the helper.\n\nI would have expected capabilities to work similarly for the credential\nhelper, where it announces \"I know to handle pre-encoded credentials\".\nBut given that I have basically no clue at all for how the credential\nhelper works there may very well be good reasons why things work so\ndifferently here.\n\n> +int credential_read(struct credential *c, FILE *fp, int op_type)\n> {\n> \tstruct strbuf line = STRBUF_INIT;\n> \n> @@ -249,6 +286,8 @@ int credential_read(struct credential *c, FILE *fp)\n> \t\t\tc->path = xstrdup(value);\n> \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n> \t\t\tstrvec_push(&c->wwwauth_headers, value);\n> +\t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n> +\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n> \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n> \t\t\terrno = 0;\n> \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n> @@ -288,14 +327,18 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,\n> \tfprintf(fp, \"%s=%s\\n\", key, value);\n> }\n> \n> -void credential_write(const struct credential *c, FILE *fp)\n> +void credential_write(const struct credential *c, FILE *fp, int op_type)\n> {\n> +\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n> +\t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n> +\t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n> +\t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n> +\t}\n> \tcredential_write_item(fp, \"protocol\", c->protocol, 1);\n> \tcredential_write_item(fp, \"host\", c->host, 1);\n> \tcredential_write_item(fp, \"path\", c->path, 0);\n> \tcredential_write_item(fp, \"username\", c->username, 0);\n> \tcredential_write_item(fp, \"password\", c->password, 0);\n> -\tcredential_write_item(fp, \"credential\", c->credential, 0);\n> \tcredential_write_item(fp, \"oauth_refresh_token\", c->oauth_refresh_token, 0);\n> \tif (c->password_expiry_utc != TIME_MAX) {\n> \t\tchar *s = xstrfmt(\"%\"PRItime, c->password_expiry_utc);\n> @@ -304,7 +347,6 @@ void credential_write(const struct credential *c, FILE *fp)\n> \t}\n> \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n> \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n> -\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n> }\n> \n> static int run_credential_helper(struct credential *c,\n> @@ -327,14 +369,14 @@ static int run_credential_helper(struct credential *c,\n> \n> \tfp = xfdopen(helper.in, \"w\");\n> \tsigchain_push(SIGPIPE, SIG_IGN);\n> -\tcredential_write(c, fp);\n> +\tcredential_write(c, fp, want_output ? CREDENTIAL_OP_HELPER : CREDENTIAL_OP_RESPONSE);\n> \tfclose(fp);\n> \tsigchain_pop(SIGPIPE);\n> \n> \tif (want_output) {\n> \t\tint r;\n> \t\tfp = xfdopen(helper.out, \"r\");\n> -\t\tr = credential_read(c, fp);\n> +\t\tr = credential_read(c, fp, CREDENTIAL_OP_HELPER);\n> \t\tfclose(fp);\n> \t\tif (r < 0) {\n> \t\t\tfinish_command(&helper);\n> @@ -367,7 +409,7 @@ static int credential_do(struct credential *c, const char *helper,\n> \treturn r;\n> }\n> \n> -void credential_fill(struct credential *c)\n> +void credential_fill(struct credential *c, int all_capabilities)\n> {\n> \tint i;\n> \n> @@ -375,6 +417,8 @@ void credential_fill(struct credential *c)\n> \t\treturn;\n> \n> \tcredential_apply_config(c);\n> +\tif (all_capabilities)\n> +\t\tcredential_set_all_capabilities(c);\n> \n> \tfor (i = 0; i < c->helpers.nr; i++) {\n> \t\tcredential_do(c, c->helpers.items[i].string, \"get\");\n> diff --git a/credential.h b/credential.h\n> index 9db892cf4d..2051d04c5a 100644\n> --- a/credential.h\n> +++ b/credential.h\n> @@ -93,6 +93,25 @@\n> * -----------------------------------------------------------------------\n> */\n> \n> +/*\n> + * These values define the kind of operation we're performing and the\n> + * capabilities at each stage. The first is either an external request (via git\n> + * credential fill) or an internal request (e.g., via the HTTP) code. The\n> + * second is the call to the credential helper, and the third is the response\n> + * we're providing.\n> + *\n> + * At each stage, we will emit the capability only if the previous stage\n> + * supported it.\n> + */\n> +#define CREDENTIAL_OP_INITIAL 1\n> +#define CREDENTIAL_OP_HELPER 2\n> +#define CREDENTIAL_OP_RESPONSE 3\n\nIs there any specific reason why you're using defines instead of an enum\nhere? I think the latter would be more self-explanatory when you see\nthat functions take `enum credential_op` as input instead of an `int`.\n\nPatrick\n\n> +struct credential_capability {\n> +\tunsigned request_initial:1,\n> +\t\t request_helper:1,\n> +\t\t response:1;\n> +};\n> \n> /**\n> * This struct represents a single username/password combination\n> @@ -136,6 +155,8 @@ struct credential {\n> \t\t use_http_path:1,\n> \t\t username_from_proto:1;\n> \n> +\tstruct credential_capability capa_authtype;\n> +\n> \tchar *username;\n> \tchar *password;\n> \tchar *credential;\n> @@ -174,8 +195,11 @@ void credential_clear(struct credential *);\n> * returns, the username and password fields of the credential are\n> * guaranteed to be non-NULL. If an error occurs, the function will\n> * die().\n> + *\n> + * If all_capabilities is set, this is an internal user that is prepared\n> + * to deal with all known capabilities, and we should advertise that fact.\n> */\n> -void credential_fill(struct credential *);\n> +void credential_fill(struct credential *, int all_capabilities);\n> \n> /**\n> * Inform the credential subsystem that the provided credentials\n> @@ -198,8 +222,8 @@ void credential_approve(struct credential *);\n> */\n> void credential_reject(struct credential *);\n> \n> -int credential_read(struct credential *, FILE *);\n> -void credential_write(const struct credential *, FILE *);\n> +int credential_read(struct credential *, FILE *, int);\n> +void credential_write(const struct credential *, FILE *, int);\n> \n> /*\n> * Parse a url into a credential struct, replacing any existing contents.\n> diff --git a/http.c b/http.c\n> index 1c2200da77..4f5df6fb14 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -569,7 +569,7 @@ static void init_curl_http_auth(CURL *result)\n> \t\treturn;\n> \t}\n> \n> -\tcredential_fill(&http_auth);\n> +\tcredential_fill(&http_auth, 0);\n> \n> \tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n> \tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n> @@ -596,7 +596,7 @@ static void init_curl_proxy_auth(CURL *result)\n> {\n> \tif (proxy_auth.username) {\n> \t\tif (!proxy_auth.password)\n> -\t\t\tcredential_fill(&proxy_auth);\n> +\t\t\tcredential_fill(&proxy_auth, 0);\n> \t\tset_proxyauth_name_password(result);\n> \t}\n> \n> @@ -630,7 +630,7 @@ static int has_cert_password(void)\n> \t\tcert_auth.host = xstrdup(\"\");\n> \t\tcert_auth.username = xstrdup(\"\");\n> \t\tcert_auth.path = xstrdup(ssl_cert);\n> -\t\tcredential_fill(&cert_auth);\n> +\t\tcredential_fill(&cert_auth, 0);\n> \t}\n> \treturn 1;\n> }\n> @@ -645,7 +645,7 @@ static int has_proxy_cert_password(void)\n> \t\tproxy_cert_auth.host = xstrdup(\"\");\n> \t\tproxy_cert_auth.username = xstrdup(\"\");\n> \t\tproxy_cert_auth.path = xstrdup(http_proxy_ssl_cert);\n> -\t\tcredential_fill(&proxy_cert_auth);\n> +\t\tcredential_fill(&proxy_cert_auth, 0);\n> \t}\n> \treturn 1;\n> }\n> @@ -2191,7 +2191,7 @@ static int http_request_reauth(const char *url,\n> \t\tBUG(\"Unknown http_request target\");\n> \t}\n> \n> -\tcredential_fill(&http_auth);\n> +\tcredential_fill(&http_auth, 0);\n> \n> \treturn http_request(url, result, target, options);\n> }\n> diff --git a/imap-send.c b/imap-send.c\n> index f2e1947e63..8c89e866b6 100644\n> --- a/imap-send.c\n> +++ b/imap-send.c\n> @@ -944,7 +944,7 @@ static void server_fill_credential(struct imap_server_conf *srvc, struct credent\n> \tcred->username = xstrdup_or_null(srvc->user);\n> \tcred->password = xstrdup_or_null(srvc->pass);\n> \n> -\tcredential_fill(cred);\n> +\tcredential_fill(cred, 1);\n> \n> \tif (!srvc->user)\n> \t\tsrvc->user = xstrdup(cred->username);\n> diff --git a/remote-curl.c b/remote-curl.c\n> index e37eaa17b7..f96bda2431 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -926,7 +926,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n> \t\tdo {\n> \t\t\terr = probe_rpc(rpc, &results);\n> \t\t\tif (err == HTTP_REAUTH)\n> -\t\t\t\tcredential_fill(&http_auth);\n> +\t\t\t\tcredential_fill(&http_auth, 0);\n> \t\t} while (err == HTTP_REAUTH);\n> \t\tif (err != HTTP_OK)\n> \t\t\treturn -1;\n> @@ -1044,7 +1044,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n> \trpc->any_written = 0;\n> \terr = run_slot(slot, NULL);\n> \tif (err == HTTP_REAUTH && !large_request) {\n> -\t\tcredential_fill(&http_auth);\n> +\t\tcredential_fill(&http_auth, 0);\n> \t\tcurl_slist_free_all(headers);\n> \t\tgoto retry;\n> \t}\n> diff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\n> index 400f6bdbca..8477108b28 100755\n> --- a/t/t0300-credentials.sh\n> +++ b/t/t0300-credentials.sh\n> @@ -12,7 +12,13 @@ test_expect_success 'setup helper scripts' '\n> \tIFS==\n> \twhile read key value; do\n> \t\techo >&2 \"$whoami: $key=$value\"\n> -\t\teval \"$key=$value\"\n> +\t\tif test -z \"${key%%*\\[\\]}\"\n> +\t\tthen\n> +\t\t\tkey=${key%%\\[\\]}\n> +\t\t\teval \"$key=\\\"\\$$key $value\\\"\"\n> +\t\telse\n> +\t\t\teval \"$key=$value\"\n> +\t\tfi\n> \tdone\n> \tIFS=$OIFS\n> \tEOF\n> @@ -35,6 +41,16 @@ test_expect_success 'setup helper scripts' '\n> \ttest -z \"$pass\" || echo password=$pass\n> \tEOF\n> \n> +\twrite_script git-credential-verbatim-cred <<-\\EOF &&\n> +\tauthtype=$1; shift\n> +\tcredential=$1; shift\n> +\t. ./dump\n> +\techo capability[]=authtype\n> +\ttest -z \"${capability##*authtype*}\" || return\n> +\ttest -z \"$authtype\" || echo authtype=$authtype\n> +\ttest -z \"$credential\" || echo credential=$credential\n> +\tEOF\n> +\n> \twrite_script git-credential-verbatim-with-expiry <<-\\EOF &&\n> \tuser=$1; shift\n> \tpass=$1; shift\n> @@ -64,6 +80,26 @@ test_expect_success 'credential_fill invokes helper' '\n> \tEOF\n> '\n> \n> +test_expect_success 'credential_fill invokes helper with credential' '\n> +\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n> +\tcapability[]=authtype\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tcapability[]=authtype\n> +\tauthtype=Bearer\n> +\tcredential=token\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tverbatim-cred: get\n> +\tverbatim-cred: capability[]=authtype\n> +\tverbatim-cred: protocol=http\n> +\tverbatim-cred: host=example.com\n> +\tEOF\n> +'\n> +\n> +\n> test_expect_success 'credential_fill invokes multiple helpers' '\n> \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n> \tprotocol=http\n> @@ -83,6 +119,42 @@ test_expect_success 'credential_fill invokes multiple helpers' '\n> \tEOF\n> '\n> \n> +test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '\n> +\tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n> +\tcapability[]=authtype\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tprotocol=http\n> +\thost=example.com\n> +\tusername=foo\n> +\tpassword=bar\n> +\t--\n> +\tuseless: get\n> +\tuseless: capability[]=authtype\n> +\tuseless: protocol=http\n> +\tuseless: host=example.com\n> +\tverbatim: get\n> +\tverbatim: capability[]=authtype\n> +\tverbatim: protocol=http\n> +\tverbatim: host=example.com\n> +\tEOF\n> +'\n> +\n> +test_expect_success 'credential_fill response does not get capabilities when caller is incapable' '\n> +\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tverbatim-cred: get\n> +\tverbatim-cred: protocol=http\n> +\tverbatim-cred: host=example.com\n> +\tEOF\n> +'\n> +\n> test_expect_success 'credential_fill stops when we get a full response' '\n> \tcheck fill \"verbatim one two\" \"verbatim three four\" <<-\\EOF\n> \tprotocol=http\n> @@ -99,6 +171,25 @@ test_expect_success 'credential_fill stops when we get a full response' '\n> \tEOF\n> '\n> \n> +test_expect_success 'credential_fill thinks a credential is a full response' '\n> +\tcheck fill \"verbatim-cred Bearer token\" \"verbatim three four\" <<-\\EOF\n> +\tcapability[]=authtype\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tcapability[]=authtype\n> +\tauthtype=Bearer\n> +\tcredential=token\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tverbatim-cred: get\n> +\tverbatim-cred: capability[]=authtype\n> +\tverbatim-cred: protocol=http\n> +\tverbatim-cred: host=example.com\n> +\tEOF\n> +'\n> +\n> test_expect_success 'credential_fill continues through partial response' '\n> \tcheck fill \"verbatim one \\\"\\\"\" \"verbatim two three\" <<-\\EOF\n> \tprotocol=http\n> @@ -175,6 +266,20 @@ test_expect_success 'credential_fill passes along metadata' '\n> \tEOF\n> '\n> \n> +test_expect_success 'credential_fill produces no credential without capability' '\n> +\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tprotocol=http\n> +\thost=example.com\n> +\t--\n> +\tverbatim-cred: get\n> +\tverbatim-cred: protocol=http\n> +\tverbatim-cred: host=example.com\n> +\tEOF\n> +'\n> +\n> test_expect_success 'credential_approve calls all helpers' '\n> \tcheck approve useless \"verbatim one two\" <<-\\EOF\n> \tprotocol=http\n> \n"},{"id":"491648","messageId":"ZgPSpYVSSmQ9Muec@tanuki","threadId":"61189","inReplyTo":"20240324011301.1553072-13-sandals@crustytoothpaste.net","subject":"Re: [PATCH 12/13] strvec: implement swapping two strvecs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-03-27T08:02:45Z","receivedAt":"2024-03-27T08:02:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Mar 24, 2024 at 01:13:00AM +0000, brian m. carlson wrote:\n> In a future commit, we'll want the ability to efficiently swap the\n> contents of two strvec instances without needing to copy any data.\n> Since a strvec is simply a pointer and two sizes, swapping them is as\n> simply as copying the two pointers and sizes, so let's do that.\n> \n> We use a temporary here simply because C doesn't provide a standard\n> swapping function, unlike C++ and Rust, but a good optimizing compiler\n> will recognize this syntax and handle it appropriately using an\n> optimization pass.\n> \n> Signed-off-by: brian m. carlson \n> ---\n> strvec.c | 7 +++++++\n> strvec.h | 5 +++++\n> 2 files changed, 12 insertions(+)\n> \n> diff --git a/strvec.c b/strvec.c\n> index 178f4f3748..93006f1e63 100644\n> --- a/strvec.c\n> +++ b/strvec.c\n> @@ -106,3 +106,10 @@ const char **strvec_detach(struct strvec *array)\n> \t\treturn ret;\n> \t}\n> }\n> +\n> +void strvec_swap(struct strvec *a, struct strvec *b)\n> +{\n> +\tstruct strvec t = *a;\n> +\t*a = *b;\n> +\t*b = t;\n> +}\n\nIsn't this equivalent to `SWAP(*a, *b)`?\n\nPatrick\n"},{"id":"491709","messageId":"xmqqbk6z1iba.fsf@gitster.g","threadId":"61189","inReplyTo":"ZgPSpYVSSmQ9Muec@tanuki","subject":"Re: [PATCH 12/13] strvec: implement swapping two strvecs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-27T21:22:17Z","receivedAt":"2024-03-27T21:22:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n>> +\n>> +void strvec_swap(struct strvec *a, struct strvec *b)\n>> +{\n>> +\tstruct strvec t = *a;\n>> +\t*a = *b;\n>> +\t*b = t;\n>> +}\n>\n> Isn't this equivalent to `SWAP(*a, *b)`?\n\nYes. \"make coccicheck\" does flag this one.\n\nLet's drop this step, and tweak the 13/13 patch to make its sole\ncaller directly use SWAP() instead, perhaps like so:\n\n credential.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git c/credential.c w/credential.c\nindex 9a08efe113..5ea52ddd68 100644\n--- c/credential.c\n+++ w/credential.c\n@@ -39,7 +39,7 @@ void credential_clear(struct credential *c)\n void credential_next_state(struct credential *c)\n {\n \tstrvec_clear(&c->state_headers_to_send);\n-\tstrvec_swap(&c->state_headers, &c->state_headers_to_send);\n+\tSWAP(c->state_headers, c->state_headers_to_send);\n }\n \n void credential_clear_secrets(struct credential *c)\n\n\n"},{"id":"491710","messageId":"ZgSQr7uQdIA8oVNn@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"ZgPSn_D8luXQyZGf@tanuki","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-27T21:33:35Z","receivedAt":"2024-03-27T21:33:43Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-27 at 08:02:39, Patrick Steinhardt wrote:\n> On Sun, Mar 24, 2024 at 01:12:53AM +0000, brian m. carlson wrote:\n> > +static int credential_has_capability(const struct credential_capability *capa, int op_type)\n> > +{\n> > +\t/*\n> > +\t * We're checking here if each previous step indicated that we had the\n> > +\t * capability. If it did, then we want to pass it along; conversely, if\n> > +\t * it did not, we don't want to report that to our caller.\n> > +\t */\n> > +\tswitch (op_type) {\n> > +\tcase CREDENTIAL_OP_HELPER:\n> > +\t\treturn capa->request_initial;\n> > +\tcase CREDENTIAL_OP_RESPONSE:\n> > +\t\treturn capa->request_initial && capa->request_helper;\n> > +\tdefault:\n> > +\t\treturn 0;\n> > +\t}\n> > +}\n> \n> I think I'm missing the bigger picture here, so please bear with me.\n> \n> What you provide here is simply an `op_type` that indicates the phase we\n> are currently in and thus allows us to check whether all of the\n> preceding phases had the capability set. But to me it seems like a phase\n> and the actual capability should be different things. So why is it that\n> the capability seems to be a mere boolean value instead of something\n> like a bitfield indicating whether a specific capability is supported or\n> not? Or is all of this infra really only to support a single capability,\n> namely the credential capability?\n> \n> I'm mostly coming from the angle of how capabilities work with remote\n> helpers. When asked, the helper will announce a set of capabilities that\n> it supports, e.g. \"capabilities stateless-connect\". So from thereon the\n> client of the helper knows that it can assume \"stateless-connect\" to be\n> understood by the helper.\n> \n> I would have expected capabilities to work similarly for the credential\n> helper, where it announces \"I know to handle pre-encoded credentials\".\n> But given that I have basically no clue at all for how the credential\n> helper works there may very well be good reasons why things work so\n> differently here.\n\nLet me explain a little bit. There are two possible flows that we can\nhave for a credential request:\n\n git-credential input -> credential.c -> helper -> credential.c -> git-credential output\n\n git-http-backend -> credential.c -> helper -> credential.c -> git-http-backend\n\nLet's look at the first one (which might, say, come from Git LFS or\nanother external tool), but the second one works similarly. When we get\na request from `git credential fill`, we need to know first whether the\nrequester supports the capability. If we're using an external tool from\nlast decade, it's not going to do so.\n\nIf it _does_ support that, then we want to pass that along to the\nhelper, but if it doesn't, we don't. That's because if the caller\ndoesn't support `credential` and `authtype`, the helper might\nlegitimately want to provide a username and password (or token) instead,\nknowing that that's more likely to work.\n\nSimilarly, in the final response, we want to indicate to the external\ncaller whether the capability was in fact supported. That's useful to\nknow in case we want to pass the response back to `git credential\nstore`, and it also discloses functionality about what the credential\nhelper in use supports.\n\nWe can't assume that the helper does or doesn't support the same\ncapabilities as Git because it might come from outside Git (e.g., Git\nCredential Manager Core, or a site-specific credential helper) or it\njust might not be capable of storing or handling that kind of\ncredential. By not making the assumption that the helper is implicitly\ncapable, we allow users to continue to use very simple shell scripts as\ncredential helpers.\n\nAs to why this functionality exists, it exists to support the two new\ncapabilities in this series, `credential` and `state`. A pie in the sky\ngoal for the future is to support additional request signing\nfunctionality, so it might learn things like method, URI, and TLS\nchannel binding info, which would be an additional capability. (I might\nimplement that, or I might not.) All of those are boolean: they either\nare supported, or not. If folks in the future want to expose\nnon-boolean capabilities, I don't think that should be a problem.\n\n> > +/*\n> > + * These values define the kind of operation we're performing and the\n> > + * capabilities at each stage. The first is either an external request (via git\n> > + * credential fill) or an internal request (e.g., via the HTTP) code. The\n> > + * second is the call to the credential helper, and the third is the response\n> > + * we're providing.\n> > + *\n> > + * At each stage, we will emit the capability only if the previous stage\n> > + * supported it.\n> > + */\n> > +#define CREDENTIAL_OP_INITIAL 1\n> > +#define CREDENTIAL_OP_HELPER 2\n> > +#define CREDENTIAL_OP_RESPONSE 3\n> \n> Is there any specific reason why you're using defines instead of an enum\n> here? I think the latter would be more self-explanatory when you see\n> that functions take `enum credential_op` as input instead of an `int`.\n\nI think an enum would be a nice improvement. I'll include that in a\nreroll.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491711","messageId":"ZgSQ5o_KyqDaxz1m@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"xmqqbk6z1iba.fsf@gitster.g","subject":"Re: [PATCH 12/13] strvec: implement swapping two strvecs","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-27T21:34:30Z","receivedAt":"2024-03-27T21:34:32Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-27 at 21:22:17, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> >> +\n> >> +void strvec_swap(struct strvec *a, struct strvec *b)\n> >> +{\n> >> +\tstruct strvec t = *a;\n> >> +\t*a = *b;\n> >> +\t*b = t;\n> >> +}\n> >\n> > Isn't this equivalent to `SWAP(*a, *b)`?\n> \n> Yes. \"make coccicheck\" does flag this one.\n> \n> Let's drop this step, and tweak the 13/13 patch to make its sole\n> caller directly use SWAP() instead, perhaps like so:\n\nGreat, I'll include that in a reroll, likely this weekend.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491725","messageId":"CAGJzqsn4Lz1T=q8NHi7kHHRdYVXKCMPGVBGzkkvZA=P1O9HtUw@mail.gmail.com","threadId":"61189","inReplyTo":"20240324011301.1553072-14-sandals@crustytoothpaste.net","subject":"Re: [PATCH 13/13] credential: add support for multistage credential rounds","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-03-28T08:00:00Z","receivedAt":"2024-03-28T08:00:13Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Sun, 24 Mar 2024 at 01:13, brian m. carlson\n wrote:\n>\n> Over HTTP, NTLM and Kerberos require two rounds of authentication on the\n> client side. It's possible that there are custom authentication schemes\n> that also implement this same approach. Since these are tricky schemes\n> to implement and the HTTP library in use may not always handle them\n> gracefully on all systems, it would be helpful to allow the credential\n> helper to implement them instead for increased portability and\n> robustness.\n\nIs this design sufficiently flexible for OAuth DPoP (RFC 9449), or at\nleast to make it work in future?\n\nOAuth 2.0 Demonstrating Proof of Possession describes \"a mechanism for\nsender-constraining OAuth 2.0 tokens via a proof-of-possession\nmechanism on the application level. This mechanism allows for the\ndetection of replay attacks with access and refresh tokens.\"\nhttps://www.rfc-editor.org/rfc/rfc9449.html\n\nPopular hosts GitHub, GitLab, Bitbucket and Gitea already support\nOAuth. OAuth DPoP \"provides a general defense in depth against the\nimpact of unanticipated token leakage\". Motivated by a 2022 GitHub\nattack involving stolen tokens\n(https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/),\nsome hosts are already experimenting with it.\nhttps://lore.kernel.org/git/20230128142827.17397-1-mirth.hickford@gmail.com/\n\nIn particular, the http request has to include both Authorization and\nDPoP headers https://www.rfc-editor.org/rfc/rfc9449.html#name-the-dpop-authentication-sch.\nThe latter depends on timestamp and a server-optional challenge in a\nDPoP-Nonce header.\nhttps://www.rfc-editor.org/rfc/rfc9449.html#name-resource-server-provided-no.\n\n\nOn Sun, 24 Mar 2024 at 01:13, brian m. carlson\n wrote:\n>\n> Over HTTP, NTLM and Kerberos require two rounds of authentication on the\n> client side. It's possible that there are custom authentication schemes\n> that also implement this same approach. Since these are tricky schemes\n> to implement and the HTTP library in use may not always handle them\n> gracefully on all systems, it would be helpful to allow the credential\n> helper to implement them instead for increased portability and\n> robustness.\n>\n> To allow this to happen, add a boolean flag, continue, that indicates\n> that instead of failing when we get a 401, we should retry another round\n> of authentication. However, this necessitates some changes in our\n> current credential code so that we can make this work.\n>\n> Keep the state[] headers between iterations, but only use them to send\n> to the helper and only consider the new ones we read from the credential\n> helper to be valid on subsequent iterations. That avoids us passing\n> stale data when we finally approve or reject the credential. Similarly,\n> clear the multistage and wwwauth[] values appropriately so that we\n> don't pass stale data or think we're trying a multiround response when\n> we're not. Remove the credential values so that we can actually fill a\n> second time with new responses.\n>\n> Limit the number of iterations of reauthentication we do to 3. This\n> means that if there's a problem, we'll terminate with an error message\n> instead of retrying indefinitely and not informing the user (and\n> possibly conducting a DoS on the server).\n>\n> In our tests, handle creating multiple response output files from our\n> helper so we can verify that each of the messages sent is correct.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/git-credential.txt | 16 +++++-\n> builtin/credential.c | 1 +\n> credential.c | 32 ++++++++++--\n> credential.h | 27 +++++++++-\n> http.c | 59 +++++++++++----------\n> t/t5563-simple-http-auth.sh | 89 ++++++++++++++++++++++++++++++--\n> 6 files changed, 187 insertions(+), 37 deletions(-)\n>\n> diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\n> index 6b7e017066..160dee5c6a 100644\n> --- a/Documentation/git-credential.txt\n> +++ b/Documentation/git-credential.txt\n> @@ -207,6 +207,19 @@ provided on input.\n> This value should not be sent unless the appropriate capability (see below) is\n> provided on input.\n>\n> +`continue`::\n> + This is a boolean value, which, if enabled, indicates that this\n> + authentication is a non-final part of a multistage authentication step. This\n> + is common in protocols such as NTLM and Kerberos, where two rounds of client\n> + authentication are required, and setting this flag allows the credential\n> + helper to implement the multistage authentication step. This flag should\n> + only be sent if a further stage is required; that is, if another round of\n> + authentication is expected.\n> ++\n> +This value should not be sent unless the appropriate capability (see below) is\n> +provided on input. This attribute is 'one-way' from a credential helper to\n> +pass information to Git (or other programs invoking `git credential`).\n> +\n> `wwwauth[]`::\n>\n> When an HTTP response is received by Git that includes one or more\n> @@ -225,7 +238,8 @@ to pass additional information to credential helpers.\n> +\n> There are two currently supported capabilities. The first is `authtype`, which\n> indicates that the `authtype` and `credential` values are understood. The\n> -second is `state`, which indicates that the `state[]` value is understood.\n> +second is `state`, which indicates that the `state[]` and `continue` values are\n> +understood.\n>\n> It is not obligatory to use the additional features just because the capability\n> is supported, but they should not be provided without this capability.\n> diff --git a/builtin/credential.c b/builtin/credential.c\n> index 5123dabcf1..f14d1b5ed6 100644\n> --- a/builtin/credential.c\n> +++ b/builtin/credential.c\n> @@ -22,6 +22,7 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n>\n> if (!strcmp(op, \"fill\")) {\n> credential_fill(&c, 0);\n> + credential_next_state(&c);\n> credential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n> } else if (!strcmp(op, \"approve\")) {\n> credential_approve(&c);\n> diff --git a/credential.c b/credential.c\n> index 0ca7c12895..9a08efe113 100644\n> --- a/credential.c\n> +++ b/credential.c\n> @@ -31,10 +31,23 @@ void credential_clear(struct credential *c)\n> string_list_clear(&c->helpers, 0);\n> strvec_clear(&c->wwwauth_headers);\n> strvec_clear(&c->state_headers);\n> + strvec_clear(&c->state_headers_to_send);\n>\n> credential_init(c);\n> }\n>\n> +void credential_next_state(struct credential *c)\n> +{\n> + strvec_clear(&c->state_headers_to_send);\n> + strvec_swap(&c->state_headers, &c->state_headers_to_send);\n> +}\n> +\n> +void credential_clear_secrets(struct credential *c)\n> +{\n> + FREE_AND_NULL(c->password);\n> + FREE_AND_NULL(c->credential);\n> +}\n> +\n> static void credential_set_all_capabilities(struct credential *c)\n> {\n> c->capa_authtype.request_initial = 1;\n> @@ -295,6 +308,8 @@ int credential_read(struct credential *c, FILE *fp, int op_type)\n> credential_set_capability(&c->capa_authtype, op_type);\n> else if (!strcmp(value, \"state\"))\n> credential_set_capability(&c->capa_state, op_type);\n> + } else if (!strcmp(key, \"continue\")) {\n> + c->multistage = !!git_config_bool(\"continue\", value);\n> } else if (!strcmp(key, \"password_expiry_utc\")) {\n> errno = 0;\n> c->password_expiry_utc = parse_timestamp(value, NULL, 10);\n> @@ -359,8 +374,10 @@ void credential_write(const struct credential *c, FILE *fp, int op_type)\n> for (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n> credential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n> if (credential_has_capability(&c->capa_state, op_type)) {\n> - for (size_t i = 0; i < c->state_headers.nr; i++)\n> - credential_write_item(fp, \"state[]\", c->state_headers.v[i], 0);\n> + if (c->multistage)\n> + credential_write_item(fp, \"continue\", \"1\", 0);\n> + for (size_t i = 0; i < c->state_headers_to_send.nr; i++)\n> + credential_write_item(fp, \"state[]\", c->state_headers_to_send.v[i], 0);\n> }\n> }\n>\n> @@ -431,6 +448,9 @@ void credential_fill(struct credential *c, int all_capabilities)\n> if ((c->username && c->password) || c->credential)\n> return;\n>\n> + credential_next_state(c);\n> + c->multistage = 0;\n> +\n> credential_apply_config(c);\n> if (all_capabilities)\n> credential_set_all_capabilities(c);\n> @@ -443,8 +463,10 @@ void credential_fill(struct credential *c, int all_capabilities)\n> /* Reset expiry to maintain consistency */\n> c->password_expiry_utc = TIME_MAX;\n> }\n> - if ((c->username && c->password) || c->credential)\n> + if ((c->username && c->password) || c->credential) {\n> + strvec_clear(&c->wwwauth_headers);\n> return;\n> + }\n> if (c->quit)\n> die(\"credential helper '%s' told us to quit\",\n> c->helpers.items[i].string);\n> @@ -464,6 +486,8 @@ void credential_approve(struct credential *c)\n> if (((!c->username || !c->password) && !c->credential) || c->password_expiry_utc < time(NULL))\n> return;\n>\n> + credential_next_state(c);\n> +\n> credential_apply_config(c);\n>\n> for (i = 0; i < c->helpers.nr; i++)\n> @@ -475,6 +499,8 @@ void credential_reject(struct credential *c)\n> {\n> int i;\n>\n> + credential_next_state(c);\n> +\n> credential_apply_config(c);\n>\n> for (i = 0; i < c->helpers.nr; i++)\n> diff --git a/credential.h b/credential.h\n> index e2021455fe..adb1fc370a 100644\n> --- a/credential.h\n> +++ b/credential.h\n> @@ -143,10 +143,15 @@ struct credential {\n> struct strvec wwwauth_headers;\n>\n> /**\n> - * A `strvec` of state headers from credential helpers.\n> + * A `strvec` of state headers received from credential helpers.\n> */\n> struct strvec state_headers;\n>\n> + /**\n> + * A `strvec` of state headers to send to credential helpers.\n> + */\n> + struct strvec state_headers_to_send;\n> +\n> /**\n> * Internal use only. Keeps track of if we previously matched against a\n> * WWW-Authenticate header line in order to re-fold future continuation\n> @@ -156,6 +161,7 @@ struct credential {\n>\n> unsigned approved:1,\n> configured:1,\n> + multistage: 1,\n> quit:1,\n> use_http_path:1,\n> username_from_proto:1;\n> @@ -184,6 +190,7 @@ struct credential {\n> .password_expiry_utc = TIME_MAX, \\\n> .wwwauth_headers = STRVEC_INIT, \\\n> .state_headers = STRVEC_INIT, \\\n> + .state_headers_to_send = STRVEC_INIT, \\\n> }\n>\n> /* Initialize a credential structure, setting all fields to empty. */\n> @@ -229,6 +236,24 @@ void credential_approve(struct credential *);\n> */\n> void credential_reject(struct credential *);\n>\n> +/**\n> + * Clear the secrets in this credential, but leave other data intact.\n> + *\n> + * This is useful for resetting credentials in preparation for a subsequent\n> + * stage of filling.\n> + */\n> +void credential_clear_secrets(struct credential *c);\n> +\n> +/**\n> + * Prepares the credential for the next iteration of the helper protocol by\n> + * updating the state headers to send with the ones read by the last iteration\n> + * of the protocol.\n> + *\n> + * Except for internal callers, this should be called exactly once between\n> + * reading credentials with `credential_fill` and writing them.\n> + */\n> +void credential_next_state(struct credential *c);\n> +\n> int credential_read(struct credential *, FILE *, int);\n> void credential_write(const struct credential *, FILE *, int);\n>\n> diff --git a/http.c b/http.c\n> index f98c520924..9d373c6460 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -1781,6 +1781,10 @@ static int handle_curl_result(struct slot_results *results)\n> else if (results->http_code == 401) {\n> if ((http_auth.username && http_auth.password) ||\\\n> (http_auth.authtype && http_auth.credential)) {\n> + if (http_auth.multistage) {\n> + credential_clear_secrets(&http_auth);\n> + return HTTP_REAUTH;\n> + }\n> credential_reject(&http_auth);\n> return HTTP_NOAUTH;\n> } else {\n> @@ -2178,6 +2182,7 @@ static int http_request_reauth(const char *url,\n> void *result, int target,\n> struct http_get_options *options)\n> {\n> + int i = 3;\n> int ret = http_request(url, result, target, options);\n>\n> if (ret != HTTP_OK && ret != HTTP_REAUTH)\n> @@ -2191,35 +2196,35 @@ static int http_request_reauth(const char *url,\n> }\n> }\n>\n> - if (ret != HTTP_REAUTH)\n> - return ret;\n> + while (ret == HTTP_REAUTH && --i) {\n> + /*\n> + * The previous request may have put cruft into our output stream; we\n> + * should clear it out before making our next request.\n> + */\n> + switch (target) {\n> + case HTTP_REQUEST_STRBUF:\n> + strbuf_reset(result);\n> + break;\n> + case HTTP_REQUEST_FILE:\n> + if (fflush(result)) {\n> + error_errno(\"unable to flush a file\");\n> + return HTTP_START_FAILED;\n> + }\n> + rewind(result);\n> + if (ftruncate(fileno(result), 0) < 0) {\n> + error_errno(\"unable to truncate a file\");\n> + return HTTP_START_FAILED;\n> + }\n> + break;\n> + default:\n> + BUG(\"Unknown http_request target\");\n> + }\n>\n> - /*\n> - * The previous request may have put cruft into our output stream; we\n> - * should clear it out before making our next request.\n> - */\n> - switch (target) {\n> - case HTTP_REQUEST_STRBUF:\n> - strbuf_reset(result);\n> - break;\n> - case HTTP_REQUEST_FILE:\n> - if (fflush(result)) {\n> - error_errno(\"unable to flush a file\");\n> - return HTTP_START_FAILED;\n> - }\n> - rewind(result);\n> - if (ftruncate(fileno(result), 0) < 0) {\n> - error_errno(\"unable to truncate a file\");\n> - return HTTP_START_FAILED;\n> - }\n> - break;\n> - default:\n> - BUG(\"Unknown http_request target\");\n> + credential_fill(&http_auth, 1);\n> +\n> + ret = http_request(url, result, target, options);\n> }\n> -\n> - credential_fill(&http_auth, 1);\n> -\n> - return http_request(url, result, target, options);\n> + return ret;\n> }\n>\n> int http_get_strbuf(const char *url,\n> diff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\n> index 515185ae00..5d5caa3f58 100755\n> --- a/t/t5563-simple-http-auth.sh\n> +++ b/t/t5563-simple-http-auth.sh\n> @@ -21,9 +21,17 @@ test_expect_success 'setup_credential_helper' '\n> CREDENTIAL_HELPER=\"$TRASH_DIRECTORY/bin/git-credential-test-helper\" &&\n> write_script \"$CREDENTIAL_HELPER\" <<-\\EOF\n> cmd=$1\n> - teefile=$cmd-query.cred\n> + teefile=$cmd-query-temp.cred\n> catfile=$cmd-reply.cred\n> sed -n -e \"/^$/q\" -e \"p\" >>$teefile\n> + state=$(sed -ne \"s/^state\\[\\]=helper://p\" \"$teefile\")\n> + if test -z \"$state\"\n> + then\n> + mv \"$teefile\" \"$cmd-query.cred\"\n> + else\n> + mv \"$teefile\" \"$cmd-query-$state.cred\"\n> + catfile=\"$cmd-reply-$state.cred\"\n> + fi\n> if test \"$cmd\" = \"get\"\n> then\n> cat $catfile\n> @@ -32,13 +40,15 @@ test_expect_success 'setup_credential_helper' '\n> '\n>\n> set_credential_reply () {\n> - cat >\"$TRASH_DIRECTORY/$1-reply.cred\"\n> + local suffix=\"$(test -n \"$2\" && echo \"-$2\")\"\n> + cat >\"$TRASH_DIRECTORY/$1-reply$suffix.cred\"\n> }\n>\n> expect_credential_query () {\n> - cat >\"$TRASH_DIRECTORY/$1-expect.cred\" &&\n> - test_cmp \"$TRASH_DIRECTORY/$1-expect.cred\" \\\n> - \"$TRASH_DIRECTORY/$1-query.cred\"\n> + local suffix=\"$(test -n \"$2\" && echo \"-$2\")\"\n> + cat >\"$TRASH_DIRECTORY/$1-expect$suffix.cred\" &&\n> + test_cmp \"$TRASH_DIRECTORY/$1-expect$suffix.cred\" \\\n> + \"$TRASH_DIRECTORY/$1-query$suffix.cred\"\n> }\n>\n> per_test_cleanup () {\n> @@ -479,4 +489,73 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n> EOF\n> '\n>\n> +test_expect_success 'access using three-legged auth' '\n> + test_when_finished \"per_test_cleanup\" &&\n> +\n> + set_credential_reply get <<-EOF &&\n> + capability[]=authtype\n> + capability[]=state\n> + authtype=Multistage\n> + credential=YS1naXQtdG9rZW4=\n> + state[]=helper:foobar\n> + continue=1\n> + EOF\n> +\n> + set_credential_reply get foobar <<-EOF &&\n> + capability[]=authtype\n> + capability[]=state\n> + authtype=Multistage\n> + credential=YW5vdGhlci10b2tlbg==\n> + state[]=helper:bazquux\n> + EOF\n> +\n> + cat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n> + id=1 creds=Multistage YS1naXQtdG9rZW4=\n> + id=2 creds=Multistage YW5vdGhlci10b2tlbg==\n> + EOF\n> +\n> + CHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n> +\n> + cat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n> + id=1 status=401 response=WWW-Authenticate: Multistage challenge=\"456\"\n> + id=1 status=401 response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n> + id=2 status=200\n> + id=default response=WWW-Authenticate: Multistage challenge=\"123\"\n> + id=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n> + EOF\n> +\n> + test_config_global credential.helper test-helper &&\n> + git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n> +\n> + expect_credential_query get <<-EOF &&\n> + capability[]=authtype\n> + capability[]=state\n> + protocol=http\n> + host=$HTTPD_DEST\n> + wwwauth[]=Multistage challenge=\"123\"\n> + wwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n> + EOF\n> +\n> + expect_credential_query get foobar <<-EOF &&\n> + capability[]=authtype\n> + capability[]=state\n> + authtype=Multistage\n> + protocol=http\n> + host=$HTTPD_DEST\n> + wwwauth[]=Multistage challenge=\"456\"\n> + wwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n> + state[]=helper:foobar\n> + EOF\n> +\n> + expect_credential_query store bazquux <<-EOF\n> + capability[]=authtype\n> + capability[]=state\n> + authtype=Multistage\n> + credential=YW5vdGhlci10b2tlbg==\n> + protocol=http\n> + host=$HTTPD_DEST\n> + state[]=helper:bazquux\n> + EOF\n> +'\n> +\n> test_done\n"},{"id":"491742","messageId":"20240328102053.GA890906@coredump.intra.peff.net","threadId":"61189","inReplyTo":"20240324011301.1553072-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-03-28T10:20:53Z","receivedAt":"2024-03-28T10:20:54Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Mar 24, 2024 at 01:12:53AM +0000, brian m. carlson wrote:\n\n> @@ -35,6 +41,16 @@ test_expect_success 'setup helper scripts' '\n> \ttest -z \"$pass\" || echo password=$pass\n> \tEOF\n> \n> +\twrite_script git-credential-verbatim-cred <<-\\EOF &&\n> +\tauthtype=$1; shift\n> +\tcredential=$1; shift\n> +\t. ./dump\n> +\techo capability[]=authtype\n> +\ttest -z \"${capability##*authtype*}\" || return\n> +\ttest -z \"$authtype\" || echo authtype=$authtype\n> +\ttest -z \"$credential\" || echo credential=$credential\n> +\tEOF\n\nI think this \"|| return\" needs to be \"|| exit 0\" or similar. The Windows\nCI jobs fail with:\n\n --- a/expect-stderr\n +++ b/stderr\n @@ -2,3 +2,4 @@ verbatim-cred: get\n verbatim-cred: capability[]=authtype\n verbatim-cred: protocol=http\n verbatim-cred: host=example.com\n +D:\\a\\git\\git\\t\\trash directory.t0300-credentials\\git-credential-verbatim-cred: line 10: return: can only `return' from a function or sourced script\n\n(actually if you count the line numbers, I think this particular case is\nthe similar \"|| return\" added to the script later, but both should be\nfixed).\n\nIt doesn't show up elsewhere because only bash complains, but not dash.\nEven running the test script with bash isn't enough, because\nwrite_script uses $SHELL_PATH under the hood. But building with \"make\nSHELL_PATH=/bin/bash test\" shows the problem on other platforms.\n\n-Peff\n"},{"id":"491755","messageId":"xmqq7chmwczx.fsf@gitster.g","threadId":"61189","inReplyTo":"20240328102053.GA890906@coredump.intra.peff.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-28T16:13:38Z","receivedAt":"2024-03-28T16:13:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> It doesn't show up elsewhere because only bash complains, but not dash.\n> Even running the test script with bash isn't enough, because\n> write_script uses $SHELL_PATH under the hood. But building with \"make\n> SHELL_PATH=/bin/bash test\" shows the problem on other platforms.\n\nCan we sneak it in to the GitHub Actions CI, I wonder, so that we\ncan catch tests that only fail with bash. Would this be sufficient,\nor can we just export it without using $use_bash to place it on the\ncommand line of make?\n\n ci/run-build-and-tests.sh | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git i/ci/run-build-and-tests.sh w/ci/run-build-and-tests.sh\nindex c192bd613c..8fb1114bc5 100755\n--- i/ci/run-build-and-tests.sh\n+++ w/ci/run-build-and-tests.sh\n@@ -11,6 +11,7 @@ windows*) cmd //c mklink //j t\\\\.prove \"$(cygpath -aw \"$cache_dir/.prove\")\";;\n esac\n \n run_tests=t\n+use_bash=\n \n case \"$jobname\" in\n linux-gcc)\n@@ -30,6 +31,7 @@ linux-TEST-vars)\n \texport GIT_TEST_NO_WRITE_REV_INDEX=1\n \texport GIT_TEST_CHECKOUT_WORKERS=2\n \texport GIT_TEST_PACK_USE_BITMAP_BOUNDARY_TRAVERSAL=1\n+\tuse_bash=/bin/bash\n \t;;\n linux-clang)\n \texport GIT_TEST_DEFAULT_HASH=sha1\n@@ -51,7 +53,8 @@ esac\n group Build make\n if test -n \"$run_tests\"\n then\n-\tgroup \"Run tests\" make test ||\n+\tgroup \"Run tests\" \\\n+\t\tmake ${use_bash:+SHELL_PATH=\"$use_bash\"} test ||\n \thandle_failed_tests\n \tgroup \"Run unit tests\" \\\n \t\tmake DEFAULT_UNIT_TEST_TARGET=unit-tests-prove unit-tests\n\n"},{"id":"491757","messageId":"20240328162923.GA1403492@coredump.intra.peff.net","threadId":"61189","inReplyTo":"xmqq7chmwczx.fsf@gitster.g","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-03-28T16:29:23Z","receivedAt":"2024-03-28T16:29:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Mar 28, 2024 at 09:13:38AM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > It doesn't show up elsewhere because only bash complains, but not dash.\n> > Even running the test script with bash isn't enough, because\n> > write_script uses $SHELL_PATH under the hood. But building with \"make\n> > SHELL_PATH=/bin/bash test\" shows the problem on other platforms.\n> \n> Can we sneak it in to the GitHub Actions CI, I wonder, so that we\n> can catch tests that only fail with bash. Would this be sufficient,\n> or can we just export it without using $use_bash to place it on the\n> command line of make?\n\nI think the sneaking has already been done, because Windows CI uses bash\n(which is after all how I noticed this). I'm not sure if using bash more\nplaces would be helpful. On the one hand, there are enough _other_\ndifferences in Windows that it is not always immediately obvious that\nthe shell is the culprit. On the other hand, I would probably forget\nthat linux-gcc is the special one with bash, and just end up reading the\ntest output anyway. So I dunno.\n\nAs to your question, yes, I think you could just export SHELL_PATH; our\nMakefile uses \"ifndef\".\n\n-Peff\n"},{"id":"491766","messageId":"xmqqfrwauv3s.fsf@gitster.g","threadId":"61189","inReplyTo":"20240328162923.GA1403492@coredump.intra.peff.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-03-28T17:25:27Z","receivedAt":"2024-03-28T17:25:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> I think the sneaking has already been done, because Windows CI uses bash\n> (which is after all how I noticed this). I'm not sure if using bash more\n> places would be helpful.\n\nYeah, you are absolutely right. Let's not pile another difference\nin the linux-TEST-vars that is already a kitchen sink.\n\nThanks for a quick dose of sanity.\n\n"},{"id":"491785","messageId":"ZgXevTHIkZoC3vnj@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"20240328102053.GA890906@coredump.intra.peff.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-28T21:18:53Z","receivedAt":"2024-03-28T21:18:55Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-28 at 10:20:53, Jeff King wrote:\n> I think this \"|| return\" needs to be \"|| exit 0\" or similar. The Windows\n> CI jobs fail with:\n> \n> --- a/expect-stderr\n> +++ b/stderr\n> @@ -2,3 +2,4 @@ verbatim-cred: get\n> verbatim-cred: capability[]=authtype\n> verbatim-cred: protocol=http\n> verbatim-cred: host=example.com\n> +D:\\a\\git\\git\\t\\trash directory.t0300-credentials\\git-credential-verbatim-cred: line 10: return: can only `return' from a function or sourced script\n> \n> (actually if you count the line numbers, I think this particular case is\n> the similar \"|| return\" added to the script later, but both should be\n> fixed).\n> \n> It doesn't show up elsewhere because only bash complains, but not dash.\n> Even running the test script with bash isn't enough, because\n> write_script uses $SHELL_PATH under the hood. But building with \"make\n> SHELL_PATH=/bin/bash test\" shows the problem on other platforms.\n\nI'll definitely make that change. I run Debian, and I've left the\ndefault dash as /bin/sh because it's faster, so I didn't notice.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491791","messageId":"ZgXmxvRv0rNBKk5K@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"CAGJzqsn4Lz1T=q8NHi7kHHRdYVXKCMPGVBGzkkvZA=P1O9HtUw@mail.gmail.com","subject":"Re: [PATCH 13/13] credential: add support for multistage credential rounds","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-28T21:53:10Z","receivedAt":"2024-03-28T21:53:18Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-28 at 08:00:00, M Hickford wrote:\n> Is this design sufficiently flexible for OAuth DPoP (RFC 9449), or at\n> least to make it work in future?\n> \n> OAuth 2.0 Demonstrating Proof of Possession describes \"a mechanism for\n> sender-constraining OAuth 2.0 tokens via a proof-of-possession\n> mechanism on the application level. This mechanism allows for the\n> detection of replay attacks with access and refresh tokens.\"\n> https://www.rfc-editor.org/rfc/rfc9449.html\n> \n> Popular hosts GitHub, GitLab, Bitbucket and Gitea already support\n> OAuth. OAuth DPoP \"provides a general defense in depth against the\n> impact of unanticipated token leakage\". Motivated by a 2022 GitHub\n> attack involving stolen tokens\n> (https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/),\n> some hosts are already experimenting with it.\n> https://lore.kernel.org/git/20230128142827.17397-1-mirth.hickford@gmail.com/\n> \n> In particular, the http request has to include both Authorization and\n> DPoP headers https://www.rfc-editor.org/rfc/rfc9449.html#name-the-dpop-authentication-sch.\n> The latter depends on timestamp and a server-optional challenge in a\n> DPoP-Nonce header.\n> https://www.rfc-editor.org/rfc/rfc9449.html#name-resource-server-provided-no.\n\nIt will likely be sufficient with further extensions. Right now, we\ndon't have a way to provide DPoP headers or send nonces to the client.\nHowever, there's no reason we cannot provide that functionality in the\nfuture via additional key/value pairs, in which case this design should\nbe fine.\n\nThis would have been sufficient if the OAuth working group had not added\nextra additional headers that other authentication mechanisms would have\nsimply put (and, honestly, should have been put) in the WWW-Authenticate\nand Authorization headers, but alas, we can't change it now.\n\nSince I think this gets us at least part of the way where we need to be,\nI think we should be able to keep it for now and implement the extra\nsupport for DPoP later.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491870","messageId":"CAGJzqsnNmz1P1bqOztiC5=-SXDD=j_oaCuPLcPf_NV4c7VNjMA@mail.gmail.com","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-03-30T08:00:00Z","receivedAt":"2024-03-30T08:02:03Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Sun, 24 Mar 2024 at 01:13, brian m. carlson\n wrote:\n>\n> Right now, HTTP authentication in Git is mostly limited to approaches\n> that require a username and password or are Kerberos (GSSAPI). In\n> addition, we effectively require that libcurl (or, for other software,\n> such as Git LFS, using the credential helper, that HTTP library) knows\n> how to implement the authentication scheme.\n>\n> However, this poses two sets of problems. First, some sites, such as\n> Azure DevOps, want to use Bearer authentication, which we don't support.\n\nGit's lack of support for Bearer auth is also a frustration for OAuth\n-- OAuth tokens are intended as Bearer tokens. As a workaround,\npopular Git hosts support Basic auth with an OAuth token as password.\nFor certain hosts, this requires a magic username such as 'oauth2'\n(GitLab) or 'x-token-auth' (Bitbucket). This is problematic if the\nuser has multiple accounts, or a different username stored in\ncredential.username or the remote URL.\n\n> This is implemented using `http.extraHeader`, which is not a secure way\n> to store credentials, since our credential helper protocol does not\n> support this functionality.\n\nConfig key http.extraHeader is particularly unsuitable to hold\nshort-lived Bearer tokens. For example, GitLab's OAuth tokens expire\nafter 2 hours.\n\n>\n> In addition, other tools using the credential helper protocol do not\n> support the variety of authentication mechanisms that Git does.\n> Specifically, making NTLM function in a useful way on Windows is\n> nontrivial and requires extensive integration and testing with C code,\n> and because of this difficulty and the fact that NTLM uses cryptography\n> known to be insecure since 1995, there is often little interest in\n> implementing this support outside of libcurl. However, it would be\n> helpful if people who want to use it can still use it.\n>\n> This series introduces new functionality to the credential helper\n> protocol that allows helpers to produce credentials for arbitrary HTTP\n> authentication schemes using the `authtype` and `credential`[0] fields.\n\nExciting! I drafted a patch to make git-credential-oauth generate\nBearer credentials\nhttps://github.com/hickford/git-credential-oauth/pull/51 . Tested\ntogether with your patch, it works for host bitbucket.org. Though of\ncourse it isn't practical while storage helpers (such as\ncredential-cache, credential-wincred, credential-libsecret and\ncredential-osxkeychain) are unable to store the new attributes -- the\nuser has to complete the OAuth flow for every authentication.\n\n> This allows a suitable credential helper to send Bearer credentials or\n> any other standard or custom authentication scheme. (It may be able to\n> be extended to other functionality in the future, such as\n> git-send-email, to implement custom SASL functionality, and due care has\n> been taken to make the protocol adequately flexible for that purpose.)\n>\n> In addition, the protocol is also expanded to include per-helper state\n> and multi-legged authentication (the former is effectively required for\n> the latter). The per-helper state can be useful to help credential\n> helpers identify where the credential is stored, or any other\n> information necessary. Because NTLM and Negotiate (Kerberos/wrapped\n> NTLM) require two rounds of authentication, the multi-legged\n> authentication support along with per-helper state allows the helper to\n> support these authentication methods without Git or other clients having\n> to be aware of how they work. (This would also be useful for SASL, as\n> mentioned above.)\n>\n> This series introduces a capability mechanism to announce this\n> functionality, which allows a helper to provide a username and password\n> on older versions of Git while supporting more advanced functionality on\n> newer versions. (This is especially important on Azure DevOps, where\n> NTLM uses a username and password but Basic or Bearer can use a personal\n> access token.) It is also designed such that extremely simple\n> credential helpers, such as the shell one-liner in the Git FAQ that\n> reads from the environment, don't accidentally claim to support\n> functionality they don't offer.\n\n\n\n\n>\n> In addition, there is documentation for the expanded protocol, although\n> none of the built-in helpers have been updated (that will be a future\n> series for those for which it's possible).\n>\n> My personal interest here is getting credentials out of config files\n> with `http.extraHeader` (which a future series will produce a warning\n> for) and also allowing Git LFS to support Digest and NTLM with a\n> suitable credential helper. Git LFS used to support NTLM using custom\n> code (because the Go standard library does not), but it was found to be\n> broken in lots of ways on Windows, and nobody with a Windows system\n> wanted to fix it or support it, so we removed it. However, there are\n> still some people who do want to use it, so allowing them to use a\n> custom credential helper they maintain themselves seems like the best\n> way forward. Despite the advantages of this series for Azure DevOps, I\n> have no personal or professional stake in their product; my only\n> interest is the general one in whether their users can securely store\n> credentials. I believe the changes here are of general advantage to the\n> Git userbase in a variety of ways such that the goal of this series\n> should be uncontroversial.\n>\n> Feedback on any portion of this series is of course welcome.\n>\n> [0] A name different from `password` was explicitly chosen to avoid\n> confusion from less capable protocol helpers so that they don't\n> accidentally send invalid data. This does have the downside that\n> credential helpers must learn a new field to not log, but that should be\n> generally easy to fix in most cases.\n>\n> brian m. carlson (13):\n> credential: add an authtype field\n> remote-curl: reset headers on new request\n> http: use new headers for each object request\n> credential: add a field for pre-encoded credentials\n> credential: gate new fields on capability\n> docs: indicate new credential protocol fields\n> http: add support for authtype and credential\n> credential: add an argument to keep state\n> credential: enable state capability\n> docs: set a limit on credential line length\n> t5563: refactor for multi-stage authentication\n> strvec: implement swapping two strvecs\n> credential: add support for multistage credential rounds\n>\n> Documentation/git-credential.txt | 59 +++++-\n> builtin/credential-cache--daemon.c | 2 +-\n> builtin/credential-store.c | 2 +-\n> builtin/credential.c | 7 +-\n> credential.c | 114 ++++++++++-\n> credential.h | 69 ++++++-\n> http.c | 128 +++++++-----\n> http.h | 5 +\n> imap-send.c | 2 +-\n> remote-curl.c | 14 +-\n> strvec.c | 7 +\n> strvec.h | 5 +\n> t/lib-httpd/nph-custom-auth.sh | 17 +-\n> t/t0300-credentials.sh | 136 ++++++++++++-\n> t/t5563-simple-http-auth.sh | 308 +++++++++++++++++++++++++----\n> 15 files changed, 760 insertions(+), 115 deletions(-)\n>\n"},{"id":"491873","messageId":"CAGJzqsnyPhR2t7yuVXGQtCoeeUM+yxOR9hDm1y2Y6hcSK6QMFQ@mail.gmail.com","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-03-30T08:16:54Z","receivedAt":"2024-03-30T08:17:34Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Sun, 24 Mar 2024 at 01:13, brian m. carlson\n wrote:\n>\n> Right now, HTTP authentication in Git is mostly limited to approaches\n> that require a username and password or are Kerberos (GSSAPI). In\n> addition, we effectively require that libcurl (or, for other software,\n> such as Git LFS, using the credential helper, that HTTP library) knows\n> how to implement the authentication scheme.\n>\n> However, this poses two sets of problems. First, some sites, such as\n> Azure DevOps, want to use Bearer authentication, which we don't support.\n> This is implemented using `http.extraHeader`, which is not a secure way\n> to store credentials, since our credential helper protocol does not\n> support this functionality.\n>\n> In addition, other tools using the credential helper protocol do not\n> support the variety of authentication mechanisms that Git does.\n> Specifically, making NTLM function in a useful way on Windows is\n> nontrivial and requires extensive integration and testing with C code,\n> and because of this difficulty and the fact that NTLM uses cryptography\n> known to be insecure since 1995, there is often little interest in\n> implementing this support outside of libcurl. However, it would be\n> helpful if people who want to use it can still use it.\n>\n> This series introduces new functionality to the credential helper\n> protocol that allows helpers to produce credentials for arbitrary HTTP\n> authentication schemes using the `authtype` and `credential`[0] fields.\n> This allows a suitable credential helper to send Bearer credentials or\n> any other standard or custom authentication scheme. (It may be able to\n> be extended to other functionality in the future, such as\n> git-send-email, to implement custom SASL functionality, and due care has\n> been taken to make the protocol adequately flexible for that purpose.)\n>\n> In addition, the protocol is also expanded to include per-helper state\n> and multi-legged authentication (the former is effectively required for\n> the latter). The per-helper state can be useful to help credential\n> helpers identify where the credential is stored, or any other\n> information necessary. Because NTLM and Negotiate (Kerberos/wrapped\n> NTLM) require two rounds of authentication, the multi-legged\n> authentication support along with per-helper state allows the helper to\n> support these authentication methods without Git or other clients having\n> to be aware of how they work. (This would also be useful for SASL, as\n> mentioned above.)\n>\n> This series introduces a capability mechanism to announce this\n> functionality, which allows a helper to provide a username and password\n> on older versions of Git while supporting more advanced functionality on\n> newer versions. (This is especially important on Azure DevOps, where\n> NTLM uses a username and password but Basic or Bearer can use a personal\n> access token.) It is also designed such that extremely simple\n> credential helpers, such as the shell one-liner in the Git FAQ that\n> reads from the environment, don't accidentally claim to support\n> functionality they don't offer.\n>\n> In addition, there is documentation for the expanded protocol, although\n> none of the built-in helpers have been updated (that will be a future\n> series for those for which it's possible).\n\nIf you've time, I think it would be instructive to add (conditional)\ntests to libcredential and support in at least one storage helper,\nsuch as credential-cache.\n\nThis would clarify the behaviour in subtle cases involving expiry and\nmultiple helpers.\n\n>\n> My personal interest here is getting credentials out of config files\n> with `http.extraHeader` (which a future series will produce a warning\n> for) and also allowing Git LFS to support Digest and NTLM with a\n> suitable credential helper. Git LFS used to support NTLM using custom\n> code (because the Go standard library does not), but it was found to be\n> broken in lots of ways on Windows, and nobody with a Windows system\n> wanted to fix it or support it, so we removed it. However, there are\n> still some people who do want to use it, so allowing them to use a\n> custom credential helper they maintain themselves seems like the best\n> way forward. Despite the advantages of this series for Azure DevOps, I\n> have no personal or professional stake in their product; my only\n> interest is the general one in whether their users can securely store\n> credentials. I believe the changes here are of general advantage to the\n> Git userbase in a variety of ways such that the goal of this series\n> should be uncontroversial.\n>\n> Feedback on any portion of this series is of course welcome.\n>\n> [0] A name different from `password` was explicitly chosen to avoid\n> confusion from less capable protocol helpers so that they don't\n> accidentally send invalid data. This does have the downside that\n> credential helpers must learn a new field to not log, but that should be\n> generally easy to fix in most cases.\n>\n> brian m. carlson (13):\n> credential: add an authtype field\n> remote-curl: reset headers on new request\n> http: use new headers for each object request\n> credential: add a field for pre-encoded credentials\n> credential: gate new fields on capability\n> docs: indicate new credential protocol fields\n> http: add support for authtype and credential\n> credential: add an argument to keep state\n> credential: enable state capability\n> docs: set a limit on credential line length\n> t5563: refactor for multi-stage authentication\n> strvec: implement swapping two strvecs\n> credential: add support for multistage credential rounds\n>\n> Documentation/git-credential.txt | 59 +++++-\n> builtin/credential-cache--daemon.c | 2 +-\n> builtin/credential-store.c | 2 +-\n> builtin/credential.c | 7 +-\n> credential.c | 114 ++++++++++-\n> credential.h | 69 ++++++-\n> http.c | 128 +++++++-----\n> http.h | 5 +\n> imap-send.c | 2 +-\n> remote-curl.c | 14 +-\n> strvec.c | 7 +\n> strvec.h | 5 +\n> t/lib-httpd/nph-custom-auth.sh | 17 +-\n> t/t0300-credentials.sh | 136 ++++++++++++-\n> t/t5563-simple-http-auth.sh | 308 +++++++++++++++++++++++++----\n> 15 files changed, 760 insertions(+), 115 deletions(-)\n>\n"},{"id":"491879","messageId":"CAGJzqs=QkoAU300V-6-w7VqQgG01rY-h5N43WkTy+sJgGV5W2Q@mail.gmail.com","threadId":"61189","inReplyTo":"ZgIKqJaE79-Erzif@tapette.crustytoothpaste.net","subject":"Re: [PATCH 06/13] docs: indicate new credential protocol fields","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-03-30T13:00:00Z","receivedAt":"2024-03-30T13:00:18Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Mon, 25 Mar 2024 at 23:37, brian m. carlson\n wrote:\n>\n> On 2024-03-25 at 23:16:09, M Hickford wrote:\n> > > +`authtype`::\n> > > + This indicates that the authentication scheme in question should be used.\n> > > + Common values for HTTP and HTTPS include `basic`, `digest`, and `ntlm`,\n> > > + although the latter two are insecure and should not be used. If `credential`\n> > > + is used, this may be set to an arbitrary string suitable for the protocol in\n> > > + question (usually HTTP).\n> >\n> > How about adding 'bearer' to this list? Popular hosts Bitbucket\n> > https://bitbucket.org and Gitea/Forgejo (such as https://codeberg.org)\n> > support Bearer auth with OAuth tokens.\n>\n> Sure, I can do that.\n>\n> > > ++\n> > > +This value should not be sent unless the appropriate capability (see below) is\n> > > +provided on input.\n> > > +\n> > > +`credential`::\n> > > + The pre-encoded credential, suitable for the protocol in question (usually\n> > > + HTTP). If this key is sent, `authtype` is mandatory, and `username` and\n> > > + `password` are not used.\n> >\n> > A credential protocol attribute named 'credential' is confusing. How\n> > about 'authorization' since it determines the HTTP Authorization\n> > header? This detail is surely worth mentioning too.\n\nWould it be accurate to add \"For HTTP, Git concatenates the authtype\nand credential attributes to determine the Authorization header\"?\n\n>\n> I don't want this to be very specific to HTTP, so I don't think that's a\n> great name. As I mentioned in the cover letter, I might well extend\n> this to IMAP and SMTP for our mail handling in the future, and that name\n> wouldn't work well there.\n\nGood point, you've dissuaded me against 'authorization'.\n\n>\n> I named it `credential` because, well, it's the credential that's used\n> in the protocol. I feel like saying that the field represents \"the\n> authorization\" sounds unnatural. It's not wrong, per se, but it sounds\n> confusing.\n\nWe already use 'credential' to describe the whole collection of\nattributes, as in \"The credential is split into a set of named\nattributes\".\n\n>\n> I'm open to other ideas if you or others have them, but between these\n> two, I think I'd prefer to stick with `credential`.\n\nIdeas anyone?\n\n\n> --\n> brian m. carlson (they/them or he/him)\n> Toronto, Ontario, CA\n"},{"id":"491928","messageId":"ZgnZFTvNyInFGn7Q@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"CAGJzqs=QkoAU300V-6-w7VqQgG01rY-h5N43WkTy+sJgGV5W2Q@mail.gmail.com","subject":"Re: [PATCH 06/13] docs: indicate new credential protocol fields","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-03-31T21:43:49Z","receivedAt":"2024-03-31T21:43:59Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-03-30 at 13:00:00, M Hickford wrote:\n> Would it be accurate to add \"For HTTP, Git concatenates the authtype\n> and credential attributes to determine the Authorization header\"?\n\nYes, I think that would be accurate.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"491985","messageId":"CAGJzqsnAby-p=XH4HnTObB=Eq7n12D_uDhdmSOcTBwLzktN60w@mail.gmail.com","threadId":"61189","inReplyTo":"ZgXmxvRv0rNBKk5K@tapette.crustytoothpaste.net","subject":"Re: [PATCH 13/13] credential: add support for multistage credential rounds","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-04-01T20:51:30Z","receivedAt":"2024-04-01T20:52:20Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Thu, 28 Mar 2024 at 21:53, brian m. carlson\n wrote:\n>\n> On 2024-03-28 at 08:00:00, M Hickford wrote:\n> > Is this design sufficiently flexible for OAuth DPoP (RFC 9449), or at\n> > least to make it work in future?\n> >\n> > OAuth 2.0 Demonstrating Proof of Possession describes \"a mechanism for\n> > sender-constraining OAuth 2.0 tokens via a proof-of-possession\n> > mechanism on the application level. This mechanism allows for the\n> > detection of replay attacks with access and refresh tokens.\"\n> > https://www.rfc-editor.org/rfc/rfc9449.html\n> >\n> > Popular hosts GitHub, GitLab, Bitbucket and Gitea already support\n> > OAuth. OAuth DPoP \"provides a general defense in depth against the\n> > impact of unanticipated token leakage\". Motivated by a 2022 GitHub\n> > attack involving stolen tokens\n> > (https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/),\n> > some hosts are already experimenting with it.\n> > https://lore.kernel.org/git/20230128142827.17397-1-mirth.hickford@gmail.com/\n> >\n> > In particular, the http request has to include both Authorization and\n> > DPoP headers https://www.rfc-editor.org/rfc/rfc9449.html#name-the-dpop-authentication-sch.\n> > The latter depends on timestamp and a server-optional challenge in a\n> > DPoP-Nonce header.\n> > https://www.rfc-editor.org/rfc/rfc9449.html#name-resource-server-provided-no.\n>\n> It will likely be sufficient with further extensions. Right now, we\n> don't have a way to provide DPoP headers or send nonces to the client.\n> However, there's no reason we cannot provide that functionality in the\n> future via additional key/value pairs, in which case this design should\n> be fine.\n>\n> This would have been sufficient if the OAuth working group had not added\n> extra additional headers that other authentication mechanisms would have\n> simply put (and, honestly, should have been put) in the WWW-Authenticate\n> and Authorization headers, but alas, we can't change it now.\n>\n> Since I think this gets us at least part of the way where we need to be,\n> I think we should be able to keep it for now and implement the extra\n> support for DPoP later.\n\nFantastic, thanks for considering this.\n\nI look forward to sharing a test Git remote with OAuth DPoP when I can\nfigure out how.\n\n> --\n> brian m. carlson (they/them or he/him)\n> Toronto, Ontario, CA\n"},{"id":"491986","messageId":"CAKcCxfBi55OoR=Li23o6p+kkCNaZHr_8NDmVXhmfvYzR_G2Qfg@mail.gmail.com","threadId":"61189","inReplyTo":"20240324011301.1553072-9-sandals@crustytoothpaste.net","subject":"Re: [PATCH 08/13] credential: add an argument to keep state","fromName":"mirth hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-04-01T21:05:28Z","receivedAt":"2024-04-01T21:05:52Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Sun, Mar 24, 2024 at 1:13 AM brian m. carlson\n wrote:\n>\n> Until now, our credential code has mostly deal with usernames and\n> passwords and we've let libcurl deal with the variant of authentication\n> to be used. However, now that we have the credential value, the\n> credential helper can take control of the authentication, so the value\n> provided might be something that's generated, such as a Digest hash\n> value.\n>\n> In such a case, it would be helpful for a credential helper that gets an\n> erase or store command to be able to keep track of an identifier for the\n> original secret that went into the computation. Furthermore, some types\n> of authentication, such as NTLM and Kerberos, actually need two round\n> trips to authenticate, which will require that the credential helper\n> keep some state.\n>\n> In order to allow for these use cases and others, allow storing state in\n> a field called \"state[]\". This value is passed back to the credential\n> helper that created it, which avoids confusion caused by parsing values\n> from different helpers.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/git-credential.txt | 29 ++++++++++++++++++-----------\n> credential.c | 20 +++++++++++++++++---\n> credential.h | 7 +++++++\n> t/t0300-credentials.sh | 29 +++++++++++++++++++++++++++++\n> 4 files changed, 71 insertions(+), 14 deletions(-)\n>\n> diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\n> index f3ed3a82fa..ef30c89c00 100644\n> --- a/Documentation/git-credential.txt\n> +++ b/Documentation/git-credential.txt\n> @@ -196,6 +196,15 @@ provided on input.\n> This value should not be sent unless the appropriate capability (see below) is\n> provided on input.\n>\n> +`state[]`::\n> + This value provides an opaque state that will be passed back to this helper\n> + if it is called again. Each different credential helper may specify this\n> + once. The value should include a prefix unique to the credential helper and\n> + should ignore values that don't match its prefix.\n\nDoes Git ever populate state[] in 'store' or 'erase' requests, or\nonly 'get' requests? It might be worthwhile to spell this out.\n\nThis seems somewhat different to other multi-valued attributes,\nparticularly the \"set at most one value\" constraint. As an\nalternative, how about a single-valued attribute stored independently\nfor each helper (vector length equal to the number of configured\nhelpers)? Then in repeat requests send the \"nth state to the nth\nhelper\". This would avoid the complexity of the prefix mechanism.\n\n> ++\n> +This value should not be sent unless the appropriate capability (see below) is\n> +provided on input.\n> +\n> `wwwauth[]`::\n>\n> When an HTTP response is received by Git that includes one or more\n> @@ -208,18 +217,16 @@ they appear in the HTTP response. This attribute is 'one-way' from Git\n> to pass additional information to credential helpers.\n>\n> `capability[]`::\n> - This signals that the caller supports the capability in question.\n> - This can be used to provide better, more specific data as part of the\n> - protocol.\n> + This signals that Git, or the helper, as appropriate, supports the\n> + capability in question. This can be used to provide better, more specific\n> + data as part of the protocol.\n> +\n> -The only capability currently supported is `authtype`, which indicates that the\n> -`authtype` and `credential` values are understood. It is not obligatory to use\n> -these values in such a case, but they should not be provided without this\n> -capability.\n> -+\n> -Callers of `git credential` and credential helpers should emit the\n> -capabilities they support unconditionally, and Git will gracefully\n> -handle passing them on.\n> +There are two currently supported capabilities. The first is `authtype`, which\n> +indicates that the `authtype` and `credential` values are understood. The\n> +second is `state`, which indicates that the `state[]` value is understood.\n>\n> +\n> +It is not obligatory to use the additional features just because the capability\n> +is supported, but they should not be provided without this capability.\n>\n> Unrecognised attributes and capabilities are silently discarded.\n>\n> diff --git a/credential.c b/credential.c\n> index f2a26b8672..0cd7dd2a00 100644\n> --- a/credential.c\n> +++ b/credential.c\n> @@ -30,6 +30,7 @@ void credential_clear(struct credential *c)\n> free(c->authtype);\n> string_list_clear(&c->helpers, 0);\n> strvec_clear(&c->wwwauth_headers);\n> + strvec_clear(&c->state_headers);\n>\n> credential_init(c);\n> }\n> @@ -286,8 +287,13 @@ int credential_read(struct credential *c, FILE *fp, int op_type)\n> c->path = xstrdup(value);\n> } else if (!strcmp(key, \"wwwauth[]\")) {\n> strvec_push(&c->wwwauth_headers, value);\n> - } else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n> - credential_set_capability(&c->capa_authtype, op_type);\n> + } else if (!strcmp(key, \"state[]\")) {\n> + strvec_push(&c->state_headers, value);\n> + } else if (!strcmp(key, \"capability[]\")) {\n> + if (!strcmp(value, \"authtype\"))\n> + credential_set_capability(&c->capa_authtype, op_type);\n> + else if (!strcmp(value, \"state\"))\n> + credential_set_capability(&c->capa_state, op_type);\n> } else if (!strcmp(key, \"password_expiry_utc\")) {\n> errno = 0;\n> c->password_expiry_utc = parse_timestamp(value, NULL, 10);\n> @@ -329,8 +335,12 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,\n>\n> void credential_write(const struct credential *c, FILE *fp, int op_type)\n> {\n> - if (credential_has_capability(&c->capa_authtype, op_type)) {\n> + if (credential_has_capability(&c->capa_authtype, op_type))\n> credential_write_item(fp, \"capability[]\", \"authtype\", 0);\n> + if (credential_has_capability(&c->capa_state, op_type))\n> + credential_write_item(fp, \"capability[]\", \"state\", 0);\n> +\n> + if (credential_has_capability(&c->capa_authtype, op_type)) {\n> credential_write_item(fp, \"authtype\", c->authtype, 0);\n> credential_write_item(fp, \"credential\", c->credential, 0);\n> }\n> @@ -347,6 +357,10 @@ void credential_write(const struct credential *c, FILE *fp, int op_type)\n> }\n> for (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n> credential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n> + if (credential_has_capability(&c->capa_state, op_type)) {\n> + for (size_t i = 0; i < c->state_headers.nr; i++)\n> + credential_write_item(fp, \"state[]\", c->state_headers.v[i], 0);\n> + }\n> }\n>\n> static int run_credential_helper(struct credential *c,\n> diff --git a/credential.h b/credential.h\n> index 2051d04c5a..e2021455fe 100644\n> --- a/credential.h\n> +++ b/credential.h\n> @@ -142,6 +142,11 @@ struct credential {\n> */\n> struct strvec wwwauth_headers;\n>\n> + /**\n> + * A `strvec` of state headers from credential helpers.\n> + */\n> + struct strvec state_headers;\n> +\n>\n> /**\n> * Internal use only. Keeps track of if we previously matched against a\n> * WWW-Authenticate header line in order to re-fold future continuation\n> @@ -156,6 +161,7 @@ struct credential {\n> username_from_proto:1;\n>\n> struct credential_capability capa_authtype;\n> + struct credential_capability capa_state;\n>\n> char *username;\n> char *password;\n> @@ -177,6 +183,7 @@ struct credential {\n> .helpers = STRING_LIST_INIT_DUP, \\\n> .password_expiry_utc = TIME_MAX, \\\n> .wwwauth_headers = STRVEC_INIT, \\\n> + .state_headers = STRVEC_INIT, \\\n> }\n>\n> /* Initialize a credential structure, setting all fields to empty. */\n> diff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\n> index 8477108b28..aa56e0dc84 100755\n> --- a/t/t0300-credentials.sh\n> +++ b/t/t0300-credentials.sh\n> @@ -46,9 +46,12 @@ test_expect_success 'setup helper scripts' '\n> credential=$1; shift\n> . ./dump\n> echo capability[]=authtype\n> + echo capability[]=state\n> test -z \"${capability##*authtype*}\" || return\n> test -z \"$authtype\" || echo authtype=$authtype\n> test -z \"$credential\" || echo credential=$credential\n> + test -z \"${capability##*state*}\" || return\n> + echo state[]=verbatim-cred:foo\n> EOF\n>\n> write_script git-credential-verbatim-with-expiry <<-\\EOF &&\n> @@ -99,6 +102,29 @@ test_expect_success 'credential_fill invokes helper with credential' '\n> EOF\n> '\n>\n> +test_expect_success 'credential_fill invokes helper with credential and state' '\n> + check fill \"verbatim-cred Bearer token\" <<-\\EOF\n> + capability[]=authtype\n> + capability[]=state\n> + protocol=http\n> + host=example.com\n> + --\n> + capability[]=authtype\n> + capability[]=state\n> + authtype=Bearer\n> + credential=token\n> + protocol=http\n> + host=example.com\n> + state[]=verbatim-cred:foo\n> + --\n> + verbatim-cred: get\n> + verbatim-cred: capability[]=authtype\n> + verbatim-cred: capability[]=state\n> + verbatim-cred: protocol=http\n> + verbatim-cred: host=example.com\n> + EOF\n> +'\n> +\n>\n> test_expect_success 'credential_fill invokes multiple helpers' '\n> check fill useless \"verbatim foo bar\" <<-\\EOF\n> @@ -122,6 +148,7 @@ test_expect_success 'credential_fill invokes multiple helpers' '\n> test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '\n> check fill useless \"verbatim foo bar\" <<-\\EOF\n> capability[]=authtype\n> + capability[]=state\n> protocol=http\n> host=example.com\n> --\n> @@ -132,10 +159,12 @@ test_expect_success 'credential_fill response does not get capabilities when hel\n> --\n> useless: get\n> useless: capability[]=authtype\n> + useless: capability[]=state\n> useless: protocol=http\n> useless: host=example.com\n> verbatim: get\n> verbatim: capability[]=authtype\n> + verbatim: capability[]=state\n> verbatim: protocol=http\n> verbatim: host=example.com\n> EOF\n"},{"id":"491999","messageId":"Zgsxqj_gkt041VCN@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"CAKcCxfBi55OoR=Li23o6p+kkCNaZHr_8NDmVXhmfvYzR_G2Qfg@mail.gmail.com","subject":"Re: [PATCH 08/13] credential: add an argument to keep state","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-01T22:14:02Z","receivedAt":"2024-04-01T22:14:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-04-01 at 21:05:28, mirth hickford wrote:\n> On Sun, Mar 24, 2024 at 1:13 AM brian m. carlson\n> wrote:\n> >\n> > Until now, our credential code has mostly deal with usernames and\n> > passwords and we've let libcurl deal with the variant of authentication\n> > to be used. However, now that we have the credential value, the\n> > credential helper can take control of the authentication, so the value\n> > provided might be something that's generated, such as a Digest hash\n> > value.\n> >\n> > In such a case, it would be helpful for a credential helper that gets an\n> > erase or store command to be able to keep track of an identifier for the\n> > original secret that went into the computation. Furthermore, some types\n> > of authentication, such as NTLM and Kerberos, actually need two round\n> > trips to authenticate, which will require that the credential helper\n> > keep some state.\n> >\n> > In order to allow for these use cases and others, allow storing state in\n> > a field called \"state[]\". This value is passed back to the credential\n> > helper that created it, which avoids confusion caused by parsing values\n> > from different helpers.\n> >\n> > Signed-off-by: brian m. carlson \n> > ---\n> > Documentation/git-credential.txt | 29 ++++++++++++++++++-----------\n> > credential.c | 20 +++++++++++++++++---\n> > credential.h | 7 +++++++\n> > t/t0300-credentials.sh | 29 +++++++++++++++++++++++++++++\n> > 4 files changed, 71 insertions(+), 14 deletions(-)\n> >\n> > diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\n> > index f3ed3a82fa..ef30c89c00 100644\n> > --- a/Documentation/git-credential.txt\n> > +++ b/Documentation/git-credential.txt\n> > @@ -196,6 +196,15 @@ provided on input.\n> > This value should not be sent unless the appropriate capability (see below) is\n> > provided on input.\n> >\n> > +`state[]`::\n> > + This value provides an opaque state that will be passed back to this helper\n> > + if it is called again. Each different credential helper may specify this\n> > + once. The value should include a prefix unique to the credential helper and\n> > + should ignore values that don't match its prefix.\n> \n> Does Git ever populate state[] in 'store' or 'erase' requests, or\n> only 'get' requests? It might be worthwhile to spell this out.\n\nYes, it's populated with whatever the last state value was from `get`.\n\n> This seems somewhat different to other multi-valued attributes,\n> particularly the \"set at most one value\" constraint. As an\n> alternative, how about a single-valued attribute stored independently\n> for each helper (vector length equal to the number of configured\n> helpers)? Then in repeat requests send the \"nth state to the nth\n> helper\". This would avoid the complexity of the prefix mechanism.\n\nI originally tried that approach, but if you have external callers of\n`git credential` (like Git LFS), that doesn't work, since you need to\nmake two separate calls: one (with `get`) to fetch the credentials that\nreturns multiple state values, and one (with `store` or `erase`) that\nsends the data back to accept or reject the credentials. Since there's\nno internal state in Git between the two calls, it's not possible to\nonly send certain data to certain helpers.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"492035","messageId":"ZgvYLxfNwBcOB_s1@tanuki","threadId":"61189","inReplyTo":"ZgSQr7uQdIA8oVNn@tapette.crustytoothpaste.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-04-02T10:04:31Z","receivedAt":"2024-04-02T10:04:39Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Mar 27, 2024 at 09:33:35PM +0000, brian m. carlson wrote:\n> On 2024-03-27 at 08:02:39, Patrick Steinhardt wrote:\n> > On Sun, Mar 24, 2024 at 01:12:53AM +0000, brian m. carlson wrote:\n> > > +static int credential_has_capability(const struct credential_capability *capa, int op_type)\n> > > +{\n> > > +\t/*\n> > > +\t * We're checking here if each previous step indicated that we had the\n> > > +\t * capability. If it did, then we want to pass it along; conversely, if\n> > > +\t * it did not, we don't want to report that to our caller.\n> > > +\t */\n> > > +\tswitch (op_type) {\n> > > +\tcase CREDENTIAL_OP_HELPER:\n> > > +\t\treturn capa->request_initial;\n> > > +\tcase CREDENTIAL_OP_RESPONSE:\n> > > +\t\treturn capa->request_initial && capa->request_helper;\n> > > +\tdefault:\n> > > +\t\treturn 0;\n> > > +\t}\n> > > +}\n> > \n> > I think I'm missing the bigger picture here, so please bear with me.\n> > \n> > What you provide here is simply an `op_type` that indicates the phase we\n> > are currently in and thus allows us to check whether all of the\n> > preceding phases had the capability set. But to me it seems like a phase\n> > and the actual capability should be different things. So why is it that\n> > the capability seems to be a mere boolean value instead of something\n> > like a bitfield indicating whether a specific capability is supported or\n> > not? Or is all of this infra really only to support a single capability,\n> > namely the credential capability?\n> > \n> > I'm mostly coming from the angle of how capabilities work with remote\n> > helpers. When asked, the helper will announce a set of capabilities that\n> > it supports, e.g. \"capabilities stateless-connect\". So from thereon the\n> > client of the helper knows that it can assume \"stateless-connect\" to be\n> > understood by the helper.\n> > \n> > I would have expected capabilities to work similarly for the credential\n> > helper, where it announces \"I know to handle pre-encoded credentials\".\n> > But given that I have basically no clue at all for how the credential\n> > helper works there may very well be good reasons why things work so\n> > differently here.\n> \n> Let me explain a little bit. There are two possible flows that we can\n> have for a credential request:\n> \n> git-credential input -> credential.c -> helper -> credential.c -> git-credential output\n> \n> git-http-backend -> credential.c -> helper -> credential.c -> git-http-backend\n> \n> Let's look at the first one (which might, say, come from Git LFS or\n> another external tool), but the second one works similarly. When we get\n> a request from `git credential fill`, we need to know first whether the\n> requester supports the capability. If we're using an external tool from\n> last decade, it's not going to do so.\n> \n> If it _does_ support that, then we want to pass that along to the\n> helper, but if it doesn't, we don't. That's because if the caller\n> doesn't support `credential` and `authtype`, the helper might\n> legitimately want to provide a username and password (or token) instead,\n> knowing that that's more likely to work.\n> \n> Similarly, in the final response, we want to indicate to the external\n> caller whether the capability was in fact supported. That's useful to\n> know in case we want to pass the response back to `git credential\n> store`, and it also discloses functionality about what the credential\n> helper in use supports.\n> \n> We can't assume that the helper does or doesn't support the same\n> capabilities as Git because it might come from outside Git (e.g., Git\n> Credential Manager Core, or a site-specific credential helper) or it\n> just might not be capable of storing or handling that kind of\n> credential. By not making the assumption that the helper is implicitly\n> capable, we allow users to continue to use very simple shell scripts as\n> credential helpers.\n\nThe intent of this is quite clear to me, but thanks for re-explaining\nthe bigger picture :)\n\n> As to why this functionality exists, it exists to support the two new\n> capabilities in this series, `credential` and `state`. A pie in the sky\n> goal for the future is to support additional request signing\n> functionality, so it might learn things like method, URI, and TLS\n> channel binding info, which would be an additional capability. (I might\n> implement that, or I might not.) All of those are boolean: they either\n> are supported, or not. If folks in the future want to expose\n> non-boolean capabilities, I don't think that should be a problem.\n\nI think you misunderstood my confusion. I didn't meant to say that there\nshould be non-boolean capabilities. I was rather missing the picture of\nhow exactly you can advertise multiple capabilities with the infra that\ncurrently exists, and why the infra supports per-phase capabilities.\n\nBasically, what I would have expected is a protocol where both Git and\nthe credential helper initially did a single \"handshake\" that also\nannounces capabilities. So something like:\n\n HELPER: capability foobar\n HELPER: capability barfoo\n GIT: capability foobar\n\nGit would only acknowledge capabilities that it both understands and\nthat have been announced by the helper. So at the end of this both have\nagreed on a single capability \"foobar\".\n\nThis is roughly how the remote helper capability subsystem works. What\nthis patch is introducing seems quite a bit more complicated than that\nthough because we have \"staged\" capabilities. I assume there is good\nreason for this complexity, but I didn't yet manage to figure out the\nreasoning behind it.\n\nTo ask more specifically: why would one side ever announce a capability\nin phase 1, but not in phase 2? Is the reason that capabilities are in\nfact tied to credentials?\n\nPatrick\n\n> > > +/*\n> > > + * These values define the kind of operation we're performing and the\n> > > + * capabilities at each stage. The first is either an external request (via git\n> > > + * credential fill) or an internal request (e.g., via the HTTP) code. The\n> > > + * second is the call to the credential helper, and the third is the response\n> > > + * we're providing.\n> > > + *\n> > > + * At each stage, we will emit the capability only if the previous stage\n> > > + * supported it.\n> > > + */\n> > > +#define CREDENTIAL_OP_INITIAL 1\n> > > +#define CREDENTIAL_OP_HELPER 2\n> > > +#define CREDENTIAL_OP_RESPONSE 3\n> > \n> > Is there any specific reason why you're using defines instead of an enum\n> > here? I think the latter would be more self-explanatory when you see\n> > that functions take `enum credential_op` as input instead of an `int`.\n> \n> I think an enum would be a nice improvement. I'll include that in a\n> reroll.\n> -- \n> brian m. carlson (they/them or he/him)\n> Toronto, Ontario, CA\n\n\n"},{"id":"492122","messageId":"20240402222619.2212650-1-calvinwan@google.com","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"Calvin Wan","fromEmail":"calvinwan@google.com","sentAt":"2024-04-02T22:26:19Z","receivedAt":"2024-04-02T22:26:24Z","isPatch":true,"sender":{"key":"calvinwan@google.com","avatar":"https://avatars.githubusercontent.com/u/92547554?v=4"},"body":"Hi Brian,\n\nWhile I personally do not know the specifics of how Git authentication\nworks at Google, I am passing this series along to the team that does\nown Git authentication (adding Jackson to this reply).\n\n\"brian m. carlson\" writes:\n> Right now, HTTP authentication in Git is mostly limited to approaches\n> that require a username and password or are Kerberos (GSSAPI). In\n> addition, we effectively require that libcurl (or, for other software,\n> such as Git LFS, using the credential helper, that HTTP library) knows\n> how to implement the authentication scheme.\n> \n> However, this poses two sets of problems. First, some sites, such as\n> Azure DevOps, want to use Bearer authentication, which we don't support.\n> This is implemented using `http.extraHeader`, which is not a secure way\n> to store credentials, since our credential helper protocol does not\n> support this functionality.\n\nMy first thought was if using `http.extraHeader` is insecure as you\nclaim and we use that internally, then how do we ensure that it is\nsecure? Or are you claiming that using `http.extraHeader` out of the box\nwithout an additional security layer is insecure?\n\nThanks,\nCalvin\n"},{"id":"492193","messageId":"Zg323pR6UYthTakT@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"ZgvYLxfNwBcOB_s1@tanuki","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-04T00:39:58Z","receivedAt":"2024-04-04T00:40:07Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-04-02 at 10:04:31, Patrick Steinhardt wrote:\n> The intent of this is quite clear to me, but thanks for re-explaining\n> the bigger picture :)\n\nSorry I misunderstood what you were getting it.\n\n> I think you misunderstood my confusion. I didn't meant to say that there\n> should be non-boolean capabilities. I was rather missing the picture of\n> how exactly you can advertise multiple capabilities with the infra that\n> currently exists, and why the infra supports per-phase capabilities.\n> \n> Basically, what I would have expected is a protocol where both Git and\n> the credential helper initially did a single \"handshake\" that also\n> announces capabilities. So something like:\n> \n> HELPER: capability foobar\n> HELPER: capability barfoo\n> GIT: capability foobar\n> \n> Git would only acknowledge capabilities that it both understands and\n> that have been announced by the helper. So at the end of this both have\n> agreed on a single capability \"foobar\".\n> \n> This is roughly how the remote helper capability subsystem works. What\n> this patch is introducing seems quite a bit more complicated than that\n> though because we have \"staged\" capabilities. I assume there is good\n> reason for this complexity, but I didn't yet manage to figure out the\n> reasoning behind it.\n> \n> To ask more specifically: why would one side ever announce a capability\n> in phase 1, but not in phase 2? Is the reason that capabilities are in\n> fact tied to credentials?\n\nMore that they're tied to the credential helper. For example, say we\nhave helpers A and B, in that order. B is incapable, but both A and Git\nunderstand the new authtype capability. If we announce the capability\nas in this series, we can get a new credential using the authtype\ncapability if A is willing to provide something to us, but we can't if A\nhas no credentials for us and B wants to provide them for us.\n\nThis would also be true if we used your proposal of negotiation, but\nbecause we have external callers (e.g., Git LFS) who may invoke `git\ncredential fill`, which would be a separate process from `git credential\ncapability`, we'd still have to have some way to tell `git credential\nfill` what capabilities the external caller supported.\n\nThe per-phase capabilities are such that we don't request functionality\nthat our callers can't use. For example, if our external caller (phase\n1) doesn't support the `authtype` credential, then we don't pass it to\nthe helper (phase 2), since the external caller might not be able to use\nthe result if we do. If the external caller (phase 1) _does_ support\nit, but the helper does not (phase 2), then we won't return the\ncapability as the result of `git credential fill` (phase 3), so our\nexternal caller will know that this isn't supported. As a practical\nmatter, that doesn't provide a great deal of useful information to the\ncaller at the moment, but it definitely could in the future (say, if we\nhad a capability for a certain form of data encoding).\n\nAll of this is also true for internal (e.g., git-http-backend) callers,\nexcept that phase 1 has all the capabilities we know about automatically\nset, and phase 3 stores the data in the internal structure we'll use for\nthe `store` and `erase` calls.\n\nI do, however, think some way to query capabilities more generically\nwould be helpful, so I'll see if I can add such changes into a v2. I\nthink we still need the current approach to make the use case I\nmentioned work, though.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"492195","messageId":"Zg38BLxLe193zYss@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"20240402222619.2212650-1-calvinwan@google.com","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-04T01:01:56Z","receivedAt":"2024-04-04T01:01:58Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-04-02 at 22:26:19, Calvin Wan wrote:\n> Hi Brian,\n> \n> While I personally do not know the specifics of how Git authentication\n> works at Google, I am passing this series along to the team that does\n> own Git authentication (adding Jackson to this reply).\n> \n> \"brian m. carlson\" writes:\n> > Right now, HTTP authentication in Git is mostly limited to approaches\n> > that require a username and password or are Kerberos (GSSAPI). In\n> > addition, we effectively require that libcurl (or, for other software,\n> > such as Git LFS, using the credential helper, that HTTP library) knows\n> > how to implement the authentication scheme.\n> > \n> > However, this poses two sets of problems. First, some sites, such as\n> > Azure DevOps, want to use Bearer authentication, which we don't support.\n> > This is implemented using `http.extraHeader`, which is not a secure way\n> > to store credentials, since our credential helper protocol does not\n> > support this functionality.\n> \n> My first thought was if using `http.extraHeader` is insecure as you\n> claim and we use that internally, then how do we ensure that it is\n> secure? Or are you claiming that using `http.extraHeader` out of the box\n> without an additional security layer is insecure?\n\nStoring plaintext credentials on disk is just not a good idea, and it's\nnot a secure way to store them. This is why `.netrc` is a less than\ngreat idea, but for Git, it's also possible to have a shared repository\nwhere information in `.git/config` can leak, and sometimes people also\njust expose Git repositories accidentally over HTTP (say, on websites)\nand leak all of their config. Sometimes people put `http.extraheader`\nin `~/.gitconfig` and then check it into their public dotfiles, and then\npush it to GitHub, for example.\n\nIt's a little less of a problem if it's a personal laptop and nobody\nelse uses it, but it's still a lot easier to accidentally expose an\narbitrary file or for an attack to exfiltrate an existing file (just\nthrough a bug in existing software) than it is to necessarily execute\nthe arbitrary code necessary to read the data out of the system\ncredential store.\n\n`http.extraheader` for Authorization headers usually necessitates that\nthe data is either stored in the config file or passed on the command\nline, and that's why it's insecure. Certainly, you could configure it\nto read only from the environment using `--config-env` or you could\nconfigure your system to store the data in the config only with a\nsingle, highly restricted service account and it might be okay.\n\nThe kind of usage of `http.extraheader` that's likely to be fine is just\npassing an extra header that some broken proxy needs to be satisfied,\nlike setting a specific language or faking a header that the proxy needs\nto think Git's a web browser (since, of course, if it's not Internet\nExplorer, it's insecure). As long as you're not storing credentials or\nsecrets in `http.extraheader`, I have no objections.\n\nI don't know what you're using it for at Google, but of course if it is\nAuthorization headers, then I'm hoping this series will help you avoid\nneeding to do that.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"492197","messageId":"Zg4neXnm2WQHYnhp@tanuki","threadId":"61189","inReplyTo":"Zg323pR6UYthTakT@tapette.crustytoothpaste.net","subject":"Re: [PATCH 05/13] credential: gate new fields on capability","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-04-04T04:07:21Z","receivedAt":"2024-04-04T04:07:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Apr 04, 2024 at 12:39:58AM +0000, brian m. carlson wrote:\n> On 2024-04-02 at 10:04:31, Patrick Steinhardt wrote:\n> > The intent of this is quite clear to me, but thanks for re-explaining\n> > the bigger picture :)\n> \n> Sorry I misunderstood what you were getting it.\n> \n> > I think you misunderstood my confusion. I didn't meant to say that there\n> > should be non-boolean capabilities. I was rather missing the picture of\n> > how exactly you can advertise multiple capabilities with the infra that\n> > currently exists, and why the infra supports per-phase capabilities.\n> > \n> > Basically, what I would have expected is a protocol where both Git and\n> > the credential helper initially did a single \"handshake\" that also\n> > announces capabilities. So something like:\n> > \n> > HELPER: capability foobar\n> > HELPER: capability barfoo\n> > GIT: capability foobar\n> > \n> > Git would only acknowledge capabilities that it both understands and\n> > that have been announced by the helper. So at the end of this both have\n> > agreed on a single capability \"foobar\".\n> > \n> > This is roughly how the remote helper capability subsystem works. What\n> > this patch is introducing seems quite a bit more complicated than that\n> > though because we have \"staged\" capabilities. I assume there is good\n> > reason for this complexity, but I didn't yet manage to figure out the\n> > reasoning behind it.\n> > \n> > To ask more specifically: why would one side ever announce a capability\n> > in phase 1, but not in phase 2? Is the reason that capabilities are in\n> > fact tied to credentials?\n> \n> More that they're tied to the credential helper. For example, say we\n> have helpers A and B, in that order. B is incapable, but both A and Git\n> understand the new authtype capability. If we announce the capability\n> as in this series, we can get a new credential using the authtype\n> capability if A is willing to provide something to us, but we can't if A\n> has no credentials for us and B wants to provide them for us.\n> \n> This would also be true if we used your proposal of negotiation, but\n> because we have external callers (e.g., Git LFS) who may invoke `git\n> credential fill`, which would be a separate process from `git credential\n> capability`, we'd still have to have some way to tell `git credential\n> fill` what capabilities the external caller supported.\n> \n> The per-phase capabilities are such that we don't request functionality\n> that our callers can't use. For example, if our external caller (phase\n> 1) doesn't support the `authtype` credential, then we don't pass it to\n> the helper (phase 2), since the external caller might not be able to use\n> the result if we do. If the external caller (phase 1) _does_ support\n> it, but the helper does not (phase 2), then we won't return the\n> capability as the result of `git credential fill` (phase 3), so our\n> external caller will know that this isn't supported. As a practical\n> matter, that doesn't provide a great deal of useful information to the\n> caller at the moment, but it definitely could in the future (say, if we\n> had a capability for a certain form of data encoding).\n> \n> All of this is also true for internal (e.g., git-http-backend) callers,\n> except that phase 1 has all the capabilities we know about automatically\n> set, and phase 3 stores the data in the internal structure we'll use for\n> the `store` and `erase` calls.\n> \n> I do, however, think some way to query capabilities more generically\n> would be helpful, so I'll see if I can add such changes into a v2. I\n> think we still need the current approach to make the use case I\n> mentioned work, though.\n\nGreat. Thanks a lot for your explanations!\n\nPatrick\n"},{"id":"492576","messageId":"CAD0vCJ=4-QoqUovaOuw6gPTfPEa+d6uJBaO_Vq9R9Btn_YXzwg@mail.gmail.com","threadId":"61189","inReplyTo":"Zg38BLxLe193zYss@tapette.crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"Jackson Toeniskoetter","fromEmail":"jackdt@google.com","sentAt":"2024-04-08T18:42:56Z","receivedAt":"2024-04-08T18:43:11Z","isPatch":true,"sender":{"key":"jackdt@google.com","avatar":null},"body":"Just to clarify, we at Google do not use extraheader to pass along\ncredentials. Instead we use the .gitcookies file[1], which iiuc gets\nread by the git process directly. I'm not a security expert but I\nimagine the risk surface of extraheader and .gitcookies is similar.\n\nReading your post on\nhttps://lore.kernel.org/git/20240324011301.1553072-1-sandals@crustytoothpaste.net/,\nit's unclear to me why the credential helper protocol needs to be\nupdated. If the goal is to support Bearer tokens, can that not just be\nimplemented using extraheader? It seems like the goal of getting\ncredentials out of the config file can be accomplished without\nupdating the credential helper protocol. So is the bigger goal to\nsupport more robust and modern auth schemes which require multiple\nsteps? That would be useful to Google; multi-step auth would probably\nbe a more elegant way for us to stop using .gitcookies than other\nsolutions we were considering.\n\n[1] This is actually only for external contributors. Google employees\nhave a more robust authentication mechanism. Concerns have been raised\ninternally about our usage of gitcookies, but it hasn't been made a\npriority to address because a leaked credential would not allow an\nattacker to commit bad code, only read it or initiate a code review.\n"},{"id":"492755","messageId":"CAGJzqs=PJX_ydB911QqBxf41wnNXaT0LZRwjDqfTw_vTTMPLow@mail.gmail.com","threadId":"61189","inReplyTo":"CAD0vCJ=4-QoqUovaOuw6gPTfPEa+d6uJBaO_Vq9R9Btn_YXzwg@mail.gmail.com","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-04-11T07:00:00Z","receivedAt":"2024-04-11T07:02:00Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Mon, 8 Apr 2024 at 19:43, Jackson Toeniskoetter wrote:\n>\n> Just to clarify, we at Google do not use extraheader to pass along\n> credentials. Instead we use the .gitcookies file[1], which iiuc gets\n> read by the git process directly. I'm not a security expert but I\n> imagine the risk surface of extraheader and .gitcookies is similar.\n\nHi Jackson. To clarify, are you describing hosts *.googlesource.com\nsuch as https://go.googlesource.com/? It always confused me that the\n'generate password' feature gives you something different to a\npassword.\n\nhttps://www.googlesource.com/new-password\nhttps://gerrit-review.googlesource.com/Documentation/user-upload.html\n\nHave you tried OAuth credential helper git-credential-oauth\nhttps://github.com/hickford/git-credential-oauth? It can authenticate\nto *.googlesource.com without setup. OAuth has security advantages\nover .gitcookies because the OAuth access tokens expire after 2 hours\nand the OAuth refresh tokens are single use. These credentials are\nstored in the user's choice of secure storage such as\ngit-credential-cache, git-credential-libsecret, git-credential-wincred\nor git-credential-osxkeychain.\n\nI encourage you to try it out. You'll need a local web browser because\nGoogle's OAuth configuration currently forbids device authorization\ngrant for the relevant scope\nhttps://issues.gerritcodereview.com/issues/300279941\n\n>\n> Reading your post on\n> https://lore.kernel.org/git/20240324011301.1553072-1-sandals@crustytoothpaste.net/,\n> it's unclear to me why the credential helper protocol needs to be\n> updated. If the goal is to support Bearer tokens, can that not just be\n> implemented using extraheader? It seems like the goal of getting\n> credentials out of the config file can be accomplished without\n> updating the credential helper protocol. So is the bigger goal to\n> support more robust and modern auth schemes which require multiple\n> steps? That would be useful to Google; multi-step auth would probably\n> be a more elegant way for us to stop using .gitcookies than other\n> solutions we were considering.\n>\n> [1] This is actually only for external contributors. Google employees\n> have a more robust authentication mechanism. Concerns have been raised\n> internally about our usage of gitcookies, but it hasn't been made a\n> priority to address because a leaked credential would not allow an\n> attacker to commit bad code, only read it or initiate a code review.\n"},{"id":"492756","messageId":"CAGJzqs=N79qLXD7CNo6m4x+KPzMFq5n9_wa5mbJw94VNatksNQ@mail.gmail.com","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"M Hickford","fromEmail":"mirth.hickford@gmail.com","sentAt":"2024-04-11T07:00:00Z","receivedAt":"2024-04-11T07:02:02Z","isPatch":true,"sender":{"key":"mirth.hickford@gmail.com","avatar":"https://avatars.githubusercontent.com/u/105314?v=4"},"body":"On Sun, 24 Mar 2024 at 01:13, brian m. carlson\n wrote:\n>\n> Right now, HTTP authentication in Git is mostly limited to approaches\n> that require a username and password or are Kerberos (GSSAPI). In\n> addition, we effectively require that libcurl (or, for other software,\n> such as Git LFS, using the credential helper, that HTTP library) knows\n> how to implement the authentication scheme.\n>\n> However, this poses two sets of problems. First, some sites, such as\n> Azure DevOps, want to use Bearer authentication, which we don't support.\n> This is implemented using `http.extraHeader`, which is not a secure way\n> to store credentials, since our credential helper protocol does not\n> support this functionality.\n>\n> In addition, other tools using the credential helper protocol do not\n> support the variety of authentication mechanisms that Git does.\n> Specifically, making NTLM function in a useful way on Windows is\n> nontrivial and requires extensive integration and testing with C code,\n> and because of this difficulty and the fact that NTLM uses cryptography\n> known to be insecure since 1995, there is often little interest in\n> implementing this support outside of libcurl. However, it would be\n> helpful if people who want to use it can still use it.\n>\n> This series introduces new functionality to the credential helper\n> protocol that allows helpers to produce credentials for arbitrary HTTP\n> authentication schemes using the `authtype` and `credential`[0] fields.\n> This allows a suitable credential helper to send Bearer credentials or\n> any other standard or custom authentication scheme. (It may be able to\n> be extended to other functionality in the future, such as\n> git-send-email, to implement custom SASL functionality, and due care has\n> been taken to make the protocol adequately flexible for that purpose.)\n>\n> In addition, the protocol is also expanded to include per-helper state\n> and multi-legged authentication (the former is effectively required for\n> the latter). The per-helper state can be useful to help credential\n> helpers identify where the credential is stored, or any other\n> information necessary. Because NTLM and Negotiate (Kerberos/wrapped\n> NTLM) require two rounds of authentication, the multi-legged\n> authentication support along with per-helper state allows the helper to\n> support these authentication methods without Git or other clients having\n> to be aware of how they work. (This would also be useful for SASL, as\n> mentioned above.)\n>\n> This series introduces a capability mechanism to announce this\n> functionality, which allows a helper to provide a username and password\n> on older versions of Git while supporting more advanced functionality on\n> newer versions. (This is especially important on Azure DevOps, where\n> NTLM uses a username and password but Basic or Bearer can use a personal\n> access token.) It is also designed such that extremely simple\n> credential helpers, such as the shell one-liner in the Git FAQ that\n> reads from the environment, don't accidentally claim to support\n> functionality they don't offer.\n>\n> In addition, there is documentation for the expanded protocol, although\n> none of the built-in helpers have been updated (that will be a future\n> series for those for which it's possible).\n>\n> My personal interest here is getting credentials out of config files\n> with `http.extraHeader` (which a future series will produce a warning\n> for) and also allowing Git LFS to support Digest and NTLM with a\n> suitable credential helper. Git LFS used to support NTLM using custom\n> code (because the Go standard library does not), but it was found to be\n> broken in lots of ways on Windows, and nobody with a Windows system\n> wanted to fix it or support it, so we removed it. However, there are\n> still some people who do want to use it, so allowing them to use a\n> custom credential helper they maintain themselves seems like the best\n> way forward. Despite the advantages of this series for Azure DevOps, I\n> have no personal or professional stake in their product; my only\n> interest is the general one in whether their users can securely store\n> credentials. I believe the changes here are of general advantage to the\n> Git userbase in a variety of ways such that the goal of this series\n> should be uncontroversial.\n\nDo you happen to know any public Git remotes that support Digest or\nNTLM authentication?\n\n\n\n>\n> Feedback on any portion of this series is of course welcome.\n>\n> [0] A name different from `password` was explicitly chosen to avoid\n> confusion from less capable protocol helpers so that they don't\n> accidentally send invalid data. This does have the downside that\n> credential helpers must learn a new field to not log, but that should be\n> generally easy to fix in most cases.\n>\n> brian m. carlson (13):\n> credential: add an authtype field\n> remote-curl: reset headers on new request\n> http: use new headers for each object request\n> credential: add a field for pre-encoded credentials\n> credential: gate new fields on capability\n> docs: indicate new credential protocol fields\n> http: add support for authtype and credential\n> credential: add an argument to keep state\n> credential: enable state capability\n> docs: set a limit on credential line length\n> t5563: refactor for multi-stage authentication\n> strvec: implement swapping two strvecs\n> credential: add support for multistage credential rounds\n>\n> Documentation/git-credential.txt | 59 +++++-\n> builtin/credential-cache--daemon.c | 2 +-\n> builtin/credential-store.c | 2 +-\n> builtin/credential.c | 7 +-\n> credential.c | 114 ++++++++++-\n> credential.h | 69 ++++++-\n> http.c | 128 +++++++-----\n> http.h | 5 +\n> imap-send.c | 2 +-\n> remote-curl.c | 14 +-\n> strvec.c | 7 +\n> strvec.h | 5 +\n> t/lib-httpd/nph-custom-auth.sh | 17 +-\n> t/t0300-credentials.sh | 136 ++++++++++++-\n> t/t5563-simple-http-auth.sh | 308 +++++++++++++++++++++++++----\n> 15 files changed, 760 insertions(+), 115 deletions(-)\n>\n"},{"id":"492825","messageId":"Zhh7xZ3mVOxqUGL3@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"CAD0vCJ=4-QoqUovaOuw6gPTfPEa+d6uJBaO_Vq9R9Btn_YXzwg@mail.gmail.com","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-12T00:09:41Z","receivedAt":"2024-04-12T00:09:49Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-04-08 at 18:42:56, Jackson Toeniskoetter wrote:\n> Just to clarify, we at Google do not use extraheader to pass along\n> credentials. Instead we use the .gitcookies file[1], which iiuc gets\n> read by the git process directly. I'm not a security expert but I\n> imagine the risk surface of extraheader and .gitcookies is similar.\n> \n> Reading your post on\n> https://lore.kernel.org/git/20240324011301.1553072-1-sandals@crustytoothpaste.net/,\n> it's unclear to me why the credential helper protocol needs to be\n> updated. If the goal is to support Bearer tokens, can that not just be\n> implemented using extraheader? It seems like the goal of getting\n> credentials out of the config file can be accomplished without\n> updating the credential helper protocol. So is the bigger goal to\n> support more robust and modern auth schemes which require multiple\n> steps? That would be useful to Google; multi-step auth would probably\n> be a more elegant way for us to stop using .gitcookies than other\n> solutions we were considering.\n\nBearer authentication certainly can be implemented using\nhttp.extraheader and the config file is also not a secure way to store\ncredentials, which is why the credential helper protocol is being\nupdated, since then people will be able to store the credentials in a\npassword manager or other secure store. Other HTTP schemes will also be\nsupported as long as they don't require headers other than Authorization\nand the credential helper can implement them on behalf of Git.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"492826","messageId":"Zhh8nHWc8J8HpJ32@tapette.crustytoothpaste.net","threadId":"61189","inReplyTo":"CAGJzqs=N79qLXD7CNo6m4x+KPzMFq5n9_wa5mbJw94VNatksNQ@mail.gmail.com","subject":"Re: [PATCH 00/13] Support for arbitrary schemes in credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-12T00:13:16Z","receivedAt":"2024-04-12T00:13:18Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-04-11 at 07:00:00, M Hickford wrote:\n> Do you happen to know any public Git remotes that support Digest or\n> NTLM authentication?\n\nAzure DevOps supports NTLM in some cases. I don't use it, so I can't\nsay in what situations you can use it, but I know from the Git LFS issue\ntracker it's a common configuration.\n\nNote that if you just want a test environment, you can use Apache with\nDigest authentication. However, it's limited to MD5, which should no\nlonger be used for any reason, so you'll probably want to try a\ndifferent implementation.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"493026","messageId":"20240417000240.3611948-3-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 02/16] remote-curl: reset headers on new request","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:26Z","receivedAt":"2024-04-17T00:03:01Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When we retry a post_rpc request, we currently reuse the same headers as\nbefore. In the future, we'd like to be able to modify them based on the\nresult we get back, so let's reset them on each retry so we can avoid\nsending potentially duplicate headers if the values change.\n\nSigned-off-by: brian m. carlson \n---\n remote-curl.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/remote-curl.c b/remote-curl.c\nindex 1161dc7fed..e37eaa17b7 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -893,7 +893,7 @@ static curl_off_t xcurl_off_t(size_t len)\n static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_received)\n {\n \tstruct active_request_slot *slot;\n-\tstruct curl_slist *headers = http_copy_default_headers();\n+\tstruct curl_slist *headers = NULL;\n \tint use_gzip = rpc->gzip_request;\n \tchar *gzip_body = NULL;\n \tsize_t gzip_size = 0;\n@@ -935,6 +935,8 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \t\t\tneeds_100_continue = 1;\n \t}\n \n+retry:\n+\theaders = http_copy_default_headers();\n \theaders = curl_slist_append(headers, rpc->hdr_content_type);\n \theaders = curl_slist_append(headers, rpc->hdr_accept);\n \theaders = curl_slist_append(headers, needs_100_continue ?\n@@ -948,7 +950,6 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \tif (rpc->protocol_header)\n \t\theaders = curl_slist_append(headers, rpc->protocol_header);\n \n-retry:\n \tslot = get_active_slot();\n \n \tcurl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0);\n@@ -1044,6 +1045,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \terr = run_slot(slot, NULL);\n \tif (err == HTTP_REAUTH && !large_request) {\n \t\tcredential_fill(&http_auth);\n+\t\tcurl_slist_free_all(headers);\n \t\tgoto retry;\n \t}\n \tif (err != HTTP_OK)\n"},{"id":"493027","messageId":"20240417000240.3611948-4-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 03/16] http: use new headers for each object request","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:27Z","receivedAt":"2024-04-17T00:03:01Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Currently we create one set of headers for all object requests and reuse\nit. However, we'll need to adjust the headers for authentication\npurposes in the future, so let's create a new set for each request so\nthat we can adjust them if the authentication changes.\n\nNote that the cost of allocation here is tiny compared to the fact that\nwe're making a network call, not to mention probably a full TLS\nconnection, so this shouldn't have a significant impact on performance.\nMoreover, nobody who cares about performance is using the dumb HTTP\nprotocol anyway, since it often makes huge numbers of requests compared\nto the smart protocol.\n\nSigned-off-by: brian m. carlson \n---\n http.c | 20 +++++++++++---------\n http.h | 2 ++\n 2 files changed, 13 insertions(+), 9 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex e73b136e58..22639b0974 100644\n--- a/http.c\n+++ b/http.c\n@@ -128,7 +128,6 @@ static unsigned long empty_auth_useless =\n \t| CURLAUTH_DIGEST;\n \n static struct curl_slist *pragma_header;\n-static struct curl_slist *no_pragma_header;\n static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;\n \n static struct curl_slist *host_resolutions;\n@@ -299,6 +298,11 @@ size_t fwrite_null(char *ptr UNUSED, size_t eltsize UNUSED, size_t nmemb,\n \treturn nmemb;\n }\n \n+static struct curl_slist *object_request_headers(void)\n+{\n+\treturn curl_slist_append(http_copy_default_headers(), \"Pragma:\");\n+}\n+\n static void closedown_active_slot(struct active_request_slot *slot)\n {\n \tactive_requests--;\n@@ -1275,8 +1279,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)\n \n \tpragma_header = curl_slist_append(http_copy_default_headers(),\n \t\t\"Pragma: no-cache\");\n-\tno_pragma_header = curl_slist_append(http_copy_default_headers(),\n-\t\t\"Pragma:\");\n \n \t{\n \t\tchar *http_max_requests = getenv(\"GIT_HTTP_MAX_REQUESTS\");\n@@ -1360,9 +1362,6 @@ void http_cleanup(void)\n \tcurl_slist_free_all(pragma_header);\n \tpragma_header = NULL;\n \n-\tcurl_slist_free_all(no_pragma_header);\n-\tno_pragma_header = NULL;\n-\n \tcurl_slist_free_all(host_resolutions);\n \thost_resolutions = NULL;\n \n@@ -2370,6 +2369,7 @@ void release_http_pack_request(struct http_pack_request *preq)\n \t}\n \tpreq->slot = NULL;\n \tstrbuf_release(&preq->tmpfile);\n+\tcurl_slist_free_all(preq->headers);\n \tfree(preq->url);\n \tfree(preq);\n }\n@@ -2454,11 +2454,11 @@ struct http_pack_request *new_direct_http_pack_request(\n \t}\n \n \tpreq->slot = get_active_slot();\n+\tpreq->headers = object_request_headers();\n \tcurl_easy_setopt(preq->slot->curl, CURLOPT_WRITEDATA, preq->packfile);\n \tcurl_easy_setopt(preq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite);\n \tcurl_easy_setopt(preq->slot->curl, CURLOPT_URL, preq->url);\n-\tcurl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER,\n-\t\tno_pragma_header);\n+\tcurl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER, preq->headers);\n \n \t/*\n \t * If there is data present from a previous transfer attempt,\n@@ -2624,13 +2624,14 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \t}\n \n \tfreq->slot = get_active_slot();\n+\tfreq->headers = object_request_headers();\n \n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_WRITEDATA, freq);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_FAILONERROR, 0);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite_sha1_file);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_ERRORBUFFER, freq->errorstr);\n \tcurl_easy_setopt(freq->slot->curl, CURLOPT_URL, freq->url);\n-\tcurl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, no_pragma_header);\n+\tcurl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, freq->headers);\n \n \t/*\n \t * If we have successfully processed data from a previous fetch\n@@ -2718,5 +2719,6 @@ void release_http_object_request(struct http_object_request *freq)\n \t\trelease_active_slot(freq->slot);\n \t\tfreq->slot = NULL;\n \t}\n+\tcurl_slist_free_all(freq->headers);\n \tstrbuf_release(&freq->tmpfile);\n }\ndiff --git a/http.h b/http.h\nindex 3af19a8bf5..c5f8cc4620 100644\n--- a/http.h\n+++ b/http.h\n@@ -196,6 +196,7 @@ struct http_pack_request {\n \tFILE *packfile;\n \tstruct strbuf tmpfile;\n \tstruct active_request_slot *slot;\n+\tstruct curl_slist *headers;\n };\n \n struct http_pack_request *new_http_pack_request(\n@@ -229,6 +230,7 @@ struct http_object_request {\n \tint zret;\n \tint rename;\n \tstruct active_request_slot *slot;\n+\tstruct curl_slist *headers;\n };\n \n struct http_object_request *new_http_object_request(\n"},{"id":"493028","messageId":"20240417000240.3611948-1-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240324011301.1553072-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 00/16] Support for arbitrary schemes in credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:24Z","receivedAt":"2024-04-17T00:03:01Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This series introduces support for multiple schemes into the credential\nhelper protocol and Git (currently only for HTTP) so that we can support\nother types of authentication, such as Bearer, that libcurl doesn't\nsupport natively. Special consideration has been given to making this\nfunctionality work for external users of `git credential`.\n\nFor the truly gory details, please see v1's cover letter.\n\nChanges from v1:\n* Add support for indicating that a credential is ephemeral.\n* Add support for the authtype capability to git credential-cache.\n* Fix a bug where credential helpers might not be passed the capability\n correctly.\n* Add a method for querying capabilities from `git credential` and\n helpers, giving due consideration to gracefully handling simple shell\n scripts as credential helpers.\n* Remove `strvec_swap` in favour of `SWAP`.\n* Add support for testing authtype support in credential helpers.\n* Fix a bug in the test that caused errors with bash.\n* Use an enum instead of define constants for credential stages.\n* Documentation improvements.\n* Possibly other cleanups and fixes mentioned in review which I have\n forgotten about fixing by now.\n\nbrian m. carlson (16):\n credential: add an authtype field\n remote-curl: reset headers on new request\n http: use new headers for each object request\n credential: add a field for pre-encoded credentials\n credential: gate new fields on capability\n credential: add a field called \"ephemeral\"\n docs: indicate new credential protocol fields\n http: add support for authtype and credential\n credential: add an argument to keep state\n credential: enable state capability\n docs: set a limit on credential line length\n t5563: refactor for multi-stage authentication\n credential: add support for multistage credential rounds\n t: add credential tests for authtype\n credential-cache: implement authtype capability\n credential: add method for querying capabilities\n\n Documentation/git-credential.txt | 104 +++++++++-\n builtin/credential-cache--daemon.c | 22 ++-\n builtin/credential-cache.c | 10 +\n builtin/credential-store.c | 2 +-\n builtin/credential.c | 15 +-\n credential.c | 138 ++++++++++++-\n credential.h | 92 ++++++++-\n http.c | 129 +++++++-----\n http.h | 5 +\n imap-send.c | 2 +-\n remote-curl.c | 14 +-\n t/lib-credential.sh | 123 ++++++++++++\n t/lib-httpd/nph-custom-auth.sh | 17 +-\n t/t0300-credentials.sh | 165 +++++++++++++++-\n t/t0301-credential-cache.sh | 1 +\n t/t5563-simple-http-auth.sh | 308 +++++++++++++++++++++++++----\n 16 files changed, 1026 insertions(+), 121 deletions(-)\n\nRange-diff against v1:\n 1: ac2507fabb = 1: ac2507fabb credential: add an authtype field\n 2: f1f3367826 = 2: f1f3367826 remote-curl: reset headers on new request\n 3: 1cccbc9d8b ! 3: cd3b1a8c3a http: use new headers for each object request\n @@ http.c: void http_cleanup(void)\n \n -\tcurl_slist_free_all(no_pragma_header);\n -\tno_pragma_header = NULL;\n - \n +-\n \tcurl_slist_free_all(host_resolutions);\n \thost_resolutions = NULL;\n + \n @@ http.c: void release_http_pack_request(struct http_pack_request *preq)\n \t}\n \tpreq->slot = NULL;\n 4: a88e6b26c8 = 4: 7b88a32294 credential: add a field for pre-encoded credentials\n 5: e0df6734a3 ! 5: 912d06b70b credential: gate new fields on capability\n @@ Commit message\n internal user, then we know about all capabilities and will request\n them.\n \n + For \"git credential approve\" and \"git credential reject\", we set the\n + helper capability before calling the helper, since we assume that the\n + input we're getting from the external program comes from a previous call\n + to \"git credential fill\", and thus we'll invoke send a capability to the\n + helper if and only if we got one from the standard input, which is the\n + correct behavior.\n +\n Signed-off-by: brian m. carlson \n \n ## builtin/credential-cache--daemon.c ##\n @@ builtin/credential.c: int cmd_credential(int argc, const char **argv, const char\n +\t\tcredential_fill(&c, 0);\n +\t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n \t} else if (!strcmp(op, \"approve\")) {\n ++\t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);\n \t\tcredential_approve(&c);\n \t} else if (!strcmp(op, \"reject\")) {\n ++\t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);\n + \t\tcredential_reject(&c);\n + \t} else {\n + \t\tusage(usage_msg);\n \n ## credential.c ##\n @@ credential.c: void credential_clear(struct credential *c)\n \tcredential_init(c);\n }\n \n -+static void credential_set_all_capabilities(struct credential *c)\n -+{\n -+\tc->capa_authtype.request_initial = 1;\n -+}\n -+\n - int credential_match(const struct credential *want,\n - \t\t const struct credential *have, int match_password)\n - {\n -@@ credential.c: static void credential_getpass(struct credential *c)\n - \t\t\t\t\t\t PROMPT_ASKPASS);\n - }\n - \n --int credential_read(struct credential *c, FILE *fp)\n -+static void credential_set_capability(struct credential_capability *capa, int op_type)\n ++static void credential_set_capability(struct credential_capability *capa,\n ++\t\t\t\t enum credential_op_type op_type)\n +{\n +\tswitch (op_type) {\n +\tcase CREDENTIAL_OP_INITIAL:\n @@ credential.c: static void credential_getpass(struct credential *c)\n +\t}\n +}\n +\n -+static int credential_has_capability(const struct credential_capability *capa, int op_type)\n ++\n ++void credential_set_all_capabilities(struct credential *c,\n ++\t\t\t\t enum credential_op_type op_type)\n ++{\n ++\tcredential_set_capability(&c->capa_authtype, op_type);\n ++}\n ++\n + int credential_match(const struct credential *want,\n + \t\t const struct credential *have, int match_password)\n + {\n +@@ credential.c: static void credential_getpass(struct credential *c)\n + \t\t\t\t\t\t PROMPT_ASKPASS);\n + }\n + \n +-int credential_read(struct credential *c, FILE *fp)\n ++static int credential_has_capability(const struct credential_capability *capa,\n ++\t\t\t\t enum credential_op_type op_type)\n +{\n +\t/*\n +\t * We're checking here if each previous step indicated that we had the\n @@ credential.c: static void credential_getpass(struct credential *c)\n +\t}\n +}\n +\n -+int credential_read(struct credential *c, FILE *fp, int op_type)\n ++int credential_read(struct credential *c, FILE *fp,\n ++\t\t enum credential_op_type op_type)\n {\n \tstruct strbuf line = STRBUF_INIT;\n \n @@ credential.c: static void credential_write_item(FILE *fp, const char *key, const\n }\n \n -void credential_write(const struct credential *c, FILE *fp)\n -+void credential_write(const struct credential *c, FILE *fp, int op_type)\n ++void credential_write(const struct credential *c, FILE *fp,\n ++\t\t enum credential_op_type op_type)\n {\n +\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n +\t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n @@ credential.c: void credential_fill(struct credential *c)\n \n \tcredential_apply_config(c);\n +\tif (all_capabilities)\n -+\t\tcredential_set_all_capabilities(c);\n ++\t\tcredential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);\n \n \tfor (i = 0; i < c->helpers.nr; i++) {\n \t\tcredential_do(c, c->helpers.items[i].string, \"get\");\n @@ credential.h\n + * At each stage, we will emit the capability only if the previous stage\n + * supported it.\n + */\n -+#define CREDENTIAL_OP_INITIAL 1\n -+#define CREDENTIAL_OP_HELPER 2\n -+#define CREDENTIAL_OP_RESPONSE 3\n ++enum credential_op_type {\n ++\tCREDENTIAL_OP_INITIAL = 1,\n ++\tCREDENTIAL_OP_HELPER = 2,\n ++\tCREDENTIAL_OP_RESPONSE = 3,\n ++};\n +\n +struct credential_capability {\n +\tunsigned request_initial:1,\n @@ credential.h: void credential_approve(struct credential *);\n \n -int credential_read(struct credential *, FILE *);\n -void credential_write(const struct credential *, FILE *);\n -+int credential_read(struct credential *, FILE *, int);\n -+void credential_write(const struct credential *, FILE *, int);\n ++/**\n ++ * Enable all of the supported credential flags in this credential.\n ++ */\n ++void credential_set_all_capabilities(struct credential *c,\n ++\t\t\t\t enum credential_op_type op_type);\n ++\n ++int credential_read(struct credential *, FILE *,\n ++\t\t enum credential_op_type);\n ++void credential_write(const struct credential *, FILE *,\n ++\t\t enum credential_op_type);\n \n /*\n * Parse a url into a credential struct, replacing any existing contents.\n @@ t/t0300-credentials.sh: test_expect_success 'setup helper scripts' '\n +\tcredential=$1; shift\n +\t. ./dump\n +\techo capability[]=authtype\n -+\ttest -z \"${capability##*authtype*}\" || return\n ++\ttest -z \"${capability##*authtype*}\" || exit 0\n +\ttest -z \"$authtype\" || echo authtype=$authtype\n +\ttest -z \"$credential\" || echo credential=$credential\n +\tEOF\n -: ---------- > 6: f96183e273 credential: add a field called \"ephemeral\"\n 6: 78fc8cdccf ! 7: efaab8eba4 docs: indicate new credential protocol fields\n @@ Documentation/git-credential.txt: empty string.\n \n +`authtype`::\n +\tThis indicates that the authentication scheme in question should be used.\n -+\tCommon values for HTTP and HTTPS include `basic`, `digest`, and `ntlm`,\n -+\talthough the latter two are insecure and should not be used. If `credential`\n ++\tCommon values for HTTP and HTTPS include `basic`, `bearer`, and `digest`,\n ++\talthough the latter is insecure and should not be used. If `credential`\n +\tis used, this may be set to an arbitrary string suitable for the protocol in\n +\tquestion (usually HTTP).\n ++\n @@ Documentation/git-credential.txt: empty string.\n +`credential`::\n +\tThe pre-encoded credential, suitable for the protocol in question (usually\n +\tHTTP). If this key is sent, `authtype` is mandatory, and `username` and\n -+\t`password` are not used.\n ++\t`password` are not used. For HTTP, Git concatenates the `authtype` value and\n ++\tthis value with a single space to determine the `Authorization` header.\n +++\n ++This value should not be sent unless the appropriate capability (see below) is\n ++provided on input.\n ++\n ++`ephemeral`::\n ++\tThis boolean value indicates, if true, that the value in the `credential`\n ++\tfield should not be saved by the credential helper because its usefulness is\n ++\tlimited in time. For example, an HTTP Digest `credential` value is computed\n ++\tusing a nonce and reusing it will not result in successful authentication.\n ++\tThis may also be used for situations with short duration (e.g., 24-hour)\n ++\tcredentials. The default value is false.\n +++\n ++The credential helper will still be invoked with `store` or `erase` so that it\n ++can determine whether the operation was successful.\n ++\n +This value should not be sent unless the appropriate capability (see below) is\n +provided on input.\n @@ Documentation/git-credential.txt: attribute 'wwwauth[]', where the order of the\n +\tprotocol.\n ++\n +The only capability currently supported is `authtype`, which indicates that the\n -+`authtype` and `credential` values are understood. It is not obligatory to use\n -+these values in such a case, but they should not be provided without this\n -+capability.\n ++`authtype`, `credential`, and `ephemeral` values are understood. It is not\n ++obligatory to use these values in such a case, but they should not be provided\n ++without this capability.\n ++\n +Callers of `git credential` and credential helpers should emit the\n +capabilities they support unconditionally, and Git will gracefully\n 7: 82f7582268 = 8: 3b0a426fa1 http: add support for authtype and credential\n 8: c67540de6f ! 9: 8897ecf5e0 credential: add an argument to keep state\n @@ Commit message\n Signed-off-by: brian m. carlson \n \n ## Documentation/git-credential.txt ##\n -@@ Documentation/git-credential.txt: provided on input.\n +@@ Documentation/git-credential.txt: can determine whether the operation was successful.\n This value should not be sent unless the appropriate capability (see below) is\n provided on input.\n \n @@ Documentation/git-credential.txt: they appear in the HTTP response. This attribu\n `capability[]`::\n -\tThis signals that the caller supports the capability in question.\n -\tThis can be used to provide better, more specific data as part of the\n --\tprotocol.\n -+ This signals that Git, or the helper, as appropriate, supports the\n -+\tcapability in question. This can be used to provide better, more specific\n -+\tdata as part of the protocol.\n ++\tThis signals that Git, or the helper, as appropriate, supports the capability\n ++\tin question. This can be used to provide better, more specific data as part\n ++\tof the protocol. A `capability[]` directive must precede any value depending\n ++\ton it and these directives _should_ be the first item announced in the\n + \tprotocol.\n +\n -The only capability currently supported is `authtype`, which indicates that the\n --`authtype` and `credential` values are understood. It is not obligatory to use\n --these values in such a case, but they should not be provided without this\n --capability.\n --+\n +-`authtype`, `credential`, and `ephemeral` values are understood. It is not\n +-obligatory to use these values in such a case, but they should not be provided\n +-without this capability.\n ++There are two currently supported capabilities. The first is `authtype`, which\n ++indicates that the `authtype`, `credential`, and `ephemeral` values are\n ++understood. The second is `state`, which indicates that the `state[]` and\n ++`continue` values are understood.\n + +\n -Callers of `git credential` and credential helpers should emit the\n -capabilities they support unconditionally, and Git will gracefully\n -handle passing them on.\n -+There are two currently supported capabilities. The first is `authtype`, which\n -+indicates that the `authtype` and `credential` values are understood. The\n -+second is `state`, which indicates that the `state[]` value is understood.\n -+\n +It is not obligatory to use the additional features just because the capability\n -+is supported, but they should not be provided without this capability.\n ++is supported, but they should not be provided without the capability.\n \n Unrecognised attributes and capabilities are silently discarded.\n \n @@ credential.c: void credential_clear(struct credential *c)\n \n \tcredential_init(c);\n }\n -@@ credential.c: int credential_read(struct credential *c, FILE *fp, int op_type)\n - \t\t\tc->path = xstrdup(value);\n +@@ credential.c: int credential_read(struct credential *c, FILE *fp,\n + \t\t\tc->ephemeral = !!git_config_bool(\"ephemeral\", value);\n \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n \t\t\tstrvec_push(&c->wwwauth_headers, value);\n -\t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n @@ credential.c: int credential_read(struct credential *c, FILE *fp, int op_type)\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n @@ credential.c: static void credential_write_item(FILE *fp, const char *key, const char *value,\n - \n - void credential_write(const struct credential *c, FILE *fp, int op_type)\n + void credential_write(const struct credential *c, FILE *fp,\n + \t\t enum credential_op_type op_type)\n {\n -\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n +\tif (credential_has_capability(&c->capa_authtype, op_type))\n @@ credential.c: static void credential_write_item(FILE *fp, const char *key, const\n +\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n \t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n \t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n - \t}\n -@@ credential.c: void credential_write(const struct credential *c, FILE *fp, int op_type)\n + \t\tif (c->ephemeral)\n +@@ credential.c: void credential_write(const struct credential *c, FILE *fp,\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n @@ t/t0300-credentials.sh: test_expect_success 'setup helper scripts' '\n \t. ./dump\n \techo capability[]=authtype\n +\techo capability[]=state\n - \ttest -z \"${capability##*authtype*}\" || return\n + \ttest -z \"${capability##*authtype*}\" || exit 0\n \ttest -z \"$authtype\" || echo authtype=$authtype\n \ttest -z \"$credential\" || echo credential=$credential\n -+\ttest -z \"${capability##*state*}\" || return\n ++\ttest -z \"${capability##*state*}\" || exit 0\n +\techo state[]=verbatim-cred:foo\n \tEOF\n \n - \twrite_script git-credential-verbatim-with-expiry <<-\\EOF &&\n -@@ t/t0300-credentials.sh: test_expect_success 'credential_fill invokes helper with credential' '\n + \twrite_script git-credential-verbatim-ephemeral <<-\\EOF &&\n +@@ t/t0300-credentials.sh: test_expect_success 'credential_fill invokes helper with ephemeral credential' '\n + \tverbatim-ephemeral: host=example.com\n \tEOF\n '\n - \n +test_expect_success 'credential_fill invokes helper with credential and state' '\n +\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n +\tcapability[]=authtype\n @@ t/t0300-credentials.sh: test_expect_success 'credential_fill invokes helper with\n +\tverbatim-cred: host=example.com\n +\tEOF\n +'\n -+\n \n test_expect_success 'credential_fill invokes multiple helpers' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n 9: 6321e0d5e6 ! 10: 6e56145ace credential: enable state capability\n @@ Commit message\n Signed-off-by: brian m. carlson \n \n ## credential.c ##\n -@@ credential.c: void credential_clear(struct credential *c)\n - static void credential_set_all_capabilities(struct credential *c)\n +@@ credential.c: void credential_set_all_capabilities(struct credential *c,\n + \t\t\t\t enum credential_op_type op_type)\n {\n - \tc->capa_authtype.request_initial = 1;\n -+\tc->capa_state.request_initial = 1;\n + \tcredential_set_capability(&c->capa_authtype, op_type);\n ++\tcredential_set_capability(&c->capa_state, op_type);\n }\n \n int credential_match(const struct credential *want,\n10: aaac77a3ce = 11: 3f5d2aa5e0 docs: set a limit on credential line length\n11: f4b01b3ba5 = 12: 1dd81f7859 t5563: refactor for multi-stage authentication\n12: a2f82a4043 < -: ---------- strvec: implement swapping two strvecs\n13: ec925ae9fb ! 13: 4a0ed175a6 credential: add support for multistage credential rounds\n @@ Documentation/git-credential.txt: provided on input.\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\n -@@ Documentation/git-credential.txt: to pass additional information to credential helpers.\n - +\n - There are two currently supported capabilities. The first is `authtype`, which\n - indicates that the `authtype` and `credential` values are understood. The\n --second is `state`, which indicates that the `state[]` value is understood.\n -+second is `state`, which indicates that the `state[]` and `continue` values are\n -+understood.\n - \n - It is not obligatory to use the additional features just because the capability\n - is supported, but they should not be provided without this capability.\n \n ## builtin/credential.c ##\n @@ builtin/credential.c: int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n @@ builtin/credential.c: int cmd_credential(int argc, const char **argv, const char\n +\t\tcredential_next_state(&c);\n \t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n \t} else if (!strcmp(op, \"approve\")) {\n - \t\tcredential_approve(&c);\n + \t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);\n \n ## credential.c ##\n @@ credential.c: void credential_clear(struct credential *c)\n @@ credential.c: void credential_clear(struct credential *c)\n +void credential_next_state(struct credential *c)\n +{\n +\tstrvec_clear(&c->state_headers_to_send);\n -+\tstrvec_swap(&c->state_headers, &c->state_headers_to_send);\n ++\tSWAP(c->state_headers, c->state_headers_to_send);\n +}\n +\n +void credential_clear_secrets(struct credential *c)\n @@ credential.c: void credential_clear(struct credential *c)\n +\tFREE_AND_NULL(c->credential);\n +}\n +\n - static void credential_set_all_capabilities(struct credential *c)\n + static void credential_set_capability(struct credential_capability *capa,\n + \t\t\t\t enum credential_op_type op_type)\n {\n - \tc->capa_authtype.request_initial = 1;\n -@@ credential.c: int credential_read(struct credential *c, FILE *fp, int op_type)\n +@@ credential.c: int credential_read(struct credential *c, FILE *fp,\n \t\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n \t\t\telse if (!strcmp(value, \"state\"))\n \t\t\t\tcredential_set_capability(&c->capa_state, op_type);\n @@ credential.c: int credential_read(struct credential *c, FILE *fp, int op_type)\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n -@@ credential.c: void credential_write(const struct credential *c, FILE *fp, int op_type)\n +@@ credential.c: void credential_write(const struct credential *c, FILE *fp,\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n \tif (credential_has_capability(&c->capa_state, op_type)) {\n @@ credential.c: void credential_fill(struct credential *c, int all_capabilities)\n +\n \tcredential_apply_config(c);\n \tif (all_capabilities)\n - \t\tcredential_set_all_capabilities(c);\n + \t\tcredential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);\n @@ credential.c: void credential_fill(struct credential *c, int all_capabilities)\n \t\t\t/* Reset expiry to maintain consistency */\n \t\t\tc->password_expiry_utc = TIME_MAX;\n @@ credential.h: struct credential {\n \t * Internal use only. Keeps track of if we previously matched against a\n \t * WWW-Authenticate header line in order to re-fold future continuation\n @@ credential.h: struct credential {\n - \n \tunsigned approved:1,\n + \t\t ephemeral:1,\n \t\t configured:1,\n +\t\t multistage: 1,\n \t\t quit:1,\n @@ credential.h: struct credential {\n }\n \n /* Initialize a credential structure, setting all fields to empty. */\n -@@ credential.h: void credential_approve(struct credential *);\n - */\n - void credential_reject(struct credential *);\n +@@ credential.h: void credential_reject(struct credential *);\n + void credential_set_all_capabilities(struct credential *c,\n + \t\t\t\t enum credential_op_type op_type);\n \n +/**\n + * Clear the secrets in this credential, but leave other data intact.\n @@ credential.h: void credential_approve(struct credential *);\n + */\n +void credential_next_state(struct credential *c);\n +\n - int credential_read(struct credential *, FILE *, int);\n - void credential_write(const struct credential *, FILE *, int);\n - \n + int credential_read(struct credential *, FILE *,\n + \t\t enum credential_op_type);\n + void credential_write(const struct credential *, FILE *,\n \n ## http.c ##\n @@ http.c: static int handle_curl_result(struct slot_results *results)\n -: ---------- > 14: c4c07c1539 t: add credential tests for authtype\n -: ---------- > 15: b82650e219 credential-cache: implement authtype capability\n -: ---------- > 16: 2f6e160693 credential: add method for querying capabilities\n"},{"id":"493029","messageId":"20240417000240.3611948-5-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 04/16] credential: add a field for pre-encoded credentials","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:28Z","receivedAt":"2024-04-17T00:03:01Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"At the moment, our credential code wants to find a username and password\nfor access, which, for HTTP, it will pass to libcurl to encode and\nprocess. However, many users want to use authentication schemes that\nlibcurl doesn't support, such as Bearer authentication. In these\nschemes, the secret is not a username and password pair, but some sort\nof token that meets the production for authentication data in the RFC.\n\nIn fact, in general, it's useful to allow our credential helper to have\nknowledge about what specifically to put in the protocol header. Thus,\nadd a field, credential, which contains data that's preencoded to be\nsuitable for the protocol in question. If we have such data, we need\nneither a username nor a password, so make that adjustment as well.\n\nIt is in theory possible to reuse the password field for this. However,\nif we do so, we must know whether the credential helper supports our new\nscheme before sending it data, which necessitates some sort of\ncapability inquiry, because otherwise an uninformed credential helper\nwould store our preencoded data as a password, which would fail the next\ntime we attempted to connect to the remote server. This design is\nsubstantially simpler, and we can hint to the credential helper that we\nsupport this approach with a simple new field instead of needing to\nquery it first.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 14 ++++++++++----\n credential.h | 1 +\n 2 files changed, 11 insertions(+), 4 deletions(-)\n\ndiff --git a/credential.c b/credential.c\nindex 3dec433df5..c521822e5a 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -25,6 +25,7 @@ void credential_clear(struct credential *c)\n \tfree(c->path);\n \tfree(c->username);\n \tfree(c->password);\n+\tfree(c->credential);\n \tfree(c->oauth_refresh_token);\n \tfree(c->authtype);\n \tstring_list_clear(&c->helpers, 0);\n@@ -234,6 +235,9 @@ int credential_read(struct credential *c, FILE *fp)\n \t\t} else if (!strcmp(key, \"password\")) {\n \t\t\tfree(c->password);\n \t\t\tc->password = xstrdup(value);\n+\t\t} else if (!strcmp(key, \"credential\")) {\n+\t\t\tfree(c->credential);\n+\t\t\tc->credential = xstrdup(value);\n \t\t} else if (!strcmp(key, \"protocol\")) {\n \t\t\tfree(c->protocol);\n \t\t\tc->protocol = xstrdup(value);\n@@ -291,6 +295,7 @@ void credential_write(const struct credential *c, FILE *fp)\n \tcredential_write_item(fp, \"path\", c->path, 0);\n \tcredential_write_item(fp, \"username\", c->username, 0);\n \tcredential_write_item(fp, \"password\", c->password, 0);\n+\tcredential_write_item(fp, \"credential\", c->credential, 0);\n \tcredential_write_item(fp, \"oauth_refresh_token\", c->oauth_refresh_token, 0);\n \tif (c->password_expiry_utc != TIME_MAX) {\n \t\tchar *s = xstrfmt(\"%\"PRItime, c->password_expiry_utc);\n@@ -366,7 +371,7 @@ void credential_fill(struct credential *c)\n {\n \tint i;\n \n-\tif (c->username && c->password)\n+\tif ((c->username && c->password) || c->credential)\n \t\treturn;\n \n \tcredential_apply_config(c);\n@@ -379,7 +384,7 @@ void credential_fill(struct credential *c)\n \t\t\t/* Reset expiry to maintain consistency */\n \t\t\tc->password_expiry_utc = TIME_MAX;\n \t\t}\n-\t\tif (c->username && c->password)\n+\t\tif ((c->username && c->password) || c->credential)\n \t\t\treturn;\n \t\tif (c->quit)\n \t\t\tdie(\"credential helper '%s' told us to quit\",\n@@ -387,7 +392,7 @@ void credential_fill(struct credential *c)\n \t}\n \n \tcredential_getpass(c);\n-\tif (!c->username && !c->password)\n+\tif (!c->username && !c->password && !c->credential)\n \t\tdie(\"unable to get password from user\");\n }\n \n@@ -397,7 +402,7 @@ void credential_approve(struct credential *c)\n \n \tif (c->approved)\n \t\treturn;\n-\tif (!c->username || !c->password || c->password_expiry_utc < time(NULL))\n+\tif (((!c->username || !c->password) && !c->credential) || c->password_expiry_utc < time(NULL))\n \t\treturn;\n \n \tcredential_apply_config(c);\n@@ -418,6 +423,7 @@ void credential_reject(struct credential *c)\n \n \tFREE_AND_NULL(c->username);\n \tFREE_AND_NULL(c->password);\n+\tFREE_AND_NULL(c->credential);\n \tFREE_AND_NULL(c->oauth_refresh_token);\n \tc->password_expiry_utc = TIME_MAX;\n \tc->approved = 0;\ndiff --git a/credential.h b/credential.h\nindex dc96ca0318..9db892cf4d 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -138,6 +138,7 @@ struct credential {\n \n \tchar *username;\n \tchar *password;\n+\tchar *credential;\n \tchar *protocol;\n \tchar *host;\n \tchar *path;\n"},{"id":"493030","messageId":"20240417000240.3611948-2-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 01/16] credential: add an authtype field","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:25Z","receivedAt":"2024-04-17T00:03:01Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When Git makes an HTTP request, it can negotiate the type of\nauthentication to use with the server provided the authentication scheme\nis one of a few well-known types (Basic, Digest, NTLM, or Negotiate).\nHowever, some servers wish to use other types of authentication, such as\nthe Bearer type from OAuth2. Since libcurl doesn't natively support\nthis type, it isn't possible to use it, and the user is forced to\nspecify the Authorization header using the http.extraheader setting.\n\nHowever, storing a plaintext token in the repository configuration is\nnot very secure, especially if a repository can be shared by multiple\nparties. We already have support for many types of secure credential\nstorage by using credential helpers, so let's teach credential helpers\nhow to produce credentials for an arbitrary scheme.\n\nIf the credential helper specifies an authtype field, then it specifies\nan authentication scheme (e.g., Bearer) and the password field specifies\nthe raw authentication token, with any encoding already specified. We\nreuse the password field for this because some credential helpers store\nthe metadata without encryption even though the password is encrypted,\nand we'd like to avoid insecure storage if an older version of the\ncredential helper gets ahold of the data.\n\nThe username is not used in this case, but it is still preserved for the\npurpose of finding the right credential if the user has multiple\naccounts.\n\nIf the authtype field is not specified, then the password behaves as\nnormal and it is passed along with the username to libcurl.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 5 +++++\n credential.h | 6 ++++++\n 2 files changed, 11 insertions(+)\n\ndiff --git a/credential.c b/credential.c\nindex 18098bd35e..3dec433df5 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -26,6 +26,7 @@ void credential_clear(struct credential *c)\n \tfree(c->username);\n \tfree(c->password);\n \tfree(c->oauth_refresh_token);\n+\tfree(c->authtype);\n \tstring_list_clear(&c->helpers, 0);\n \tstrvec_clear(&c->wwwauth_headers);\n \n@@ -252,6 +253,9 @@ int credential_read(struct credential *c, FILE *fp)\n \t\t} else if (!strcmp(key, \"oauth_refresh_token\")) {\n \t\t\tfree(c->oauth_refresh_token);\n \t\t\tc->oauth_refresh_token = xstrdup(value);\n+\t\t} else if (!strcmp(key, \"authtype\")) {\n+\t\t\tfree(c->authtype);\n+\t\t\tc->authtype = xstrdup(value);\n \t\t} else if (!strcmp(key, \"url\")) {\n \t\t\tcredential_from_url(c, value);\n \t\t} else if (!strcmp(key, \"quit\")) {\n@@ -295,6 +299,7 @@ void credential_write(const struct credential *c, FILE *fp)\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n+\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n }\n \n static int run_credential_helper(struct credential *c,\ndiff --git a/credential.h b/credential.h\nindex acc41adf54..dc96ca0318 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -143,6 +143,12 @@ struct credential {\n \tchar *path;\n \tchar *oauth_refresh_token;\n \ttimestamp_t password_expiry_utc;\n+\n+\t/**\n+\t * The authorization scheme to use. If this is NULL, libcurl is free to\n+\t * negotiate any scheme it likes.\n+\t */\n+\tchar *authtype;\n };\n \n #define CREDENTIAL_INIT { \\\n"},{"id":"493031","messageId":"20240417000240.3611948-7-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 06/16] credential: add a field called \"ephemeral\"","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:30Z","receivedAt":"2024-04-17T00:03:03Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we have support for a wide variety of types of authentication,\nit's important to indicate to other credential helpers whether they\nshould store credentials, since not every credential helper may\nintuitively understand all possible values of the authtype field. Do so\nwith a boolean field called \"ephemeral\", to indicate whether the\ncredential is expected to be temporary.\n\nFor example, in HTTP Digest authentication, the Authorization header\nvalue is based off a nonce. It isn't useful to store this value\nfor later use because reusing the credential long term will not result\nin successful authentication due to the nonce necessarily differing.\n\nAn additional case is potentially short-lived credentials, which may\nlast only a few hours. It similarly wouldn't be helper for other\ncredential helpers to attempt to provide these much later.\n\nWe do still pass the value to \"git credential store\" or \"git credential\nerase\", since it may be helpful to the original helper to know whether\nthe operation was successful.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 4 ++++\n credential.h | 1 +\n t/t0300-credentials.sh | 30 ++++++++++++++++++++++++++++++\n 3 files changed, 35 insertions(+)\n\ndiff --git a/credential.c b/credential.c\nindex f5396629df..3531d74346 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -289,6 +289,8 @@ int credential_read(struct credential *c, FILE *fp,\n \t\t} else if (!strcmp(key, \"path\")) {\n \t\t\tfree(c->path);\n \t\t\tc->path = xstrdup(value);\n+\t\t} else if (!strcmp(key, \"ephemeral\")) {\n+\t\t\tc->ephemeral = !!git_config_bool(\"ephemeral\", value);\n \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n \t\t\tstrvec_push(&c->wwwauth_headers, value);\n \t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n@@ -339,6 +341,8 @@ void credential_write(const struct credential *c, FILE *fp,\n \t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n \t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n \t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n+\t\tif (c->ephemeral)\n+\t\t\tcredential_write_item(fp, \"ephemeral\", \"1\", 0);\n \t}\n \tcredential_write_item(fp, \"protocol\", c->protocol, 1);\n \tcredential_write_item(fp, \"host\", c->host, 1);\ndiff --git a/credential.h b/credential.h\nindex b524fdba59..da2a4802b7 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -152,6 +152,7 @@ struct credential {\n \tunsigned header_is_last_match:1;\n \n \tunsigned approved:1,\n+\t\t ephemeral:1,\n \t\t configured:1,\n \t\t quit:1,\n \t\t use_http_path:1,\ndiff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\nindex daf330ddd8..eceb6bbfbe 100755\n--- a/t/t0300-credentials.sh\n+++ b/t/t0300-credentials.sh\n@@ -51,6 +51,17 @@ test_expect_success 'setup helper scripts' '\n \ttest -z \"$credential\" || echo credential=$credential\n \tEOF\n \n+\twrite_script git-credential-verbatim-ephemeral <<-\\EOF &&\n+\tauthtype=$1; shift\n+\tcredential=$1; shift\n+\t. ./dump\n+\techo capability[]=authtype\n+\ttest -z \"${capability##*authtype*}\" || exit 0\n+\ttest -z \"$authtype\" || echo authtype=$authtype\n+\ttest -z \"$credential\" || echo credential=$credential\n+\techo \"ephemeral=1\"\n+\tEOF\n+\n \twrite_script git-credential-verbatim-with-expiry <<-\\EOF &&\n \tuser=$1; shift\n \tpass=$1; shift\n@@ -99,6 +110,25 @@ test_expect_success 'credential_fill invokes helper with credential' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill invokes helper with ephemeral credential' '\n+\tcheck fill \"verbatim-ephemeral Bearer token\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=token\n+\tephemeral=1\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-ephemeral: get\n+\tverbatim-ephemeral: capability[]=authtype\n+\tverbatim-ephemeral: protocol=http\n+\tverbatim-ephemeral: host=example.com\n+\tEOF\n+'\n \n test_expect_success 'credential_fill invokes multiple helpers' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n"},{"id":"493033","messageId":"20240417000240.3611948-6-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 05/16] credential: gate new fields on capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:29Z","receivedAt":"2024-04-17T00:03:03Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We support the new credential and authtype fields, but we lack a way to\nindicate to a credential helper that we'd like them to be used. Without\nsome sort of indication, the credential helper doesn't know if it should\ntry to provide us a username and password, or a pre-encoded credential.\nFor example, the helper might prefer a more restricted Bearer token if\npre-encoded credentials are possible, but might have to fall back to\nmore general username and password if not.\n\nLet's provide a simple way to indicate whether Git (or, for that matter,\nthe helper) is capable of understanding the authtype and credential\nfields. We send this capability when we generate a request, and the\nother side may reply to indicate to us that it does, too.\n\nFor now, don't enable sending capabilities for the HTTP code. In a\nfuture commit, we'll introduce appropriate handling for that code,\nwhich requires more in-depth work.\n\nThe logic for determining whether a capability is supported may seem\ncomplex, but it is not. At each stage, we emit the capability to the\nfollowing stage if all preceding stages have declared it. Thus, if the\ncaller to git credential fill didn't declare it, then we won't send it\nto the helper, and if fill's caller did send but the helper doesn't\nunderstand it, then we won't send it on in the response. If we're an\ninternal user, then we know about all capabilities and will request\nthem.\n\nFor \"git credential approve\" and \"git credential reject\", we set the\nhelper capability before calling the helper, since we assume that the\ninput we're getting from the external program comes from a previous call\nto \"git credential fill\", and thus we'll invoke send a capability to the\nhelper if and only if we got one from the standard input, which is the\ncorrect behavior.\n\nSigned-off-by: brian m. carlson \n---\n builtin/credential-cache--daemon.c | 2 +-\n builtin/credential-store.c | 2 +-\n builtin/credential.c | 8 ++-\n credential.c | 64 +++++++++++++++--\n credential.h | 40 ++++++++++-\n http.c | 10 +--\n imap-send.c | 2 +-\n remote-curl.c | 4 +-\n t/t0300-credentials.sh | 107 ++++++++++++++++++++++++++++-\n 9 files changed, 215 insertions(+), 24 deletions(-)\n\ndiff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c\nindex 3a6a750a8e..ccbcf99ac1 100644\n--- a/builtin/credential-cache--daemon.c\n+++ b/builtin/credential-cache--daemon.c\n@@ -115,7 +115,7 @@ static int read_request(FILE *fh, struct credential *c,\n \t\treturn error(\"client sent bogus timeout line: %s\", item.buf);\n \t*timeout = atoi(p);\n \n-\tif (credential_read(c, fh) < 0)\n+\tif (credential_read(c, fh, CREDENTIAL_OP_HELPER) < 0)\n \t\treturn -1;\n \treturn 0;\n }\ndiff --git a/builtin/credential-store.c b/builtin/credential-store.c\nindex 4a492411bb..494c809332 100644\n--- a/builtin/credential-store.c\n+++ b/builtin/credential-store.c\n@@ -205,7 +205,7 @@ int cmd_credential_store(int argc, const char **argv, const char *prefix)\n \tif (!fns.nr)\n \t\tdie(\"unable to set up default path; use --file\");\n \n-\tif (credential_read(&c, stdin) < 0)\n+\tif (credential_read(&c, stdin, CREDENTIAL_OP_HELPER) < 0)\n \t\tdie(\"unable to read credential\");\n \n \tif (!strcmp(op, \"get\"))\ndiff --git a/builtin/credential.c b/builtin/credential.c\nindex 7010752987..643bf0b5e4 100644\n--- a/builtin/credential.c\n+++ b/builtin/credential.c\n@@ -17,15 +17,17 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n \t\tusage(usage_msg);\n \top = argv[1];\n \n-\tif (credential_read(&c, stdin) < 0)\n+\tif (credential_read(&c, stdin, CREDENTIAL_OP_INITIAL) < 0)\n \t\tdie(\"unable to read credential from stdin\");\n \n \tif (!strcmp(op, \"fill\")) {\n-\t\tcredential_fill(&c);\n-\t\tcredential_write(&c, stdout);\n+\t\tcredential_fill(&c, 0);\n+\t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n \t} else if (!strcmp(op, \"approve\")) {\n+\t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);\n \t\tcredential_approve(&c);\n \t} else if (!strcmp(op, \"reject\")) {\n+\t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);\n \t\tcredential_reject(&c);\n \t} else {\n \t\tusage(usage_msg);\ndiff --git a/credential.c b/credential.c\nindex c521822e5a..f5396629df 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -34,6 +34,29 @@ void credential_clear(struct credential *c)\n \tcredential_init(c);\n }\n \n+static void credential_set_capability(struct credential_capability *capa,\n+\t\t\t\t enum credential_op_type op_type)\n+{\n+\tswitch (op_type) {\n+\tcase CREDENTIAL_OP_INITIAL:\n+\t\tcapa->request_initial = 1;\n+\t\tbreak;\n+\tcase CREDENTIAL_OP_HELPER:\n+\t\tcapa->request_helper = 1;\n+\t\tbreak;\n+\tcase CREDENTIAL_OP_RESPONSE:\n+\t\tcapa->response = 1;\n+\t\tbreak;\n+\t}\n+}\n+\n+\n+void credential_set_all_capabilities(struct credential *c,\n+\t\t\t\t enum credential_op_type op_type)\n+{\n+\tcredential_set_capability(&c->capa_authtype, op_type);\n+}\n+\n int credential_match(const struct credential *want,\n \t\t const struct credential *have, int match_password)\n {\n@@ -210,7 +233,26 @@ static void credential_getpass(struct credential *c)\n \t\t\t\t\t\t PROMPT_ASKPASS);\n }\n \n-int credential_read(struct credential *c, FILE *fp)\n+static int credential_has_capability(const struct credential_capability *capa,\n+\t\t\t\t enum credential_op_type op_type)\n+{\n+\t/*\n+\t * We're checking here if each previous step indicated that we had the\n+\t * capability. If it did, then we want to pass it along; conversely, if\n+\t * it did not, we don't want to report that to our caller.\n+\t */\n+\tswitch (op_type) {\n+\tcase CREDENTIAL_OP_HELPER:\n+\t\treturn capa->request_initial;\n+\tcase CREDENTIAL_OP_RESPONSE:\n+\t\treturn capa->request_initial && capa->request_helper;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+}\n+\n+int credential_read(struct credential *c, FILE *fp,\n+\t\t enum credential_op_type op_type)\n {\n \tstruct strbuf line = STRBUF_INIT;\n \n@@ -249,6 +291,8 @@ int credential_read(struct credential *c, FILE *fp)\n \t\t\tc->path = xstrdup(value);\n \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n \t\t\tstrvec_push(&c->wwwauth_headers, value);\n+\t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n+\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n@@ -288,14 +332,19 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,\n \tfprintf(fp, \"%s=%s\\n\", key, value);\n }\n \n-void credential_write(const struct credential *c, FILE *fp)\n+void credential_write(const struct credential *c, FILE *fp,\n+\t\t enum credential_op_type op_type)\n {\n+\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n+\t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n+\t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n+\t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n+\t}\n \tcredential_write_item(fp, \"protocol\", c->protocol, 1);\n \tcredential_write_item(fp, \"host\", c->host, 1);\n \tcredential_write_item(fp, \"path\", c->path, 0);\n \tcredential_write_item(fp, \"username\", c->username, 0);\n \tcredential_write_item(fp, \"password\", c->password, 0);\n-\tcredential_write_item(fp, \"credential\", c->credential, 0);\n \tcredential_write_item(fp, \"oauth_refresh_token\", c->oauth_refresh_token, 0);\n \tif (c->password_expiry_utc != TIME_MAX) {\n \t\tchar *s = xstrfmt(\"%\"PRItime, c->password_expiry_utc);\n@@ -304,7 +353,6 @@ void credential_write(const struct credential *c, FILE *fp)\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n-\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n }\n \n static int run_credential_helper(struct credential *c,\n@@ -327,14 +375,14 @@ static int run_credential_helper(struct credential *c,\n \n \tfp = xfdopen(helper.in, \"w\");\n \tsigchain_push(SIGPIPE, SIG_IGN);\n-\tcredential_write(c, fp);\n+\tcredential_write(c, fp, want_output ? CREDENTIAL_OP_HELPER : CREDENTIAL_OP_RESPONSE);\n \tfclose(fp);\n \tsigchain_pop(SIGPIPE);\n \n \tif (want_output) {\n \t\tint r;\n \t\tfp = xfdopen(helper.out, \"r\");\n-\t\tr = credential_read(c, fp);\n+\t\tr = credential_read(c, fp, CREDENTIAL_OP_HELPER);\n \t\tfclose(fp);\n \t\tif (r < 0) {\n \t\t\tfinish_command(&helper);\n@@ -367,7 +415,7 @@ static int credential_do(struct credential *c, const char *helper,\n \treturn r;\n }\n \n-void credential_fill(struct credential *c)\n+void credential_fill(struct credential *c, int all_capabilities)\n {\n \tint i;\n \n@@ -375,6 +423,8 @@ void credential_fill(struct credential *c)\n \t\treturn;\n \n \tcredential_apply_config(c);\n+\tif (all_capabilities)\n+\t\tcredential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);\n \n \tfor (i = 0; i < c->helpers.nr; i++) {\n \t\tcredential_do(c, c->helpers.items[i].string, \"get\");\ndiff --git a/credential.h b/credential.h\nindex 9db892cf4d..b524fdba59 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -93,6 +93,27 @@\n * -----------------------------------------------------------------------\n */\n \n+/*\n+ * These values define the kind of operation we're performing and the\n+ * capabilities at each stage. The first is either an external request (via git\n+ * credential fill) or an internal request (e.g., via the HTTP) code. The\n+ * second is the call to the credential helper, and the third is the response\n+ * we're providing.\n+ *\n+ * At each stage, we will emit the capability only if the previous stage\n+ * supported it.\n+ */\n+enum credential_op_type {\n+\tCREDENTIAL_OP_INITIAL = 1,\n+\tCREDENTIAL_OP_HELPER = 2,\n+\tCREDENTIAL_OP_RESPONSE = 3,\n+};\n+\n+struct credential_capability {\n+\tunsigned request_initial:1,\n+\t\t request_helper:1,\n+\t\t response:1;\n+};\n \n /**\n * This struct represents a single username/password combination\n@@ -136,6 +157,8 @@ struct credential {\n \t\t use_http_path:1,\n \t\t username_from_proto:1;\n \n+\tstruct credential_capability capa_authtype;\n+\n \tchar *username;\n \tchar *password;\n \tchar *credential;\n@@ -174,8 +197,11 @@ void credential_clear(struct credential *);\n * returns, the username and password fields of the credential are\n * guaranteed to be non-NULL. If an error occurs, the function will\n * die().\n+ *\n+ * If all_capabilities is set, this is an internal user that is prepared\n+ * to deal with all known capabilities, and we should advertise that fact.\n */\n-void credential_fill(struct credential *);\n+void credential_fill(struct credential *, int all_capabilities);\n \n /**\n * Inform the credential subsystem that the provided credentials\n@@ -198,8 +224,16 @@ void credential_approve(struct credential *);\n */\n void credential_reject(struct credential *);\n \n-int credential_read(struct credential *, FILE *);\n-void credential_write(const struct credential *, FILE *);\n+/**\n+ * Enable all of the supported credential flags in this credential.\n+ */\n+void credential_set_all_capabilities(struct credential *c,\n+\t\t\t\t enum credential_op_type op_type);\n+\n+int credential_read(struct credential *, FILE *,\n+\t\t enum credential_op_type);\n+void credential_write(const struct credential *, FILE *,\n+\t\t enum credential_op_type);\n \n /*\n * Parse a url into a credential struct, replacing any existing contents.\ndiff --git a/http.c b/http.c\nindex 22639b0974..54ddff03fb 100644\n--- a/http.c\n+++ b/http.c\n@@ -569,7 +569,7 @@ static void init_curl_http_auth(CURL *result)\n \t\treturn;\n \t}\n \n-\tcredential_fill(&http_auth);\n+\tcredential_fill(&http_auth, 0);\n \n \tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n \tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n@@ -596,7 +596,7 @@ static void init_curl_proxy_auth(CURL *result)\n {\n \tif (proxy_auth.username) {\n \t\tif (!proxy_auth.password)\n-\t\t\tcredential_fill(&proxy_auth);\n+\t\t\tcredential_fill(&proxy_auth, 0);\n \t\tset_proxyauth_name_password(result);\n \t}\n \n@@ -630,7 +630,7 @@ static int has_cert_password(void)\n \t\tcert_auth.host = xstrdup(\"\");\n \t\tcert_auth.username = xstrdup(\"\");\n \t\tcert_auth.path = xstrdup(ssl_cert);\n-\t\tcredential_fill(&cert_auth);\n+\t\tcredential_fill(&cert_auth, 0);\n \t}\n \treturn 1;\n }\n@@ -645,7 +645,7 @@ static int has_proxy_cert_password(void)\n \t\tproxy_cert_auth.host = xstrdup(\"\");\n \t\tproxy_cert_auth.username = xstrdup(\"\");\n \t\tproxy_cert_auth.path = xstrdup(http_proxy_ssl_cert);\n-\t\tcredential_fill(&proxy_cert_auth);\n+\t\tcredential_fill(&proxy_cert_auth, 0);\n \t}\n \treturn 1;\n }\n@@ -2190,7 +2190,7 @@ static int http_request_reauth(const char *url,\n \t\tBUG(\"Unknown http_request target\");\n \t}\n \n-\tcredential_fill(&http_auth);\n+\tcredential_fill(&http_auth, 0);\n \n \treturn http_request(url, result, target, options);\n }\ndiff --git a/imap-send.c b/imap-send.c\nindex f2e1947e63..8c89e866b6 100644\n--- a/imap-send.c\n+++ b/imap-send.c\n@@ -944,7 +944,7 @@ static void server_fill_credential(struct imap_server_conf *srvc, struct credent\n \tcred->username = xstrdup_or_null(srvc->user);\n \tcred->password = xstrdup_or_null(srvc->pass);\n \n-\tcredential_fill(cred);\n+\tcredential_fill(cred, 1);\n \n \tif (!srvc->user)\n \t\tsrvc->user = xstrdup(cred->username);\ndiff --git a/remote-curl.c b/remote-curl.c\nindex e37eaa17b7..f96bda2431 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -926,7 +926,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \t\tdo {\n \t\t\terr = probe_rpc(rpc, &results);\n \t\t\tif (err == HTTP_REAUTH)\n-\t\t\t\tcredential_fill(&http_auth);\n+\t\t\t\tcredential_fill(&http_auth, 0);\n \t\t} while (err == HTTP_REAUTH);\n \t\tif (err != HTTP_OK)\n \t\t\treturn -1;\n@@ -1044,7 +1044,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \trpc->any_written = 0;\n \terr = run_slot(slot, NULL);\n \tif (err == HTTP_REAUTH && !large_request) {\n-\t\tcredential_fill(&http_auth);\n+\t\tcredential_fill(&http_auth, 0);\n \t\tcurl_slist_free_all(headers);\n \t\tgoto retry;\n \t}\ndiff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\nindex 400f6bdbca..daf330ddd8 100755\n--- a/t/t0300-credentials.sh\n+++ b/t/t0300-credentials.sh\n@@ -12,7 +12,13 @@ test_expect_success 'setup helper scripts' '\n \tIFS==\n \twhile read key value; do\n \t\techo >&2 \"$whoami: $key=$value\"\n-\t\teval \"$key=$value\"\n+\t\tif test -z \"${key%%*\\[\\]}\"\n+\t\tthen\n+\t\t\tkey=${key%%\\[\\]}\n+\t\t\teval \"$key=\\\"\\$$key $value\\\"\"\n+\t\telse\n+\t\t\teval \"$key=$value\"\n+\t\tfi\n \tdone\n \tIFS=$OIFS\n \tEOF\n@@ -35,6 +41,16 @@ test_expect_success 'setup helper scripts' '\n \ttest -z \"$pass\" || echo password=$pass\n \tEOF\n \n+\twrite_script git-credential-verbatim-cred <<-\\EOF &&\n+\tauthtype=$1; shift\n+\tcredential=$1; shift\n+\t. ./dump\n+\techo capability[]=authtype\n+\ttest -z \"${capability##*authtype*}\" || exit 0\n+\ttest -z \"$authtype\" || echo authtype=$authtype\n+\ttest -z \"$credential\" || echo credential=$credential\n+\tEOF\n+\n \twrite_script git-credential-verbatim-with-expiry <<-\\EOF &&\n \tuser=$1; shift\n \tpass=$1; shift\n@@ -64,6 +80,26 @@ test_expect_success 'credential_fill invokes helper' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill invokes helper with credential' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=token\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: capability[]=authtype\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n+\n test_expect_success 'credential_fill invokes multiple helpers' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n \tprotocol=http\n@@ -83,6 +119,42 @@ test_expect_success 'credential_fill invokes multiple helpers' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '\n+\tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tprotocol=http\n+\thost=example.com\n+\tusername=foo\n+\tpassword=bar\n+\t--\n+\tuseless: get\n+\tuseless: capability[]=authtype\n+\tuseless: protocol=http\n+\tuseless: host=example.com\n+\tverbatim: get\n+\tverbatim: capability[]=authtype\n+\tverbatim: protocol=http\n+\tverbatim: host=example.com\n+\tEOF\n+'\n+\n+test_expect_success 'credential_fill response does not get capabilities when caller is incapable' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n test_expect_success 'credential_fill stops when we get a full response' '\n \tcheck fill \"verbatim one two\" \"verbatim three four\" <<-\\EOF\n \tprotocol=http\n@@ -99,6 +171,25 @@ test_expect_success 'credential_fill stops when we get a full response' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill thinks a credential is a full response' '\n+\tcheck fill \"verbatim-cred Bearer token\" \"verbatim three four\" <<-\\EOF\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=token\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: capability[]=authtype\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n test_expect_success 'credential_fill continues through partial response' '\n \tcheck fill \"verbatim one \\\"\\\"\" \"verbatim two three\" <<-\\EOF\n \tprotocol=http\n@@ -175,6 +266,20 @@ test_expect_success 'credential_fill passes along metadata' '\n \tEOF\n '\n \n+test_expect_success 'credential_fill produces no credential without capability' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n+\n test_expect_success 'credential_approve calls all helpers' '\n \tcheck approve useless \"verbatim one two\" <<-\\EOF\n \tprotocol=http\n"},{"id":"493032","messageId":"20240417000240.3611948-8-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 07/16] docs: indicate new credential protocol fields","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:31Z","receivedAt":"2024-04-17T00:03:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we have new fields (authtype and credential), let's document\nthem for users and credential helper implementers.\n\nIndicate specifically what common values of authtype are and what values\nare allowed. Note that, while common, digest and NTLM authentication\nare insecure because they require unsalted, uniterated password hashes\nto be stored.\n\nTell users that they can continue to use a username and password even if\nthe new capability is supported.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 49 +++++++++++++++++++++++++++++++-\n 1 file changed, 48 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex 918a0aa42b..230ac4c2c3 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -178,6 +178,39 @@ empty string.\n Components which are missing from the URL (e.g., there is no\n username in the example above) will be left unset.\n \n+`authtype`::\n+\tThis indicates that the authentication scheme in question should be used.\n+\tCommon values for HTTP and HTTPS include `basic`, `bearer`, and `digest`,\n+\talthough the latter is insecure and should not be used. If `credential`\n+\tis used, this may be set to an arbitrary string suitable for the protocol in\n+\tquestion (usually HTTP).\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n+`credential`::\n+\tThe pre-encoded credential, suitable for the protocol in question (usually\n+\tHTTP). If this key is sent, `authtype` is mandatory, and `username` and\n+\t`password` are not used. For HTTP, Git concatenates the `authtype` value and\n+\tthis value with a single space to determine the `Authorization` header.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n+`ephemeral`::\n+\tThis boolean value indicates, if true, that the value in the `credential`\n+\tfield should not be saved by the credential helper because its usefulness is\n+\tlimited in time. For example, an HTTP Digest `credential` value is computed\n+\tusing a nonce and reusing it will not result in successful authentication.\n+\tThis may also be used for situations with short duration (e.g., 24-hour)\n+\tcredentials. The default value is false.\n++\n+The credential helper will still be invoked with `store` or `erase` so that it\n+can determine whether the operation was successful.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\n@@ -189,7 +222,21 @@ attribute 'wwwauth[]', where the order of the attributes is the same as\n they appear in the HTTP response. This attribute is 'one-way' from Git\n to pass additional information to credential helpers.\n \n-Unrecognised attributes are silently discarded.\n+`capability[]`::\n+\tThis signals that the caller supports the capability in question.\n+\tThis can be used to provide better, more specific data as part of the\n+\tprotocol.\n++\n+The only capability currently supported is `authtype`, which indicates that the\n+`authtype`, `credential`, and `ephemeral` values are understood. It is not\n+obligatory to use these values in such a case, but they should not be provided\n+without this capability.\n++\n+Callers of `git credential` and credential helpers should emit the\n+capabilities they support unconditionally, and Git will gracefully\n+handle passing them on.\n+\n+Unrecognised attributes and capabilities are silently discarded.\n \n GIT\n ---\n"},{"id":"493034","messageId":"20240417000240.3611948-12-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 11/16] docs: set a limit on credential line length","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:35Z","receivedAt":"2024-04-17T00:03:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We recently introduced a way for credential helpers to add arbitrary\nstate as part of the protocol. Set some limits on line length to avoid\nhelpers passing extremely large amounts of data. While Git doesn't have\na fixed parsing length, there are other tools which support this\nprotocol and it's kind to allow them to use a reasonable fixed-size\nbuffer for parsing. In addition, we would like to be moderate in our\nmemory usage and imposing reasonable limits is helpful for that purpose.\n\nIn the event a credential helper is incapable of storing its serialized\nstate in 64 KiB, it can feel free to serialize it on disk and store a\nreference instead.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 4 +++-\n 1 file changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex f63a8e0458..4bbf2db9ca 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -111,7 +111,9 @@ attribute per line. Each attribute is specified by a key-value pair,\n separated by an `=` (equals) sign, followed by a newline.\n \n The key may contain any bytes except `=`, newline, or NUL. The value may\n-contain any bytes except newline or NUL.\n+contain any bytes except newline or NUL. A line, including the trailing\n+newline, may not exceed 65535 bytes in order to allow implementations to\n+parse efficiently.\n \n Attributes with keys that end with C-style array brackets `[]` can have\n multiple values. Each instance of a multi-valued attribute forms an\n"},{"id":"493035","messageId":"20240417000240.3611948-10-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 09/16] credential: add an argument to keep state","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:33Z","receivedAt":"2024-04-17T00:03:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Until now, our credential code has mostly deal with usernames and\npasswords and we've let libcurl deal with the variant of authentication\nto be used. However, now that we have the credential value, the\ncredential helper can take control of the authentication, so the value\nprovided might be something that's generated, such as a Digest hash\nvalue.\n\nIn such a case, it would be helpful for a credential helper that gets an\nerase or store command to be able to keep track of an identifier for the\noriginal secret that went into the computation. Furthermore, some types\nof authentication, such as NTLM and Kerberos, actually need two round\ntrips to authenticate, which will require that the credential helper\nkeep some state.\n\nIn order to allow for these use cases and others, allow storing state in\na field called \"state[]\". This value is passed back to the credential\nhelper that created it, which avoids confusion caused by parsing values\nfrom different helpers.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 28 +++++++++++++++++++---------\n credential.c | 20 +++++++++++++++++---\n credential.h | 7 +++++++\n t/t0300-credentials.sh | 28 ++++++++++++++++++++++++++++\n 4 files changed, 71 insertions(+), 12 deletions(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex 230ac4c2c3..f63a8e0458 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -211,6 +211,15 @@ can determine whether the operation was successful.\n This value should not be sent unless the appropriate capability (see below) is\n provided on input.\n \n+`state[]`::\n+\tThis value provides an opaque state that will be passed back to this helper\n+\tif it is called again. Each different credential helper may specify this\n+\tonce. The value should include a prefix unique to the credential helper and\n+\tshould ignore values that don't match its prefix.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input.\n+\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\n@@ -223,18 +232,19 @@ they appear in the HTTP response. This attribute is 'one-way' from Git\n to pass additional information to credential helpers.\n \n `capability[]`::\n-\tThis signals that the caller supports the capability in question.\n-\tThis can be used to provide better, more specific data as part of the\n+\tThis signals that Git, or the helper, as appropriate, supports the capability\n+\tin question. This can be used to provide better, more specific data as part\n+\tof the protocol. A `capability[]` directive must precede any value depending\n+\ton it and these directives _should_ be the first item announced in the\n \tprotocol.\n +\n-The only capability currently supported is `authtype`, which indicates that the\n-`authtype`, `credential`, and `ephemeral` values are understood. It is not\n-obligatory to use these values in such a case, but they should not be provided\n-without this capability.\n+There are two currently supported capabilities. The first is `authtype`, which\n+indicates that the `authtype`, `credential`, and `ephemeral` values are\n+understood. The second is `state`, which indicates that the `state[]` and\n+`continue` values are understood.\n +\n-Callers of `git credential` and credential helpers should emit the\n-capabilities they support unconditionally, and Git will gracefully\n-handle passing them on.\n+It is not obligatory to use the additional features just because the capability\n+is supported, but they should not be provided without the capability.\n \n Unrecognised attributes and capabilities are silently discarded.\n \ndiff --git a/credential.c b/credential.c\nindex 3531d74346..48826fb5a2 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -30,6 +30,7 @@ void credential_clear(struct credential *c)\n \tfree(c->authtype);\n \tstring_list_clear(&c->helpers, 0);\n \tstrvec_clear(&c->wwwauth_headers);\n+\tstrvec_clear(&c->state_headers);\n \n \tcredential_init(c);\n }\n@@ -293,8 +294,13 @@ int credential_read(struct credential *c, FILE *fp,\n \t\t\tc->ephemeral = !!git_config_bool(\"ephemeral\", value);\n \t\t} else if (!strcmp(key, \"wwwauth[]\")) {\n \t\t\tstrvec_push(&c->wwwauth_headers, value);\n-\t\t} else if (!strcmp(key, \"capability[]\") && !strcmp(value, \"authtype\")) {\n-\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n+\t\t} else if (!strcmp(key, \"state[]\")) {\n+\t\t\tstrvec_push(&c->state_headers, value);\n+\t\t} else if (!strcmp(key, \"capability[]\")) {\n+\t\t\tif (!strcmp(value, \"authtype\"))\n+\t\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n+\t\t\telse if (!strcmp(value, \"state\"))\n+\t\t\t\tcredential_set_capability(&c->capa_state, op_type);\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n@@ -337,8 +343,12 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,\n void credential_write(const struct credential *c, FILE *fp,\n \t\t enum credential_op_type op_type)\n {\n-\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n+\tif (credential_has_capability(&c->capa_authtype, op_type))\n \t\tcredential_write_item(fp, \"capability[]\", \"authtype\", 0);\n+\tif (credential_has_capability(&c->capa_state, op_type))\n+\t\tcredential_write_item(fp, \"capability[]\", \"state\", 0);\n+\n+\tif (credential_has_capability(&c->capa_authtype, op_type)) {\n \t\tcredential_write_item(fp, \"authtype\", c->authtype, 0);\n \t\tcredential_write_item(fp, \"credential\", c->credential, 0);\n \t\tif (c->ephemeral)\n@@ -357,6 +367,10 @@ void credential_write(const struct credential *c, FILE *fp,\n \t}\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n+\tif (credential_has_capability(&c->capa_state, op_type)) {\n+\t\tfor (size_t i = 0; i < c->state_headers.nr; i++)\n+\t\t\tcredential_write_item(fp, \"state[]\", c->state_headers.v[i], 0);\n+\t}\n }\n \n static int run_credential_helper(struct credential *c,\ndiff --git a/credential.h b/credential.h\nindex da2a4802b7..c307300d12 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -144,6 +144,11 @@ struct credential {\n \t */\n \tstruct strvec wwwauth_headers;\n \n+\t/**\n+\t * A `strvec` of state headers from credential helpers.\n+\t */\n+\tstruct strvec state_headers;\n+\n \t/**\n \t * Internal use only. Keeps track of if we previously matched against a\n \t * WWW-Authenticate header line in order to re-fold future continuation\n@@ -159,6 +164,7 @@ struct credential {\n \t\t username_from_proto:1;\n \n \tstruct credential_capability capa_authtype;\n+\tstruct credential_capability capa_state;\n \n \tchar *username;\n \tchar *password;\n@@ -180,6 +186,7 @@ struct credential {\n \t.helpers = STRING_LIST_INIT_DUP, \\\n \t.password_expiry_utc = TIME_MAX, \\\n \t.wwwauth_headers = STRVEC_INIT, \\\n+\t.state_headers = STRVEC_INIT, \\\n }\n \n /* Initialize a credential structure, setting all fields to empty. */\ndiff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh\nindex eceb6bbfbe..432f029d48 100755\n--- a/t/t0300-credentials.sh\n+++ b/t/t0300-credentials.sh\n@@ -46,9 +46,12 @@ test_expect_success 'setup helper scripts' '\n \tcredential=$1; shift\n \t. ./dump\n \techo capability[]=authtype\n+\techo capability[]=state\n \ttest -z \"${capability##*authtype*}\" || exit 0\n \ttest -z \"$authtype\" || echo authtype=$authtype\n \ttest -z \"$credential\" || echo credential=$credential\n+\ttest -z \"${capability##*state*}\" || exit 0\n+\techo state[]=verbatim-cred:foo\n \tEOF\n \n \twrite_script git-credential-verbatim-ephemeral <<-\\EOF &&\n@@ -129,6 +132,28 @@ test_expect_success 'credential_fill invokes helper with ephemeral credential' '\n \tverbatim-ephemeral: host=example.com\n \tEOF\n '\n+test_expect_success 'credential_fill invokes helper with credential and state' '\n+\tcheck fill \"verbatim-cred Bearer token\" <<-\\EOF\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tprotocol=http\n+\thost=example.com\n+\t--\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Bearer\n+\tcredential=token\n+\tprotocol=http\n+\thost=example.com\n+\tstate[]=verbatim-cred:foo\n+\t--\n+\tverbatim-cred: get\n+\tverbatim-cred: capability[]=authtype\n+\tverbatim-cred: capability[]=state\n+\tverbatim-cred: protocol=http\n+\tverbatim-cred: host=example.com\n+\tEOF\n+'\n \n test_expect_success 'credential_fill invokes multiple helpers' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n@@ -152,6 +177,7 @@ test_expect_success 'credential_fill invokes multiple helpers' '\n test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '\n \tcheck fill useless \"verbatim foo bar\" <<-\\EOF\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=example.com\n \t--\n@@ -162,10 +188,12 @@ test_expect_success 'credential_fill response does not get capabilities when hel\n \t--\n \tuseless: get\n \tuseless: capability[]=authtype\n+\tuseless: capability[]=state\n \tuseless: protocol=http\n \tuseless: host=example.com\n \tverbatim: get\n \tverbatim: capability[]=authtype\n+\tverbatim: capability[]=state\n \tverbatim: protocol=http\n \tverbatim: host=example.com\n \tEOF\n"},{"id":"493036","messageId":"20240417000240.3611948-11-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 10/16] credential: enable state capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:34Z","receivedAt":"2024-04-17T00:03:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we've implemented the state capability, let's send it along by\ndefault when filling credentials so we can make use of it.\n\nSigned-off-by: brian m. carlson \n---\n credential.c | 1 +\n t/t5563-simple-http-auth.sh | 10 ++++++++++\n 2 files changed, 11 insertions(+)\n\ndiff --git a/credential.c b/credential.c\nindex 48826fb5a2..c93de92f65 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -56,6 +56,7 @@ void credential_set_all_capabilities(struct credential *c,\n \t\t\t\t enum credential_op_type op_type)\n {\n \tcredential_set_capability(&c->capa_authtype, op_type);\n+\tcredential_set_capability(&c->capa_state, op_type);\n }\n \n int credential_match(const struct credential *want,\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex b3ed0d9fc2..b098cd0fdf 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -75,6 +75,7 @@ test_expect_success 'access using basic auth' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -111,6 +112,7 @@ test_expect_success 'access using basic auth via authtype' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -147,6 +149,7 @@ test_expect_success 'access using basic auth invalid credentials' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -185,6 +188,7 @@ test_expect_success 'access using basic auth with extra challenges' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -224,6 +228,7 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=foobar param1=\"value1\" param2=\"value2\"\n@@ -268,6 +273,7 @@ test_expect_success 'access using basic auth with wwwauth header continuations'\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -314,6 +320,7 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -356,6 +363,7 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -397,6 +405,7 @@ test_expect_success 'access using bearer auth' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -440,6 +449,7 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \n \texpect_credential_query get <<-EOF &&\n \tcapability[]=authtype\n+\tcapability[]=state\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n"},{"id":"493037","messageId":"20240417000240.3611948-13-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 12/16] t5563: refactor for multi-stage authentication","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:36Z","receivedAt":"2024-04-17T00:03:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Some HTTP authentication schemes, such as NTLM- and Kerberos-based\noptions, require more than one round trip to authenticate. Currently,\nthese can only be supported in libcurl, since Git does not have support\nfor this in the credential helper protocol.\n\nHowever, in a future commit, we'll add support for this functionality\ninto the credential helper protocol and Git itself. Because we don't\nreally want to implement either NTLM or Kerberos, both of which are\ncomplex protocols, we'll want to test this using a fake credential\nauthentication scheme. In order to do so, update t5563 and its backend\nto allow us to accept multiple sets of credentials and respond with\ndifferent behavior in each case.\n\nSince we can now provide any number of possible status codes, provide a\nnon-specific reason phrase so we don't have to generate a more specific\none based on the response. The reason phrase is mandatory according to\nthe status-line production in RFC 7230, but clients SHOULD ignore it,\nand curl does (except to print it).\n\nEach entry in the authorization and challenge fields contains an ID,\nwhich indicates a corresponding credential and response. If the\nresponse is a 200 status, then we continue to execute git-http-backend.\nOtherwise, we print the corresponding status and response. If no ID is\nmatched, we use the default response with a status of 401.\n\nNote that there is an implicit order to the parameters. The ID is\nalways first and the creds or response value is always last, and\ntherefore may contain spaces, equals signs, or other arbitrary data.\n\nSigned-off-by: brian m. carlson \n---\n t/lib-httpd/nph-custom-auth.sh | 17 ++++--\n t/t5563-simple-http-auth.sh | 96 +++++++++++++++++++---------------\n 2 files changed, 66 insertions(+), 47 deletions(-)\n\ndiff --git a/t/lib-httpd/nph-custom-auth.sh b/t/lib-httpd/nph-custom-auth.sh\nindex f5345e775e..d408d2caad 100644\n--- a/t/lib-httpd/nph-custom-auth.sh\n+++ b/t/lib-httpd/nph-custom-auth.sh\n@@ -19,21 +19,30 @@ CHALLENGE_FILE=custom-auth.challenge\n #\n \n if test -n \"$HTTP_AUTHORIZATION\" && \\\n-\tgrep -Fqsx \"${HTTP_AUTHORIZATION}\" \"$VALID_CREDS_FILE\"\n+\tgrep -Fqs \"creds=${HTTP_AUTHORIZATION}\" \"$VALID_CREDS_FILE\"\n then\n+\tidno=$(grep -F \"creds=${HTTP_AUTHORIZATION}\" \"$VALID_CREDS_FILE\" | sed -e 's/^id=\\([a-z0-9-][a-z0-9-]*\\) .*$/\\1/')\n+\tstatus=$(sed -ne \"s/^id=$idno.*status=\\\\([0-9][0-9][0-9]\\\\).*\\$/\\\\1/p\" \"$CHALLENGE_FILE\" | head -n1)\n \t# Note that although git-http-backend returns a status line, it\n \t# does so using a CGI 'Status' header. Because this script is an\n \t# No Parsed Headers (NPH) script, we must return a real HTTP\n \t# status line.\n \t# This is only a test script, so we don't bother to check for\n \t# the actual status from git-http-backend and always return 200.\n-\techo 'HTTP/1.1 200 OK'\n-\texec \"$GIT_EXEC_PATH\"/git-http-backend\n+\techo \"HTTP/1.1 $status Nonspecific Reason Phrase\"\n+\tif test \"$status\" -eq 200\n+\tthen\n+\t\texec \"$GIT_EXEC_PATH\"/git-http-backend\n+\telse\n+\t\tsed -ne \"s/^id=$idno.*response=//p\" \"$CHALLENGE_FILE\"\n+\t\techo\n+\t\texit\n+\tfi\n fi\n \n echo 'HTTP/1.1 401 Authorization Required'\n if test -f \"$CHALLENGE_FILE\"\n then\n-\tcat \"$CHALLENGE_FILE\"\n+\tsed -ne 's/^id=default.*response=//p' \"$CHALLENGE_FILE\"\n fi\n echo\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex b098cd0fdf..515185ae00 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -63,11 +63,12 @@ test_expect_success 'access using basic auth' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -100,11 +101,12 @@ test_expect_success 'access using basic auth via authtype' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -137,11 +139,12 @@ test_expect_success 'access using basic auth invalid credentials' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -174,13 +177,14 @@ test_expect_success 'access using basic auth with extra challenges' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -214,13 +218,14 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\twww-authenticate: foobar param1=\"value1\" param2=\"value2\"\n-\tWWW-AUTHENTICATE: BEARER authorize_uri=\"id.example.com\" p=1 q=0\n-\tWwW-aUtHeNtIcAtE: baSiC realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=www-authenticate: foobar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-AUTHENTICATE: BEARER authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WwW-aUtHeNtIcAtE: baSiC realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -254,18 +259,19 @@ test_expect_success 'access using basic auth with wwwauth header continuations'\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \t# Note that leading and trailing whitespace is important to correctly\n \t# simulate a continuation/folded header.\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\"\n-\t param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\n-\t p=1\n-\t q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\"\n+\tid=default response= param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\n+\tid=default response= p=1\n+\tid=default response= q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -299,21 +305,22 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \t# Note that leading and trailing whitespace is important to correctly\n \t# simulate a continuation/folded header.\n-\tprintf \"WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >\"$CHALLENGE\" &&\n-\tprintf \" \\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" param2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"WWW-Authenticate: Bearer authorize_uri=\\\"id.example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" p=1\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" \\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \" q=0\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"WWW-Authenticate: Basic realm=\\\"example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=1 status=200\\n\" >\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= \\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= param2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: Bearer authorize_uri=\\\"id.example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= p=1\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= \\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= q=0\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: Basic realm=\\\"example.com\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n \n \ttest_config_global credential.helper test-helper &&\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n@@ -346,17 +353,18 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \n \t# Basic base64(alice:secret-passwd)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tid=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \t# Note that leading and trailing whitespace is important to correctly\n \t# simulate a continuation/folded header.\n-\tprintf \"WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >\"$CHALLENGE\" &&\n-\tprintf \" \\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"\\tparam2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n-\tprintf \"WWW-Authenticate: Basic realm=\\\"example.com\\\"\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=1 status=200\\n\" >\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: FooBar param1=\\\"value1\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response= \\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=\\tparam2=\\\"value2\\\"\\r\\n\" >>\"$CHALLENGE\" &&\n+\tprintf \"id=default response=WWW-Authenticate: Basic realm=\\\"example.com\\\"\" >>\"$CHALLENGE\" &&\n \n \ttest_config_global credential.helper test-helper &&\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n@@ -389,15 +397,16 @@ test_expect_success 'access using bearer auth' '\n \n \t# Basic base64(a-git-token)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBearer YS1naXQtdG9rZW4=\n+\tid=1 creds=Bearer YS1naXQtdG9rZW4=\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n@@ -433,15 +442,16 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \n \t# Basic base64(a-git-token)\n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n-\tBearer YS1naXQtdG9rZW4=\n+\tid=1 creds=Bearer YS1naXQtdG9rZW4=\n \tEOF\n \n \tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n \n \tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n-\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n-\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n-\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tid=1 status=200\n+\tid=default response=WWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=default response=WWW-Authenticate: Basic realm=\"example.com\"\n \tEOF\n \n \ttest_config_global credential.helper test-helper &&\n"},{"id":"493041","messageId":"20240417000240.3611948-9-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 08/16] http: add support for authtype and credential","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:32Z","receivedAt":"2024-04-17T00:03:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we have the credential helper code set up to handle arbitrary\nauthentications schemes, let's add support for this in the HTTP code,\nwhere we really want to use it. If we're using this new functionality,\ndon't set a username and password, and instead set a header wherever\nwe'd normally do so, including for proxy authentication.\n\nSince we can now handle this case, ask the credential helper to enable\nthe appropriate capabilities.\n\nFinally, if we're using the authtype value, set \"Expect: 100-continue\".\nAny type of authentication that requires multiple rounds (such as NTLM\nor Kerberos) requires a 100 Continue (if we're larger than\nhttp.postBuffer) because otherwise we send the pack data before we're\nauthenticated, the push gets a 401 response, and we can't rewind the\nstream. We don't know for certain what other custom schemes might\nrequire this, the HTTP/1.1 standard has required handling this since\n1999, the broken HTTP server for which we disabled this (Google's) is\nnow fixed and has been for some time, and libcurl has a 1-second\nfallback in case the HTTP server is still broken. In addition, it is\nnot unreasonable to require compliance with a 25-year old standard to\nuse new Git features. For all of these reasons, do so here.\n\nSigned-off-by: brian m. carlson \n---\n http.c | 48 ++++++++++---\n http.h | 3 +\n remote-curl.c | 4 +-\n t/t5563-simple-http-auth.sh | 133 ++++++++++++++++++++++++++++++++++++\n 4 files changed, 176 insertions(+), 12 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 54ddff03fb..906eb098c8 100644\n--- a/http.c\n+++ b/http.c\n@@ -561,18 +561,34 @@ static int curl_empty_auth_enabled(void)\n \treturn 0;\n }\n \n+struct curl_slist *http_append_auth_header(const struct credential *c,\n+\t\t\t\t\t struct curl_slist *headers)\n+{\n+\tif (c->authtype && c->credential) {\n+\t\tstruct strbuf auth = STRBUF_INIT;\n+\t\tstrbuf_addf(&auth, \"Authorization: %s %s\",\n+\t\t\t c->authtype, c->credential);\n+\t\theaders = curl_slist_append(headers, auth.buf);\n+\t\tstrbuf_release(&auth);\n+\t}\n+\treturn headers;\n+}\n+\n static void init_curl_http_auth(CURL *result)\n {\n-\tif (!http_auth.username || !*http_auth.username) {\n+\tif ((!http_auth.username || !*http_auth.username) &&\n+\t (!http_auth.credential || !*http_auth.credential)) {\n \t\tif (curl_empty_auth_enabled())\n \t\t\tcurl_easy_setopt(result, CURLOPT_USERPWD, \":\");\n \t\treturn;\n \t}\n \n-\tcredential_fill(&http_auth, 0);\n+\tcredential_fill(&http_auth, 1);\n \n-\tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n-\tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n+\tif (http_auth.password) {\n+\t\tcurl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);\n+\t\tcurl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);\n+\t}\n }\n \n /* *var must be free-able */\n@@ -586,17 +602,22 @@ static void var_override(const char **var, char *value)\n \n static void set_proxyauth_name_password(CURL *result)\n {\n+\tif (proxy_auth.password) {\n \t\tcurl_easy_setopt(result, CURLOPT_PROXYUSERNAME,\n \t\t\tproxy_auth.username);\n \t\tcurl_easy_setopt(result, CURLOPT_PROXYPASSWORD,\n \t\t\tproxy_auth.password);\n+\t} else if (proxy_auth.authtype && proxy_auth.credential) {\n+\t\tcurl_easy_setopt(result, CURLOPT_PROXYHEADER,\n+\t\t\t\t http_append_auth_header(&proxy_auth, NULL));\n+\t}\n }\n \n static void init_curl_proxy_auth(CURL *result)\n {\n \tif (proxy_auth.username) {\n-\t\tif (!proxy_auth.password)\n-\t\t\tcredential_fill(&proxy_auth, 0);\n+\t\tif (!proxy_auth.password && !proxy_auth.credential)\n+\t\t\tcredential_fill(&proxy_auth, 1);\n \t\tset_proxyauth_name_password(result);\n \t}\n \n@@ -1468,7 +1489,7 @@ struct active_request_slot *get_active_slot(void)\n \n \tcurl_easy_setopt(slot->curl, CURLOPT_IPRESOLVE, git_curl_ipresolve);\n \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, http_auth_methods);\n-\tif (http_auth.password || curl_empty_auth_enabled())\n+\tif (http_auth.password || http_auth.credential || curl_empty_auth_enabled())\n \t\tinit_curl_http_auth(slot->curl);\n \n \treturn slot;\n@@ -1757,7 +1778,8 @@ static int handle_curl_result(struct slot_results *results)\n \t} else if (missing_target(results))\n \t\treturn HTTP_MISSING_TARGET;\n \telse if (results->http_code == 401) {\n-\t\tif (http_auth.username && http_auth.password) {\n+\t\tif ((http_auth.username && http_auth.password) ||\\\n+\t\t (http_auth.authtype && http_auth.credential)) {\n \t\t\tcredential_reject(&http_auth);\n \t\t\treturn HTTP_NOAUTH;\n \t\t} else {\n@@ -2065,11 +2087,15 @@ static int http_request(const char *url,\n \t/* Add additional headers here */\n \tif (options && options->extra_headers) {\n \t\tconst struct string_list_item *item;\n-\t\tfor_each_string_list_item(item, options->extra_headers) {\n-\t\t\theaders = curl_slist_append(headers, item->string);\n+\t\tif (options && options->extra_headers) {\n+\t\t\tfor_each_string_list_item(item, options->extra_headers) {\n+\t\t\t\theaders = curl_slist_append(headers, item->string);\n+\t\t\t}\n \t\t}\n \t}\n \n+\theaders = http_append_auth_header(&http_auth, headers);\n+\n \tcurl_easy_setopt(slot->curl, CURLOPT_URL, url);\n \tcurl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);\n \tcurl_easy_setopt(slot->curl, CURLOPT_ENCODING, \"\");\n@@ -2190,7 +2216,7 @@ static int http_request_reauth(const char *url,\n \t\tBUG(\"Unknown http_request target\");\n \t}\n \n-\tcredential_fill(&http_auth, 0);\n+\tcredential_fill(&http_auth, 1);\n \n \treturn http_request(url, result, target, options);\n }\ndiff --git a/http.h b/http.h\nindex c5f8cc4620..a516ca4a9a 100644\n--- a/http.h\n+++ b/http.h\n@@ -175,6 +175,9 @@ int http_get_file(const char *url, const char *filename,\n \n int http_fetch_ref(const char *base, struct ref *ref);\n \n+struct curl_slist *http_append_auth_header(const struct credential *c,\n+\t\t\t\t\t struct curl_slist *headers);\n+\n /* Helpers for fetching packs */\n int http_get_info_packs(const char *base_url,\n \t\t\tstruct packed_git **packs_head);\ndiff --git a/remote-curl.c b/remote-curl.c\nindex f96bda2431..1c5416812a 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -931,7 +931,7 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \t\tif (err != HTTP_OK)\n \t\t\treturn -1;\n \n-\t\tif (results.auth_avail & CURLAUTH_GSSNEGOTIATE)\n+\t\tif (results.auth_avail & CURLAUTH_GSSNEGOTIATE || http_auth.authtype)\n \t\t\tneeds_100_continue = 1;\n \t}\n \n@@ -942,6 +942,8 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece\n \theaders = curl_slist_append(headers, needs_100_continue ?\n \t\t\"Expect: 100-continue\" : \"Expect:\");\n \n+\theaders = http_append_auth_header(&http_auth, headers);\n+\n \t/* Add Accept-Language header */\n \tif (rpc->hdr_accept_language)\n \t\theaders = curl_slist_append(headers, rpc->hdr_accept_language);\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex ab8a721ccc..b3ed0d9fc2 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -74,6 +74,7 @@ test_expect_success 'access using basic auth' '\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -87,6 +88,43 @@ test_expect_success 'access using basic auth' '\n \tEOF\n '\n \n+test_expect_success 'access using basic auth via authtype' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Basic\n+\tcredential=YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tEOF\n+\n+\t# Basic base64(alice:secret-passwd)\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tBasic YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tEOF\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\tGIT_CURL_VERBOSE=1 git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+\n+\texpect_credential_query store <<-EOF\n+\tcapability[]=authtype\n+\tauthtype=Basic\n+\tcredential=YWxpY2U6c2VjcmV0LXBhc3N3ZA==\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\tEOF\n+'\n+\n test_expect_success 'access using basic auth invalid credentials' '\n \ttest_when_finished \"per_test_cleanup\" &&\n \n@@ -108,6 +146,7 @@ test_expect_success 'access using basic auth invalid credentials' '\n \ttest_must_fail git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=Basic realm=\"example.com\"\n@@ -145,6 +184,7 @@ test_expect_success 'access using basic auth with extra challenges' '\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -183,6 +223,7 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=foobar param1=\"value1\" param2=\"value2\"\n@@ -226,6 +267,7 @@ test_expect_success 'access using basic auth with wwwauth header continuations'\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -271,6 +313,7 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -312,6 +355,7 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n \n \texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n \tprotocol=http\n \thost=$HTTPD_DEST\n \twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n@@ -326,4 +370,93 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi\n \tEOF\n '\n \n+test_expect_success 'access using bearer auth' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\t# Basic base64(a-git-token)\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tBearer YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+\n+\texpect_credential_query store <<-EOF\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=YS1naXQtdG9rZW4=\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\tEOF\n+'\n+\n+test_expect_success 'access using bearer auth with invalid credentials' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=incorrect-token\n+\tEOF\n+\n+\t# Basic base64(a-git-token)\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tBearer YS1naXQtdG9rZW4=\n+\tEOF\n+\n+\tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tWWW-Authenticate: FooBar param1=\"value1\" param2=\"value2\"\n+\tWWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tWWW-Authenticate: Basic realm=\"example.com\"\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\ttest_must_fail git ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+\n+\texpect_credential_query erase <<-EOF\n+\tcapability[]=authtype\n+\tauthtype=Bearer\n+\tcredential=incorrect-token\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=FooBar param1=\"value1\" param2=\"value2\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\twwwauth[]=Basic realm=\"example.com\"\n+\tEOF\n+'\n+\n test_done\n"},{"id":"493038","messageId":"20240417000240.3611948-15-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 14/16] t: add credential tests for authtype","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:38Z","receivedAt":"2024-04-17T00:03:05Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"It's helpful to have some basic tests for credential helpers supporting\nthe authtype and credential fields. Let's add some tests for this case\nso that we can make sure newly supported helpers work correctly.\n\nNote that we explicitly check that credential helpers can produce\ndifferent sets of authtype and credential values based on the username.\nWhile the username is not used in the HTTP protocol with authtype and\ncredential, it can still be specified in the URL and thus may be part of\nthe protocol. Additionally, because it is common for users to have\nmultiple accounts on one service (say, both personal and professional\naccounts), it's very helpful to be able to store different credentials\nfor different accounts in the same helper, and that doesn't become less\nuseful if one is using, say, Bearer authentication instead of Basic.\nThus, credential helpers should be expected to support this\nfunctionality as basic functionality, so verify here that they do so.\n\nSigned-off-by: brian m. carlson \n---\n t/lib-credential.sh | 123 ++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 123 insertions(+)\n\ndiff --git a/t/lib-credential.sh b/t/lib-credential.sh\nindex 44799c0d38..58b9c74060 100644\n--- a/t/lib-credential.sh\n+++ b/t/lib-credential.sh\n@@ -538,6 +538,129 @@ helper_test_oauth_refresh_token() {\n \t'\n }\n \n+helper_test_authtype() {\n+\tHELPER=$1\n+\n+\ttest_expect_success \"helper ($HELPER) stores authtype and credential\" '\n+\t\tcheck approve $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tauthtype=Bearer\n+\t\tcredential=random-token\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\tEOF\n+\t'\n+\n+\ttest_expect_success \"helper ($HELPER) gets authtype and credential\" '\n+\t\tcheck fill $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\t--\n+\t\tcapability[]=authtype\n+\t\tauthtype=Bearer\n+\t\tcredential=random-token\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\t--\n+\t\tEOF\n+\t'\n+\n+\ttest_expect_success \"helper ($HELPER) stores authtype and credential with username\" '\n+\t\tcheck approve $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tauthtype=Bearer\n+\t\tcredential=other-token\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\tusername=foobar\n+\t\tEOF\n+\t'\n+\n+\ttest_expect_success \"helper ($HELPER) gets authtype and credential with username\" '\n+\t\tcheck fill $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\tusername=foobar\n+\t\t--\n+\t\tcapability[]=authtype\n+\t\tauthtype=Bearer\n+\t\tcredential=other-token\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\tusername=foobar\n+\t\t--\n+\t\tEOF\n+\t'\n+\n+\ttest_expect_success \"helper ($HELPER) does not get authtype and credential with different username\" '\n+\t\tcheck fill $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\tusername=barbaz\n+\t\t--\n+\t\tprotocol=https\n+\t\thost=git.example.com\n+\t\tusername=barbaz\n+\t\tpassword=askpass-password\n+\t\t--\n+\t\taskpass: Password for '\\''https://barbaz@git.example.com'\\'':\n+\t\tEOF\n+\t'\n+\n+\ttest_expect_success \"helper ($HELPER) does not store ephemeral authtype and credential\" '\n+\t\tcheck approve $HELPER <<-\\EOF &&\n+\t\tcapability[]=authtype\n+\t\tauthtype=Bearer\n+\t\tcredential=git2-token\n+\t\tprotocol=https\n+\t\thost=git2.example.com\n+\t\tephemeral=1\n+\t\tEOF\n+\n+\t\tcheck fill $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tprotocol=https\n+\t\thost=git2.example.com\n+\t\t--\n+\t\tprotocol=https\n+\t\thost=git2.example.com\n+\t\tusername=askpass-username\n+\t\tpassword=askpass-password\n+\t\t--\n+\t\taskpass: Username for '\\''https://git2.example.com'\\'':\n+\t\taskpass: Password for '\\''https://askpass-username@git2.example.com'\\'':\n+\t\tEOF\n+\t'\n+\n+\ttest_expect_success \"helper ($HELPER) does not store ephemeral username and password\" '\n+\t\tcheck approve $HELPER <<-\\EOF &&\n+\t\tcapability[]=authtype\n+\t\tprotocol=https\n+\t\thost=git2.example.com\n+\t\tuser=barbaz\n+\t\tpassword=secret\n+\t\tephemeral=1\n+\t\tEOF\n+\n+\t\tcheck fill $HELPER <<-\\EOF\n+\t\tcapability[]=authtype\n+\t\tprotocol=https\n+\t\thost=git2.example.com\n+\t\t--\n+\t\tprotocol=https\n+\t\thost=git2.example.com\n+\t\tusername=askpass-username\n+\t\tpassword=askpass-password\n+\t\t--\n+\t\taskpass: Username for '\\''https://git2.example.com'\\'':\n+\t\taskpass: Password for '\\''https://askpass-username@git2.example.com'\\'':\n+\t\tEOF\n+\t'\n+}\n+\n write_script askpass <<\\EOF\n echo >&2 askpass: $*\n what=$(echo $1 | cut -d\" \" -f1 | tr A-Z a-z | tr -cd a-z)\n"},{"id":"493039","messageId":"20240417000240.3611948-14-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 13/16] credential: add support for multistage credential rounds","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:37Z","receivedAt":"2024-04-17T00:03:05Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Over HTTP, NTLM and Kerberos require two rounds of authentication on the\nclient side. It's possible that there are custom authentication schemes\nthat also implement this same approach. Since these are tricky schemes\nto implement and the HTTP library in use may not always handle them\ngracefully on all systems, it would be helpful to allow the credential\nhelper to implement them instead for increased portability and\nrobustness.\n\nTo allow this to happen, add a boolean flag, continue, that indicates\nthat instead of failing when we get a 401, we should retry another round\nof authentication. However, this necessitates some changes in our\ncurrent credential code so that we can make this work.\n\nKeep the state[] headers between iterations, but only use them to send\nto the helper and only consider the new ones we read from the credential\nhelper to be valid on subsequent iterations. That avoids us passing\nstale data when we finally approve or reject the credential. Similarly,\nclear the multistage and wwwauth[] values appropriately so that we\ndon't pass stale data or think we're trying a multiround response when\nwe're not. Remove the credential values so that we can actually fill a\nsecond time with new responses.\n\nLimit the number of iterations of reauthentication we do to 3. This\nmeans that if there's a problem, we'll terminate with an error message\ninstead of retrying indefinitely and not informing the user (and\npossibly conducting a DoS on the server).\n\nIn our tests, handle creating multiple response output files from our\nhelper so we can verify that each of the messages sent is correct.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 13 +++++\n builtin/credential.c | 1 +\n credential.c | 32 ++++++++++--\n credential.h | 27 +++++++++-\n http.c | 59 +++++++++++----------\n t/t5563-simple-http-auth.sh | 89 ++++++++++++++++++++++++++++++--\n 6 files changed, 185 insertions(+), 36 deletions(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex 4bbf2db9ca..3d3accc273 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -222,6 +222,19 @@ provided on input.\n This value should not be sent unless the appropriate capability (see below) is\n provided on input.\n \n+`continue`::\n+\tThis is a boolean value, which, if enabled, indicates that this\n+\tauthentication is a non-final part of a multistage authentication step. This\n+\tis common in protocols such as NTLM and Kerberos, where two rounds of client\n+\tauthentication are required, and setting this flag allows the credential\n+\thelper to implement the multistage authentication step. This flag should\n+\tonly be sent if a further stage is required; that is, if another round of\n+\tauthentication is expected.\n++\n+This value should not be sent unless the appropriate capability (see below) is\n+provided on input. This attribute is 'one-way' from a credential helper to\n+pass information to Git (or other programs invoking `git credential`).\n+\n `wwwauth[]`::\n \n \tWhen an HTTP response is received by Git that includes one or more\ndiff --git a/builtin/credential.c b/builtin/credential.c\nindex 643bf0b5e4..3568e57025 100644\n--- a/builtin/credential.c\n+++ b/builtin/credential.c\n@@ -22,6 +22,7 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n \n \tif (!strcmp(op, \"fill\")) {\n \t\tcredential_fill(&c, 0);\n+\t\tcredential_next_state(&c);\n \t\tcredential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);\n \t} else if (!strcmp(op, \"approve\")) {\n \t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);\ndiff --git a/credential.c b/credential.c\nindex c93de92f65..98b040cf11 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -31,10 +31,23 @@ void credential_clear(struct credential *c)\n \tstring_list_clear(&c->helpers, 0);\n \tstrvec_clear(&c->wwwauth_headers);\n \tstrvec_clear(&c->state_headers);\n+\tstrvec_clear(&c->state_headers_to_send);\n \n \tcredential_init(c);\n }\n \n+void credential_next_state(struct credential *c)\n+{\n+\tstrvec_clear(&c->state_headers_to_send);\n+\tSWAP(c->state_headers, c->state_headers_to_send);\n+}\n+\n+void credential_clear_secrets(struct credential *c)\n+{\n+\tFREE_AND_NULL(c->password);\n+\tFREE_AND_NULL(c->credential);\n+}\n+\n static void credential_set_capability(struct credential_capability *capa,\n \t\t\t\t enum credential_op_type op_type)\n {\n@@ -302,6 +315,8 @@ int credential_read(struct credential *c, FILE *fp,\n \t\t\t\tcredential_set_capability(&c->capa_authtype, op_type);\n \t\t\telse if (!strcmp(value, \"state\"))\n \t\t\t\tcredential_set_capability(&c->capa_state, op_type);\n+\t\t} else if (!strcmp(key, \"continue\")) {\n+\t\t\tc->multistage = !!git_config_bool(\"continue\", value);\n \t\t} else if (!strcmp(key, \"password_expiry_utc\")) {\n \t\t\terrno = 0;\n \t\t\tc->password_expiry_utc = parse_timestamp(value, NULL, 10);\n@@ -369,8 +384,10 @@ void credential_write(const struct credential *c, FILE *fp,\n \tfor (size_t i = 0; i < c->wwwauth_headers.nr; i++)\n \t\tcredential_write_item(fp, \"wwwauth[]\", c->wwwauth_headers.v[i], 0);\n \tif (credential_has_capability(&c->capa_state, op_type)) {\n-\t\tfor (size_t i = 0; i < c->state_headers.nr; i++)\n-\t\t\tcredential_write_item(fp, \"state[]\", c->state_headers.v[i], 0);\n+\t\tif (c->multistage)\n+\t\t\tcredential_write_item(fp, \"continue\", \"1\", 0);\n+\t\tfor (size_t i = 0; i < c->state_headers_to_send.nr; i++)\n+\t\t\tcredential_write_item(fp, \"state[]\", c->state_headers_to_send.v[i], 0);\n \t}\n }\n \n@@ -441,6 +458,9 @@ void credential_fill(struct credential *c, int all_capabilities)\n \tif ((c->username && c->password) || c->credential)\n \t\treturn;\n \n+\tcredential_next_state(c);\n+\tc->multistage = 0;\n+\n \tcredential_apply_config(c);\n \tif (all_capabilities)\n \t\tcredential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);\n@@ -453,8 +473,10 @@ void credential_fill(struct credential *c, int all_capabilities)\n \t\t\t/* Reset expiry to maintain consistency */\n \t\t\tc->password_expiry_utc = TIME_MAX;\n \t\t}\n-\t\tif ((c->username && c->password) || c->credential)\n+\t\tif ((c->username && c->password) || c->credential) {\n+\t\t\tstrvec_clear(&c->wwwauth_headers);\n \t\t\treturn;\n+\t\t}\n \t\tif (c->quit)\n \t\t\tdie(\"credential helper '%s' told us to quit\",\n \t\t\t c->helpers.items[i].string);\n@@ -474,6 +496,8 @@ void credential_approve(struct credential *c)\n \tif (((!c->username || !c->password) && !c->credential) || c->password_expiry_utc < time(NULL))\n \t\treturn;\n \n+\tcredential_next_state(c);\n+\n \tcredential_apply_config(c);\n \n \tfor (i = 0; i < c->helpers.nr; i++)\n@@ -485,6 +509,8 @@ void credential_reject(struct credential *c)\n {\n \tint i;\n \n+\tcredential_next_state(c);\n+\n \tcredential_apply_config(c);\n \n \tfor (i = 0; i < c->helpers.nr; i++)\ndiff --git a/credential.h b/credential.h\nindex c307300d12..19163fc6a0 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -145,10 +145,15 @@ struct credential {\n \tstruct strvec wwwauth_headers;\n \n \t/**\n-\t * A `strvec` of state headers from credential helpers.\n+\t * A `strvec` of state headers received from credential helpers.\n \t */\n \tstruct strvec state_headers;\n \n+\t/**\n+\t * A `strvec` of state headers to send to credential helpers.\n+\t */\n+\tstruct strvec state_headers_to_send;\n+\n \t/**\n \t * Internal use only. Keeps track of if we previously matched against a\n \t * WWW-Authenticate header line in order to re-fold future continuation\n@@ -159,6 +164,7 @@ struct credential {\n \tunsigned approved:1,\n \t\t ephemeral:1,\n \t\t configured:1,\n+\t\t multistage: 1,\n \t\t quit:1,\n \t\t use_http_path:1,\n \t\t username_from_proto:1;\n@@ -187,6 +193,7 @@ struct credential {\n \t.password_expiry_utc = TIME_MAX, \\\n \t.wwwauth_headers = STRVEC_INIT, \\\n \t.state_headers = STRVEC_INIT, \\\n+\t.state_headers_to_send = STRVEC_INIT, \\\n }\n \n /* Initialize a credential structure, setting all fields to empty. */\n@@ -238,6 +245,24 @@ void credential_reject(struct credential *);\n void credential_set_all_capabilities(struct credential *c,\n \t\t\t\t enum credential_op_type op_type);\n \n+/**\n+ * Clear the secrets in this credential, but leave other data intact.\n+ *\n+ * This is useful for resetting credentials in preparation for a subsequent\n+ * stage of filling.\n+ */\n+void credential_clear_secrets(struct credential *c);\n+\n+/**\n+ * Prepares the credential for the next iteration of the helper protocol by\n+ * updating the state headers to send with the ones read by the last iteration\n+ * of the protocol.\n+ *\n+ * Except for internal callers, this should be called exactly once between\n+ * reading credentials with `credential_fill` and writing them.\n+ */\n+void credential_next_state(struct credential *c);\n+\n int credential_read(struct credential *, FILE *,\n \t\t enum credential_op_type);\n void credential_write(const struct credential *, FILE *,\ndiff --git a/http.c b/http.c\nindex 906eb098c8..9a514404d0 100644\n--- a/http.c\n+++ b/http.c\n@@ -1780,6 +1780,10 @@ static int handle_curl_result(struct slot_results *results)\n \telse if (results->http_code == 401) {\n \t\tif ((http_auth.username && http_auth.password) ||\\\n \t\t (http_auth.authtype && http_auth.credential)) {\n+\t\t\tif (http_auth.multistage) {\n+\t\t\t\tcredential_clear_secrets(&http_auth);\n+\t\t\t\treturn HTTP_REAUTH;\n+\t\t\t}\n \t\t\tcredential_reject(&http_auth);\n \t\t\treturn HTTP_NOAUTH;\n \t\t} else {\n@@ -2177,6 +2181,7 @@ static int http_request_reauth(const char *url,\n \t\t\t void *result, int target,\n \t\t\t struct http_get_options *options)\n {\n+\tint i = 3;\n \tint ret = http_request(url, result, target, options);\n \n \tif (ret != HTTP_OK && ret != HTTP_REAUTH)\n@@ -2190,35 +2195,35 @@ static int http_request_reauth(const char *url,\n \t\t}\n \t}\n \n-\tif (ret != HTTP_REAUTH)\n-\t\treturn ret;\n+\twhile (ret == HTTP_REAUTH && --i) {\n+\t\t/*\n+\t\t * The previous request may have put cruft into our output stream; we\n+\t\t * should clear it out before making our next request.\n+\t\t */\n+\t\tswitch (target) {\n+\t\tcase HTTP_REQUEST_STRBUF:\n+\t\t\tstrbuf_reset(result);\n+\t\t\tbreak;\n+\t\tcase HTTP_REQUEST_FILE:\n+\t\t\tif (fflush(result)) {\n+\t\t\t\terror_errno(\"unable to flush a file\");\n+\t\t\t\treturn HTTP_START_FAILED;\n+\t\t\t}\n+\t\t\trewind(result);\n+\t\t\tif (ftruncate(fileno(result), 0) < 0) {\n+\t\t\t\terror_errno(\"unable to truncate a file\");\n+\t\t\t\treturn HTTP_START_FAILED;\n+\t\t\t}\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tBUG(\"Unknown http_request target\");\n+\t\t}\n \n-\t/*\n-\t * The previous request may have put cruft into our output stream; we\n-\t * should clear it out before making our next request.\n-\t */\n-\tswitch (target) {\n-\tcase HTTP_REQUEST_STRBUF:\n-\t\tstrbuf_reset(result);\n-\t\tbreak;\n-\tcase HTTP_REQUEST_FILE:\n-\t\tif (fflush(result)) {\n-\t\t\terror_errno(\"unable to flush a file\");\n-\t\t\treturn HTTP_START_FAILED;\n-\t\t}\n-\t\trewind(result);\n-\t\tif (ftruncate(fileno(result), 0) < 0) {\n-\t\t\terror_errno(\"unable to truncate a file\");\n-\t\t\treturn HTTP_START_FAILED;\n-\t\t}\n-\t\tbreak;\n-\tdefault:\n-\t\tBUG(\"Unknown http_request target\");\n+\t\tcredential_fill(&http_auth, 1);\n+\n+\t\tret = http_request(url, result, target, options);\n \t}\n-\n-\tcredential_fill(&http_auth, 1);\n-\n-\treturn http_request(url, result, target, options);\n+\treturn ret;\n }\n \n int http_get_strbuf(const char *url,\ndiff --git a/t/t5563-simple-http-auth.sh b/t/t5563-simple-http-auth.sh\nindex 515185ae00..5d5caa3f58 100755\n--- a/t/t5563-simple-http-auth.sh\n+++ b/t/t5563-simple-http-auth.sh\n@@ -21,9 +21,17 @@ test_expect_success 'setup_credential_helper' '\n \tCREDENTIAL_HELPER=\"$TRASH_DIRECTORY/bin/git-credential-test-helper\" &&\n \twrite_script \"$CREDENTIAL_HELPER\" <<-\\EOF\n \tcmd=$1\n-\tteefile=$cmd-query.cred\n+\tteefile=$cmd-query-temp.cred\n \tcatfile=$cmd-reply.cred\n \tsed -n -e \"/^$/q\" -e \"p\" >>$teefile\n+\tstate=$(sed -ne \"s/^state\\[\\]=helper://p\" \"$teefile\")\n+\tif test -z \"$state\"\n+\tthen\n+\t\tmv \"$teefile\" \"$cmd-query.cred\"\n+\telse\n+\t\tmv \"$teefile\" \"$cmd-query-$state.cred\"\n+\t\tcatfile=\"$cmd-reply-$state.cred\"\n+\tfi\n \tif test \"$cmd\" = \"get\"\n \tthen\n \t\tcat $catfile\n@@ -32,13 +40,15 @@ test_expect_success 'setup_credential_helper' '\n '\n \n set_credential_reply () {\n-\tcat >\"$TRASH_DIRECTORY/$1-reply.cred\"\n+\tlocal suffix=\"$(test -n \"$2\" && echo \"-$2\")\"\n+\tcat >\"$TRASH_DIRECTORY/$1-reply$suffix.cred\"\n }\n \n expect_credential_query () {\n-\tcat >\"$TRASH_DIRECTORY/$1-expect.cred\" &&\n-\ttest_cmp \"$TRASH_DIRECTORY/$1-expect.cred\" \\\n-\t\t \"$TRASH_DIRECTORY/$1-query.cred\"\n+\tlocal suffix=\"$(test -n \"$2\" && echo \"-$2\")\"\n+\tcat >\"$TRASH_DIRECTORY/$1-expect$suffix.cred\" &&\n+\ttest_cmp \"$TRASH_DIRECTORY/$1-expect$suffix.cred\" \\\n+\t\t \"$TRASH_DIRECTORY/$1-query$suffix.cred\"\n }\n \n per_test_cleanup () {\n@@ -479,4 +489,73 @@ test_expect_success 'access using bearer auth with invalid credentials' '\n \tEOF\n '\n \n+test_expect_success 'access using three-legged auth' '\n+\ttest_when_finished \"per_test_cleanup\" &&\n+\n+\tset_credential_reply get <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tcredential=YS1naXQtdG9rZW4=\n+\tstate[]=helper:foobar\n+\tcontinue=1\n+\tEOF\n+\n+\tset_credential_reply get foobar <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tcredential=YW5vdGhlci10b2tlbg==\n+\tstate[]=helper:bazquux\n+\tEOF\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.valid\" <<-EOF &&\n+\tid=1 creds=Multistage YS1naXQtdG9rZW4=\n+\tid=2 creds=Multistage YW5vdGhlci10b2tlbg==\n+\tEOF\n+\n+\tCHALLENGE=\"$HTTPD_ROOT_PATH/custom-auth.challenge\" &&\n+\n+\tcat >\"$HTTPD_ROOT_PATH/custom-auth.challenge\" <<-EOF &&\n+\tid=1 status=401 response=WWW-Authenticate: Multistage challenge=\"456\"\n+\tid=1 status=401 response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tid=2 status=200\n+\tid=default response=WWW-Authenticate: Multistage challenge=\"123\"\n+\tid=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tEOF\n+\n+\ttest_config_global credential.helper test-helper &&\n+\tgit ls-remote \"$HTTPD_URL/custom_auth/repo.git\" &&\n+\n+\texpect_credential_query get <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=Multistage challenge=\"123\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tEOF\n+\n+\texpect_credential_query get foobar <<-EOF &&\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\twwwauth[]=Multistage challenge=\"456\"\n+\twwwauth[]=Bearer authorize_uri=\"id.example.com\" p=1 q=0\n+\tstate[]=helper:foobar\n+\tEOF\n+\n+\texpect_credential_query store bazquux <<-EOF\n+\tcapability[]=authtype\n+\tcapability[]=state\n+\tauthtype=Multistage\n+\tcredential=YW5vdGhlci10b2tlbg==\n+\tprotocol=http\n+\thost=$HTTPD_DEST\n+\tstate[]=helper:bazquux\n+\tEOF\n+'\n+\n test_done\n"},{"id":"493040","messageId":"20240417000240.3611948-16-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 15/16] credential-cache: implement authtype capability","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:39Z","receivedAt":"2024-04-17T00:03:06Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Now that we have full support in Git for the authtype capability, let's\nadd support to the cache credential helper.\n\nWhen parsing data, we always set the initial capabilities because we're\nthe helper, and we need both the initial and helper capabilities to be\nset in order to have the helper capabilities take effect.\n\nWhen emitting data, always emit the supported capability and make sure\nwe emit items only if we have them and they're supported by the caller.\nSince we may no longer have a username or password, be sure to emit\nthose conditionally as well so we don't segfault on a NULL pointer.\nSimilarly, when comparing credentials, consider both the password and\ncredential fields when we're matching passwords.\n\nAdjust the partial credential detection code so that we can store\ncredentials missing a username or password as long as they have an\nauthtype and credential.\n\nSigned-off-by: brian m. carlson \n---\n builtin/credential-cache--daemon.c | 20 +++++++++++++++++---\n credential.c | 7 ++++---\n credential.h | 6 ++++++\n t/t0301-credential-cache.sh | 1 +\n 4 files changed, 28 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c\nindex ccbcf99ac1..6ffedcd17c 100644\n--- a/builtin/credential-cache--daemon.c\n+++ b/builtin/credential-cache--daemon.c\n@@ -115,6 +115,8 @@ static int read_request(FILE *fh, struct credential *c,\n \t\treturn error(\"client sent bogus timeout line: %s\", item.buf);\n \t*timeout = atoi(p);\n \n+\tcredential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);\n+\n \tif (credential_read(c, fh, CREDENTIAL_OP_HELPER) < 0)\n \t\treturn -1;\n \treturn 0;\n@@ -131,8 +133,18 @@ static void serve_one_client(FILE *in, FILE *out)\n \telse if (!strcmp(action.buf, \"get\")) {\n \t\tstruct credential_cache_entry *e = lookup_credential(&c);\n \t\tif (e) {\n-\t\t\tfprintf(out, \"username=%s\\n\", e->item.username);\n-\t\t\tfprintf(out, \"password=%s\\n\", e->item.password);\n+\t\t\te->item.capa_authtype.request_initial = 1;\n+\t\t\te->item.capa_authtype.request_helper = 1;\n+\n+\t\t\tfprintf(out, \"capability[]=authtype\\n\");\n+\t\t\tif (e->item.username)\n+\t\t\t\tfprintf(out, \"username=%s\\n\", e->item.username);\n+\t\t\tif (e->item.password)\n+\t\t\t\tfprintf(out, \"password=%s\\n\", e->item.password);\n+\t\t\tif (credential_has_capability(&c.capa_authtype, CREDENTIAL_OP_HELPER) && e->item.authtype)\n+\t\t\t\tfprintf(out, \"authtype=%s\\n\", e->item.authtype);\n+\t\t\tif (credential_has_capability(&c.capa_authtype, CREDENTIAL_OP_HELPER) && e->item.credential)\n+\t\t\t\tfprintf(out, \"credential=%s\\n\", e->item.credential);\n \t\t\tif (e->item.password_expiry_utc != TIME_MAX)\n \t\t\t\tfprintf(out, \"password_expiry_utc=%\"PRItime\"\\n\",\n \t\t\t\t\te->item.password_expiry_utc);\n@@ -157,8 +169,10 @@ static void serve_one_client(FILE *in, FILE *out)\n \telse if (!strcmp(action.buf, \"store\")) {\n \t\tif (timeout < 0)\n \t\t\twarning(\"cache client didn't specify a timeout\");\n-\t\telse if (!c.username || !c.password)\n+\t\telse if ((!c.username || !c.password) && (!c.authtype && !c.credential))\n \t\t\twarning(\"cache client gave us a partial credential\");\n+\t\telse if (c.ephemeral)\n+\t\t\twarning(\"not storing ephemeral credential\");\n \t\telse {\n \t\t\tremove_credential(&c, 0);\n \t\t\tcache_credential(&c, timeout);\ndiff --git a/credential.c b/credential.c\nindex 98b040cf11..ffaf31499e 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -80,7 +80,8 @@ int credential_match(const struct credential *want,\n \t CHECK(host) &&\n \t CHECK(path) &&\n \t CHECK(username) &&\n-\t (!match_password || CHECK(password));\n+\t (!match_password || CHECK(password)) &&\n+\t (!match_password || CHECK(credential));\n #undef CHECK\n }\n \n@@ -248,8 +249,8 @@ static void credential_getpass(struct credential *c)\n \t\t\t\t\t\t PROMPT_ASKPASS);\n }\n \n-static int credential_has_capability(const struct credential_capability *capa,\n-\t\t\t\t enum credential_op_type op_type)\n+int credential_has_capability(const struct credential_capability *capa,\n+\t\t\t enum credential_op_type op_type)\n {\n \t/*\n \t * We're checking here if each previous step indicated that we had the\ndiff --git a/credential.h b/credential.h\nindex 19163fc6a0..f8df10937c 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -263,6 +263,12 @@ void credential_clear_secrets(struct credential *c);\n */\n void credential_next_state(struct credential *c);\n \n+/**\n+ * Return true if the capability is enabled for an operation of op_type.\n+ */\n+int credential_has_capability(const struct credential_capability *capa,\n+\t\t\t enum credential_op_type op_type);\n+\n int credential_read(struct credential *, FILE *,\n \t\t enum credential_op_type);\n void credential_write(const struct credential *, FILE *,\ndiff --git a/t/t0301-credential-cache.sh b/t/t0301-credential-cache.sh\nindex 8300faadea..3897353d55 100755\n--- a/t/t0301-credential-cache.sh\n+++ b/t/t0301-credential-cache.sh\n@@ -31,6 +31,7 @@ test_atexit 'git credential-cache exit'\n helper_test cache\n helper_test_password_expiry_utc cache\n helper_test_oauth_refresh_token cache\n+helper_test_authtype cache\n \n test_expect_success 'socket defaults to ~/.cache/git/credential/socket' '\n \ttest_when_finished \"\n"},{"id":"493042","messageId":"20240417000240.3611948-17-sandals@crustytoothpaste.net","threadId":"61189","inReplyTo":"20240417000240.3611948-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 16/16] credential: add method for querying capabilities","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-04-17T00:02:40Z","receivedAt":"2024-04-17T00:03:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Right now, there's no specific way to determine whether a credential\nhelper or git credential itself supports a given set of capabilities.\nIt would be helpful to have such a way, so let's let credential helpers\nand git credential take an argument, \"capability\", which has it list the\ncapabilities and a version number on standard output.\n\nSpecifically choose a format that is slightly different from regular\ncredential output and assume that no capabilities are supported if a\nnon-zero exit status occurs or the data deviates from the format. It is\ncommon for users to write small shell scripts as the argument to\ncredential.helper, which will almost never be designed to emit\ncapabilities. We want callers to gracefully handle this case by\nassuming that they are not capable of extended support because that is\nalmost certainly the case, and specifying the error behavior up front\ndoes this and preserves backwards compatibility in a graceful way.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-credential.txt | 28 +++++++++++++++++++++++++++-\n builtin/credential-cache.c | 10 ++++++++++\n builtin/credential.c | 6 ++++++\n credential.c | 11 +++++++++++\n credential.h | 6 ++++++\n 5 files changed, 60 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt\nindex 3d3accc273..e41493292f 100644\n--- a/Documentation/git-credential.txt\n+++ b/Documentation/git-credential.txt\n@@ -8,7 +8,7 @@ git-credential - Retrieve and store user credentials\n SYNOPSIS\n --------\n ------------------\n-'git credential' (fill|approve|reject)\n+'git credential' (fill|approve|reject|capability)\n ------------------\n \n DESCRIPTION\n@@ -41,6 +41,9 @@ If the action is `reject`, git-credential will send the description to\n any configured credential helpers, which may erase any stored\n credentials matching the description.\n \n+If the action is `capability`, git-credential will announce any capabilities\n+it supports to standard output.\n+\n If the action is `approve` or `reject`, no output should be emitted.\n \n TYPICAL USE OF GIT CREDENTIAL\n@@ -263,6 +266,29 @@ is supported, but they should not be provided without the capability.\n \n Unrecognised attributes and capabilities are silently discarded.\n \n+[[CAPA-IOFMT]]\n+CAPABILITY INPUT/OUTPUT FORMAT\n+------------------------------\n+\n+For `git credential capability`, the format is slightly different. First, a\n+`version 0` announcement is made to indicate the current version of the\n+protocol, and then each capability is announced with a line like `capability\n+authtype`. Credential helpers may also implement this format, again with the\n+`capability` argument. Additional lines may be added in the future; callers\n+should ignore lines which they don't understand.\n+\n+Because this is a new part of the credential helper protocol, older versions of\n+Git, as well as some credential helpers, may not support it. If a non-zero\n+exit status is received, or if the first line doesn't start with the word\n+`version` and a space, callers should assume that no capabilities are supported.\n+\n+The intention of this format is to differentiate it from the credential output\n+in an unambiguous way. It is possible to use very simple credential helpers\n+(e.g., inline shell scripts) which always produce identical output. Using a\n+distinct format allows users to continue to use this syntax without having to\n+worry about correctly implementing capability advertisements or accidentally\n+confusing callers querying for capabilities.\n+\n GIT\n ---\n Part of the linkgit:git[1] suite\ndiff --git a/builtin/credential-cache.c b/builtin/credential-cache.c\nindex bba96d4ffd..f5c989e2b2 100644\n--- a/builtin/credential-cache.c\n+++ b/builtin/credential-cache.c\n@@ -1,4 +1,5 @@\n #include \"builtin.h\"\n+#include \"credential.h\"\n #include \"gettext.h\"\n #include \"parse-options.h\"\n #include \"path.h\"\n@@ -127,6 +128,13 @@ static char *get_socket_path(void)\n \treturn socket;\n }\n \n+static void announce_capabilities(void)\n+{\n+\tstruct credential c = CREDENTIAL_INIT;\n+\tc.capa_authtype.request_initial = 1;\n+\tcredential_announce_capabilities(&c, stdout);\n+}\n+\n int cmd_credential_cache(int argc, const char **argv, const char *prefix)\n {\n \tchar *socket_path = NULL;\n@@ -160,6 +168,8 @@ int cmd_credential_cache(int argc, const char **argv, const char *prefix)\n \t\tdo_cache(socket_path, op, timeout, FLAG_RELAY);\n \telse if (!strcmp(op, \"store\"))\n \t\tdo_cache(socket_path, op, timeout, FLAG_RELAY|FLAG_SPAWN);\n+\telse if (!strcmp(op, \"capability\"))\n+\t\tannounce_capabilities();\n \telse\n \t\t; /* ignore unknown operation */\n \ndiff --git a/builtin/credential.c b/builtin/credential.c\nindex 3568e57025..5100d441f2 100644\n--- a/builtin/credential.c\n+++ b/builtin/credential.c\n@@ -17,6 +17,12 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)\n \t\tusage(usage_msg);\n \top = argv[1];\n \n+\tif (!strcmp(op, \"capability\")) {\n+\t\tcredential_set_all_capabilities(&c, CREDENTIAL_OP_INITIAL);\n+\t\tcredential_announce_capabilities(&c, stdout);\n+\t\treturn 0;\n+\t}\n+\n \tif (credential_read(&c, stdin, CREDENTIAL_OP_INITIAL) < 0)\n \t\tdie(\"unable to read credential from stdin\");\n \ndiff --git a/credential.c b/credential.c\nindex ffaf31499e..758528b291 100644\n--- a/credential.c\n+++ b/credential.c\n@@ -72,6 +72,17 @@ void credential_set_all_capabilities(struct credential *c,\n \tcredential_set_capability(&c->capa_state, op_type);\n }\n \n+static void announce_one(struct credential_capability *cc, const char *name, FILE *fp) {\n+\tif (cc->request_initial)\n+\t\tfprintf(fp, \"capability %s\\n\", name);\n+}\n+\n+void credential_announce_capabilities(struct credential *c, FILE *fp) {\n+\tfprintf(fp, \"version 0\\n\");\n+\tannounce_one(&c->capa_authtype, \"authtype\", fp);\n+\tannounce_one(&c->capa_state, \"state\", fp);\n+}\n+\n int credential_match(const struct credential *want,\n \t\t const struct credential *have, int match_password)\n {\ndiff --git a/credential.h b/credential.h\nindex f8df10937c..af8c287ff2 100644\n--- a/credential.h\n+++ b/credential.h\n@@ -253,6 +253,12 @@ void credential_set_all_capabilities(struct credential *c,\n */\n void credential_clear_secrets(struct credential *c);\n \n+/**\n+ * Print a list of supported capabilities and version numbers to standard\n+ * output.\n+ */\n+void credential_announce_capabilities(struct credential *c, FILE *fp);\n+\n /**\n * Prepares the credential for the next iteration of the helper protocol by\n * updating the state headers to send with the ones read by the last iteration\n"}]}