{"thread":{"id":"65516","subject":"[BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages","startedAt":"2026-04-20T08:59:12Z","lastAt":"2026-05-12T05:54:38Z","messageCount":13,"participants":["Kushal Das","brian m. carlson","Elijah Newren","D. Ben Knoble","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"541948","messageId":"4d5d04e2-49c4-4781-a289-f8cf79570643@sunet.se","threadId":"65516","inReplyTo":null,"subject":"[BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages","fromName":"Kushal Das","fromEmail":"kushal@sunet.se","sentAt":"2026-04-20T08:59:05Z","receivedAt":"2026-04-20T08:59:12Z","isPatch":false,"body":"Hi all,\n\nEvery `git commit -S` since v2.45.0 produces a permanently-BAD\nsignature when the commit message contains bytes that are not valid\nUTF-8 AND `i18n.commitEncoding` is unset (i.e. the default case).\nVerification fails under both `gpg --verify` and any non-GnuPG signer.\nThe failure is deterministic: it happens every time, on every\nnon-UTF-8 commit, no card or external tooling needed.\n\nMy best guess is commit 6206089cbd0b1cb30a017ec904567f040ab4cea0 \nstarting this (and I am maybe 100% wrong in identifying the cause).\n\nIn pre-6206089cbd `commit_tree_extended`, `verify_utf8(&buffer)` ran\nBEFORE `sign_with_header(&buffer, sign_commit)`. `verify_utf8` is not\na simple validator -- it mutates the strbuf in place, replacing\ninvalid-UTF-8 bytes with their Latin-1 -> UTF-8 two-byte form. The\nsigner therefore saw the transcoded bytes, and the same transcoded\nbytes were then written to the object database. Signer and\nverifier agreed.\n\nAfter 6206089cbd, the sequence in `commit_tree_extended` is:\n\n write_commit_tree(&buffer, ...);\n sign_commit_to_strbuf(&sig, &buffer, sign_commit); /* pre-transcode */\n ...\n add_commit_signature(&buffer, bufs[i].sig, bufs[i].algo);\n /* and then */\n if (encoding_is_utf8 && (!verify_utf8(&buffer) || \n!verify_utf8(&compat_buffer)))\n fprintf(stderr, _(commit_utf8_warn)); /* post-sign \ntranscode */\n ...\n odb_write_object_ext(..., buffer.buf, buffer.len, OBJ_COMMIT, ret, ...);\n\nThe signature in `bufs[i].sig` covers the raw (non-UTF-8) buffer. The\n`verify_utf8` call after `add_commit_signature` rewrites the message\nportion of the stored object to UTF-8. The object that hits the ODB\ntherefore contains bytes that no longer match what the signer hashed,\nand any verifier that reads the commit back and re-hashes the\nsig-stripped buffer will find a mismatch.\n\n\nAs a reproducer I ran the following command with the below bash script \nto verify so far\n`podman run --rm -it -v ./git_bug.sh:/git_bug.sh:Z fedora:41 bash -c \n'dnf -y install git gnupg2 >/dev/null 2>&1 && /git_bug.sh 2>&1'`\n\n\n```\n#!/usr/bin/env bash\n# git_bug.sh -- minimal in-container reproducer for the git 2.45+\n# sign/store divergence on non-UTF-8 commit messages.\n#\n# Two cases:\n# CASE A: default git config. On git 2.45+ the signer hashes the\n# raw (Latin-1) bytes but the ODB stores the UTF-8-transcoded\n# bytes. Verify fails with BAD signature.\n# CASE B: same commit with `i18n.commitEncoding=iso-8859-1`. git\n# tags the object with an `encoding` header and skips the\n# in-place transcode, so sign and store bytes agree.\n#\n# Exit codes:\n# 0 Not reproduced. Git either predates v2.45 or carries a fix.\n# 1 Reproduced. CASE A BAD, CASE B GOOD (workaround confirmed).\n# 2 Unexpected state or missing tool.\n#\n# Dependencies: git + gpg + awk + od + a POSIX shell. the script mints \nits own\n# throwaway Ed25519 key in a temp GNUPGHOME. Host state untouched.\n#\nset -euo pipefail\n\nstep() { printf '>>> %s\\n' \"$*\"; }\nsay() { printf '%s\\n' \"$*\"; }\ndie() { printf 'git_bug.sh: %s\\n' \"$*\" >&2; exit 2; }\n\nfor tool in git gpg awk od; do\n command -v \"$tool\" >/dev/null || die \"$tool not found on PATH\"\ndone\n\nSANDBOX=$(mktemp -d -t git-bug-XXXXXX)\nexport GNUPGHOME=\"$SANDBOX/gnupg\"\nmkdir -p \"$GNUPGHOME\"\nchmod 700 \"$GNUPGHOME\"\n# Pre-create agent + gpg configs so loopback pinentry works and no\n# dirmngr / tty prompts can fire.\ncat > \"$GNUPGHOME/gpg-agent.conf\" < \"$GNUPGHOME/gpg.conf\" </dev/null 2>&1 || true\n rm -rf \"$SANDBOX\"\n}\ntrap cleanup EXIT\n\nPASS=\"\"\n\nstep \"detected: $(git --version), $(gpg --version | head -n1)\"\n\n# ---------------------------------------------------------------------\n# Mint a throwaway Ed25519 signing key. Ed25519 signs in well under a\n# second, unlike RSA-4096 which on some containers takes tens of\n# seconds and easily reads as \"the script is hung\".\n# ---------------------------------------------------------------------\nstep \"minting throwaway Ed25519 key\"\ncat > \"$SANDBOX/keygen\" <\"$SANDBOX/keygen.err\" \\\n || { cat \"$SANDBOX/keygen.err\" >&2; die \"key generation failed\"; }\nKEY_FP=$(gpg --list-keys --with-colons bug@example.local \\\n | awk -F: '/^fpr:/ {print $10; exit}')\n[ -n \"$KEY_FP\" ] || die \"could not read key fingerprint\"\nsay \"key: $KEY_FP\"\n\n# Kill the gpg-agent spawned by keygen so the next invocation starts\n# fresh with our loopback-pinentry policy.\ngpgconf --kill gpg-agent >/dev/null 2>&1 || true\n\n# gpg wrapper git will invoke as `gpg.program`. Branches on whether\n# --verify is in argv so we can capture the two stdin streams to\n# separate files without clobbering each other.\ncat > \"$SANDBOX/gpg-capture\" <}\"\n\n (\n cd \"$repo\"\n step \"git init + config\"\n git init --quiet\n git config user.name tester\n git config user.email bug@example.local\n git config user.signingkey \"$KEY_FP\"\n git config commit.gpgsign true\n git config gpg.program \"$SANDBOX/gpg-capture\"\n for kv in \"${extra[@]}\"; do\n git config \"${kv%%=*}\" \"${kv#*=}\"\n done\n\n printf 'section \\xa7\\n' > \"$SANDBOX/msg.txt\"\n echo content > f.txt\n git add f.txt\n\n step \"git commit -F msg.txt (invokes gpg-capture to sign)\"\n git commit -F \"$SANDBOX/msg.txt\" --quiet \n2>\"$SANDBOX/commit_${label}.err\" \\\n || { cat \"$SANDBOX/commit_${label}.err\" >&2; exit 1; }\n )\n\n say \"-- msg.txt (bytes we asked git to commit): --\"\n od -An -tx1 -v \"$SANDBOX/msg.txt\"\n\n say \"-- sign_stdin (bytes git fed gpg at SIGN time, tail): --\"\n od -An -tx1 -v \"$SANDBOX/sign_stdin.bin\" | tail -n 1\n\n say \"-- commit object body (bytes in the ODB, tail): --\"\n (cd \"$repo\" && git cat-file commit HEAD) | od -An -tx1 -v | tail -n 1\n\n step \"git verify-commit HEAD\"\n local rc=0\n (cd \"$repo\" && git verify-commit HEAD) >/dev/null 2>&1 || rc=$?\n\n if [[ -f \"$SANDBOX/verify_stdin.bin\" ]]; then\n say \"-- verify_stdin (bytes git fed gpg at VERIFY time, tail): --\"\n od -An -tx1 -v \"$SANDBOX/verify_stdin.bin\" | tail -n 1\n fi\n\n if (( rc == 0 )); then\n say \"verify: GOOD (exit 0)\"\n return 0\n else\n say \"verify: BAD (exit $rc)\"\n return 1\n fi\n}\n\nif run_case A; then\n STATE_A=GOOD\nelse\n STATE_A=BAD\nfi\n\nif run_case B i18n.commitEncoding=iso-8859-1; then\n STATE_B=GOOD\nelse\n STATE_B=BAD\nfi\n\nsay \"\"\nsay \"=== summary ===\"\nsay \"CASE A (default commitEncoding): $STATE_A\"\nsay \"CASE B (i18n.commitEncoding=iso-8859-1): $STATE_B\"\n\nif [[ \"$STATE_A\" == BAD && \"$STATE_B\" == GOOD ]]; then\n say \"RESULT: reproduced. This git has the v2.45+ regression:\"\n say \" verify_utf8 runs after the signature is already computed,\"\n say \" so the signer sees raw bytes and the ODB stores transcoded\"\n say \" bytes. Workaround confirmed: i18n.commitEncoding=iso-8859-1.\"\n exit 1\nfi\nif [[ \"$STATE_A\" == GOOD && \"$STATE_B\" == GOOD ]]; then\n say \"RESULT: not reproduced. Either git predates v2.45 or has a fix.\"\n exit 0\nfi\nsay \"RESULT: unexpected state (A=$STATE_A B=$STATE_B).\"\nexit 2\n```\n\nOutput:\n\n```\n >>> detected: git version 2.52.0, gpg (GnuPG) 2.4.5\n >>> minting throwaway Ed25519 key\nkey: 5DDD17B4C7A2BB447EAC16F5280A03FBE5C7A8DF\n\n=== CASE A ===\nextra git config: \n >>> git init + config\n >>> git commit -F msg.txt (invokes gpg-capture to sign)\n-- msg.txt (bytes we asked git to commit): --\n 73 65 63 74 69 6f 6e 20 a7 0a\n-- sign_stdin (bytes git fed gpg at SIGN time, tail): --\n a7 0a\n-- commit object body (bytes in the ODB, tail): --\n 20 c2 a7 0a\n >>> git verify-commit HEAD\n-- verify_stdin (bytes git fed gpg at VERIFY time, tail): --\n c2 a7 0a\nverify: BAD (exit 1)\n\n=== CASE B ===\nextra git config: i18n.commitEncoding=iso-8859-1\n >>> git init + config\n >>> git commit -F msg.txt (invokes gpg-capture to sign)\n-- msg.txt (bytes we asked git to commit): --\n 73 65 63 74 69 6f 6e 20 a7 0a\n-- sign_stdin (bytes git fed gpg at SIGN time, tail): --\n 69 6f 6e 20 a7 0a\n-- commit object body (bytes in the ODB, tail): --\n 74 69 6f 6e 20 a7 0a\n >>> git verify-commit HEAD\n-- verify_stdin (bytes git fed gpg at VERIFY time, tail): --\n 69 6f 6e 20 a7 0a\nverify: GOOD (exit 0)\n\n=== summary ===\nCASE A (default commitEncoding): BAD\nCASE B (i18n.commitEncoding=iso-8859-1): GOOD\nRESULT: reproduced. This git has the v2.45+ regression:\n verify_utf8 runs after the signature is already computed,\n so the signer sees raw bytes and the ODB stores transcoded\n bytes. Workaround confirmed: i18n.commitEncoding=iso-8859-1.\n```\n\nI tested this in different ways.\n\n- git 2.53.0 from Fedora 43 reproduces.\n- git 2.53.0 from ubuntu-latest (GitHub Actions) reproduces.\n- git 2.43.0 from Ubuntu 24.04 does NOT reproduce (v2.44 is the last\n tag without 6206089cbd).\n- Any signer produces the bug -- tested with GnuPG 2.4.9 and with\n my non-GnuPG tclig [0]; both hash over the same pre-transcode bytes\n the command-line signer is piped, so both get invalidated by the\n post-sign transcode.\n\n\n[0] https://crates.io/crates/tumpa-cli\n\nKushal\n\n"},{"id":"541993","messageId":"aeakf0xcjSteTMZp@fruit.crustytoothpaste.net","threadId":"65516","inReplyTo":"4d5d04e2-49c4-4781-a289-f8cf79570643@sunet.se","subject":"Re: [BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-20T22:11:11Z","receivedAt":"2026-04-20T22:11:19Z","isPatch":false,"body":"On 2026-04-20 at 08:59:05, Kushal Das wrote:\n> Hi all,\n> \n> Every `git commit -S` since v2.45.0 produces a permanently-BAD\n> signature when the commit message contains bytes that are not valid\n> UTF-8 AND `i18n.commitEncoding` is unset (i.e. the default case).\n> Verification fails under both `gpg --verify` and any non-GnuPG signer.\n> The failure is deterministic: it happens every time, on every\n> non-UTF-8 commit, no card or external tooling needed.\n\nI'm not sure that's a valid configuration. The commit message either\nneeds to be UTF-8 or you need to declare the encoding so Git can convert\nit.\n\n> My best guess is commit 6206089cbd0b1cb30a017ec904567f040ab4cea0 starting\n> this (and I am maybe 100% wrong in identifying the cause).\n\nIt does bisect to that commit. I wrote that patch originally, but it\ngot modified and sent upstream by someone else. I'm not sure where it\ngot introduced, though.\n\n> In pre-6206089cbd `commit_tree_extended`, `verify_utf8(&buffer)` ran\n> BEFORE `sign_with_header(&buffer, sign_commit)`. `verify_utf8` is not\n> a simple validator -- it mutates the strbuf in place, replacing\n> invalid-UTF-8 bytes with their Latin-1 -> UTF-8 two-byte form. The\n> signer therefore saw the transcoded bytes, and the same transcoded\n> bytes were then written to the object database. Signer and\n> verifier agreed.\n\nThe fact that we have a function called `verify_utf8` that does more\nthan verify is a problem. I'll send out a two-patch series in a minute\nor two that first fixes that to be called `ensure_utf8` and then fixes\nthe issue.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"541994","messageId":"20260420221425.2763661-1-sandals@crustytoothpaste.net","threadId":"65516","inReplyTo":"aeakf0xcjSteTMZp@fruit.crustytoothpaste.net","subject":"[PATCH 1/2] commit: name UTF-8 function appropriately","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-20T22:14:24Z","receivedAt":"2026-04-20T22:14:38Z","isPatch":true,"body":"We have a function named verify_utf8, but it does more than verify, it\nmodifies the buffer if it is not UTF-8. This is different from what\nmost people would expect, so call the function ensure_utf8, since it\nmutates the buffer in some cases.\n\nSigned-off-by: brian m. carlson \n---\n commit.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/commit.c b/commit.c\nindex 80d8d07875..790dd2faed 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -1637,12 +1637,12 @@ static int find_invalid_utf8(const char *buf, int len)\n }\n \n /*\n- * This verifies that the buffer is in proper utf8 format.\n+ * This ensures that the buffer is in proper utf8 format.\n *\n * If it isn't, it assumes any non-utf8 characters are Latin1,\n * and does the conversion.\n */\n-static int verify_utf8(struct strbuf *buf)\n+static int ensure_utf8(struct strbuf *buf)\n {\n \tint ok = 1;\n \tlong pos = 0;\n@@ -1819,7 +1819,7 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t}\n \n \t/* And check the encoding. */\n-\tif (encoding_is_utf8 && (!verify_utf8(&buffer) || !verify_utf8(&compat_buffer)))\n+\tif (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n \t\tfprintf(stderr, _(commit_utf8_warn));\n \n \tif (r->compat_hash_algo) {\n"},{"id":"541995","messageId":"20260420221425.2763661-2-sandals@crustytoothpaste.net","threadId":"65516","inReplyTo":"20260420221425.2763661-1-sandals@crustytoothpaste.net","subject":"[PATCH 2/2] commit: sign commit after mutating buffer","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-20T22:14:25Z","receivedAt":"2026-04-20T22:14:38Z","isPatch":true,"body":"The ensure_utf8 function can mutate the buffer to change its encoding,\nso we must call it before signing the buffer so that we do not\ninvalidate the signature, which is made over raw bytes. Add a test for\nthis case as well using 0xfe and 0xff, which are never valid in UTF-8.\n\nReported-by: Kushal Das \nSigned-off-by: brian m. carlson \n---\n commit.c | 12 ++++++++----\n t/t7510-signed-commit.sh | 8 ++++++++\n 2 files changed, 16 insertions(+), 4 deletions(-)\n\ndiff --git a/commit.c b/commit.c\nindex 790dd2faed..bc41859be1 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -1747,6 +1747,11 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t\toidcpy(&parent_buf[i++], &p->item->object.oid);\n \n \twrite_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra);\n+\n+\t/* And check the encoding. */\n+\tif (encoding_is_utf8 && !ensure_utf8(&buffer))\n+\t\tfprintf(stderr, _(commit_utf8_warn));\n+\n \tif (sign_commit && sign_buffer(&buffer, &sig, sign_commit,\n \t\t\t\t SIGN_BUFFER_USE_DEFAULT_KEY)) {\n \t\tresult = -1;\n@@ -1780,6 +1785,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t\tfree_commit_extra_headers(compat_extra);\n \t\tfree(mapped_parents);\n \n+\t\tif (encoding_is_utf8 && !ensure_utf8(&compat_buffer))\n+\t\t\tfprintf(stderr, _(commit_utf8_warn));\n+\n \t\tif (sign_commit && sign_buffer(&compat_buffer, &compat_sig,\n \t\t\t\t\t sign_commit,\n \t\t\t\t\t SIGN_BUFFER_USE_DEFAULT_KEY)) {\n@@ -1818,10 +1826,6 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t\t}\n \t}\n \n-\t/* And check the encoding. */\n-\tif (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n-\t\tfprintf(stderr, _(commit_utf8_warn));\n-\n \tif (r->compat_hash_algo) {\n \t\thash_object_file(r->compat_hash_algo, compat_buffer.buf, compat_buffer.len,\n \t\t\tOBJ_COMMIT, &compat_oid_buf);\ndiff --git a/t/t7510-signed-commit.sh b/t/t7510-signed-commit.sh\nindex 1201c85ba6..071dbb3d39 100755\n--- a/t/t7510-signed-commit.sh\n+++ b/t/t7510-signed-commit.sh\n@@ -462,4 +462,12 @@ test_expect_success 'custom `gpg.program`' '\n \tgit commit -S --allow-empty -m signed-commit\n '\n \n+test_expect_success GPG 'commit verifies with non-UTF-8 commit message' '\n+\tprintf \"I hate\\\\376\\\\377UTF-8\\\\n\" >message &&\n+\techo unusual-message >file &&\n+\tgit add file &&\n+\ttest_tick && git commit -S -F message &&\n+\tgit verify-commit HEAD\n+'\n+\n test_done\n"},{"id":"542029","messageId":"c0df6dbd-47f9-4a2d-b68d-cb0c1e19ca5a@sunet.se","threadId":"65516","inReplyTo":"aeakf0xcjSteTMZp@fruit.crustytoothpaste.net","subject":"Re: [BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages","fromName":"Kushal Das","fromEmail":"kushal@sunet.se","sentAt":"2026-04-21T07:39:11Z","receivedAt":"2026-04-21T07:39:15Z","isPatch":false,"body":"Hi,\n\nOn 4/21/26 12:11 AM, brian m. carlson wrote:\n> On 2026-04-20 at 08:59:05, Kushal Das wrote:\n>> Hi all,\n>>\n>> Every `git commit -S` since v2.45.0 produces a permanently-BAD\n>> signature when the commit message contains bytes that are not valid\n>> UTF-8 AND `i18n.commitEncoding` is unset (i.e. the default case).\n>> Verification fails under both `gpg --verify` and any non-GnuPG signer.\n>> The failure is deterministic: it happens every time, on every\n>> non-UTF-8 commit, no card or external tooling needed.\n> \n> I'm not sure that's a valid configuration. The commit message either\n> needs to be UTF-8 or you need to declare the encoding so Git can convert\n> it.\n> \n\nIt is not a valid configuration, but I am guessing there are more people \nlike me who never knew this configuration and just freaked out by seeing \nbad signatures over own commits :)\n\nThank you for quick fix.\n\nI am also wondering in the test harness for git signing, if you want to \ninclude other tools than gnupg for testing.\n\nKushal\n\n"},{"id":"542087","messageId":"aef2g0j-ws4zZ2Zp@fruit.crustytoothpaste.net","threadId":"65516","inReplyTo":"c0df6dbd-47f9-4a2d-b68d-cb0c1e19ca5a@sunet.se","subject":"Re: [BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-21T22:13:23Z","receivedAt":"2026-04-21T22:13:31Z","isPatch":false,"body":"On 2026-04-21 at 07:39:11, Kushal Das wrote:\n> I am also wondering in the test harness for git signing, if you want to\n> include other tools than gnupg for testing.\n\nThe signing code is abstract at that point in the code, so it should\nwork identically with SSH or X.509 and I don't think a separate test is\nnecessary.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"542138","messageId":"CABPp-BEy0dENdPG3XyLvqkKgWsP2kN=EF+-e8OHXOGkPrGXQog@mail.gmail.com","threadId":"65516","inReplyTo":"20260420221425.2763661-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/2] commit: name UTF-8 function appropriately","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2026-04-22T15:10:25Z","receivedAt":"2026-04-22T15:10:37Z","isPatch":true,"body":"On Mon, Apr 20, 2026 at 3:14 PM brian m. carlson\n wrote:\n>\n> We have a function named verify_utf8, but it does more than verify, it\n> modifies the buffer if it is not UTF-8. This is different from what\n> most people would expect, so call the function ensure_utf8, since it\n> mutates the buffer in some cases.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> commit.c | 6 +++---\n> 1 file changed, 3 insertions(+), 3 deletions(-)\n>\n> diff --git a/commit.c b/commit.c\n> index 80d8d07875..790dd2faed 100644\n> --- a/commit.c\n> +++ b/commit.c\n> @@ -1637,12 +1637,12 @@ static int find_invalid_utf8(const char *buf, int len)\n> }\n>\n> /*\n> - * This verifies that the buffer is in proper utf8 format.\n> + * This ensures that the buffer is in proper utf8 format.\n> *\n> * If it isn't, it assumes any non-utf8 characters are Latin1,\n> * and does the conversion.\n> */\n> -static int verify_utf8(struct strbuf *buf)\n> +static int ensure_utf8(struct strbuf *buf)\n> {\n> int ok = 1;\n> long pos = 0;\n> @@ -1819,7 +1819,7 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> }\n>\n> /* And check the encoding. */\n> - if (encoding_is_utf8 && (!verify_utf8(&buffer) || !verify_utf8(&compat_buffer)))\n> + if (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n> fprintf(stderr, _(commit_utf8_warn));\n>\n> if (r->compat_hash_algo) {\n\nMakes sense.\n"},{"id":"542139","messageId":"CABPp-BEd9saiMWVdcSaZBHqGreshpA=fGZc4AvYkoU=swSSuxA@mail.gmail.com","threadId":"65516","inReplyTo":"20260420221425.2763661-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 2/2] commit: sign commit after mutating buffer","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2026-04-22T15:10:29Z","receivedAt":"2026-04-22T15:10:42Z","isPatch":true,"body":"On Mon, Apr 20, 2026 at 3:14 PM brian m. carlson\n wrote:\n>\n> The ensure_utf8 function can mutate the buffer to change its encoding,\n> so we must call it before signing the buffer so that we do not\n> invalidate the signature, which is made over raw bytes. Add a test for\n> this case as well using 0xfe and 0xff, which are never valid in UTF-8.\n>\n> Reported-by: Kushal Das \n> Signed-off-by: brian m. carlson \n> ---\n> commit.c | 12 ++++++++----\n> t/t7510-signed-commit.sh | 8 ++++++++\n> 2 files changed, 16 insertions(+), 4 deletions(-)\n>\n> diff --git a/commit.c b/commit.c\n> index 790dd2faed..bc41859be1 100644\n> --- a/commit.c\n> +++ b/commit.c\n> @@ -1747,6 +1747,11 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> oidcpy(&parent_buf[i++], &p->item->object.oid);\n>\n> write_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra);\n> +\n> + /* And check the encoding. */\n> + if (encoding_is_utf8 && !ensure_utf8(&buffer))\n> + fprintf(stderr, _(commit_utf8_warn));\n> +\n> if (sign_commit && sign_buffer(&buffer, &sig, sign_commit,\n> SIGN_BUFFER_USE_DEFAULT_KEY)) {\n> result = -1;\n> @@ -1780,6 +1785,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> free_commit_extra_headers(compat_extra);\n> free(mapped_parents);\n>\n> + if (encoding_is_utf8 && !ensure_utf8(&compat_buffer))\n> + fprintf(stderr, _(commit_utf8_warn));\n> +\n\nSo the users might see \"commit message did not conform to UTF-8...\"\ntwice? (Isn't compat_buffer likely to have invalid UTF-8 whenever\nbuffer does?) Do we want to avoid that double printing?\n\n> if (sign_commit && sign_buffer(&compat_buffer, &compat_sig,\n> sign_commit,\n> SIGN_BUFFER_USE_DEFAULT_KEY)) {\n> @@ -1818,10 +1826,6 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> }\n> }\n>\n> - /* And check the encoding. */\n> - if (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n> - fprintf(stderr, _(commit_utf8_warn));\n> -\n\nDid the change in this patch also fix a short-circuiting error?\nPreviously, when both buffers had invalid UTF-8, we'd only call\nensure_utf8() on the first one and fix it, and then short-circuit and\nnot handle compat_buffer, right?\n\n> if (r->compat_hash_algo) {\n> hash_object_file(r->compat_hash_algo, compat_buffer.buf, compat_buffer.len,\n> OBJ_COMMIT, &compat_oid_buf);\n> diff --git a/t/t7510-signed-commit.sh b/t/t7510-signed-commit.sh\n> index 1201c85ba6..071dbb3d39 100755\n> --- a/t/t7510-signed-commit.sh\n> +++ b/t/t7510-signed-commit.sh\n> @@ -462,4 +462,12 @@ test_expect_success 'custom `gpg.program`' '\n> git commit -S --allow-empty -m signed-commit\n> '\n>\n> +test_expect_success GPG 'commit verifies with non-UTF-8 commit message' '\n> + printf \"I hate\\\\376\\\\377UTF-8\\\\n\" >message &&\n> + echo unusual-message >file &&\n> + git add file &&\n> + test_tick && git commit -S -F message &&\n> + git verify-commit HEAD\n> +'\n\nNice test.\n"},{"id":"542150","messageId":"CALnO6CB5TX=zi7Ghhnvv4wCatLPLKJFT5g1_hf4c+pno2zC3fQ@mail.gmail.com","threadId":"65516","inReplyTo":"aeakf0xcjSteTMZp@fruit.crustytoothpaste.net","subject":"Re: [BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2026-04-22T18:13:30Z","receivedAt":"2026-04-22T18:13:43Z","isPatch":false,"body":"On Mon, Apr 20, 2026 at 6:12 PM brian m. carlson\n wrote:\n>\n> On 2026-04-20 at 08:59:05, Kushal Das wrote:\n> > Hi all,\n> >\n> > Every `git commit -S` since v2.45.0 produces a permanently-BAD\n> > signature when the commit message contains bytes that are not valid\n> > UTF-8 AND `i18n.commitEncoding` is unset (i.e. the default case).\n> > Verification fails under both `gpg --verify` and any non-GnuPG signer.\n> > The failure is deterministic: it happens every time, on every\n> > non-UTF-8 commit, no card or external tooling needed.\n>\n> I'm not sure that's a valid configuration. The commit message either\n> needs to be UTF-8 or you need to declare the encoding so Git can convert\n> it.\n>\n> > My best guess is commit 6206089cbd0b1cb30a017ec904567f040ab4cea0 starting\n> > this (and I am maybe 100% wrong in identifying the cause).\n>\n> It does bisect to that commit. I wrote that patch originally, but it\n> got modified and sent upstream by someone else. I'm not sure where it\n> got introduced, though.\n\nAccording to the `amlog` notes ref: <20231002024034.2611-9-ebiederm@gmail.com>\n\n\n-- \nD. Ben Knoble\n"},{"id":"542273","messageId":"aevPxrN3xWq8SP71@fruit.crustytoothpaste.net","threadId":"65516","inReplyTo":"CABPp-BEd9saiMWVdcSaZBHqGreshpA=fGZc4AvYkoU=swSSuxA@mail.gmail.com","subject":"Re: [PATCH 2/2] commit: sign commit after mutating buffer","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-24T20:17:10Z","receivedAt":"2026-04-24T20:17:12Z","isPatch":true,"body":"On 2026-04-22 at 15:10:29, Elijah Newren wrote:\n> On Mon, Apr 20, 2026 at 3:14 PM brian m. carlson\n> wrote:\n> > diff --git a/commit.c b/commit.c\n> > index 790dd2faed..bc41859be1 100644\n> > --- a/commit.c\n> > +++ b/commit.c\n> > @@ -1747,6 +1747,11 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> > oidcpy(&parent_buf[i++], &p->item->object.oid);\n> >\n> > write_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra);\n> > +\n> > + /* And check the encoding. */\n> > + if (encoding_is_utf8 && !ensure_utf8(&buffer))\n> > + fprintf(stderr, _(commit_utf8_warn));\n> > +\n> > if (sign_commit && sign_buffer(&buffer, &sig, sign_commit,\n> > SIGN_BUFFER_USE_DEFAULT_KEY)) {\n> > result = -1;\n> > @@ -1780,6 +1785,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> > free_commit_extra_headers(compat_extra);\n> > free(mapped_parents);\n> >\n> > + if (encoding_is_utf8 && !ensure_utf8(&compat_buffer))\n> > + fprintf(stderr, _(commit_utf8_warn));\n> > +\n> \n> So the users might see \"commit message did not conform to UTF-8...\"\n> twice? (Isn't compat_buffer likely to have invalid UTF-8 whenever\n> buffer does?) Do we want to avoid that double printing?\n\nYeah, I'll fix that in v2.\n\n> Did the change in this patch also fix a short-circuiting error?\n> Previously, when both buffers had invalid UTF-8, we'd only call\n> ensure_utf8() on the first one and fix it, and then short-circuit and\n> not handle compat_buffer, right?\n\nI believe it did, yes.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"542399","messageId":"20260427221834.1824543-1-sandals@crustytoothpaste.net","threadId":"65516","inReplyTo":"aeakf0xcjSteTMZp@fruit.crustytoothpaste.net","subject":"[PATCH v2 1/2] commit: name UTF-8 function appropriately","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-27T22:18:33Z","receivedAt":"2026-04-27T22:18:45Z","isPatch":true,"body":"We have a function named verify_utf8, but it does more than verify, it\nmodifies the buffer if it is not UTF-8. This is different from what\nmost people would expect, so call the function ensure_utf8, since it\nmutates the buffer in some cases.\n\nSigned-off-by: brian m. carlson \n---\n commit.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/commit.c b/commit.c\nindex 80d8d07875..790dd2faed 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -1637,12 +1637,12 @@ static int find_invalid_utf8(const char *buf, int len)\n }\n \n /*\n- * This verifies that the buffer is in proper utf8 format.\n+ * This ensures that the buffer is in proper utf8 format.\n *\n * If it isn't, it assumes any non-utf8 characters are Latin1,\n * and does the conversion.\n */\n-static int verify_utf8(struct strbuf *buf)\n+static int ensure_utf8(struct strbuf *buf)\n {\n \tint ok = 1;\n \tlong pos = 0;\n@@ -1819,7 +1819,7 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t}\n \n \t/* And check the encoding. */\n-\tif (encoding_is_utf8 && (!verify_utf8(&buffer) || !verify_utf8(&compat_buffer)))\n+\tif (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n \t\tfprintf(stderr, _(commit_utf8_warn));\n \n \tif (r->compat_hash_algo) {\n"},{"id":"542400","messageId":"20260427221834.1824543-2-sandals@crustytoothpaste.net","threadId":"65516","inReplyTo":"20260427221834.1824543-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 2/2] commit: sign commit after mutating buffer","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2026-04-27T22:18:34Z","receivedAt":"2026-04-27T22:18:45Z","isPatch":true,"body":"The ensure_utf8 function can mutate the buffer to change its encoding,\nso we must call it before signing the buffer so that we do not\ninvalidate the signature, which is made over raw bytes. Fix a bug which\ncaused the compatibility code to not convert the compatibility buffer if\nthe main buffer was invalid UTF-8. We expect both buffers to be valid\nUTF-8 or both invalid, since the only data that would differ between\nthem would be hex object IDs, which are always valid UTF-8.\n\nAdd a test for this case using 0xfe and 0xff, which are never valid in\nUTF-8.\n\nReported-by: Kushal Das \nSigned-off-by: brian m. carlson \n---\n commit.c | 15 +++++++++++----\n t/t7510-signed-commit.sh | 10 ++++++++++\n 2 files changed, 21 insertions(+), 4 deletions(-)\n\ndiff --git a/commit.c b/commit.c\nindex 790dd2faed..e5d725fe93 100644\n--- a/commit.c\n+++ b/commit.c\n@@ -1726,6 +1726,7 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \tstruct repository *r = the_repository;\n \tint result = 0;\n \tint encoding_is_utf8;\n+\tbool warned = false;\n \tstruct strbuf buffer = STRBUF_INIT, compat_buffer = STRBUF_INIT;\n \tstruct strbuf sig = STRBUF_INIT, compat_sig = STRBUF_INIT;\n \tstruct object_id *parent_buf = NULL, *compat_oid = NULL;\n@@ -1747,6 +1748,13 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t\toidcpy(&parent_buf[i++], &p->item->object.oid);\n \n \twrite_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra);\n+\n+\t/* And check the encoding. */\n+\tif (encoding_is_utf8 && !ensure_utf8(&buffer)) {\n+\t\tfprintf(stderr, _(commit_utf8_warn));\n+\t\twarned = true;\n+\t}\n+\n \tif (sign_commit && sign_buffer(&buffer, &sig, sign_commit,\n \t\t\t\t SIGN_BUFFER_USE_DEFAULT_KEY)) {\n \t\tresult = -1;\n@@ -1780,6 +1788,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t\tfree_commit_extra_headers(compat_extra);\n \t\tfree(mapped_parents);\n \n+\t\tif (encoding_is_utf8 && !ensure_utf8(&compat_buffer) && !warned)\n+\t\t\tfprintf(stderr, _(commit_utf8_warn));\n+\n \t\tif (sign_commit && sign_buffer(&compat_buffer, &compat_sig,\n \t\t\t\t\t sign_commit,\n \t\t\t\t\t SIGN_BUFFER_USE_DEFAULT_KEY)) {\n@@ -1818,10 +1829,6 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n \t\t}\n \t}\n \n-\t/* And check the encoding. */\n-\tif (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n-\t\tfprintf(stderr, _(commit_utf8_warn));\n-\n \tif (r->compat_hash_algo) {\n \t\thash_object_file(r->compat_hash_algo, compat_buffer.buf, compat_buffer.len,\n \t\t\tOBJ_COMMIT, &compat_oid_buf);\ndiff --git a/t/t7510-signed-commit.sh b/t/t7510-signed-commit.sh\nindex 1201c85ba6..aa9108da54 100755\n--- a/t/t7510-signed-commit.sh\n+++ b/t/t7510-signed-commit.sh\n@@ -462,4 +462,14 @@ test_expect_success 'custom `gpg.program`' '\n \tgit commit -S --allow-empty -m signed-commit\n '\n \n+test_expect_success GPG 'commit verifies with non-UTF-8 commit message' '\n+\tprintf \"I hate\\\\376\\\\377UTF-8\\\\n\" >message &&\n+\techo unusual-message >file &&\n+\tgit add file &&\n+\ttest_tick && git commit -S -F message 2>err &&\n+\tgit verify-commit HEAD &&\n+\tgrep \"commit message did not conform to UTF-8\" err >lines &&\n+\ttest_line_count = 1 lines\n+'\n+\n test_done\n"},{"id":"543136","messageId":"xmqqtssdnpf7.fsf@gitster.g","threadId":"65516","inReplyTo":"20260427221834.1824543-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2 2/2] commit: sign commit after mutating buffer","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-05-12T05:54:20Z","receivedAt":"2026-05-12T05:54:38Z","isPatch":true,"body":"\"brian m. carlson\" writes:\n\n> The ensure_utf8 function can mutate the buffer to change its encoding,\n> so we must call it before signing the buffer so that we do not\n> invalidate the signature, which is made over raw bytes. Fix a bug which\n> caused the compatibility code to not convert the compatibility buffer if\n> the main buffer was invalid UTF-8. We expect both buffers to be valid\n> UTF-8 or both invalid, since the only data that would differ between\n> them would be hex object IDs, which are always valid UTF-8.\n>\n> Add a test for this case using 0xfe and 0xff, which are never valid in\n> UTF-8.\n>\n> Reported-by: Kushal Das \n> Signed-off-by: brian m. carlson \n> ---\n> commit.c | 15 +++++++++++----\n> t/t7510-signed-commit.sh | 10 ++++++++++\n> 2 files changed, 21 insertions(+), 4 deletions(-)\n\nThis iteration hasn't seen any reaction but comparing it with the\nprevious round and peeking at comments that the previous round\nreceived, I guess everybody commented on the previous round is happy\nwith this version.\n\nLet me mark the topic for 'next'.\n\nThanks.\n\n\n>\n> diff --git a/commit.c b/commit.c\n> index 790dd2faed..e5d725fe93 100644\n> --- a/commit.c\n> +++ b/commit.c\n> @@ -1726,6 +1726,7 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> \tstruct repository *r = the_repository;\n> \tint result = 0;\n> \tint encoding_is_utf8;\n> +\tbool warned = false;\n> \tstruct strbuf buffer = STRBUF_INIT, compat_buffer = STRBUF_INIT;\n> \tstruct strbuf sig = STRBUF_INIT, compat_sig = STRBUF_INIT;\n> \tstruct object_id *parent_buf = NULL, *compat_oid = NULL;\n> @@ -1747,6 +1748,13 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> \t\toidcpy(&parent_buf[i++], &p->item->object.oid);\n> \n> \twrite_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra);\n> +\n> +\t/* And check the encoding. */\n> +\tif (encoding_is_utf8 && !ensure_utf8(&buffer)) {\n> +\t\tfprintf(stderr, _(commit_utf8_warn));\n> +\t\twarned = true;\n> +\t}\n> +\n> \tif (sign_commit && sign_buffer(&buffer, &sig, sign_commit,\n> \t\t\t\t SIGN_BUFFER_USE_DEFAULT_KEY)) {\n> \t\tresult = -1;\n> @@ -1780,6 +1788,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> \t\tfree_commit_extra_headers(compat_extra);\n> \t\tfree(mapped_parents);\n> \n> +\t\tif (encoding_is_utf8 && !ensure_utf8(&compat_buffer) && !warned)\n> +\t\t\tfprintf(stderr, _(commit_utf8_warn));\n> +\n> \t\tif (sign_commit && sign_buffer(&compat_buffer, &compat_sig,\n> \t\t\t\t\t sign_commit,\n> \t\t\t\t\t SIGN_BUFFER_USE_DEFAULT_KEY)) {\n> @@ -1818,10 +1829,6 @@ int commit_tree_extended(const char *msg, size_t msg_len,\n> \t\t}\n> \t}\n> \n> -\t/* And check the encoding. */\n> -\tif (encoding_is_utf8 && (!ensure_utf8(&buffer) || !ensure_utf8(&compat_buffer)))\n> -\t\tfprintf(stderr, _(commit_utf8_warn));\n> -\n> \tif (r->compat_hash_algo) {\n> \t\thash_object_file(r->compat_hash_algo, compat_buffer.buf, compat_buffer.len,\n> \t\t\tOBJ_COMMIT, &compat_oid_buf);\n> diff --git a/t/t7510-signed-commit.sh b/t/t7510-signed-commit.sh\n> index 1201c85ba6..aa9108da54 100755\n> --- a/t/t7510-signed-commit.sh\n> +++ b/t/t7510-signed-commit.sh\n> @@ -462,4 +462,14 @@ test_expect_success 'custom `gpg.program`' '\n> \tgit commit -S --allow-empty -m signed-commit\n> '\n> \n> +test_expect_success GPG 'commit verifies with non-UTF-8 commit message' '\n> +\tprintf \"I hate\\\\376\\\\377UTF-8\\\\n\" >message &&\n> +\techo unusual-message >file &&\n> +\tgit add file &&\n> +\ttest_tick && git commit -S -F message 2>err &&\n> +\tgit verify-commit HEAD &&\n> +\tgrep \"commit message did not conform to UTF-8\" err >lines &&\n> +\ttest_line_count = 1 lines\n> +'\n> +\n> test_done\n"}]}