git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages

From
KDKushal Das <kushal@sunet.se>
Date
Apr 20, 2026, 08:59 UTC
Message-ID
<4d5d04e2-49c4-4781-a289-f8cf79570643@sunet.se>
Hi all,

Every `git commit -S` since v2.45.0 produces a permanently-BAD signature when the commit message contains bytes that are not valid UTF-8 AND `i18n.commitEncoding` is unset (i.e. the default case). Verification fails under both `gpg --verify` and any non-GnuPG signer. The failure is deterministic: it happens every time, on every non-UTF-8 commit, no card or external tooling needed.

My best guess is commit 6206089cbd0b1cb30a017ec904567f040ab4cea0 starting this (and I am maybe 100% wrong in identifying the cause).

In pre-6206089cbd `commit_tree_extended`, `verify_utf8(&buffer)` ran BEFORE `sign_with_header(&buffer, sign_commit)`. `verify_utf8` is not a simple validator -- it mutates the strbuf in place, replacing invalid-UTF-8 bytes with their Latin-1 -> UTF-8 two-byte form. The signer therefore saw the transcoded bytes, and the same transcoded bytes were then written to the object database. Signer and verifier agreed.

After 6206089cbd, the sequence in `commit_tree_extended` is:
   write_commit_tree(&buffer, ...);
   sign_commit_to_strbuf(&sig, &buffer, sign_commit);   /* pre-transcode */
   ...
   add_commit_signature(&buffer, bufs[i].sig, bufs[i].algo);
   /* and then */
   if (encoding_is_utf8 && (!verify_utf8(&buffer) || 
!verify_utf8(&compat_buffer)))
       fprintf(stderr, _(commit_utf8_warn));            /* post-sign 
transcode */
   ...
   odb_write_object_ext(..., buffer.buf, buffer.len, OBJ_COMMIT, ret, ...);

The signature in `bufs[i].sig` covers the raw (non-UTF-8) buffer. The `verify_utf8` call after `add_commit_signature` rewrites the message portion of the stored object to UTF-8. The object that hits the ODB therefore contains bytes that no longer match what the signer hashed, and any verifier that reads the commit back and re-hashes the sig-stripped buffer will find a mismatch.

As a reproducer I ran the following command with the below bash script to verify so far `podman run --rm -it -v ./git_bug.sh:/git_bug.sh:Z fedora:41 bash -c 'dnf -y install git gnupg2 >/dev/null 2>&1 && /git_bug.sh 2>&1'`

``` #!/usr/bin/env bash # git_bug.sh -- minimal in-container reproducer for the git 2.45+ # sign/store divergence on non-UTF-8 commit messages. # # Two cases: # CASE A: default git config. On git 2.45+ the signer hashes the # raw (Latin-1) bytes but the ODB stores the UTF-8-transcoded # bytes. Verify fails with BAD signature. # CASE B: same commit with `i18n.commitEncoding=iso-8859-1`. git # tags the object with an `encoding` header and skips the # in-place transcode, so sign and store bytes agree. # # Exit codes: # 0 Not reproduced. Git either predates v2.45 or carries a fix. # 1 Reproduced. CASE A BAD, CASE B GOOD (workaround confirmed). # 2 Unexpected state or missing tool. # # Dependencies: git + gpg + awk + od + a POSIX shell. the script mints its own # throwaway Ed25519 key in a temp GNUPGHOME. Host state untouched. # set -euo pipefail

step() { printf '>>> %s\n' "$*"; } say() { printf '%s\n' "$*"; } die() { printf 'git_bug.sh: %s\n' "$*" >&2; exit 2; }

for tool in git gpg awk od; do
     command -v "$tool" >/dev/null || die "$tool not found on PATH"
done

SANDBOX=$(mktemp -d -t git-bug-XXXXXX) export GNUPGHOME="$SANDBOX/gnupg" mkdir -p "$GNUPGHOME" chmod 700 "$GNUPGHOME" # Pre-create agent + gpg configs so loopback pinentry works and no # dirmngr / tty prompts can fire. cat > "$GNUPGHOME/gpg-agent.conf" <<EOF allow-loopback-pinentry default-cache-ttl 3600 max-cache-ttl 3600 EOF cat > "$GNUPGHOME/gpg.conf" <<EOF pinentry-mode loopback batch trust-model always EOF

cleanup() {
     gpgconf --homedir "$GNUPGHOME" --kill all >/dev/null 2>&1 || true
     rm -rf "$SANDBOX"
}
trap cleanup EXIT
PASS=""
step "detected: $(git --version), $(gpg --version | head -n1)"
# ---------------------------------------------------------------------
# Mint a throwaway Ed25519 signing key. Ed25519 signs in well under a
# second, unlike RSA-4096 which on some containers takes tens of
# seconds and easily reads as "the script is hung".
# ---------------------------------------------------------------------
step "minting throwaway Ed25519 key"
cat > "$SANDBOX/keygen" <<EOF
%no-protection
Key-Type: EDDSA
Key-Curve: ed25519
Key-Usage: sign,cert
Name-Real: git-bug-tester
Name-Email: bug@example.local
Expire-Date: 1y
%commit
EOF
gpg --batch --gen-key "$SANDBOX/keygen" 2>"$SANDBOX/keygen.err" \
     || { cat "$SANDBOX/keygen.err" >&2; die "key generation failed"; }
KEY_FP=$(gpg --list-keys --with-colons bug@example.local \
     | awk -F: '/^fpr:/ {print $10; exit}')
[ -n "$KEY_FP" ] || die "could not read key fingerprint"
say "key: $KEY_FP"

# Kill the gpg-agent spawned by keygen so the next invocation starts # fresh with our loopback-pinentry policy. gpgconf --kill gpg-agent >/dev/null 2>&1 || true

# gpg wrapper git will invoke as `gpg.program`. Branches on whether
# --verify is in argv so we can capture the two stdin streams to
# separate files without clobbering each other.
cat > "$SANDBOX/gpg-capture" <<WRAP
#!/usr/bin/env bash
OUT="$SANDBOX/sign_stdin.bin"
VERIFY=0
for a in "\$@"; do
     if [[ "\$a" == "--verify" ]]; then
         OUT="$SANDBOX/verify_stdin.bin"
         VERIFY=1
         break
     fi
done
if (( VERIFY )); then
     tee "\$OUT" | exec gpg "\$@"
else
     tee "\$OUT" | exec gpg \\
         --pinentry-mode loopback --passphrase "$PASS" --batch --yes "\$@"
fi
WRAP
chmod +x "$SANDBOX/gpg-capture"
# ---------------------------------------------------------------------
# Run one case: fresh repo, optional extra git config, sign a
# lone-0xa7 commit, dump the bytes, verify. Returns 0 on GOOD, 1 on
# BAD. Prints a progress line for every potentially-slow operation so
# a "is it stuck?" question is answerable from the terminal.
# ---------------------------------------------------------------------
run_case() {
     local label="$1"
     shift
     local extra=("$@")
     local repo="$SANDBOX/repo_${label}"
     rm -f "$SANDBOX/sign_stdin.bin" "$SANDBOX/verify_stdin.bin"
     mkdir -p "$repo"
     say ""
     say "=== CASE $label ==="
     say "extra git config: ${extra[*]:-<none>}"
     (
         cd "$repo"
         step "git init + config"
         git init --quiet
         git config user.name tester
         git config user.email bug@example.local
         git config user.signingkey "$KEY_FP"
         git config commit.gpgsign true
         git config gpg.program "$SANDBOX/gpg-capture"
         for kv in "${extra[@]}"; do
             git config "${kv%%=*}" "${kv#*=}"
         done
         printf 'section \xa7\n' > "$SANDBOX/msg.txt"
         echo content > f.txt
         git add f.txt
         step "git commit -F msg.txt (invokes gpg-capture to sign)"
         git commit -F "$SANDBOX/msg.txt" --quiet 
2>"$SANDBOX/commit_${label}.err" \
             || { cat "$SANDBOX/commit_${label}.err" >&2; exit 1; }
     )
     say "-- msg.txt (bytes we asked git to commit): --"
     od -An -tx1 -v "$SANDBOX/msg.txt"
     say "-- sign_stdin (bytes git fed gpg at SIGN time, tail): --"
     od -An -tx1 -v "$SANDBOX/sign_stdin.bin" | tail -n 1
     say "-- commit object body (bytes in the ODB, tail): --"
     (cd "$repo" && git cat-file commit HEAD) | od -An -tx1 -v | tail -n 1
     step "git verify-commit HEAD"
     local rc=0
     (cd "$repo" && git verify-commit HEAD) >/dev/null 2>&1 || rc=$?
     if [[ -f "$SANDBOX/verify_stdin.bin" ]]; then
         say "-- verify_stdin (bytes git fed gpg at VERIFY time, tail): --"
         od -An -tx1 -v "$SANDBOX/verify_stdin.bin" | tail -n 1
     fi
     if (( rc == 0 )); then
         say "verify: GOOD (exit 0)"
         return 0
     else
         say "verify: BAD (exit $rc)"
         return 1
     fi
}
if run_case A; then
     STATE_A=GOOD
else
     STATE_A=BAD
fi
if run_case B i18n.commitEncoding=iso-8859-1; then
     STATE_B=GOOD
else
     STATE_B=BAD
fi

say "" say "=== summary ===" say "CASE A (default commitEncoding): $STATE_A" say "CASE B (i18n.commitEncoding=iso-8859-1): $STATE_B"

if [[ "$STATE_A" == BAD && "$STATE_B" == GOOD ]]; then
     say "RESULT: reproduced. This git has the v2.45+ regression:"
     say "  verify_utf8 runs after the signature is already computed,"
     say "  so the signer sees raw bytes and the ODB stores transcoded"
     say "  bytes. Workaround confirmed: i18n.commitEncoding=iso-8859-1."
     exit 1
fi
if [[ "$STATE_A" == GOOD && "$STATE_B" == GOOD ]]; then
     say "RESULT: not reproduced. Either git predates v2.45 or has a fix."
     exit 0
fi
say "RESULT: unexpected state (A=$STATE_A B=$STATE_B)."
exit 2
```
Output:
```
 >>> detected: git version 2.52.0, gpg (GnuPG) 2.4.5
 >>> minting throwaway Ed25519 key
key: 5DDD17B4C7A2BB447EAC16F5280A03FBE5C7A8DF
=== CASE A ===
extra git config: <none>
 >>> git init + config
 >>> git commit -F msg.txt (invokes gpg-capture to sign)
-- msg.txt (bytes we asked git to commit): --
  73 65 63 74 69 6f 6e 20 a7 0a
-- sign_stdin (bytes git fed gpg at SIGN time, tail): --
  a7 0a
-- commit object body (bytes in the ODB, tail): --
  20 c2 a7 0a
 >>> git verify-commit HEAD
-- verify_stdin (bytes git fed gpg at VERIFY time, tail): --
  c2 a7 0a
verify: BAD (exit 1)
=== CASE B ===
extra git config: i18n.commitEncoding=iso-8859-1
 >>> git init + config
 >>> git commit -F msg.txt (invokes gpg-capture to sign)
-- msg.txt (bytes we asked git to commit): --
  73 65 63 74 69 6f 6e 20 a7 0a
-- sign_stdin (bytes git fed gpg at SIGN time, tail): --
  69 6f 6e 20 a7 0a
-- commit object body (bytes in the ODB, tail): --
  74 69 6f 6e 20 a7 0a
 >>> git verify-commit HEAD
-- verify_stdin (bytes git fed gpg at VERIFY time, tail): --
  69 6f 6e 20 a7 0a
verify: GOOD (exit 0)
=== summary ===
CASE A (default commitEncoding):         BAD
CASE B (i18n.commitEncoding=iso-8859-1): GOOD
RESULT: reproduced. This git has the v2.45+ regression:
   verify_utf8 runs after the signature is already computed,
   so the signer sees raw bytes and the ODB stores transcoded
   bytes. Workaround confirmed: i18n.commitEncoding=iso-8859-1.
```
I tested this in different ways.
- git 2.53.0 from Fedora 43 reproduces.
- git 2.53.0 from ubuntu-latest (GitHub Actions) reproduces.
- git 2.43.0 from Ubuntu 24.04 does NOT reproduce (v2.44 is the last
   tag without 6206089cbd).
- Any signer produces the bug -- tested with GnuPG 2.4.9 and with
   my non-GnuPG tclig [0]; both hash over the same pre-transcode bytes
   the command-line signer is piped, so both get invalidated by the
   post-sign transcode.
[0] https://crates.io/crates/tumpa-cli
Kushal
Next: brian m. carlson
Message 1 of 13 in “[BUG] v2.45+: git commit -S invalidates signature for non-UTF-8 messages”
  1. Kushal DasApr 20, 2026
  2. brian m. carlsonApr 20, 2026
  3. 1/2 commit: name UTF-8 function appropriatelybrian m. carlson, Apr 20, 2026
  4. 2/2 commit: sign commit after mutating bufferbrian m. carlson, Apr 20, 2026
  5. Elijah NewrenApr 22, 2026
  6. brian m. carlsonApr 24, 2026
  7. Elijah NewrenApr 22, 2026
  8. Kushal DasApr 21, 2026
  9. brian m. carlsonApr 21, 2026
  10. D. Ben KnobleApr 22, 2026
  11. 1/2 commit: name UTF-8 function appropriatelybrian m. carlson, Apr 27, 2026
  12. 2/2 commit: sign commit after mutating bufferbrian m. carlson, Apr 27, 2026
  13. Junio C HamanoMay 12, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.