Re: [RFC PATCH 0/4] sign a SHA-256 digest of the tree in commits and tags

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Oct 6, 2026, 21:55 UTC
Message-ID: <asVuadq79SNc-1y1@fruit.crustytoothpaste.net>
In-Reply-To: <d59dfe7e-5958-4a72-92d7-788521f3e55f@app.fastmail.com>

On 2026-10-06 at 16:16:07, Kristoffer Haugsbakk wrote:

Show 21 quoted lines

> And that was okay. The Git repo seems to have a bit of cruft, and
> git-fast-export(1) fails on the first error then suggests a fix so that
> you can continue on to the next error. But that’s fine for a one-shot
> program. For anyone interested:
> 
>     git fast-export --all --reencode=yes --mark-tags \
>         --signed-tags=verbatim \
>         --tag-of-filtered-object=rewrite >SHA1HERE
> 
> Some real loss of fidelity was there though:
> 
> 1. You can’t for some reason export refs that point to blobs or trees
> 2. Your Git notes will be effectively lost since they will retain their
>    SHA-1 filenames. (This is mentioned in hash-function-transition)
> 
> Then you import it with
> 
>     git fast-import
> 
> But to no one’s surprise (here) this does not work because of the SHA-1
> collision submodule.

We actually have support for rewriting submodules in fast-export and fast-import. It's a little fussy because you have to rewrite all the submodules before you rewrite the main repository, but it works. Here's a command to handle git.git:

---- #!/bin/sh

temp=$(mktemp -d) trap 'rm -fr "$temp"' EXIT

GIT_LOCATION="$1" RESULT="$2"

git -C "$GIT_LOCATION/sha1collisiondetection" fast-export --signed-tags=verbatim --tag-of-filtered-object=drop --export-marks="$temp/sha1dc-sha1.marks" --all >"$temp/sha1dc.export"

git init --bare --object-format=sha256 "$temp/sha1dc" git -C "$temp/sha1dc" fast-import --export-marks="$temp/sha1dc-sha256.marks" < "$temp/sha1dc.export"

git -C "$GIT_LOCATION" fast-export --reencode=no --signed-tags=verbatim --tag-of-filtered-object=drop --branches --tags >"$temp/git.export" git init --object-format=sha256 "$RESULT" git -C "$RESULT" fast-import --rewrite-submodules-from=sha1dc:"$temp/sha1dc-sha1.marks" --rewrite-submodules-to=sha1dc:"$temp/sha1dc-sha256.marks" <"$temp/git.export" ----

And here's an example running it right now (my main branch following `master` is `dev`):

---- % ./convert-git ~/checkouts/git git-sha256.git [elided] % git -C git-sha256.git log -1 --format=oneline dev 05370fd7088edf77bfcd09c8c909450e764a9d10d8304dc333f39f01697c5a84 4th batch for -rc1 ----

The downside is that it doesn't produce the same results as the true interoperability code and it's much slower, and, as I pointed out above, the user experience is poor. The advantage is that it's been available since the original SHA-256 work in about 2.30 or so, so you can totally make it work almost anywhere. The above script could also probably be nicely converted into a generic script that would work on any repository without too much effort.

Show 10 quoted lines

> Okay, dropping that exercise for a second. I would personally be okay
> with trying out this migration on my existing repos that are “local
> only”. It would clearly be in my interest to find any bugs that are
> particular to my workflows. But for that I would that migration where
> you keep a mapping of SHA-1 to SHA-256. Or else I will lose Git notes
> forever (which I use a lot).
> 
> But reading brian’s cousin response:
> <asQrWAKQXV9zn1Vq@fruit.crustytoothpaste.net> ... it seems that there is
> not enough in git(1) or anywhere else to do that.

The interoperability work doesn't rewrite notes because it only happens when cloning or fetching from a repository and notes aren't usually copied in that case. In-place rewriting is not yet implemented, although that's a thing I'd like to work on. Hooking notes into that shouldn't be very difficult to do.

The reason more of the interoperability work has not gone upstream is because the pluggable ODB work has really ended up breaking a lot of things[0], so sending almost anything requires a bunch of rebasing and fixing, and I'm presently very burnt out, so I'm doing very little coding in my free time and doing more cycling, reading, and Factorio: Space Age.

Show 5 quoted lines

> The above scenario would be very hyperbolic and too cynical if not for
> the context: one person is leading the direct implementation work[2] in
> their spare time. In order to migrate Git from a to-be government-wide
> banned hash algorithm. That seems like an institutional malfunction.
> Somewhere.

This is the problem with open source, unfortunately. In the ideal world, would other people and very especially major companies help out more? Sure. But macOS and FreeBSD also ship one person's bc/dc implementation as a core part of the OS, there's only one maintainer each for bash and ncurses, and a lot of other cases. This is basically https://xkcd.com/2347/, which, as we all know, is a widespread problem.

As I said elsewhere, everyone is interested in scaling Git to larger and larger repositories and improving performance, but little else gets attention. Those are things I _don't_ really want to work on, which is why my job is not working on Git.

[0] To be clear, I think it's a great project and I'm very happy to see the work come in, but it has impacts throughout the codebase.

-- 
brian m. carlson (they/them)
Toronto, Ontario, CA