[RFC][PATCH 0/32] SHA256 and SHA1 interoperability
- From
Eric W. Biederman <ebiederm@xmission.com>
- Date
- Sep 8, 2023, 23:05 UTC
- Message-ID
- <87sf7ol0z3.fsf@email.froward.int.ebiederm.org>
I would like to see the SHA256 transition happen so I started playing with the k2204-transition-interop branch of brian m. carlson's tree.
Before I go farther I need to some other folks to look at this and see if this is a general direction that the git project can stand.
This patchset is not complete it does not implement converting a received pack of the compatibility hash into the hash function of the repository, nor have I written any automated tests. Both need to happen before this is finalized.
That said I think I have working implementations of all of the interesting cases. In particular I have "git index-pack" computing the compatibility hash of every object in a pack file, and I can tell you the sha256 of every sha1 in the git://git.kernel.org/pub/scm/git/git.git
To get there I have tweaked the transition plan a little.
So far I have just aimed for code that works, so there is doubtless room for improvement. My hope is that I have implemented enough that people can play with this, and that people can see all of the weird little details that need to be taken care of to make this work.
What do everyone else think? Does this direction look plausible?
Eric W. Biederman (24):
doc hash-file-transition: A map file for mapping between sha1 and sha256
doc hash-function-transition: Replace compatObjectFormat with compatMap
object-file-convert: Stubs for converting from one object format to another
object-name: Initial support for ^{sha1} and ^{sha256}
repository: add a compatibility hash algorithm
loose: Compatibilty short name support
object-file: Update the loose object map when writing loose objects
bulk-checkin: Only accept blobs
pack: Communicate the compat_oid through struct pack_idx_entry
object-file: Add a compat_oid_in parameter to write_object_file_flags
object: Factor out parse_mode out of fast-import and tree-walk into in object.h
builtin/cat-file: Let the oid determine the output algorithm
tree-walk: init_tree_desc take an oid to get the hash algorithm
object-file: Handle compat objects in check_object_signature
builtin/ls-tree: Let the oid determine the output algorithm
builtin/pack-objects: Communicate the compatibility hash through struct pack_idx_entry
pack-compat-map: Add support for .compat files of a packfile
object-file-convert: Implement convert_object_file_{begin,step,end}
builtin/fast-import: compute compatibility hashs for imported objects
builtin/index-pack: Add a simple oid index
builtin/index-pack: Compute the compatibility hash
builtin/index-pack: Make the stack in compute_compat_oid explicit
unpack-objects: Update to compute and write the compatibility hashes
object-file-convert: Implement repo_submodule_oid_to_algopbrian m. carlson (8):
repository: Implement core.compatMap
loose: add a mapping between SHA-1 and SHA-256 for loose objects
bulk-checkin: hash object with compatibility algorithm
commit: write commits for both hashes
cache: add a function to read an OID of a specific algorithm
object-file-convert: add a function to convert trees between algorithms
object-file-convert: convert commit objects when writing
object-file-convert: convert tag commits when writingDocumentation/config/core.txt | 6 + .../technical/hash-function-transition.txt | 56 ++- Makefile | 4 + archive.c | 3 +- builtin.h | 1 + builtin/am.c | 6 +- builtin/cat-file.c | 8 +- builtin/checkout.c | 8 +- builtin/clone.c | 2 +- builtin/commit.c | 2 +- builtin/fast-import.c | 110 +++-- builtin/grep.c | 8 +- builtin/index-pack.c | 441 ++++++++++++++++++++- builtin/ls-tree.c | 5 +- builtin/merge.c | 3 +- builtin/pack-objects.c | 13 +- builtin/read-tree.c | 2 +- builtin/show-compat-map.c | 139 +++++++ builtin/stash.c | 5 +- builtin/unpack-objects.c | 14 +- bulk-checkin.c | 55 ++- bulk-checkin.h | 6 +- cache-tree.c | 4 +- commit.c | 176 +++++--- commit.h | 1 + delta-islands.c | 2 +- diff-lib.c | 2 +- fsck.c | 6 +- git.c | 1 + hash-ll.h | 3 + hash.h | 9 +- http-push.c | 2 +- list-objects.c | 2 +- loose.c | 256 ++++++++++++ loose.h | 20 + match-trees.c | 4 +- merge-ort.c | 11 +- merge-recursive.c | 2 +- merge.c | 3 +- object-file-convert.c | 366 +++++++++++++++++ object-file-convert.h | 50 +++ object-file.c | 197 +++++++-- object-name.c | 77 +++- object-store-ll.h | 13 +- object.c | 2 + object.h | 18 + pack-bitmap-write.c | 2 +- pack-compat-map.c | 334 ++++++++++++++++ pack-compat-map.h | 27 ++ pack-write.c | 158 ++++++++ pack.h | 1 + packfile.c | 15 +- reflog.c | 2 +- repository.c | 17 + repository.h | 4 + revision.c | 4 +- setup.c | 5 + setup.h | 1 + tree-walk.c | 58 ++- tree-walk.h | 7 +- tree.c | 2 +- walker.c | 2 +- 62 files changed, 2525 insertions(+), 238 deletions(-)
Eric