{"thread":{"id":"64168","subject":"[PATCH 3/9] docs: reflect actual double signature for tags","startedAt":"2025-09-19T01:09:42Z","lastAt":"2025-10-14T05:54:09Z","messageCount":67,"participants":["brian m. carlson","Junio C Hamano","Patrick Steinhardt"],"isPatch":true,"patchVersion":1,"patchTotal":9},"messages":[{"id":"526747","messageId":"20250919010911.649831-4-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 3/9] docs: reflect actual double signature for tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:05Z","receivedAt":"2025-09-19T01:09:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The documentation for the hash function transition reflects the original\ndesign where the SHA-256 signature would always be placed in a header.\nHowever, due to a missed patch in Git 2.29, we shipped SHA-256 support\nsuch that the signature for the current algorithm is always an in-body\nsignature and the opposite algorithm is always in a header. Since the\ndocumentation is inaccurate, update it to reflect the correct\ninformation.\n\nSigned-off-by: brian m. carlson \n---\n .../technical/hash-function-transition.adoc | 20 ++++++++++---------\n 1 file changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex 11c4f2950a..27c90e3729 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -425,17 +425,19 @@ ordinary unsigned commit.\n \n Signed Tags\n ~~~~~~~~~~~\n-We add a new field \"gpgsig-sha256\" to the tag object format to allow\n-signing tags without relying on SHA-1. Its signed payload is the\n-SHA-256 content of the tag with its gpgsig-sha256 field and \"-----BEGIN PGP\n-SIGNATURE-----\" delimited in-body signature removed.\n+We add new fields \"gpgsig\" and \"gpgsig-sha256\" to the tag object format to\n+allow signing tags in both formats. The in-body signature is used for the\n+signature in the current hash algorithm and the header is used for the\n+signature in the other algorithm. Thus, a dual-signature tag will contain both\n+an in-body signature and a gpgsig-sha256 header for the SHA-1 format of an\n+object or both an in-body signature and a gpgsig header for the SHA-256 format\n+of and object.\n \n-This means tags can be signed\n+The signed payload of the tag is the content of the tag in the current\n+algorithm with both its gpgsig and gpgsig-sha256 fields and\n+\"-----BEGIN PGP SIGNATURE-----\" delimited in-body signature removed.\n \n-1. using SHA-1 only, as in existing signed tag objects\n-2. using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body\n- signature.\n-3. using only SHA-256, by only using the gpgsig-sha256 field.\n+This means tags can be signed using one or both algorithms.\n \n Mergetag embedding\n ~~~~~~~~~~~~~~~~~~\n"},{"id":"526748","messageId":"20250919010911.649831-2-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 1/9] docs: update pack index v3 format","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:03Z","receivedAt":"2025-09-19T01:09:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our current pack index v3 format uses 4-byte integers to find the\ntrailer of the file. This effectively means that the file cannot be\nmuch larger than 2^32. While this might at first seem to be okay, we\nexpect that each object will have at least 64 bytes worth of data, which\nmeans that no more than about 67 million objects can be stored.\n\nAgain, this might seem fine, but unfortunately, we know of many users\nwho attempt to create repos with extremely large numbers of commits to\nget a \"high score,\" and we've already seen repositories with at least 55\nmillion commits. In the interests of gracefully handling repositories\neven for these well-intentioned but ultimately misguided users, let's\nchange these lengths to 8 bytes.\n\nFor the checksums at the end of the file, we're producing 32-byte\nSHA-256 checksums because that's what we already do with pack index v2\nand SHA-256. Truncating SHA-256 doesn't pose any actual security\nproblems other than those related to the reduced size, but our pack\nchecksum must already be 32 bytes (since SHA-256 packs have 32-byte\nchecksums) and it simplifies the code to use the existing hashfile logic\nfor these cases for the index checksum as well.\n\nIn addition, even though we may not need cryptographic security for the\nindex checksum, we'd like to avoid arguments from auditors and such for\norganizations that may have compliance or security requirements. Using\nthe simple, boring choice of the full SHA-256 hash avoids all possible\ndiscussion related to hash truncation and removes impediments for these\norganizations.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/technical/hash-function-transition.adoc | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex f047fd80ca..f2df1d618d 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -227,9 +227,9 @@ network byte order):\n ** 4-byte length in bytes of shortened object names. This is the\n shortest possible length needed to make names in the shortened\n object name table unambiguous.\n- ** 4-byte integer, recording where tables relating to this format\n+ ** 8-byte integer, recording where tables relating to this format\n are stored in this index file, as an offset from the beginning.\n- * 4-byte offset to the trailer from the beginning of this file.\n+ * 8-byte offset to the trailer from the beginning of this file.\n * Zero or more additional key/value pairs (4-byte key, 4-byte\n value). Only one key is supported: 'PSRC'. See the \"Loose objects\n and unreachable objects\" section for supported values and how this\n@@ -276,10 +276,10 @@ network byte order):\n up to and not including the table of CRC32 values.\n - Zero or more NUL bytes.\n - The trailer consists of the following:\n- * A copy of the 20-byte SHA-256 checksum at the end of the\n+ * A copy of the 32-byte SHA-256 checksum at the end of the\n corresponding packfile.\n \n- * 20-byte SHA-256 checksum of all of the above.\n+ * 32-byte SHA-256 checksum of all of the above.\n \n Loose object index\n ~~~~~~~~~~~~~~~~~~\n"},{"id":"526749","messageId":"20250919010911.649831-3-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 2/9] docs: update offset order for pack index v3","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:04Z","receivedAt":"2025-09-19T01:09:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The current design of pack index v3 has items in two different orders:\nsorted shortened object ID order and pack order. The shortened object\nIDs and the pack index offset values are in the former order and\neverything else is in the latter.\n\nThis, however, poses some problems. We have many parts of the packfile\ncode that expect to find out data about an object knowing only its index\nin pack order. With the current design, to find the pack offset after\nhaving looked up the index in pack order, we must then look up the full\nobject ID and use that to look up the shortened object ID to find the\npack offset, which is inconvenient, inefficient, and leads to poor cache\nusage.\n\nInstead, let's change the offset values to be looked up by pack order.\nThis works better because once we know the pack order offset, we can\nfind the full object name and its location in the pack with a simple\nindex into their respective tables. This makes many operations much\nmore efficient, especially with the functions we already have, and it\navoids the need for the revindex with pack index v3.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/technical/hash-function-transition.adoc | 10 ++++------\n 1 file changed, 4 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex f2df1d618d..11c4f2950a 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -260,12 +260,10 @@ network byte order):\n compressed data to be copied directly from pack to pack during\n repacking without undetected data corruption.\n \n- * A table of 4-byte offset values. For an object in the table of\n- sorted shortened object names, the value at the corresponding\n- index in this table indicates where that object can be found in\n- the pack file. These are usually 31-bit pack file offsets, but\n- large offsets are encoded as an index into the next table with the\n- most significant bit set.\n+ * A table of 4-byte offset values. The index of this table in pack order\n+ indicates where that object can be found in the pack file. These are\n+ usually 31-bit pack file offsets, but large offsets are encoded as\n+ an index into the next table with the most significant bit set.\n \n * A table of 8-byte offset entries (empty for pack files less than\n 2 GiB). Pack files are organized with heavily used objects toward\n"},{"id":"526750","messageId":"20250919010911.649831-5-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 4/9] docs: improve ambiguous areas of pack format documentation","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:06Z","receivedAt":"2025-09-19T01:09:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"It is fair to say that our pack and indexing code is quite complex.\nContributors who wish to work on this code or implementors of other\nimplementations would benefit from clear, unambiguous documentation\nabout how our data formats are structured and encoded and what data is\nused in the computation of certain values. Unfortunately, some of this\ndata is missing, which leads to confusion and frustration.\n\nLet's document some of this data to help clarify things. Specify over\nwhat data CRC32 values are computed and also note which CRC32 algorithm\nis used, since Wikipedia mentions at least four 32-bit CRC algorithms\nand notes that it's possible to use different bit orderings.\n\nIn addition, note how we encode objects in the pack. One might be led\nto believe that packed objects are always stored with the \"\n\\0\" prefix of loose objects, but that is not the case, although\nfor obvious reasons this data is included in the computation of the\nobject ID. Explain why this is for the curious reader.\n\nFinally, indicate what the size field of the packed object represents.\nOtherwise, a reader might think that the size of a delta is the size of\nthe full object or that it might contain the offset or object ID,\nneither of which are the case. Explain clearly, however, that the\nvalues represent uncompressed sizes to avoid confusion.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/gitformat-pack.adoc | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/Documentation/gitformat-pack.adoc b/Documentation/gitformat-pack.adoc\nindex d6ae229be5..9b7af5c184 100644\n--- a/Documentation/gitformat-pack.adoc\n+++ b/Documentation/gitformat-pack.adoc\n@@ -32,6 +32,10 @@ In a repository using the traditional SHA-1, pack checksums, index checksums,\n and object IDs (object names) mentioned below are all computed using SHA-1.\n Similarly, in SHA-256 repositories, these values are computed using SHA-256.\n \n+CRC32 checksums are always computed over the entire packed object, including\n+the header (n-byte type and length); the base object name or offset, if any;\n+and the entire compressed object. The CRC32 algorithm used is that of zlib.\n+\n == pack-*.pack files have the following format:\n \n - A header appears at the beginning and consists of the following:\n@@ -80,6 +84,15 @@ Valid object types are:\n \n Type 5 is reserved for future expansion. Type 0 is invalid.\n \n+=== Object encoding\n+\n+Unlike loose objects, packed objects do not have a prefix containing the type,\n+size, and a NUL byte. These are not necessary because they can be determined by\n+the n-byte type and length that prefixes the data and so they are omitted from\n+the compressed and deltified data.\n+\n+The computation of the object ID still uses this prefix, however.\n+\n === Size encoding\n \n This document uses the following \"size encoding\" of non-negative\n@@ -92,6 +105,11 @@ values are more significant.\n This size encoding should not be confused with the \"offset encoding\",\n which is also used in this document.\n \n+When encoding the size of an undeltified object in a pack, the size is that of\n+the uncompressed raw object. For deltified objects, it is the size of the\n+uncompressed delta. The base object name or offset is not included in the size\n+computation.\n+\n === Deltified representation\n \n Conceptually there are only four object types: commit, tree, tag and\n"},{"id":"526751","messageId":"20250919010911.649831-1-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":null,"subject":"[PATCH 0/9] SHA-1/SHA-256 interoperability, part 1","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:02Z","receivedAt":"2025-09-19T01:09:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This is the first of several series for SHA-1 and SHA-256\ninteroperability, all of which will hopefully land in Git 3.0.\n\nThe first set of commits here is documentation updates for documentation\nwhich was incorrect, missing, or simply wrong. I have spent more time\nthan I'd like in the pack code and felt our documentation there could be\nmore helpful. I also am correcting some things about the\ninteroperability formats that I've found are not correct or efficient in\nterms of implementation and thus I will be implementing differently.\n\nThe loose object documentation will be updated with the loose object\nmapping in a future commit, but I felt I should send a basic loose\nobject document first, so here it is.\n\nThe remaining commits are for expected gpgsig headers in tags, which\ncauses some tests which use strict fsck to fail, as well as for\nprerequisites for compatibility hashes in the testsuite. Actually using\nthis configuration is not possible since the tests are still very broken\nusing it, but declaring these prerequisites allows me and others to send\nin patches that use them and thus make our testsuite more resilient.\n\nFor example, in interoperability mode we cannot write objects that\nare not valid since we cannot convert them into the other hash\nalgorithm. Thus, when we're testing in a mode that has a compatibility\nalgorithm, we skip these tests.\n\nThe goal is to run the tests in a full compatibility mode where\neverything is dual-hash as well as introduce some specific tests for\ninteroperability that run in all configurations of the tests.\n\nThis series should be relatively boring and noncontroversial. The more\nsubstantive, interesting, and controversial patches (in that they\ncontain Rust code) will require `ps/rust-balloon` to hit master and will\nfollow as part 2. You need not clutch your pearls yet.\n\nbrian m. carlson (9):\n docs: update pack index v3 format\n docs: update offset order for pack index v3\n docs: reflect actual double signature for tags\n docs: improve ambiguous areas of pack format documentation\n docs: add documentation for loose objects\n rev-parse: allow printing compatibility hash\n fsck: consider gpgsig headers expected in tags\n Allow specifying compatibility hash\n t: add a prerequisite for a compatibility hash\n\n Documentation/fsck-msgids.adoc | 6 +++\n Documentation/git-rev-parse.adoc | 11 ++--\n Documentation/gitformat-loose.adoc | 49 +++++++++++++++++\n Documentation/gitformat-pack.adoc | 18 +++++++\n .../technical/hash-function-transition.adoc | 38 ++++++-------\n builtin/rev-parse.c | 11 +++-\n fsck.c | 18 +++++++\n fsck.h | 2 +\n t/t1450-fsck.sh | 54 +++++++++++++++++++\n t/t1500-rev-parse.sh | 34 ++++++++++++\n t/test-lib-functions.sh | 9 +++-\n t/test-lib.sh | 13 +++++\n 12 files changed, 236 insertions(+), 27 deletions(-)\n create mode 100644 Documentation/gitformat-loose.adoc\n\n"},{"id":"526752","messageId":"20250919010911.649831-9-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 8/9] Allow specifying compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:10Z","receivedAt":"2025-09-19T01:09:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We want to specify a compatibility hash for testing interactions for\nSHA-256 repositories where we have SHA-1 compatibility enabled. Allow\nthe user to specify this scenario in the test suite by setting\nGIT_TEST_DEFAULT_HASH to \"sha256:sha1\".\n\nNote that this will get passed into GIT_DEFAULT_HASH, which Git itself\ndoes not presently support. However, we will support this in a future\ncommit.\n\nSince we'll now want to know the value for a specific version, let's add\nthe ability to specify either the storage hash (in this case, SHA-256)\nor the compatibility hash (SHA-1). We use a different value for the\ncompatibility hash that will be enabled for all repositories\n(test_repo_compat_hash_algo) versus the one that is used individually in\nsome tests (test_compat_hash_algo), since we want to still run those\nindividual tests without requiring that the testsuite be run fully in a\ncompatibility mode.\n\nFinally, in this scenario, we can no longer rely on having broken\nobjects work since we lack compatibility mappings to rewrite objects in\nthe repository. Add a prerequisite, BROKEN_OBJECTS, that checks to see\nif creating deliberately broken objects is possible, so that we can\ndisable these tests if not.\n\nSigned-off-by: brian m. carlson \n---\n t/test-lib-functions.sh | 9 +++++++--\n t/test-lib.sh | 7 +++++++\n 2 files changed, 14 insertions(+), 2 deletions(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex a28de7b19b..52d7759bf5 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1708,11 +1708,16 @@ test_set_hash () {\n # Detect the hash algorithm in use.\n test_detect_hash () {\n \tcase \"${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}\" in\n-\t\"sha256\")\n+\t*:*)\n+\t test_hash_algo=\"${GIT_TEST_DEFAULT_HASH%%:*}\"\n+\t test_compat_hash_algo=\"${GIT_TEST_DEFAULT_HASH##*:}\"\n+\t test_repo_compat_hash_algo=\"$test_compat_hash_algo\"\n+\t ;;\n+\tsha256)\n \t test_hash_algo=sha256\n \t test_compat_hash_algo=sha1\n \t ;;\n-\t*)\n+\tsha1)\n \t test_hash_algo=sha1\n \t test_compat_hash_algo=sha256\n \t ;;\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 621cd31ae1..14c777e4e2 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1917,6 +1917,13 @@ test_lazy_prereq DEFAULT_HASH_ALGORITHM '\n test_lazy_prereq DEFAULT_REPO_FORMAT '\n \ttest_have_prereq SHA1,REFFILES\n '\n+# BROKEN_OBJECTS is a test if we can write deliberately broken objects and\n+# expect them to work. When running using SHA-256 mode with SHA-1\n+# compatibility, we cannot write such objects because there's no SHA-1\n+# compatibility value for a nonexistent object.\n+test_lazy_prereq BROKEN_OBJECTS '\n+\ttest -z \"$test_repo_compat_hash_algo\"\n+'\n \n # Ensure that no test accidentally triggers a Git command\n # that runs the actual maintenance scheduler, affecting a user's\n"},{"id":"526753","messageId":"20250919010911.649831-6-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 5/9] docs: add documentation for loose objects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:07Z","receivedAt":"2025-09-19T01:09:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We currently have no documentation for how loose objects are stored.\nLet's add some here so its easy for people to understand how they\nwork.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/gitformat-loose.adoc | 49 ++++++++++++++++++++++++++++++\n 1 file changed, 49 insertions(+)\n create mode 100644 Documentation/gitformat-loose.adoc\n\ndiff --git a/Documentation/gitformat-loose.adoc b/Documentation/gitformat-loose.adoc\nnew file mode 100644\nindex 0000000000..c8bef606fb\n--- /dev/null\n+++ b/Documentation/gitformat-loose.adoc\n@@ -0,0 +1,49 @@\n+gitformat-loose(5)\n+==================\n+\n+NAME\n+----\n+gitformat-loose - Git loose object format\n+\n+\n+SYNOPSIS\n+--------\n+[verse]\n+$GIT_DIR/objects/[0-9a-f][0-9a-f]/*\n+$GIT_DIR/objects/loose-object-idx\n+$GIT_DIR/objects/loose-map/map-*.map\n+\n+DESCRIPTION\n+-----------\n+\n+Loose objects are how Git initially stores most of its primary repository data.\n+Over the lifetime of a repository, objects are usually written as loose objects\n+initially and then converted into packs.\n+\n+== Loose objects\n+\n+Each loose object contains a prefix, followed immediately by the data of the\n+object. The prefix contains ` \\0`. `` is one of `blob`,\n+`tree`, `commit`, or `tag` and `size` is the size of the data (without the\n+prefix) as a decimal integer expressed in ASCII.\n+\n+The entire contents, prefix and data concatenated, is then compressed with zlib\n+and the compressed data is stored in the file. The object ID of the object is\n+the SHA-1 or SHA-256 (as appropriate) hash of the uncompressed data.\n+\n+The file for the loose object is stored under the `objects` directory, with the\n+first two hex characters of the object ID being the directory and the remaining\n+characters being the file name.\n+\n+As an example, the empty tree contains the data (when uncompressed) `tree 0\\0`\n+and, in a SHA-256 repository, would have the object ID\n+`6ef19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321` and would be\n+stored under\n+`$GIT_DIR/objects/6e/f19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321`.\n+\n+Similarly, a blob containing the contents `abc` would have the uncompressed\n+data of `blob 3\\0abc`.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\n"},{"id":"526754","messageId":"20250919010911.649831-7-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 6/9] rev-parse: allow printing compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:08Z","receivedAt":"2025-09-19T01:09:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Right now, we have a way to print the storage hash, the input hash, and\nthe output hash, but we lack a way to print the compatibility hash. Add\na new type to --show-object-format, compat, which prints this value.\n\nIf no compatibility hash exists, simply print a newline. This is\nimportant to allow users to use multiple options at once while still\ngetting unambiguous output.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-rev-parse.adoc | 11 ++++++-----\n builtin/rev-parse.c | 11 ++++++++++-\n t/t1500-rev-parse.sh | 34 ++++++++++++++++++++++++++++++++\n 3 files changed, 50 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-rev-parse.adoc b/Documentation/git-rev-parse.adoc\nindex cc32b4b4f0..465ae3e29d 100644\n--- a/Documentation/git-rev-parse.adoc\n+++ b/Documentation/git-rev-parse.adoc\n@@ -324,11 +324,12 @@ The following options are unaffected by `--path-format`:\n \tpath of the current directory relative to the top-level\n \tdirectory.\n \n---show-object-format[=(storage|input|output)]::\n-\tShow the object format (hash algorithm) used for the repository\n-\tfor storage inside the `.git` directory, input, or output. For\n-\tinput, multiple algorithms may be printed, space-separated.\n-\tIf not specified, the default is \"storage\".\n+--show-object-format[=(storage|input|output|compat)]::\n+\tShow the object format (hash algorithm) used for the repository for storage\n+\tinside the `.git` directory, input, output, or compatibility. For input,\n+\tmultiple algorithms may be printed, space-separated. If `compat` is\n+\trequested and no compatibility algorithm is enabled, prints an empty line. If\n+\tnot specified, the default is \"storage\".\n \n --show-ref-format::\n \tShow the reference storage format used for the repository.\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 44ff1b8342..187b7e8be9 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -1108,11 +1108,20 @@ int cmd_rev_parse(int argc,\n \t\t\t\tconst char *val = arg ? arg : \"storage\";\n \n \t\t\t\tif (strcmp(val, \"storage\") &&\n+\t\t\t\t strcmp(val, \"compat\") &&\n \t\t\t\t strcmp(val, \"input\") &&\n \t\t\t\t strcmp(val, \"output\"))\n \t\t\t\t\tdie(_(\"unknown mode for --show-object-format: %s\"),\n \t\t\t\t\t arg);\n-\t\t\t\tputs(the_hash_algo->name);\n+\n+\t\t\t\tif (!strcmp(val, \"compat\")) {\n+\t\t\t\t\tif (the_repository->compat_hash_algo)\n+\t\t\t\t\t\tputs(the_repository->compat_hash_algo->name);\n+\t\t\t\t\telse\n+\t\t\t\t\t\tputchar('\\n');\n+\t\t\t\t} else {\n+\t\t\t\t\tputs(the_hash_algo->name);\n+\t\t\t\t}\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 58a4583088..98c5a772bd 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -207,6 +207,40 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n '\n \n+\n+test_expect_success RUST 'rev-parse --show-object-format in repo with compat mode' '\n+\tmkdir repo &&\n+\t(\n+\t\tsane_unset GIT_DEFAULT_HASH &&\n+\t\tcd repo &&\n+\t\tgit init --object-format=sha256 &&\n+\t\tgit config extensions.compatobjectformat sha1 &&\n+\t\techo sha256 >expect &&\n+\t\tgit rev-parse --show-object-format >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=storage >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=input >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=output >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\techo sha1 >expect &&\n+\t\tgit rev-parse --show-object-format=compat >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\ttest_must_fail git rev-parse --show-object-format=squeamish-ossifrage 2>err &&\n+\t\tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n+\t) &&\n+\tmkdir repo2 &&\n+\t(\n+\t\tsane_unset GIT_DEFAULT_HASH &&\n+\t\tcd repo2 &&\n+\t\tgit init --object-format=sha256 &&\n+\t\techo >expect &&\n+\t\tgit rev-parse --show-object-format=compat >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_expect_success 'rev-parse --show-ref-format' '\n \ttest_detect_ref_format >expect &&\n \tgit rev-parse --show-ref-format >actual &&\n"},{"id":"526755","messageId":"20250919010911.649831-8-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 7/9] fsck: consider gpgsig headers expected in tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:09Z","receivedAt":"2025-09-19T01:09:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When we're creating a tag, we want to make sure that gpgsig and\ngpgsig-sha256 headers are allowed for the commit. The default fsck\nbehavior is to ignore the fact that they're left over, but some of our\ntests enable strict checking which flags them nonetheless. Add\nimproved checking for these headers as well as documentation and several\ntests.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/fsck-msgids.adoc | 6 ++++\n fsck.c | 18 ++++++++++++\n fsck.h | 2 ++\n t/t1450-fsck.sh | 54 ++++++++++++++++++++++++++++++++++\n 4 files changed, 80 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 0ba4f9a27e..52d9a8a811 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -10,6 +10,12 @@\n `badFilemode`::\n \t(INFO) A tree contains a bad filemode entry.\n \n+`badGpgsig`::\n+\t(ERROR) A tag contains a bad (truncated) signature (e.g., `gpgsig`) header.\n+\n+`badHeaderContinuation`::\n+\t(ERROR) A continuation header (such as for `gpgsig`) is unexpectedly truncated.\n+\n `badName`::\n \t(ERROR) An author/committer name is empty.\n \ndiff --git a/fsck.c b/fsck.c\nindex 171b424dd5..341e100d24 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1067,6 +1067,24 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,\n \telse\n \t\tret = fsck_ident(&buffer, oid, OBJ_TAG, options);\n \n+\tif (buffer < buffer_end && (skip_prefix(buffer, \"gpgsig \", &buffer) || skip_prefix(buffer, \"gpgsig-sha256 \", &buffer))) {\n+\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n+\t\tif (!eol) {\n+\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_GPGSIG, \"invalid format - unexpected end after 'gpgsig' or 'gpgsig-sha256' line\");\n+\t\t\tgoto done;\n+\t\t}\n+\t\tbuffer = eol + 1;\n+\n+\t\twhile (buffer < buffer_end && starts_with(buffer, \" \")) {\n+\t\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n+\t\t\tif (!eol) {\n+\t\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_HEADER_CONTINUATION, \"invalid format - unexpected end in 'gpgsig' or 'gpgsig-sha256' continuation line\");\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t\tbuffer = eol + 1;\n+\t\t}\n+\t}\n+\n \tif (buffer < buffer_end && !starts_with(buffer, \"\\n\")) {\n \t\t/*\n \t\t * The verify_headers() check will allow\ndiff --git a/fsck.h b/fsck.h\nindex dd7df3d5b3..c26616d7eb 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -25,9 +25,11 @@ enum fsck_msg_type {\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n \t/* errors */ \\\n+\tFUNC(BAD_HEADER_CONTINUATION, ERROR) \\\n \tFUNC(BAD_DATE, ERROR) \\\n \tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n \tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_GPGSIG, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex 5ae86c42be..c4b651c2dc 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -454,6 +454,60 @@ test_expect_success 'tag with NUL in header' '\n \ttest_grep \"error in tag $tag.*unterminated header: NUL at offset\" out\n '\n \n+test_expect_success 'tag accepts gpgsig header even if not validly signed' '\n+\ttest_oid_cache <<-\\EOF &&\n+\theader sha1:gpgsig-sha256\n+\theader sha256:gpgsig\n+\tEOF\n+\theader=$(test_oid header) &&\n+\tsha=$(git rev-parse HEAD) &&\n+\tcat >good-tag <<-EOF &&\n+\tobject $sha\n+\ttype commit\n+\ttag good\n+\ttagger T A Gger 1234567890 -0000\n+\t$header -----BEGIN PGP SIGNATURE-----\n+\t Not a valid signature\n+\t -----END PGP SIGNATURE-----\n+\n+\tThis is a good tag.\n+\tEOF\n+\n+\ttag=$(git hash-object --literally -t tag -w --stdin bad-tag <<-EOF &&\n+\tobject $sha\n+\ttype commit\n+\ttag good\n+\ttagger T A Gger 1234567890 -0000\n+\t$header -----BEGIN PGP SIGNATURE-----\n+\t Not a valid signature\n+\t -----END PGP SIGNATURE-----\n+\tjunk\n+\n+\tThis is a bad tag with junk at the end of the headers.\n+\tEOF\n+\n+\ttag=$(git hash-object --literally -t tag -w --stdin out &&\n+\ttest_grep \"error in tag $tag.*invalid format - extra header\" out\n+'\n+\n test_expect_success 'cleaned up' '\n \tgit fsck >actual 2>&1 &&\n \ttest_must_be_empty actual\n"},{"id":"526756","messageId":"20250919010911.649831-10-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH 9/9] t: add a prerequisite for a compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T01:09:11Z","receivedAt":"2025-09-19T01:09:45Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"In some cases, we'll need to adjust our test suite to work in a proper\nway with a compatibility hash. For example, in such a case, we'll only\nuse pack index v3, since v1 and v2 lack support for multiple algorithms.\nSince we won't want to write those older formats, we'll need to skip\ntests that do so.\n\nLet's add a COMPAT_HASH prerequisite and define the BROKEN_OBJECTS\nprerequisite in terms of it.\n\nSigned-off-by: brian m. carlson \n---\n t/test-lib.sh | 8 +++++++-\n 1 file changed, 7 insertions(+), 1 deletion(-)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 14c777e4e2..a4bb9ab2d8 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1922,7 +1922,13 @@ test_lazy_prereq DEFAULT_REPO_FORMAT '\n # compatibility, we cannot write such objects because there's no SHA-1\n # compatibility value for a nonexistent object.\n test_lazy_prereq BROKEN_OBJECTS '\n-\ttest -z \"$test_repo_compat_hash_algo\"\n+\t! test_have_prereq COMPAT_HASH\n+'\n+\n+# COMPAT_HASH is a test if we're operating in a repository with SHA-256 with\n+# SHA-1 compatibility.\n+test_lazy_prereq COMPAT_HASH '\n+\ttest -n \"$test_repo_compat_hash_algo\"\n '\n \n # Ensure that no test accidentally triggers a Git command\n"},{"id":"526799","messageId":"xmqqv7le45t3.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T19:10:16Z","receivedAt":"2025-09-19T19:10:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> We currently have no documentation for how loose objects are stored.\n> Let's add some here so its easy for people to understand how they\n> work.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/gitformat-loose.adoc | 49 ++++++++++++++++++++++++++++++\n> 1 file changed, 49 insertions(+)\n> create mode 100644 Documentation/gitformat-loose.adoc\n\nFails a build, unfortunately.\n\n ...\n LINT DOCSTYLE includes/cmd-config-section-rest.adoc\n GEN lint-docs-manpages\n LINT DOCSTYLE includes/cmd-config-section-all.adoc\ntmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: char 3297, line 176\nMeson man pages differ from actual man pages:\n--- tmp-meson-diff/meson.adoc 2025-09-19 12:04:55.145229743 -0700\n+++ tmp-meson-diff/actual.adoc 2025-09-19 12:04:55.149229734 -0700\n@@ -173,6 +173,7 @@\n gitformat-chunk.adoc\n gitformat-commit-graph.adoc\n gitformat-index.adoc\n+gitformat-loose.adoc\n gitformat-pack.adoc\n gitformat-signature.adoc\n gitglossary.adoc\n\nThanks.\n"},{"id":"526800","messageId":"xmqqms6q45nl.fsf@gitster.g","threadId":"64168","inReplyTo":"xmqqv7le45t3.fsf@gitster.g","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T19:13:34Z","receivedAt":"2025-09-19T19:13:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> \"brian m. carlson\" writes:\n>\n>> We currently have no documentation for how loose objects are stored.\n>> Let's add some here so its easy for people to understand how they\n>> work.\n>>\n>> Signed-off-by: brian m. carlson \n>> ---\n>> Documentation/gitformat-loose.adoc | 49 ++++++++++++++++++++++++++++++\n>> 1 file changed, 49 insertions(+)\n>> create mode 100644 Documentation/gitformat-loose.adoc\n>\n> Fails a build, unfortunately.\n>\n> ...\n> LINT DOCSTYLE includes/cmd-config-section-rest.adoc\n> GEN lint-docs-manpages\n> LINT DOCSTYLE includes/cmd-config-section-all.adoc\n> tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: char 3297, line 176\n> Meson man pages differ from actual man pages:\n> --- tmp-meson-diff/meson.adoc 2025-09-19 12:04:55.145229743 -0700\n> +++ tmp-meson-diff/actual.adoc 2025-09-19 12:04:55.149229734 -0700\n> @@ -173,6 +173,7 @@\n> gitformat-chunk.adoc\n> gitformat-commit-graph.adoc\n> gitformat-index.adoc\n> +gitformat-loose.adoc\n> gitformat-pack.adoc\n> gitformat-signature.adoc\n> gitglossary.adoc\n>\n> Thanks.\n\nProbably this should be sufficient? Not tested (yet).\n\n\n\ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex 4404c623f0..93fa3dee8b 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -171,6 +171,7 @@ manpages = {\n 'gitformat-chunk.adoc' : 5,\n 'gitformat-commit-graph.adoc' : 5,\n 'gitformat-index.adoc' : 5,\n+ 'gitformat-loose.adoc' : 5,\n 'gitformat-pack.adoc' : 5,\n 'gitformat-signature.adoc' : 5,\n 'githooks.adoc' : 5,\n-- \n2.51.0-409-gb2b0f57e0f\n\n"},{"id":"526801","messageId":"aM2rv33nL8swECpn@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqqms6q45nl.fsf@gitster.g","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-19T19:15:11Z","receivedAt":"2025-09-19T19:15:19Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-19 at 19:13:34, Junio C Hamano wrote:\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 4404c623f0..93fa3dee8b 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -171,6 +171,7 @@ manpages = {\n> 'gitformat-chunk.adoc' : 5,\n> 'gitformat-commit-graph.adoc' : 5,\n> 'gitformat-index.adoc' : 5,\n> + 'gitformat-loose.adoc' : 5,\n> 'gitformat-pack.adoc' : 5,\n> 'gitformat-signature.adoc' : 5,\n> 'githooks.adoc' : 5,\n\nI'll figure it out for v2. Thanks for the tip; I'll send out v2 once\nothers have provided any other comments.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"526813","messageId":"xmqqwm5u2o2d.fsf@gitster.g","threadId":"64168","inReplyTo":"xmqqms6q45nl.fsf@gitster.g","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T20:18:50Z","receivedAt":"2025-09-19T20:18:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Probably this should be sufficient? Not tested (yet).\n\nI did test and my local build of 'seen' no longer barfs with this.\n\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 4404c623f0..93fa3dee8b 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -171,6 +171,7 @@ manpages = {\n> 'gitformat-chunk.adoc' : 5,\n> 'gitformat-commit-graph.adoc' : 5,\n> 'gitformat-index.adoc' : 5,\n> + 'gitformat-loose.adoc' : 5,\n> 'gitformat-pack.adoc' : 5,\n> 'gitformat-signature.adoc' : 5,\n> 'githooks.adoc' : 5,\n"},{"id":"526826","messageId":"xmqq7bxu14fw.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/9] docs: update pack index v3 format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T22:08:03Z","receivedAt":"2025-09-19T22:08:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> Our current pack index v3 format uses 4-byte integers to find the\n> trailer of the file. This effectively means that the file cannot be\n> much larger than 2^32. While this might at first seem to be okay, we\n> expect that each object will have at least 64 bytes worth of data, which\n> means that no more than about 67 million objects can be stored.\n>\n> Again, this might seem fine, but unfortunately, we know of many users\n> who attempt to create repos with extremely large numbers of commits to\n> get a \"high score,\" and we've already seen repositories with at least 55\n> million commits. In the interests of gracefully handling repositories\n> even for these well-intentioned but ultimately misguided users, let's\n> change these lengths to 8 bytes.\n\nVery sensible.\n\nI do also agree that 32-byte is the natural size for the trailing\nhash, but I found that the two paragraphs below was far more than\nnecessary. As they argue, we use a truncated hash anywhere in our\nfile formats, so I would have understood if the explanation were\n\n \"20\" in \"A copy of the 20-byte SHA-256 checksum\" is an obvious\n typo, as SHA-256 is longer than that. Fix it to \"32\".\n\ninstead of these two paragraphs.\n\nOr did we mean to use a truncated hash back when this transition\ndesign was proposed originally?\n\n> For the checksums at the end of the file, we're producing 32-byte\n> SHA-256 checksums because that's what we already do with pack index v2\n> and SHA-256. Truncating SHA-256 doesn't pose any actual security\n> problems other than those related to the reduced size, but our pack\n> checksum must already be 32 bytes (since SHA-256 packs have 32-byte\n> checksums) and it simplifies the code to use the existing hashfile logic\n> for these cases for the index checksum as well.\n>\n> In addition, even though we may not need cryptographic security for the\n> index checksum, we'd like to avoid arguments from auditors and such for\n> organizations that may have compliance or security requirements. Using\n> the simple, boring choice of the full SHA-256 hash avoids all possible\n> discussion related to hash truncation and removes impediments for these\n> organizations.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/technical/hash-function-transition.adoc | 8 ++++----\n> 1 file changed, 4 insertions(+), 4 deletions(-)\n>\n> diff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\n> index f047fd80ca..f2df1d618d 100644\n> --- a/Documentation/technical/hash-function-transition.adoc\n> +++ b/Documentation/technical/hash-function-transition.adoc\n> @@ -227,9 +227,9 @@ network byte order):\n> ** 4-byte length in bytes of shortened object names. This is the\n> shortest possible length needed to make names in the shortened\n> object name table unambiguous.\n> - ** 4-byte integer, recording where tables relating to this format\n> + ** 8-byte integer, recording where tables relating to this format\n> are stored in this index file, as an offset from the beginning.\n> - * 4-byte offset to the trailer from the beginning of this file.\n> + * 8-byte offset to the trailer from the beginning of this file.\n> * Zero or more additional key/value pairs (4-byte key, 4-byte\n> value). Only one key is supported: 'PSRC'. See the \"Loose objects\n> and unreachable objects\" section for supported values and how this\n> @@ -276,10 +276,10 @@ network byte order):\n> up to and not including the table of CRC32 values.\n> - Zero or more NUL bytes.\n> - The trailer consists of the following:\n> - * A copy of the 20-byte SHA-256 checksum at the end of the\n> + * A copy of the 32-byte SHA-256 checksum at the end of the\n> corresponding packfile.\n> \n> - * 20-byte SHA-256 checksum of all of the above.\n> + * 32-byte SHA-256 checksum of all of the above.\n> \n> Loose object index\n> ~~~~~~~~~~~~~~~~~~\n"},{"id":"526828","messageId":"xmqq348i138l.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-4-sandals@crustytoothpaste.net","subject":"Re: [PATCH 3/9] docs: reflect actual double signature for tags","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T22:34:02Z","receivedAt":"2025-09-19T22:34:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> Signed Tags\n> ~~~~~~~~~~~\n> +We add new fields \"gpgsig\" and \"gpgsig-sha256\" to the tag object format to\n> +allow signing tags in both formats. The in-body signature is used for the\n> +signature in the current hash algorithm and the header is used for the\n> +signature in the other algorithm. Thus, a dual-signature tag will contain both\n\nNot suggesting a change in the text, but to make sure I am reading\nthe new text correctly. Does \"the other algorithm\" refer to the\ncompatibility hash algorithm specified by the compatObjectFormat\nextension and the \"current\" algorithm refers to the objectFormat\nextension?\n\n> +an in-body signature and a gpgsig-sha256 header for the SHA-1 format of an\n> +object or both an in-body signature and a gpgsig header for the SHA-256 format\n> +of and object.\n> \n> -This means tags can be signed\n> +The signed payload of the tag is the content of the tag in the current\n> +algorithm with both its gpgsig and gpgsig-sha256 fields and\n\nMy reading of the previous paragraph is that we cannot have gpgsig\nand gpgsig-sha256 fields on a single object at the same time.\n\nShould we say \"gpgsig or gpgsig-sha256\" (instead of \"and\"), to get\nthe resulting text parsable as:\n\n\tboth \n\t\tits gpgsig or gpgsig-sha256 fields\n\tand\n\t\t\"-----BEGIN PGP SIGNATURE-----\" delimited in-body signature\n\tremoved.\n\ninstead?\n\n> +\"-----BEGIN PGP SIGNATURE-----\" delimited in-body signature removed.\n> \n> -1. using SHA-1 only, as in existing signed tag objects\n> -2. using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body\n> - signature.\n> -3. using only SHA-256, by only using the gpgsig-sha256 field.\n> +This means tags can be signed using one or both algorithms.\n> \n> Mergetag embedding\n> ~~~~~~~~~~~~~~~~~~\n"},{"id":"526836","messageId":"xmqqtt0yyrgn.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-5-sandals@crustytoothpaste.net","subject":"Re: [PATCH 4/9] docs: improve ambiguous areas of pack format documentation","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T23:04:24Z","receivedAt":"2025-09-19T23:04:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> +=== Object encoding\n> +\n> +Unlike loose objects, packed objects do not have a prefix containing the type,\n> +size, and a NUL byte. These are not necessary because they can be determined by\n> +the n-byte type and length that prefixes the data and so they are omitted from\n> +the compressed and deltified data.\n> +\n> +The computation of the object ID still uses this prefix, however.\n\nNot wrong per-se, but I've always viewd that the in-pack object\nheader with n-byte type and length was an optimized representation\nthat stands in for the textual type+size+NUL, just like the payload\npart also uses object representation different from that is used for\nloose objects for performance.\n\nAnd when you view the in-pack object header that way, \"are not\nnecessary\" and everything follows in the above appear to somewhat\nmiss the point. It is not just \"type size\" that is recreated\non the fly for computation of the same object name as in the loose\nobject form, but the payload also is recreated on the fly to match\nwhat loose object would have had, e.g., a deltified representation\nwould be reconstituted into non-deltified form, etc.\n\nIOW, I would have exprected the description to go more along this\nline intead.\n\n Packed objects use the n-byte type and length in-pack object\n header, with in-pack specific representation of the object data.\n In order to compute the same object name as if the object were\n loose, the object representation used in the loose object is\n virtually recreated by translating n-byte type and length to the\n textual type + size + NUL, concatenated with the undeltified and\n inflated object data and hashing the result.\n\n> === Size encoding\n> \n> This document uses the following \"size encoding\" of non-negative\n> @@ -92,6 +105,11 @@ values are more significant.\n> This size encoding should not be confused with the \"offset encoding\",\n> which is also used in this document.\n> \n> +When encoding the size of an undeltified object in a pack, the size is that of\n> +the uncompressed raw object. For deltified objects, it is the size of the\n> +uncompressed delta. The base object name or offset is not included in the size\n> +computation.\n\nThis is an important point worth describing. Very nice.\n\nIf we wanted to help the curious, we can say that these are used to\nboth help us know beforehand how much memory to allocate, before we\ninflate and/or to run patch-delta on the payload.\n\nThanks.\n"},{"id":"526839","messageId":"xmqqms6qyqw9.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T23:16:38Z","receivedAt":"2025-09-19T23:16:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> We currently have no documentation for how loose objects are stored.\n> Let's add some here so its easy for people to understand how they\n> work.\n>\n> Signed-off-by: brian m. carlson \n> ---\n> Documentation/gitformat-loose.adoc | 49 ++++++++++++++++++++++++++++++\n> 1 file changed, 49 insertions(+)\n> create mode 100644 Documentation/gitformat-loose.adoc\n>\n> diff --git a/Documentation/gitformat-loose.adoc b/Documentation/gitformat-loose.adoc\n> new file mode 100644\n> index 0000000000..c8bef606fb\n> --- /dev/null\n> +++ b/Documentation/gitformat-loose.adoc\n> @@ -0,0 +1,49 @@\n> +gitformat-loose(5)\n> +==================\n> +\n> +NAME\n> +----\n> +gitformat-loose - Git loose object format\n> +\n> +\n> +SYNOPSIS\n> +--------\n> +[verse]\n> +$GIT_DIR/objects/[0-9a-f][0-9a-f]/*\n> +$GIT_DIR/objects/loose-object-idx\n> +$GIT_DIR/objects/loose-map/map-*.map\n> +\n> +DESCRIPTION\n> +-----------\n> +\n> +Loose objects are how Git initially stores most of its primary repository data.\n\n\"most of\" is a bit misleading, I would think. Those who start (what\neventuall becomes) a large project from scratch are only minority of\nthe users, and all others start with \"git clone\" from elsewhere, and\nin the resulting repository, Git initially stores most of its data\nin a packfile (or two).\n\nI think it may become a bit clearer if we drop \"initially\", and end\nthe sentence with \"data that are created locally\", perhaps?\n\n> +Over the lifetime of a repository, objects are usually written as loose objects\n> +initially and then converted into packs.\n\nThis one is good.\n\n> +== Loose objects\n> +\n> +Each loose object contains a prefix, followed immediately by the data of the\n> +object. The prefix contains ` \\0`. `` is one of `blob`,\n> +`tree`, `commit`, or `tag` and `size` is the size of the data (without the\n> +prefix) as a decimal integer expressed in ASCII.\n> +\n> +The entire contents, prefix and data concatenated, is then compressed with zlib\n> +and the compressed data is stored in the file. The object ID of the object is\n\nThe glossary calls this \"object name\", not \"ID\".\n\n> +the SHA-1 or SHA-256 (as appropriate) hash of the uncompressed data.\n\nWe should clarify \"data\" in uncompressed \"data\", as we earlier said\n\"prefix and data concatenated\", it can be misread as the payload\nalone. You have \"The entire contents\" that stands for \"prefix and\ndata concatenated\" above, so \"has of the entire contents\" may work.\n\nAlso \"uncompressed \" at the end of\nthis sentence should be followed by \"in hexadecimal\". The \"first\ntwo hex characters are used for fan-out\" etc., depends on that you\ndo not use binary result of the hash as the object name.\n\nThanks.\n\n"},{"id":"526840","messageId":"xmqqikheyqil.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-7-sandals@crustytoothpaste.net","subject":"Re: [PATCH 6/9] rev-parse: allow printing compatibility hash","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T23:24:50Z","receivedAt":"2025-09-19T23:24:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> Right now, we have a way to print the storage hash, the input hash, and\n> the output hash, but we lack a way to print the compatibility hash. Add\n> a new type to --show-object-format, compat, which prints this value.\n>\n> If no compatibility hash exists, simply print a newline. This is\n> important to allow users to use multiple options at once while still\n> getting unambiguous output.\n>\n> Signed-off-by: brian m. carlson \n> ---\n\nNice.\n\nAt first I somehow thought \n\n $ git rev-parse --show-object-format=compat HEAD\n\nin a SHA-1 primary repository would give the equivalent object name\nfor the commit at HEAD in SHA-256 world, but that is expecting too\nmuch out of a simple 50-line patch ;-).\n\n> \n> \t\t\t\tif (strcmp(val, \"storage\") &&\n> +\t\t\t\t strcmp(val, \"compat\") &&\n> \t\t\t\t strcmp(val, \"input\") &&\n> \t\t\t\t strcmp(val, \"output\"))\n> \t\t\t\t\tdie(_(\"unknown mode for --show-object-format: %s\"),\n> \t\t\t\t\t arg);\n> -\t\t\t\tputs(the_hash_algo->name);\n> +\n> +\t\t\t\tif (!strcmp(val, \"compat\")) {\n> +\t\t\t\t\tif (the_repository->compat_hash_algo)\n> +\t\t\t\t\t\tputs(the_repository->compat_hash_algo->name);\n> +\t\t\t\t\telse\n> +\t\t\t\t\t\tputchar('\\n');\n> +\t\t\t\t} else {\n> +\t\t\t\t\tputs(the_hash_algo->name);\n> +\t\t\t\t}\n> \t\t\t\tcontinue;\n> \t\t\t}\n> \t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\n\nPretty straight-forward.\n\nThanks.\n"},{"id":"526841","messageId":"xmqqecs2yq83.fsf@gitster.g","threadId":"64168","inReplyTo":"20250919010911.649831-8-sandals@crustytoothpaste.net","subject":"Re: [PATCH 7/9] fsck: consider gpgsig headers expected in tags","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-19T23:31:08Z","receivedAt":"2025-09-19T23:31:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> diff --git a/fsck.c b/fsck.c\n> index 171b424dd5..341e100d24 100644\n> --- a/fsck.c\n> +++ b/fsck.c\n> @@ -1067,6 +1067,24 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,\n> \telse\n> \t\tret = fsck_ident(&buffer, oid, OBJ_TAG, options);\n> \n> +\tif (buffer < buffer_end && (skip_prefix(buffer, \"gpgsig \", &buffer) || skip_prefix(buffer, \"gpgsig-sha256 \", &buffer))) {\n\nCould you wrap this overly long line?\n\n\tif (buffer < buffer_end && \n\t (skip_prefix(buffer, \"gpgsig \", &buffer) ||\n\t skip_prefix(buffer, \"gpgsig-sha256 \", &buffer))) {\n\n> +\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n> +\t\tif (!eol) {\n> +\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_GPGSIG, \"invalid format - unexpected end after 'gpgsig' or 'gpgsig-sha256' line\");\n> +\t\t\tgoto done;\n> +\t\t}\n> +\t\tbuffer = eol + 1;\n> +\n> +\t\twhile (buffer < buffer_end && starts_with(buffer, \" \")) {\n> +\t\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n> +\t\t\tif (!eol) {\n> +\t\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_HEADER_CONTINUATION, \"invalid format - unexpected end in 'gpgsig' or 'gpgsig-sha256' continuation line\");\n> +\t\t\t\tgoto done;\n> +\t\t\t}\n> +\t\t\tbuffer = eol + 1;\n> +\t\t}\n> +\t}\n> +\n\nDo we allow a tag object with both \"gpgsig\" and \"gpgsig-sha256\" or\ndetect as an error? I think the most natural way to extend this\nsystem in the future with a third hash function would be to still\nhave the primary hash in the payload and signatures created with\nother compatibility hash functions on the header, so if we were to\ndetect, the rule may be \"gpgsig* in the headers ought to be unique\nand should not include the primary hash algorithm\" plus \"if you have\ngpgsig* in the header, the body must also have inline signature, and\nif you don't, the body must not\", perhaps?\n"},{"id":"526852","messageId":"aM7G22LJ47nx8Wqk@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqq7bxu14fw.fsf@gitster.g","subject":"Re: [PATCH 1/9] docs: update pack index v3 format","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-20T15:23:07Z","receivedAt":"2025-09-20T15:23:10Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-19 at 22:08:03, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > Our current pack index v3 format uses 4-byte integers to find the\n> > trailer of the file. This effectively means that the file cannot be\n> > much larger than 2^32. While this might at first seem to be okay, we\n> > expect that each object will have at least 64 bytes worth of data, which\n> > means that no more than about 67 million objects can be stored.\n> >\n> > Again, this might seem fine, but unfortunately, we know of many users\n> > who attempt to create repos with extremely large numbers of commits to\n> > get a \"high score,\" and we've already seen repositories with at least 55\n> > million commits. In the interests of gracefully handling repositories\n> > even for these well-intentioned but ultimately misguided users, let's\n> > change these lengths to 8 bytes.\n> \n> Very sensible.\n> \n> I do also agree that 32-byte is the natural size for the trailing\n> hash, but I found that the two paragraphs below was far more than\n> necessary. As they argue, we use a truncated hash anywhere in our\n> file formats, so I would have understood if the explanation were\n> \n> \"20\" in \"A copy of the 20-byte SHA-256 checksum\" is an obvious\n> typo, as SHA-256 is longer than that. Fix it to \"32\".\n> \n> instead of these two paragraphs.\n> \n> Or did we mean to use a truncated hash back when this transition\n> design was proposed originally?\n\nI think we intended to use a 20-byte value originally because we felt we\ndidn't need the full 32 bytes for an index or pack checksum. However,\nas I mentioned, we use the 32-byte checksum for SHA-256 already, so all\nit does is add complexity to try to mandate a 20-byte value.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"526853","messageId":"aM7IQibwwKFJZcYE@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqq348i138l.fsf@gitster.g","subject":"Re: [PATCH 3/9] docs: reflect actual double signature for tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-20T15:29:06Z","receivedAt":"2025-09-20T15:29:08Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-19 at 22:34:02, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > Signed Tags\n> > ~~~~~~~~~~~\n> > +We add new fields \"gpgsig\" and \"gpgsig-sha256\" to the tag object format to\n> > +allow signing tags in both formats. The in-body signature is used for the\n> > +signature in the current hash algorithm and the header is used for the\n> > +signature in the other algorithm. Thus, a dual-signature tag will contain both\n> \n> Not suggesting a change in the text, but to make sure I am reading\n> the new text correctly. Does \"the other algorithm\" refer to the\n> compatibility hash algorithm specified by the compatObjectFormat\n> extension and the \"current\" algorithm refers to the objectFormat\n> extension?\n\nThe \"current algorithm\" is usually the main algorithm (that is, SHA-256\nwhere `extensions.objectformat` is `sha256`) and the \"other algorithm\"\nis the compatibility algorithm (SHA-1 in that case). However, when you\nconvert that object to SHA-1 to hash it in SHA-1, the \"current\nalgorithm\" becomes SHA-1 and the \"other algorithm\" is SHA-256.\n\nDoes that make sense?\n\n> > +an in-body signature and a gpgsig-sha256 header for the SHA-1 format of an\n> > +object or both an in-body signature and a gpgsig header for the SHA-256 format\n> > +of and object.\n> > \n> > -This means tags can be signed\n> > +The signed payload of the tag is the content of the tag in the current\n> > +algorithm with both its gpgsig and gpgsig-sha256 fields and\n> \n> My reading of the previous paragraph is that we cannot have gpgsig\n> and gpgsig-sha256 fields on a single object at the same time.\n\nCorrect, unless we come up with a third hash algorithm. Hopefully that\nis a long way away, and we are not considering that case here.\n\n> Should we say \"gpgsig or gpgsig-sha256\" (instead of \"and\"), to get\n> the resulting text parsable as:\n> \n> \tboth \n> \t\tits gpgsig or gpgsig-sha256 fields\n> \tand\n> \t\t\"-----BEGIN PGP SIGNATURE-----\" delimited in-body signature\n> \tremoved.\n> \n> instead?\n\nSure, I'll include that in a reroll.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"526855","messageId":"xmqqseghxdlm.fsf@gitster.g","threadId":"64168","inReplyTo":"aM7G22LJ47nx8Wqk@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/9] docs: update pack index v3 format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-20T17:01:25Z","receivedAt":"2025-09-20T17:01:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n>> I do also agree that 32-byte is the natural size for the trailing\n>> hash, but I found that the two paragraphs below was far more than\n>> necessary. As they argue, we use a truncated hash anywhere in our\n>> file formats, so I would have understood if the explanation were\n>> \n>> \"20\" in \"A copy of the 20-byte SHA-256 checksum\" is an obvious\n>> typo, as SHA-256 is longer than that. Fix it to \"32\".\n>> \n>> instead of these two paragraphs.\n>> \n>> Or did we mean to use a truncated hash back when this transition\n>> design was proposed originally?\n>\n> I think we intended to use a 20-byte value originally because we felt we\n> didn't need the full 32 bytes for an index or pack checksum. However,\n> as I mentioned, we use the 32-byte checksum for SHA-256 already, so all\n> it does is add complexity to try to mandate a 20-byte value.\n\nI think we are saying the same thing but from different sides of the\nsame mirror.\n\nSHA-256 packs and any csum-file based file would be using 32-byte\nchecksum because with CSUM_HASH_IN_STREAM, finalize_hashfile() does\nnot know any way to produce the trailing hash other than writing the\nfull hash value, and that would be 32 bytes for SHA-256. This was\nexactly where my \"20 certainly is a typo\" impression came from.\n\nBe it a typo or misdesign, picking 32 instead of 20 is a good thing\nto do now for a subsystem and fileformat that is not used anywhere\nin producation yet.\n\nThanks.\n"},{"id":"526856","messageId":"xmqqms6pxdg4.fsf@gitster.g","threadId":"64168","inReplyTo":"aM7IQibwwKFJZcYE@fruit.crustytoothpaste.net","subject":"Re: [PATCH 3/9] docs: reflect actual double signature for tags","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-20T17:04:43Z","receivedAt":"2025-09-20T17:04:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> On 2025-09-19 at 22:34:02, Junio C Hamano wrote:\n>> \"brian m. carlson\" writes:\n>> \n>> > Signed Tags\n>> > ~~~~~~~~~~~\n>> > +We add new fields \"gpgsig\" and \"gpgsig-sha256\" to the tag object format to\n>> > +allow signing tags in both formats. The in-body signature is used for the\n>> > +signature in the current hash algorithm and the header is used for the\n>> > +signature in the other algorithm. Thus, a dual-signature tag will contain both\n>> \n>> Not suggesting a change in the text, but to make sure I am reading\n>> the new text correctly. Does \"the other algorithm\" refer to the\n>> compatibility hash algorithm specified by the compatObjectFormat\n>> extension and the \"current\" algorithm refers to the objectFormat\n>> extension?\n>\n> The \"current algorithm\" is usually the main algorithm (that is, SHA-256\n> where `extensions.objectformat` is `sha256`) and the \"other algorithm\"\n> is the compatibility algorithm (SHA-1 in that case). However, when you\n> convert that object to SHA-1 to hash it in SHA-1, the \"current\n> algorithm\" becomes SHA-1 and the \"other algorithm\" is SHA-256.\n>\n> Does that make sense?\n\nLet me see if I got it right by trying to paraphrase the above.\n\nFor any object that is suitable to be stored in a repository with\nobjectFormat and compatObjectFormat set, \"current\" is the former,\nand \"the other\" is the latter.\n\nYour goal is not educating me, though. I wanted to make sure that\nthe text would be understood by the target audience of this document\nin a way you intended it to be.\n\nThanks.\n"},{"id":"526993","messageId":"aNHB4V7figj9lmfk@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqqecs2yq83.fsf@gitster.g","subject":"Re: [PATCH 7/9] fsck: consider gpgsig headers expected in tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-22T21:38:41Z","receivedAt":"2025-09-22T21:38:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-19 at 23:31:08, Junio C Hamano wrote:\n> Could you wrap this overly long line?\n> \n> \tif (buffer < buffer_end && \n> \t (skip_prefix(buffer, \"gpgsig \", &buffer) ||\n> \t skip_prefix(buffer, \"gpgsig-sha256 \", &buffer))) {\n\nWill fix in v2.\n\n> Do we allow a tag object with both \"gpgsig\" and \"gpgsig-sha256\" or\n> detect as an error? I think the most natural way to extend this\n> system in the future with a third hash function would be to still\n> have the primary hash in the payload and signatures created with\n> other compatibility hash functions on the header, so if we were to\n> detect, the rule may be \"gpgsig* in the headers ought to be unique\n> and should not include the primary hash algorithm\" plus \"if you have\n> gpgsig* in the header, the body must also have inline signature, and\n> if you don't, the body must not\", perhaps?\n\nIn v2, I'll make it such that `gpgsig` is allowed only when we're not\nusing SHA-1 and `gpgsig-sha256` is allowed only when we're not using\nSHA-256. It may be that we don't have a trailing signature, though,\nsince we might turn a SHA-1 tag (signed only with SHA-1) into a SHA-256\ntag (which would have only a `gpgsig` header and no trailing SHA-256\nsignature).\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527187","messageId":"aNOj8fFTvkQ6jsaT@pks.im","threadId":"64168","inReplyTo":"20250919010911.649831-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH 1/9] docs: update pack index v3 format","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:55:29Z","receivedAt":"2025-09-24T07:55:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 19, 2025 at 01:09:03AM +0000, brian m. carlson wrote:\n> Our current pack index v3 format uses 4-byte integers to find the\n> trailer of the file. This effectively means that the file cannot be\n> much larger than 2^32. While this might at first seem to be okay, we\n> expect that each object will have at least 64 bytes worth of data, which\n> means that no more than about 67 million objects can be stored.\n> \n> Again, this might seem fine, but unfortunately, we know of many users\n> who attempt to create repos with extremely large numbers of commits to\n> get a \"high score,\" and we've already seen repositories with at least 55\n> million commits. In the interests of gracefully handling repositories\n> even for these well-intentioned but ultimately misguided users, let's\n> change these lengths to 8 bytes.\n\nYeah, this makes sense. We can only assume that repositories will\ncontinue to grow, so it makes sense to future proof.\n\nWe also have the 4-byte number of objects contained in the pack. But as\nyou explain, it's nothing we should need to worry about given that this\nis a mere counter, and not an offset into the file. I doubt that there's\nrepositories out there that'll have more than 4 billion objects anytime\nsoon.\n\n> For the checksums at the end of the file, we're producing 32-byte\n> SHA-256 checksums because that's what we already do with pack index v2\n> and SHA-256. Truncating SHA-256 doesn't pose any actual security\n> problems other than those related to the reduced size, but our pack\n> checksum must already be 32 bytes (since SHA-256 packs have 32-byte\n> checksums) and it simplifies the code to use the existing hashfile logic\n> for these cases for the index checksum as well.\n> \n> In addition, even though we may not need cryptographic security for the\n> index checksum, we'd like to avoid arguments from auditors and such for\n> organizations that may have compliance or security requirements. Using\n> the simple, boring choice of the full SHA-256 hash avoids all possible\n> discussion related to hash truncation and removes impediments for these\n> organizations.\n\nFor now we only have SHA256 and SHA1. But thinking about the future,\nthere will be a time when SHA256 will be considered broken. I wonder\nwhether we should safeguard against that and also specify the trailer\nhash to be agile? That is, instead of hardcoding the hash function, we\nadd something like a \"primary\" hash to the packfile and then use the\nfull output of that hash as checksum.\n\nIn any case, please feel free to say \"no\" to the above thought. It's\njust something that popped into my mind upon reading this.\n\nI guess one thing that should be explicitly pointed out in the commit\nmessage is that there are no implementations of the v3 format yet, so\nthis is basically updating our envisioned design, only. Otherwise one\nmight wonder why we can update the spec just so.\n\nPatrick\n"},{"id":"527188","messageId":"aNOj-1xy-UTP_3E1@pks.im","threadId":"64168","inReplyTo":"aM7IQibwwKFJZcYE@fruit.crustytoothpaste.net","subject":"Re: [PATCH 3/9] docs: reflect actual double signature for tags","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:55:39Z","receivedAt":"2025-09-24T07:55:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sat, Sep 20, 2025 at 03:29:06PM +0000, brian m. carlson wrote:\n> On 2025-09-19 at 22:34:02, Junio C Hamano wrote:\n> > \"brian m. carlson\" writes:\n> > > +an in-body signature and a gpgsig-sha256 header for the SHA-1 format of an\n> > > +object or both an in-body signature and a gpgsig header for the SHA-256 format\n> > > +of and object.\n> > > \n> > > -This means tags can be signed\n> > > +The signed payload of the tag is the content of the tag in the current\n> > > +algorithm with both its gpgsig and gpgsig-sha256 fields and\n> > \n> > My reading of the previous paragraph is that we cannot have gpgsig\n> > and gpgsig-sha256 fields on a single object at the same time.\n> \n> Correct, unless we come up with a third hash algorithm. Hopefully that\n> is a long way away, and we are not considering that case here.\n\nYou mentioned a \"missed patch\" in the commit message. So is this design\nhere intentional or merely an oversight?\n\nI'm mostly asking because it feels weird to me that an object shouldn't\nhave both fields. I would assume that it's easier to implement and\nreason about if this signature always was a header, or multiple that is.\nBut I'm not familiar enough with the logic here to really judge, so I\nassume that there are good reasons that I miss.\n\nPatrick\n"},{"id":"527189","messageId":"aNOkAYsqavZjnYSO@pks.im","threadId":"64168","inReplyTo":"xmqqms6q45nl.fsf@gitster.g","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:55:45Z","receivedAt":"2025-09-24T07:55:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 19, 2025 at 12:13:34PM -0700, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> > \"brian m. carlson\" writes:\n> >\n> >> We currently have no documentation for how loose objects are stored.\n> >> Let's add some here so its easy for people to understand how they\n> >> work.\n> >>\n> >> Signed-off-by: brian m. carlson \n> >> ---\n> >> Documentation/gitformat-loose.adoc | 49 ++++++++++++++++++++++++++++++\n> >> 1 file changed, 49 insertions(+)\n> >> create mode 100644 Documentation/gitformat-loose.adoc\n> >\n> > Fails a build, unfortunately.\n> >\n> > ...\n> > LINT DOCSTYLE includes/cmd-config-section-rest.adoc\n> > GEN lint-docs-manpages\n> > LINT DOCSTYLE includes/cmd-config-section-all.adoc\n> > tmp-meson-diff/meson.adoc tmp-meson-diff/actual.adoc differ: char 3297, line 176\n> > Meson man pages differ from actual man pages:\n> > --- tmp-meson-diff/meson.adoc 2025-09-19 12:04:55.145229743 -0700\n> > +++ tmp-meson-diff/actual.adoc 2025-09-19 12:04:55.149229734 -0700\n> > @@ -173,6 +173,7 @@\n> > gitformat-chunk.adoc\n> > gitformat-commit-graph.adoc\n> > gitformat-index.adoc\n> > +gitformat-loose.adoc\n> > gitformat-pack.adoc\n> > gitformat-signature.adoc\n> > gitglossary.adoc\n> >\n> > Thanks.\n> \n> Probably this should be sufficient? Not tested (yet).\n> \n> \n> \n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 4404c623f0..93fa3dee8b 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -171,6 +171,7 @@ manpages = {\n> 'gitformat-chunk.adoc' : 5,\n> 'gitformat-commit-graph.adoc' : 5,\n> 'gitformat-index.adoc' : 5,\n> + 'gitformat-loose.adoc' : 5,\n> 'gitformat-pack.adoc' : 5,\n> 'gitformat-signature.adoc' : 5,\n> 'githooks.adoc' : 5,\n\nYup, this one looks correct. But in fact, we also need a similar change\nto our Makefile.\n\nPatrick\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 6fb83d0c6e..e1d38fbfe6 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -34,6 +34,7 @@ MAN5_TXT += gitformat-bundle.adoc\n MAN5_TXT += gitformat-chunk.adoc\n MAN5_TXT += gitformat-commit-graph.adoc\n MAN5_TXT += gitformat-index.adoc\n+MAN5_TXT += gitformat-loose.adoc\n MAN5_TXT += gitformat-pack.adoc\n MAN5_TXT += gitformat-signature.adoc\n MAN5_TXT += githooks.adoc\ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex 41f43e0336..64f70ac724 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -172,6 +172,7 @@ manpages = {\n 'gitformat-chunk.adoc' : 5,\n 'gitformat-commit-graph.adoc' : 5,\n 'gitformat-index.adoc' : 5,\n+ 'gitformat-loose.adoc' : 5,\n 'gitformat-pack.adoc' : 5,\n 'gitformat-signature.adoc' : 5,\n 'githooks.adoc' : 5,\n"},{"id":"527190","messageId":"aNOkBuigxCODIcVN@pks.im","threadId":"64168","inReplyTo":"20250919010911.649831-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:55:50Z","receivedAt":"2025-09-24T07:55:56Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 19, 2025 at 01:09:07AM +0000, brian m. carlson wrote:\n> We currently have no documentation for how loose objects are stored.\n> Let's add some here so its easy for people to understand how they\n\nNit: s/its/it is/\n\n> diff --git a/Documentation/gitformat-loose.adoc b/Documentation/gitformat-loose.adoc\n> new file mode 100644\n> index 0000000000..c8bef606fb\n> --- /dev/null\n> +++ b/Documentation/gitformat-loose.adoc\n\nDo we maybe want to call this \"gitformat-loose-objects(5)\"? \"loose\"\nfeels rather generic.\n\n> @@ -0,0 +1,49 @@\n> +gitformat-loose(5)\n> +==================\n\nMakes me wonder whether we should also have gitformat-reffiles(5) and\ngitformat-reftables(5). Obviously nothing you have to do, but rather an\naction item for myself or others interested in the ref backends.\n\n> +NAME\n> +----\n> +gitformat-loose - Git loose object format\n> +\n> +\n> +SYNOPSIS\n> +--------\n> +[verse]\n> +$GIT_DIR/objects/[0-9a-f][0-9a-f]/*\n> +$GIT_DIR/objects/loose-object-idx\n> +$GIT_DIR/objects/loose-map/map-*.map\n\nIt's a bit weird to list the mapping files here without explaining them.\nShould we maybe drop them for now and only add them once we also add a\nsection explaining their format?\n\nOn the other hand, maybe it's better to list those files and not explain\nthem compared to not mentioning them at all. Not quite sure.\n\n> +DESCRIPTION\n> +-----------\n> +\n> +Loose objects are how Git initially stores most of its primary repository data.\n> +Over the lifetime of a repository, objects are usually written as loose objects\n> +initially and then converted into packs.\n\nI feel that \"most of its primary repository data\" is a bit misleading,\nas one can expect that most of the data should be in packfiles instead.\nHow about the following instead:\n\n Loose objects are how Git stores individual objects, where every\n object is written as a separate file.\n\n Over the lifetime of a repository, new objects are typically written\n as loose objects initially. Eventually, these loose objects will be\n compacted into packfiles via repository maintenance to improve disk\n space usage and speed up the lookup of those objects.\n\n> +== Loose objects\n> +\n> +Each loose object contains a prefix, followed immediately by the data of the\n> +object. The prefix contains ` \\0`. `` is one of `blob`,\n> +`tree`, `commit`, or `tag` and `size` is the size of the data (without the\n> +prefix) as a decimal integer expressed in ASCII.\n> +\n> +The entire contents, prefix and data concatenated, is then compressed with zlib\n> +and the compressed data is stored in the file. The object ID of the object is\n> +the SHA-1 or SHA-256 (as appropriate) hash of the uncompressed data.\n> +\n> +The file for the loose object is stored under the `objects` directory, with the\n> +first two hex characters of the object ID being the directory and the remaining\n> +characters being the file name.\n\nShould we maybe give a hint why we have these sharding directories?\n\nPatrick\n"},{"id":"527191","messageId":"aNOkDEsJIfq0Ufgg@pks.im","threadId":"64168","inReplyTo":"20250919010911.649831-7-sandals@crustytoothpaste.net","subject":"Re: [PATCH 6/9] rev-parse: allow printing compatibility hash","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:55:56Z","receivedAt":"2025-09-24T07:56:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 19, 2025 at 01:09:08AM +0000, brian m. carlson wrote:\n> diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\n> index 58a4583088..98c5a772bd 100755\n> --- a/t/t1500-rev-parse.sh\n> +++ b/t/t1500-rev-parse.sh\n> @@ -207,6 +207,40 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n> \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n> '\n> \n> +\n> +test_expect_success RUST 'rev-parse --show-object-format in repo with compat mode' '\n\nDoes this test really depend on the RUST prereq? I cannot see anything\nhere that would require it.\n\nPatrick\n"},{"id":"527192","messageId":"aNOkEVUag91bv2RP@pks.im","threadId":"64168","inReplyTo":"20250919010911.649831-9-sandals@crustytoothpaste.net","subject":"Re: [PATCH 8/9] Allow specifying compatibility hash","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:56:01Z","receivedAt":"2025-09-24T07:56:07Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 19, 2025 at 01:09:10AM +0000, brian m. carlson wrote:\n\nTiny nit: it would help if the patch subject was prefixed with \"t:\".\n\n> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\n> index a28de7b19b..52d7759bf5 100644\n> --- a/t/test-lib-functions.sh\n> +++ b/t/test-lib-functions.sh\n> @@ -1708,11 +1708,16 @@ test_set_hash () {\n> # Detect the hash algorithm in use.\n> test_detect_hash () {\n> \tcase \"${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}\" in\n> -\t\"sha256\")\n> +\t*:*)\n> +\t test_hash_algo=\"${GIT_TEST_DEFAULT_HASH%%:*}\"\n> +\t test_compat_hash_algo=\"${GIT_TEST_DEFAULT_HASH##*:}\"\n> +\t test_repo_compat_hash_algo=\"$test_compat_hash_algo\"\n> +\t ;;\n> +\tsha256)\n> \t test_hash_algo=sha256\n> \t test_compat_hash_algo=sha1\n> \t ;;\n> -\t*)\n> +\tsha1)\n> \t test_hash_algo=sha1\n> \t test_compat_hash_algo=sha256\n> \t ;;\n\nMakes sense, I guess. It's a bit hard to judge without seeing any actual\ntests, but I don't think that should hold off adding the infrastructure.\nWorst case we can still adjust it at later point in time.\n\n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index 621cd31ae1..14c777e4e2 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -1917,6 +1917,13 @@ test_lazy_prereq DEFAULT_HASH_ALGORITHM '\n> test_lazy_prereq DEFAULT_REPO_FORMAT '\n> \ttest_have_prereq SHA1,REFFILES\n> '\n> +# BROKEN_OBJECTS is a test if we can write deliberately broken objects and\n\ns/if/whether/\n\n> +# expect them to work. When running using SHA-256 mode with SHA-1\n> +# compatibility, we cannot write such objects because there's no SHA-1\n> +# compatibility value for a nonexistent object.\n> +test_lazy_prereq BROKEN_OBJECTS '\n> +\ttest -z \"$test_repo_compat_hash_algo\"\n> +'\n> \n> # Ensure that no test accidentally triggers a Git command\n> # that runs the actual maintenance scheduler, affecting a user's\n\nPatrick\n"},{"id":"527193","messageId":"aNOkF7Isc7OihClu@pks.im","threadId":"64168","inReplyTo":"20250919010911.649831-10-sandals@crustytoothpaste.net","subject":"Re: [PATCH 9/9] t: add a prerequisite for a compatibility hash","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-24T07:56:07Z","receivedAt":"2025-09-24T07:56:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 19, 2025 at 01:09:11AM +0000, brian m. carlson wrote:\n> In some cases, we'll need to adjust our test suite to work in a proper\n> way with a compatibility hash. For example, in such a case, we'll only\n> use pack index v3, since v1 and v2 lack support for multiple algorithms.\n> Since we won't want to write those older formats, we'll need to skip\n> tests that do so.\n> \n> Let's add a COMPAT_HASH prerequisite and define the BROKEN_OBJECTS\n> prerequisite in terms of it.\n\nNit: I think this commit could easily be squashed into the preceding\ncommit.\n\nPatrick\n"},{"id":"527359","messageId":"aNW2riVWtLQbacSR@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"aNOj8fFTvkQ6jsaT@pks.im","subject":"Re: [PATCH 1/9] docs: update pack index v3 format","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-25T21:39:58Z","receivedAt":"2025-09-25T21:40:06Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-24 at 07:55:29, Patrick Steinhardt wrote:\n> On Fri, Sep 19, 2025 at 01:09:03AM +0000, brian m. carlson wrote:\n> > Our current pack index v3 format uses 4-byte integers to find the\n> > trailer of the file. This effectively means that the file cannot be\n> > much larger than 2^32. While this might at first seem to be okay, we\n> > expect that each object will have at least 64 bytes worth of data, which\n> > means that no more than about 67 million objects can be stored.\n> > \n> > Again, this might seem fine, but unfortunately, we know of many users\n> > who attempt to create repos with extremely large numbers of commits to\n> > get a \"high score,\" and we've already seen repositories with at least 55\n> > million commits. In the interests of gracefully handling repositories\n> > even for these well-intentioned but ultimately misguided users, let's\n> > change these lengths to 8 bytes.\n> \n> Yeah, this makes sense. We can only assume that repositories will\n> continue to grow, so it makes sense to future proof.\n> \n> We also have the 4-byte number of objects contained in the pack. But as\n> you explain, it's nothing we should need to worry about given that this\n> is a mere counter, and not an offset into the file. I doubt that there's\n> repositories out there that'll have more than 4 billion objects anytime\n> soon.\n\nThere are certainly some users who try to do that at $DAYJOB, but they\ncome to our attention (because our maintenance job fails due to taking\ntoo long) before they get there. I am not, however, aware of any\nactually legitimate and productive uses of repositories that threaten to\nbreak that limit, which is what I think what we should really care\nabout.\n\nIn the event we start seeing those kinds of problems, it should be easy\nto implement pack v5 with a corresponding index, just with a larger\nnumber of objects.\n\n> For now we only have SHA256 and SHA1. But thinking about the future,\n> there will be a time when SHA256 will be considered broken. I wonder\n> whether we should safeguard against that and also specify the trailer\n> hash to be agile? That is, instead of hardcoding the hash function, we\n> add something like a \"primary\" hash to the packfile and then use the\n> full output of that hash as checksum.\n> \n> In any case, please feel free to say \"no\" to the above thought. It's\n> just something that popped into my mind upon reading this.\n\nIt is actually that it's the main hash algorithm in use. So if we add a\nthird algorithm which is SHA-3-512, then the trailer checksum will be\nSHA-3-512 when that's the main algorithm.\n\nTechnically, it's also SHA-1 if we're in a SHA-1 repository with SHA-256\ncompatibility. That's not a use case I really encourage, but it is a\nuse case I'm testing because it exposes bugs in our codebase and I\nexpect people will want to do in-place conversion from SHA-1 only to\nSHA-1 with SHA-256 at some point.\n\nI'll fix that for v2.\n\n> I guess one thing that should be explicitly pointed out in the commit\n> message is that there are no implementations of the v3 format yet, so\n> this is basically updating our envisioned design, only. Otherwise one\n> might wonder why we can update the spec just so.\n\nThat isn't completely true. There is an implementation, but it is not\nyet on the list, and it follows the spec written here. I will provide\ndocumentation with the rest of the pack index code when index v3 comes\nin, but I wanted to update this in case people are trying to add it in\nother implementations as well.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527360","messageId":"aNW2yds9UFFpntQr@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"aNOkAYsqavZjnYSO@pks.im","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-25T21:40:25Z","receivedAt":"2025-09-25T21:40:27Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-24 at 07:55:45, Patrick Steinhardt wrote:\n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index 6fb83d0c6e..e1d38fbfe6 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -34,6 +34,7 @@ MAN5_TXT += gitformat-bundle.adoc\n> MAN5_TXT += gitformat-chunk.adoc\n> MAN5_TXT += gitformat-commit-graph.adoc\n> MAN5_TXT += gitformat-index.adoc\n> +MAN5_TXT += gitformat-loose.adoc\n> MAN5_TXT += gitformat-pack.adoc\n> MAN5_TXT += gitformat-signature.adoc\n> MAN5_TXT += githooks.adoc\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 41f43e0336..64f70ac724 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -172,6 +172,7 @@ manpages = {\n> 'gitformat-chunk.adoc' : 5,\n> 'gitformat-commit-graph.adoc' : 5,\n> 'gitformat-index.adoc' : 5,\n> + 'gitformat-loose.adoc' : 5,\n> 'gitformat-pack.adoc' : 5,\n> 'gitformat-signature.adoc' : 5,\n> 'githooks.adoc' : 5,\n\nI've got these already fixed up for v2.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527362","messageId":"aNW4UgPE_sitLgGU@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"aNOj-1xy-UTP_3E1@pks.im","subject":"Re: [PATCH 3/9] docs: reflect actual double signature for tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-25T21:46:58Z","receivedAt":"2025-09-25T21:47:01Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-24 at 07:55:39, Patrick Steinhardt wrote:\n> You mentioned a \"missed patch\" in the commit message. So is this design\n> here intentional or merely an oversight?\n\nThe original design was to implement all SHA-256 signatures in the\n`gpgsig-sha256` header, but the patch to do that got dropped\naccidentally for 2.29, so we shipped without it. I decided to fix it in\na compatible way for 2.30 using the design here so that users who had\ncreated SHA-256 tags with 2.29 would not have them be mistaken for\nsignatures over the SHA-1 values of the tag by Git 2.30.\n\nI knew that people would try things out nearly immediately and that some\npeople would use very old versions of Git from their LTS distro and did\nnot want to risk making an incompatible change that would break the\nobject format, even while things were marked experimental.\n\n> I'm mostly asking because it feels weird to me that an object shouldn't\n> have both fields. I would assume that it's easier to implement and\n> reason about if this signature always was a header, or multiple that is.\n> But I'm not familiar enough with the logic here to really judge, so I\n> assume that there are good reasons that I miss.\n\nWe should not have both fields. In the SHA-256 version of the tag, the\nin-body signature is SHA-256 and there is optionally a `gpgsig` header\nfor the SHA-1 version of the tag. When that tag is converted into SHA-1\nformat, the in-body signature moves to the `gpgsig-sha256` header and\nthe one that was formerly in the `gpgsig` header is placed in body and\nthat header is removed.\n\nSo we will never have both unless we have an additional hash algorithm,\nsay, SHA-3-512, where, when in SHA-3-512 format, the in-body signature\nis over SHA-3-512 and there may be both `gpgsig` and `gpgsig-sha256`\nheaders.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527363","messageId":"aNW4x6Tb1g-YKqzw@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"aNOkDEsJIfq0Ufgg@pks.im","subject":"Re: [PATCH 6/9] rev-parse: allow printing compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-25T21:48:55Z","receivedAt":"2025-09-25T21:48:57Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-24 at 07:55:56, Patrick Steinhardt wrote:\n> On Fri, Sep 19, 2025 at 01:09:08AM +0000, brian m. carlson wrote:\n> > diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\n> > index 58a4583088..98c5a772bd 100755\n> > --- a/t/t1500-rev-parse.sh\n> > +++ b/t/t1500-rev-parse.sh\n> > @@ -207,6 +207,40 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n> > \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n> > '\n> > \n> > +\n> > +test_expect_success RUST 'rev-parse --show-object-format in repo with compat mode' '\n> \n> Does this test really depend on the RUST prereq? I cannot see anything\n> here that would require it.\n\nI think that should move up into part 2. Will fix for v2.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527644","messageId":"aNwH3wqMzbNMpnEB@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"aNOkBuigxCODIcVN@pks.im","subject":"Re: [PATCH 5/9] docs: add documentation for loose objects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-30T16:39:59Z","receivedAt":"2025-09-30T16:40:02Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-24 at 07:55:50, Patrick Steinhardt wrote:\n> On Fri, Sep 19, 2025 at 01:09:07AM +0000, brian m. carlson wrote:\n> > We currently have no documentation for how loose objects are stored.\n> > Let's add some here so its easy for people to understand how they\n> \n> Nit: s/its/it is/\n\nWill be fixed in v2.\n\n> Do we maybe want to call this \"gitformat-loose-objects(5)\"? \"loose\"\n> feels rather generic.\n\nWe have \"index\" and \"pack\", so I'd rather keep it short.\n\n> > @@ -0,0 +1,49 @@\n> > +gitformat-loose(5)\n> > +==================\n> \n> Makes me wonder whether we should also have gitformat-reffiles(5) and\n> gitformat-reftables(5). Obviously nothing you have to do, but rather an\n> action item for myself or others interested in the ref backends.\n\nYeah, I definitely think that would be useful.\n\n> > +SYNOPSIS\n> > +--------\n> > +[verse]\n> > +$GIT_DIR/objects/[0-9a-f][0-9a-f]/*\n> > +$GIT_DIR/objects/loose-object-idx\n> > +$GIT_DIR/objects/loose-map/map-*.map\n> \n> It's a bit weird to list the mapping files here without explaining them.\n> Should we maybe drop them for now and only add them once we also add a\n> section explaining their format?\n\nI've bumped those two lines to a future commit (probably part 2).\n\n> I feel that \"most of its primary repository data\" is a bit misleading,\n> as one can expect that most of the data should be in packfiles instead.\n> How about the following instead:\n> \n> Loose objects are how Git stores individual objects, where every\n> object is written as a separate file.\n> \n> Over the lifetime of a repository, new objects are typically written\n> as loose objects initially. Eventually, these loose objects will be\n> compacted into packfiles via repository maintenance to improve disk\n> space usage and speed up the lookup of those objects.\n\nI adopted most of this language.\n\n> Should we maybe give a hint why we have these sharding directories?\n\nDone in v2.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527645","messageId":"aNwI9gLAGp7QvpI4@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"aNOkEVUag91bv2RP@pks.im","subject":"Re: [PATCH 8/9] Allow specifying compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-30T16:44:38Z","receivedAt":"2025-09-30T16:44:39Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-24 at 07:56:01, Patrick Steinhardt wrote:\n> Makes sense, I guess. It's a bit hard to judge without seeing any actual\n> tests, but I don't think that should hold off adding the infrastructure.\n> Worst case we can still adjust it at later point in time.\n\nI'll send in an example test as part of v2.\n\n> > '\n> > +# BROKEN_OBJECTS is a test if we can write deliberately broken objects and\n> \n> s/if/whether/\n\nFixed in v2.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527846","messageId":"20251002223855.1022847-2-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 1/9] docs: update pack index v3 format","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:47Z","receivedAt":"2025-10-02T22:39:06Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our current pack index v3 format uses 4-byte integers to find the\ntrailer of the file. This effectively means that the file cannot be\nmuch larger than 2^32. While this might at first seem to be okay, we\nexpect that each object will have at least 64 bytes worth of data, which\nmeans that no more than about 67 million objects can be stored.\n\nAgain, this might seem fine, but unfortunately, we know of many users\nwho attempt to create repos with extremely large numbers of commits to\nget a \"high score,\" and we've already seen repositories with at least 55\nmillion commits. In the interests of gracefully handling repositories\neven for these well-intentioned but ultimately misguided users, let's\nchange these lengths to 8 bytes.\n\nFor the checksums at the end of the file, we're producing 32-byte\nSHA-256 checksums because that's what we already do with pack index v2\nand SHA-256. Truncating SHA-256 doesn't pose any actual security\nproblems other than those related to the reduced size, but our pack\nchecksum must already be 32 bytes (since SHA-256 packs have 32-byte\nchecksums) and it simplifies the code to use the existing hashfile logic\nfor these cases for the index checksum as well.\n\nIn addition, even though we may not need cryptographic security for the\nindex checksum, we'd like to avoid arguments from auditors and such for\norganizations that may have compliance or security requirements. Using\nthe simple, boring choice of the full SHA-256 hash avoids all possible\ndiscussion related to hash truncation and removes impediments for these\norganizations.\n\nNote that we do not yet have a pack index v3 implementation in Git, so\nit should be fine to change this format. However, such an\nimplementation has been written for future inclusion following this\nformat.\n\nSigned-off-by: brian m. carlson \n---\n .../technical/hash-function-transition.adoc | 12 ++++++++----\n 1 file changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex f047fd80ca..274dc993d4 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -227,9 +227,9 @@ network byte order):\n ** 4-byte length in bytes of shortened object names. This is the\n shortest possible length needed to make names in the shortened\n object name table unambiguous.\n- ** 4-byte integer, recording where tables relating to this format\n+ ** 8-byte integer, recording where tables relating to this format\n are stored in this index file, as an offset from the beginning.\n- * 4-byte offset to the trailer from the beginning of this file.\n+ * 8-byte offset to the trailer from the beginning of this file.\n * Zero or more additional key/value pairs (4-byte key, 4-byte\n value). Only one key is supported: 'PSRC'. See the \"Loose objects\n and unreachable objects\" section for supported values and how this\n@@ -276,10 +276,14 @@ network byte order):\n up to and not including the table of CRC32 values.\n - Zero or more NUL bytes.\n - The trailer consists of the following:\n- * A copy of the 20-byte SHA-256 checksum at the end of the\n+ * A copy of the full main hash checksum at the end of the\n corresponding packfile.\n \n- * 20-byte SHA-256 checksum of all of the above.\n+ * Full main hash checksum of all of the above.\n+\n+The \"full main hash\" is a full-length hash of the main (not compatibility)\n+algorithm in the repository. Thus, if the main algorithm is SHA-256, this is\n+a 32-byte SHA-256 hash and for SHA-1, it's a 20-byte SHA-1 hash.\n \n Loose object index\n ~~~~~~~~~~~~~~~~~~\n"},{"id":"527843","messageId":"20251002223855.1022847-4-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 3/9] docs: reflect actual double signature for tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:49Z","receivedAt":"2025-10-02T22:39:07Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The documentation for the hash function transition reflects the original\ndesign where the SHA-256 signature would always be placed in a header.\nHowever, due to a missed patch in Git 2.29, we shipped SHA-256 support\nsuch that the signature for the current algorithm is always an in-body\nsignature and the opposite algorithm is always in a header. Since the\ndocumentation is inaccurate, update it to reflect the correct\ninformation.\n\nSigned-off-by: brian m. carlson \n---\n .../technical/hash-function-transition.adoc | 20 ++++++++++---------\n 1 file changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex adb0c61e53..2359d7d106 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -429,17 +429,19 @@ ordinary unsigned commit.\n \n Signed Tags\n ~~~~~~~~~~~\n-We add a new field \"gpgsig-sha256\" to the tag object format to allow\n-signing tags without relying on SHA-1. Its signed payload is the\n-SHA-256 content of the tag with its gpgsig-sha256 field and \"-----BEGIN PGP\n-SIGNATURE-----\" delimited in-body signature removed.\n+We add new fields \"gpgsig\" and \"gpgsig-sha256\" to the tag object format to\n+allow signing tags in both formats. The in-body signature is used for the\n+signature in the current hash algorithm and the header is used for the\n+signature in the other algorithm. Thus, a dual-signature tag will contain both\n+an in-body signature and a gpgsig-sha256 header for the SHA-1 format of an\n+object or both an in-body signature and a gpgsig header for the SHA-256 format\n+of and object.\n \n-This means tags can be signed\n+The signed payload of the tag is the content of the tag in the current\n+algorithm with both its gpgsig and gpgsig-sha256 fields and\n+\"-----BEGIN PGP SIGNATURE-----\" delimited in-body signature removed.\n \n-1. using SHA-1 only, as in existing signed tag objects\n-2. using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body\n- signature.\n-3. using only SHA-256, by only using the gpgsig-sha256 field.\n+This means tags can be signed using one or both algorithms.\n \n Mergetag embedding\n ~~~~~~~~~~~~~~~~~~\n"},{"id":"527844","messageId":"20251002223855.1022847-3-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 2/9] docs: update offset order for pack index v3","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:48Z","receivedAt":"2025-10-02T22:39:07Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The current design of pack index v3 has items in two different orders:\nsorted shortened object ID order and pack order. The shortened object\nIDs and the pack index offset values are in the former order and\neverything else is in the latter.\n\nThis, however, poses some problems. We have many parts of the packfile\ncode that expect to find out data about an object knowing only its index\nin pack order. With the current design, to find the pack offset after\nhaving looked up the index in pack order, we must then look up the full\nobject ID and use that to look up the shortened object ID to find the\npack offset, which is inconvenient, inefficient, and leads to poor cache\nusage.\n\nInstead, let's change the offset values to be looked up by pack order.\nThis works better because once we know the pack order offset, we can\nfind the full object name and its location in the pack with a simple\nindex into their respective tables. This makes many operations much\nmore efficient, especially with the functions we already have, and it\navoids the need for the revindex with pack index v3.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/technical/hash-function-transition.adoc | 10 ++++------\n 1 file changed, 4 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex 274dc993d4..adb0c61e53 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -260,12 +260,10 @@ network byte order):\n compressed data to be copied directly from pack to pack during\n repacking without undetected data corruption.\n \n- * A table of 4-byte offset values. For an object in the table of\n- sorted shortened object names, the value at the corresponding\n- index in this table indicates where that object can be found in\n- the pack file. These are usually 31-bit pack file offsets, but\n- large offsets are encoded as an index into the next table with the\n- most significant bit set.\n+ * A table of 4-byte offset values. The index of this table in pack order\n+ indicates where that object can be found in the pack file. These are\n+ usually 31-bit pack file offsets, but large offsets are encoded as\n+ an index into the next table with the most significant bit set.\n \n * A table of 8-byte offset entries (empty for pack files less than\n 2 GiB). Pack files are organized with heavily used objects toward\n"},{"id":"527845","messageId":"20251002223855.1022847-1-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 0/9] SHA-1/SHA-256 interoperability, part 1","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:46Z","receivedAt":"2025-10-02T22:39:07Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This is the first of several series for SHA-1 and SHA-256\ninteroperability, all of which will hopefully land in Git 3.0.\n\nThe first set of commits here is documentation updates for documentation\nwhich was incorrect, missing, or simply wrong. I have spent more time\nthan I'd like in the pack code and felt our documentation there could be\nmore helpful. I also am correcting some things about the\ninteroperability formats that I've found are not correct or efficient in\nterms of implementation and thus I will be implementing differently.\n\nThe loose object documentation will be updated with the loose object\nmapping in a future commit, but I felt I should send a basic loose\nobject document first, so here it is.\n\nThe remaining commits are for expected gpgsig headers in tags, which\ncauses some tests which use strict fsck to fail, as well as for\nprerequisites for compatibility hashes in the testsuite. Actually using\nthis configuration is not possible since the tests are still very broken\nusing it, but declaring these prerequisites allows me and others to send\nin patches that use them and thus make our testsuite more resilient.\n\nFor example, in interoperability mode we cannot write objects that\nare not valid since we cannot convert them into the other hash\nalgorithm. Thus, when we're testing in a mode that has a compatibility\nalgorithm, we skip these tests.\n\nThe goal is to run the tests in a full compatibility mode where\neverything is dual-hash as well as introduce some specific tests for\ninteroperability that run in all configurations of the tests.\n\nChanges from v1:\n* Squash the two test changes into one commit.\n* Include a new commit showing the use of the BROKEN_OBJECTS prereq.\n* Mention using main algorithm hash in pack index v3.\n* Hopefully clarify signed tags.\n* Improve text for pack format documentation.\n* Wire up build of loose object documentation.\n* Remove loose object map documentation.\n* Rephrase text about loose objects.\n* Remove needless RUST prerequisite.\n* Wrap overly long line.\n* Reject invalid signature algorithms in tag headers.\n* Fix if/whether problem in test comment.\n\nbrian m. carlson (9):\n docs: update pack index v3 format\n docs: update offset order for pack index v3\n docs: reflect actual double signature for tags\n docs: improve ambiguous areas of pack format documentation\n docs: add documentation for loose objects\n rev-parse: allow printing compatibility hash\n fsck: consider gpgsig headers expected in tags\n t: allow specifying compatibility hash\n t1010: use BROKEN_OBJECTS prerequisite\n\n Documentation/Makefile | 1 +\n Documentation/fsck-msgids.adoc | 6 +++\n Documentation/git-rev-parse.adoc | 11 ++--\n Documentation/gitformat-loose.adoc | 53 ++++++++++++++++++\n Documentation/gitformat-pack.adoc | 18 +++++++\n Documentation/meson.build | 1 +\n .../technical/hash-function-transition.adoc | 42 ++++++++-------\n builtin/rev-parse.c | 11 +++-\n fsck.c | 18 +++++++\n fsck.h | 2 +\n t/t1010-mktree.sh | 13 +++--\n t/t1450-fsck.sh | 54 +++++++++++++++++++\n t/t1500-rev-parse.sh | 34 ++++++++++++\n t/test-lib-functions.sh | 9 +++-\n t/test-lib.sh | 13 +++++\n 15 files changed, 254 insertions(+), 32 deletions(-)\n create mode 100644 Documentation/gitformat-loose.adoc\n\n"},{"id":"527847","messageId":"20251002223855.1022847-5-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 4/9] docs: improve ambiguous areas of pack format documentation","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:50Z","receivedAt":"2025-10-02T22:39:07Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"It is fair to say that our pack and indexing code is quite complex.\nContributors who wish to work on this code or implementors of other\nimplementations would benefit from clear, unambiguous documentation\nabout how our data formats are structured and encoded and what data is\nused in the computation of certain values. Unfortunately, some of this\ndata is missing, which leads to confusion and frustration.\n\nLet's document some of this data to help clarify things. Specify over\nwhat data CRC32 values are computed and also note which CRC32 algorithm\nis used, since Wikipedia mentions at least four 32-bit CRC algorithms\nand notes that it's possible to use different bit orderings.\n\nIn addition, note how we encode objects in the pack. One might be led\nto believe that packed objects are always stored with the \"\n\\0\" prefix of loose objects, but that is not the case, although\nfor obvious reasons this data is included in the computation of the\nobject ID. Explain why this is for the curious reader.\n\nFinally, indicate what the size field of the packed object represents.\nOtherwise, a reader might think that the size of a delta is the size of\nthe full object or that it might contain the offset or object ID,\nneither of which are the case. Explain clearly, however, that the\nvalues represent uncompressed sizes to avoid confusion.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/gitformat-pack.adoc | 18 ++++++++++++++++++\n 1 file changed, 18 insertions(+)\n\ndiff --git a/Documentation/gitformat-pack.adoc b/Documentation/gitformat-pack.adoc\nindex d6ae229be5..9b7af5c184 100644\n--- a/Documentation/gitformat-pack.adoc\n+++ b/Documentation/gitformat-pack.adoc\n@@ -32,6 +32,10 @@ In a repository using the traditional SHA-1, pack checksums, index checksums,\n and object IDs (object names) mentioned below are all computed using SHA-1.\n Similarly, in SHA-256 repositories, these values are computed using SHA-256.\n \n+CRC32 checksums are always computed over the entire packed object, including\n+the header (n-byte type and length); the base object name or offset, if any;\n+and the entire compressed object. The CRC32 algorithm used is that of zlib.\n+\n == pack-*.pack files have the following format:\n \n - A header appears at the beginning and consists of the following:\n@@ -80,6 +84,15 @@ Valid object types are:\n \n Type 5 is reserved for future expansion. Type 0 is invalid.\n \n+=== Object encoding\n+\n+Unlike loose objects, packed objects do not have a prefix containing the type,\n+size, and a NUL byte. These are not necessary because they can be determined by\n+the n-byte type and length that prefixes the data and so they are omitted from\n+the compressed and deltified data.\n+\n+The computation of the object ID still uses this prefix, however.\n+\n === Size encoding\n \n This document uses the following \"size encoding\" of non-negative\n@@ -92,6 +105,11 @@ values are more significant.\n This size encoding should not be confused with the \"offset encoding\",\n which is also used in this document.\n \n+When encoding the size of an undeltified object in a pack, the size is that of\n+the uncompressed raw object. For deltified objects, it is the size of the\n+uncompressed delta. The base object name or offset is not included in the size\n+computation.\n+\n === Deltified representation\n \n Conceptually there are only four object types: commit, tree, tag and\n"},{"id":"527848","messageId":"20251002223855.1022847-6-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 5/9] docs: add documentation for loose objects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:51Z","receivedAt":"2025-10-02T22:39:09Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We currently have no documentation for how loose objects are stored.\nLet's add some here so it's easy for people to understand how they\nwork.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/Makefile | 1 +\n Documentation/gitformat-loose.adoc | 53 ++++++++++++++++++++++++++++++\n Documentation/meson.build | 1 +\n 3 files changed, 55 insertions(+)\n create mode 100644 Documentation/gitformat-loose.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 6fb83d0c6e..e1d38fbfe6 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -34,6 +34,7 @@ MAN5_TXT += gitformat-bundle.adoc\n MAN5_TXT += gitformat-chunk.adoc\n MAN5_TXT += gitformat-commit-graph.adoc\n MAN5_TXT += gitformat-index.adoc\n+MAN5_TXT += gitformat-loose.adoc\n MAN5_TXT += gitformat-pack.adoc\n MAN5_TXT += gitformat-signature.adoc\n MAN5_TXT += githooks.adoc\ndiff --git a/Documentation/gitformat-loose.adoc b/Documentation/gitformat-loose.adoc\nnew file mode 100644\nindex 0000000000..947993663e\n--- /dev/null\n+++ b/Documentation/gitformat-loose.adoc\n@@ -0,0 +1,53 @@\n+gitformat-loose(5)\n+==================\n+\n+NAME\n+----\n+gitformat-loose - Git loose object format\n+\n+\n+SYNOPSIS\n+--------\n+[verse]\n+$GIT_DIR/objects/[0-9a-f][0-9a-f]/*\n+\n+DESCRIPTION\n+-----------\n+\n+Loose objects are how Git stores individual objects, where every object is\n+written as a separate file.\n+\n+Over the lifetime of a repository, objects are usually written as loose objects\n+initially. Eventually, these loose objects will be compacted into packfiles\n+via repository maintenance to improve disk space usage and speed up the lookup\n+of these objects.\n+\n+== Loose objects\n+\n+Each loose object contains a prefix, followed immediately by the data of the\n+object. The prefix contains ` \\0`. `` is one of `blob`,\n+`tree`, `commit`, or `tag` and `size` is the size of the data (without the\n+prefix) as a decimal integer expressed in ASCII.\n+\n+The entire contents, prefix and data concatenated, is then compressed with zlib\n+and the compressed data is stored in the file. The object ID of the object is\n+the SHA-1 or SHA-256 (as appropriate) hash of the uncompressed data.\n+\n+The file for the loose object is stored under the `objects` directory, with the\n+first two hex characters of the object ID being the directory and the remaining\n+characters being the file name. This is done to shard the data and avoid too\n+many files being in one directory, since some file systems perform poorly with\n+many items in a directory.\n+\n+As an example, the empty tree contains the data (when uncompressed) `tree 0\\0`\n+and, in a SHA-256 repository, would have the object ID\n+`6ef19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321` and would be\n+stored under\n+`$GIT_DIR/objects/6e/f19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321`.\n+\n+Similarly, a blob containing the contents `abc` would have the uncompressed\n+data of `blob 3\\0abc`.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex 41f43e0336..64f70ac724 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -172,6 +172,7 @@ manpages = {\n 'gitformat-chunk.adoc' : 5,\n 'gitformat-commit-graph.adoc' : 5,\n 'gitformat-index.adoc' : 5,\n+ 'gitformat-loose.adoc' : 5,\n 'gitformat-pack.adoc' : 5,\n 'gitformat-signature.adoc' : 5,\n 'githooks.adoc' : 5,\n"},{"id":"527849","messageId":"20251002223855.1022847-8-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 7/9] fsck: consider gpgsig headers expected in tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:53Z","receivedAt":"2025-10-02T22:39:09Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When we're creating a tag, we want to make sure that gpgsig and\ngpgsig-sha256 headers are allowed for the commit. The default fsck\nbehavior is to ignore the fact that they're left over, but some of our\ntests enable strict checking which flags them nonetheless. Add\nimproved checking for these headers as well as documentation and several\ntests.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/fsck-msgids.adoc | 6 ++++\n fsck.c | 18 ++++++++++++\n fsck.h | 2 ++\n t/t1450-fsck.sh | 54 ++++++++++++++++++++++++++++++++++\n 4 files changed, 80 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 0ba4f9a27e..52d9a8a811 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -10,6 +10,12 @@\n `badFilemode`::\n \t(INFO) A tree contains a bad filemode entry.\n \n+`badGpgsig`::\n+\t(ERROR) A tag contains a bad (truncated) signature (e.g., `gpgsig`) header.\n+\n+`badHeaderContinuation`::\n+\t(ERROR) A continuation header (such as for `gpgsig`) is unexpectedly truncated.\n+\n `badName`::\n \t(ERROR) An author/committer name is empty.\n \ndiff --git a/fsck.c b/fsck.c\nindex 171b424dd5..341e100d24 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1067,6 +1067,24 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,\n \telse\n \t\tret = fsck_ident(&buffer, oid, OBJ_TAG, options);\n \n+\tif (buffer < buffer_end && (skip_prefix(buffer, \"gpgsig \", &buffer) || skip_prefix(buffer, \"gpgsig-sha256 \", &buffer))) {\n+\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n+\t\tif (!eol) {\n+\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_GPGSIG, \"invalid format - unexpected end after 'gpgsig' or 'gpgsig-sha256' line\");\n+\t\t\tgoto done;\n+\t\t}\n+\t\tbuffer = eol + 1;\n+\n+\t\twhile (buffer < buffer_end && starts_with(buffer, \" \")) {\n+\t\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n+\t\t\tif (!eol) {\n+\t\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_HEADER_CONTINUATION, \"invalid format - unexpected end in 'gpgsig' or 'gpgsig-sha256' continuation line\");\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t\tbuffer = eol + 1;\n+\t\t}\n+\t}\n+\n \tif (buffer < buffer_end && !starts_with(buffer, \"\\n\")) {\n \t\t/*\n \t\t * The verify_headers() check will allow\ndiff --git a/fsck.h b/fsck.h\nindex dd7df3d5b3..c26616d7eb 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -25,9 +25,11 @@ enum fsck_msg_type {\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n \t/* errors */ \\\n+\tFUNC(BAD_HEADER_CONTINUATION, ERROR) \\\n \tFUNC(BAD_DATE, ERROR) \\\n \tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n \tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_GPGSIG, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex 5ae86c42be..c4b651c2dc 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -454,6 +454,60 @@ test_expect_success 'tag with NUL in header' '\n \ttest_grep \"error in tag $tag.*unterminated header: NUL at offset\" out\n '\n \n+test_expect_success 'tag accepts gpgsig header even if not validly signed' '\n+\ttest_oid_cache <<-\\EOF &&\n+\theader sha1:gpgsig-sha256\n+\theader sha256:gpgsig\n+\tEOF\n+\theader=$(test_oid header) &&\n+\tsha=$(git rev-parse HEAD) &&\n+\tcat >good-tag <<-EOF &&\n+\tobject $sha\n+\ttype commit\n+\ttag good\n+\ttagger T A Gger 1234567890 -0000\n+\t$header -----BEGIN PGP SIGNATURE-----\n+\t Not a valid signature\n+\t -----END PGP SIGNATURE-----\n+\n+\tThis is a good tag.\n+\tEOF\n+\n+\ttag=$(git hash-object --literally -t tag -w --stdin bad-tag <<-EOF &&\n+\tobject $sha\n+\ttype commit\n+\ttag good\n+\ttagger T A Gger 1234567890 -0000\n+\t$header -----BEGIN PGP SIGNATURE-----\n+\t Not a valid signature\n+\t -----END PGP SIGNATURE-----\n+\tjunk\n+\n+\tThis is a bad tag with junk at the end of the headers.\n+\tEOF\n+\n+\ttag=$(git hash-object --literally -t tag -w --stdin out &&\n+\ttest_grep \"error in tag $tag.*invalid format - extra header\" out\n+'\n+\n test_expect_success 'cleaned up' '\n \tgit fsck >actual 2>&1 &&\n \ttest_must_be_empty actual\n"},{"id":"527850","messageId":"20251002223855.1022847-7-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 6/9] rev-parse: allow printing compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:52Z","receivedAt":"2025-10-02T22:39:09Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Right now, we have a way to print the storage hash, the input hash, and\nthe output hash, but we lack a way to print the compatibility hash. Add\na new type to --show-object-format, compat, which prints this value.\n\nIf no compatibility hash exists, simply print a newline. This is\nimportant to allow users to use multiple options at once while still\ngetting unambiguous output.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-rev-parse.adoc | 11 ++++++-----\n builtin/rev-parse.c | 11 ++++++++++-\n t/t1500-rev-parse.sh | 34 ++++++++++++++++++++++++++++++++\n 3 files changed, 50 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-rev-parse.adoc b/Documentation/git-rev-parse.adoc\nindex cc32b4b4f0..465ae3e29d 100644\n--- a/Documentation/git-rev-parse.adoc\n+++ b/Documentation/git-rev-parse.adoc\n@@ -324,11 +324,12 @@ The following options are unaffected by `--path-format`:\n \tpath of the current directory relative to the top-level\n \tdirectory.\n \n---show-object-format[=(storage|input|output)]::\n-\tShow the object format (hash algorithm) used for the repository\n-\tfor storage inside the `.git` directory, input, or output. For\n-\tinput, multiple algorithms may be printed, space-separated.\n-\tIf not specified, the default is \"storage\".\n+--show-object-format[=(storage|input|output|compat)]::\n+\tShow the object format (hash algorithm) used for the repository for storage\n+\tinside the `.git` directory, input, output, or compatibility. For input,\n+\tmultiple algorithms may be printed, space-separated. If `compat` is\n+\trequested and no compatibility algorithm is enabled, prints an empty line. If\n+\tnot specified, the default is \"storage\".\n \n --show-ref-format::\n \tShow the reference storage format used for the repository.\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 44ff1b8342..187b7e8be9 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -1108,11 +1108,20 @@ int cmd_rev_parse(int argc,\n \t\t\t\tconst char *val = arg ? arg : \"storage\";\n \n \t\t\t\tif (strcmp(val, \"storage\") &&\n+\t\t\t\t strcmp(val, \"compat\") &&\n \t\t\t\t strcmp(val, \"input\") &&\n \t\t\t\t strcmp(val, \"output\"))\n \t\t\t\t\tdie(_(\"unknown mode for --show-object-format: %s\"),\n \t\t\t\t\t arg);\n-\t\t\t\tputs(the_hash_algo->name);\n+\n+\t\t\t\tif (!strcmp(val, \"compat\")) {\n+\t\t\t\t\tif (the_repository->compat_hash_algo)\n+\t\t\t\t\t\tputs(the_repository->compat_hash_algo->name);\n+\t\t\t\t\telse\n+\t\t\t\t\t\tputchar('\\n');\n+\t\t\t\t} else {\n+\t\t\t\t\tputs(the_hash_algo->name);\n+\t\t\t\t}\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 58a4583088..7739ab611b 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -207,6 +207,40 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n '\n \n+\n+test_expect_success 'rev-parse --show-object-format in repo with compat mode' '\n+\tmkdir repo &&\n+\t(\n+\t\tsane_unset GIT_DEFAULT_HASH &&\n+\t\tcd repo &&\n+\t\tgit init --object-format=sha256 &&\n+\t\tgit config extensions.compatobjectformat sha1 &&\n+\t\techo sha256 >expect &&\n+\t\tgit rev-parse --show-object-format >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=storage >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=input >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=output >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\techo sha1 >expect &&\n+\t\tgit rev-parse --show-object-format=compat >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\ttest_must_fail git rev-parse --show-object-format=squeamish-ossifrage 2>err &&\n+\t\tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n+\t) &&\n+\tmkdir repo2 &&\n+\t(\n+\t\tsane_unset GIT_DEFAULT_HASH &&\n+\t\tcd repo2 &&\n+\t\tgit init --object-format=sha256 &&\n+\t\techo >expect &&\n+\t\tgit rev-parse --show-object-format=compat >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_expect_success 'rev-parse --show-ref-format' '\n \ttest_detect_ref_format >expect &&\n \tgit rev-parse --show-ref-format >actual &&\n"},{"id":"527851","messageId":"20251002223855.1022847-9-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 8/9] t: allow specifying compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:54Z","receivedAt":"2025-10-02T22:39:09Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We want to specify a compatibility hash for testing interactions for\nSHA-256 repositories where we have SHA-1 compatibility enabled. Allow\nthe user to specify this scenario in the test suite by setting\nGIT_TEST_DEFAULT_HASH to \"sha256:sha1\".\n\nNote that this will get passed into GIT_DEFAULT_HASH, which Git itself\ndoes not presently support. However, we will support this in a future\ncommit.\n\nSince we'll now want to know the value for a specific version, let's add\nthe ability to specify either the storage hash (in this case, SHA-256)\nor the compatibility hash (SHA-1). We use a different value for the\ncompatibility hash that will be enabled for all repositories\n(test_repo_compat_hash_algo) versus the one that is used individually in\nsome tests (test_compat_hash_algo), since we want to still run those\nindividual tests without requiring that the testsuite be run fully in a\ncompatibility mode.\n\nIn some cases, we'll need to adjust our test suite to work in a proper\nway with a compatibility hash. For example, in such a case, we'll only\nuse pack index v3, since v1 and v2 lack support for multiple algorithms.\nSince we won't want to write those older formats, we'll need to skip\ntests that do so. Let's add a COMPAT_HASH prerequisite for this\npurpose.\n\nFinally, in this scenario, we can no longer rely on having broken\nobjects work since we lack compatibility mappings to rewrite objects in\nthe repository. Add a prerequisite, BROKEN_OBJECTS, that we define in\nterms of COMPAT_HASH and checks to see if creating deliberately broken\nobjects is possible, so that we can disable these tests if not.\n\nSigned-off-by: brian m. carlson \n---\n t/test-lib-functions.sh | 9 +++++++--\n t/test-lib.sh | 13 +++++++++++++\n 2 files changed, 20 insertions(+), 2 deletions(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex a28de7b19b..52d7759bf5 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1708,11 +1708,16 @@ test_set_hash () {\n # Detect the hash algorithm in use.\n test_detect_hash () {\n \tcase \"${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}\" in\n-\t\"sha256\")\n+\t*:*)\n+\t test_hash_algo=\"${GIT_TEST_DEFAULT_HASH%%:*}\"\n+\t test_compat_hash_algo=\"${GIT_TEST_DEFAULT_HASH##*:}\"\n+\t test_repo_compat_hash_algo=\"$test_compat_hash_algo\"\n+\t ;;\n+\tsha256)\n \t test_hash_algo=sha256\n \t test_compat_hash_algo=sha1\n \t ;;\n-\t*)\n+\tsha1)\n \t test_hash_algo=sha1\n \t test_compat_hash_algo=sha256\n \t ;;\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 621cd31ae1..9eb79324ee 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1917,6 +1917,19 @@ test_lazy_prereq DEFAULT_HASH_ALGORITHM '\n test_lazy_prereq DEFAULT_REPO_FORMAT '\n \ttest_have_prereq SHA1,REFFILES\n '\n+# BROKEN_OBJECTS is a test whether we can write deliberately broken objects and\n+# expect them to work. When running using SHA-256 mode with SHA-1\n+# compatibility, we cannot write such objects because there's no SHA-1\n+# compatibility value for a nonexistent object.\n+test_lazy_prereq BROKEN_OBJECTS '\n+\t! test_have_prereq COMPAT_HASH\n+'\n+\n+# COMPAT_HASH is a test if we're operating in a repository with SHA-256 with\n+# SHA-1 compatibility.\n+test_lazy_prereq COMPAT_HASH '\n+\ttest -n \"$test_repo_compat_hash_algo\"\n+'\n \n # Ensure that no test accidentally triggers a Git command\n # that runs the actual maintenance scheduler, affecting a user's\n"},{"id":"527852","messageId":"20251002223855.1022847-10-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251002223855.1022847-1-sandals@crustytoothpaste.net","subject":"[PATCH v2 9/9] t1010: use BROKEN_OBJECTS prerequisite","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-02T22:38:55Z","receivedAt":"2025-10-02T22:39:09Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When hash compatibility mode is enabled, we cannot write broken objects\nbecause they cannot be mapped into the other hash algorithm. Use the\nBROKEN_OBJECTS prerequisite to disable these tests and the writing of\nbroken objects in this mode.\n\nSigned-off-by: brian m. carlson \n---\n t/t1010-mktree.sh | 13 ++++++++-----\n 1 file changed, 8 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh\nindex e9973f7494..312fe6717a 100755\n--- a/t/t1010-mktree.sh\n+++ b/t/t1010-mktree.sh\n@@ -11,10 +11,13 @@ test_expect_success setup '\n \t\tgit add \"$d\" || return 1\n \tdone &&\n \techo zero >one &&\n-\tgit update-index --add --info-only one &&\n-\tgit write-tree --missing-ok >tree.missing &&\n-\tgit ls-tree $(cat tree.missing) >top.missing &&\n-\tgit ls-tree -r $(cat tree.missing) >all.missing &&\n+\tif test_have_prereq BROKEN_OBJECTS\n+\tthen\n+\t\tgit update-index --add --info-only one &&\n+\t\tgit write-tree --missing-ok >tree.missing &&\n+\t\tgit ls-tree $(cat tree.missing) >top.missing &&\n+\t\tgit ls-tree -r $(cat tree.missing) >all.missing\n+\tfi &&\n \techo one >one &&\n \tgit add one &&\n \tgit write-tree >tree &&\n@@ -53,7 +56,7 @@ test_expect_success 'ls-tree output in wrong order given to mktree (2)' '\n \ttest_cmp tree.withsub actual\n '\n \n-test_expect_success 'allow missing object with --missing' '\n+test_expect_success BROKEN_OBJECTS 'allow missing object with --missing' '\n \tgit mktree --missing actual &&\n \ttest_cmp tree.missing actual\n '\n"},{"id":"527890","messageId":"xmqq7bxbvs1n.fsf@gitster.g","threadId":"64168","inReplyTo":"20251002223855.1022847-2-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2 1/9] docs: update pack index v3 format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-03T17:00:52Z","receivedAt":"2025-10-03T17:00:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> diff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\n> index f047fd80ca..274dc993d4 100644\n> --- a/Documentation/technical/hash-function-transition.adoc\n> +++ b/Documentation/technical/hash-function-transition.adoc\n> @@ -227,9 +227,9 @@ network byte order):\n> ** 4-byte length in bytes of shortened object names. This is the\n> shortest possible length needed to make names in the shortened\n> object name table unambiguous.\n> - ** 4-byte integer, recording where tables relating to this format\n> + ** 8-byte integer, recording where tables relating to this format\n> are stored in this index file, as an offset from the beginning.\n> - * 4-byte offset to the trailer from the beginning of this file.\n> + * 8-byte offset to the trailer from the beginning of this file.\n> * Zero or more additional key/value pairs (4-byte key, 4-byte\n> value). Only one key is supported: 'PSRC'. See the \"Loose objects\n> and unreachable objects\" section for supported values and how this\n> @@ -276,10 +276,14 @@ network byte order):\n> up to and not including the table of CRC32 values.\n> - Zero or more NUL bytes.\n> - The trailer consists of the following:\n> - * A copy of the 20-byte SHA-256 checksum at the end of the\n> + * A copy of the full main hash checksum at the end of the\n> corresponding packfile.\n> - * 20-byte SHA-256 checksum of all of the above.\n> + * Full main hash checksum of all of the above.\n> +\n> +The \"full main hash\" is a full-length hash of the main (not compatibility)\n> +algorithm in the repository. Thus, if the main algorithm is SHA-256, this is\n> +a 32-byte SHA-256 hash and for SHA-1, it's a 20-byte SHA-1 hash.\n\nI see a nice improvement over v1 here. Very good.\n"},{"id":"527891","messageId":"xmqqzfa7ud9z.fsf@gitster.g","threadId":"64168","inReplyTo":"20251002223855.1022847-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2 5/9] docs: add documentation for loose objects","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-03T17:05:12Z","receivedAt":"2025-10-03T17:05:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> +DESCRIPTION\n> +-----------\n> +\n> +Loose objects are how Git stores individual objects, where every object is\n> +written as a separate file.\n> +\n> +Over the lifetime of a repository, objects are usually written as loose objects\n> +initially. Eventually, these loose objects will be compacted into packfiles\n> +via repository maintenance to improve disk space usage and speed up the lookup\n> +of these objects.\n\nMuch easier to follow relative to v1. Very much appreciated.\n\n> +== Loose objects\n> +\n> +Each loose object contains a prefix, followed immediately by the data of the\n> +object. The prefix contains ` \\0`. `` is one of `blob`,\n> +`tree`, `commit`, or `tag` and `size` is the size of the data (without the\n> +prefix) as a decimal integer expressed in ASCII.\n> +\n> +The entire contents, prefix and data concatenated, is then compressed with zlib\n> +and the compressed data is stored in the file. The object ID of the object is\n> +the SHA-1 or SHA-256 (as appropriate) hash of the uncompressed data.\n> +\n> +The file for the loose object is stored under the `objects` directory, with the\n> +first two hex characters of the object ID being the directory and the remaining\n> +characters being the file name. This is done to shard the data and avoid too\n> +many files being in one directory, since some file systems perform poorly with\n> +many items in a directory.\n\nAdditional explanation new in v2 look quite sensible.\n\n> +As an example, the empty tree contains the data (when uncompressed) `tree 0\\0`\n> +and, in a SHA-256 repository, would have the object ID\n> +`6ef19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321` and would be\n> +stored under\n> +`$GIT_DIR/objects/6e/f19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321`.\n> +\n> +Similarly, a blob containing the contents `abc` would have the uncompressed\n> +data of `blob 3\\0abc`.\n> +\n> +GIT\n> +---\n> +Part of the linkgit:git[1] suite\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index 41f43e0336..64f70ac724 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -172,6 +172,7 @@ manpages = {\n> 'gitformat-chunk.adoc' : 5,\n> 'gitformat-commit-graph.adoc' : 5,\n> 'gitformat-index.adoc' : 5,\n> + 'gitformat-loose.adoc' : 5,\n> 'gitformat-pack.adoc' : 5,\n> 'gitformat-signature.adoc' : 5,\n> 'githooks.adoc' : 5,\n\n"},{"id":"527892","messageId":"xmqqv7kvud5o.fsf@gitster.g","threadId":"64168","inReplyTo":"20251002223855.1022847-5-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2 4/9] docs: improve ambiguous areas of pack format documentation","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-03T17:07:47Z","receivedAt":"2025-10-03T17:07:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> +=== Object encoding\n> +\n> +Unlike loose objects, packed objects do not have a prefix containing the type,\n> +size, and a NUL byte. These are not necessary because they can be determined by\n> +the n-byte type and length that prefixes the data and so they are omitted from\n> +the compressed and deltified data.\n> +\n> +The computation of the object ID still uses this prefix, however.\n\n\"however\" -> \"by reconstructing it from the type/length data as\nneeded\"?\n\nOther than that, the new text reads very well.\n\nThanks.\n"},{"id":"527893","messageId":"xmqqqzvjucuh.fsf@gitster.g","threadId":"64168","inReplyTo":"20251002223855.1022847-9-sandals@crustytoothpaste.net","subject":"Re: [PATCH v2 8/9] t: allow specifying compatibility hash","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-03T17:14:30Z","receivedAt":"2025-10-03T17:14:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> Finally, in this scenario, we can no longer rely on having broken\n> objects work since we lack compatibility mappings to rewrite objects in\n> the repository. Add a prerequisite, BROKEN_OBJECTS, that we define in\n> terms of COMPAT_HASH and checks to see if creating deliberately broken\n> objects is possible, so that we can disable these tests if not.\n\nThanks for an attention for this kind of details.\n\n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index 621cd31ae1..9eb79324ee 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -1917,6 +1917,19 @@ test_lazy_prereq DEFAULT_HASH_ALGORITHM '\n> test_lazy_prereq DEFAULT_REPO_FORMAT '\n> \ttest_have_prereq SHA1,REFFILES\n> '\n> +# BROKEN_OBJECTS is a test whether we can write deliberately broken objects and\n> +# expect them to work. When running using SHA-256 mode with SHA-1\n> +# compatibility, we cannot write such objects because there's no SHA-1\n> +# compatibility value for a nonexistent object.\n> +test_lazy_prereq BROKEN_OBJECTS '\n> +\t! test_have_prereq COMPAT_HASH\n> +'\n> +\n> +# COMPAT_HASH is a test if we're operating in a repository with SHA-256 with\n> +# SHA-1 compatibility.\n> +test_lazy_prereq COMPAT_HASH '\n> +\ttest -n \"$test_repo_compat_hash_algo\"\n> +'\n> \n> # Ensure that no test accidentally triggers a Git command\n> # that runs the actual maintenance scheduler, affecting a user's\n"},{"id":"527905","messageId":"aOA11cS_vuVwSbkr@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqqqzvjucuh.fsf@gitster.g","subject":"Re: [PATCH v2 8/9] t: allow specifying compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-03T20:45:09Z","receivedAt":"2025-10-03T20:45:17Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-10-03 at 17:14:30, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > Finally, in this scenario, we can no longer rely on having broken\n> > objects work since we lack compatibility mappings to rewrite objects in\n> > the repository. Add a prerequisite, BROKEN_OBJECTS, that we define in\n> > terms of COMPAT_HASH and checks to see if creating deliberately broken\n> > objects is possible, so that we can disable these tests if not.\n> \n> Thanks for an attention for this kind of details.\n\nI appreciate the kind words. To be fair, this is one of a few major\ncauses of testsuite failures when running in compatibility mode, so\naddressing these is important to getting the testsuite to work properly\nin such a mode. Fortunately, they are relatively easy to spot and fix.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"527908","messageId":"aOA6z-kFCd78XtxX@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqqv7kvud5o.fsf@gitster.g","subject":"Re: [PATCH v2 4/9] docs: improve ambiguous areas of pack format documentation","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-03T21:06:23Z","receivedAt":"2025-10-03T21:06:25Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-10-03 at 17:07:47, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > +=== Object encoding\n> > +\n> > +Unlike loose objects, packed objects do not have a prefix containing the type,\n> > +size, and a NUL byte. These are not necessary because they can be determined by\n> > +the n-byte type and length that prefixes the data and so they are omitted from\n> > +the compressed and deltified data.\n> > +\n> > +The computation of the object ID still uses this prefix, however.\n> \n> \"however\" -> \"by reconstructing it from the type/length data as\n> needed\"?\n> \n> Other than that, the new text reads very well.\n\nI've already squashed in a very similar change. I'll wait to see if\nPatrick or anyone else has more comments and then send v3.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"528412","messageId":"20251009215626.3089287-4-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 3/9] docs: reflect actual double signature for tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:20Z","receivedAt":"2025-10-09T21:56:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The documentation for the hash function transition reflects the original\ndesign where the SHA-256 signature would always be placed in a header.\nHowever, due to a missed patch in Git 2.29, we shipped SHA-256 support\nsuch that the signature for the current algorithm is always an in-body\nsignature and the opposite algorithm is always in a header. Since the\ndocumentation is inaccurate, update it to reflect the correct\ninformation.\n\nSigned-off-by: brian m. carlson \n---\n .../technical/hash-function-transition.adoc | 20 ++++++++++---------\n 1 file changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex adb0c61e53..2359d7d106 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -429,17 +429,19 @@ ordinary unsigned commit.\n \n Signed Tags\n ~~~~~~~~~~~\n-We add a new field \"gpgsig-sha256\" to the tag object format to allow\n-signing tags without relying on SHA-1. Its signed payload is the\n-SHA-256 content of the tag with its gpgsig-sha256 field and \"-----BEGIN PGP\n-SIGNATURE-----\" delimited in-body signature removed.\n+We add new fields \"gpgsig\" and \"gpgsig-sha256\" to the tag object format to\n+allow signing tags in both formats. The in-body signature is used for the\n+signature in the current hash algorithm and the header is used for the\n+signature in the other algorithm. Thus, a dual-signature tag will contain both\n+an in-body signature and a gpgsig-sha256 header for the SHA-1 format of an\n+object or both an in-body signature and a gpgsig header for the SHA-256 format\n+of and object.\n \n-This means tags can be signed\n+The signed payload of the tag is the content of the tag in the current\n+algorithm with both its gpgsig and gpgsig-sha256 fields and\n+\"-----BEGIN PGP SIGNATURE-----\" delimited in-body signature removed.\n \n-1. using SHA-1 only, as in existing signed tag objects\n-2. using both SHA-1 and SHA-256, by using gpgsig-sha256 and an in-body\n- signature.\n-3. using only SHA-256, by only using the gpgsig-sha256 field.\n+This means tags can be signed using one or both algorithms.\n \n Mergetag embedding\n ~~~~~~~~~~~~~~~~~~\n"},{"id":"528413","messageId":"20251009215626.3089287-5-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 4/9] docs: improve ambiguous areas of pack format documentation","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:21Z","receivedAt":"2025-10-09T21:56:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"It is fair to say that our pack and indexing code is quite complex.\nContributors who wish to work on this code or implementors of other\nimplementations would benefit from clear, unambiguous documentation\nabout how our data formats are structured and encoded and what data is\nused in the computation of certain values. Unfortunately, some of this\ndata is missing, which leads to confusion and frustration.\n\nLet's document some of this data to help clarify things. Specify over\nwhat data CRC32 values are computed and also note which CRC32 algorithm\nis used, since Wikipedia mentions at least four 32-bit CRC algorithms\nand notes that it's possible to use different bit orderings.\n\nIn addition, note how we encode objects in the pack. One might be led\nto believe that packed objects are always stored with the \"\n\\0\" prefix of loose objects, but that is not the case, although\nfor obvious reasons this data is included in the computation of the\nobject ID. Explain why this is for the curious reader.\n\nFinally, indicate what the size field of the packed object represents.\nOtherwise, a reader might think that the size of a delta is the size of\nthe full object or that it might contain the offset or object ID,\nneither of which are the case. Explain clearly, however, that the\nvalues represent uncompressed sizes to avoid confusion.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/gitformat-pack.adoc | 19 +++++++++++++++++++\n 1 file changed, 19 insertions(+)\n\ndiff --git a/Documentation/gitformat-pack.adoc b/Documentation/gitformat-pack.adoc\nindex d6ae229be5..1b4db4aa61 100644\n--- a/Documentation/gitformat-pack.adoc\n+++ b/Documentation/gitformat-pack.adoc\n@@ -32,6 +32,10 @@ In a repository using the traditional SHA-1, pack checksums, index checksums,\n and object IDs (object names) mentioned below are all computed using SHA-1.\n Similarly, in SHA-256 repositories, these values are computed using SHA-256.\n \n+CRC32 checksums are always computed over the entire packed object, including\n+the header (n-byte type and length); the base object name or offset, if any;\n+and the entire compressed object. The CRC32 algorithm used is that of zlib.\n+\n == pack-*.pack files have the following format:\n \n - A header appears at the beginning and consists of the following:\n@@ -80,6 +84,16 @@ Valid object types are:\n \n Type 5 is reserved for future expansion. Type 0 is invalid.\n \n+=== Object encoding\n+\n+Unlike loose objects, packed objects do not have a prefix containing the type,\n+size, and a NUL byte. These are not necessary because they can be determined by\n+the n-byte type and length that prefixes the data and so they are omitted from\n+the compressed and deltified data.\n+\n+The computation of the object ID still uses this prefix by reconstructing it\n+from the type and length as needed.\n+\n === Size encoding\n \n This document uses the following \"size encoding\" of non-negative\n@@ -92,6 +106,11 @@ values are more significant.\n This size encoding should not be confused with the \"offset encoding\",\n which is also used in this document.\n \n+When encoding the size of an undeltified object in a pack, the size is that of\n+the uncompressed raw object. For deltified objects, it is the size of the\n+uncompressed delta. The base object name or offset is not included in the size\n+computation.\n+\n === Deltified representation\n \n Conceptually there are only four object types: commit, tree, tag and\n"},{"id":"528414","messageId":"20251009215626.3089287-1-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20250919010911.649831-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 0/9] SHA-1/SHA-256 interoperability, part 1","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:17Z","receivedAt":"2025-10-09T21:56:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This is the first of several series for SHA-1 and SHA-256\ninteroperability, all of which will hopefully land in Git 3.0.\n\nThe first set of commits here is documentation updates for documentation\nwhich was incorrect, missing, or simply wrong. I have spent more time\nthan I'd like in the pack code and felt our documentation there could be\nmore helpful. I also am correcting some things about the\ninteroperability formats that I've found are not correct or efficient in\nterms of implementation and thus I will be implementing differently.\n\nThe loose object documentation will be updated with the loose object\nmapping in a future commit, but I felt I should send a basic loose\nobject document first, so here it is.\n\nThe remaining commits are for expected gpgsig headers in tags, which\ncauses some tests which use strict fsck to fail, as well as for\nprerequisites for compatibility hashes in the testsuite. Actually using\nthis configuration is not possible since the tests are still very broken\nusing it, but declaring these prerequisites allows me and others to send\nin patches that use them and thus make our testsuite more resilient.\n\nFor example, in interoperability mode we cannot write objects that\nare not valid since we cannot convert them into the other hash\nalgorithm. Thus, when we're testing in a mode that has a compatibility\nalgorithm, we skip these tests.\n\nThe goal is to run the tests in a full compatibility mode where\neverything is dual-hash as well as introduce some specific tests for\ninteroperability that run in all configurations of the tests.\n\nChanges from v2:\n* Improve the language slightly in the pack documentation.\n\nChanges from v1:\n* Squash the two test changes into one commit.\n* Include a new commit showing the use of the BROKEN_OBJECTS prereq.\n* Mention using main algorithm hash in pack index v3.\n* Hopefully clarify signed tags.\n* Improve text for pack format documentation.\n* Wire up build of loose object documentation.\n* Remove loose object map documentation.\n* Rephrase text about loose objects.\n* Remove needless RUST prerequisite.\n* Wrap overly long line.\n* Reject invalid signature algorithms in tag headers.\n* Fix if/whether problem in test comment.\n\nbrian m. carlson (9):\n docs: update pack index v3 format\n docs: update offset order for pack index v3\n docs: reflect actual double signature for tags\n docs: improve ambiguous areas of pack format documentation\n docs: add documentation for loose objects\n rev-parse: allow printing compatibility hash\n fsck: consider gpgsig headers expected in tags\n t: allow specifying compatibility hash\n t1010: use BROKEN_OBJECTS prerequisite\n\n Documentation/Makefile | 1 +\n Documentation/fsck-msgids.adoc | 6 +++\n Documentation/git-rev-parse.adoc | 11 ++--\n Documentation/gitformat-loose.adoc | 53 ++++++++++++++++++\n Documentation/gitformat-pack.adoc | 19 +++++++\n Documentation/meson.build | 1 +\n .../technical/hash-function-transition.adoc | 42 ++++++++-------\n builtin/rev-parse.c | 11 +++-\n fsck.c | 18 +++++++\n fsck.h | 2 +\n t/t1010-mktree.sh | 13 +++--\n t/t1450-fsck.sh | 54 +++++++++++++++++++\n t/t1500-rev-parse.sh | 34 ++++++++++++\n t/test-lib-functions.sh | 9 +++-\n t/test-lib.sh | 13 +++++\n 15 files changed, 255 insertions(+), 32 deletions(-)\n create mode 100644 Documentation/gitformat-loose.adoc\n\n"},{"id":"528415","messageId":"20251009215626.3089287-3-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 2/9] docs: update offset order for pack index v3","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:19Z","receivedAt":"2025-10-09T21:56:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The current design of pack index v3 has items in two different orders:\nsorted shortened object ID order and pack order. The shortened object\nIDs and the pack index offset values are in the former order and\neverything else is in the latter.\n\nThis, however, poses some problems. We have many parts of the packfile\ncode that expect to find out data about an object knowing only its index\nin pack order. With the current design, to find the pack offset after\nhaving looked up the index in pack order, we must then look up the full\nobject ID and use that to look up the shortened object ID to find the\npack offset, which is inconvenient, inefficient, and leads to poor cache\nusage.\n\nInstead, let's change the offset values to be looked up by pack order.\nThis works better because once we know the pack order offset, we can\nfind the full object name and its location in the pack with a simple\nindex into their respective tables. This makes many operations much\nmore efficient, especially with the functions we already have, and it\navoids the need for the revindex with pack index v3.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/technical/hash-function-transition.adoc | 10 ++++------\n 1 file changed, 4 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex 274dc993d4..adb0c61e53 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -260,12 +260,10 @@ network byte order):\n compressed data to be copied directly from pack to pack during\n repacking without undetected data corruption.\n \n- * A table of 4-byte offset values. For an object in the table of\n- sorted shortened object names, the value at the corresponding\n- index in this table indicates where that object can be found in\n- the pack file. These are usually 31-bit pack file offsets, but\n- large offsets are encoded as an index into the next table with the\n- most significant bit set.\n+ * A table of 4-byte offset values. The index of this table in pack order\n+ indicates where that object can be found in the pack file. These are\n+ usually 31-bit pack file offsets, but large offsets are encoded as\n+ an index into the next table with the most significant bit set.\n \n * A table of 8-byte offset entries (empty for pack files less than\n 2 GiB). Pack files are organized with heavily used objects toward\n"},{"id":"528416","messageId":"20251009215626.3089287-2-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 1/9] docs: update pack index v3 format","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:18Z","receivedAt":"2025-10-09T21:56:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Our current pack index v3 format uses 4-byte integers to find the\ntrailer of the file. This effectively means that the file cannot be\nmuch larger than 2^32. While this might at first seem to be okay, we\nexpect that each object will have at least 64 bytes worth of data, which\nmeans that no more than about 67 million objects can be stored.\n\nAgain, this might seem fine, but unfortunately, we know of many users\nwho attempt to create repos with extremely large numbers of commits to\nget a \"high score,\" and we've already seen repositories with at least 55\nmillion commits. In the interests of gracefully handling repositories\neven for these well-intentioned but ultimately misguided users, let's\nchange these lengths to 8 bytes.\n\nFor the checksums at the end of the file, we're producing 32-byte\nSHA-256 checksums because that's what we already do with pack index v2\nand SHA-256. Truncating SHA-256 doesn't pose any actual security\nproblems other than those related to the reduced size, but our pack\nchecksum must already be 32 bytes (since SHA-256 packs have 32-byte\nchecksums) and it simplifies the code to use the existing hashfile logic\nfor these cases for the index checksum as well.\n\nIn addition, even though we may not need cryptographic security for the\nindex checksum, we'd like to avoid arguments from auditors and such for\norganizations that may have compliance or security requirements. Using\nthe simple, boring choice of the full SHA-256 hash avoids all possible\ndiscussion related to hash truncation and removes impediments for these\norganizations.\n\nNote that we do not yet have a pack index v3 implementation in Git, so\nit should be fine to change this format. However, such an\nimplementation has been written for future inclusion following this\nformat.\n\nSigned-off-by: brian m. carlson \n---\n .../technical/hash-function-transition.adoc | 12 ++++++++----\n 1 file changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/technical/hash-function-transition.adoc b/Documentation/technical/hash-function-transition.adoc\nindex f047fd80ca..274dc993d4 100644\n--- a/Documentation/technical/hash-function-transition.adoc\n+++ b/Documentation/technical/hash-function-transition.adoc\n@@ -227,9 +227,9 @@ network byte order):\n ** 4-byte length in bytes of shortened object names. This is the\n shortest possible length needed to make names in the shortened\n object name table unambiguous.\n- ** 4-byte integer, recording where tables relating to this format\n+ ** 8-byte integer, recording where tables relating to this format\n are stored in this index file, as an offset from the beginning.\n- * 4-byte offset to the trailer from the beginning of this file.\n+ * 8-byte offset to the trailer from the beginning of this file.\n * Zero or more additional key/value pairs (4-byte key, 4-byte\n value). Only one key is supported: 'PSRC'. See the \"Loose objects\n and unreachable objects\" section for supported values and how this\n@@ -276,10 +276,14 @@ network byte order):\n up to and not including the table of CRC32 values.\n - Zero or more NUL bytes.\n - The trailer consists of the following:\n- * A copy of the 20-byte SHA-256 checksum at the end of the\n+ * A copy of the full main hash checksum at the end of the\n corresponding packfile.\n \n- * 20-byte SHA-256 checksum of all of the above.\n+ * Full main hash checksum of all of the above.\n+\n+The \"full main hash\" is a full-length hash of the main (not compatibility)\n+algorithm in the repository. Thus, if the main algorithm is SHA-256, this is\n+a 32-byte SHA-256 hash and for SHA-1, it's a 20-byte SHA-1 hash.\n \n Loose object index\n ~~~~~~~~~~~~~~~~~~\n"},{"id":"528417","messageId":"20251009215626.3089287-6-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 5/9] docs: add documentation for loose objects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:22Z","receivedAt":"2025-10-09T21:56:38Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We currently have no documentation for how loose objects are stored.\nLet's add some here so it's easy for people to understand how they\nwork.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/Makefile | 1 +\n Documentation/gitformat-loose.adoc | 53 ++++++++++++++++++++++++++++++\n Documentation/meson.build | 1 +\n 3 files changed, 55 insertions(+)\n create mode 100644 Documentation/gitformat-loose.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 6fb83d0c6e..e1d38fbfe6 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -34,6 +34,7 @@ MAN5_TXT += gitformat-bundle.adoc\n MAN5_TXT += gitformat-chunk.adoc\n MAN5_TXT += gitformat-commit-graph.adoc\n MAN5_TXT += gitformat-index.adoc\n+MAN5_TXT += gitformat-loose.adoc\n MAN5_TXT += gitformat-pack.adoc\n MAN5_TXT += gitformat-signature.adoc\n MAN5_TXT += githooks.adoc\ndiff --git a/Documentation/gitformat-loose.adoc b/Documentation/gitformat-loose.adoc\nnew file mode 100644\nindex 0000000000..947993663e\n--- /dev/null\n+++ b/Documentation/gitformat-loose.adoc\n@@ -0,0 +1,53 @@\n+gitformat-loose(5)\n+==================\n+\n+NAME\n+----\n+gitformat-loose - Git loose object format\n+\n+\n+SYNOPSIS\n+--------\n+[verse]\n+$GIT_DIR/objects/[0-9a-f][0-9a-f]/*\n+\n+DESCRIPTION\n+-----------\n+\n+Loose objects are how Git stores individual objects, where every object is\n+written as a separate file.\n+\n+Over the lifetime of a repository, objects are usually written as loose objects\n+initially. Eventually, these loose objects will be compacted into packfiles\n+via repository maintenance to improve disk space usage and speed up the lookup\n+of these objects.\n+\n+== Loose objects\n+\n+Each loose object contains a prefix, followed immediately by the data of the\n+object. The prefix contains ` \\0`. `` is one of `blob`,\n+`tree`, `commit`, or `tag` and `size` is the size of the data (without the\n+prefix) as a decimal integer expressed in ASCII.\n+\n+The entire contents, prefix and data concatenated, is then compressed with zlib\n+and the compressed data is stored in the file. The object ID of the object is\n+the SHA-1 or SHA-256 (as appropriate) hash of the uncompressed data.\n+\n+The file for the loose object is stored under the `objects` directory, with the\n+first two hex characters of the object ID being the directory and the remaining\n+characters being the file name. This is done to shard the data and avoid too\n+many files being in one directory, since some file systems perform poorly with\n+many items in a directory.\n+\n+As an example, the empty tree contains the data (when uncompressed) `tree 0\\0`\n+and, in a SHA-256 repository, would have the object ID\n+`6ef19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321` and would be\n+stored under\n+`$GIT_DIR/objects/6e/f19b41225c5369f1c104d45d8d85efa9b057b53b14b4b9b939dd74decc5321`.\n+\n+Similarly, a blob containing the contents `abc` would have the uncompressed\n+data of `blob 3\\0abc`.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex 41f43e0336..64f70ac724 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -172,6 +172,7 @@ manpages = {\n 'gitformat-chunk.adoc' : 5,\n 'gitformat-commit-graph.adoc' : 5,\n 'gitformat-index.adoc' : 5,\n+ 'gitformat-loose.adoc' : 5,\n 'gitformat-pack.adoc' : 5,\n 'gitformat-signature.adoc' : 5,\n 'githooks.adoc' : 5,\n"},{"id":"528418","messageId":"20251009215626.3089287-7-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 6/9] rev-parse: allow printing compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:23Z","receivedAt":"2025-10-09T21:56:38Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Right now, we have a way to print the storage hash, the input hash, and\nthe output hash, but we lack a way to print the compatibility hash. Add\na new type to --show-object-format, compat, which prints this value.\n\nIf no compatibility hash exists, simply print a newline. This is\nimportant to allow users to use multiple options at once while still\ngetting unambiguous output.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/git-rev-parse.adoc | 11 ++++++-----\n builtin/rev-parse.c | 11 ++++++++++-\n t/t1500-rev-parse.sh | 34 ++++++++++++++++++++++++++++++++\n 3 files changed, 50 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-rev-parse.adoc b/Documentation/git-rev-parse.adoc\nindex cc32b4b4f0..465ae3e29d 100644\n--- a/Documentation/git-rev-parse.adoc\n+++ b/Documentation/git-rev-parse.adoc\n@@ -324,11 +324,12 @@ The following options are unaffected by `--path-format`:\n \tpath of the current directory relative to the top-level\n \tdirectory.\n \n---show-object-format[=(storage|input|output)]::\n-\tShow the object format (hash algorithm) used for the repository\n-\tfor storage inside the `.git` directory, input, or output. For\n-\tinput, multiple algorithms may be printed, space-separated.\n-\tIf not specified, the default is \"storage\".\n+--show-object-format[=(storage|input|output|compat)]::\n+\tShow the object format (hash algorithm) used for the repository for storage\n+\tinside the `.git` directory, input, output, or compatibility. For input,\n+\tmultiple algorithms may be printed, space-separated. If `compat` is\n+\trequested and no compatibility algorithm is enabled, prints an empty line. If\n+\tnot specified, the default is \"storage\".\n \n --show-ref-format::\n \tShow the reference storage format used for the repository.\ndiff --git a/builtin/rev-parse.c b/builtin/rev-parse.c\nindex 44ff1b8342..187b7e8be9 100644\n--- a/builtin/rev-parse.c\n+++ b/builtin/rev-parse.c\n@@ -1108,11 +1108,20 @@ int cmd_rev_parse(int argc,\n \t\t\t\tconst char *val = arg ? arg : \"storage\";\n \n \t\t\t\tif (strcmp(val, \"storage\") &&\n+\t\t\t\t strcmp(val, \"compat\") &&\n \t\t\t\t strcmp(val, \"input\") &&\n \t\t\t\t strcmp(val, \"output\"))\n \t\t\t\t\tdie(_(\"unknown mode for --show-object-format: %s\"),\n \t\t\t\t\t arg);\n-\t\t\t\tputs(the_hash_algo->name);\n+\n+\t\t\t\tif (!strcmp(val, \"compat\")) {\n+\t\t\t\t\tif (the_repository->compat_hash_algo)\n+\t\t\t\t\t\tputs(the_repository->compat_hash_algo->name);\n+\t\t\t\t\telse\n+\t\t\t\t\t\tputchar('\\n');\n+\t\t\t\t} else {\n+\t\t\t\t\tputs(the_hash_algo->name);\n+\t\t\t\t}\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tif (!strcmp(arg, \"--show-ref-format\")) {\ndiff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh\nindex 58a4583088..7739ab611b 100755\n--- a/t/t1500-rev-parse.sh\n+++ b/t/t1500-rev-parse.sh\n@@ -207,6 +207,40 @@ test_expect_success 'rev-parse --show-object-format in repo' '\n \tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n '\n \n+\n+test_expect_success 'rev-parse --show-object-format in repo with compat mode' '\n+\tmkdir repo &&\n+\t(\n+\t\tsane_unset GIT_DEFAULT_HASH &&\n+\t\tcd repo &&\n+\t\tgit init --object-format=sha256 &&\n+\t\tgit config extensions.compatobjectformat sha1 &&\n+\t\techo sha256 >expect &&\n+\t\tgit rev-parse --show-object-format >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=storage >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=input >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit rev-parse --show-object-format=output >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\techo sha1 >expect &&\n+\t\tgit rev-parse --show-object-format=compat >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\ttest_must_fail git rev-parse --show-object-format=squeamish-ossifrage 2>err &&\n+\t\tgrep \"unknown mode for --show-object-format: squeamish-ossifrage\" err\n+\t) &&\n+\tmkdir repo2 &&\n+\t(\n+\t\tsane_unset GIT_DEFAULT_HASH &&\n+\t\tcd repo2 &&\n+\t\tgit init --object-format=sha256 &&\n+\t\techo >expect &&\n+\t\tgit rev-parse --show-object-format=compat >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n test_expect_success 'rev-parse --show-ref-format' '\n \ttest_detect_ref_format >expect &&\n \tgit rev-parse --show-ref-format >actual &&\n"},{"id":"528419","messageId":"20251009215626.3089287-9-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 8/9] t: allow specifying compatibility hash","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:25Z","receivedAt":"2025-10-09T21:56:39Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"We want to specify a compatibility hash for testing interactions for\nSHA-256 repositories where we have SHA-1 compatibility enabled. Allow\nthe user to specify this scenario in the test suite by setting\nGIT_TEST_DEFAULT_HASH to \"sha256:sha1\".\n\nNote that this will get passed into GIT_DEFAULT_HASH, which Git itself\ndoes not presently support. However, we will support this in a future\ncommit.\n\nSince we'll now want to know the value for a specific version, let's add\nthe ability to specify either the storage hash (in this case, SHA-256)\nor the compatibility hash (SHA-1). We use a different value for the\ncompatibility hash that will be enabled for all repositories\n(test_repo_compat_hash_algo) versus the one that is used individually in\nsome tests (test_compat_hash_algo), since we want to still run those\nindividual tests without requiring that the testsuite be run fully in a\ncompatibility mode.\n\nIn some cases, we'll need to adjust our test suite to work in a proper\nway with a compatibility hash. For example, in such a case, we'll only\nuse pack index v3, since v1 and v2 lack support for multiple algorithms.\nSince we won't want to write those older formats, we'll need to skip\ntests that do so. Let's add a COMPAT_HASH prerequisite for this\npurpose.\n\nFinally, in this scenario, we can no longer rely on having broken\nobjects work since we lack compatibility mappings to rewrite objects in\nthe repository. Add a prerequisite, BROKEN_OBJECTS, that we define in\nterms of COMPAT_HASH and checks to see if creating deliberately broken\nobjects is possible, so that we can disable these tests if not.\n\nSigned-off-by: brian m. carlson \n---\n t/test-lib-functions.sh | 9 +++++++--\n t/test-lib.sh | 13 +++++++++++++\n 2 files changed, 20 insertions(+), 2 deletions(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex a28de7b19b..52d7759bf5 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1708,11 +1708,16 @@ test_set_hash () {\n # Detect the hash algorithm in use.\n test_detect_hash () {\n \tcase \"${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}\" in\n-\t\"sha256\")\n+\t*:*)\n+\t test_hash_algo=\"${GIT_TEST_DEFAULT_HASH%%:*}\"\n+\t test_compat_hash_algo=\"${GIT_TEST_DEFAULT_HASH##*:}\"\n+\t test_repo_compat_hash_algo=\"$test_compat_hash_algo\"\n+\t ;;\n+\tsha256)\n \t test_hash_algo=sha256\n \t test_compat_hash_algo=sha1\n \t ;;\n-\t*)\n+\tsha1)\n \t test_hash_algo=sha1\n \t test_compat_hash_algo=sha256\n \t ;;\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 621cd31ae1..9eb79324ee 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1917,6 +1917,19 @@ test_lazy_prereq DEFAULT_HASH_ALGORITHM '\n test_lazy_prereq DEFAULT_REPO_FORMAT '\n \ttest_have_prereq SHA1,REFFILES\n '\n+# BROKEN_OBJECTS is a test whether we can write deliberately broken objects and\n+# expect them to work. When running using SHA-256 mode with SHA-1\n+# compatibility, we cannot write such objects because there's no SHA-1\n+# compatibility value for a nonexistent object.\n+test_lazy_prereq BROKEN_OBJECTS '\n+\t! test_have_prereq COMPAT_HASH\n+'\n+\n+# COMPAT_HASH is a test if we're operating in a repository with SHA-256 with\n+# SHA-1 compatibility.\n+test_lazy_prereq COMPAT_HASH '\n+\ttest -n \"$test_repo_compat_hash_algo\"\n+'\n \n # Ensure that no test accidentally triggers a Git command\n # that runs the actual maintenance scheduler, affecting a user's\n"},{"id":"528420","messageId":"20251009215626.3089287-8-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 7/9] fsck: consider gpgsig headers expected in tags","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:24Z","receivedAt":"2025-10-09T21:56:39Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When we're creating a tag, we want to make sure that gpgsig and\ngpgsig-sha256 headers are allowed for the commit. The default fsck\nbehavior is to ignore the fact that they're left over, but some of our\ntests enable strict checking which flags them nonetheless. Add\nimproved checking for these headers as well as documentation and several\ntests.\n\nSigned-off-by: brian m. carlson \n---\n Documentation/fsck-msgids.adoc | 6 ++++\n fsck.c | 18 ++++++++++++\n fsck.h | 2 ++\n t/t1450-fsck.sh | 54 ++++++++++++++++++++++++++++++++++\n 4 files changed, 80 insertions(+)\n\ndiff --git a/Documentation/fsck-msgids.adoc b/Documentation/fsck-msgids.adoc\nindex 0ba4f9a27e..52d9a8a811 100644\n--- a/Documentation/fsck-msgids.adoc\n+++ b/Documentation/fsck-msgids.adoc\n@@ -10,6 +10,12 @@\n `badFilemode`::\n \t(INFO) A tree contains a bad filemode entry.\n \n+`badGpgsig`::\n+\t(ERROR) A tag contains a bad (truncated) signature (e.g., `gpgsig`) header.\n+\n+`badHeaderContinuation`::\n+\t(ERROR) A continuation header (such as for `gpgsig`) is unexpectedly truncated.\n+\n `badName`::\n \t(ERROR) An author/committer name is empty.\n \ndiff --git a/fsck.c b/fsck.c\nindex 171b424dd5..341e100d24 100644\n--- a/fsck.c\n+++ b/fsck.c\n@@ -1067,6 +1067,24 @@ int fsck_tag_standalone(const struct object_id *oid, const char *buffer,\n \telse\n \t\tret = fsck_ident(&buffer, oid, OBJ_TAG, options);\n \n+\tif (buffer < buffer_end && (skip_prefix(buffer, \"gpgsig \", &buffer) || skip_prefix(buffer, \"gpgsig-sha256 \", &buffer))) {\n+\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n+\t\tif (!eol) {\n+\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_GPGSIG, \"invalid format - unexpected end after 'gpgsig' or 'gpgsig-sha256' line\");\n+\t\t\tgoto done;\n+\t\t}\n+\t\tbuffer = eol + 1;\n+\n+\t\twhile (buffer < buffer_end && starts_with(buffer, \" \")) {\n+\t\t\teol = memchr(buffer, '\\n', buffer_end - buffer);\n+\t\t\tif (!eol) {\n+\t\t\t\tret = report(options, oid, OBJ_TAG, FSCK_MSG_BAD_HEADER_CONTINUATION, \"invalid format - unexpected end in 'gpgsig' or 'gpgsig-sha256' continuation line\");\n+\t\t\t\tgoto done;\n+\t\t\t}\n+\t\t\tbuffer = eol + 1;\n+\t\t}\n+\t}\n+\n \tif (buffer < buffer_end && !starts_with(buffer, \"\\n\")) {\n \t\t/*\n \t\t * The verify_headers() check will allow\ndiff --git a/fsck.h b/fsck.h\nindex dd7df3d5b3..c26616d7eb 100644\n--- a/fsck.h\n+++ b/fsck.h\n@@ -25,9 +25,11 @@ enum fsck_msg_type {\n \tFUNC(NUL_IN_HEADER, FATAL) \\\n \tFUNC(UNTERMINATED_HEADER, FATAL) \\\n \t/* errors */ \\\n+\tFUNC(BAD_HEADER_CONTINUATION, ERROR) \\\n \tFUNC(BAD_DATE, ERROR) \\\n \tFUNC(BAD_DATE_OVERFLOW, ERROR) \\\n \tFUNC(BAD_EMAIL, ERROR) \\\n+\tFUNC(BAD_GPGSIG, ERROR) \\\n \tFUNC(BAD_NAME, ERROR) \\\n \tFUNC(BAD_OBJECT_SHA1, ERROR) \\\n \tFUNC(BAD_PACKED_REF_ENTRY, ERROR) \\\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex 5ae86c42be..c4b651c2dc 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -454,6 +454,60 @@ test_expect_success 'tag with NUL in header' '\n \ttest_grep \"error in tag $tag.*unterminated header: NUL at offset\" out\n '\n \n+test_expect_success 'tag accepts gpgsig header even if not validly signed' '\n+\ttest_oid_cache <<-\\EOF &&\n+\theader sha1:gpgsig-sha256\n+\theader sha256:gpgsig\n+\tEOF\n+\theader=$(test_oid header) &&\n+\tsha=$(git rev-parse HEAD) &&\n+\tcat >good-tag <<-EOF &&\n+\tobject $sha\n+\ttype commit\n+\ttag good\n+\ttagger T A Gger 1234567890 -0000\n+\t$header -----BEGIN PGP SIGNATURE-----\n+\t Not a valid signature\n+\t -----END PGP SIGNATURE-----\n+\n+\tThis is a good tag.\n+\tEOF\n+\n+\ttag=$(git hash-object --literally -t tag -w --stdin bad-tag <<-EOF &&\n+\tobject $sha\n+\ttype commit\n+\ttag good\n+\ttagger T A Gger 1234567890 -0000\n+\t$header -----BEGIN PGP SIGNATURE-----\n+\t Not a valid signature\n+\t -----END PGP SIGNATURE-----\n+\tjunk\n+\n+\tThis is a bad tag with junk at the end of the headers.\n+\tEOF\n+\n+\ttag=$(git hash-object --literally -t tag -w --stdin out &&\n+\ttest_grep \"error in tag $tag.*invalid format - extra header\" out\n+'\n+\n test_expect_success 'cleaned up' '\n \tgit fsck >actual 2>&1 &&\n \ttest_must_be_empty actual\n"},{"id":"528421","messageId":"20251009215626.3089287-10-sandals@crustytoothpaste.net","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"[PATCH v3 9/9] t1010: use BROKEN_OBJECTS prerequisite","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-09T21:56:26Z","receivedAt":"2025-10-09T21:56:39Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"When hash compatibility mode is enabled, we cannot write broken objects\nbecause they cannot be mapped into the other hash algorithm. Use the\nBROKEN_OBJECTS prerequisite to disable these tests and the writing of\nbroken objects in this mode.\n\nSigned-off-by: brian m. carlson \n---\n t/t1010-mktree.sh | 13 ++++++++-----\n 1 file changed, 8 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t1010-mktree.sh b/t/t1010-mktree.sh\nindex e9973f7494..312fe6717a 100755\n--- a/t/t1010-mktree.sh\n+++ b/t/t1010-mktree.sh\n@@ -11,10 +11,13 @@ test_expect_success setup '\n \t\tgit add \"$d\" || return 1\n \tdone &&\n \techo zero >one &&\n-\tgit update-index --add --info-only one &&\n-\tgit write-tree --missing-ok >tree.missing &&\n-\tgit ls-tree $(cat tree.missing) >top.missing &&\n-\tgit ls-tree -r $(cat tree.missing) >all.missing &&\n+\tif test_have_prereq BROKEN_OBJECTS\n+\tthen\n+\t\tgit update-index --add --info-only one &&\n+\t\tgit write-tree --missing-ok >tree.missing &&\n+\t\tgit ls-tree $(cat tree.missing) >top.missing &&\n+\t\tgit ls-tree -r $(cat tree.missing) >all.missing\n+\tfi &&\n \techo one >one &&\n \tgit add one &&\n \tgit write-tree >tree &&\n@@ -53,7 +56,7 @@ test_expect_success 'ls-tree output in wrong order given to mktree (2)' '\n \ttest_cmp tree.withsub actual\n '\n \n-test_expect_success 'allow missing object with --missing' '\n+test_expect_success BROKEN_OBJECTS 'allow missing object with --missing' '\n \tgit mktree --missing actual &&\n \ttest_cmp tree.missing actual\n '\n"},{"id":"528632","messageId":"xmqqzf9u3jug.fsf@gitster.g","threadId":"64168","inReplyTo":"20251009215626.3089287-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH v3 0/9] SHA-1/SHA-256 interoperability, part 1","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-13T15:24:55Z","receivedAt":"2025-10-13T15:24:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> brian m. carlson (9):\n> docs: update pack index v3 format\n> docs: update offset order for pack index v3\n> docs: reflect actual double signature for tags\n> docs: improve ambiguous areas of pack format documentation\n> docs: add documentation for loose objects\n> rev-parse: allow printing compatibility hash\n> fsck: consider gpgsig headers expected in tags\n> t: allow specifying compatibility hash\n> t1010: use BROKEN_OBJECTS prerequisite\n\nThe topic has been quiet ovre the weekend. Shall I mark it for\n'next' now?\n"},{"id":"528639","messageId":"aO0qJwFQfSXsi7lA@fruit.crustytoothpaste.net","threadId":"64168","inReplyTo":"xmqqzf9u3jug.fsf@gitster.g","subject":"Re: [PATCH v3 0/9] SHA-1/SHA-256 interoperability, part 1","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-10-13T16:34:47Z","receivedAt":"2025-10-13T16:42:41Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-10-13 at 15:24:55, Junio C Hamano wrote:\n> \"brian m. carlson\" writes:\n> \n> > brian m. carlson (9):\n> > docs: update pack index v3 format\n> > docs: update offset order for pack index v3\n> > docs: reflect actual double signature for tags\n> > docs: improve ambiguous areas of pack format documentation\n> > docs: add documentation for loose objects\n> > rev-parse: allow printing compatibility hash\n> > fsck: consider gpgsig headers expected in tags\n> > t: allow specifying compatibility hash\n> > t1010: use BROKEN_OBJECTS prerequisite\n> \n> The topic has been quiet ovre the weekend. Shall I mark it for\n> 'next' now?\n\nYes, I think it's ready. The only difference between v2 and v3 was your\ncomment on the text and there were no other comments.\n\nOf course, if Patrick or anyone else would like more time to review, I'm\nhappy to wait.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"528686","messageId":"aO3ld4kz4Ru-OP7r@pks.im","threadId":"64168","inReplyTo":"aO0qJwFQfSXsi7lA@fruit.crustytoothpaste.net","subject":"Re: [PATCH v3 0/9] SHA-1/SHA-256 interoperability, part 1","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-10-14T05:53:59Z","receivedAt":"2025-10-14T05:54:09Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 13, 2025 at 04:34:47PM +0000, brian m. carlson wrote:\n> On 2025-10-13 at 15:24:55, Junio C Hamano wrote:\n> > \"brian m. carlson\" writes:\n> > \n> > > brian m. carlson (9):\n> > > docs: update pack index v3 format\n> > > docs: update offset order for pack index v3\n> > > docs: reflect actual double signature for tags\n> > > docs: improve ambiguous areas of pack format documentation\n> > > docs: add documentation for loose objects\n> > > rev-parse: allow printing compatibility hash\n> > > fsck: consider gpgsig headers expected in tags\n> > > t: allow specifying compatibility hash\n> > > t1010: use BROKEN_OBJECTS prerequisite\n> > \n> > The topic has been quiet ovre the weekend. Shall I mark it for\n> > 'next' now?\n> \n> Yes, I think it's ready. The only difference between v2 and v3 was your\n> comment on the text and there were no other comments.\n> \n> Of course, if Patrick or anyone else would like more time to review, I'm\n> happy to wait.\n\nI just had another read through the series and am happy with this\nversion. Thanks!\n\nPatrick\n"}]}