{"thread":{"id":"50474","subject":"[PATCH 00/31] Hash function transition part 16","startedAt":"2019-02-12T01:23:16Z","lastAt":"2019-02-18T21:00:39Z","messageCount":56,"participants":["brian m. carlson","Eric Sunshine","Jeff King","René Scharfe","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":31},"messages":[{"id":"369080","messageId":"20190212012256.1005924-1-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":null,"subject":"[PATCH 00/31] Hash function transition part 16","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:25Z","receivedAt":"2019-02-12T01:23:16Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This is the sixteenth series of hash function transition patches. This\nseries contains various fixes, mostly focused around the pack bitmap\ncode, the HTTP code, the archive code, the index, and parts of our Perl\ncode.\n\nThis is the second to last series required for a \"stage 0\" Git; that is,\none that can operate only with SHA-256, but not SHA-1. Observers will\nnotice a focus on getting rid of sha1_to_hex and null_sha1 as well as\nthe normal types of transforms; the next series will remove both of\nthese.\n\nThis series modifies the index code such that it can work with a hash\nalgorithm of any length. In order to do so, the structs involved were\nchanged to use flex array members and not store the hash in a fixed\narray member. This design was chosen over a multiple struct approach\nbecause it ensures that we have one consistent, well-tested code path\nthat works for both algorithms, as well as any algorithms in the future.\nComments on the approach or arguments for other designs are welcome.\n\nThis is a rather long series, but most of it is concentrated in a few\nsmall areas, so hopefully it's a little easier to review because of\nthat.\n\nTo preview the series that come after this, there is an additional\nseries for stage 0 Git (object-id-part17 plus part of sha256-fixes),\nplus potentially several series of test fixes (test-fixes-part4 and part\nof sha256-fixes). Following that, I plan to introduce, under the\nDEVELOPER Makefile flag, the actual code which supports\nextensions.objectFormat and makes it so that it works\n(transition-stage-4).\n\nbrian m. carlson (31):\n  t/lib-submodule-update: use appropriate length constant\n  pack-bitmap: make bitmap header handling hash agnostic\n  pack-bitmap: convert struct stored_bitmap to object_id\n  pack-bitmap: replace sha1_to_hex\n  pack-bitmap: switch hard-coded constants to the_hash_algo\n  submodule: avoid hard-coded constants\n  notes-merge: switch to use the_hash_algo\n  notes: make hash size independent\n  notes: replace sha1_to_hex\n  object-store: rename and expand packed_git's sha1 member\n  builtin/name-rev: make hash-size independent\n  fast-import: make hash-size independent\n  fast-import: replace sha1_to_hex\n  builtin/am: make hash size independent\n  builtin/pull: make hash-size independent\n  http-push: convert to use the_hash_algo\n  http-backend: allow 64-character hex names\n  http-push: remove remaining uses of sha1_to_hex\n  http-walker: replace sha1_to_hex\n  http: replace hard-coded constant with the_hash_algo\n  http: compute hash of downloaded objects using the_hash_algo\n  http: replace sha1_to_hex\n  remote-curl: make hash size independent\n  archive-tar: make hash size independent\n  archive: convert struct archiver_args to object_id\n  refspec: make hash size independent\n  builtin/difftool: use parse_oid_hex\n  dir: make untracked cache extension hash size independent\n  read-cache: read data in a hash-independent way\n  Git.pm: make hash size independent\n  gitweb: make hash size independent\n\n archive-tar.c               |  7 ++--\n archive-zip.c               | 10 ++---\n archive.c                   |  8 ++--\n archive.h                   |  2 +-\n builtin/am.c                |  9 +++--\n builtin/difftool.c          |  6 +--\n builtin/get-tar-commit-id.c | 11 +++++-\n builtin/name-rev.c          | 14 ++++---\n builtin/pack-redundant.c    |  2 +-\n builtin/pull.c              |  5 ++-\n dir.c                       | 28 +++++++-------\n fast-import.c               | 48 +++++++++++++-----------\n gitweb/gitweb.perl          | 63 ++++++++++++++++---------------\n http-backend.c              |  3 ++\n http-push.c                 | 29 ++++++++-------\n http-walker.c               | 18 ++++-----\n http.c                      | 33 +++++++++--------\n http.h                      |  2 +-\n merge-recursive.c           |  2 +-\n notes-merge.c               |  6 +--\n notes.c                     | 44 +++++++++++-----------\n object-store.h              |  2 +-\n pack-bitmap-write.c         |  8 ++--\n pack-bitmap.c               | 20 +++++-----\n pack-bitmap.h               |  2 +-\n packfile.c                  |  6 +--\n perl/Git.pm                 |  2 +-\n read-cache.c                | 74 +++++++++++++++----------------------\n refspec.c                   |  2 +-\n remote-curl.c               | 11 +++---\n submodule.c                 |  2 +-\n t/lib-submodule-update.sh   |  3 +-\n 32 files changed, 246 insertions(+), 236 deletions(-)\n\n"},{"id":"369081","messageId":"20190212012256.1005924-2-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 01/31] t/lib-submodule-update: use appropriate length constant","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:26Z","receivedAt":"2019-02-12T01:23:18Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Instead of using a specific invalid hard-coded object ID, produce one\nof the appropriate length by using test_oid.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n t/lib-submodule-update.sh | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5b56b23166..1dd17fc03e 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -139,7 +139,7 @@ create_lib_submodule_repo () {\n \t\tgit revert HEAD &&\n \n \t\tgit checkout -b invalid_sub1 add_sub1 &&\n-\t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n+\t\tgit update-index --cacheinfo 160000 $(test_oid numeric) sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n@@ -196,6 +196,7 @@ test_git_directory_exists() {\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n prolog () {\n+\ttest_oid_init &&\n \t(test -d submodule_update_repo || create_lib_submodule_repo) &&\n \ttest_config_global diff.ignoreSubmodules all &&\n \ttest_config diff.ignoreSubmodules all\n"},{"id":"369082","messageId":"20190212012256.1005924-4-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 03/31] pack-bitmap: convert struct stored_bitmap to object_id","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:28Z","receivedAt":"2019-02-12T01:23:20Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Convert struct stored_bitmap to use struct object_id.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n pack-bitmap.c | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex b53f37243c..c760913cea 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -18,7 +18,7 @@\n  * commit.\n  */\n struct stored_bitmap {\n-\tunsigned char sha1[20];\n+\tstruct object_id oid;\n \tstruct ewah_bitmap *root;\n \tstruct stored_bitmap *xor;\n \tint flags;\n@@ -181,9 +181,9 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \tstored->root = root;\n \tstored->xor = xor_with;\n \tstored->flags = flags;\n-\thashcpy(stored->sha1, sha1);\n+\toidread(&stored->oid, sha1);\n \n-\thash_pos = kh_put_sha1(index->bitmaps, stored->sha1, &ret);\n+\thash_pos = kh_put_sha1(index->bitmaps, stored->oid.hash, &ret);\n \n \t/* a 0 return code means the insertion succeeded with no changes,\n \t * because the SHA1 already existed on the map. this is bad, there\n@@ -1080,7 +1080,7 @@ int rebuild_existing_bitmaps(struct bitmap_index *bitmap_git,\n \t\t\t\t\t    lookup_stored_bitmap(stored),\n \t\t\t\t\t    rebuild)) {\n \t\t\t\thash_pos = kh_put_sha1(reused_bitmaps,\n-\t\t\t\t\t\t       stored->sha1,\n+\t\t\t\t\t\t       stored->oid.hash,\n \t\t\t\t\t\t       &hash_ret);\n \t\t\t\tkh_value(reused_bitmaps, hash_pos) =\n \t\t\t\t\tbitmap_to_ewah(rebuild);\n"},{"id":"369083","messageId":"20190212012256.1005924-3-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 02/31] pack-bitmap: make bitmap header handling hash agnostic","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:27Z","receivedAt":"2019-02-12T01:23:20Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Increase the checksum field in struct bitmap_disk_header to be\nGIT_MAX_RAWSZ bytes in length and ensure that we hash the proper number\nof bytes out when computing the bitmap checksum.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n pack-bitmap-write.c | 2 +-\n pack-bitmap.c       | 2 +-\n pack-bitmap.h       | 2 +-\n 3 files changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/pack-bitmap-write.c b/pack-bitmap-write.c\nindex 5566e94abe..c82fb01fd7 100644\n--- a/pack-bitmap-write.c\n+++ b/pack-bitmap-write.c\n@@ -535,7 +535,7 @@ void bitmap_writer_finish(struct pack_idx_entry **index,\n \theader.entry_count = htonl(writer.selected_nr);\n \thashcpy(header.checksum, writer.pack_checksum);\n \n-\thashwrite(f, &header, sizeof(header));\n+\thashwrite(f, &header, sizeof(header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz);\n \tdump_bitmap(f, writer.commits);\n \tdump_bitmap(f, writer.trees);\n \tdump_bitmap(f, writer.blobs);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 4695aaf6b4..b53f37243c 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -163,7 +163,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \t}\n \n \tindex->entry_count = ntohl(header->entry_count);\n-\tindex->map_pos += sizeof(*header);\n+\tindex->map_pos += sizeof(*header) - GIT_MAX_RAWSZ + the_hash_algo->rawsz;\n \treturn 0;\n }\n \ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 8418ba8c79..344ba23af9 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -14,7 +14,7 @@ struct bitmap_disk_header {\n \tuint16_t version;\n \tuint16_t options;\n \tuint32_t entry_count;\n-\tunsigned char checksum[20];\n+\tunsigned char checksum[GIT_MAX_RAWSZ];\n };\n \n static const char BITMAP_IDX_SIGNATURE[] = {'B', 'I', 'T', 'M'};\n"},{"id":"369084","messageId":"20190212012256.1005924-6-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 05/31] pack-bitmap: switch hard-coded constants to the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:30Z","receivedAt":"2019-02-12T01:23:23Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Switch two hard-coded uses of 20 to references to the_hash_algo.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n pack-bitmap.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 6d6fa68563..603492c237 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -138,7 +138,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n {\n \tstruct bitmap_disk_header *header = (void *)index->map;\n \n-\tif (index->map_size < sizeof(*header) + 20)\n+\tif (index->map_size < sizeof(*header) + the_hash_algo->rawsz)\n \t\treturn error(\"Corrupted bitmap index (missing header data)\");\n \n \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n@@ -157,7 +157,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n \n \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n-\t\t\tunsigned char *end = index->map + index->map_size - 20;\n+\t\t\tunsigned char *end = index->map + index->map_size - the_hash_algo->rawsz;\n \t\t\tindex->hashes = ((uint32_t *)end) - index->pack->num_objects;\n \t\t}\n \t}\n"},{"id":"369085","messageId":"20190212012256.1005924-5-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 04/31] pack-bitmap: replace sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:29Z","receivedAt":"2019-02-12T01:23:24Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Replace the uses of sha1_to_hex in the pack bitmap code with hash_to_hex\nto allow the use of SHA-256 as well.  Rename a few variables since they\nare no longer limited to SHA-1.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n pack-bitmap-write.c | 6 +++---\n pack-bitmap.c       | 8 ++++----\n 2 files changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap-write.c b/pack-bitmap-write.c\nindex c82fb01fd7..802ed62677 100644\n--- a/pack-bitmap-write.c\n+++ b/pack-bitmap-write.c\n@@ -142,13 +142,13 @@ static inline void reset_all_seen(void)\n \tseen_objects_nr = 0;\n }\n \n-static uint32_t find_object_pos(const unsigned char *sha1)\n+static uint32_t find_object_pos(const unsigned char *hash)\n {\n-\tstruct object_entry *entry = packlist_find(writer.to_pack, sha1, NULL);\n+\tstruct object_entry *entry = packlist_find(writer.to_pack, hash, NULL);\n \n \tif (!entry) {\n \t\tdie(\"Failed to write bitmap index. Packfile doesn't have full closure \"\n-\t\t\t\"(object %s is missing)\", sha1_to_hex(sha1));\n+\t\t\t\"(object %s is missing)\", hash_to_hex(hash));\n \t}\n \n \treturn oe_in_pack_pos(writer.to_pack, entry);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex c760913cea..6d6fa68563 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -169,7 +169,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n \n static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t\t\t\t\t  struct ewah_bitmap *root,\n-\t\t\t\t\t  const unsigned char *sha1,\n+\t\t\t\t\t  const unsigned char *hash,\n \t\t\t\t\t  struct stored_bitmap *xor_with,\n \t\t\t\t\t  int flags)\n {\n@@ -181,7 +181,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \tstored->root = root;\n \tstored->xor = xor_with;\n \tstored->flags = flags;\n-\toidread(&stored->oid, sha1);\n+\toidread(&stored->oid, hash);\n \n \thash_pos = kh_put_sha1(index->bitmaps, stored->oid.hash, &ret);\n \n@@ -189,7 +189,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n \t * because the SHA1 already existed on the map. this is bad, there\n \t * shouldn't be duplicated commits in the index */\n \tif (ret == 0) {\n-\t\terror(\"Duplicate entry in bitmap index: %s\", sha1_to_hex(sha1));\n+\t\terror(\"Duplicate entry in bitmap index: %s\", hash_to_hex(hash));\n \t\treturn NULL;\n \t}\n \n@@ -805,7 +805,7 @@ int reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,\n \n \t\tfprintf(stderr, \"Failed to reuse at %d (%016llx)\\n\",\n \t\t\treuse_objects, result->words[i]);\n-\t\tfprintf(stderr, \" %s\\n\", sha1_to_hex(sha1));\n+\t\tfprintf(stderr, \" %s\\n\", hash_to_hex(sha1));\n \t}\n #endif\n \n"},{"id":"369086","messageId":"20190212012256.1005924-9-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 08/31] notes: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:33Z","receivedAt":"2019-02-12T01:23:25Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Switch out various uses of the GIT_SHA1_* constants with GIT_MAX_*\nconstants for allocations and the_hash_algo for general parsing.  Update\na comment to no longer be SHA-1 specific.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n notes.c | 32 +++++++++++++++++---------------\n 1 file changed, 17 insertions(+), 15 deletions(-)\n\ndiff --git a/notes.c b/notes.c\nindex 7f7cc4d511..96647f1d2d 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -67,8 +67,9 @@ struct non_note {\n \n #define GET_NIBBLE(n, sha1) ((((sha1)[(n) >> 1]) >> ((~(n) & 0x01) << 2)) & 0x0f)\n \n-#define KEY_INDEX (GIT_SHA1_RAWSZ - 1)\n-#define FANOUT_PATH_SEPARATORS ((GIT_SHA1_HEXSZ / 2) - 1)\n+#define KEY_INDEX (the_hash_algo->rawsz - 1)\n+#define FANOUT_PATH_SEPARATORS (the_hash_algo->rawsz - 1)\n+#define FANOUT_PATH_SEPARATORS_MAX ((GIT_MAX_HEXSZ / 2) - 1)\n #define SUBTREE_SHA1_PREFIXCMP(key_sha1, subtree_sha1) \\\n \t(memcmp(key_sha1, subtree_sha1, subtree_sha1[KEY_INDEX]))\n \n@@ -198,7 +199,7 @@ static void note_tree_remove(struct notes_tree *t,\n \t\tstruct leaf_node *entry)\n {\n \tstruct leaf_node *l;\n-\tstruct int_node *parent_stack[GIT_SHA1_RAWSZ];\n+\tstruct int_node *parent_stack[GIT_MAX_RAWSZ];\n \tunsigned char i, j;\n \tvoid **p = note_tree_search(t, &tree, &n, entry->key_oid.hash);\n \n@@ -394,6 +395,7 @@ static void load_subtree(struct notes_tree *t, struct leaf_node *subtree,\n \tvoid *buf;\n \tstruct tree_desc desc;\n \tstruct name_entry entry;\n+\tconst unsigned hashsz = the_hash_algo->rawsz;\n \n \tbuf = fill_tree_descriptor(&desc, &subtree->val_oid);\n \tif (!buf)\n@@ -401,7 +403,7 @@ static void load_subtree(struct notes_tree *t, struct leaf_node *subtree,\n \t\t     oid_to_hex(&subtree->val_oid));\n \n \tprefix_len = subtree->key_oid.hash[KEY_INDEX];\n-\tif (prefix_len >= GIT_SHA1_RAWSZ)\n+\tif (prefix_len >= hashsz)\n \t\tBUG(\"prefix_len (%\"PRIuMAX\") is out of range\", (uintmax_t)prefix_len);\n \tif (prefix_len * 2 < n)\n \t\tBUG(\"prefix_len (%\"PRIuMAX\") is too small\", (uintmax_t)prefix_len);\n@@ -411,7 +413,7 @@ static void load_subtree(struct notes_tree *t, struct leaf_node *subtree,\n \t\tstruct leaf_node *l;\n \t\tsize_t path_len = strlen(entry.path);\n \n-\t\tif (path_len == 2 * (GIT_SHA1_RAWSZ - prefix_len)) {\n+\t\tif (path_len == 2 * (hashsz - prefix_len)) {\n \t\t\t/* This is potentially the remainder of the SHA-1 */\n \n \t\t\tif (!S_ISREG(entry.mode))\n@@ -419,7 +421,7 @@ static void load_subtree(struct notes_tree *t, struct leaf_node *subtree,\n \t\t\t\tgoto handle_non_note;\n \n \t\t\tif (hex_to_bytes(object_oid.hash + prefix_len, entry.path,\n-\t\t\t\t\t GIT_SHA1_RAWSZ - prefix_len))\n+\t\t\t\t\t hashsz - prefix_len))\n \t\t\t\tgoto handle_non_note; /* entry.path is not a SHA1 */\n \n \t\t\ttype = PTR_TYPE_NOTE;\n@@ -439,7 +441,7 @@ static void load_subtree(struct notes_tree *t, struct leaf_node *subtree,\n \t\t\t * except for the last byte, where we write\n \t\t\t * the length:\n \t\t\t */\n-\t\t\tmemset(object_oid.hash + len, 0, GIT_SHA1_RAWSZ - len - 1);\n+\t\t\tmemset(object_oid.hash + len, 0, hashsz - len - 1);\n \t\t\tobject_oid.hash[KEY_INDEX] = (unsigned char)len;\n \n \t\t\ttype = PTR_TYPE_SUBTREE;\n@@ -527,15 +529,15 @@ static unsigned char determine_fanout(struct int_node *tree, unsigned char n,\n \treturn fanout + 1;\n }\n \n-/* hex SHA1 + 19 * '/' + NUL */\n-#define FANOUT_PATH_MAX GIT_SHA1_HEXSZ + FANOUT_PATH_SEPARATORS + 1\n+/* hex oid + one slash between each pair + NUL */\n+#define FANOUT_PATH_MAX GIT_MAX_HEXSZ + FANOUT_PATH_SEPARATORS_MAX + 1\n \n static void construct_path_with_fanout(const unsigned char *sha1,\n \t\tunsigned char fanout, char *path)\n {\n \tunsigned int i = 0, j = 0;\n \tconst char *hex_sha1 = sha1_to_hex(sha1);\n-\tassert(fanout < GIT_SHA1_RAWSZ);\n+\tassert(fanout < the_hash_algo->rawsz);\n \twhile (fanout) {\n \t\tpath[i++] = hex_sha1[j++];\n \t\tpath[i++] = hex_sha1[j++];\n@@ -637,10 +639,10 @@ static inline int matches_tree_write_stack(struct tree_write_stack *tws,\n \n static void write_tree_entry(struct strbuf *buf, unsigned int mode,\n \t\tconst char *path, unsigned int path_len, const\n-\t\tunsigned char *sha1)\n+\t\tunsigned char *hash)\n {\n \tstrbuf_addf(buf, \"%o %.*s%c\", mode, path_len, path, '\\0');\n-\tstrbuf_add(buf, sha1, GIT_SHA1_RAWSZ);\n+\tstrbuf_add(buf, hash, the_hash_algo->rawsz);\n }\n \n static void tree_write_stack_init_subtree(struct tree_write_stack *tws,\n@@ -652,7 +654,7 @@ static void tree_write_stack_init_subtree(struct tree_write_stack *tws,\n \tn = (struct tree_write_stack *)\n \t\txmalloc(sizeof(struct tree_write_stack));\n \tn->next = NULL;\n-\tstrbuf_init(&n->buf, 256 * (32 + GIT_SHA1_HEXSZ)); /* assume 256 entries per tree */\n+\tstrbuf_init(&n->buf, 256 * (32 + the_hash_algo->hexsz)); /* assume 256 entries per tree */\n \tn->path[0] = n->path[1] = '\\0';\n \ttws->next = n;\n \ttws->path[0] = path[0];\n@@ -757,7 +759,7 @@ static int write_each_note(const struct object_id *object_oid,\n \t\tnote_path[note_path_len] = '\\0';\n \t\tmode = 040000;\n \t}\n-\tassert(note_path_len <= GIT_SHA1_HEXSZ + FANOUT_PATH_SEPARATORS);\n+\tassert(note_path_len <= GIT_MAX_HEXSZ + FANOUT_PATH_SEPARATORS);\n \n \t/* Weave non-note entries into note entries */\n \treturn  write_each_non_note_until(note_path, d) ||\n@@ -1137,7 +1139,7 @@ int write_notes_tree(struct notes_tree *t, struct object_id *result)\n \n \t/* Prepare for traversal of current notes tree */\n \troot.next = NULL; /* last forward entry in list is grounded */\n-\tstrbuf_init(&root.buf, 256 * (32 + GIT_SHA1_HEXSZ)); /* assume 256 entries */\n+\tstrbuf_init(&root.buf, 256 * (32 + the_hash_algo->hexsz)); /* assume 256 entries */\n \troot.path[0] = root.path[1] = '\\0';\n \tcb_data.root = &root;\n \tcb_data.next_non_note = t->first_non_note;\n"},{"id":"369087","messageId":"20190212012256.1005924-10-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 09/31] notes: replace sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:34Z","receivedAt":"2019-02-12T01:23:26Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Replace the uses of sha1_to_hex in this function with hash_to_hex to\nallow the use of SHA-256 as well.  Rename some variables since this code\nis no longer limited to SHA-1.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n notes.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/notes.c b/notes.c\nindex 96647f1d2d..a9a3ddd6d7 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -532,19 +532,19 @@ static unsigned char determine_fanout(struct int_node *tree, unsigned char n,\n /* hex oid + one slash between each pair + NUL */\n #define FANOUT_PATH_MAX GIT_MAX_HEXSZ + FANOUT_PATH_SEPARATORS_MAX + 1\n \n-static void construct_path_with_fanout(const unsigned char *sha1,\n+static void construct_path_with_fanout(const unsigned char *hash,\n \t\tunsigned char fanout, char *path)\n {\n \tunsigned int i = 0, j = 0;\n-\tconst char *hex_sha1 = sha1_to_hex(sha1);\n+\tconst char *hex_hash = hash_to_hex(hash);\n \tassert(fanout < the_hash_algo->rawsz);\n \twhile (fanout) {\n-\t\tpath[i++] = hex_sha1[j++];\n-\t\tpath[i++] = hex_sha1[j++];\n+\t\tpath[i++] = hex_hash[j++];\n+\t\tpath[i++] = hex_hash[j++];\n \t\tpath[i++] = '/';\n \t\tfanout--;\n \t}\n-\txsnprintf(path + i, FANOUT_PATH_MAX - i, \"%s\", hex_sha1 + j);\n+\txsnprintf(path + i, FANOUT_PATH_MAX - i, \"%s\", hex_hash + j);\n }\n \n static int for_each_note_helper(struct notes_tree *t, struct int_node *tree,\n@@ -1167,7 +1167,7 @@ void prune_notes(struct notes_tree *t, int flags)\n \n \twhile (l) {\n \t\tif (flags & NOTES_PRUNE_VERBOSE)\n-\t\t\tprintf(\"%s\\n\", sha1_to_hex(l->sha1));\n+\t\t\tprintf(\"%s\\n\", hash_to_hex(l->sha1));\n \t\tif (!(flags & NOTES_PRUNE_DRYRUN))\n \t\t\tremove_note(t, l->sha1);\n \t\tl = l->next;\n"},{"id":"369088","messageId":"20190212012256.1005924-12-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 11/31] builtin/name-rev: make hash-size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:36Z","receivedAt":"2019-02-12T01:23:28Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Use the_hash_algo when parsing instead of GIT_SHA1_HEXSZ so that this\nfunction works with any size hash.  Rename the variable forty to\ncounter, as this is a better name and is independent of the hash size.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n builtin/name-rev.c | 14 ++++++++------\n 1 file changed, 8 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/name-rev.c b/builtin/name-rev.c\nindex f1cb45c227..05ccf53e00 100644\n--- a/builtin/name-rev.c\n+++ b/builtin/name-rev.c\n@@ -361,23 +361,25 @@ static char const * const name_rev_usage[] = {\n static void name_rev_line(char *p, struct name_ref_data *data)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tint forty = 0;\n+\tint counter = 0;\n \tchar *p_start;\n+\tconst unsigned hexsz = the_hash_algo->hexsz;\n+\n \tfor (p_start = p; *p; p++) {\n #define ishex(x) (isdigit((x)) || ((x) >= 'a' && (x) <= 'f'))\n \t\tif (!ishex(*p))\n-\t\t\tforty = 0;\n-\t\telse if (++forty == GIT_SHA1_HEXSZ &&\n+\t\t\tcounter = 0;\n+\t\telse if (++counter == hexsz &&\n \t\t\t !ishex(*(p+1))) {\n \t\t\tstruct object_id oid;\n \t\t\tconst char *name = NULL;\n \t\t\tchar c = *(p+1);\n \t\t\tint p_len = p - p_start + 1;\n \n-\t\t\tforty = 0;\n+\t\t\tcounter = 0;\n \n \t\t\t*(p+1) = 0;\n-\t\t\tif (!get_oid(p - (GIT_SHA1_HEXSZ - 1), &oid)) {\n+\t\t\tif (!get_oid(p - (hexsz - 1), &oid)) {\n \t\t\t\tstruct object *o =\n \t\t\t\t\tlookup_object(the_repository,\n \t\t\t\t\t\t      oid.hash);\n@@ -390,7 +392,7 @@ static void name_rev_line(char *p, struct name_ref_data *data)\n \t\t\t\tcontinue;\n \n \t\t\tif (data->name_only)\n-\t\t\t\tprintf(\"%.*s%s\", p_len - GIT_SHA1_HEXSZ, p_start, name);\n+\t\t\t\tprintf(\"%.*s%s\", p_len - hexsz, p_start, name);\n \t\t\telse\n \t\t\t\tprintf(\"%.*s (%s)\", p_len, p_start, name);\n \t\t\tp_start = p + 1;\n"},{"id":"369089","messageId":"20190212012256.1005924-7-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 06/31] submodule: avoid hard-coded constants","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:31Z","receivedAt":"2019-02-12T01:23:28Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Instead of using hard-coded 40-based constants, express these values in\nterms of the_hash_algo and GIT_MAX_HEXSZ.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n merge-recursive.c | 2 +-\n submodule.c       | 2 +-\n 2 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/merge-recursive.c b/merge-recursive.c\nindex 4851825aeb..b8bd5d4f8d 100644\n--- a/merge-recursive.c\n+++ b/merge-recursive.c\n@@ -1122,7 +1122,7 @@ static int find_first_merges(struct repository *repo,\n \tstruct commit *commit;\n \tint contains_another;\n \n-\tchar merged_revision[42];\n+\tchar merged_revision[GIT_MAX_HEXSZ + 2];\n \tconst char *rev_args[] = { \"rev-list\", \"--merges\", \"--ancestry-path\",\n \t\t\t\t   \"--all\", merged_revision, NULL };\n \tstruct rev_info revs;\ndiff --git a/submodule.c b/submodule.c\nindex 934ecfa294..0e2faaa41b 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -990,7 +990,7 @@ static int submodule_needs_pushing(struct repository *r,\n \t\tif (start_command(&cp))\n \t\t\tdie(\"Could not run 'git rev-list <commits> --not --remotes -n 1' command in submodule %s\",\n \t\t\t\t\tpath);\n-\t\tif (strbuf_read(&buf, cp.out, 41))\n+\t\tif (strbuf_read(&buf, cp.out, the_hash_algo->hexsz + 1))\n \t\t\tneeds_pushing = 1;\n \t\tfinish_command(&cp);\n \t\tclose(cp.out);\n"},{"id":"369091","messageId":"20190212012256.1005924-11-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 10/31] object-store: rename and expand packed_git's sha1 member","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:35Z","receivedAt":"2019-02-12T01:23:31Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"This member is used to represent the pack checksum of the pack in\nquestion.  Expand this member to be GIT_MAX_RAWSZ bytes in length so it\nworks with longer hashes and rename it to be \"hash\" instead of \"sha1\".\nThis transformation was made with a change to the definition and the\nfollowing semantic patch:\n\n@@\nstruct packed_git *E1;\n@@\n- E1->sha1\n+ E1->hash\n\n@@\nstruct packed_git E1;\n@@\n- E1.sha1\n+ E1.hash\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n builtin/pack-redundant.c |  2 +-\n fast-import.c            | 17 +++++++++--------\n http-push.c              |  3 ++-\n http-walker.c            |  2 +-\n http.c                   | 13 +++++++------\n object-store.h           |  2 +-\n packfile.c               |  6 +++---\n 7 files changed, 24 insertions(+), 21 deletions(-)\n\ndiff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\nindex 11bc514566..d9af7f5414 100644\n--- a/builtin/pack-redundant.c\n+++ b/builtin/pack-redundant.c\n@@ -689,7 +689,7 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)\n \tpl = red = pack_list_difference(local_packs, min);\n \twhile (pl) {\n \t\tprintf(\"%s\\n%s\\n\",\n-\t\t       sha1_pack_index_name(pl->pack->sha1),\n+\t\t       sha1_pack_index_name(pl->pack->hash),\n \t\t       pl->pack->pack_name);\n \t\tpl = pl->next;\n \t}\ndiff --git a/fast-import.c b/fast-import.c\nindex b7ba755c2b..7c9a10a77b 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -742,7 +742,8 @@ static const char *create_index(void)\n \tif (c != last)\n \t\tdie(\"internal consistency error creating the index\");\n \n-\ttmpfile = write_idx_file(NULL, idx, object_count, &pack_idx_opts, pack_data->sha1);\n+\ttmpfile = write_idx_file(NULL, idx, object_count, &pack_idx_opts,\n+\t\t\t\t pack_data->hash);\n \tfree(idx);\n \treturn tmpfile;\n }\n@@ -753,7 +754,7 @@ static char *keep_pack(const char *curr_index_name)\n \tstruct strbuf name = STRBUF_INIT;\n \tint keep_fd;\n \n-\todb_pack_name(&name, pack_data->sha1, \"keep\");\n+\todb_pack_name(&name, pack_data->hash, \"keep\");\n \tkeep_fd = odb_pack_keep(name.buf);\n \tif (keep_fd < 0)\n \t\tdie_errno(\"cannot create keep file\");\n@@ -761,11 +762,11 @@ static char *keep_pack(const char *curr_index_name)\n \tif (close(keep_fd))\n \t\tdie_errno(\"failed to write keep file\");\n \n-\todb_pack_name(&name, pack_data->sha1, \"pack\");\n+\todb_pack_name(&name, pack_data->hash, \"pack\");\n \tif (finalize_object_file(pack_data->pack_name, name.buf))\n \t\tdie(\"cannot store pack file\");\n \n-\todb_pack_name(&name, pack_data->sha1, \"idx\");\n+\todb_pack_name(&name, pack_data->hash, \"idx\");\n \tif (finalize_object_file(curr_index_name, name.buf))\n \t\tdie(\"cannot store index file\");\n \tfree((void *)curr_index_name);\n@@ -779,7 +780,7 @@ static void unkeep_all_packs(void)\n \n \tfor (k = 0; k < pack_id; k++) {\n \t\tstruct packed_git *p = all_packs[k];\n-\t\todb_pack_name(&name, p->sha1, \"keep\");\n+\t\todb_pack_name(&name, p->hash, \"keep\");\n \t\tunlink_or_warn(name.buf);\n \t}\n \tstrbuf_release(&name);\n@@ -821,9 +822,9 @@ static void end_packfile(void)\n \n \t\tclose_pack_windows(pack_data);\n \t\tfinalize_hashfile(pack_file, cur_pack_oid.hash, 0);\n-\t\tfixup_pack_header_footer(pack_data->pack_fd, pack_data->sha1,\n-\t\t\t\t    pack_data->pack_name, object_count,\n-\t\t\t\t    cur_pack_oid.hash, pack_size);\n+\t\tfixup_pack_header_footer(pack_data->pack_fd, pack_data->hash,\n+\t\t\t\t\t pack_data->pack_name, object_count,\n+\t\t\t\t\t cur_pack_oid.hash, pack_size);\n \n \t\tif (object_count <= unpack_limit) {\n \t\t\tif (!loosen_small_pack(pack_data)) {\ndiff --git a/http-push.c b/http-push.c\nindex b22c7caea0..b313ada515 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -315,7 +315,8 @@ static void start_fetch_packed(struct transfer_request *request)\n \t\treturn;\n \t}\n \n-\tfprintf(stderr,\t\"Fetching pack %s\\n\", sha1_to_hex(target->sha1));\n+\tfprintf(stderr,\t\"Fetching pack %s\\n\",\n+\t\tsha1_to_hex(target->hash));\n \tfprintf(stderr, \" which contains %s\\n\", oid_to_hex(&request->obj->oid));\n \n \tpreq = new_http_pack_request(target, repo->url);\ndiff --git a/http-walker.c b/http-walker.c\nindex 8ae5d76c6a..8063896cf6 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -434,7 +434,7 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo, unsigne\n \n \tif (walker->get_verbosely) {\n \t\tfprintf(stderr, \"Getting pack %s\\n\",\n-\t\t\tsha1_to_hex(target->sha1));\n+\t\t\tsha1_to_hex(target->hash));\n \t\tfprintf(stderr, \" which contains %s\\n\",\n \t\t\tsha1_to_hex(sha1));\n \t}\ndiff --git a/http.c b/http.c\nindex a32ad36ddf..a09adc518f 100644\n--- a/http.c\n+++ b/http.c\n@@ -2236,10 +2236,10 @@ int finish_http_pack_request(struct http_pack_request *preq)\n \t\treturn -1;\n \t}\n \n-\tunlink(sha1_pack_index_name(p->sha1));\n+\tunlink(sha1_pack_index_name(p->hash));\n \n-\tif (finalize_object_file(preq->tmpfile.buf, sha1_pack_name(p->sha1))\n-\t || finalize_object_file(tmp_idx, sha1_pack_index_name(p->sha1))) {\n+\tif (finalize_object_file(preq->tmpfile.buf, sha1_pack_name(p->hash))\n+\t || finalize_object_file(tmp_idx, sha1_pack_index_name(p->hash))) {\n \t\tfree(tmp_idx);\n \t\treturn -1;\n \t}\n@@ -2262,10 +2262,10 @@ struct http_pack_request *new_http_pack_request(\n \n \tend_url_with_slash(&buf, base_url);\n \tstrbuf_addf(&buf, \"objects/pack/pack-%s.pack\",\n-\t\tsha1_to_hex(target->sha1));\n+\t\tsha1_to_hex(target->hash));\n \tpreq->url = strbuf_detach(&buf, NULL);\n \n-\tstrbuf_addf(&preq->tmpfile, \"%s.temp\", sha1_pack_name(target->sha1));\n+\tstrbuf_addf(&preq->tmpfile, \"%s.temp\", sha1_pack_name(target->hash));\n \tpreq->packfile = fopen(preq->tmpfile.buf, \"a\");\n \tif (!preq->packfile) {\n \t\terror(\"Unable to open local file %s for pack\",\n@@ -2289,7 +2289,8 @@ struct http_pack_request *new_http_pack_request(\n \t\tif (http_is_verbose)\n \t\t\tfprintf(stderr,\n \t\t\t\t\"Resuming fetch of pack %s at byte %\"PRIuMAX\"\\n\",\n-\t\t\t\tsha1_to_hex(target->sha1), (uintmax_t)prev_posn);\n+\t\t\t\tsha1_to_hex(target->hash),\n+\t\t\t\t(uintmax_t)prev_posn);\n \t\thttp_opt_request_remainder(preq->slot->curl, prev_posn);\n \t}\n \ndiff --git a/object-store.h b/object-store.h\nindex 14fc935bd1..56f8aea1cc 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -77,7 +77,7 @@ struct packed_git {\n \t\t freshened:1,\n \t\t do_not_close:1,\n \t\t pack_promisor:1;\n-\tunsigned char sha1[20];\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n \tstruct revindex_entry *revindex;\n \t/* something like \".git/objects/pack/xxxxx.pack\" */\n \tchar pack_name[FLEX_ARRAY]; /* more */\ndiff --git a/packfile.c b/packfile.c\nindex 16bcb75262..e6e8861650 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -235,7 +235,7 @@ struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_path)\n \tstruct packed_git *p = alloc_packed_git(alloc);\n \n \tmemcpy(p->pack_name, path, alloc); /* includes NUL */\n-\thashcpy(p->sha1, sha1);\n+\thashcpy(p->hash, sha1);\n \tif (check_packed_git_idx(idx_path, p)) {\n \t\tfree(p);\n \t\treturn NULL;\n@@ -722,8 +722,8 @@ struct packed_git *add_packed_git(const char *path, size_t path_len, int local)\n \tp->pack_local = local;\n \tp->mtime = st.st_mtime;\n \tif (path_len < the_hash_algo->hexsz ||\n-\t    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->sha1))\n-\t\thashclr(p->sha1);\n+\t    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))\n+\t\thashclr(p->hash);\n \treturn p;\n }\n \n"},{"id":"369090","messageId":"20190212012256.1005924-15-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 14/31] builtin/am: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:39Z","receivedAt":"2019-02-12T01:23:32Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Instead of using GIT_SHA1_HEXSZ, switch to using the_hash_algo and\nparse_oid_hex to parse the lines involved in rebasing notes.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n builtin/am.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/am.c b/builtin/am.c\nindex 58a2aef28b..584baf1c7e 100644\n--- a/builtin/am.c\n+++ b/builtin/am.c\n@@ -485,23 +485,24 @@ static int copy_notes_for_rebase(const struct am_state *state)\n \n \twhile (!strbuf_getline_lf(&sb, fp)) {\n \t\tstruct object_id from_obj, to_obj;\n+\t\tconst char *p;\n \n-\t\tif (sb.len != GIT_SHA1_HEXSZ * 2 + 1) {\n+\t\tif (sb.len != the_hash_algo->hexsz * 2 + 1) {\n \t\t\tret = error(invalid_line, sb.buf);\n \t\t\tgoto finish;\n \t\t}\n \n-\t\tif (get_oid_hex(sb.buf, &from_obj)) {\n+\t\tif (parse_oid_hex(sb.buf, &from_obj, &p)) {\n \t\t\tret = error(invalid_line, sb.buf);\n \t\t\tgoto finish;\n \t\t}\n \n-\t\tif (sb.buf[GIT_SHA1_HEXSZ] != ' ') {\n+\t\tif (*p != ' ') {\n \t\t\tret = error(invalid_line, sb.buf);\n \t\t\tgoto finish;\n \t\t}\n \n-\t\tif (get_oid_hex(sb.buf + GIT_SHA1_HEXSZ + 1, &to_obj)) {\n+\t\tif (get_oid_hex(p + 1, &to_obj)) {\n \t\t\tret = error(invalid_line, sb.buf);\n \t\t\tgoto finish;\n \t\t}\n"},{"id":"369092","messageId":"20190212012256.1005924-16-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 15/31] builtin/pull: make hash-size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:40Z","receivedAt":"2019-02-12T01:23:33Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Instead of using get_oid_hex and GIT_SHA1_HEXSZ, use parse_oid_hex to\navoid the need for a constant and simplify the code.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n builtin/pull.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/pull.c b/builtin/pull.c\nindex 701d1473dc..52d93c1e3b 100644\n--- a/builtin/pull.c\n+++ b/builtin/pull.c\n@@ -365,9 +365,10 @@ static void get_merge_heads(struct oid_array *merge_heads)\n \n \tfp = xfopen(filename, \"r\");\n \twhile (strbuf_getline_lf(&sb, fp) != EOF) {\n-\t\tif (get_oid_hex(sb.buf, &oid))\n+\t\tconst char *p;\n+\t\tif (parse_oid_hex(sb.buf, &oid, &p))\n \t\t\tcontinue;  /* invalid line: does not start with SHA1 */\n-\t\tif (starts_with(sb.buf + GIT_SHA1_HEXSZ, \"\\tnot-for-merge\\t\"))\n+\t\tif (starts_with(p, \"\\tnot-for-merge\\t\"))\n \t\t\tcontinue;  /* ref is not-for-merge */\n \t\toid_array_append(merge_heads, &oid);\n \t}\n"},{"id":"369093","messageId":"20190212012256.1005924-14-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 13/31] fast-import: replace sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:38Z","receivedAt":"2019-02-12T01:23:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Replace the uses of sha1_to_hex in this function with hash_to_hex to\nallow the use of SHA-256 as well.  Rename a variable since it is no\nlonger limited to SHA-1.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n fast-import.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex eba8c8c919..3d6e2bfa4d 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -2945,7 +2945,7 @@ static struct object_entry *parse_treeish_dataref(const char **p)\n \treturn e;\n }\n \n-static void print_ls(int mode, const unsigned char *sha1, const char *path)\n+static void print_ls(int mode, const unsigned char *hash, const char *path)\n {\n \tstatic struct strbuf line = STRBUF_INIT;\n \n@@ -2965,7 +2965,7 @@ static void print_ls(int mode, const unsigned char *sha1, const char *path)\n \t\t/* mode SP type SP object_name TAB path LF */\n \t\tstrbuf_reset(&line);\n \t\tstrbuf_addf(&line, \"%06o %s %s\\t\",\n-\t\t\t\tmode & ~NO_DELTA, type, sha1_to_hex(sha1));\n+\t\t\t\tmode & ~NO_DELTA, type, hash_to_hex(hash));\n \t\tquote_c_style(path, &line, NULL, 0);\n \t\tstrbuf_addch(&line, '\\n');\n \t}\n"},{"id":"369094","messageId":"20190212012256.1005924-18-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 17/31] http-backend: allow 64-character hex names","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:42Z","receivedAt":"2019-02-12T01:23:36Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"In an SHA-256-backed repository using the http-backend handler for dumb\nprotocol clients, it may be necessary to access the raw packs using\ntheir full SHA-256-specified names.  Allow packs and loose objects to be\naccessed using their full SHA-256-specified 64-character hex names.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http-backend.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/http-backend.c b/http-backend.c\nindex 29e68e38b5..ec3144b444 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -711,8 +711,11 @@ static struct service_cmd {\n \t{\"GET\", \"/objects/info/http-alternates$\", get_text_file},\n \t{\"GET\", \"/objects/info/packs$\", get_info_packs},\n \t{\"GET\", \"/objects/[0-9a-f]{2}/[0-9a-f]{38}$\", get_loose_object},\n+\t{\"GET\", \"/objects/[0-9a-f]{2}/[0-9a-f]{62}$\", get_loose_object},\n \t{\"GET\", \"/objects/pack/pack-[0-9a-f]{40}\\\\.pack$\", get_pack_file},\n+\t{\"GET\", \"/objects/pack/pack-[0-9a-f]{64}\\\\.pack$\", get_pack_file},\n \t{\"GET\", \"/objects/pack/pack-[0-9a-f]{40}\\\\.idx$\", get_idx_file},\n+\t{\"GET\", \"/objects/pack/pack-[0-9a-f]{64}\\\\.idx$\", get_idx_file},\n \n \t{\"POST\", \"/git-upload-pack$\", service_rpc},\n \t{\"POST\", \"/git-receive-pack$\", service_rpc}\n"},{"id":"369095","messageId":"20190212012256.1005924-17-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 16/31] http-push: convert to use the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:41Z","receivedAt":"2019-02-12T01:23:37Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Switch the lock token code to use the_hash_algo and increase its buffers\nto be allocated using GIT_MAX_* constants.  Update the parsing of object\npaths to use the_hash_algo as well.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http-push.c | 20 ++++++++++----------\n 1 file changed, 10 insertions(+), 10 deletions(-)\n\ndiff --git a/http-push.c b/http-push.c\nindex b313ada515..c4e5142bc9 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -145,7 +145,7 @@ struct remote_lock {\n \tchar *url;\n \tchar *owner;\n \tchar *token;\n-\tchar tmpfile_suffix[41];\n+\tchar tmpfile_suffix[GIT_MAX_HEXSZ + 1];\n \ttime_t start_time;\n \tlong timeout;\n \tint refreshing;\n@@ -399,7 +399,7 @@ static void start_put(struct transfer_request *request)\n \trequest->dest = strbuf_detach(&buf, NULL);\n \n \tappend_remote_object_url(&buf, repo->url, hex, 0);\n-\tstrbuf_add(&buf, request->lock->tmpfile_suffix, 41);\n+\tstrbuf_add(&buf, request->lock->tmpfile_suffix, the_hash_algo->hexsz + 1);\n \trequest->url = strbuf_detach(&buf, NULL);\n \n \tslot = get_active_slot();\n@@ -758,8 +758,8 @@ static void handle_lockprop_ctx(struct xml_ctx *ctx, int tag_closed)\n static void handle_new_lock_ctx(struct xml_ctx *ctx, int tag_closed)\n {\n \tstruct remote_lock *lock = (struct remote_lock *)ctx->userData;\n-\tgit_SHA_CTX sha_ctx;\n-\tunsigned char lock_token_sha1[20];\n+\tgit_hash_ctx hash_ctx;\n+\tunsigned char lock_token_hash[GIT_MAX_RAWSZ];\n \n \tif (tag_closed && ctx->cdata) {\n \t\tif (!strcmp(ctx->name, DAV_ACTIVELOCK_OWNER)) {\n@@ -771,12 +771,12 @@ static void handle_new_lock_ctx(struct xml_ctx *ctx, int tag_closed)\n \t\t} else if (!strcmp(ctx->name, DAV_ACTIVELOCK_TOKEN)) {\n \t\t\tlock->token = xstrdup(ctx->cdata);\n \n-\t\t\tgit_SHA1_Init(&sha_ctx);\n-\t\t\tgit_SHA1_Update(&sha_ctx, lock->token, strlen(lock->token));\n-\t\t\tgit_SHA1_Final(lock_token_sha1, &sha_ctx);\n+\t\t\tthe_hash_algo->init_fn(&hash_ctx);\n+\t\t\tthe_hash_algo->update_fn(&hash_ctx, lock->token, strlen(lock->token));\n+\t\t\tthe_hash_algo->final_fn(lock_token_hash, &hash_ctx);\n \n \t\t\tlock->tmpfile_suffix[0] = '_';\n-\t\t\tmemcpy(lock->tmpfile_suffix + 1, sha1_to_hex(lock_token_sha1), 40);\n+\t\t\tmemcpy(lock->tmpfile_suffix + 1, hash_to_hex(lock_token_hash), the_hash_algo->hexsz);\n \t\t}\n \t}\n }\n@@ -1018,7 +1018,7 @@ static void remote_ls(const char *path, int flags,\n /* extract hex from sharded \"xx/x{38}\" filename */\n static int get_oid_hex_from_objpath(const char *path, struct object_id *oid)\n {\n-\tif (strlen(path) != GIT_SHA1_HEXSZ + 1)\n+\tif (strlen(path) != the_hash_algo->hexsz + 1)\n \t\treturn -1;\n \n \tif (hex_to_bytes(oid->hash, path, 1))\n@@ -1026,7 +1026,7 @@ static int get_oid_hex_from_objpath(const char *path, struct object_id *oid)\n \tpath += 2;\n \tpath++; /* skip '/' */\n \n-\treturn hex_to_bytes(oid->hash + 1, path, GIT_SHA1_RAWSZ - 1);\n+\treturn hex_to_bytes(oid->hash + 1, path, the_hash_algo->rawsz - 1);\n }\n \n static void process_ls_object(struct remote_ls_ctx *ls)\n"},{"id":"369096","messageId":"20190212012256.1005924-13-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 12/31] fast-import: make hash-size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:37Z","receivedAt":"2019-02-12T01:23:38Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Replace several uses of GIT_SHA1_HEXSZ and 40-based constants with\nreferences to the_hash_algo.  Update the note handling code here to\ncompute path sizes based on GIT_MAX_RAWSZ as well.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n fast-import.c | 27 +++++++++++++++------------\n 1 file changed, 15 insertions(+), 12 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex 7c9a10a77b..eba8c8c919 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -1241,7 +1241,7 @@ static void load_tree(struct tree_entry *root)\n \t\tc += e->name->str_len + 1;\n \t\thashcpy(e->versions[0].oid.hash, (unsigned char *)c);\n \t\thashcpy(e->versions[1].oid.hash, (unsigned char *)c);\n-\t\tc += GIT_SHA1_RAWSZ;\n+\t\tc += the_hash_algo->rawsz;\n \t}\n \tfree(buf);\n }\n@@ -1288,7 +1288,7 @@ static void mktree(struct tree_content *t, int v, struct strbuf *b)\n \t\tstrbuf_addf(b, \"%o %s%c\",\n \t\t\t(unsigned int)(e->versions[v].mode & ~NO_DELTA),\n \t\t\te->name->str_dat, '\\0');\n-\t\tstrbuf_add(b, e->versions[v].oid.hash, GIT_SHA1_RAWSZ);\n+\t\tstrbuf_add(b, e->versions[v].oid.hash, the_hash_algo->rawsz);\n \t}\n }\n \n@@ -2047,7 +2047,8 @@ static uintmax_t do_change_note_fanout(\n \tunsigned int i, tmp_hex_oid_len, tmp_fullpath_len;\n \tuintmax_t num_notes = 0;\n \tstruct object_id oid;\n-\tchar realpath[60];\n+\tchar realpath[GIT_MAX_RAWSZ * 3];\n+\tconst unsigned hexsz = the_hash_algo->hexsz;\n \n \tif (!root->tree)\n \t\tload_tree(root);\n@@ -2067,7 +2068,7 @@ static uintmax_t do_change_note_fanout(\n \t\t * of 2 chars.\n \t\t */\n \t\tif (!e->versions[1].mode ||\n-\t\t    tmp_hex_oid_len > GIT_SHA1_HEXSZ ||\n+\t\t    tmp_hex_oid_len > hexsz ||\n \t\t    e->name->str_len % 2)\n \t\t\tcontinue;\n \n@@ -2081,7 +2082,7 @@ static uintmax_t do_change_note_fanout(\n \t\ttmp_fullpath_len += e->name->str_len;\n \t\tfullpath[tmp_fullpath_len] = '\\0';\n \n-\t\tif (tmp_hex_oid_len == GIT_SHA1_HEXSZ && !get_oid_hex(hex_oid, &oid)) {\n+\t\tif (tmp_hex_oid_len == hexsz && !get_oid_hex(hex_oid, &oid)) {\n \t\t\t/* This is a note entry */\n \t\t\tif (fanout == 0xff) {\n \t\t\t\t/* Counting mode, no rename */\n@@ -2352,7 +2353,7 @@ static void note_change_n(const char *p, struct branch *b, unsigned char *old_fa\n \tstruct object_entry *oe;\n \tstruct branch *s;\n \tstruct object_id oid, commit_oid;\n-\tchar path[60];\n+\tchar path[GIT_MAX_RAWSZ * 3];\n \tuint16_t inline_data = 0;\n \tunsigned char new_fanout;\n \n@@ -2405,7 +2406,7 @@ static void note_change_n(const char *p, struct branch *b, unsigned char *old_fa\n \t\tchar *buf = read_object_with_reference(&commit_oid,\n \t\t\t\t\t\t       commit_type, &size,\n \t\t\t\t\t\t       &commit_oid);\n-\t\tif (!buf || size < 46)\n+\t\tif (!buf || size < the_hash_algo->hexsz)\n \t\t\tdie(\"Not a valid commit: %s\", p);\n \t\tfree(buf);\n \t} else\n@@ -2456,7 +2457,7 @@ static void file_change_deleteall(struct branch *b)\n \n static void parse_from_commit(struct branch *b, char *buf, unsigned long size)\n {\n-\tif (!buf || size < GIT_SHA1_HEXSZ + 6)\n+\tif (!buf || size < the_hash_algo->hexsz + 6)\n \t\tdie(\"Not a valid commit: %s\", oid_to_hex(&b->oid));\n \tif (memcmp(\"tree \", buf, 5)\n \t\t|| get_oid_hex(buf + 5, &b->branch_tree.versions[1].oid))\n@@ -2555,7 +2556,7 @@ static struct hash_list *parse_merge(unsigned int *count)\n \t\t\tchar *buf = read_object_with_reference(&n->oid,\n \t\t\t\t\t\t\t       commit_type,\n \t\t\t\t\t\t\t       &size, &n->oid);\n-\t\t\tif (!buf || size < 46)\n+\t\t\tif (!buf || size < the_hash_algo->hexsz)\n \t\t\t\tdie(\"Not a valid commit: %s\", from);\n \t\t\tfree(buf);\n \t\t} else\n@@ -2842,7 +2843,7 @@ static void parse_get_mark(const char *p)\n \t\tdie(\"Unknown mark: %s\", command_buf.buf);\n \n \txsnprintf(output, sizeof(output), \"%s\\n\", oid_to_hex(&oe->idx.oid));\n-\tcat_blob_write(output, GIT_SHA1_HEXSZ + 1);\n+\tcat_blob_write(output, the_hash_algo->hexsz + 1);\n }\n \n static void parse_cat_blob(const char *p)\n@@ -2872,6 +2873,8 @@ static struct object_entry *dereference(struct object_entry *oe,\n {\n \tunsigned long size;\n \tchar *buf = NULL;\n+\tconst unsigned hexsz = the_hash_algo->hexsz;\n+\n \tif (!oe) {\n \t\tenum object_type type = oid_object_info(the_repository, oid,\n \t\t\t\t\t\t\tNULL);\n@@ -2905,12 +2908,12 @@ static struct object_entry *dereference(struct object_entry *oe,\n \t/* Peel one layer. */\n \tswitch (oe->type) {\n \tcase OBJ_TAG:\n-\t\tif (size < GIT_SHA1_HEXSZ + strlen(\"object \") ||\n+\t\tif (size < hexsz + strlen(\"object \") ||\n \t\t    get_oid_hex(buf + strlen(\"object \"), oid))\n \t\t\tdie(\"Invalid SHA1 in tag: %s\", command_buf.buf);\n \t\tbreak;\n \tcase OBJ_COMMIT:\n-\t\tif (size < GIT_SHA1_HEXSZ + strlen(\"tree \") ||\n+\t\tif (size < hexsz + strlen(\"tree \") ||\n \t\t    get_oid_hex(buf + strlen(\"tree \"), oid))\n \t\t\tdie(\"Invalid SHA1 in commit: %s\", command_buf.buf);\n \t}\n"},{"id":"369097","messageId":"20190212012256.1005924-19-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 18/31] http-push: remove remaining uses of sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:43Z","receivedAt":"2019-02-12T01:23:39Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Since sha1_to_hex is limited to SHA-1, switch all remaining uses of it\nin this file to hash_to_hex or oid_to_hex.  Modify update_remote to take\na pointer to struct object_id, and since we don't modify that parameter\nin the function, set it to be const as well.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http-push.c | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/http-push.c b/http-push.c\nindex c4e5142bc9..f675a96316 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -316,7 +316,7 @@ static void start_fetch_packed(struct transfer_request *request)\n \t}\n \n \tfprintf(stderr,\t\"Fetching pack %s\\n\",\n-\t\tsha1_to_hex(target->hash));\n+\t\thash_to_hex(target->hash));\n \tfprintf(stderr, \" which contains %s\\n\", oid_to_hex(&request->obj->oid));\n \n \tpreq = new_http_pack_request(target, repo->url);\n@@ -1374,7 +1374,7 @@ static int get_delta(struct rev_info *revs, struct remote_lock *lock)\n \treturn count;\n }\n \n-static int update_remote(unsigned char *sha1, struct remote_lock *lock)\n+static int update_remote(const struct object_id *oid, struct remote_lock *lock)\n {\n \tstruct active_request_slot *slot;\n \tstruct slot_results results;\n@@ -1383,7 +1383,7 @@ static int update_remote(unsigned char *sha1, struct remote_lock *lock)\n \n \tdav_headers = get_dav_token_headers(lock, DAV_HEADER_IF);\n \n-\tstrbuf_addf(&out_buffer.buf, \"%s\\n\", sha1_to_hex(sha1));\n+\tstrbuf_addf(&out_buffer.buf, \"%s\\n\", oid_to_hex(oid));\n \n \tslot = get_active_slot();\n \tslot->results = &results;\n@@ -1948,7 +1948,7 @@ int cmd_main(int argc, const char **argv)\n \t\trun_request_queue();\n \n \t\t/* Update the remote branch if all went well */\n-\t\tif (aborted || !update_remote(ref->new_oid.hash, ref_lock))\n+\t\tif (aborted || !update_remote(&ref->new_oid, ref_lock))\n \t\t\trc = 1;\n \n \t\tif (!rc)\n"},{"id":"369098","messageId":"20190212012256.1005924-21-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 20/31] http: replace hard-coded constant with the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:45Z","receivedAt":"2019-02-12T01:23:41Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Replace a hard-coded 40 with a reference to the_hash_algo.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/http.c b/http.c\nindex a09adc518f..993ddc956a 100644\n--- a/http.c\n+++ b/http.c\n@@ -2065,7 +2065,7 @@ int http_fetch_ref(const char *base, struct ref *ref)\n \turl = quote_ref_url(base, ref->name);\n \tif (http_get_strbuf(url, &buffer, &options) == HTTP_OK) {\n \t\tstrbuf_rtrim(&buffer);\n-\t\tif (buffer.len == 40)\n+\t\tif (buffer.len == the_hash_algo->hexsz)\n \t\t\tret = get_oid_hex(buffer.buf, &ref->old_oid);\n \t\telse if (starts_with(buffer.buf, \"ref: \")) {\n \t\t\tref->symref = xstrdup(buffer.buf + 5);\n"},{"id":"369099","messageId":"20190212012256.1005924-23-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 22/31] http: replace sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:47Z","receivedAt":"2019-02-12T01:23:42Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Since sha1_to_hex is limited to SHA-1, replace it with hash_to_hex.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http.c | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 458d07fabb..8ab07458e4 100644\n--- a/http.c\n+++ b/http.c\n@@ -2079,19 +2079,19 @@ int http_fetch_ref(const char *base, struct ref *ref)\n }\n \n /* Helpers for fetching packs */\n-static char *fetch_pack_index(unsigned char *sha1, const char *base_url)\n+static char *fetch_pack_index(unsigned char *hash, const char *base_url)\n {\n \tchar *url, *tmp;\n \tstruct strbuf buf = STRBUF_INIT;\n \n \tif (http_is_verbose)\n-\t\tfprintf(stderr, \"Getting index for pack %s\\n\", sha1_to_hex(sha1));\n+\t\tfprintf(stderr, \"Getting index for pack %s\\n\", hash_to_hex(hash));\n \n \tend_url_with_slash(&buf, base_url);\n-\tstrbuf_addf(&buf, \"objects/pack/pack-%s.idx\", sha1_to_hex(sha1));\n+\tstrbuf_addf(&buf, \"objects/pack/pack-%s.idx\", hash_to_hex(hash));\n \turl = strbuf_detach(&buf, NULL);\n \n-\tstrbuf_addf(&buf, \"%s.temp\", sha1_pack_index_name(sha1));\n+\tstrbuf_addf(&buf, \"%s.temp\", sha1_pack_index_name(hash));\n \ttmp = strbuf_detach(&buf, NULL);\n \n \tif (http_get_file(url, tmp, NULL) != HTTP_OK) {\n@@ -2262,7 +2262,7 @@ struct http_pack_request *new_http_pack_request(\n \n \tend_url_with_slash(&buf, base_url);\n \tstrbuf_addf(&buf, \"objects/pack/pack-%s.pack\",\n-\t\tsha1_to_hex(target->hash));\n+\t\thash_to_hex(target->hash));\n \tpreq->url = strbuf_detach(&buf, NULL);\n \n \tstrbuf_addf(&preq->tmpfile, \"%s.temp\", sha1_pack_name(target->hash));\n@@ -2289,7 +2289,7 @@ struct http_pack_request *new_http_pack_request(\n \t\tif (http_is_verbose)\n \t\t\tfprintf(stderr,\n \t\t\t\t\"Resuming fetch of pack %s at byte %\"PRIuMAX\"\\n\",\n-\t\t\t\tsha1_to_hex(target->hash),\n+\t\t\t\thash_to_hex(target->hash),\n \t\t\t\t(uintmax_t)prev_posn);\n \t\thttp_opt_request_remainder(preq->slot->curl, prev_posn);\n \t}\n"},{"id":"369100","messageId":"20190212012256.1005924-22-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 21/31] http: compute hash of downloaded objects using the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:46Z","receivedAt":"2019-02-12T01:23:43Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http.c | 10 +++++-----\n http.h |  2 +-\n 2 files changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex 993ddc956a..458d07fabb 100644\n--- a/http.c\n+++ b/http.c\n@@ -2337,8 +2337,8 @@ static size_t fwrite_sha1_file(char *ptr, size_t eltsize, size_t nmemb,\n \t\tfreq->stream.next_out = expn;\n \t\tfreq->stream.avail_out = sizeof(expn);\n \t\tfreq->zret = git_inflate(&freq->stream, Z_SYNC_FLUSH);\n-\t\tgit_SHA1_Update(&freq->c, expn,\n-\t\t\t\tsizeof(expn) - freq->stream.avail_out);\n+\t\tthe_hash_algo->update_fn(&freq->c, expn,\n+\t\t\t\t\t sizeof(expn) - freq->stream.avail_out);\n \t} while (freq->stream.avail_in && freq->zret == Z_OK);\n \treturn size;\n }\n@@ -2396,7 +2396,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \n \tgit_inflate_init(&freq->stream);\n \n-\tgit_SHA1_Init(&freq->c);\n+\tthe_hash_algo->init_fn(&freq->c);\n \n \tfreq->url = get_remote_object_url(base_url, hex, 0);\n \n@@ -2431,7 +2431,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \tif (prev_read == -1) {\n \t\tmemset(&freq->stream, 0, sizeof(freq->stream));\n \t\tgit_inflate_init(&freq->stream);\n-\t\tgit_SHA1_Init(&freq->c);\n+\t\tthe_hash_algo->init_fn(&freq->c);\n \t\tif (prev_posn>0) {\n \t\t\tprev_posn = 0;\n \t\t\tlseek(freq->localfile, 0, SEEK_SET);\n@@ -2502,7 +2502,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t}\n \n \tgit_inflate_end(&freq->stream);\n-\tgit_SHA1_Final(freq->real_oid.hash, &freq->c);\n+\tthe_hash_algo->final_fn(freq->real_oid.hash, &freq->c);\n \tif (freq->zret != Z_STREAM_END) {\n \t\tunlink_or_warn(freq->tmpfile.buf);\n \t\treturn -1;\ndiff --git a/http.h b/http.h\nindex 4eb4e808e5..10d3cfdb80 100644\n--- a/http.h\n+++ b/http.h\n@@ -225,7 +225,7 @@ struct http_object_request {\n \tlong http_code;\n \tstruct object_id oid;\n \tstruct object_id real_oid;\n-\tgit_SHA_CTX c;\n+\tgit_hash_ctx c;\n \tgit_zstream stream;\n \tint zret;\n \tint rename;\n"},{"id":"369101","messageId":"20190212012256.1005924-24-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 23/31] remote-curl: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:48Z","receivedAt":"2019-02-12T01:23:46Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Change one hard-coded use of the constant 40 to a reference to\nthe_hash_algo.  In addition, switch a use of get_oid_hex to\nparse_oid_hex to avoid the need to use a constant.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n remote-curl.c | 11 ++++++-----\n 1 file changed, 6 insertions(+), 5 deletions(-)\n\ndiff --git a/remote-curl.c b/remote-curl.c\nindex bb7421023b..8395b71bbb 100644\n--- a/remote-curl.c\n+++ b/remote-curl.c\n@@ -249,7 +249,7 @@ static struct ref *parse_info_refs(struct discovery *heads)\n \t\tif (data[i] == '\\t')\n \t\t\tmid = &data[i];\n \t\tif (data[i] == '\\n') {\n-\t\t\tif (mid - start != 40)\n+\t\t\tif (mid - start != the_hash_algo->hexsz)\n \t\t\t\tdie(\"%sinfo/refs not valid: is this a git repository?\",\n \t\t\t\t    url.buf);\n \t\t\tdata[i] = 0;\n@@ -1013,12 +1013,13 @@ static void parse_fetch(struct strbuf *buf)\n \t\t\tconst char *name;\n \t\t\tstruct ref *ref;\n \t\t\tstruct object_id old_oid;\n+\t\t\tconst char *q;\n \n-\t\t\tif (get_oid_hex(p, &old_oid))\n+\t\t\tif (parse_oid_hex(p, &old_oid, &q))\n \t\t\t\tdie(\"protocol error: expected sha/ref, got %s'\", p);\n-\t\t\tif (p[GIT_SHA1_HEXSZ] == ' ')\n-\t\t\t\tname = p + GIT_SHA1_HEXSZ + 1;\n-\t\t\telse if (!p[GIT_SHA1_HEXSZ])\n+\t\t\tif (*q == ' ')\n+\t\t\t\tname = q + 1;\n+\t\t\telse if (!*q)\n \t\t\t\tname = \"\";\n \t\t\telse\n \t\t\t\tdie(\"protocol error: expected sha/ref, got %s'\", p);\n"},{"id":"369102","messageId":"20190212012256.1005924-20-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 19/31] http-walker: replace sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:44Z","receivedAt":"2019-02-12T01:23:48Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Since sha1_to_hex is limited to SHA-1, replace the uses of it in this\nfile with hasH_to_hex.  Rename several variables accordingly to reflect\nthat they are no longer limited to SHA-1.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n http-walker.c | 18 +++++++++---------\n 1 file changed, 9 insertions(+), 9 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex 8063896cf6..e11670eee2 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -434,9 +434,9 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo, unsigne\n \n \tif (walker->get_verbosely) {\n \t\tfprintf(stderr, \"Getting pack %s\\n\",\n-\t\t\tsha1_to_hex(target->hash));\n+\t\t\thash_to_hex(target->hash));\n \t\tfprintf(stderr, \" which contains %s\\n\",\n-\t\t\tsha1_to_hex(sha1));\n+\t\t\thash_to_hex(sha1));\n \t}\n \n \tpreq = new_http_pack_request(target, repo->base);\n@@ -473,9 +473,9 @@ static void abort_object_request(struct object_request *obj_req)\n \trelease_object_request(obj_req);\n }\n \n-static int fetch_object(struct walker *walker, unsigned char *sha1)\n+static int fetch_object(struct walker *walker, unsigned char *hash)\n {\n-\tchar *hex = sha1_to_hex(sha1);\n+\tchar *hex = hash_to_hex(hash);\n \tint ret = 0;\n \tstruct object_request *obj_req = NULL;\n \tstruct http_object_request *req;\n@@ -483,7 +483,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)\n \n \tlist_for_each(pos, head) {\n \t\tobj_req = list_entry(pos, struct object_request, node);\n-\t\tif (hasheq(obj_req->oid.hash, sha1))\n+\t\tif (hasheq(obj_req->oid.hash, hash))\n \t\t\tbreak;\n \t}\n \tif (obj_req == NULL)\n@@ -557,20 +557,20 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)\n \treturn ret;\n }\n \n-static int fetch(struct walker *walker, unsigned char *sha1)\n+static int fetch(struct walker *walker, unsigned char *hash)\n {\n \tstruct walker_data *data = walker->data;\n \tstruct alt_base *altbase = data->alt;\n \n-\tif (!fetch_object(walker, sha1))\n+\tif (!fetch_object(walker, hash))\n \t\treturn 0;\n \twhile (altbase) {\n-\t\tif (!http_fetch_pack(walker, altbase, sha1))\n+\t\tif (!http_fetch_pack(walker, altbase, hash))\n \t\t\treturn 0;\n \t\tfetch_alternates(walker, data->alt->base);\n \t\taltbase = altbase->next;\n \t}\n-\treturn error(\"Unable to find %s under %s\", sha1_to_hex(sha1),\n+\treturn error(\"Unable to find %s under %s\", hash_to_hex(hash),\n \t\t     data->alt->base);\n }\n \n"},{"id":"369103","messageId":"20190212012256.1005924-30-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 29/31] read-cache: read data in a hash-independent way","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:54Z","receivedAt":"2019-02-12T01:23:49Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Index entries are structured with a variety of fields up front, followed\nby a hash and one or two flags fields.  Because the hash field is stored\nin the middle of the structure, it's difficult to use one fixed-size\nstructure that easily allows access to the hash and flags fields.\nAdjust the structure to hold the maximum amount of data that may be\nneeded using a member called \"data\" and read and write this field\nindependently in the various places that need to read and write the\nstructure.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n read-cache.c | 74 ++++++++++++++++++++--------------------------------\n 1 file changed, 29 insertions(+), 45 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 0e0c93edc9..d9f12c568f 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1634,39 +1634,24 @@ struct ondisk_cache_entry {\n \tuint32_t uid;\n \tuint32_t gid;\n \tuint32_t size;\n-\tunsigned char sha1[20];\n-\tuint16_t flags;\n-\tchar name[FLEX_ARRAY]; /* more */\n-};\n-\n-/*\n- * This struct is used when CE_EXTENDED bit is 1\n- * The struct must match ondisk_cache_entry exactly from\n- * ctime till flags\n- */\n-struct ondisk_cache_entry_extended {\n-\tstruct cache_time ctime;\n-\tstruct cache_time mtime;\n-\tuint32_t dev;\n-\tuint32_t ino;\n-\tuint32_t mode;\n-\tuint32_t uid;\n-\tuint32_t gid;\n-\tuint32_t size;\n-\tunsigned char sha1[20];\n-\tuint16_t flags;\n-\tuint16_t flags2;\n-\tchar name[FLEX_ARRAY]; /* more */\n+\t/*\n+\t * unsigned char hash[hashsz];\n+\t * uint16_t flags;\n+\t * if (flags & CE_EXTENDED)\n+\t *\tuint16_t flags2;\n+\t */\n+\tunsigned char data[GIT_MAX_RAWSZ + 2 * sizeof(uint16_t)];\n+\tchar name[FLEX_ARRAY];\n };\n \n /* These are only used for v3 or lower */\n #define align_padding_size(size, len) ((size + (len) + 8) & ~7) - (size + len)\n-#define align_flex_name(STRUCT,len) ((offsetof(struct STRUCT,name) + (len) + 8) & ~7)\n+#define align_flex_name(STRUCT,len) ((offsetof(struct STRUCT,data) + (len) + 8) & ~7)\n #define ondisk_cache_entry_size(len) align_flex_name(ondisk_cache_entry,len)\n-#define ondisk_cache_entry_extended_size(len) align_flex_name(ondisk_cache_entry_extended,len)\n-#define ondisk_ce_size(ce) (((ce)->ce_flags & CE_EXTENDED) ? \\\n-\t\t\t    ondisk_cache_entry_extended_size(ce_namelen(ce)) : \\\n-\t\t\t    ondisk_cache_entry_size(ce_namelen(ce)))\n+#define ondisk_data_size(flags, len) (the_hash_algo->rawsz + \\\n+\t\t\t\t     ((flags & CE_EXTENDED) ? 2 : 1) * sizeof(uint16_t) + len)\n+#define ondisk_data_size_max(len) (ondisk_data_size(CE_EXTENDED, len))\n+#define ondisk_ce_size(ce) (ondisk_cache_entry_size(ondisk_data_size((ce)->ce_flags, ce_namelen(ce))))\n \n /* Allow fsck to force verification of the index checksum. */\n int verify_index_checksum;\n@@ -1740,6 +1725,8 @@ static struct cache_entry *create_from_disk(struct mem_pool *ce_mem_pool,\n \tstruct cache_entry *ce;\n \tsize_t len;\n \tconst char *name;\n+\tconst unsigned hashsz = the_hash_algo->rawsz;\n+\tconst uint16_t *flagsp = (const uint16_t *)(ondisk->data + hashsz);\n \tunsigned int flags;\n \tsize_t copy_len = 0;\n \t/*\n@@ -1752,22 +1739,20 @@ static struct cache_entry *create_from_disk(struct mem_pool *ce_mem_pool,\n \tint expand_name_field = version == 4;\n \n \t/* On-disk flags are just 16 bits */\n-\tflags = get_be16(&ondisk->flags);\n+\tflags = get_be16(flagsp);\n \tlen = flags & CE_NAMEMASK;\n \n \tif (flags & CE_EXTENDED) {\n-\t\tstruct ondisk_cache_entry_extended *ondisk2;\n \t\tint extended_flags;\n-\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n-\t\textended_flags = get_be16(&ondisk2->flags2) << 16;\n+\t\textended_flags = get_be16(flagsp + 1) << 16;\n \t\t/* We do not yet understand any bit out of CE_EXTENDED_FLAGS */\n \t\tif (extended_flags & ~CE_EXTENDED_FLAGS)\n \t\t\tdie(_(\"unknown index entry format 0x%08x\"), extended_flags);\n \t\tflags |= extended_flags;\n-\t\tname = ondisk2->name;\n+\t\tname = (const char *)(flagsp + 2);\n \t}\n \telse\n-\t\tname = ondisk->name;\n+\t\tname = (const char *)(flagsp + 1);\n \n \tif (expand_name_field) {\n \t\tconst unsigned char *cp = (const unsigned char *)name;\n@@ -1806,7 +1791,9 @@ static struct cache_entry *create_from_disk(struct mem_pool *ce_mem_pool,\n \tce->ce_flags = flags & ~CE_NAMEMASK;\n \tce->ce_namelen = len;\n \tce->index = 0;\n-\thashcpy(ce->oid.hash, ondisk->sha1);\n+\thashcpy(ce->oid.hash, ondisk->data);\n+\tmemcpy(ce->name, name, len);\n+\tce->name[len] = '\\0';\n \n \tif (expand_name_field) {\n \t\tif (copy_len)\n@@ -2528,6 +2515,8 @@ static void copy_cache_entry_to_ondisk(struct ondisk_cache_entry *ondisk,\n \t\t\t\t       struct cache_entry *ce)\n {\n \tshort flags;\n+\tconst unsigned hashsz = the_hash_algo->rawsz;\n+\tuint16_t *flagsp = (uint16_t *)(ondisk->data + hashsz);\n \n \tondisk->ctime.sec = htonl(ce->ce_stat_data.sd_ctime.sec);\n \tondisk->mtime.sec = htonl(ce->ce_stat_data.sd_mtime.sec);\n@@ -2539,15 +2528,13 @@ static void copy_cache_entry_to_ondisk(struct ondisk_cache_entry *ondisk,\n \tondisk->uid  = htonl(ce->ce_stat_data.sd_uid);\n \tondisk->gid  = htonl(ce->ce_stat_data.sd_gid);\n \tondisk->size = htonl(ce->ce_stat_data.sd_size);\n-\thashcpy(ondisk->sha1, ce->oid.hash);\n+\thashcpy(ondisk->data, ce->oid.hash);\n \n \tflags = ce->ce_flags & ~CE_NAMEMASK;\n \tflags |= (ce_namelen(ce) >= CE_NAMEMASK ? CE_NAMEMASK : ce_namelen(ce));\n-\tondisk->flags = htons(flags);\n+\tflagsp[0] = htons(flags);\n \tif (ce->ce_flags & CE_EXTENDED) {\n-\t\tstruct ondisk_cache_entry_extended *ondisk2;\n-\t\tondisk2 = (struct ondisk_cache_entry_extended *)ondisk;\n-\t\tondisk2->flags2 = htons((ce->ce_flags & CE_EXTENDED_FLAGS) >> 16);\n+\t\tflagsp[1] = htons((ce->ce_flags & CE_EXTENDED_FLAGS) >> 16);\n \t}\n }\n \n@@ -2566,10 +2553,7 @@ static int ce_write_entry(git_hash_ctx *c, int fd, struct cache_entry *ce,\n \t\tstripped_name = 1;\n \t}\n \n-\tif (ce->ce_flags & CE_EXTENDED)\n-\t\tsize = offsetof(struct ondisk_cache_entry_extended, name);\n-\telse\n-\t\tsize = offsetof(struct ondisk_cache_entry, name);\n+\tsize = offsetof(struct ondisk_cache_entry,data) + ondisk_data_size(ce->ce_flags, 0);\n \n \tif (!previous_name) {\n \t\tint len = ce_namelen(ce);\n@@ -2727,7 +2711,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile,\n \tstruct cache_entry **cache = istate->cache;\n \tint entries = istate->cache_nr;\n \tstruct stat st;\n-\tstruct ondisk_cache_entry_extended ondisk;\n+\tstruct ondisk_cache_entry ondisk;\n \tstruct strbuf previous_name_buf = STRBUF_INIT, *previous_name;\n \tint drop_cache_tree = istate->drop_cache_tree;\n \toff_t offset;\n"},{"id":"369104","messageId":"20190212012256.1005924-29-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 28/31] dir: make untracked cache extension hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:53Z","receivedAt":"2019-02-12T01:23:50Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Instead of using a struct with a flex array member to read and write the\nuntracked cache extension, use a shorter, fixed-length struct and add\nthe name and hash data explicitly.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n dir.c | 28 ++++++++++++++--------------\n 1 file changed, 14 insertions(+), 14 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex b2cabadf25..7503b56918 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2545,13 +2545,9 @@ struct ondisk_untracked_cache {\n \tstruct stat_data info_exclude_stat;\n \tstruct stat_data excludes_file_stat;\n \tuint32_t dir_flags;\n-\tunsigned char info_exclude_sha1[20];\n-\tunsigned char excludes_file_sha1[20];\n-\tchar exclude_per_dir[FLEX_ARRAY];\n };\n \n #define ouc_offset(x) offsetof(struct ondisk_untracked_cache, x)\n-#define ouc_size(len) (ouc_offset(exclude_per_dir) + len + 1)\n \n struct write_data {\n \tint index;\t   /* number of written untracked_cache_dir */\n@@ -2634,20 +2630,21 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra\n \tstruct write_data wd;\n \tunsigned char varbuf[16];\n \tint varint_len;\n-\tsize_t len = strlen(untracked->exclude_per_dir);\n+\tconst unsigned hashsz = the_hash_algo->rawsz;\n \n-\tFLEX_ALLOC_MEM(ouc, exclude_per_dir, untracked->exclude_per_dir, len);\n+\touc = xcalloc(1, sizeof(*ouc));\n \tstat_data_to_disk(&ouc->info_exclude_stat, &untracked->ss_info_exclude.stat);\n \tstat_data_to_disk(&ouc->excludes_file_stat, &untracked->ss_excludes_file.stat);\n-\thashcpy(ouc->info_exclude_sha1, untracked->ss_info_exclude.oid.hash);\n-\thashcpy(ouc->excludes_file_sha1, untracked->ss_excludes_file.oid.hash);\n \touc->dir_flags = htonl(untracked->dir_flags);\n \n \tvarint_len = encode_varint(untracked->ident.len, varbuf);\n \tstrbuf_add(out, varbuf, varint_len);\n \tstrbuf_addbuf(out, &untracked->ident);\n \n-\tstrbuf_add(out, ouc, ouc_size(len));\n+\tstrbuf_add(out, ouc, sizeof(*ouc));\n+\tstrbuf_add(out, untracked->ss_info_exclude.oid.hash, hashsz);\n+\tstrbuf_add(out, untracked->ss_excludes_file.oid.hash, hashsz);\n+\tstrbuf_add(out, untracked->exclude_per_dir, strlen(untracked->exclude_per_dir) + 1);\n \tFREE_AND_NULL(ouc);\n \n \tif (!untracked->root) {\n@@ -2834,6 +2831,9 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long\n \tint ident_len;\n \tssize_t len;\n \tconst char *exclude_per_dir;\n+\tconst unsigned hashsz = the_hash_algo->rawsz;\n+\tconst unsigned offset = sizeof(struct ondisk_untracked_cache);\n+\tconst unsigned exclude_per_dir_offset = offset + 2 * hashsz;\n \n \tif (sz <= 1 || end[-1] != '\\0')\n \t\treturn NULL;\n@@ -2845,7 +2845,7 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long\n \tident = (const char *)next;\n \tnext += ident_len;\n \n-\tif (next + ouc_size(0) > end)\n+\tif (next + exclude_per_dir_offset + 1 > end)\n \t\treturn NULL;\n \n \tuc = xcalloc(1, sizeof(*uc));\n@@ -2853,15 +2853,15 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long\n \tstrbuf_add(&uc->ident, ident, ident_len);\n \tload_oid_stat(&uc->ss_info_exclude,\n \t\t      next + ouc_offset(info_exclude_stat),\n-\t\t      next + ouc_offset(info_exclude_sha1));\n+\t\t      next + offset);\n \tload_oid_stat(&uc->ss_excludes_file,\n \t\t      next + ouc_offset(excludes_file_stat),\n-\t\t      next + ouc_offset(excludes_file_sha1));\n+\t\t      next + offset + hashsz);\n \tuc->dir_flags = get_be32(next + ouc_offset(dir_flags));\n-\texclude_per_dir = (const char *)next + ouc_offset(exclude_per_dir);\n+\texclude_per_dir = (const char *)next + exclude_per_dir_offset;\n \tuc->exclude_per_dir = xstrdup(exclude_per_dir);\n \t/* NUL after exclude_per_dir is covered by sizeof(*ouc) */\n-\tnext += ouc_size(strlen(exclude_per_dir));\n+\tnext += exclude_per_dir_offset + strlen(exclude_per_dir) + 1;\n \tif (next >= end)\n \t\tgoto done2;\n \n"},{"id":"369105","messageId":"20190212012256.1005924-26-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 25/31] archive: convert struct archiver_args to object_id","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:50Z","receivedAt":"2019-02-12T01:23:50Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Change the commit_sha1 member to be called \"commit_oid\" and change it to\nbe a pointer to struct object_id.  Additionally, update two uses of\nGIT_SHA1_HEXSZ to use the_hash_algo instead.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n archive-tar.c |  6 +++---\n archive-zip.c | 10 +++++-----\n archive.c     |  8 ++++----\n archive.h     |  2 +-\n 4 files changed, 13 insertions(+), 13 deletions(-)\n\ndiff --git a/archive-tar.c b/archive-tar.c\nindex a5ba55c11e..3e53aac1e6 100644\n--- a/archive-tar.c\n+++ b/archive-tar.c\n@@ -326,14 +326,14 @@ static int write_tar_entry(struct archiver_args *args,\n \n static void write_global_extended_header(struct archiver_args *args)\n {\n-\tconst unsigned char *hash = args->commit_sha1;\n+\tconst struct object_id *oid = args->commit_oid;\n \tstruct strbuf ext_header = STRBUF_INIT;\n \tstruct ustar_header header;\n \tunsigned int mode;\n \n-\tif (hash)\n+\tif (oid)\n \t\tstrbuf_append_ext_header(&ext_header, \"comment\",\n-\t\t\t\t\t hash_to_hex(hash),\n+\t\t\t\t\t oid_to_hex(oid),\n \t\t\t\t\t the_hash_algo->hexsz);\n \tif (args->time > USTAR_MAX_MTIME) {\n \t\tstrbuf_append_ext_header_uint(&ext_header, \"mtime\",\ndiff --git a/archive-zip.c b/archive-zip.c\nindex 155ee4a779..4d66b5be6e 100644\n--- a/archive-zip.c\n+++ b/archive-zip.c\n@@ -577,7 +577,7 @@ static void write_zip64_trailer(void)\n \twrite_or_die(1, &locator64, ZIP64_DIR_TRAILER_LOCATOR_SIZE);\n }\n \n-static void write_zip_trailer(const unsigned char *sha1)\n+static void write_zip_trailer(const struct object_id *oid)\n {\n \tstruct zip_dir_trailer trailer;\n \tint clamped = 0;\n@@ -590,14 +590,14 @@ static void write_zip_trailer(const unsigned char *sha1)\n \tcopy_le16_clamp(trailer.entries, zip_dir_entries, &clamped);\n \tcopy_le32(trailer.size, zip_dir.len);\n \tcopy_le32_clamp(trailer.offset, zip_offset, &clamped);\n-\tcopy_le16(trailer.comment_length, sha1 ? GIT_SHA1_HEXSZ : 0);\n+\tcopy_le16(trailer.comment_length, oid ? the_hash_algo->hexsz : 0);\n \n \twrite_or_die(1, zip_dir.buf, zip_dir.len);\n \tif (clamped)\n \t\twrite_zip64_trailer();\n \twrite_or_die(1, &trailer, ZIP_DIR_TRAILER_SIZE);\n-\tif (sha1)\n-\t\twrite_or_die(1, sha1_to_hex(sha1), GIT_SHA1_HEXSZ);\n+\tif (oid)\n+\t\twrite_or_die(1, oid_to_hex(oid), the_hash_algo->hexsz);\n }\n \n static void dos_time(timestamp_t *timestamp, int *dos_date, int *dos_time)\n@@ -635,7 +635,7 @@ static int write_zip_archive(const struct archiver *ar,\n \n \terr = write_archive_entries(args, write_zip_entry);\n \tif (!err)\n-\t\twrite_zip_trailer(args->commit_sha1);\n+\t\twrite_zip_trailer(args->commit_oid);\n \n \tstrbuf_release(&zip_dir);\n \ndiff --git a/archive.c b/archive.c\nindex 1f98324a93..f2c78a2712 100644\n--- a/archive.c\n+++ b/archive.c\n@@ -380,7 +380,7 @@ static void parse_treeish_arg(const char **argv,\n \t\tint remote)\n {\n \tconst char *name = argv[0];\n-\tconst unsigned char *commit_sha1;\n+\tconst struct object_id *commit_oid;\n \ttime_t archive_time;\n \tstruct tree *tree;\n \tconst struct commit *commit;\n@@ -402,10 +402,10 @@ static void parse_treeish_arg(const char **argv,\n \n \tcommit = lookup_commit_reference_gently(ar_args->repo, &oid, 1);\n \tif (commit) {\n-\t\tcommit_sha1 = commit->object.oid.hash;\n+\t\tcommit_oid = &commit->object.oid;\n \t\tarchive_time = commit->date;\n \t} else {\n-\t\tcommit_sha1 = NULL;\n+\t\tcommit_oid = NULL;\n \t\tarchive_time = time(NULL);\n \t}\n \n@@ -426,7 +426,7 @@ static void parse_treeish_arg(const char **argv,\n \t\ttree = parse_tree_indirect(&tree_oid);\n \t}\n \tar_args->tree = tree;\n-\tar_args->commit_sha1 = commit_sha1;\n+\tar_args->commit_oid = commit_oid;\n \tar_args->commit = commit;\n \tar_args->time = archive_time;\n }\ndiff --git a/archive.h b/archive.h\nindex 21ac010699..dd022a6b46 100644\n--- a/archive.h\n+++ b/archive.h\n@@ -11,7 +11,7 @@ struct archiver_args {\n \tconst char *base;\n \tsize_t baselen;\n \tstruct tree *tree;\n-\tconst unsigned char *commit_sha1;\n+\tconst struct object_id *commit_oid;\n \tconst struct commit *commit;\n \ttimestamp_t time;\n \tstruct pathspec pathspec;\n"},{"id":"369106","messageId":"20190212012256.1005924-27-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 26/31] refspec: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:51Z","receivedAt":"2019-02-12T01:23:54Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Switch a use of GIT_SHA1_HEXSZ to use the_hash_algo.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n refspec.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refspec.c b/refspec.c\nindex f529092fd6..9a9bf21934 100644\n--- a/refspec.c\n+++ b/refspec.c\n@@ -72,7 +72,7 @@ static int parse_refspec(struct refspec_item *item, const char *refspec, int fet\n \t\t/* LHS */\n \t\tif (!*item->src)\n \t\t\t; /* empty is ok; it means \"HEAD\" */\n-\t\telse if (llen == GIT_SHA1_HEXSZ && !get_oid_hex(item->src, &unused))\n+\t\telse if (llen == the_hash_algo->hexsz && !get_oid_hex(item->src, &unused))\n \t\t\titem->exact_sha1 = 1; /* ok */\n \t\telse if (!check_refname_format(item->src, flags))\n \t\t\t; /* valid looking ref is ok */\n"},{"id":"369107","messageId":"20190212012256.1005924-32-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 31/31] gitweb: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:56Z","receivedAt":"2019-02-12T01:23:55Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Gitweb has several hard-coded 40 values throughout it to check for\nvalues that are passed in or acquired from Git.  To simplify the code,\nintroduce a regex variable that matches either exactly 40 or exactly 64\nhex characters, and use this variable anywhere we would have previously\nhard-coded a 40 in a regex.\n\nSimilarly, switch the code that looks for deleted diffinfo information\nto look for either 40 or 64 zeros, and update one piece of code to use\nthis function.  Finally, when formatting a log line, allow an\nabbreviated describe output to contain up to 64 characters.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n gitweb/gitweb.perl | 63 ++++++++++++++++++++++++----------------------\n 1 file changed, 33 insertions(+), 30 deletions(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 2594a4badb..aec8ca3256 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -788,6 +788,9 @@ sub check_loadavg {\n # ======================================================================\n # input validation and dispatch\n \n+# A regex matching a valid object ID.\n+our $oid_regex = qr/(?:[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})?)/;\n+\n # input parameters can be collected from a variety of sources (presently, CGI\n # and PATH_INFO), so we define an %input_params hash that collects them all\n # together during validation: this allows subsequent uses (e.g. href()) to be\n@@ -1516,7 +1519,7 @@ sub is_valid_refname {\n \n \treturn undef unless defined $input;\n \t# textual hashes are O.K.\n-\tif ($input =~ m/^[0-9a-fA-F]{40}$/) {\n+\tif ($input =~ m/^$oid_regex$/) {\n \t\treturn 1;\n \t}\n \t# it must be correct pathname\n@@ -2037,10 +2040,10 @@ sub format_log_line_html {\n             (?<!-) # see strbuf_check_tag_ref(). Tags can't start with -\n             [A-Za-z0-9.-]+\n             (?!\\.) # refs can't end with \".\", see check_refname_format()\n-            -g[0-9a-fA-F]{7,40}\n+            -g[0-9a-fA-F]{7,64}\n             |\n             # Just a normal looking Git SHA1\n-            [0-9a-fA-F]{7,40}\n+            [0-9a-fA-F]{7,64}\n         )\n         \\b\n     }{\n@@ -2286,7 +2289,7 @@ sub format_extended_diff_header_line {\n \t\t         ')</span>';\n \t}\n \t# match <hash>\n-\tif ($line =~ m/^index [0-9a-fA-F]{40},[0-9a-fA-F]{40}/) {\n+\tif ($line =~ m/^index $oid_regex,$oid_regex/) {\n \t\t# can match only for combined diff\n \t\t$line = 'index ';\n \t\tfor (my $i = 0; $i < $diffinfo->{'nparents'}; $i++) {\n@@ -2308,7 +2311,7 @@ sub format_extended_diff_header_line {\n \t\t\t$line .= '0' x 7;\n \t\t}\n \n-\t} elsif ($line =~ m/^index [0-9a-fA-F]{40}..[0-9a-fA-F]{40}/) {\n+\t} elsif ($line =~ m/^index $oid_regex..$oid_regex/) {\n \t\t# can match only for ordinary diff\n \t\tmy ($from_link, $to_link);\n \t\tif ($from->{'href'}) {\n@@ -2834,7 +2837,7 @@ sub git_get_hash_by_path {\n \t}\n \n \t#'100644 blob 0fa3f3a66fb6a137f6ec2c19351ed4d807070ffa\tpanic.c'\n-\t$line =~ m/^([0-9]+) (.+) ([0-9a-fA-F]{40})\\t/;\n+\t$line =~ m/^([0-9]+) (.+) ($oid_regex)\\t/;\n \tif (defined $type && $type ne $2) {\n \t\t# type doesn't match\n \t\treturn undef;\n@@ -3333,7 +3336,7 @@ sub git_get_references {\n \n \twhile (my $line = <$fd>) {\n \t\tchomp $line;\n-\t\tif ($line =~ m!^([0-9a-fA-F]{40})\\srefs/($type.*)$!) {\n+\t\tif ($line =~ m!^($oid_regex)\\srefs/($type.*)$!) {\n \t\t\tif (defined $refs{$1}) {\n \t\t\t\tpush @{$refs{$1}}, $2;\n \t\t\t} else {\n@@ -3407,7 +3410,7 @@ sub parse_tag {\n \t$tag{'id'} = $tag_id;\n \twhile (my $line = <$fd>) {\n \t\tchomp $line;\n-\t\tif ($line =~ m/^object ([0-9a-fA-F]{40})$/) {\n+\t\tif ($line =~ m/^object ($oid_regex)$/) {\n \t\t\t$tag{'object'} = $1;\n \t\t} elsif ($line =~ m/^type (.+)$/) {\n \t\t\t$tag{'type'} = $1;\n@@ -3451,15 +3454,15 @@ sub parse_commit_text {\n \t}\n \n \tmy $header = shift @commit_lines;\n-\tif ($header !~ m/^[0-9a-fA-F]{40}/) {\n+\tif ($header !~ m/^$oid_regex/) {\n \t\treturn;\n \t}\n \t($co{'id'}, my @parents) = split ' ', $header;\n \twhile (my $line = shift @commit_lines) {\n \t\tlast if $line eq \"\\n\";\n-\t\tif ($line =~ m/^tree ([0-9a-fA-F]{40})$/) {\n+\t\tif ($line =~ m/^tree ($oid_regex)$/) {\n \t\t\t$co{'tree'} = $1;\n-\t\t} elsif ((!defined $withparents) && ($line =~ m/^parent ([0-9a-fA-F]{40})$/)) {\n+\t\t} elsif ((!defined $withparents) && ($line =~ m/^parent ($oid_regex)$/)) {\n \t\t\tpush @parents, $1;\n \t\t} elsif ($line =~ m/^author (.*) ([0-9]+) (.*)$/) {\n \t\t\t$co{'author'} = to_utf8($1);\n@@ -3591,7 +3594,7 @@ sub parse_difftree_raw_line {\n \n \t# ':100644 100644 03b218260e99b78c6df0ed378e59ed9205ccc96d 3b93d5e7cc7f7dd4ebed13a5cc1a4ad976fc94d8 M\tls-files.c'\n \t# ':100644 100644 7f9281985086971d3877aca27704f2aaf9c448ce bc190ebc71bbd923f2b728e505408f5e54bd073a M\trev-tree.c'\n-\tif ($line =~ m/^:([0-7]{6}) ([0-7]{6}) ([0-9a-fA-F]{40}) ([0-9a-fA-F]{40}) (.)([0-9]{0,3})\\t(.*)$/) {\n+\tif ($line =~ m/^:([0-7]{6}) ([0-7]{6}) ($oid_regex) ($oid_regex) (.)([0-9]{0,3})\\t(.*)$/) {\n \t\t$res{'from_mode'} = $1;\n \t\t$res{'to_mode'} = $2;\n \t\t$res{'from_id'} = $3;\n@@ -3606,7 +3609,7 @@ sub parse_difftree_raw_line {\n \t}\n \t# '::100755 100755 100755 60e79ca1b01bc8b057abe17ddab484699a7f5fdb 94067cc5f73388f33722d52ae02f44692bc07490 94067cc5f73388f33722d52ae02f44692bc07490 MR\tgit-gui/git-gui.sh'\n \t# combined diff (for merge commit)\n-\telsif ($line =~ s/^(::+)((?:[0-7]{6} )+)((?:[0-9a-fA-F]{40} )+)([a-zA-Z]+)\\t(.*)$//) {\n+\telsif ($line =~ s/^(::+)((?:[0-7]{6} )+)((?:$oid_regex )+)([a-zA-Z]+)\\t(.*)$//) {\n \t\t$res{'nparents'}  = length($1);\n \t\t$res{'from_mode'} = [ split(' ', $2) ];\n \t\t$res{'to_mode'} = pop @{$res{'from_mode'}};\n@@ -3616,7 +3619,7 @@ sub parse_difftree_raw_line {\n \t\t$res{'to_file'} = unquote($5);\n \t}\n \t# 'c512b523472485aef4fff9e57b229d9d243c967f'\n-\telsif ($line =~ m/^([0-9a-fA-F]{40})$/) {\n+\telsif ($line =~ m/^($oid_regex)$/) {\n \t\t$res{'commit'} = $1;\n \t}\n \n@@ -3644,7 +3647,7 @@ sub parse_ls_tree_line {\n \n \tif ($opts{'-l'}) {\n \t\t#'100644 blob 0fa3f3a66fb6a137f6ec2c19351ed4d807070ffa   16717\tpanic.c'\n-\t\t$line =~ m/^([0-9]+) (.+) ([0-9a-fA-F]{40}) +(-|[0-9]+)\\t(.+)$/s;\n+\t\t$line =~ m/^([0-9]+) (.+) ($oid_regex) +(-|[0-9]+)\\t(.+)$/s;\n \n \t\t$res{'mode'} = $1;\n \t\t$res{'type'} = $2;\n@@ -3657,7 +3660,7 @@ sub parse_ls_tree_line {\n \t\t}\n \t} else {\n \t\t#'100644 blob 0fa3f3a66fb6a137f6ec2c19351ed4d807070ffa\tpanic.c'\n-\t\t$line =~ m/^([0-9]+) (.+) ([0-9a-fA-F]{40})\\t(.+)$/s;\n+\t\t$line =~ m/^([0-9]+) (.+) ($oid_regex)\\t(.+)$/s;\n \n \t\t$res{'mode'} = $1;\n \t\t$res{'type'} = $2;\n@@ -4799,7 +4802,7 @@ sub fill_from_file_info {\n sub is_deleted {\n \tmy $diffinfo = shift;\n \n-\treturn $diffinfo->{'to_id'} eq ('0' x 40);\n+\treturn $diffinfo->{'to_id'} eq ('0' x 40) || $diffinfo->{'to_id'} eq ('0' x 64);\n }\n \n # does patch correspond to [previous] difftree raw line\n@@ -6285,7 +6288,7 @@ sub git_search_changes {\n \t\t\t              -class => \"list subject\"},\n \t\t\t              chop_and_escape_str($co{'title'}, 50) . \"<br/>\");\n \t\t} elsif (defined $set{'to_id'}) {\n-\t\t\tnext if ($set{'to_id'} =~ m/^0{40}$/);\n+\t\t\tnext if is_deleted(\\%set);\n \n \t\t\tprint $cgi->a({-href => href(action=>\"blob\", hash_base=>$co{'id'},\n \t\t\t                             hash=>$set{'to_id'}, file_name=>$set{'to_file'}),\n@@ -6829,7 +6832,7 @@ sub git_blame_common {\n \t\t\t# the header: <SHA-1> <src lineno> <dst lineno> [<lines in group>]\n \t\t\t# no <lines in group> for subsequent lines in group of lines\n \t\t\tmy ($full_rev, $orig_lineno, $lineno, $group_size) =\n-\t\t\t   ($line =~ /^([0-9a-f]{40}) (\\d+) (\\d+)(?: (\\d+))?$/);\n+\t\t\t   ($line =~ /^($oid_regex) (\\d+) (\\d+)(?: (\\d+))?$/);\n \t\t\tif (!exists $metainfo{$full_rev}) {\n \t\t\t\t$metainfo{$full_rev} = { 'nprevious' => 0 };\n \t\t\t}\n@@ -6879,7 +6882,7 @@ sub git_blame_common {\n \t\t\t}\n \t\t\t# 'previous' <sha1 of parent commit> <filename at commit>\n \t\t\tif (exists $meta->{'previous'} &&\n-\t\t\t    $meta->{'previous'} =~ /^([a-fA-F0-9]{40}) (.*)$/) {\n+\t\t\t    $meta->{'previous'} =~ /^($oid_regex) (.*)$/) {\n \t\t\t\t$meta->{'parent'} = $1;\n \t\t\t\t$meta->{'file_parent'} = unquote($2);\n \t\t\t}\n@@ -6996,7 +6999,7 @@ sub git_blob_plain {\n \t\t} else {\n \t\t\tdie_error(400, \"No file name defined\");\n \t\t}\n-\t} elsif ($hash =~ m/^[0-9a-fA-F]{40}$/) {\n+\t} elsif ($hash =~ m/^$oid_regex$/) {\n \t\t# blobs defined by non-textual hash id's can be cached\n \t\t$expires = \"+1d\";\n \t}\n@@ -7057,7 +7060,7 @@ sub git_blob {\n \t\t} else {\n \t\t\tdie_error(400, \"No file name defined\");\n \t\t}\n-\t} elsif ($hash =~ m/^[0-9a-fA-F]{40}$/) {\n+\t} elsif ($hash =~ m/^$oid_regex$/) {\n \t\t# blobs defined by non-textual hash id's can be cached\n \t\t$expires = \"+1d\";\n \t}\n@@ -7515,7 +7518,7 @@ sub git_commit {\n \n \t# non-textual hash id's can be cached\n \tmy $expires;\n-\tif ($hash =~ m/^[0-9a-fA-F]{40}$/) {\n+\tif ($hash =~ m/^$oid_regex$/) {\n \t\t$expires = \"+1d\";\n \t}\n \tmy $refs = git_get_references();\n@@ -7609,7 +7612,7 @@ sub git_object {\n \t\tclose $fd;\n \n \t\t#'100644 blob 0fa3f3a66fb6a137f6ec2c19351ed4d807070ffa\tpanic.c'\n-\t\tunless ($line && $line =~ m/^([0-9]+) (.+) ([0-9a-fA-F]{40})\\t/) {\n+\t\tunless ($line && $line =~ m/^([0-9]+) (.+) ($oid_regex)\\t/) {\n \t\t\tdie_error(404, \"File or directory for given base does not exist\");\n \t\t}\n \t\t$type = $2;\n@@ -7649,7 +7652,7 @@ sub git_blobdiff {\n \t\t\t\tor die_error(404, \"Blob diff not found\");\n \n \t\t} elsif (defined $hash &&\n-\t\t         $hash =~ /[0-9a-fA-F]{40}/) {\n+\t\t         $hash =~ $oid_regex) {\n \t\t\t# try to find filename from $hash\n \n \t\t\t# read filtered raw output\n@@ -7659,7 +7662,7 @@ sub git_blobdiff {\n \t\t\t@difftree =\n \t\t\t\t# ':100644 100644 03b21826... 3b93d5e7... M\tls-files.c'\n \t\t\t\t# $hash == to_id\n-\t\t\t\tgrep { /^:[0-7]{6} [0-7]{6} [0-9a-fA-F]{40} $hash/ }\n+\t\t\t\tgrep { /^:[0-7]{6} [0-7]{6} $oid_regex $hash/ }\n \t\t\t\tmap { chomp; $_ } <$fd>;\n \t\t\tclose $fd\n \t\t\t\tor die_error(404, \"Reading git-diff-tree failed\");\n@@ -7682,8 +7685,8 @@ sub git_blobdiff {\n \t\t$hash        ||= $diffinfo{'to_id'};\n \n \t\t# non-textual hash id's can be cached\n-\t\tif ($hash_base =~ m/^[0-9a-fA-F]{40}$/ &&\n-\t\t    $hash_parent_base =~ m/^[0-9a-fA-F]{40}$/) {\n+\t\tif ($hash_base =~ m/^$oid_regex$/ &&\n+\t\t    $hash_parent_base =~ m/^$oid_regex$/) {\n \t\t\t$expires = '+1d';\n \t\t}\n \n@@ -7819,7 +7822,7 @@ sub git_commitdiff {\n \t\t    $hash_parent ne '-c' && $hash_parent ne '--cc') {\n \t\t\t# commitdiff with two commits given\n \t\t\tmy $hash_parent_short = $hash_parent;\n-\t\t\tif ($hash_parent =~ m/^[0-9a-fA-F]{40}$/) {\n+\t\t\tif ($hash_parent =~ m/^$oid_regex$/) {\n \t\t\t\t$hash_parent_short = substr($hash_parent, 0, 7);\n \t\t\t}\n \t\t\t$formats_nav .=\n@@ -7928,7 +7931,7 @@ sub git_commitdiff {\n \n \t# non-textual hash id's can be cached\n \tmy $expires;\n-\tif ($hash =~ m/^[0-9a-fA-F]{40}$/) {\n+\tif ($hash =~ m/^$oid_regex$/) {\n \t\t$expires = \"+1d\";\n \t}\n \n"},{"id":"369108","messageId":"20190212012256.1005924-31-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 30/31] Git.pm: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:55Z","receivedAt":"2019-02-12T01:23:55Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"The cat_blob function was matching on exactly 40 hex characters.  This\nwon't work with SHA-256, which uses 64-character hex object IDs.  While\nit should be fine to simply match any number of hex characters since the\noutput is space delimited, be extra safe by matching either exactly 40\nor exactly 64 hex characters.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n perl/Git.pm | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/perl/Git.pm b/perl/Git.pm\nindex d856930b2e..62c472e0ce 100644\n--- a/perl/Git.pm\n+++ b/perl/Git.pm\n@@ -980,7 +980,7 @@ sub cat_blob {\n \t\treturn -1;\n \t}\n \n-\tif ($description !~ /^[0-9a-fA-F]{40} \\S+ (\\d+)$/) {\n+\tif ($description !~ /^[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})? \\S+ (\\d+)$/) {\n \t\tcarp \"Unexpected result returned from git cat-file\";\n \t\treturn -1;\n \t}\n"},{"id":"369109","messageId":"20190212012256.1005924-25-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 24/31] archive-tar: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:49Z","receivedAt":"2019-02-12T01:23:57Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Make the tar generation code hash size independent by using\nthe_hash_algo.  Make the tar parsing code compute the header value based\non the hash algorithm in use. Update a variable name and switch to\nhash_to_hex.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n archive-tar.c               |  7 ++++---\n builtin/get-tar-commit-id.c | 11 +++++++++--\n 2 files changed, 13 insertions(+), 5 deletions(-)\n\ndiff --git a/archive-tar.c b/archive-tar.c\nindex 4aabd566fb..a5ba55c11e 100644\n--- a/archive-tar.c\n+++ b/archive-tar.c\n@@ -326,14 +326,15 @@ static int write_tar_entry(struct archiver_args *args,\n \n static void write_global_extended_header(struct archiver_args *args)\n {\n-\tconst unsigned char *sha1 = args->commit_sha1;\n+\tconst unsigned char *hash = args->commit_sha1;\n \tstruct strbuf ext_header = STRBUF_INIT;\n \tstruct ustar_header header;\n \tunsigned int mode;\n \n-\tif (sha1)\n+\tif (hash)\n \t\tstrbuf_append_ext_header(&ext_header, \"comment\",\n-\t\t\t\t\t sha1_to_hex(sha1), 40);\n+\t\t\t\t\t hash_to_hex(hash),\n+\t\t\t\t\t the_hash_algo->hexsz);\n \tif (args->time > USTAR_MAX_MTIME) {\n \t\tstrbuf_append_ext_header_uint(&ext_header, \"mtime\",\n \t\t\t\t\t      args->time);\ndiff --git a/builtin/get-tar-commit-id.c b/builtin/get-tar-commit-id.c\nindex 2706fcfaf2..2760549e91 100644\n--- a/builtin/get-tar-commit-id.c\n+++ b/builtin/get-tar-commit-id.c\n@@ -5,6 +5,7 @@\n #include \"commit.h\"\n #include \"tar.h\"\n #include \"builtin.h\"\n+#include \"strbuf.h\"\n #include \"quote.h\"\n \n static const char builtin_get_tar_commit_id_usage[] =\n@@ -21,6 +22,8 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n \tchar *content = buffer + RECORDSIZE;\n \tconst char *comment;\n \tssize_t n;\n+\tchar *hdrprefix;\n+\tint ret;\n \n \tif (argc != 1)\n \t\tusage(builtin_get_tar_commit_id_usage);\n@@ -32,10 +35,14 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(\"git get-tar-commit-id: EOF before reading tar header\");\n \tif (header->typeflag[0] != 'g')\n \t\treturn 1;\n-\tif (!skip_prefix(content, \"52 comment=\", &comment))\n+\n+\thdrprefix = xstrfmt(\"%zu comment=\", the_hash_algo->hexsz + strlen(\" comment=\") + 2 + 1);\n+\tret = skip_prefix(content, hdrprefix, &comment);\n+\tfree(hdrprefix);\n+\tif (!ret)\n \t\treturn 1;\n \n-\tif (write_in_full(1, comment, 41) < 0)\n+\tif (write_in_full(1, comment, the_hash_algo->hexsz + 1) < 0)\n \t\tdie_errno(\"git get-tar-commit-id: write error\");\n \n \treturn 0;\n"},{"id":"369110","messageId":"20190212012256.1005924-28-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 27/31] builtin/difftool: use parse_oid_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:52Z","receivedAt":"2019-02-12T01:23:58Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Instead of using get_oid_hex and adding constants to the result, use\nparse_oid_hex to make this code independent of the hash size.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n builtin/difftool.c | 6 ++----\n 1 file changed, 2 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/difftool.c b/builtin/difftool.c\nindex a3ea60ea71..1b648226dc 100644\n--- a/builtin/difftool.c\n+++ b/builtin/difftool.c\n@@ -65,14 +65,12 @@ static int parse_index_info(char *p, int *mode1, int *mode2,\n \t*mode2 = (int)strtol(p + 1, &p, 8);\n \tif (*p != ' ')\n \t\treturn error(\"expected ' ', got '%c'\", *p);\n-\tif (get_oid_hex(++p, oid1))\n+\tif (parse_oid_hex(++p, oid1, (const char **)&p))\n \t\treturn error(\"expected object ID, got '%s'\", p + 1);\n-\tp += GIT_SHA1_HEXSZ;\n \tif (*p != ' ')\n \t\treturn error(\"expected ' ', got '%c'\", *p);\n-\tif (get_oid_hex(++p, oid2))\n+\tif (parse_oid_hex(++p, oid2, (const char **)&p))\n \t\treturn error(\"expected object ID, got '%s'\", p + 1);\n-\tp += GIT_SHA1_HEXSZ;\n \tif (*p != ' ')\n \t\treturn error(\"expected ' ', got '%c'\", *p);\n \t*status = *++p;\n"},{"id":"369111","messageId":"20190212012256.1005924-8-sandals@crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"[PATCH 07/31] notes-merge: switch to use the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:22:32Z","receivedAt":"2019-02-12T01:23:59Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"Switch from using GIT_SHA1_HEXSZ to GIT_MAX_HEXSZ and the_hash_algo so\nthat the code works with any hash algorithm.\n\nSigned-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n---\n notes-merge.c | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/notes-merge.c b/notes-merge.c\nindex 280aa8e6c1..2fe724f1cf 100644\n--- a/notes-merge.c\n+++ b/notes-merge.c\n@@ -29,14 +29,14 @@ void init_notes_merge_options(struct repository *r,\n \n static int path_to_oid(const char *path, struct object_id *oid)\n {\n-\tchar hex_oid[GIT_SHA1_HEXSZ];\n+\tchar hex_oid[GIT_MAX_HEXSZ];\n \tint i = 0;\n-\twhile (*path && i < GIT_SHA1_HEXSZ) {\n+\twhile (*path && i < the_hash_algo->hexsz) {\n \t\tif (*path != '/')\n \t\t\thex_oid[i++] = *path;\n \t\tpath++;\n \t}\n-\tif (*path || i != GIT_SHA1_HEXSZ)\n+\tif (*path || i != the_hash_algo->hexsz)\n \t\treturn -1;\n \treturn get_oid_hex(hex_oid, oid);\n }\n"},{"id":"369112","messageId":"CAPig+cRQrkeD6rL8M9pL4kjPTwbPdjUYyH4Kes3jk9Wn0O-NtA@mail.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-9-sandals@crustytoothpaste.net","subject":"Re: [PATCH 08/31] notes: make hash size independent","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-02-12T01:37:49Z","receivedAt":"2019-02-12T01:38:02Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> Switch out various uses of the GIT_SHA1_* constants with GIT_MAX_*\n> constants for allocations and the_hash_algo for general parsing.  Update\n> a comment to no longer be SHA-1 specific.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n> diff --git a/notes.c b/notes.c\n> @@ -527,15 +529,15 @@ static unsigned char determine_fanout(struct int_node *tree, unsigned char n,\n> -/* hex SHA1 + 19 * '/' + NUL */\n> -#define FANOUT_PATH_MAX GIT_SHA1_HEXSZ + FANOUT_PATH_SEPARATORS + 1\n> +/* hex oid + one slash between each pair + NUL */\n> +#define FANOUT_PATH_MAX GIT_MAX_HEXSZ + FANOUT_PATH_SEPARATORS_MAX + 1\n\nI had trouble understanding to what pair you are referring. I _think_\nyou mean \"pair of hex digits\". If so, perhaps:\n\n    /* hex oid + '/' between each pair of hex digits + NUL */\n\n(but not worth a re-roll).\n"},{"id":"369113","messageId":"20190212014215.GE684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"CAPig+cRQrkeD6rL8M9pL4kjPTwbPdjUYyH4Kes3jk9Wn0O-NtA@mail.gmail.com","subject":"Re: [PATCH 08/31] notes: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T01:42:15Z","receivedAt":"2019-02-12T01:42:23Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Mon, Feb 11, 2019 at 08:37:49PM -0500, Eric Sunshine wrote:\n> On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n> <sandals@crustytoothpaste.net> wrote:\n> > diff --git a/notes.c b/notes.c\n> > @@ -527,15 +529,15 @@ static unsigned char determine_fanout(struct int_node *tree, unsigned char n,\n> > -/* hex SHA1 + 19 * '/' + NUL */\n> > -#define FANOUT_PATH_MAX GIT_SHA1_HEXSZ + FANOUT_PATH_SEPARATORS + 1\n> > +/* hex oid + one slash between each pair + NUL */\n> > +#define FANOUT_PATH_MAX GIT_MAX_HEXSZ + FANOUT_PATH_SEPARATORS_MAX + 1\n> \n> I had trouble understanding to what pair you are referring. I _think_\n> you mean \"pair of hex digits\". If so, perhaps:\n> \n>     /* hex oid + '/' between each pair of hex digits + NUL */\n> \n> (but not worth a re-roll).\n\nYes, I did. I fully expect to do at least one reroll here, so I'll\nupdate it when I do.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369121","messageId":"CAPig+cS3i2FWf6mFmpPpctyJJbx4xdHgWXrveRUkjMHct5SbJg@mail.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-11-sandals@crustytoothpaste.net","subject":"Re: [PATCH 10/31] object-store: rename and expand packed_git's sha1 member","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-02-12T03:32:15Z","receivedAt":"2019-02-12T03:32:28Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> This member is used to represent the pack checksum of the pack in\n> question.  Expand this member to be GIT_MAX_RAWSZ bytes in length so it\n> works with longer hashes and rename it to be \"hash\" instead of \"sha1\".\n> [...]\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n> diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\n> @@ -689,7 +689,7 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)\n>         while (pl) {\n>                 printf(\"%s\\n%s\\n\",\n> -                      sha1_pack_index_name(pl->pack->sha1),\n> +                      sha1_pack_index_name(pl->pack->hash),\n\nI guess there is no oid_pack_index_name() function yet?\n\n> diff --git a/http-walker.c b/http-walker.c\n> @@ -434,7 +434,7 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo, unsigne\n>         if (walker->get_verbosely) {\n>                 fprintf(stderr, \"Getting pack %s\\n\",\n> -                       sha1_to_hex(target->sha1));\n> +                       sha1_to_hex(target->hash));\n\nShould this have become oid_to_hex()?\n\n>                 fprintf(stderr, \" which contains %s\\n\",\n>                         sha1_to_hex(sha1));\n\nThe patch isn't touching this sha1 yet, so this is okay(?).\n\n> diff --git a/http.c b/http.c\n> @@ -2262,10 +2262,10 @@ struct http_pack_request *new_http_pack_request(\n>         strbuf_addf(&buf, \"objects/pack/pack-%s.pack\",\n> -               sha1_to_hex(target->sha1));\n> +               sha1_to_hex(target->hash));\n\noid_to_hex()?\n\n> @@ -2289,7 +2289,8 @@ struct http_pack_request *new_http_pack_request(\n>                         fprintf(stderr,\n>                                 \"Resuming fetch of pack %s at byte %\"PRIuMAX\"\\n\",\n> -                               sha1_to_hex(target->sha1), (uintmax_t)prev_posn);\n> +                               sha1_to_hex(target->hash),\n\noid_to_hex()?\n\n> diff --git a/packfile.c b/packfile.c\n> @@ -722,8 +722,8 @@ struct packed_git *add_packed_git(const char *path, size_t path_len, int local)\n>         if (path_len < the_hash_algo->hexsz ||\n> -           get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->sha1))\n> -               hashclr(p->sha1);\n> +           get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))\n> +               hashclr(p->hash);\n\nget_oid_hex()?\n"},{"id":"369122","messageId":"CAPig+cQivuPjDztK30QO0xKx1Tx+nYDPzTvgXvi9WVyRX5-aRQ@mail.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-13-sandals@crustytoothpaste.net","subject":"Re: [PATCH 12/31] fast-import: make hash-size independent","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-02-12T03:44:12Z","receivedAt":"2019-02-12T03:44:26Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> Replace several uses of GIT_SHA1_HEXSZ and 40-based constants with\n> references to the_hash_algo.  Update the note handling code here to\n> compute path sizes based on GIT_MAX_RAWSZ as well.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n> diff --git a/fast-import.c b/fast-import.c\n> @@ -2047,7 +2047,8 @@ static uintmax_t do_change_note_fanout(\n> -       char realpath[60];\n> +       char realpath[GIT_MAX_RAWSZ * 3];\n\nI wonder if the fixed multiplier deserves a comment explaining that\nthis is reserving enough space for a hex representation with '/'\nbetween each digit pair plus NUL. Which leads to the next question: Is\nthere is GIT_MAX_HEXSZ constant? If so, this might be more clearly\nrepresented (or not) by taking advantage of that value.\n\nAlso, there are a number of hardcoded 60's in the code earlier in this\nfile, such as:\n\n    if ((max_packsize && (pack_size + 60 + len) > max_packsize)\n        || (pack_size + 60 + len) < pack_size)\n        cycle_packfile();\n\nIs that just a coincidence or is it related to the 60 characters\nallocated for 'realpath'?\n\n> @@ -2405,7 +2406,7 @@ static void note_change_n(const char *p, struct branch *b, unsigned char *old_fa\n>                 char *buf = read_object_with_reference(&commit_oid,\n>                                                        commit_type, &size,\n>                                                        &commit_oid);\n> -               if (!buf || size < 46)\n> +               if (!buf || size < the_hash_algo->hexsz)\n\nWhat exactly did the 46 represent and how does it relate to 'hexsz'?\nStated differently, why didn't this become:\n\n    the_hash_algo->hexsz + 6'\n\n?\n\n> @@ -2456,7 +2457,7 @@ static void file_change_deleteall(struct branch *b)\n>  static void parse_from_commit(struct branch *b, char *buf, unsigned long size)\n>  {\n> -       if (!buf || size < GIT_SHA1_HEXSZ + 6)\n> +       if (!buf || size < the_hash_algo->hexsz + 6)\n\n...as it seems to have here...\n\n> @@ -2555,7 +2556,7 @@ static struct hash_list *parse_merge(unsigned int *count)\n> -                       if (!buf || size < 46)\n> +                       if (!buf || size < the_hash_algo->hexsz)\n\n...but not here.\n"},{"id":"369123","messageId":"CAPig+cRv04mcbxNUn6bRXbJiQvgj2cpwDGLJP=7NdmZ2LXJJfQ@mail.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-16-sandals@crustytoothpaste.net","subject":"Re: [PATCH 15/31] builtin/pull: make hash-size independent","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-02-12T03:47:46Z","receivedAt":"2019-02-12T03:48:00Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> Instead of using get_oid_hex and GIT_SHA1_HEXSZ, use parse_oid_hex to\n> avoid the need for a constant and simplify the code.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n> diff --git a/builtin/pull.c b/builtin/pull.c\n> @@ -365,9 +365,10 @@ static void get_merge_heads(struct oid_array *merge_heads)\n>         while (strbuf_getline_lf(&sb, fp) != EOF) {\n> -               if (get_oid_hex(sb.buf, &oid))\n> +               const char *p;\n> +               if (parse_oid_hex(sb.buf, &oid, &p))\n>                         continue;  /* invalid line: does not start with SHA1 */\n\nWhile you're here, perhaps fix the comment so it doesn't talk about\nSHA1 anymore.\n\n> -               if (starts_with(sb.buf + GIT_SHA1_HEXSZ, \"\\tnot-for-merge\\t\"))\n> +               if (starts_with(p, \"\\tnot-for-merge\\t\"))\n>                         continue;  /* ref is not-for-merge */\n>                 oid_array_append(merge_heads, &oid);\n>         }\n"},{"id":"369124","messageId":"CAPig+cQsWrqapzd7XqDEf9Dx+pg3NjnJUG8Sgh21LEu3YWKRcg@mail.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-20-sandals@crustytoothpaste.net","subject":"Re: [PATCH 19/31] http-walker: replace sha1_to_hex","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-02-12T03:51:48Z","receivedAt":"2019-02-12T03:52:01Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> Since sha1_to_hex is limited to SHA-1, replace the uses of it in this\n> file with hasH_to_hex.  Rename several variables accordingly to reflect\n\ns/hasH/hash/\n\n> that they are no longer limited to SHA-1.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n"},{"id":"369131","messageId":"20190212063749.GC21872@sigill.intra.peff.net","threadId":"50474","inReplyTo":"20190212012256.1005924-5-sandals@crustytoothpaste.net","subject":"Re: [PATCH 04/31] pack-bitmap: replace sha1_to_hex","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-02-12T06:37:49Z","receivedAt":"2019-02-12T06:37:53Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Feb 12, 2019 at 01:22:29AM +0000, brian m. carlson wrote:\n\n> Replace the uses of sha1_to_hex in the pack bitmap code with hash_to_hex\n> to allow the use of SHA-256 as well.  Rename a few variables since they\n> are no longer limited to SHA-1.\n\nSounds good, although...\n\n> -static uint32_t find_object_pos(const unsigned char *sha1)\n> +static uint32_t find_object_pos(const unsigned char *hash)\n\nIsn't this really just a \"struct object_id\"? Why don't we want to use\nthat here?\n\nI suspect it may be partially because our khash is storing raw sha1s.\nBut shouldn't we also be converted that to store object_ids?\n\nIn particular:\n\n> @@ -181,7 +181,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n>  \tstored->root = root;\n>  \tstored->xor = xor_with;\n>  \tstored->flags = flags;\n> -\toidread(&stored->oid, sha1);\n> +\toidread(&stored->oid, hash);\n>  \n>  \thash_pos = kh_put_sha1(index->bitmaps, stored->oid.hash, &ret);\n\nThis last line (which is actually from the previous patch) is going to\nalways truncate the stored data to 20 bytes, isn't it?\n\nI think we need to define a kh_oid. We have the \"set\" version already in\noidset.[ch]; I think we need to make that more public.\n\n-Peff\n"},{"id":"369132","messageId":"2a6cf99a-c297-225c-b3c9-c30e1ac83948@web.de","threadId":"50474","inReplyTo":"20190212012256.1005924-25-sandals@crustytoothpaste.net","subject":"Re: [PATCH 24/31] archive-tar: make hash size independent","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2019-02-12T07:20:38Z","receivedAt":"2019-02-12T07:20:52Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 12.02.2019 um 02:22 schrieb brian m. carlson:\n> diff --git a/builtin/get-tar-commit-id.c b/builtin/get-tar-commit-id.c\n> index 2706fcfaf2..2760549e91 100644\n> --- a/builtin/get-tar-commit-id.c\n> +++ b/builtin/get-tar-commit-id.c\n> @@ -5,6 +5,7 @@\n>  #include \"commit.h\"\n>  #include \"tar.h\"\n>  #include \"builtin.h\"\n> +#include \"strbuf.h\"\n>  #include \"quote.h\"\n>\n>  static const char builtin_get_tar_commit_id_usage[] =\n> @@ -21,6 +22,8 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n>  \tchar *content = buffer + RECORDSIZE;\n>  \tconst char *comment;\n>  \tssize_t n;\n> +\tchar *hdrprefix;\n> +\tint ret;\n>\n>  \tif (argc != 1)\n>  \t\tusage(builtin_get_tar_commit_id_usage);\n> @@ -32,10 +35,14 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n>  \t\tdie_errno(\"git get-tar-commit-id: EOF before reading tar header\");\n>  \tif (header->typeflag[0] != 'g')\n>  \t\treturn 1;\n> -\tif (!skip_prefix(content, \"52 comment=\", &comment))\n> +\n> +\thdrprefix = xstrfmt(\"%zu comment=\", the_hash_algo->hexsz + strlen(\" comment=\") + 2 + 1);\n> +\tret = skip_prefix(content, hdrprefix, &comment);\n> +\tfree(hdrprefix);\n> +\tif (!ret)\n>  \t\treturn 1;\n>\n> -\tif (write_in_full(1, comment, 41) < 0)\n> +\tif (write_in_full(1, comment, the_hash_algo->hexsz + 1) < 0)\n>  \t\tdie_errno(\"git get-tar-commit-id: write error\");\n>\n>  \treturn 0;\n\nThat command currently prints the pax comment in tar archives if it\nlooks like a SHA1 hash based on its length.  It should continue to do\nso, and _also_ show longer hashes.  Your change makes it _only_ show\nthose longer hashes.\n\nSo it could check for all known valid hash lengths in turn, or accept\nany payload length between 40 and the_hash_algo->hexsz, or loosen up\ntotally and show comments of any length.\n\nRené\n"},{"id":"369133","messageId":"CAPig+cRTKTWPA66YTnX=RMJyCZmWK6efkxOyfYH_zn9OOSb8_w@mail.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-28-sandals@crustytoothpaste.net","subject":"Re: [PATCH 27/31] builtin/difftool: use parse_oid_hex","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-02-12T08:27:23Z","receivedAt":"2019-02-12T08:27:37Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 11, 2019 at 8:24 PM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> Instead of using get_oid_hex and adding constants to the result, use\n> parse_oid_hex to make this code independent of the hash size.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n> diff --git a/builtin/difftool.c b/builtin/difftool.c\n> @@ -65,14 +65,12 @@ static int parse_index_info(char *p, int *mode1, int *mode2,\n>         if (*p != ' ')\n>                 return error(\"expected ' ', got '%c'\", *p);\n> -       if (get_oid_hex(++p, oid1))\n> +       if (parse_oid_hex(++p, oid1, (const char **)&p))\n>                 return error(\"expected object ID, got '%s'\", p + 1);\n\nNot a problem introduced by this patch, but is the 'p + 1' in the\nerror message correct? 'p' has already been incremented via '++p' in\nthe call to parse_oid_hex() to point at what _should_ be the start of\nOID, so one would think that the error message would want to print out\nwhatever was found there rather than what was found one character\nafter the start of OID.\n\n> -       if (get_oid_hex(++p, oid2))\n> +       if (parse_oid_hex(++p, oid2, (const char **)&p))\n>                 return error(\"expected object ID, got '%s'\", p + 1);\n\nDitto.\n"},{"id":"369143","messageId":"871s4dl1e7.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-32-sandals@crustytoothpaste.net","subject":"Re: [PATCH 31/31] gitweb: make hash size independent","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-12T10:57:52Z","receivedAt":"2019-02-12T10:57:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Feb 12 2019, brian m. carlson wrote:\n\n> Gitweb has several hard-coded 40 values throughout it to check for\n> values that are passed in or acquired from Git.  To simplify the code,\n> introduce a regex variable that matches either exactly 40 or exactly 64\n> hex characters, and use this variable anywhere we would have previously\n> hard-coded a 40 in a regex.\n>\n> Similarly, switch the code that looks for deleted diffinfo information\n> to look for either 40 or 64 zeros, and update one piece of code to use\n> this function.  Finally, when formatting a log line, allow an\n> abbreviated describe output to contain up to 64 characters.\n\nThis might be going a bit overboard but I tried this with a variant\nwhere...\n\n> +# A regex matching a valid object ID.\n> +our $oid_regex = qr/(?:[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})?)/;\n> +\n\nInstead of this dense regex I did:\n\n    my $sha1_len = 40;\n    my $sha256_extra_len = 24;\n    my $sha256_len = $sha1_len + $sha256_extra_len;\n\n    sub oid_nlen_regex {\n    \tmy $len = shift;\n    \tmy $hchr = qr/[0-9a-fA-F]/;\n    \treturn qr/(?:(?:$hchr){$len})/\n    }\n\n    our $oid_regex;\n    {\n        my $x = oid_nlen_regex($sha1_len);\n        my $y = oid_nlen_regex($sha256_extra_len);\n        $oid_regex = qr/(?:$x(?:$y)?)/\n    }\n\nThen most of the rest of this is the same, e.g.:\n> -\tif ($input =~ m/^[0-9a-fA-F]{40}$/) {\n\nBut...\n\n> @@ -2037,10 +2040,10 @@ sub format_log_line_html {\n>              (?<!-) # see strbuf_check_tag_ref(). Tags can't start with -\n>              [A-Za-z0-9.-]+\n>              (?!\\.) # refs can't end with \".\", see check_refname_format()\n> -            -g[0-9a-fA-F]{7,40}\n> +            -g[0-9a-fA-F]{7,64}\n>              |\n>              # Just a normal looking Git SHA1\n> -            [0-9a-fA-F]{7,40}\n> +            [0-9a-fA-F]{7,64}\n>          )\n>          \\b\n>      }{\n\nE.g. here we can do call oid_nlen_regex(\"7,64\") to produce this blurb.\n\n> -\tif ($line =~ m/^index [0-9a-fA-F]{40},[0-9a-fA-F]{40}/) {\n> +\tif ($line =~ m/^index $oid_regex,$oid_regex/) {\n> -\t} elsif ($line =~ m/^index [0-9a-fA-F]{40}..[0-9a-fA-F]{40}/) {\n> +\t} elsif ($line =~ m/^index $oid_regex..$oid_regex/) {\n\nAnd here, maybe nobody cares, but we now implicitly accept mixed SHA-1 &\nSHA-256 input. Whereas we could have a helper on top of the above code\nlike:\n\n    sub oid_nlen_prefix_infix_regex {\n        my $nlen = shift;\n        my $prefix = shift;\n        my $infix = shift;\n\n        my $rx = oid_nlen_regex($nlen);\n\n        return qr/^\\Q$prefix\\E$rx\\Q$infix\\E$rx$/;\n    }\n\nAnd then e.g.:\n\n    } elsif ($line =~ oid_nlen_prefix_infix_regex($sha1_len, \"index \", \"..\") ||\n             $line =~ oid_nlen_prefix_infix_regex($sha256_len, \"index \", \"..\")) {\n\nSo only accept SHA1..SHA1 or SHA256..SHA256, not SHA1..SHA256 or\nSHA256..SHA1.\n"},{"id":"369144","messageId":"87zhr1jmr4.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-31-sandals@crustytoothpaste.net","subject":"Re: [PATCH 30/31] Git.pm: make hash size independent","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-12T10:59:27Z","receivedAt":"2019-02-12T10:59:35Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Feb 12 2019, brian m. carlson wrote:\n\n> The cat_blob function was matching on exactly 40 hex characters.  This\n> won't work with SHA-256, which uses 64-character hex object IDs.  While\n> it should be fine to simply match any number of hex characters since the\n> output is space delimited, be extra safe by matching either exactly 40\n> or exactly 64 hex characters.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n>  perl/Git.pm | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/perl/Git.pm b/perl/Git.pm\n> index d856930b2e..62c472e0ce 100644\n> --- a/perl/Git.pm\n> +++ b/perl/Git.pm\n> @@ -980,7 +980,7 @@ sub cat_blob {\n>  \t\treturn -1;\n>  \t}\n>\n> -\tif ($description !~ /^[0-9a-fA-F]{40} \\S+ (\\d+)$/) {\n> +\tif ($description !~ /^[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})? \\S+ (\\d+)$/) {\n>  \t\tcarp \"Unexpected result returned from git cat-file\";\n>  \t\treturn -1;\n>  \t}\n\nThe gitweb code doesn't load Git.pm now, but does anyone know a reason\nfor why we'd avoid any perl/* dependency in the gitweb code? If not the\nregex here & in gitweb could be factored into e.g. a tiny Git::OID\nlibrary which would either just expose a $GIT::OID_REGEX, or something\nlike the sort of interface (might not be worth it) that I suggested in\nmy feedback to 31/31.\n"},{"id":"369145","messageId":"87y36ljmcu.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-29-sandals@crustytoothpaste.net","subject":"Re: [PATCH 28/31] dir: make untracked cache extension hash size independent","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-12T11:08:01Z","receivedAt":"2019-02-12T11:08:05Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Feb 12 2019, brian m. carlson wrote:\n\n> Instead of using a struct with a flex array member to read and write the\n> untracked cache extension, use a shorter, fixed-length struct and add\n> the name and hash data explicitly.\n> [...]\n>  \tstruct stat_data info_exclude_stat;\n>  \tstruct stat_data excludes_file_stat;\n>  \tuint32_t dir_flags;\n> -\tunsigned char info_exclude_sha1[20];\n> -\tunsigned char excludes_file_sha1[20];\n> -\tchar exclude_per_dir[FLEX_ARRAY];\n>  };\n\nBoth this & the follow-up 29/31 look scary since this is an on-disk\nstructure and this patch & the next one rather than implementing some\ntransition, just changes the structs & code we use to read & write to\nuse the current hash size.\n\nSo what are we going to do when the \"current\" size is SHA-256 and our\non-disk cache is SHA-1? It seems like with this we'd at best (I haven't\ntested) throw away the SHA-1 UC data, and at worst introduce some silent\npersistent read failure.\n\nIn any case that seems like something we should have tests for with an\non-disk format, i.e. write in one hash, see what happens when we read in\nanother, and perhaps instead of not understanding SHA-1 hashes in\nSHA-256 mode we'd read the SHA-1 and consult the SHA-1<->SHA-256 lookup\ntable we're going to be eventually maintaining on the side?\n"},{"id":"369146","messageId":"87wom5jm7j.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-24-sandals@crustytoothpaste.net","subject":"Re: [PATCH 23/31] remote-curl: make hash size independent","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-12T11:11:12Z","receivedAt":"2019-02-12T11:11:16Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Feb 12 2019, brian m. carlson wrote:\n\n> Change one hard-coded use of the constant 40 to a reference to\n> the_hash_algo.  In addition, switch a use of get_oid_hex to\n> parse_oid_hex to avoid the need to use a constant.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n>  remote-curl.c | 11 ++++++-----\n>  1 file changed, 6 insertions(+), 5 deletions(-)\n>\n> diff --git a/remote-curl.c b/remote-curl.c\n> index bb7421023b..8395b71bbb 100644\n> --- a/remote-curl.c\n> +++ b/remote-curl.c\n> @@ -249,7 +249,7 @@ static struct ref *parse_info_refs(struct discovery *heads)\n>  \t\tif (data[i] == '\\t')\n>  \t\t\tmid = &data[i];\n>  \t\tif (data[i] == '\\n') {\n> -\t\t\tif (mid - start != 40)\n> +\t\t\tif (mid - start != the_hash_algo->hexsz)\n>  \t\t\t\tdie(\"%sinfo/refs not valid: is this a git repository?\",\n>  \t\t\t\t    url.buf);\n>  \t\t\tdata[i] = 0;\n\nSimilar to my comment on 28/31 I may be missing something, but this is\nthe part where we're parsing the ref advertisement from a remote\nrepository, which may e.g. be SHA-1, and our local \"the hash algo\" may\nbe SHA-256, no?\n\nSo isn't this a point where we need to accept both, and parse it into\nsome structure where we mark it as either/or SHA-1 or SHA-256, and late\nor consult SHA-1<->SHA-256 lookup tables etc.?\n"},{"id":"369147","messageId":"87va1pjm4d.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-6-sandals@crustytoothpaste.net","subject":"Re: [PATCH 05/31] pack-bitmap: switch hard-coded constants to the_hash_algo","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-12T11:13:06Z","receivedAt":"2019-02-12T11:13:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Feb 12 2019, brian m. carlson wrote:\n\n> Switch two hard-coded uses of 20 to references to the_hash_algo.\n>\n> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> ---\n>  pack-bitmap.c | 4 ++--\n>  1 file changed, 2 insertions(+), 2 deletions(-)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 6d6fa68563..603492c237 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -138,7 +138,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n>  {\n>  \tstruct bitmap_disk_header *header = (void *)index->map;\n>\n> -\tif (index->map_size < sizeof(*header) + 20)\n> +\tif (index->map_size < sizeof(*header) + the_hash_algo->rawsz)\n>  \t\treturn error(\"Corrupted bitmap index (missing header data)\");\n>\n>  \tif (memcmp(header->magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE)) != 0)\n> @@ -157,7 +157,7 @@ static int load_bitmap_header(struct bitmap_index *index)\n>  \t\t\t\t\"(Git requires BITMAP_OPT_FULL_DAG)\");\n>\n>  \t\tif (flags & BITMAP_OPT_HASH_CACHE) {\n> -\t\t\tunsigned char *end = index->map + index->map_size - 20;\n> +\t\t\tunsigned char *end = index->map + index->map_size - the_hash_algo->rawsz;\n>  \t\t\tindex->hashes = ((uint32_t *)end) - index->pack->num_objects;\n>  \t\t}\n>  \t}\n\nSimilarly to my other comments upthread, what happens in a repo where we\nhave a SHA-1 bitmap file & a SHA-1<->SHA-256 mapping?\n"},{"id":"369148","messageId":"87tvh9jm0l.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190212012256.1005924-1-sandals@crustytoothpaste.net","subject":"Re: [PATCH 00/31] Hash function transition part 16","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-12T11:15:22Z","receivedAt":"2019-02-12T11:15:26Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Feb 12 2019, brian m. carlson wrote:\n\n> This is the sixteenth series of hash function transition patches. This\n> series contains various fixes, mostly focused around the pack bitmap\n> code, the HTTP code, the archive code, the index, and parts of our Perl\n> code.\n>\n> This is the second to last series required for a \"stage 0\" Git; that is,\n\nI skimmed most of this, but decided to stop when I came to \"am\" since I\nwas just going to start repeating the same question I had in other\npatches, i.e. for the parts of this that deal with on-disk formats how\ndoes just e.g. search-replacing s/40/64/ interact with needing to read\nexisting files (bitmaps, \"am\" patches, untracked cache etc.) which may\nbe in the \"old\" format.\n"},{"id":"369173","messageId":"2e0ab97b-2a9f-6820-46fa-ba664c42cd87@web.de","threadId":"50474","inReplyTo":"2a6cf99a-c297-225c-b3c9-c30e1ac83948@web.de","subject":"Re: [PATCH 24/31] archive-tar: make hash size independent","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2019-02-12T17:33:39Z","receivedAt":"2019-02-12T17:34:01Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"Am 12.02.2019 um 08:20 schrieb René Scharfe:\n> That command currently prints the pax comment in tar archives if it\n> looks like a SHA1 hash based on its length.  It should continue to do\n> so, and _also_ show longer hashes.  Your change makes it _only_ show\n> those longer hashes.\n>\n> So it could check for all known valid hash lengths in turn, or accept\n> any payload length between 40 and the_hash_algo->hexsz, or loosen up\n> totally and show comments of any length.\n\nHow about the following patch, as a preparation?\n\n-- >8 --\nFrom: Rene Scharfe <l.s.r@web.de>\nSubject: [PATCH] get-tar-commit-id: parse comment record\n\nParse pax comment records properly and get rid of magic numbers for\nacceptable comment length.  This simplifies a later change to handle\nlonger hashes.\n\nSigned-off-by: Rene Scharfe <l.s.r@web.de>\n---\n builtin/get-tar-commit-id.c | 13 +++++++++++--\n 1 file changed, 11 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/get-tar-commit-id.c b/builtin/get-tar-commit-id.c\nindex 2706fcfaf2..312e44ed05 100644\n--- a/builtin/get-tar-commit-id.c\n+++ b/builtin/get-tar-commit-id.c\n@@ -21,6 +21,8 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n \tchar *content = buffer + RECORDSIZE;\n \tconst char *comment;\n \tssize_t n;\n+\tlong len;\n+\tchar *end;\n\n \tif (argc != 1)\n \t\tusage(builtin_get_tar_commit_id_usage);\n@@ -32,10 +34,17 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n \t\tdie_errno(\"git get-tar-commit-id: EOF before reading tar header\");\n \tif (header->typeflag[0] != 'g')\n \t\treturn 1;\n-\tif (!skip_prefix(content, \"52 comment=\", &comment))\n+\n+\tlen = strtol(content, &end, 10);\n+\tif (errno == ERANGE || end == content || len < 0)\n+\t\treturn 1;\n+\tif (!skip_prefix(end, \" comment=\", &comment))\n+\t\treturn 1;\n+\tlen -= comment - content;\n+\tif (len != GIT_SHA1_HEXSZ + 1)\n \t\treturn 1;\n\n-\tif (write_in_full(1, comment, 41) < 0)\n+\tif (write_in_full(1, comment, len) < 0)\n \t\tdie_errno(\"git get-tar-commit-id: write error\");\n\n \treturn 0;\n--\n2.20.1\n"},{"id":"369199","messageId":"20190212233648.GF684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"CAPig+cQivuPjDztK30QO0xKx1Tx+nYDPzTvgXvi9WVyRX5-aRQ@mail.gmail.com","subject":"Re: [PATCH 12/31] fast-import: make hash-size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-12T23:36:48Z","receivedAt":"2019-02-12T23:36:57Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Mon, Feb 11, 2019 at 10:44:12PM -0500, Eric Sunshine wrote:\n> On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n> <sandals@crustytoothpaste.net> wrote:\n> > Replace several uses of GIT_SHA1_HEXSZ and 40-based constants with\n> > references to the_hash_algo.  Update the note handling code here to\n> > compute path sizes based on GIT_MAX_RAWSZ as well.\n> >\n> > Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> > ---\n> > diff --git a/fast-import.c b/fast-import.c\n> > @@ -2047,7 +2047,8 @@ static uintmax_t do_change_note_fanout(\n> > -       char realpath[60];\n> > +       char realpath[GIT_MAX_RAWSZ * 3];\n> \n> I wonder if the fixed multiplier deserves a comment explaining that\n> this is reserving enough space for a hex representation with '/'\n> between each digit pair plus NUL. Which leads to the next question: Is\n> there is GIT_MAX_HEXSZ constant? If so, this might be more clearly\n> represented (or not) by taking advantage of that value.\n\nThere is such a constant. I'll add a comment and see if I can write it\nin a way that makes it more intuitive what we're computing.\n\n> Also, there are a number of hardcoded 60's in the code earlier in this\n> file, such as:\n> \n>     if ((max_packsize && (pack_size + 60 + len) > max_packsize)\n>         || (pack_size + 60 + len) < pack_size)\n>         cycle_packfile();\n> \n> Is that just a coincidence or is it related to the 60 characters\n> allocated for 'realpath'?\n\nThat's an interesting question. It looks like these are indeed related\nto the hash size, but they're not related to the realpath call above.\nI'll convert them as well.\n\n> > @@ -2405,7 +2406,7 @@ static void note_change_n(const char *p, struct branch *b, unsigned char *old_fa\n> >                 char *buf = read_object_with_reference(&commit_oid,\n> >                                                        commit_type, &size,\n> >                                                        &commit_oid);\n> > -               if (!buf || size < 46)\n> > +               if (!buf || size < the_hash_algo->hexsz)\n> \n> What exactly did the 46 represent and how does it relate to 'hexsz'?\n> Stated differently, why didn't this become:\n> \n>     the_hash_algo->hexsz + 6'\n> \n> ?\n\nGood catch. I believe that 46 is the number of bytes in \"tree %s\\n\",\nwhich is the smallest possible commit (a root commit without an author\nor committer).\n\nI'll fix this misconversion.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369200","messageId":"20190213000007.GG684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"20190212063749.GC21872@sigill.intra.peff.net","subject":"Re: [PATCH 04/31] pack-bitmap: replace sha1_to_hex","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-13T00:00:07Z","receivedAt":"2019-02-13T00:00:15Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 12, 2019 at 01:37:49AM -0500, Jeff King wrote:\n> On Tue, Feb 12, 2019 at 01:22:29AM +0000, brian m. carlson wrote:\n> > -static uint32_t find_object_pos(const unsigned char *sha1)\n> > +static uint32_t find_object_pos(const unsigned char *hash)\n> \n> Isn't this really just a \"struct object_id\"? Why don't we want to use\n> that here?\n> \n> I suspect it may be partially because our khash is storing raw sha1s.\n> But shouldn't we also be converted that to store object_ids?\n\nI think probably there are some more places that we could convert here.\nThere may have been one or two places that weren't convertible because\nwe ended up passing data from some sort of buffer around. I'll take\nanother look.\n\n> In particular:\n> \n> > @@ -181,7 +181,7 @@ static struct stored_bitmap *store_bitmap(struct bitmap_index *index,\n> >  \tstored->root = root;\n> >  \tstored->xor = xor_with;\n> >  \tstored->flags = flags;\n> > -\toidread(&stored->oid, sha1);\n> > +\toidread(&stored->oid, hash);\n> >  \n> >  \thash_pos = kh_put_sha1(index->bitmaps, stored->oid.hash, &ret);\n> \n> This last line (which is actually from the previous patch) is going to\n> always truncate the stored data to 20 bytes, isn't it?\n\nNo, I don't think it does. The _sha1 variant stores pointers to unsigned\nchar, while the _oid variant stores the entire struct object_id (not\njust a pointer to it). We don't care how much data the pointer points\nto.\n\n> I think we need to define a kh_oid. We have the \"set\" version already in\n> oidset.[ch]; I think we need to make that more public.\n\nI wrote this quite a bit before that code came in, which is probably why\nI didn't do that originally. I seem to recall last time I looked at this\nthat there was some reason that hoisting this didn't work as I expected\ndue to header include order, but I'll take a look and see if I can\nfigure out a way to do this.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369201","messageId":"20190213001111.GH684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"2e0ab97b-2a9f-6820-46fa-ba664c42cd87@web.de","subject":"Re: [PATCH 24/31] archive-tar: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-13T00:11:11Z","receivedAt":"2019-02-13T00:11:19Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 12, 2019 at 06:33:39PM +0100, René Scharfe wrote:\n> Am 12.02.2019 um 08:20 schrieb René Scharfe:\n> > That command currently prints the pax comment in tar archives if it\n> > looks like a SHA1 hash based on its length.  It should continue to do\n> > so, and _also_ show longer hashes.  Your change makes it _only_ show\n> > those longer hashes.\n> >\n> > So it could check for all known valid hash lengths in turn, or accept\n> > any payload length between 40 and the_hash_algo->hexsz, or loosen up\n> > totally and show comments of any length.\n> \n> How about the following patch, as a preparation?\n> \n> -- >8 --\n> From: Rene Scharfe <l.s.r@web.de>\n> Subject: [PATCH] get-tar-commit-id: parse comment record\n> \n> Parse pax comment records properly and get rid of magic numbers for\n> acceptable comment length.  This simplifies a later change to handle\n> longer hashes.\n> \n> Signed-off-by: Rene Scharfe <l.s.r@web.de>\n> ---\n>  builtin/get-tar-commit-id.c | 13 +++++++++++--\n>  1 file changed, 11 insertions(+), 2 deletions(-)\n> \n> diff --git a/builtin/get-tar-commit-id.c b/builtin/get-tar-commit-id.c\n> index 2706fcfaf2..312e44ed05 100644\n> --- a/builtin/get-tar-commit-id.c\n> +++ b/builtin/get-tar-commit-id.c\n> @@ -21,6 +21,8 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n>  \tchar *content = buffer + RECORDSIZE;\n>  \tconst char *comment;\n>  \tssize_t n;\n> +\tlong len;\n> +\tchar *end;\n> \n>  \tif (argc != 1)\n>  \t\tusage(builtin_get_tar_commit_id_usage);\n> @@ -32,10 +34,17 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix)\n>  \t\tdie_errno(\"git get-tar-commit-id: EOF before reading tar header\");\n>  \tif (header->typeflag[0] != 'g')\n>  \t\treturn 1;\n> -\tif (!skip_prefix(content, \"52 comment=\", &comment))\n> +\n> +\tlen = strtol(content, &end, 10);\n> +\tif (errno == ERANGE || end == content || len < 0)\n> +\t\treturn 1;\n> +\tif (!skip_prefix(end, \" comment=\", &comment))\n> +\t\treturn 1;\n> +\tlen -= comment - content;\n> +\tif (len != GIT_SHA1_HEXSZ + 1)\n>  \t\treturn 1;\n\nSo it turns out I wrote a different patch for this in a later series,\nbut I like this style better. I'm going to squash my existing patches\ntogether and then rework them such that they're based off yours.\n\nMy technique iterated over the entire header, comparing the entire\nheader, which is much less elegant than this solution.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369202","messageId":"20190213003041.GI684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"87y36ljmcu.fsf@evledraar.gmail.com","subject":"Re: [PATCH 28/31] dir: make untracked cache extension hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-13T00:30:42Z","receivedAt":"2019-02-13T00:30:51Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 12, 2019 at 12:08:01PM +0100, Ævar Arnfjörð Bjarmason wrote:\n> Both this & the follow-up 29/31 look scary since this is an on-disk\n> structure and this patch & the next one rather than implementing some\n> transition, just changes the structs & code we use to read & write to\n> use the current hash size.\n> \n> So what are we going to do when the \"current\" size is SHA-256 and our\n> on-disk cache is SHA-1? It seems like with this we'd at best (I haven't\n> tested) throw away the SHA-1 UC data, and at worst introduce some silent\n> persistent read failure.\n\nThat's a good question. The current design has everything in the .git\ndirectory other than the loose object table and the pack indices in one\nalgorithm. That is, the untracked cache extension, like the rest of the\nindex, will always be SHA-256 when the objects are stored in SHA-256.\n\nHaving mixed algorithms in the .git directory would add a lot of\ncomplexity.\n\n> In any case that seems like something we should have tests for with an\n> on-disk format, i.e. write in one hash, see what happens when we read in\n> another, and perhaps instead of not understanding SHA-1 hashes in\n> SHA-256 mode we'd read the SHA-1 and consult the SHA-1<->SHA-256 lookup\n> table we're going to be eventually maintaining on the side?\n\nCorrect. The current plan is that when the transition code is fully\nimplemented, we'll read object IDs from the user in whatever algorithms\nwe support, translate them into the default algorithm, and then use\nthem.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369271","messageId":"20190214033315.GK684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"CAPig+cS3i2FWf6mFmpPpctyJJbx4xdHgWXrveRUkjMHct5SbJg@mail.gmail.com","subject":"Re: [PATCH 10/31] object-store: rename and expand packed_git's sha1 member","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-14T03:33:16Z","receivedAt":"2019-02-14T03:33:25Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Mon, Feb 11, 2019 at 10:32:15PM -0500, Eric Sunshine wrote:\n> On Mon, Feb 11, 2019 at 8:23 PM brian m. carlson\n> <sandals@crustytoothpaste.net> wrote:\n> > This member is used to represent the pack checksum of the pack in\n> > question.  Expand this member to be GIT_MAX_RAWSZ bytes in length so it\n> > works with longer hashes and rename it to be \"hash\" instead of \"sha1\".\n> > [...]\n> > Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>\n> > ---\n> > diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\n> > @@ -689,7 +689,7 @@ int cmd_pack_redundant(int argc, const char **argv, const char *prefix)\n> >         while (pl) {\n> >                 printf(\"%s\\n%s\\n\",\n> > -                      sha1_pack_index_name(pl->pack->sha1),\n> > +                      sha1_pack_index_name(pl->pack->hash),\n> \n> I guess there is no oid_pack_index_name() function yet?\n\nCorrect, although see below.\n\n> > diff --git a/http-walker.c b/http-walker.c\n> > @@ -434,7 +434,7 @@ static int http_fetch_pack(struct walker *walker, struct alt_base *repo, unsigne\n> >         if (walker->get_verbosely) {\n> >                 fprintf(stderr, \"Getting pack %s\\n\",\n> > -                       sha1_to_hex(target->sha1));\n> > +                       sha1_to_hex(target->hash));\n> \n> Should this have become oid_to_hex()?\n\nNo, I think what's misleading here is that the member for both is named\n\"hash\". This is a struct packed_git, not a struct object_id. The reason\nI didn't convert this to a struct object_id is because this isn't\nactually an object ID, it's a pack checksum hash.\n\nI've tried to be diligent about separating the object IDs from other\nhashes because I think it makes it easier to reason about the code, but\nalso because it may help us avoid tricky behavior when we have to deal\nwith multiple algorithms for input and storage.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369279","messageId":"20190214044145.GA19071@sigill.intra.peff.net","threadId":"50474","inReplyTo":"20190213000007.GG684736@genre.crustytoothpaste.net","subject":"Re: [PATCH 04/31] pack-bitmap: replace sha1_to_hex","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-02-14T04:41:46Z","receivedAt":"2019-02-14T04:41:49Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 13, 2019 at 12:00:07AM +0000, brian m. carlson wrote:\n\n> On Tue, Feb 12, 2019 at 01:37:49AM -0500, Jeff King wrote:\n> > On Tue, Feb 12, 2019 at 01:22:29AM +0000, brian m. carlson wrote:\n> > > -static uint32_t find_object_pos(const unsigned char *sha1)\n> > > +static uint32_t find_object_pos(const unsigned char *hash)\n> > \n> > Isn't this really just a \"struct object_id\"? Why don't we want to use\n> > that here?\n> > \n> > I suspect it may be partially because our khash is storing raw sha1s.\n> > But shouldn't we also be converted that to store object_ids?\n> \n> I think probably there are some more places that we could convert here.\n> There may have been one or two places that weren't convertible because\n> we ended up passing data from some sort of buffer around. I'll take\n> another look.\n\nThanks. I don't want to derail you too much if you have a series of\nother changes on top. And moving to \"hash\" here is a step in the right\ndirection. But if we can take it all the way to object_id while we're\nlooking at it, I think that's preferable.\n\n> > >  \thash_pos = kh_put_sha1(index->bitmaps, stored->oid.hash, &ret);\n> > \n> > This last line (which is actually from the previous patch) is going to\n> > always truncate the stored data to 20 bytes, isn't it?\n> \n> No, I don't think it does. The _sha1 variant stores pointers to unsigned\n> char, while the _oid variant stores the entire struct object_id (not\n> just a pointer to it). We don't care how much data the pointer points\n> to.\n\nOh, you're right. I was thinking it actually stored the 20-byte\nsequences, but I was just reading it wrong. Sorry for the confusion.\n\n> > I think we need to define a kh_oid. We have the \"set\" version already in\n> > oidset.[ch]; I think we need to make that more public.\n> \n> I wrote this quite a bit before that code came in, which is probably why\n> I didn't do that originally. I seem to recall last time I looked at this\n> that there was some reason that hoisting this didn't work as I expected\n> due to header include order, but I'll take a look and see if I can\n> figure out a way to do this.\n\nGreat, thanks!\n\n-Peff\n"},{"id":"369547","messageId":"20190218190920.GL684736@genre.crustytoothpaste.net","threadId":"50474","inReplyTo":"87zhr1jmr4.fsf@evledraar.gmail.com","subject":"Re: [PATCH 30/31] Git.pm: make hash size independent","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-02-18T19:09:20Z","receivedAt":"2019-02-18T19:09:29Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 12, 2019 at 11:59:27AM +0100, Ævar Arnfjörð Bjarmason wrote:\n> The gitweb code doesn't load Git.pm now, but does anyone know a reason\n> for why we'd avoid any perl/* dependency in the gitweb code? If not the\n> regex here & in gitweb could be factored into e.g. a tiny Git::OID\n> library which would either just expose a $GIT::OID_REGEX, or something\n> like the sort of interface (might not be worth it) that I suggested in\n> my feedback to 31/31.\n\nI think one potential issue here is that some distributors bundle the\nPerl modules with git send-email and not gitweb, and they're packaged\nindependently. I'm not opposed to seeing a patch to do that, but it\nprobably belongs in its own series.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"369563","messageId":"87r2c4olqm.fsf@evledraar.gmail.com","threadId":"50474","inReplyTo":"20190218190920.GL684736@genre.crustytoothpaste.net","subject":"Re: [PATCH 30/31] Git.pm: make hash size independent","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-02-18T21:00:33Z","receivedAt":"2019-02-18T21:00:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Feb 18 2019, brian m. carlson wrote:\n\n> On Tue, Feb 12, 2019 at 11:59:27AM +0100, Ævar Arnfjörð Bjarmason wrote:\n>> The gitweb code doesn't load Git.pm now, but does anyone know a reason\n>> for why we'd avoid any perl/* dependency in the gitweb code? If not the\n>> regex here & in gitweb could be factored into e.g. a tiny Git::OID\n>> library which would either just expose a $GIT::OID_REGEX, or something\n>> like the sort of interface (might not be worth it) that I suggested in\n>> my feedback to 31/31.\n>\n> I think one potential issue here is that some distributors bundle the\n> Perl modules with git send-email and not gitweb, and they're packaged\n> independently. I'm not opposed to seeing a patch to do that, but it\n> probably belongs in its own series.\n\nIn packaging terms that one's easy to juggle, FWIW I'm just aware of\ne.g. RedHat packaging perl-Git as its own thing depended on by git-svn,\ngit-email & friends, not anyone who packages it with git-send-email\nspecifically.\n\nBut according to gitweb/README it's also meant to be used as a\nstand-alone script. So:\n\n * You can wget it from the git.git repo & run it\n * You can run it with any (reasonable) version of git\n * If you're missing perl modules, those are all on CPAN (just CGI.pm\n   should be missing these days)\n\nMaking it depend on any module we ship would break all those things. I\ndon't know if anyone cares, but avoiding this very minor copy/pasting\nseems like a bad reason to wake that particular dragon.\n"}]}