{"thread":{"id":"60022","subject":"[PATCH] rerere: match the hash algorithm with its length","startedAt":"2023-07-21T23:38:29Z","lastAt":"2023-07-24T23:11:13Z","messageCount":5,"participants":["Junio C Hamano","brian m. carlson"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"479773","messageId":"xmqqa5vou9ar.fsf@gitster.g","threadId":"60022","inReplyTo":null,"subject":"[PATCH] rerere: match the hash algorithm with its length","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-07-21T23:36:12Z","receivedAt":"2023-07-21T23:38:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The \"conflict ID\" used by \"git rerere\" to identify past conflicts we\nsaw has been a SHA-1 hash of the normalized text taken from the\nconflicted region.  0d7c419a (rerere: convert to use the_hash_algo,\n2018-10-15) updated the rerere machinery to use more general \"hash\"\ninstead of hardcoded SHA-1 by using the_hash_algo, GIT_MAX_RAWSZ and\ntheir friends, but the code that read from the MERGE_RR records were\nleft unconverted to still use get_sha1_hex(), possibly breaking the\noperation in SHA-256 repositories.\n\nWe enumerate the subdirectories of $GIT_DIR/rr-cache/ and use the\nones whose name passes parse_oid_hex() in full as conflict IDs,\nso they are always of correct length relative to the choice of the\nhash the repository makes, and they are written to the MERGE_RR\nfile.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\n * The \"conflict ID\" uses SHA-1 not because we needed a secure hash.\n   We only needed something that is reasonably long with fewer\n   collisions (the \"rerere\" machinery tolerates collisions).  We\n   just had SHA-1 readily available to us and that was the only\n   reason we used it.  As these \"conflict ID\" are not security\n   sensitive, we could leave them as SHA-1 even in SHA-256\n   repositories and reverting 0d7c419a might be a good first step if\n   we want to go in that direction, but let's be consistent.\n\n rerere.c | 9 +++++++--\n 1 file changed, 7 insertions(+), 2 deletions(-)\n\ndiff --git a/rerere.c b/rerere.c\nindex 7070f75014..f06172253b 100644\n--- a/rerere.c\n+++ b/rerere.c\n@@ -203,8 +203,13 @@ static void read_rr(struct repository *r, struct string_list *rr)\n \t\tint variant;\n \t\tconst unsigned hexsz = the_hash_algo->hexsz;\n \n-\t\t/* There has to be the hash, tab, path and then NUL */\n-\t\tif (buf.len < hexsz + 2 || get_sha1_hex(buf.buf, hash))\n+\t\t/*\n+\t\t * There has to be the \"conflict ID\", tab, path and then NUL.\n+\t\t * \"conflict ID\" would be a hash, possibly suffixed by \".\" and\n+\t\t * a small integer (variant number).\n+\t\t */\n+\t\tif (buf.len < hexsz + 2 ||\n+\t\t    get_hash_hex_algop(buf.buf, hash, the_hash_algo))\n \t\t\tdie(_(\"corrupt MERGE_RR\"));\n \n \t\tif (buf.buf[hexsz] != '.') {\n-- \n2.41.0-394-ge43f4fd0bd\n\n"},{"id":"479781","messageId":"ZL1BNxVWKGx0Gi1b@tapette.crustytoothpaste.net","threadId":"60022","inReplyTo":"xmqqa5vou9ar.fsf@gitster.g","subject":"Re: [PATCH] rerere: match the hash algorithm with its length","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2023-07-23T15:03:19Z","receivedAt":"2023-07-23T15:03:25Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2023-07-21 at 23:36:12, Junio C Hamano wrote:\n> The \"conflict ID\" used by \"git rerere\" to identify past conflicts we\n> saw has been a SHA-1 hash of the normalized text taken from the\n> conflicted region.  0d7c419a (rerere: convert to use the_hash_algo,\n> 2018-10-15) updated the rerere machinery to use more general \"hash\"\n> instead of hardcoded SHA-1 by using the_hash_algo, GIT_MAX_RAWSZ and\n> their friends, but the code that read from the MERGE_RR records were\n> left unconverted to still use get_sha1_hex(), possibly breaking the\n> operation in SHA-256 repositories.\n\nI agree consistency here is a good idea.  However, I should point out\nthe definition of `get_sha1_hex`:\n\nint get_sha1_hex(const char *hex, unsigned char *sha1)\n{\n\treturn get_hash_hex_algop(hex, sha1, the_hash_algo);\n}\n\nThus `get_sha1_hex` uses `the_hash_algo`, and therefore your change is\nequivalent to what was there before, I believe.  That's because during\nthe SHA-256 code work, we could either send a bunch of patches to fix\nall of the instance of `get_sha1_hex` or we could just patch that\nfunction to use the default hash algorithm, and I, for better or for\nworse, made the decision to avoid the churn.\n\nI still firmly agree that your change is better, because it is easier to\nread and less confusing, and that is a major improvement in itself.\nHowever, I would suggest that the commit message be updated to reflect\nthat if possible.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"479782","messageId":"xmqqr0oyr3y0.fsf@gitster.g","threadId":"60022","inReplyTo":"ZL1BNxVWKGx0Gi1b@tapette.crustytoothpaste.net","subject":"Re: [PATCH] rerere: match the hash algorithm with its length","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-07-23T16:24:39Z","receivedAt":"2023-07-23T16:26:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n\n> I agree consistency here is a good idea.  However, I should point out\n> the definition of `get_sha1_hex`:\n>\n> int get_sha1_hex(const char *hex, unsigned char *sha1)\n> {\n> \treturn get_hash_hex_algop(hex, sha1, the_hash_algo);\n> }\n\nYeah, I think I lifted the inlining from there, and you are\nabsolutely right.  I think the main source of the confusion is that\nget_sha1_hex(), while it was a perfectly good name before the\n\"struct object_id\" world, has now become a misnomer.\n\nI'd retract the patch you reviewed, but now I wonder if the\nfollowing is a good idea.\n\n------- >8 ------------- >8 ------------- >8 -------\nSubject: [PATCH] hex: retire get_sha1_hex()\n\nThe naming convention around get_sha1_hex() and its friends is\nawkward these days (post \"struct object_id\".  There are three public\nfunctions:\n\n * get_sha1_hex()       - use the implied the_hash_algo, fill uchar *\n * get_oid_hex()        - use the implied the_hash_algo, fill oid *\n * get_oid_hex_algop()  - use the passed algop, fill oid *\n\nBetween the latter two, the \"_algop\" suffix signals whether the\nthe_hash_algo is used as the implied algorithm or the caller should\npass an algorithm explicitly.  That is very much understandable and\nis a good convention.\n\nBetween the former two, however, the \"SHA1\" vs \"OID\" in the names\ndifferentiate in what type of variable the result is stored.\n\nWe could argue that it makes sense to use \"SHA1\" to mean \"flat byte\nbuffer\" to honor the historical practice in the days before \"struct\nobject_id\" was invented, but when we introduce and name the natural\nfourth friend to the mix that takes an algop and fills a flat byte\nbuffer, it would get an awkward name: get_sha1_hex_algop().\n\nDo we use the passed in algo, or are we limited to SHA-1 ;-)?\n\nCorrect the misnomer and use \"hash\" as \"flat byte buffer that stores\nbinary (as opposed to hexadecimal) representation of the hash\".  The\nfour (2x2) friends now become:\n\n * get_hash_hex()       - use the implied the_hash_algo, fill uchar *\n * get_oid_hex()        - use the implied the_hash_algo, fill oid *\n * get_hash_hex_algop() - use the passed algop, fill uchar *\n * get_oid_hex_algop()  - use the passed algop, fill oid *\n\nAs there are only two remaining calls to get_sha1_hex() in the\ncodebase right now, the blast radious is fairly small.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n hex.c      |  6 +++---\n hex.h      | 13 +++++++++----\n packfile.c |  2 +-\n rerere.c   |  2 +-\n 4 files changed, 14 insertions(+), 9 deletions(-)\n\ndiff --git c/hex.c w/hex.c\nindex 7bb440e794..9ec4e674ad 100644\n--- c/hex.c\n+++ w/hex.c\n@@ -49,8 +49,8 @@ int hex_to_bytes(unsigned char *binary, const char *hex, size_t len)\n \treturn 0;\n }\n \n-static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n-\t\t\t      const struct git_hash_algo *algop)\n+int get_hash_hex_algop(const char *hex, unsigned char *hash,\n+\t\t       const struct git_hash_algo *algop)\n {\n \tint i;\n \tfor (i = 0; i < algop->rawsz; i++) {\n@@ -63,7 +63,7 @@ static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n \treturn 0;\n }\n \n-int get_sha1_hex(const char *hex, unsigned char *sha1)\n+int get_hash_hex(const char *hex, unsigned char *sha1)\n {\n \treturn get_hash_hex_algop(hex, sha1, the_hash_algo);\n }\ndiff --git c/hex.h w/hex.h\nindex 7df4b3c460..9fa9c11fd0 100644\n--- c/hex.h\n+++ w/hex.h\n@@ -20,14 +20,19 @@ static inline int hex2chr(const char *s)\n }\n \n /*\n- * Try to read a SHA1 in hexadecimal format from the 40 characters\n- * starting at hex.  Write the 20-byte result to sha1 in binary form.\n+ * Try to read a hash (specified by the_hash_algo) in hexadecimal format from\n+ * the 40 (or whatever length the hash algorithm uses) characters\n+ * starting at hex.  Write the 20-byte (or the length of the hash) result to\n+ * hash in binary form.\n  * Return 0 on success.  Reading stops if a NUL is encountered in the\n  * input, so it is safe to pass this function an arbitrary\n  * null-terminated string.\n  */\n-int get_sha1_hex(const char *hex, unsigned char *sha1);\n-int get_oid_hex(const char *hex, struct object_id *sha1);\n+int get_hash_hex(const char *hex, unsigned char *hash);\n+int get_oid_hex(const char *hex, struct object_id *oid);\n+\n+/* Like get_hash_hex, but for an arbitrary hash algorithm. */\n+int get_hash_hex_algop(const char *hex, unsigned char *, const struct git_hash_algo *);\n \n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\ndiff --git c/packfile.c w/packfile.c\nindex 030b7ec7a8..3076fc8d6f 100644\n--- c/packfile.c\n+++ w/packfile.c\n@@ -751,7 +751,7 @@ struct packed_git *add_packed_git(const char *path, size_t path_len, int local)\n \tp->pack_local = local;\n \tp->mtime = st.st_mtime;\n \tif (path_len < the_hash_algo->hexsz ||\n-\t    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))\n+\t    get_hash_hex(path + path_len - the_hash_algo->hexsz, p->hash))\n \t\thashclr(p->hash);\n \treturn p;\n }\ndiff --git c/rerere.c w/rerere.c\nindex 7070f75014..725c1b6a95 100644\n--- c/rerere.c\n+++ w/rerere.c\n@@ -204,7 +204,7 @@ static void read_rr(struct repository *r, struct string_list *rr)\n \t\tconst unsigned hexsz = the_hash_algo->hexsz;\n \n \t\t/* There has to be the hash, tab, path and then NUL */\n-\t\tif (buf.len < hexsz + 2 || get_sha1_hex(buf.buf, hash))\n+\t\tif (buf.len < hexsz + 2 || get_hash_hex(buf.buf, hash))\n \t\t\tdie(_(\"corrupt MERGE_RR\"));\n \n \t\tif (buf.buf[hexsz] != '.') {\n"},{"id":"479832","messageId":"ZL7rn+LTVjSFzrI8@tapette.crustytoothpaste.net","threadId":"60022","inReplyTo":"xmqqr0oyr3y0.fsf@gitster.g","subject":"Re: [PATCH] rerere: match the hash algorithm with its length","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2023-07-24T21:22:39Z","receivedAt":"2023-07-24T21:22:57Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2023-07-23 at 16:24:39, Junio C Hamano wrote:\n> \"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n> \n> > I agree consistency here is a good idea.  However, I should point out\n> > the definition of `get_sha1_hex`:\n> >\n> > int get_sha1_hex(const char *hex, unsigned char *sha1)\n> > {\n> > \treturn get_hash_hex_algop(hex, sha1, the_hash_algo);\n> > }\n> \n> Yeah, I think I lifted the inlining from there, and you are\n> absolutely right.  I think the main source of the confusion is that\n> get_sha1_hex(), while it was a perfectly good name before the\n> \"struct object_id\" world, has now become a misnomer.\n> \n> I'd retract the patch you reviewed, but now I wonder if the\n> following is a good idea.\n\nYeah, I think that's a great idea, especially since now there are only a\nhandful of those calls left.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"479834","messageId":"xmqq1qgwoqgo.fsf_-_@gitster.g","threadId":"60022","inReplyTo":"ZL7rn+LTVjSFzrI8@tapette.crustytoothpaste.net","subject":"Re* [PATCH] rerere: match the hash algorithm with its length","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-07-24T23:11:03Z","receivedAt":"2023-07-24T23:11:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n\n>> I'd retract the patch you reviewed, but now I wonder if the\n>> following is a good idea.\n>\n> Yeah, I think that's a great idea, especially since now there are only a\n> handful of those calls left.\n\nThere are exactly two ;-).\n\nI've toned it down somewhat, as the fourth friend in the group\nalready exists and there is no need to make it public.  So here is\nwhat I am going to queue.\n\nThanks.\n\n---- >8 -----\nSubject: [PATCH v2] hex: retire get_sha1_hex()\n\nThe naming convention around get_sha1_hex() and its friends is\nawkward these days, after \"struct object_id\" was introduced.\n\nThere are three public functions around this area:\n\n * get_sha1_hex()       - use the implied the_hash_algo, fill uchar *\n * get_oid_hex()        - use the implied the_hash_algo, fill oid *\n * get_oid_hex_algop()  - use the passed algop, fill oid *\n\nBetween the latter two, the \"_algop\" suffix signals whether the\nthe_hash_algo is used as the implied algorithm or the caller should\npass an algorithm explicitly.  That is very much understandable and\nis a good convention.\n\nBetween the former two, however, the \"SHA1\" vs \"OID\" in the names\ndifferentiate in what type of variable the result is stored.\n\nWe could argue that it makes sense to use \"SHA1\" to mean \"flat byte\nbuffer\" to honor the historical practice in the days before \"struct\nobject_id\" was invented, but the natural fourth friend of the above\ngroup would take an algop and fill a flat byte buffer, and it would\nbe strange to name it get_sha1_hex_algop().  Do we use the passed in\nalgo, or are we limited to SHA-1 ;-)?\n\nIn fact, such a function exists, albeit as a private helper function\nused by the implementation of these functions, and is named a lot\nmore sensibly: get_hash_hex_algop().\n\nCorrect the misnomer of get_sha1_hex() and use \"hash\", instead of\n\"sha1\", as \"flat byte buffer that stores binary (as opposed to\nhexadecimal) representation of the hash\".\n\nThe four (2x2) friends now become:\n\n * get_hash_hex()       - use the implied the_hash_algo, fill uchar *\n * get_oid_hex()        - use the implied the_hash_algo, fill oid *\n * get_hash_hex_algop() - use the passed algop, fill uchar *\n * get_oid_hex_algop()  - use the passed algop, fill oid *\n\nAs there are only two remaining calls to get_sha1_hex() in the\ncodebase right now, the blast radious of this change is fairly\nsmall.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n hex.c      |  2 +-\n hex.h      | 10 ++++++----\n packfile.c |  2 +-\n rerere.c   |  2 +-\n 4 files changed, 9 insertions(+), 7 deletions(-)\n\ndiff --git a/hex.c b/hex.c\nindex 7bb440e794..01f17fe5c9 100644\n--- a/hex.c\n+++ b/hex.c\n@@ -63,7 +63,7 @@ static int get_hash_hex_algop(const char *hex, unsigned char *hash,\n \treturn 0;\n }\n \n-int get_sha1_hex(const char *hex, unsigned char *sha1)\n+int get_hash_hex(const char *hex, unsigned char *sha1)\n {\n \treturn get_hash_hex_algop(hex, sha1, the_hash_algo);\n }\ndiff --git a/hex.h b/hex.h\nindex 7df4b3c460..87abf66602 100644\n--- a/hex.h\n+++ b/hex.h\n@@ -20,14 +20,16 @@ static inline int hex2chr(const char *s)\n }\n \n /*\n- * Try to read a SHA1 in hexadecimal format from the 40 characters\n- * starting at hex.  Write the 20-byte result to sha1 in binary form.\n+ * Try to read a hash (specified by the_hash_algo) in hexadecimal\n+ * format from the 40 (or whatever length the hash algorithm uses)\n+ * characters starting at hex.  Write the 20-byte (or the length of\n+ * the hash) result to hash in binary form.\n  * Return 0 on success.  Reading stops if a NUL is encountered in the\n  * input, so it is safe to pass this function an arbitrary\n  * null-terminated string.\n  */\n-int get_sha1_hex(const char *hex, unsigned char *sha1);\n-int get_oid_hex(const char *hex, struct object_id *sha1);\n+int get_hash_hex(const char *hex, unsigned char *hash);\n+int get_oid_hex(const char *hex, struct object_id *oid);\n \n /* Like get_oid_hex, but for an arbitrary hash algorithm. */\n int get_oid_hex_algop(const char *hex, struct object_id *oid, const struct git_hash_algo *algop);\ndiff --git a/packfile.c b/packfile.c\nindex fd083c86e0..aa7a7ad8c3 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -753,7 +753,7 @@ struct packed_git *add_packed_git(const char *path, size_t path_len, int local)\n \tp->pack_local = local;\n \tp->mtime = st.st_mtime;\n \tif (path_len < the_hash_algo->hexsz ||\n-\t    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))\n+\t    get_hash_hex(path + path_len - the_hash_algo->hexsz, p->hash))\n \t\thashclr(p->hash);\n \treturn p;\n }\ndiff --git a/rerere.c b/rerere.c\nindex e968d413d6..228af65a5b 100644\n--- a/rerere.c\n+++ b/rerere.c\n@@ -204,7 +204,7 @@ static void read_rr(struct repository *r, struct string_list *rr)\n \t\tconst unsigned hexsz = the_hash_algo->hexsz;\n \n \t\t/* There has to be the hash, tab, path and then NUL */\n-\t\tif (buf.len < hexsz + 2 || get_sha1_hex(buf.buf, hash))\n+\t\tif (buf.len < hexsz + 2 || get_hash_hex(buf.buf, hash))\n \t\t\tdie(_(\"corrupt MERGE_RR\"));\n \n \t\tif (buf.buf[hexsz] != '.') {\n-- \n2.41.0-394-ge43f4fd0bd\n\n"}]}