{"thread":{"id":"60179","subject":"[REGRESSION] Can't clone GitHub repos (fetch-pack error) due to avoiding deprecated OpenSSL SHA-1 routines","startedAt":"2023-08-31T12:47:25Z","lastAt":"2023-09-01T11:09:26Z","messageCount":8,"participants":["Bagas Sanjaya","brian m. carlson","Eric Wong","Junio C Hamano","Oswald Buddenhagen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"481237","messageId":"ZPCL11k38PXTkFga@debian.me","threadId":"60179","inReplyTo":null,"subject":"[REGRESSION] Can't clone GitHub repos (fetch-pack error) due to avoiding deprecated OpenSSL SHA-1 routines","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2023-08-31T12:47:19Z","receivedAt":"2023-08-31T12:47:25Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"Hi,\n\nI built Git v2.42.0 on Debian testing, linked with OpenSSL (v3.0.10 from\ndistribution) with Makefile knob `OPENSSL_SHA1=YesPlease \nOPENSSL_SHA256=YesPlease`. I tried to shallow clone git.git repository:\n\n```\n$ git clone https://github.com/git/git --depth=1 git-scm\n```\n\nAll the necessary objects were fetched but the clone errored instead with:\n\n```\nfatal: fetch-pack: invalid index-pack output\n```\n\nThis issue is a regression since v2.41.0 doesn't have it. Bisecting, the\nculprit is commit bda9c12073e7 (avoid SHA-1 functions deprecated in OpenSSL 3+,\n2023-08-01). AFAIK, the culprit doesn't touch `fetch-pack.c` as I hoped.\n\nThe full bisection log is:\n\n```\ngit bisect start '--term-good=ok' '--term-bad=oops'\n# status: waiting for both good and bad commits\n# ok: [fe86abd7511a9a6862d5706c6fa1d9b57a63ba09] Git 2.41\ngit bisect ok fe86abd7511a9a6862d5706c6fa1d9b57a63ba09\n# status: waiting for bad commit, 1 good commit known\n# oops: [43c8a30d150ecede9709c1f2527c8fba92c65f40] Git 2.42\ngit bisect oops 43c8a30d150ecede9709c1f2527c8fba92c65f40\n# ok: [1d76e69212102c3373b552186590b76d6ad8d84c] Merge branch 'jc/doc-hash-object-types'\ngit bisect ok 1d76e69212102c3373b552186590b76d6ad8d84c\n# ok: [914a353a128d4d885e138f189e235ad6094d436e] Merge branch 'jc/am-parseopt-fix'\ngit bisect ok 914a353a128d4d885e138f189e235ad6094d436e\n# ok: [e48d9c78cc00805660b83ac809188d0c413e4c46] Merge branch 'am/doc-sha256'\ngit bisect ok e48d9c78cc00805660b83ac809188d0c413e4c46\n# oops: [cecd6a5ffce2c35f18e8ac537c9e2f71ac99932b] Merge branch 'jc/send-email-pre-process-fix'\ngit bisect oops cecd6a5ffce2c35f18e8ac537c9e2f71ac99932b\n# oops: [8cdd5e713d7ba54b9d26ac997408bb745ab55088] Merge branch 'ma/locate-in-path-for-windows'\ngit bisect oops 8cdd5e713d7ba54b9d26ac997408bb745ab55088\n# ok: [a82fb66fed250e16d3010c75404503bea3f0ab61] A few more topics before -rc1\ngit bisect ok a82fb66fed250e16d3010c75404503bea3f0ab61\n# oops: [cf07e53bae8492fc6ee8a8d394e2fba858daa0a4] Merge branch 'bc/ident-dot-is-no-longer-crud-letter'\ngit bisect oops cf07e53bae8492fc6ee8a8d394e2fba858daa0a4\n# oops: [bda9c12073e786e2ffa2c3ec479c7fe098d49999] avoid SHA-1 functions deprecated in OpenSSL 3+\ngit bisect oops bda9c12073e786e2ffa2c3ec479c7fe098d49999\n# ok: [3e440ea0aba0660f356a3e5b9fc366d5d6960847] sha256: avoid functions deprecated in OpenSSL 3+\ngit bisect ok 3e440ea0aba0660f356a3e5b9fc366d5d6960847\n# first oops commit: [bda9c12073e786e2ffa2c3ec479c7fe098d49999] avoid SHA-1 functions deprecated in OpenSSL 3+\n```\n\nThanks.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"481275","messageId":"ZPEf8kbBUFqLO25W@tapette.crustytoothpaste.net","threadId":"60179","inReplyTo":"ZPCL11k38PXTkFga@debian.me","subject":"Re: [REGRESSION] Can't clone GitHub repos (fetch-pack error) due to avoiding deprecated OpenSSL SHA-1 routines","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2023-08-31T23:19:14Z","receivedAt":"2023-08-31T23:19:19Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2023-08-31 at 12:47:19, Bagas Sanjaya wrote:\n> Hi,\n> \n> I built Git v2.42.0 on Debian testing, linked with OpenSSL (v3.0.10 from\n> distribution) with Makefile knob `OPENSSL_SHA1=YesPlease \n> OPENSSL_SHA256=YesPlease`. I tried to shallow clone git.git repository:\n\nI should point out that using OpenSSL's SHA-1 support is insecure\nbecause it doesn't check for collisions.  As a practical matter, no\ndistro builds that way, and if you distributed that build, it would\nprobably qualify for a CVE.\n\nHowever, OPENSSL_SHA256 being set is fine for a local build or a build\nwhere you're not distributing OpenSSL itself.\n\n> ```\n> $ git clone https://github.com/git/git --depth=1 git-scm\n> ```\n> \n> All the necessary objects were fetched but the clone errored instead with:\n> \n> ```\n> fatal: fetch-pack: invalid index-pack output\n> ```\n> \n> This issue is a regression since v2.41.0 doesn't have it. Bisecting, the\n> culprit is commit bda9c12073e7 (avoid SHA-1 functions deprecated in OpenSSL 3+,\n> 2023-08-01). AFAIK, the culprit doesn't touch `fetch-pack.c` as I hoped.\n\nI also see this with that configuration on Debian sid, and it appears to\naffect SHA-256 as well.  The testsuite fails in a variety of spectacular\nways.  For example, t0410 is broken in exactly this way.\n\nA simple git index-pack on Git's codebase shows a segfault in\n`EVP_DigestUpdate` after calling `flush`.  It's not clear to me why this\nwould occur, but I did note that the context is a static variable.  I\nwonder if there's something about this configuration that results in\nbreakage if a context is reused, although looking at the code, nothing\njumps out to me.\n\nI did, however, apply this patch, which I think makes the problem really\nclear:\n\n----\ndiff --git a/sha1/openssl.h b/sha1/openssl.h\nindex 006c1f4ba5..0390ba9da6 100644\n--- a/sha1/openssl.h\n+++ b/sha1/openssl.h\n@@ -32,6 +32,7 @@ static inline void openssl_SHA1_Final(unsigned char *digest,\n {\n \tEVP_DigestFinal_ex(ctx->ectx, digest, NULL);\n \tEVP_MD_CTX_free(ctx->ectx);\n+\tctx->ectx = NULL;\n }\n \n static inline void openssl_SHA1_Clone(struct openssl_SHA1_CTX *dst,\n----\n\nNow we see that when the segfault happens, `input_ctx.sha1.ectx` is\nNULL.  I'm not sure why that is, or what needs to be fixed, but I think\nit's clear that _someone_ isn't calling the `init_fn` method before\nre-using the context, and they definitely should be.\n\nHopefully this gives someone a good push in the right direction on\nsolving the problem.\n\nIf someone wants to pick up the above patch to help make this problem\nmore obvious in the future (don't forget to do the same for SHA-256),\nplease do so with my blessing.  I wouldn't say you need my sign-off\nsince it's so trivial, but feel free to forge it if it makes you feel\nbetter.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"481276","messageId":"20230901005742.M783359@dcvr","threadId":"60179","inReplyTo":"ZPEf8kbBUFqLO25W@tapette.crustytoothpaste.net","subject":"Re: [REGRESSION] Can't clone GitHub repos (fetch-pack error) due to avoiding deprecated OpenSSL SHA-1 routines","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2023-09-01T00:57:42Z","receivedAt":"2023-09-01T00:57:48Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"\"brian m. carlson\" <sandals@crustytoothpaste.net> wrote:\n> Hopefully this gives someone a good push in the right direction on\n> solving the problem.\n\nThanks.  Here's my WIP which fixes clones on SHA-1 and SHA-256\n <https://80x24.org/sha256test.git> and also t1050-large.sh, but\nt1514-rev-parse-push.sh is still broken...\n\nThat said, I don't much understand some of the code I'm modifying\nand just poking at it until tests pass and valgrind is happy :x\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 006ffdc9c5..dda94a9f46 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1166,6 +1166,7 @@ static void parse_pack_objects(unsigned char *hash)\n \tstruct ofs_delta_entry *ofs_delta = ofs_deltas;\n \tstruct object_id ref_delta_oid;\n \tstruct stat st;\n+\tgit_hash_ctx tmp_ctx;\n \n \tif (verbose)\n \t\tprogress = start_progress(\n@@ -1202,7 +1203,9 @@ static void parse_pack_objects(unsigned char *hash)\n \n \t/* Check pack integrity */\n \tflush();\n-\tthe_hash_algo->final_fn(hash, &input_ctx);\n+\tthe_hash_algo->init_fn(&tmp_ctx);\n+\tthe_hash_algo->clone_fn(&tmp_ctx, &input_ctx);\n+\tthe_hash_algo->final_fn(hash, &tmp_ctx);\n \tif (!hasheq(fill(the_hash_algo->rawsz), hash))\n \t\tdie(_(\"pack is corrupted (SHA1 mismatch)\"));\n \tuse(the_hash_algo->rawsz);\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex 73bff3a23d..92b9c8598b 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -268,6 +268,7 @@ static int deflate_to_pack(struct bulk_checkin_packfile *state,\n \t\t\t\t\t  type, size);\n \tthe_hash_algo->init_fn(&ctx);\n \tthe_hash_algo->update_fn(&ctx, obuf, header_len);\n+\tthe_hash_algo->init_fn(&checkpoint.ctx);\n \n \t/* Note: idx is non-NULL when we are writing */\n \tif ((flags & HASH_WRITE_OBJECT) != 0)\ndiff --git a/csum-file.c b/csum-file.c\nindex cd01713244..870748e016 100644\n--- a/csum-file.c\n+++ b/csum-file.c\n@@ -207,7 +207,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint\n \t    lseek(f->fd, offset, SEEK_SET) != offset)\n \t\treturn -1;\n \tf->total = offset;\n-\tf->ctx = checkpoint->ctx;\n+\tthe_hash_algo->clone_fn(&f->ctx, &checkpoint->ctx);\n \tf->offset = 0; /* hashflush() was called in checkpoint */\n \treturn 0;\n }\n"},{"id":"481277","messageId":"20230901020928.M610756@dcvr","threadId":"60179","inReplyTo":"20230901005742.M783359@dcvr","subject":"[PATCH] treewide: fix various bugs w/ OpenSSL 3+ EVP API","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2023-09-01T02:09:28Z","receivedAt":"2023-09-01T02:09:46Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"The OpenSSL 3+ EVP API for SHA-* cannot support our prior use cases\nsupported by other SHA-* implementations.  It has the following\ndifferences:\n\n1. ->init_fn is required before all use\n2. struct assignments don't work and requires ->clone_fn\n3. can't support ->update_fn after ->final_*fn\n\nWhile fixing cases 1 and 2 is merely the matter of calling ->init_fn and\n->clone_fn as appropriate, fixing case 3 requires calling ->final_*fn on\na temporary context that's cloned from the primary context.\n\nReported-by: Bagas Sanjaya <bagasdotme@gmail.com>\nLink: https://lore.kernel.org/ZPCL11k38PXTkFga@debian.me/\nHelped-by: brian m. carlson <sandals@crustytoothpaste.net>\nFixes: 3e440ea0aba0 (\"sha256: avoid functions deprecated in OpenSSL 3+\")\nFixes: bda9c12073e7 (\"avoid SHA-1 functions deprecated in OpenSSL 3+\")\nSigned-off-by: Eric Wong <e@80x24.org>\n---\n Ugh, I wonder if I setup my config.mak incorrectly when testing\n 3e440ea0aba0 and bda9c12073e7 :x\n\n There may be other misuses not exposed by the test suite.  Making\n git_hash_ctx opaque could flush out some of them (but I dislike\n APIs which force heap allocations in the first place).  In any case,\n I really wish git relied less on globals so object lifetimes could be\n more obvious and really wish all C projects could rely on\n gcc/tinycc/clang-supported __attribute__((__cleanup__)) to make\n lifetimes easier-to-manage...\n\n builtin/fast-import.c    | 1 +\n builtin/index-pack.c     | 5 ++++-\n builtin/unpack-objects.c | 5 ++++-\n bulk-checkin.c           | 1 +\n csum-file.c              | 2 +-\n 5 files changed, 11 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/fast-import.c b/builtin/fast-import.c\nindex 4dbb10aff3..444f41cf8c 100644\n--- a/builtin/fast-import.c\n+++ b/builtin/fast-import.c\n@@ -1102,6 +1102,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n \t\t|| (pack_size + PACK_SIZE_THRESHOLD + len) < pack_size)\n \t\tcycle_packfile();\n \n+\tthe_hash_algo->init_fn(&checkpoint.ctx);\n \thashfile_checkpoint(pack_file, &checkpoint);\n \toffset = checkpoint.offset;\n \ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 006ffdc9c5..dda94a9f46 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1166,6 +1166,7 @@ static void parse_pack_objects(unsigned char *hash)\n \tstruct ofs_delta_entry *ofs_delta = ofs_deltas;\n \tstruct object_id ref_delta_oid;\n \tstruct stat st;\n+\tgit_hash_ctx tmp_ctx;\n \n \tif (verbose)\n \t\tprogress = start_progress(\n@@ -1202,7 +1203,9 @@ static void parse_pack_objects(unsigned char *hash)\n \n \t/* Check pack integrity */\n \tflush();\n-\tthe_hash_algo->final_fn(hash, &input_ctx);\n+\tthe_hash_algo->init_fn(&tmp_ctx);\n+\tthe_hash_algo->clone_fn(&tmp_ctx, &input_ctx);\n+\tthe_hash_algo->final_fn(hash, &tmp_ctx);\n \tif (!hasheq(fill(the_hash_algo->rawsz), hash))\n \t\tdie(_(\"pack is corrupted (SHA1 mismatch)\"));\n \tuse(the_hash_algo->rawsz);\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex 32505255a0..fef7423448 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -609,6 +609,7 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n {\n \tint i;\n \tstruct object_id oid;\n+\tgit_hash_ctx tmp_ctx;\n \n \tdisable_replace_refs();\n \n@@ -669,7 +670,9 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n \tthe_hash_algo->init_fn(&ctx);\n \tunpack_all();\n \tthe_hash_algo->update_fn(&ctx, buffer, offset);\n-\tthe_hash_algo->final_oid_fn(&oid, &ctx);\n+\tthe_hash_algo->init_fn(&tmp_ctx);\n+\tthe_hash_algo->clone_fn(&tmp_ctx, &ctx);\n+\tthe_hash_algo->final_oid_fn(&oid, &tmp_ctx);\n \tif (strict) {\n \t\twrite_rest();\n \t\tif (fsck_finish(&fsck_options))\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex 73bff3a23d..92b9c8598b 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -268,6 +268,7 @@ static int deflate_to_pack(struct bulk_checkin_packfile *state,\n \t\t\t\t\t  type, size);\n \tthe_hash_algo->init_fn(&ctx);\n \tthe_hash_algo->update_fn(&ctx, obuf, header_len);\n+\tthe_hash_algo->init_fn(&checkpoint.ctx);\n \n \t/* Note: idx is non-NULL when we are writing */\n \tif ((flags & HASH_WRITE_OBJECT) != 0)\ndiff --git a/csum-file.c b/csum-file.c\nindex cd01713244..870748e016 100644\n--- a/csum-file.c\n+++ b/csum-file.c\n@@ -207,7 +207,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint\n \t    lseek(f->fd, offset, SEEK_SET) != offset)\n \t\treturn -1;\n \tf->total = offset;\n-\tf->ctx = checkpoint->ctx;\n+\tthe_hash_algo->clone_fn(&f->ctx, &checkpoint->ctx);\n \tf->offset = 0; /* hashflush() was called in checkpoint */\n \treturn 0;\n }\n"},{"id":"481278","messageId":"xmqqledqsbl5.fsf@gitster.g","threadId":"60179","inReplyTo":"20230901020928.M610756@dcvr","subject":"Re: [PATCH] treewide: fix various bugs w/ OpenSSL 3+ EVP API","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-09-01T05:32:06Z","receivedAt":"2023-09-01T05:32:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong <e@80x24.org> writes:\n\n> The OpenSSL 3+ EVP API for SHA-* cannot support our prior use cases\n> supported by other SHA-* implementations.  It has the following\n> differences:\n>\n> 1. ->init_fn is required before all use\n> 2. struct assignments don't work and requires ->clone_fn\n> 3. can't support ->update_fn after ->final_*fn\n>\n> While fixing cases 1 and 2 is merely the matter of calling ->init_fn and\n> ->clone_fn as appropriate, fixing case 3 requires calling ->final_*fn on\n> a temporary context that's cloned from the primary context.\n>\n> Reported-by: Bagas Sanjaya <bagasdotme@gmail.com>\n> Link: https://lore.kernel.org/ZPCL11k38PXTkFga@debian.me/\n> Helped-by: brian m. carlson <sandals@crustytoothpaste.net>\n> Fixes: 3e440ea0aba0 (\"sha256: avoid functions deprecated in OpenSSL 3+\")\n> Fixes: bda9c12073e7 (\"avoid SHA-1 functions deprecated in OpenSSL 3+\")\n> Signed-off-by: Eric Wong <e@80x24.org>\n> ---\n>  Ugh, I wonder if I setup my config.mak incorrectly when testing\n>  3e440ea0aba0 and bda9c12073e7 :x\n\nThe third kind looks like a fun one to diagnoise and fix.\n\nThanks.  Will queue.\n\n>  There may be other misuses not exposed by the test suite.  Making\n>  git_hash_ctx opaque could flush out some of them (but I dislike\n>  APIs which force heap allocations in the first place).  In any case,\n>  I really wish git relied less on globals so object lifetimes could be\n>  more obvious and really wish all C projects could rely on\n>  gcc/tinycc/clang-supported __attribute__((__cleanup__)) to make\n>  lifetimes easier-to-manage...\n>\n>  builtin/fast-import.c    | 1 +\n>  builtin/index-pack.c     | 5 ++++-\n>  builtin/unpack-objects.c | 5 ++++-\n>  bulk-checkin.c           | 1 +\n>  csum-file.c              | 2 +-\n>  5 files changed, 11 insertions(+), 3 deletions(-)\n>\n> diff --git a/builtin/fast-import.c b/builtin/fast-import.c\n> index 4dbb10aff3..444f41cf8c 100644\n> --- a/builtin/fast-import.c\n> +++ b/builtin/fast-import.c\n> @@ -1102,6 +1102,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n>  \t\t|| (pack_size + PACK_SIZE_THRESHOLD + len) < pack_size)\n>  \t\tcycle_packfile();\n>  \n> +\tthe_hash_algo->init_fn(&checkpoint.ctx);\n>  \thashfile_checkpoint(pack_file, &checkpoint);\n>  \toffset = checkpoint.offset;\n>  \n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 006ffdc9c5..dda94a9f46 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -1166,6 +1166,7 @@ static void parse_pack_objects(unsigned char *hash)\n>  \tstruct ofs_delta_entry *ofs_delta = ofs_deltas;\n>  \tstruct object_id ref_delta_oid;\n>  \tstruct stat st;\n> +\tgit_hash_ctx tmp_ctx;\n>  \n>  \tif (verbose)\n>  \t\tprogress = start_progress(\n> @@ -1202,7 +1203,9 @@ static void parse_pack_objects(unsigned char *hash)\n>  \n>  \t/* Check pack integrity */\n>  \tflush();\n> -\tthe_hash_algo->final_fn(hash, &input_ctx);\n> +\tthe_hash_algo->init_fn(&tmp_ctx);\n> +\tthe_hash_algo->clone_fn(&tmp_ctx, &input_ctx);\n> +\tthe_hash_algo->final_fn(hash, &tmp_ctx);\n>  \tif (!hasheq(fill(the_hash_algo->rawsz), hash))\n>  \t\tdie(_(\"pack is corrupted (SHA1 mismatch)\"));\n>  \tuse(the_hash_algo->rawsz);\n> diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\n> index 32505255a0..fef7423448 100644\n> --- a/builtin/unpack-objects.c\n> +++ b/builtin/unpack-objects.c\n> @@ -609,6 +609,7 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n>  {\n>  \tint i;\n>  \tstruct object_id oid;\n> +\tgit_hash_ctx tmp_ctx;\n>  \n>  \tdisable_replace_refs();\n>  \n> @@ -669,7 +670,9 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n>  \tthe_hash_algo->init_fn(&ctx);\n>  \tunpack_all();\n>  \tthe_hash_algo->update_fn(&ctx, buffer, offset);\n> -\tthe_hash_algo->final_oid_fn(&oid, &ctx);\n> +\tthe_hash_algo->init_fn(&tmp_ctx);\n> +\tthe_hash_algo->clone_fn(&tmp_ctx, &ctx);\n> +\tthe_hash_algo->final_oid_fn(&oid, &tmp_ctx);\n>  \tif (strict) {\n>  \t\twrite_rest();\n>  \t\tif (fsck_finish(&fsck_options))\n> diff --git a/bulk-checkin.c b/bulk-checkin.c\n> index 73bff3a23d..92b9c8598b 100644\n> --- a/bulk-checkin.c\n> +++ b/bulk-checkin.c\n> @@ -268,6 +268,7 @@ static int deflate_to_pack(struct bulk_checkin_packfile *state,\n>  \t\t\t\t\t  type, size);\n>  \tthe_hash_algo->init_fn(&ctx);\n>  \tthe_hash_algo->update_fn(&ctx, obuf, header_len);\n> +\tthe_hash_algo->init_fn(&checkpoint.ctx);\n>  \n>  \t/* Note: idx is non-NULL when we are writing */\n>  \tif ((flags & HASH_WRITE_OBJECT) != 0)\n> diff --git a/csum-file.c b/csum-file.c\n> index cd01713244..870748e016 100644\n> --- a/csum-file.c\n> +++ b/csum-file.c\n> @@ -207,7 +207,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint\n>  \t    lseek(f->fd, offset, SEEK_SET) != offset)\n>  \t\treturn -1;\n>  \tf->total = offset;\n> -\tf->ctx = checkpoint->ctx;\n> +\tthe_hash_algo->clone_fn(&f->ctx, &checkpoint->ctx);\n>  \tf->offset = 0; /* hashflush() was called in checkpoint */\n>  \treturn 0;\n>  }\n"},{"id":"481279","messageId":"ZPGI02eZLmFGKCaE@ugly","threadId":"60179","inReplyTo":"20230901020928.M610756@dcvr","subject":"Re: [PATCH] treewide: fix various bugs w/ OpenSSL 3+ EVP API","fromName":"Oswald Buddenhagen","fromEmail":"oswald.buddenhagen@gmx.de","sentAt":"2023-09-01T06:46:43Z","receivedAt":"2023-09-01T06:46:49Z","isPatch":true,"sender":{"key":"oswald.buddenhagen@gmx.de","avatar":"https://avatars.githubusercontent.com/u/812380?v=4"},"body":"On Fri, Sep 01, 2023 at 02:09:28AM +0000, Eric Wong wrote:\n>@@ -1202,7 +1203,9 @@ static void parse_pack_objects(unsigned char *hash)\n> \n> \t/* Check pack integrity */\n> \tflush();\n>+\tthe_hash_algo->init_fn(&tmp_ctx);\n>\ndoes it make sense (and doesn't it potentially even cause a leak) to \ninit the target before cloning into it? at least the fallback simply \nmemcpy()s over it.\n\n>@@ -669,7 +670,9 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n> \tthe_hash_algo->init_fn(&ctx);\n> \tunpack_all();\n> \tthe_hash_algo->update_fn(&ctx, buffer, offset);\n>+\tthe_hash_algo->init_fn(&tmp_ctx);\n>\nditto\n\n>+\tthe_hash_algo->clone_fn(&tmp_ctx, &ctx);\n>+\tthe_hash_algo->final_oid_fn(&oid, &tmp_ctx);\n\nregards\n"},{"id":"481283","messageId":"ZPHEu6qjwP478vPg@debian.me","threadId":"60179","inReplyTo":"20230901020928.M610756@dcvr","subject":"Re: [PATCH] treewide: fix various bugs w/ OpenSSL 3+ EVP API","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2023-09-01T11:02:19Z","receivedAt":"2023-09-01T11:02:27Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On Fri, Sep 01, 2023 at 02:09:28AM +0000, Eric Wong wrote:\n> diff --git a/builtin/fast-import.c b/builtin/fast-import.c\n> index 4dbb10aff3..444f41cf8c 100644\n> --- a/builtin/fast-import.c\n> +++ b/builtin/fast-import.c\n> @@ -1102,6 +1102,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n>  \t\t|| (pack_size + PACK_SIZE_THRESHOLD + len) < pack_size)\n>  \t\tcycle_packfile();\n>  \n> +\tthe_hash_algo->init_fn(&checkpoint.ctx);\n>  \thashfile_checkpoint(pack_file, &checkpoint);\n>  \toffset = checkpoint.offset;\n>  \n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 006ffdc9c5..dda94a9f46 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -1166,6 +1166,7 @@ static void parse_pack_objects(unsigned char *hash)\n>  \tstruct ofs_delta_entry *ofs_delta = ofs_deltas;\n>  \tstruct object_id ref_delta_oid;\n>  \tstruct stat st;\n> +\tgit_hash_ctx tmp_ctx;\n>  \n>  \tif (verbose)\n>  \t\tprogress = start_progress(\n> @@ -1202,7 +1203,9 @@ static void parse_pack_objects(unsigned char *hash)\n>  \n>  \t/* Check pack integrity */\n>  \tflush();\n> -\tthe_hash_algo->final_fn(hash, &input_ctx);\n> +\tthe_hash_algo->init_fn(&tmp_ctx);\n> +\tthe_hash_algo->clone_fn(&tmp_ctx, &input_ctx);\n> +\tthe_hash_algo->final_fn(hash, &tmp_ctx);\n>  \tif (!hasheq(fill(the_hash_algo->rawsz), hash))\n>  \t\tdie(_(\"pack is corrupted (SHA1 mismatch)\"));\n>  \tuse(the_hash_algo->rawsz);\n> diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\n> index 32505255a0..fef7423448 100644\n> --- a/builtin/unpack-objects.c\n> +++ b/builtin/unpack-objects.c\n> @@ -609,6 +609,7 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n>  {\n>  \tint i;\n>  \tstruct object_id oid;\n> +\tgit_hash_ctx tmp_ctx;\n>  \n>  \tdisable_replace_refs();\n>  \n> @@ -669,7 +670,9 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix UNUSED)\n>  \tthe_hash_algo->init_fn(&ctx);\n>  \tunpack_all();\n>  \tthe_hash_algo->update_fn(&ctx, buffer, offset);\n> -\tthe_hash_algo->final_oid_fn(&oid, &ctx);\n> +\tthe_hash_algo->init_fn(&tmp_ctx);\n> +\tthe_hash_algo->clone_fn(&tmp_ctx, &ctx);\n> +\tthe_hash_algo->final_oid_fn(&oid, &tmp_ctx);\n>  \tif (strict) {\n>  \t\twrite_rest();\n>  \t\tif (fsck_finish(&fsck_options))\n> diff --git a/bulk-checkin.c b/bulk-checkin.c\n> index 73bff3a23d..92b9c8598b 100644\n> --- a/bulk-checkin.c\n> +++ b/bulk-checkin.c\n> @@ -268,6 +268,7 @@ static int deflate_to_pack(struct bulk_checkin_packfile *state,\n>  \t\t\t\t\t  type, size);\n>  \tthe_hash_algo->init_fn(&ctx);\n>  \tthe_hash_algo->update_fn(&ctx, obuf, header_len);\n> +\tthe_hash_algo->init_fn(&checkpoint.ctx);\n>  \n>  \t/* Note: idx is non-NULL when we are writing */\n>  \tif ((flags & HASH_WRITE_OBJECT) != 0)\n> diff --git a/csum-file.c b/csum-file.c\n> index cd01713244..870748e016 100644\n> --- a/csum-file.c\n> +++ b/csum-file.c\n> @@ -207,7 +207,7 @@ int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint\n>  \t    lseek(f->fd, offset, SEEK_SET) != offset)\n>  \t\treturn -1;\n>  \tf->total = offset;\n> -\tf->ctx = checkpoint->ctx;\n> +\tthe_hash_algo->clone_fn(&f->ctx, &checkpoint->ctx);\n>  \tf->offset = 0; /* hashflush() was called in checkpoint */\n>  \treturn 0;\n>  }\n\nThe regression gone away, thanks!\n\nTested-by: Bagas Sanjaya <bagasdotme@gmail.com>\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"481284","messageId":"ZPHGX4Dd0Mc1VVAQ@debian.me","threadId":"60179","inReplyTo":"ZPEf8kbBUFqLO25W@tapette.crustytoothpaste.net","subject":"Re: [REGRESSION] Can't clone GitHub repos (fetch-pack error) due to avoiding deprecated OpenSSL SHA-1 routines","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2023-09-01T11:09:19Z","receivedAt":"2023-09-01T11:09:26Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On Thu, Aug 31, 2023 at 11:19:14PM +0000, brian m. carlson wrote:\n> On 2023-08-31 at 12:47:19, Bagas Sanjaya wrote:\n> > Hi,\n> > \n> > I built Git v2.42.0 on Debian testing, linked with OpenSSL (v3.0.10 from\n> > distribution) with Makefile knob `OPENSSL_SHA1=YesPlease \n> > OPENSSL_SHA256=YesPlease`. I tried to shallow clone git.git repository:\n> \n> I should point out that using OpenSSL's SHA-1 support is insecure\n> because it doesn't check for collisions.  As a practical matter, no\n> distro builds that way, and if you distributed that build, it would\n> probably qualify for a CVE.\n> \n> However, OPENSSL_SHA256 being set is fine for a local build or a build\n> where you're not distributing OpenSSL itself.\n\nThanks for the disclaimer. I did such build for myself since the distro\nversion always lagging.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"}]}