git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 4/4] unbundle: use VERIFY_BUNDLE_FSCK_FOLLOW_FETCH for fetches

From
XGXing Xin via GitGitGadget <gitgitgadget@gmail.com>
Date
Jun 11, 2024, 06:42 UTC
Message-ID
<eb9f21f16b5d2b01e67510eaec160cecabbcc049.1718088127.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1730.v5.git.1718088126.gitgitgadget@gmail.com>
From: Xing Xin <xingxin.xx@bytedance.com>

This commit passes `VERIFY_BUNDLE_FSCK_FOLLOW_FETCH` to `unbundle` in the fetching process, including:

- `transport.c:fetch_refs_from_bundle` for direct bundle fetches.
- `bundle-uri.c:unbundle_from_file` for bundle-uri enabled fetches.

This addition ensures a consistent logic for object verification during fetch operations. Tests have been added to confirm functionality in the scenarios mentioned above.

Reviewed-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Xing Xin <xingxin.xx@bytedance.com>
---
 bundle-uri.c                |  2 +-
 t/t5558-clone-bundle-uri.sh | 35 ++++++++++++++++++++++++++++++++++-
 t/t5607-clone-bundle.sh     | 33 +++++++++++++++++++++++++++++++++
 transport.c                 |  2 +-
 4 files changed, 69 insertions(+), 3 deletions(-)
diff --git a/bundle-uri.c b/bundle-uri.c
index 65666a11d9c..e7ebac6ce57 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -373,7 +373,7 @@ static int unbundle_from_file(struct repository *r, const char *file)
 	 * the prerequisite commits.
 	 */
 	if ((result = unbundle(r, &header, bundle_fd, NULL,
-			       VERIFY_BUNDLE_QUIET)))
+			       VERIFY_BUNDLE_QUIET | VERIFY_BUNDLE_FSCK_FOLLOW_FETCH)))
 		return 1;
 
 	/*
diff --git a/t/t5558-clone-bundle-uri.sh b/t/t5558-clone-bundle-uri.sh
index 8f4f802e4f1..48be1b18802 100755
--- a/t/t5558-clone-bundle-uri.sh
+++ b/t/t5558-clone-bundle-uri.sh
@@ -30,7 +30,21 @@ test_expect_success 'create bundle' '
 		git bundle create B.bundle topic &&
 
 		# Create a bundle with reference pointing to non-existent object.
-		sed "s/$(git rev-parse A)/$(git rev-parse B)/" <A.bundle >bad-header.bundle
+		sed "s/$(git rev-parse A)/$(git rev-parse B)/" <A.bundle >bad-header.bundle &&
+
+		cat >data <<-EOF &&
+		tree $(git rev-parse HEAD^{tree})
+		parent $(git rev-parse HEAD)
+		author A U Thor
+		committer A U Thor
+
+		commit: this is a commit with bad emails
+
+		EOF
+		git hash-object --literally -t commit -w --stdin <data >commit &&
+		git branch bad $(cat commit) &&
+		git bundle create bad-object.bundle bad &&
+		git update-ref -d refs/heads/bad
 	)
 '
 
@@ -52,6 +66,25 @@ test_expect_success 'clone with bundle that has bad header' '
 	! grep "refs/bundles/" refs
 '
 
+test_expect_success 'clone with bundle that has bad object' '
+	# Unbundle succeeds if no fsckObjects confugured.
+	git clone --bundle-uri="clone-from/bad-object.bundle" \
+		clone-from clone-bad-object-no-fsck &&
+	git -C clone-bad-object-no-fsck for-each-ref --format="%(refname)" >refs &&
+	grep "refs/bundles/" refs >actual &&
+	cat >expect <<-\EOF &&
+	refs/bundles/bad
+	EOF
+	test_cmp expect actual &&
+
+	# Unbundle fails with fsckObjects set true, but clone can still proceed.
+	git -c fetch.fsckObjects=true clone --bundle-uri="clone-from/bad-object.bundle" \
+		clone-from clone-bad-object-fsck 2>err &&
+	test_grep "missingEmail" err &&
+	git -C clone-bad-object-fsck for-each-ref --format="%(refname)" >refs &&
+	! grep "refs/bundles/" refs
+'
+
 test_expect_success 'clone with path bundle and non-default hash' '
 	test_when_finished "rm -rf clone-path-non-default-hash" &&
 	GIT_DEFAULT_HASH=sha256 git clone --bundle-uri="clone-from/B.bundle" \
diff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh
index 0d1e92d9963..5182efc0b45 100755
--- a/t/t5607-clone-bundle.sh
+++ b/t/t5607-clone-bundle.sh
@@ -138,6 +138,39 @@ test_expect_success 'fetch SHA-1 from bundle' '
 	git fetch --no-tags foo/tip.bundle "$(cat hash)"
 '
 
+test_expect_success 'clone bundle with different fsckObjects configurations' '
+	test_create_repo bundle-fsck &&
+	(
+		cd bundle-fsck &&
+		test_commit first &&
+		cat >data <<-EOF &&
+		tree $(git rev-parse HEAD^{tree})
+		parent $(git rev-parse HEAD)
+		author A U Thor
+		committer A U Thor
+
+		commit: this is a commit with bad emails
+
+		EOF
+		git hash-object --literally -t commit -w --stdin <data >commit &&
+		git branch bad $(cat commit) &&
+		git bundle create bad.bundle bad
+	) &&
+
+	git clone bundle-fsck/bad.bundle bundle-no-fsck &&
+
+	git -c fetch.fsckObjects=false -c transfer.fsckObjects=true \
+		clone bundle-fsck/bad.bundle bundle-fetch-no-fsck &&
+
+	test_must_fail git -c fetch.fsckObjects=true \
+		clone bundle-fsck/bad.bundle bundle-fetch-fsck 2>err &&
+	test_grep "missingEmail" err &&
+
+	test_must_fail git -c transfer.fsckObjects=true \
+		clone bundle-fsck/bad.bundle bundle-transfer-fsck 2>err &&
+	test_grep "missingEmail" err
+'
+
 test_expect_success 'git bundle uses expected default format' '
 	git bundle create bundle HEAD^.. &&
 	cat >expect <<-EOF &&
diff --git a/transport.c b/transport.c
index 0ad04b77fd2..6cd5683bb45 100644
--- a/transport.c
+++ b/transport.c
@@ -184,7 +184,7 @@ static int fetch_refs_from_bundle(struct transport *transport,
 	if (!data->get_refs_from_bundle_called)
 		get_refs_from_bundle_inner(transport);
 	ret = unbundle(the_repository, &data->header, data->fd,
-		       &extra_index_pack_args, 0);
+		       &extra_index_pack_args, VERIFY_BUNDLE_FSCK_FOLLOW_FETCH);
 	transport->hash_algo = data->header.hash_algo;
 	return ret;
 }
-- 
gitgitgadget
Previous: Xing XinNext: blanet via GitGitGadget
Message 46 of 66 in “bundle-uri: refresh packed_git if unbundle succeed”
  1. bundle-uri: refresh packed_git if unbundle succeedblanet via GitGitGadget, May 15, 2024
  2. Patrick SteinhardtMay 17, 2024
  3. Junio C HamanoMay 17, 2024
  4. Xing XinMay 20, 2024
  5. Junio C HamanoMay 20, 2024
  6. Xing XinMay 27, 2024
  7. Xing XinMay 20, 2024
  8. Karthik NayakMay 17, 2024
  9. Xing XinMay 20, 2024
  10. bundle-uri: verify oid before writing refsblanet via GitGitGadget, May 20, 2024
  11. Karthik NayakMay 21, 2024
  12. 0/4 object checking related additions and fixes for bundles in fetchesblanet via GitGitGadget, May 27, 2024
  13. 1/4 bundle-uri: verify oid before writing refsXing Xin via GitGitGadget, May 27, 2024
  14. Patrick SteinhardtMay 28, 2024
  15. Xing XinMay 30, 2024
  16. 2/4 unbundle: introduce unbundle_fsck_flags for fsckobjects handlingXing Xin via GitGitGadget, May 27, 2024
  17. Patrick SteinhardtMay 28, 2024
  18. Xing XinMay 29, 2024
  19. Patrick SteinhardtMay 30, 2024
  20. Xing XinMay 30, 2024
  21. 3/4 fetch-pack: expose fsckObjects configuration logicXing Xin via GitGitGadget, May 27, 2024
  22. Patrick SteinhardtMay 28, 2024
  23. Junio C HamanoMay 28, 2024
  24. Junio C HamanoMay 28, 2024
  25. Patrick SteinhardtMay 29, 2024
  26. Xing XinMay 30, 2024
  27. Patrick SteinhardtMay 29, 2024
  28. 4/4 unbundle: introduce new option UNBUNDLE_FSCK_FOLLOW_FETCHXing Xin via GitGitGadget, May 27, 2024
  29. Patrick SteinhardtMay 28, 2024
  30. Xing XinMay 30, 2024
  31. 0/4 object checking related additions and fixes for bundles in fetchesblanet via GitGitGadget, May 30, 2024
  32. 1/4 bundle-uri: verify oid before writing refsXing Xin via GitGitGadget, May 30, 2024
  33. 2/4 unbundle: extend verify_bundle_flags to support fsck-objectsXing Xin via GitGitGadget, May 30, 2024
  34. Patrick SteinhardtJun 6, 2024
  35. Xing XinJun 11, 2024
  36. 3/4 fetch-pack: expose fsckObjects configuration logicXing Xin via GitGitGadget, May 30, 2024
  37. 4/4 unbundle: introduce option VERIFY_BUNDLE_FSCK_FOLLOW_FETCHXing Xin via GitGitGadget, May 30, 2024
  38. Patrick SteinhardtJun 6, 2024
  39. Xing XinJun 11, 2024
  40. 0/4 object checking related additions and fixes for bundles in fetchesblanet via GitGitGadget, Jun 11, 2024
  41. 2/4 fetch-pack: expose fsckObjects configuration logicXing Xin via GitGitGadget, Jun 11, 2024
  42. 1/4 bundle-uri: verify oid before writing refsXing Xin via GitGitGadget, Jun 11, 2024
  43. 3/4 unbundle: extend options to support object verificationXing Xin via GitGitGadget, Jun 11, 2024
  44. Patrick SteinhardtJun 11, 2024
  45. Xing XinJun 11, 2024
  46. 4/4 unbundle: use VERIFY_BUNDLE_FSCK_FOLLOW_FETCH for fetchesXing Xin via GitGitGadget, Jun 11, 2024
  47. 0/3 object checking related additions and fixes for bundles in fetchesblanet via GitGitGadget, Jun 11, 2024
  48. 1/3 bundle-uri: verify oid before writing refsXing Xin via GitGitGadget, Jun 11, 2024
  49. Junio C HamanoJun 11, 2024
  50. Xing XinJun 17, 2024
  51. 2/3 fetch-pack: expose fsckObjects configuration logicXing Xin via GitGitGadget, Jun 11, 2024
  52. Junio C HamanoJun 11, 2024
  53. 3/3 unbundle: support object verification for fetchesXing Xin via GitGitGadget, Jun 11, 2024
  54. Junio C HamanoJun 11, 2024
  55. Xing XinJun 12, 2024
  56. Patrick SteinhardtJun 11, 2024
  57. 0/3 object checking related additions and fixes for bundles in fetchesblanet via GitGitGadget, Jun 17, 2024
  58. 1/3 bundle-uri: verify oid before writing refsXing Xin via GitGitGadget, Jun 17, 2024
  59. Junio C HamanoJun 18, 2024
  60. Xing XinJun 19, 2024
  61. 2/3 fetch-pack: expose fsckObjects configuration logicXing Xin via GitGitGadget, Jun 17, 2024
  62. 3/3 unbundle: extend object verification for fetchesXing Xin via GitGitGadget, Jun 17, 2024
  63. 0/3 object checking related additions and fixes for bundles in fetchesblanet via GitGitGadget, Jun 19, 2024
  64. 2/3 fetch-pack: expose fsckObjects configuration logicXing Xin via GitGitGadget, Jun 19, 2024
  65. 1/3 bundle-uri: verify oid before writing refsXing Xin via GitGitGadget, Jun 19, 2024
  66. 3/3 unbundle: extend object verification for fetchesXing Xin via GitGitGadget, Jun 19, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.