git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2] index-pack: remove fetch_if_missing=0

From
Kousik Sanagavarapu <five231003@gmail.com>
Date
Mar 10, 2023, 18:30 UTC
Message-ID
<20230310183029.19429-1-five231003@gmail.com>
In-Reply-To
<20230225052439.27096-1-five231003@gmail.com>

A collision test is triggered in sha1_object(), whenever there is an object file in our repo. If our repo is a partial clone, then checking for this file existence does not lazy-fetch the object (if the object is missing and if there are one or more promisor remotes) when fetch_if_missing is set to 0.

This global was added as a temporary measure to suppress the fetching of missing objects and can be removed once the commandshave been taught to handle these cases.

Hence, use has_object() to check for the existence of an object, which has the default behavior of not lazy-fetching in a partial clone. It is worth mentioning that this is the only place where there is potential for lazy-fetching and all other cases are properly handled, making it safe to remove this global here.

Signed-off-by: Kousik Sanagavarapu <five231003@gmail.com>
---
Sorry for the late reroll, I was having semester-end exams.
Changes since v1:
- Changed the commit message to be more clear about the
  change done here.
- Changed the test according to the previous review.
 builtin/index-pack.c     | 11 +----------
 t/t5616-partial-clone.sh | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 10 deletions(-)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 6648f2daef..8c0f36a49e 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -800,8 +800,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,
 
 	if (startup_info->have_repository) {
 		read_lock();
-		collision_test_needed =
-			has_object_file_with_flags(oid, OBJECT_INFO_QUICK);
+		collision_test_needed = has_object(the_repository, oid, 0);
 		read_unlock();
 	}
 
@@ -1728,14 +1727,6 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
 	int report_end_of_input = 0;
 	int hash_algo = 0;
 
-	/*
-	 * index-pack never needs to fetch missing objects except when
-	 * REF_DELTA bases are missing (which are explicitly handled). It only
-	 * accesses the repo to do hash collision checks and to check which
-	 * REF_DELTA bases need to be fetched.
-	 */
-	fetch_if_missing = 0;
-
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage(index_pack_usage);
 
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index f519d2a87a..46af8698ce 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -644,6 +644,41 @@ test_expect_success 'repack does not loosen promisor objects' '
 	grep "loosen_unused_packed_objects/loosened:0" trace
 '
 
+test_expect_success 'index-pack does not lazy-fetch when checking for sha1 collsions' '
+	rm -rf server promisor-remote client repo trace &&
+
+	# setup
+	git init server &&
+	for i in 1 2 3 4
+	do
+		echo $i >server/file$i &&
+		git -C server add file$i &&
+		git -C server commit -am "Commit $i" || return 1
+	done &&
+	git -C server config --local uploadpack.allowFilter 1 &&
+	git -C server config --local uploadpack.allowAnySha1InWant 1 &&
+	HASH=$(git -C server hash-object file3) &&
+
+	git init promisor-remote &&
+	git -C promisor-remote fetch --keep "file://$(pwd)/server" &&
+
+	git clone --no-checkout --filter=blob:none "file://$(pwd)/server" client &&
+	git -C client remote set-url origin "file://$(pwd)/promisor-remote" &&
+	git -C client config extensions.partialClone 1 &&
+	git -C client config remote.origin.promisor 1 &&
+
+	git init repo &&
+	echo "5" >repo/file5 &&
+	git -C repo config --local uploadpack.allowFilter 1 &&
+	git -C repo config --local uploadpack.allowAnySha1InWant 1 &&
+
+	# verify that no lazy-fetching is done when fetching from another repo
+	GIT_TRACE_PACKET="$(pwd)/trace" git -C client \
+					fetch --keep "file://$(pwd)/repo" main &&
+
+	! grep "want $HASH" trace
+'
+
 test_expect_success 'lazy-fetch in submodule succeeds' '
 	# setup
 	test_config_global protocol.file.allow always &&
-- 
2.25.1
Previous: Kousik SanagavarapuNext: Junio C Hamano
Message 5 of 20 in “index-pack: remove fetch_if_missing=0”
  1. index-pack: remove fetch_if_missing=0Kousik Sanagavarapu, Feb 25, 2023
  2. Kousik SanagavarapuFeb 27, 2023
  3. Jonathan TanFeb 27, 2023
  4. Kousik SanagavarapuFeb 28, 2023
  5. index-pack: remove fetch_if_missing=0Kousik Sanagavarapu, Mar 10, 2023
  6. Junio C HamanoMar 10, 2023
  7. Jonathan TanMar 10, 2023
  8. Junio C HamanoMar 10, 2023
  9. Jonathan TanMar 11, 2023
  10. Kousik SanagavarapuMar 12, 2023
  11. Kousik SanagavarapuMar 11, 2023
  12. Kousik SanagavarapuMar 11, 2023
  13. index-pack: remove fetch_if_missing=0Kousik Sanagavarapu, Mar 13, 2023
  14. Junio C HamanoMar 13, 2023
  15. Junio C HamanoMar 13, 2023
  16. index-pack: remove fetch_if_missing=0Kousik Sanagavarapu, Mar 17, 2023
  17. Junio C HamanoMar 17, 2023
  18. Kousik SanagavarapuMar 19, 2023
  19. Sean AllredMar 11, 2023
  20. Junio C HamanoMar 11, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.