git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] fetch-pack: in partial clone, pass --promisor

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Aug 24, 2020, 19:16 UTC
Message-ID
<20200824191656.3870231-1-jonathantanmy@google.com>
In-Reply-To
<cover.1598296530.git.jonathantanmy@google.com>

When fetching a pack from a promisor remote, the corresponding .promisor file needs to be created. "fetch-pack" originally did this by passing "--promisor" to "index-pack", but in 5374a290aa ("fetch-pack: write fetched refs to .promisor", 2019-10-16), "fetch-pack" was taught to do this itself instead, because it needed to store ref information in the .promisor file.

This causes a problem with superprojects when transfer.fsckobjects is set, because in the current implementation, it is "index-pack" that calls fsck_finish() to check the objects; before 5374a290aa, fsck_finish() would see that .gitmodules is a promisor object and tolerate it being missing, but after, there is no .promisor file (at the time of the invocation of fsck_finish() by "index-pack") to tell it that .gitmodules is a promisor object, so it returns an error.

Therefore, teach "fetch-pack" to pass "--promisor" to index pack once again. "fetch-pack" will subsequently overwrite this file with the ref information.

An alternative is to instead move object checking to "fetch-pack", and let "index-pack" only index the files. However, since "index-pack" has to inflate objects in order to index them, it seems reasonable to also let it check the objects (which also require inflated files).

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 fetch-pack.c             | 17 ++++++++++-------
 t/t5616-partial-clone.sh | 16 ++++++++++++++++
 2 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 7f20eca4f8..d467edc24e 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -866,13 +866,16 @@ static int get_pack(struct fetch_pack_args *args,
 			 * have this responsibility.
 			 */
 			args->check_self_contained_and_connected = 0;
-		/*
-		 * If we're obtaining the filename of a lockfile, we'll use
-		 * that filename to write a .promisor file with more
-		 * information below. If not, we need index-pack to do it for
-		 * us.
-		 */
-		if (!(do_keep && pack_lockfiles) && args->from_promisor)
+
+		if (args->from_promisor)
+			/*
+			 * write_promisor_file() may be called afterwards but
+			 * we still need index-pack to know that this is a
+			 * promisor pack. For example, if transfer.fsckobjects
+			 * is true, index-pack needs to know that .gitmodules
+			 * is a promisor object (so that it won't complain if
+			 * it is missing).
+			 */
 			strvec_push(&cmd.args, "--promisor");
 	}
 	else {
diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index 8827c2ed18..5a01466db4 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -163,6 +163,22 @@ test_expect_success 'manual prefetch of missing objects' '
 	test_line_count = 0 observed.oids
 '
 
+test_expect_success 'partial clone with transfer.fsckobjects=1 works with submodules' '
+	test_create_repo submodule &&
+	test_commit -C submodule mycommit &&
+
+	test_create_repo src_with_sub &&
+	test_config -C src_with_sub uploadpack.allowfilter 1 &&
+	test_config -C src_with_sub uploadpack.allowanysha1inwant 1 &&
+
+	git -C src_with_sub submodule add "file://$(pwd)/submodule" mysub &&
+	git -C src_with_sub commit -m "commit with submodule" &&
+
+	git -c transfer.fsckobjects=1 \
+		clone --filter="blob:none" "file://$(pwd)/src_with_sub" dst &&
+	test_when_finished rm -rf dst
+'
+
 test_expect_success 'partial clone with transfer.fsckobjects=1 uses index-pack --fsck-objects' '
 	git init src &&
 	test_commit -C src x &&
-- 
2.28.0.297.g1956fa8f8d-goog
Previous: Jonathan TanNext: Jonathan Tan
Message 3 of 28 in “Better threaded delta resolution in index-pack (another try)”
  1. 0/7 Better threaded delta resolution in index-pack (another try)Jonathan Tan, Aug 24, 2020
  2. 1/7 Documentation: deltaBaseCacheLimit is per-threadJonathan Tan, Aug 24, 2020
  3. fetch-pack: in partial clone, pass --promisorJonathan Tan, Aug 24, 2020
  4. Jonathan TanAug 24, 2020
  5. 2/7 index-pack: remove redundant parameterJonathan Tan, Aug 24, 2020
  6. Junio C HamanoAug 24, 2020
  7. 3/7 index-pack: unify threaded and unthreaded codeJonathan Tan, Aug 24, 2020
  8. Junio C HamanoAug 24, 2020
  9. 5/7 index-pack: calculate {ref,ofs}_{first,last} earlyJonathan Tan, Aug 24, 2020
  10. 7/7 index-pack: make quantum of work smallerJonathan Tan, Aug 24, 2020
  11. Junio C HamanoAug 24, 2020
  12. 6/7 index-pack: make resolve_delta() assume base dataJonathan Tan, Aug 24, 2020
  13. 4/7 index-pack: remove redundant child fieldJonathan Tan, Aug 24, 2020
  14. Junio C HamanoAug 24, 2020
  15. fixup! index-pack: make quantum of work smallerJonathan Tan, Aug 24, 2020
  16. Jeff KingAug 24, 2020
  17. Jonathan TanAug 25, 2020
  18. Jeff KingAug 25, 2020
  19. Jeff KingAug 25, 2020
  20. 0/7 Better threaded delta resolution in index-pack (another try)Jonathan Tan, Sep 8, 2020
  21. 2/7 index-pack: remove redundant parameterJonathan Tan, Sep 8, 2020
  22. 3/7 index-pack: unify threaded and unthreaded codeJonathan Tan, Sep 8, 2020
  23. 7/7 index-pack: make quantum of work smallerJonathan Tan, Sep 8, 2020
  24. 6/7 index-pack: make resolve_delta() assume base dataJonathan Tan, Sep 8, 2020
  25. 5/7 index-pack: calculate {ref,ofs}_{first,last} earlyJonathan Tan, Sep 8, 2020
  26. 4/7 index-pack: remove redundant child fieldJonathan Tan, Sep 8, 2020
  27. 1/7 Documentation: deltaBaseCacheLimit is per-threadJonathan Tan, Sep 8, 2020
  28. Junio C HamanoSep 8, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.