git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 2/4] fetch: refactor --submodule-prefix

From
Glen Choo <chooglen@google.com>
Date
Nov 9, 2022, 00:47 UTC
Message-ID
<20221109004708.97668-3-chooglen@google.com>
In-Reply-To
<20221109004708.97668-1-chooglen@google.com>

Since "git fetch" learned to recurse into submodules [1], it has passed "--submodule-prefix=<path from root of superproject tree to submodule>" to print full paths to the submodules being fetched. This is the same value as "--super-prefix"'s when calling "git read-tree --recurse-submodules", as well as "git (grep|ls-files) --recurse-submodules" before they were implemented in-process [2] [3], (when they used repository.submodule_prefix to replace "--super-prefix).

Let's standardize the way we pass such prefixes. Having a per-command "--submodule-prefix" is preferable to a top-level "--super-prefix", since it gives us fine-grained control over which commands support it and which don't, and we can preserve the effort-saving properties of the top-level flag by sharing an implementation under the hood.

Refactor "git fetch" so that "--submodule-prefix" is stored in repository.submodule_prefix, and create functions in submodule.h for parsing and reading its value. Also, mark it with PARSE_OPT_HIDDEN and drop it from our documentation. We will follow this pattern for "git read-tree" in the next commit.

Since we've previously documented that repository.submodule_prefix is only set for submodules, introduce repository.is_submodule for distinguishing between the_repository and a submodule (although, this turns out to be unused, since no caller uses repository.submodule_prefix for that purpose).

[1] 7dce19d374 (fetch/pull: Add the --recurse-submodules option, 2010-11-12) [2] 188dce131f (ls-files: use repository object, 2017-06-22) [3] f9ee2fcdfa (grep: recurse in-process using 'struct repository', 2017-08-02)

Signed-off-by: Glen Choo <chooglen@google.com>
---
 Documentation/fetch-options.txt |  5 -----
 builtin/fetch.c                 |  7 +++----
 repository.c                    |  1 +
 repository.h                    |  9 +++++++--
 submodule.c                     | 35 +++++++++++++++++++++------------
 submodule.h                     | 12 +++++++++--
 6 files changed, 43 insertions(+), 26 deletions(-)
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 622bd84768..20cbd2c291 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -241,11 +241,6 @@ endif::git-pull[]
 	linkgit:git-config[1].
 
 ifndef::git-pull[]
---submodule-prefix=<path>::
-	Prepend <path> to paths printed in informative messages
-	such as "Fetching submodule foo".  This option is used
-	internally when recursing over submodules.
-
 --recurse-submodules-default=[yes|on-demand]::
 	This option is used internally to temporarily provide a
 	non-negative default value for the --recurse-submodules
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b06e454cbd..78a46389ff 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -74,7 +74,6 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
 static struct strbuf default_rla = STRBUF_INIT;
 static struct transport *gtransport;
 static struct transport *gsecondary;
-static const char *submodule_prefix = "";
 static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
 static int recurse_submodules_cli = RECURSE_SUBMODULES_DEFAULT;
 static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND;
@@ -195,8 +194,9 @@ static struct option builtin_fetch_options[] = {
 	OPT_SET_INT_F(0, "refetch", &refetch,
 		      N_("re-fetch without negotiating common commits"),
 		      1, PARSE_OPT_NONEG),
-	{ OPTION_STRING, 0, "submodule-prefix", &submodule_prefix, N_("dir"),
-		   N_("prepend this to submodule path output"), PARSE_OPT_HIDDEN },
+	OPT_CALLBACK_F(0, "submodule-prefix", NULL, "path",
+			"path from top-level superproject root to current repo root",
+			PARSE_OPT_HIDDEN, option_parse_submodule_prefix),
 	OPT_CALLBACK_F(0, "recurse-submodules-default",
 		   &recurse_submodules_default, N_("on-demand"),
 		   N_("default for recursive fetching of submodules "
@@ -2297,7 +2297,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
 		add_options_to_argv(&options);
 		result = fetch_submodules(the_repository,
 					  &options,
-					  submodule_prefix,
 					  recurse_submodules,
 					  recurse_submodules_default,
 					  verbosity < 0,
diff --git a/repository.c b/repository.c
index 5d166b692c..1d454a0ac4 100644
--- a/repository.c
+++ b/repository.c
@@ -229,6 +229,7 @@ int repo_submodule_init(struct repository *subrepo,
 		}
 	}
 
+	subrepo->is_submodule = 1;
 	subrepo->submodule_prefix = xstrfmt("%s%s/",
 					    superproject->submodule_prefix ?
 					    superproject->submodule_prefix :
diff --git a/repository.h b/repository.h
index 24316ac944..1bc4afc2b6 100644
--- a/repository.h
+++ b/repository.h
@@ -119,10 +119,15 @@ struct repository {
 	 */
 	char *worktree;
 
+	/*
+	 * Whether this struct represents a submodule of this process's main
+	 * repository.
+	 */
+	int is_submodule;
+
 	/*
 	 * Path from the root of the top-level superproject down to this
-	 * repository.  This is only non-NULL if the repository is initialized
-	 * as a submodule of another repository.
+	 * repository.
 	 */
 	char *submodule_prefix;
 
diff --git a/submodule.c b/submodule.c
index a6db8c4bbe..d84345a0b4 100644
--- a/submodule.c
+++ b/submodule.c
@@ -257,6 +257,21 @@ const char *get_toplevel_cwd_prefix(void)
 	return toplevel_cwd_prefix;
 }
 
+int option_parse_submodule_prefix(const struct option *opt,
+				  const char *arg, int unset)
+{
+	if (arg)
+		the_repository->submodule_prefix = xstrdup(arg);
+	return 0;
+}
+
+const char *get_submodule_prefix(void)
+{
+	if (!the_repository->submodule_prefix)
+		return "";
+	return the_repository->submodule_prefix;
+}
+
 /*
  * Determine if a submodule has been initialized at a given 'path'
  */
@@ -1392,7 +1407,6 @@ struct submodule_parallel_fetch {
 	int changed_count;
 	struct strvec args;
 	struct repository *r;
-	const char *prefix;
 	int command_line_option;
 	int default_option;
 	int quiet;
@@ -1583,7 +1597,7 @@ get_fetch_task_from_index(struct submodule_parallel_fetch *spf,
 		if (task->repo) {
 			if (!spf->quiet)
 				strbuf_addf(err, _("Fetching submodule %s%s\n"),
-					    spf->prefix, ce->name);
+					    get_submodule_prefix(), ce->name);
 
 			spf->index_count++;
 			return task;
@@ -1645,7 +1659,7 @@ get_fetch_task_from_changed(struct submodule_parallel_fetch *spf,
 		if (!spf->quiet)
 			strbuf_addf(err,
 				    _("Fetching submodule %s%s at commit %s\n"),
-				    spf->prefix, task->sub->path,
+				    get_submodule_prefix(), task->sub->path,
 				    find_unique_abbrev(cs_data->super_oid,
 						       DEFAULT_ABBREV));
 
@@ -1692,8 +1706,6 @@ static int get_next_submodule(struct child_process *cp, struct strbuf *err,
 		task = get_fetch_task_from_changed(spf, err);
 
 	if (task) {
-		struct strbuf submodule_prefix = STRBUF_INIT;
-
 		child_process_init(cp);
 		cp->dir = task->repo->gitdir;
 		prepare_submodule_repo_env_in_gitdir(&cp->env);
@@ -1705,13 +1717,11 @@ static int get_next_submodule(struct child_process *cp, struct strbuf *err,
 		strvec_push(&cp->args, task->default_argv);
 		strvec_push(&cp->args, "--submodule-prefix");
 
-		strbuf_addf(&submodule_prefix, "%s%s/",
-						spf->prefix,
-						task->sub->path);
-		strvec_push(&cp->args, submodule_prefix.buf);
+		strvec_pushf(&cp->args, "%s%s/",
+					get_submodule_prefix(),
+					task->sub->path);
 		*task_cb = task;
 
-		strbuf_release(&submodule_prefix);
 		string_list_insert(&spf->seen_submodule_names, task->sub->name);
 		return 1;
 	}
@@ -1723,7 +1733,7 @@ static int get_next_submodule(struct child_process *cp, struct strbuf *err,
 		spf->oid_fetch_tasks_nr--;
 
 		strbuf_addf(&submodule_prefix, "%s%s/",
-			    spf->prefix, task->sub->path);
+			    get_submodule_prefix(), task->sub->path);
 
 		child_process_init(cp);
 		prepare_submodule_repo_env_in_gitdir(&cp->env);
@@ -1829,7 +1839,7 @@ static int fetch_finish(int retvalue, struct strbuf *err,
 
 int fetch_submodules(struct repository *r,
 		     const struct strvec *options,
-		     const char *prefix, int command_line_option,
+		     int command_line_option,
 		     int default_option,
 		     int quiet, int max_parallel_jobs)
 {
@@ -1851,7 +1861,6 @@ int fetch_submodules(struct repository *r,
 	spf.command_line_option = command_line_option;
 	spf.default_option = default_option;
 	spf.quiet = quiet;
-	spf.prefix = prefix;
 
 	if (!r->worktree)
 		goto out;
diff --git a/submodule.h b/submodule.h
index 436dbb2e11..f2701c4869 100644
--- a/submodule.h
+++ b/submodule.h
@@ -55,7 +55,7 @@ struct option;
 int option_parse_recurse_submodules_worktree_updater(const struct option *opt,
 						     const char *arg, int unset);
 int option_parse_toplevel_cwd_prefix(const struct option *opt,
-				   const char *arg, int unset);
+				     const char *arg, int unset);
 /*
  * Return the relative path of the top-level process's cwd to the root of the
  * working tree. When printing paths to submodules in the working tree, this
@@ -63,6 +63,15 @@ int option_parse_toplevel_cwd_prefix(const struct option *opt,
  * top-level process's cwd instead of this process's cwd.
  */
 const char *get_toplevel_cwd_prefix(void);
+int option_parse_submodule_prefix(const struct option *opt,
+				      const char *arg, int unset);
+/*
+ * Return the path from the root of the top-level superproject to root of this
+ * repository. When printing paths to submodules in a tree, this value should be
+ * prepended to the path so that they originate from the top-level
+ * superproject's tree instead of this repository's tree.
+ */
+const char *get_submodule_prefix(void);
 
 int is_tree_submodule_active(struct repository *repo,
 			     const struct object_id *treeish_name,
@@ -100,7 +109,6 @@ const struct submodule *submodule_from_ce(const struct cache_entry *ce);
 void check_for_new_submodule_commits(struct object_id *oid);
 int fetch_submodules(struct repository *r,
 		     const struct strvec *options,
-		     const char *prefix,
 		     int command_line_option,
 		     int default_option,
 		     int quiet, int max_parallel_jobs);
-- 
2.38.1.431.g37b22c650d-goog
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 4 of 105 in “git: remove --super-prefix”
  1. 0/4 git: remove --super-prefixGlen Choo, Nov 9, 2022
  2. 1/4 submodule--helper: teach --toplevel-cwd-prefixGlen Choo, Nov 9, 2022
  3. Ævar Arnfjörð BjarmasonNov 9, 2022
  4. 2/4 fetch: refactor --submodule-prefixGlen Choo, Nov 9, 2022
  5. Ævar Arnfjörð BjarmasonNov 9, 2022
  6. 3/4 read-tree: teach --submodule-prefixGlen Choo, Nov 9, 2022
  7. Ævar Arnfjörð BjarmasonNov 9, 2022
  8. 4/4 git: remove --super-prefixGlen Choo, Nov 9, 2022
  9. 0/8 Get rid of "git --super-prefix"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  10. 1/8 submodule--helper: don't use global --super-prefix in "absorbgitdirs"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  11. Glen ChooNov 11, 2022
  12. 2/8 submodule--helper: "deinit" has never used "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  13. 3/8 submodule--helper: convert "foreach" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  14. 4/8 submodule--helper: convert "sync" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  15. 5/8 submodule--helper: convert "status" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  16. 6/8 submodule--helper: convert "{update,clone}" to their own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 9, 2022
  17. 7/8 submodule tests: test "git branch -t" output and stderrÆvar Arnfjörð Bjarmason, Nov 9, 2022
  18. 8/8 read-tree: add "--super-prefix" option, eliminate globalÆvar Arnfjörð Bjarmason, Nov 9, 2022
  19. Glen ChooNov 11, 2022
  20. Taylor BlauNov 9, 2022
  21. Ævar Arnfjörð BjarmasonNov 9, 2022
  22. Taylor BlauNov 9, 2022
  23. Ævar Arnfjörð BjarmasonNov 9, 2022
  24. Glen ChooNov 10, 2022
  25. Ævar Arnfjörð BjarmasonNov 10, 2022
  26. Glen ChooNov 11, 2022
  27. Glen ChooNov 11, 2022
  28. Ævar Arnfjörð BjarmasonNov 11, 2022
  29. Taylor BlauNov 11, 2022
  30. Glen ChooNov 12, 2022
  31. Ævar Arnfjörð BjarmasonNov 14, 2022
  32. Glen ChooNov 14, 2022
  33. Ævar Arnfjörð BjarmasonNov 15, 2022
  34. 00/10 Get rid of "git --super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  35. 01/10 read-tree + fetch tests: test failing "--super-prefix" interactionÆvar Arnfjörð Bjarmason, Nov 14, 2022
  36. Glen ChooNov 14, 2022
  37. Ævar Arnfjörð BjarmasonNov 14, 2022
  38. Glen ChooNov 14, 2022
  39. 02/10 submodule--helper: don't use global --super-prefix in "absorbgitdirs"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  40. Glen ChooNov 14, 2022
  41. Ævar Arnfjörð BjarmasonNov 17, 2022
  42. 03/10 submodule--helper: "deinit" has never used "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  43. 04/10 submodule--helper: convert "foreach" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  44. Glen ChooNov 14, 2022
  45. Ævar Arnfjörð BjarmasonNov 17, 2022
  46. 05/10 submodule--helper: convert "sync" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  47. 06/10 submodule--helper: convert "status" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  48. 07/10 submodule--helper: convert "{update,clone}" to their own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  49. Glen ChooNov 14, 2022
  50. 08/10 submodule tests: test "git branch -t" output and stderrÆvar Arnfjörð Bjarmason, Nov 14, 2022
  51. Glen ChooNov 14, 2022
  52. 10/10 fetch: rename "--submodule-prefix" to "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 14, 2022
  53. Glen ChooNov 14, 2022
  54. 09/10 read-tree: add "--super-prefix" option, eliminate globalÆvar Arnfjörð Bjarmason, Nov 14, 2022
  55. Glen ChooNov 14, 2022
  56. Taylor BlauNov 14, 2022
  57. Glen ChooNov 14, 2022
  58. Glen ChooNov 14, 2022
  59. Ævar Arnfjörð BjarmasonNov 15, 2022
  60. Glen ChooNov 16, 2022
  61. Ævar Arnfjörð BjarmasonNov 17, 2022
  62. Glen ChooNov 21, 2022
  63. 0/9 Get rid of "git --super-prefix"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  64. 1/9 read-tree + fetch tests: test failing "--super-prefix" interactionÆvar Arnfjörð Bjarmason, Nov 19, 2022
  65. 3/9 submodule--helper: don't use global --super-prefix in "absorbgitdirs"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  66. Glen ChooNov 22, 2022
  67. 2/9 submodule.c & submodule--helper: pass along "super_prefix" paramÆvar Arnfjörð Bjarmason, Nov 19, 2022
  68. 4/9 submodule--helper: convert "foreach" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  69. 6/9 submodule--helper: convert "status" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  70. 5/9 submodule--helper: convert "sync" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  71. 7/9 submodule--helper: convert "{update,clone}" to their own "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  72. 9/9 fetch: rename "--submodule-prefix" to "--super-prefix"Ævar Arnfjörð Bjarmason, Nov 19, 2022
  73. 8/9 read-tree: add "--super-prefix" option, eliminate globalÆvar Arnfjörð Bjarmason, Nov 19, 2022
  74. Glen ChooNov 22, 2022
  75. Glen ChooNov 22, 2022
  76. 0/9 Get rid of "git --super-prefix"Ævar Arnfjörð Bjarmason, Dec 15, 2022
  77. 2/9 read-tree + fetch tests: test failing "--super-prefix" interactionÆvar Arnfjörð Bjarmason, Dec 15, 2022
  78. 1/9 submodule absorbgitdirs tests: add missing "Migrating git..." testsÆvar Arnfjörð Bjarmason, Dec 15, 2022
  79. Glen ChooDec 15, 2022
  80. Ævar Arnfjörð BjarmasonDec 20, 2022
  81. 3/9 submodule.c & submodule--helper: pass along "super_prefix" paramÆvar Arnfjörð Bjarmason, Dec 15, 2022
  82. 5/9 submodule--helper: convert "foreach" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 15, 2022
  83. 4/9 submodule--helper: don't use global --super-prefix in "absorbgitdirs"Ævar Arnfjörð Bjarmason, Dec 15, 2022
  84. Glen ChooDec 15, 2022
  85. 6/9 submodule--helper: convert "sync" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 15, 2022
  86. 7/9 submodule--helper: convert "status" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 15, 2022
  87. 8/9 submodule--helper: convert "{update,clone}" to their own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 15, 2022
  88. 9/9 read-tree: add "--super-prefix" option, eliminate globalÆvar Arnfjörð Bjarmason, Dec 15, 2022
  89. Glen ChooDec 15, 2022
  90. Junio C HamanoDec 15, 2022
  91. Junio C HamanoDec 15, 2022
  92. 0/9 Get rid of "git --super-prefix"Ævar Arnfjörð Bjarmason, Dec 20, 2022
  93. 2/9 read-tree + fetch tests: test failing "--super-prefix" interactionÆvar Arnfjörð Bjarmason, Dec 20, 2022
  94. 1/9 submodule absorbgitdirs tests: add missing "Migrating git..." testsÆvar Arnfjörð Bjarmason, Dec 20, 2022
  95. 3/9 submodule.c & submodule--helper: pass along "super_prefix" paramÆvar Arnfjörð Bjarmason, Dec 20, 2022
  96. 5/9 submodule--helper: convert "foreach" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 20, 2022
  97. 4/9 submodule--helper: don't use global --super-prefix in "absorbgitdirs"Ævar Arnfjörð Bjarmason, Dec 20, 2022
  98. 6/9 submodule--helper: convert "sync" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 20, 2022
  99. 7/9 submodule--helper: convert "status" to its own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 20, 2022
  100. 8/9 submodule--helper: convert "{update,clone}" to their own "--super-prefix"Ævar Arnfjörð Bjarmason, Dec 20, 2022
  101. 9/9 read-tree: add "--super-prefix" option, eliminate globalÆvar Arnfjörð Bjarmason, Dec 20, 2022
  102. Taylor BlauNov 9, 2022
  103. Glen ChooNov 9, 2022
  104. Taylor BlauNov 10, 2022
  105. Glen ChooNov 10, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.