git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv3 11/11] clone: allow an explicit argument for parallel submodule clones

From
Stefan Beller <sbeller@google.com>
Date
Nov 4, 2015, 00:37 UTC
Message-ID
<1446597434-1740-12-git-send-email-sbeller@google.com>
In-Reply-To
<1446597434-1740-1-git-send-email-sbeller@google.com>

Just pass it along to "git submodule update", which may pick reasonable defaults if you don't specify an explicit number.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-clone.txt |  6 +++++-
 builtin/clone.c             | 19 +++++++++++++------
 t/t7406-submodule-update.sh | 15 +++++++++++++++
 3 files changed, 33 insertions(+), 7 deletions(-)
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index f1f2a3f..01bd6b7 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,7 +14,7 @@ SYNOPSIS
 	  [-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
 	  [--dissociate] [--separate-git-dir <git dir>]
 	  [--depth <depth>] [--[no-]single-branch]
-	  [--recursive | --recurse-submodules] [--] <repository>
+	  [--recursive | --recurse-submodules] [--jobs <n>] [--] <repository>
 	  [<directory>]
 
 DESCRIPTION
@@ -216,6 +216,10 @@ objects from the source repository into a pack in the cloned repository.
 	The result is Git repository can be separated from working
 	tree.
 
+-j <n>::
+--jobs <n>::
+	The number of submodules fetched at the same time.
+	Defaults to the `submodule.jobs` option.
 
 <repository>::
 	The (possibly remote) repository to clone from.  See the
diff --git a/builtin/clone.c b/builtin/clone.c
index 9eaecd9..ce578d2 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -50,6 +50,7 @@ static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
 static int option_dissociate;
+static int max_jobs = -1;
 
 static struct option builtin_clone_options[] = {
 	OPT__VERBOSITY(&option_verbosity),
@@ -72,6 +73,8 @@ static struct option builtin_clone_options[] = {
 		    N_("initialize submodules in the clone")),
 	OPT_BOOL(0, "recurse-submodules", &option_recursive,
 		    N_("initialize submodules in the clone")),
+	OPT_INTEGER('j', "jobs", &max_jobs,
+		    N_("number of submodules cloned in parallel")),
 	OPT_STRING(0, "template", &option_template, N_("template-directory"),
 		   N_("directory from which templates will be used")),
 	OPT_STRING_LIST(0, "reference", &option_reference, N_("repo"),
@@ -95,10 +98,6 @@ static struct option builtin_clone_options[] = {
 	OPT_END()
 };
 
-static const char *argv_submodule[] = {
-	"submodule", "update", "--init", "--recursive", NULL
-};
-
 static const char *get_repo_path_1(struct strbuf *path, int *is_bundle)
 {
 	static char *suffix[] = { "/.git", "", ".git/.git", ".git" };
@@ -724,8 +723,16 @@ static int checkout(void)
 	err |= run_hook_le(NULL, "post-checkout", sha1_to_hex(null_sha1),
 			   sha1_to_hex(sha1), "1", NULL);
 
-	if (!err && option_recursive)
-		err = run_command_v_opt(argv_submodule, RUN_GIT_CMD);
+	if (!err && option_recursive) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		argv_array_pushl(&args, "submodule", "update", "--init", "--recursive", NULL);
+
+		if (max_jobs != -1)
+			argv_array_pushf(&args, "--jobs=%d", max_jobs);
+
+		err = run_command_v_opt(args.argv, RUN_GIT_CMD);
+		argv_array_clear(&args);
+	}
 
 	return err;
 }
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 05ea66f..ade0524 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -786,4 +786,19 @@ test_expect_success 'submodule update can be run in parallel' '
 	 grep "9 children" trace.out
 	)
 '
+
+test_expect_success 'git clone passes the parallel jobs config on to submodules' '
+	test_when_finished "rm -rf super4" &&
+	GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 7 . super4 &&
+	grep "7 children" trace.out &&
+	rm -rf super4 &&
+	git config --global submodule.jobs 8 &&
+	GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules . super4 &&
+	grep "8 children" trace.out &&
+	rm -rf super4 &&
+	GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 9 . super4 &&
+	grep "9 children" trace.out &&
+	rm -rf super4
+'
+
 test_done

-- 
2.6.1.247.ge8f2a41.dirty
Previous: Stefan BellerNext: Junio C Hamano
Message 31 of 34 in “[PATCHv3 00/11] Expose the submodule parallelism to the user”
  1. Stefan BellerNov 4, 2015
  2. 01/11 run_processes_parallel: delimit intermixed task outputStefan Beller, Nov 4, 2015
  3. 02/11 run-command: report failure for degraded output just onceStefan Beller, Nov 4, 2015
  4. Junio C HamanoNov 4, 2015
  5. Stefan BellerNov 4, 2015
  6. Johannes SixtNov 4, 2015
  7. Junio C HamanoNov 4, 2015
  8. Jeff KingNov 4, 2015
  9. Junio C HamanoNov 5, 2015
  10. Jeff KingNov 5, 2015
  11. Junio C HamanoNov 5, 2015
  12. Stefan BellerNov 5, 2015
  13. Junio C HamanoNov 4, 2015
  14. Stefan BellerNov 4, 2015
  15. Junio C HamanoNov 4, 2015
  16. Stefan BellerNov 4, 2015
  17. 03/11 run-command: omit setting file descriptors to non blocking in WindowsStefan Beller, Nov 4, 2015
  18. 04/11 submodule-config: keep update strategy aroundStefan Beller, Nov 4, 2015
  19. 05/11 submodule-config: drop check against NULLStefan Beller, Nov 4, 2015
  20. 06/11 submodule-config: remove name_and_item_from_varStefan Beller, Nov 4, 2015
  21. 07/11 submodule-config: introduce parse_generic_submodule_configStefan Beller, Nov 4, 2015
  22. 08/11 fetching submodules: respect `submodule.jobs` config optionStefan Beller, Nov 4, 2015
  23. Jens LehmannNov 10, 2015
  24. Stefan BellerNov 10, 2015
  25. Jens LehmannNov 11, 2015
  26. Stefan BellerNov 11, 2015
  27. Jens LehmannNov 13, 2015
  28. Stefan BellerNov 13, 2015
  29. 09/11 git submodule update: have a dedicated helper for cloningStefan Beller, Nov 4, 2015
  30. 10/11 submodule update: expose parallelism to the userStefan Beller, Nov 4, 2015
  31. 11/11 clone: allow an explicit argument for parallel submodule clonesStefan Beller, Nov 4, 2015
  32. Junio C HamanoNov 4, 2015
  33. Stefan BellerNov 4, 2015
  34. Junio C HamanoNov 4, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.