git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/8] fetching submodules: respect `submodule.fetchJobs` config option

From
Junio C Hamano <gitster@pobox.com>
Date
Feb 3, 2016, 23:09 UTC
Message-ID
<xmqq60y5l7bq.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<1454435497-26429-6-git-send-email-sbeller@google.com>
Stefan Beller <sbeller@google.com> writes:
Show 32 quoted lines
> This allows to configure fetching and updating in parallel
> without having the command line option.
>
> This moved the responsibility to determine how many parallel processes
> to start from builtin/fetch to submodule.c as we need a way to communicate
> "The user did not specify the number of parallel processes in the command
> line options" in the builtin fetch. The submodule code takes care of
> the precedence (CLI > config > default)
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/config.txt    |  7 +++++++
>  builtin/fetch.c             |  2 +-
>  submodule-config.c          | 15 +++++++++++++++
>  submodule-config.h          |  2 ++
>  submodule.c                 |  5 +++++
>  t/t5526-fetch-submodules.sh | 14 ++++++++++++++
>  6 files changed, 44 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/config.txt b/Documentation/config.txt
> index 2d06b11..eda3276 100644
> --- a/Documentation/config.txt
> +++ b/Documentation/config.txt
> @@ -2646,6 +2646,13 @@ submodule.<name>.ignore::
>  	"--ignore-submodules" option. The 'git submodule' commands are not
>  	affected by this setting.
>  
> +submodule.fetchJobs::
> +	This is used to determine how many submodules will be
> +	fetched/cloned at the same time. Specifying a positive integer
> +	allows up to that number of submodules being fetched in parallel.

s/being //? Either would be fine, but shorter might be better? I dunno.

s/is used to determine/specifies/? That may shorten the whole thing.

	Specifies how many submodules are fetched/cloned at the same
	time.  A positive integer allows up to that number of ...
> +	This is used in fetch and clone operations only. A value of 0 will
> +	give some reasonable configuration. It defaults to 1.

s/used/respected/, perhaps? s/reasonable configuration/reasonable default/, perhaps?

Show 31 quoted lines
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index 586840d..5aa1c2d 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -37,7 +37,7 @@ static int prune = -1; /* unspecified */
>  static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity;
>  static int progress = -1, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
>  static int tags = TAGS_DEFAULT, unshallow, update_shallow;
> -static int max_children = 1;
> +static int max_children = -1;
>  static const char *depth;
>  static const char *upload_pack;
>  static struct strbuf default_rla = STRBUF_INIT;
> diff --git a/submodule-config.c b/submodule-config.c
> index 29e21b2..a32259e 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -32,6 +32,7 @@ enum lookup_type {
>  
>  static struct submodule_cache cache;
>  static int is_cache_init;
> +static int parallel_jobs = -1;
>  
>  static int config_path_cmp(const struct submodule_entry *a,
>  			   const struct submodule_entry *b,
> @@ -239,6 +240,15 @@ static int parse_generic_submodule_config(const char *key,
>  					  const char *value,
>  					  struct parse_config_parameter *me)
>  {
> +	if (!strcmp(key, "fetchjobs")) {
> +		parallel_jobs = strtol(value, NULL, 10);

We do not notice trailing 'x' in "[submodule] fetchjobs = 10x"; intended? I am wondering this is a good excuse to invent git_parse_long() and git_config_long() that do not exist yet. In general, I'd personally prefer to reduce the number of calls to strto[u]l that does not check trailing garbage, not increase it.

If we reject negatives anyway, we might be able to just use git_parse_ulong() without adding new "signed" variants.

Previous: Stefan BellerNext: Stefan Beller
Message 14 of 22 in “[PATCHv7 0/8] Expose submodule parallelism to the user”
  1. Stefan BellerFeb 2, 2016
  2. 1/8 submodule-config: keep update strategy aroundStefan Beller, Feb 2, 2016
  3. Junio C HamanoFeb 3, 2016
  4. Jeff KingFeb 4, 2016
  5. 2/8 submodule-config: drop check against NULLStefan Beller, Feb 2, 2016
  6. Junio C HamanoFeb 3, 2016
  7. 3/8 submodule-config: remove name_and_item_from_varStefan Beller, Feb 2, 2016
  8. Junio C HamanoFeb 3, 2016
  9. 4/8 submodule-config: introduce parse_generic_submodule_configStefan Beller, Feb 2, 2016
  10. Junio C HamanoFeb 3, 2016
  11. Stefan BellerFeb 3, 2016
  12. Junio C HamanoFeb 4, 2016
  13. 5/8 fetching submodules: respect `submodule.fetchJobs` config optionStefan Beller, Feb 2, 2016
  14. Junio C HamanoFeb 3, 2016
  15. 6/8 git submodule update: have a dedicated helper for cloningStefan Beller, Feb 2, 2016
  16. Junio C HamanoFeb 3, 2016
  17. Stefan BellerFeb 3, 2016
  18. Junio C HamanoFeb 4, 2016
  19. Stefan BellerFeb 4, 2016
  20. Junio C HamanoFeb 4, 2016
  21. 7/8 submodule update: expose parallelism to the userStefan Beller, Feb 2, 2016
  22. 8/8 clone: allow an explicit argument for parallel submodule clonesStefan Beller, Feb 2, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.