git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 05/10] submodule: decouple url and submodule existence

From
Junio C Hamano <gitster@pobox.com>
Date
Mar 14, 2017, 18:42 UTC
Message-ID
<xmqqzignittf.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<20170313214341.172676-6-bmwill@google.com>
Brandon Williams <bmwill@google.com> writes:
Show 5 quoted lines
> Currently the submodule.<name>.url config option is used to determine
> if a given submodule exists and is interesting to the user.  This
> however doesn't work very well because the URL is a config option for
> the scope of a repository, whereas the existence of a submodule is an
> option scoped to the working tree.

A submodule exists if it is exists, whether the user is interested in it or not. If it should be checked out in the working tree is a different matter, but that should be a logical AND between "is it of interest?" and "is the superproject tree has a gitlink for it in its working tree?".

So I do not agree with "This however doesn't work" at all. I'd understand it if you said "This is cumbersome if we want to do this and that, which are different from what we have done traditionally" and explain what this and that are and how they are different.

> In a future with worktree support for submodules, there will be multiple
> working trees, each of which may only need a subset of the submodules
> checked out.  The URL (which is where the submodule repository can be
> obtained) should not differ between different working trees.

And this makes the motivation a bit clearer. When the user wants to have multiple worktrees for the same superproject. In such a setting, the same submodule in two worktrees typically want to have the same URL. It may be different from what the upstream suggests in the .gitmodules file, but the difference, i.e. the site specific customization of the URL, should be the same between the two worktrees. But one worktree may be and the other worktree may not be interested in that submodule, and with shared .git/config file, you cannot have submodule.<name>.url set to one value and unset at the same time.

This series does not solve the "two worktrees cannot have private parts in the configuration namespace" issue, but assuming it will be solved by some other series, it anticipates that submodule.<name>.URL would want to be shared between two worktrees most of time (note that there will be users who want two separate .URL for the same submodule while sharing the object database via worktrees mechanism, and you'll need to prepare for them, too), and another "bit" that tells if the submodule is of interest would want to be private to each worktree.

That is the motivation, the reason why you want .URL to stop serving the dual purpose of overriding upstream-suggested URL and indicating the submodule is interesting to the user.

Show 10 quoted lines
> It may also be convenient for users to more easily specify groups of
> submodules they are interested in as apposed to running "git submodule
> init <path>" on each submodule they want checked out in their working
> tree.
>
> To this end two config options are introduced, submodule.active and
> submodule.<name>.active.  The submodule.active config holds a pathspec
> that specifies which submodules should exist in the working tree.  The
> submodule.<name>.active config is a boolean flag used to indicate if
> that particular submodule should exist in the working tree.

And because two worktrees always share their .git/config, these new configuration variables are useless to help workflow with multiple worktrees with the current system, until "per-worktree configuration" is invented. But we prepare for that future in this step.

Also submodule.active that takes pathspec and not name is an oddball (use of "name" not "path" is to prepare for a submodule whose location in the superproject changes depending on the commit in the superproject), and we need to justify with an explanation. I think you envision two cases. 1. we encourage projects to adopt a convention that submodules are grouped with leading directory, so that pathspec e.g. lib/, would cover _all_ library-ish modules to allow those who are interested in library-ish modules to set ".active = lib/" just once to say any and all modules in lib/ are interesting. 2. another convention the projects can adopt, when pathspec-attribute feature is invented, is to label submodules with attribute to group them, so that a broad pathspec with attribute requirement, e.g. .:(attr:lib), can be used to say any and all modules with 'lib' attribute are interesting.

The above two points (justifications, intended uses and future plans) need to be clarified around here (and possibly in the documentation), I would think.

Show 49 quoted lines
> diff --git a/submodule.c b/submodule.c
> index 0a2831d84..2b33bd70f 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -217,13 +217,41 @@ void gitmodules_config_sha1(const unsigned char *commit_sha1)
>  int is_submodule_initialized(const char *path)
>  {
>  	int ret = 0;
> -	const struct submodule *module = NULL;
> +	char *key;
> +	const struct string_list *sl;
> +	const struct submodule *module = submodule_from_path(null_sha1, path);
>  
> -	module = submodule_from_path(null_sha1, path);
> +	/* early return if there isn't a path->module mapping */
> +	if (!module)
> +		return 0;
> +
> +	/* submodule.<name>.active is set */
> +	key = xstrfmt("submodule.%s.active", module->name);
> +	if (!git_config_get_bool(key, &ret)) {
> +		free(key);
> +		return ret;
> +	}
> +	free(key);
> +
> +	sl = git_config_get_value_multi("submodule.active");
>  
> -	if (module) {
> -		char *key = xstrfmt("submodule.%s.url", module->name);
> +	if (sl) {
> +		struct pathspec ps;
> +		struct argv_array args = ARGV_ARRAY_INIT;
> +		const struct string_list_item *item;
> +
> +		for_each_string_list_item(item, sl) {
> +			argv_array_push(&args, item->string);
> +		}
> +
> +		parse_pathspec(&ps, 0, 0, 0, args.argv);
> +		ret = match_pathspec(&ps, path, strlen(path), 0, NULL, 1);
> +
> +		argv_array_clear(&args);
> +		clear_pathspec(&ps);
> +	} else {
>  		char *value = NULL;
> +		key = xstrfmt("submodule.%s.url", module->name);
>  
>  		ret = !git_config_get_string(key, &value);

It probably is easier to read if you had a final "return ret" in the "if (sl) {...}" part, just like you have one for the codepath that deals with "submodule.<name>.active", and flatten the else clause. That would make it clear that we have three ways with decreasing precedence.

At this point, the answer from function is even less about "is it initialized?" but about "is it of interest?" (or "is it to be initialized?"). We'd probably want a /* NEEDSWORK */ comment before the function to remind us to come up with a better name after the dust settles.

Show 19 quoted lines
> diff --git a/t/t7413-submodule-is-active.sh b/t/t7413-submodule-is-active.sh
> index f18e0c925..c41b899ab 100755
> --- a/t/t7413-submodule-is-active.sh
> +++ b/t/t7413-submodule-is-active.sh
> @@ -28,4 +28,59 @@ test_expect_success 'is-active works with urls' '
>  	git -C super submodule--helper is-active sub1
>  '
>  
> +test_expect_success 'is-active works with submodule.<name>.active config' '
> +	git -C super config --bool submodule.sub1.active "false" &&
> +	test_must_fail git -C super submodule--helper is-active sub1 &&
> +
> +	git -C super config --bool submodule.sub1.active "true" &&
> +	git -C super config --unset submodule.sub1.URL &&
> +	git -C super submodule--helper is-active sub1 &&
> +
> +	git -C super config submodule.sub1.URL ../sub &&
> +	git -C super config --unset submodule.sub1.active
> +'

The last "unset" is done to clean the customization this test did, in order to give a predictable beginning state to the next test? If so, use test_when_finished instead of &&-cascading it at the end.

Show 13 quoted lines
> + ...
> +test_expect_success 'is-active with submodule.active and submodule.<name>.active' '
> +	git -C super config --add submodule.active "sub1" &&
> +	git -C super config --bool submodule.sub1.active "false" &&
> +	git -C super config --bool submodule.sub2.active "true" &&
> +
> +	test_must_fail git -C super submodule--helper is-active sub1 &&
> +	git -C super submodule--helper is-active sub2 &&
> +
> +	git -C super config --unset-all submodule.active &&
> +	git -C super config --unset submodule.sub1.active &&
> +	git -C super config --unset submodule.sub2.active
> +'
Likewise for all the new tests in this patch.
Thanks.
Previous: Brandon WilliamsNext: Junio C Hamano
Message 58 of 123 in “decoupling a submodule's existence and its url”
  1. 00/10 decoupling a submodule's existence and its urlBrandon Williams, Feb 23, 2017
  2. 06/10 submodule add: respect submodule.activeBrandon Williams, Feb 23, 2017
  3. 09/10 submodule sync: use submodule--helper is-activeBrandon Williams, Feb 23, 2017
  4. 04/10 completion: clone can initialize specific submodulesBrandon Williams, Feb 23, 2017
  5. 01/10 submodule: decouple url and submodule existenceBrandon Williams, Feb 23, 2017
  6. Junio C HamanoFeb 24, 2017
  7. Brandon WilliamsMar 1, 2017
  8. Stefan BellerMar 1, 2017
  9. Brandon WilliamsMar 6, 2017
  10. Junio C HamanoMar 2, 2017
  11. 03/10 clone: add --submodule-spec=<pathspec> switchBrandon Williams, Feb 23, 2017
  12. 07/10 submodule status: use submodule--helper is-activeBrandon Williams, Feb 23, 2017
  13. 08/10 submodule deinit: use most reliable urlBrandon Williams, Feb 23, 2017
  14. 10/10 submodule--helper clone: check for configured submodules using helperBrandon Williams, Feb 23, 2017
  15. Stefan BellerFeb 24, 2017
  16. Brandon WilliamsFeb 27, 2017
  17. 02/10 submodule update: add `--init-active` switchBrandon Williams, Feb 23, 2017
  18. 05/10 submodule--helper: add is_active commandBrandon Williams, Feb 23, 2017
  19. Stefan BellerFeb 24, 2017
  20. Brandon WilliamsFeb 27, 2017
  21. Stefan BellerFeb 23, 2017
  22. 00/11 decoupling a submodule's existence and its urlBrandon Williams, Mar 9, 2017
  23. 05/11 submodule--helper clone: check for configured submodules using helperBrandon Williams, Mar 9, 2017
  24. 02/11 submodule status: use submodule--helper is-activeBrandon Williams, Mar 9, 2017
  25. 04/11 submodule sync: use submodule--helper is-activeBrandon Williams, Mar 9, 2017
  26. 01/11 submodule--helper: add is_active commandBrandon Williams, Mar 9, 2017
  27. 11/11 submodule add: respect submodule.active and submodule.<name>.activeBrandon Williams, Mar 9, 2017
  28. 06/11 submodule: decouple url and submodule existenceBrandon Williams, Mar 9, 2017
  29. 09/11 completion: clone can initialize specific submodulesBrandon Williams, Mar 9, 2017
  30. 07/11 submodule update: add `--init-active` switchBrandon Williams, Mar 9, 2017
  31. Stefan BellerMar 9, 2017
  32. Brandon WilliamsMar 9, 2017
  33. 08/11 clone: add --submodule-spec=<pathspec> switchBrandon Williams, Mar 9, 2017
  34. 10/11 submodule--helper init: set submodule.<name>.activeBrandon Williams, Mar 9, 2017
  35. Stefan BellerMar 9, 2017
  36. Brandon WilliamsMar 9, 2017
  37. 03/11 submodule deinit: use most reliable urlBrandon Williams, Mar 9, 2017
  38. Stefan BellerMar 9, 2017
  39. Brandon WilliamsMar 9, 2017
  40. 00/10 decoupling a submodule's existence and its urlBrandon Williams, Mar 13, 2017
  41. 01/10 submodule--helper: add is_active commandBrandon Williams, Mar 13, 2017
  42. Stefan BellerMar 13, 2017
  43. Junio C HamanoMar 14, 2017
  44. Brandon WilliamsMar 14, 2017
  45. 02/10 submodule status: use submodule--helper is-activeBrandon Williams, Mar 13, 2017
  46. Junio C HamanoMar 14, 2017
  47. Stefan BellerMar 14, 2017
  48. Junio C HamanoMar 14, 2017
  49. Junio C HamanoMar 14, 2017
  50. Brandon WilliamsMar 14, 2017
  51. Brandon WilliamsMar 14, 2017
  52. 03/10 submodule sync: use submodule--helper is-activeBrandon Williams, Mar 13, 2017
  53. Junio C HamanoMar 14, 2017
  54. Brandon WilliamsMar 14, 2017
  55. 05/10 submodule: decouple url and submodule existenceBrandon Williams, Mar 13, 2017
  56. Stefan BellerMar 13, 2017
  57. Brandon WilliamsMar 15, 2017
  58. Junio C HamanoMar 14, 2017
  59. Junio C HamanoMar 14, 2017
  60. Brandon WilliamsMar 15, 2017
  61. 07/10 clone: add --submodule-spec=<pathspec> switchBrandon Williams, Mar 13, 2017
  62. Junio C HamanoMar 14, 2017
  63. Brandon WilliamsMar 15, 2017
  64. Stefan BellerMar 15, 2017
  65. 08/10 completion: clone can initialize specific submodulesBrandon Williams, Mar 13, 2017
  66. 09/10 submodule--helper init: set submodule.<name>.activeBrandon Williams, Mar 13, 2017
  67. Junio C HamanoMar 14, 2017
  68. Brandon WilliamsMar 15, 2017
  69. Junio C HamanoMar 16, 2017
  70. 04/10 submodule--helper clone: check for configured submodules using helperBrandon Williams, Mar 13, 2017
  71. Junio C HamanoMar 14, 2017
  72. Stefan BellerMar 14, 2017
  73. Junio C HamanoMar 14, 2017
  74. Brandon WilliamsMar 15, 2017
  75. 10/10 submodule add: respect submodule.active and submodule.<name>.activeBrandon Williams, Mar 13, 2017
  76. Junio C HamanoMar 14, 2017
  77. Brandon WilliamsMar 15, 2017
  78. 06/10 submodule update: add `--init-active` switchBrandon Williams, Mar 13, 2017
  79. Junio C HamanoMar 14, 2017
  80. Brandon WilliamsMar 15, 2017
  81. Junio C HamanoMar 14, 2017
  82. Brandon WilliamsMar 15, 2017
  83. Stefan BellerMar 13, 2017
  84. Junio C HamanoMar 14, 2017
  85. Brandon WilliamsMar 14, 2017
  86. 00/10 decoupling url and submodule interestBrandon Williams, Mar 16, 2017
  87. 06/10 submodule: decouple url and submodule interestBrandon Williams, Mar 16, 2017
  88. Stefan BellerMar 16, 2017
  89. 03/10 submodule sync: skip work for inactive submodulesBrandon Williams, Mar 16, 2017
  90. Stefan BellerMar 16, 2017
  91. Brandon WilliamsMar 16, 2017
  92. Stefan BellerMar 16, 2017
  93. Junio C HamanoMar 17, 2017
  94. Brandon WilliamsMar 17, 2017
  95. Junio C HamanoMar 17, 2017
  96. 05/10 submodule--helper clone: check for configured submodules using helperBrandon Williams, Mar 16, 2017
  97. 02/10 submodule status: use submodule--helper is-activeBrandon Williams, Mar 16, 2017
  98. 08/10 clone: teach --recurse-submodules to optionally take a pathspecBrandon Williams, Mar 16, 2017
  99. Stefan BellerMar 17, 2017
  100. Brandon WilliamsMar 17, 2017
  101. 04/10 submodule sync: use submodule--helper is-activeBrandon Williams, Mar 16, 2017
  102. 07/10 submodule init: initialize active submodulesBrandon Williams, Mar 16, 2017
  103. Stefan BellerMar 17, 2017
  104. Brandon WilliamsMar 17, 2017
  105. Stefan BellerMar 17, 2017
  106. 01/10 submodule--helper: add is-active subcommandBrandon Williams, Mar 16, 2017
  107. 10/10 submodule add: respect submodule.active and submodule.<name>.activeBrandon Williams, Mar 16, 2017
  108. 09/10 submodule--helper init: set submodule.<name>.activeBrandon Williams, Mar 16, 2017
  109. Stefan BellerMar 17, 2017
  110. Brandon WilliamsMar 17, 2017
  111. 00/10 decoupling url and submodule interestBrandon Williams, Mar 17, 2017
  112. 06/10 submodule: decouple url and submodule interestBrandon Williams, Mar 17, 2017
  113. 09/10 submodule--helper init: set submodule.<name>.activeBrandon Williams, Mar 17, 2017
  114. 05/10 submodule--helper clone: check for configured submodules using helperBrandon Williams, Mar 17, 2017
  115. 03/10 submodule sync: skip work for inactive submodulesBrandon Williams, Mar 17, 2017
  116. 02/10 submodule status: use submodule--helper is-activeBrandon Williams, Mar 17, 2017
  117. Stefan BellerMar 17, 2017
  118. 08/10 clone: teach --recurse-submodules to optionally take a pathspecBrandon Williams, Mar 17, 2017
  119. 01/10 submodule--helper: add is-active subcommandBrandon Williams, Mar 17, 2017
  120. 07/10 submodule init: initialize active submodulesBrandon Williams, Mar 17, 2017
  121. 04/10 submodule sync: use submodule--helper is-activeBrandon Williams, Mar 17, 2017
  122. 10/10 submodule add: respect submodule.active and submodule.<name>.activeBrandon Williams, Mar 17, 2017
  123. Junio C HamanoMar 18, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.