git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFCv8 PATCH 00/17] Checkout aware of Submodules!

From
Stefan Beller <sbeller@google.com>
Date
Mar 9, 2017, 22:15 UTC
Message-ID
<20170309221543.15897-1-sbeller@google.com>
In-Reply-To
<20170306205919.9713-1-sbeller@google.com>

previous work: https://public-inbox.org/git/20170306205919.9713-1-sbeller@google.com/

v8:
 * Thanks Ramsay; another variable is static now!
 * Thanks Junio; fixed typos (in comments and commit messages)!
 * dropped the patch touching read-cache.c; I realized it is bogus after
   inspecting it and trying to write a commit message.
 * All commits have commit messages now.
 * diff to Junios version below.
 
v7:
 * addressed Erics comment by fixing the bashism in t/lib-submodule-update.sh
diff to v6:
  diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
  index 5caae06bc5..949ebd546c 100755
  --- a/t/lib-submodule-update.sh
  +++ b/t/lib-submodule-update.sh
  @@ -255,7 +255,7 @@ test_superproject_content () {
   # Test that the given submodule at path "$1" contains the content according
   # to the submodule commit recorded in the superproject's commit "$2"
   test_submodule_content () {
  -       if test "$1" == "-C"
  +       if test x"$1" = "x-C"
          then
                  cd "$2"
                  shift; shift;
 * interdiff to v5 below
                  
v6:
 * added support for read-tree (see last patch) to see how generic the
   code of the previous patches is. I am pretty pleased with that patch.
 * marked two functions static. Thanks Ramsay!
 * fixed the recursive test; it still fails but it is the code that fails,
   not the test. For this I had to change the setup code slightly.
 * 2 new patches adding tiny refactoring to the submodule test lib.  
 * interdiff (to origin/sb/checkout-recurse-submodules, which is v5) below.
v5:
 * as v4 was the first version queued by Junio, we do have an interdiff below!
 * renamed functions
 * changed the API, now the caller has to take care of the submodule strategy
   themselves. (Note this can be different for different situations, e.g.
   when a submodule is deleted, we can do that for any strategy except none and
   !command. But for moving to a new state of the submodule we currently
   only implement "checkout" [unspecified defaults to checkout]. warning about
   the others, doing nothing there.)
v4:
 * addressed all comments of Brian, Junio and Brandon.
 Thanks!
 * one major point of change is the introduction of another patch
   "lib-submodule-update.sh: do not use ./. as submodule remote",
   as that took some time to track down the existing bug.
 
v3:
 * moved tests from t2013 to the generic submodule library.
 * factored out the refactoring patches to be up front
 * As I redid the complete implementation, I have the impression this time
   it is cleaner than previous versions.
 
 I think we still have to fix the corner cases of directory/file/submodule 
 conflicts before merging, but this serves as a status update on my current
 way of thinking how to implement the worktree commands being aware of
 submodules.
 
Thanks,
Stefan
v2:
* based on top of the series sent out an hour ago
  "[PATCHv4 0/5] submodule embedgitdirs"
* Try to embed a submodule if we need to remove it.
* Strictly do not change behavior if not giving the new flag.
* I think I missed some review comments from v1, but I'd like to get
  the current state out over the weekend, as a lot has changed so far.
  On Monday I'll go through the previous discussion with a comb to see
  if I missed something.
  
v1:
When working with submodules, nearly anytime after checking out
a different state of the projects, that has submodules changed
you'd run "git submodule update" with a current version of Git.
There are two problems with this approach:
* The "submodule update" command is dangerous as it
  doesn't check for work that may be lost in the submodule
  (e.g. a dangling commit).
* you may forget to run the command as checkout is supposed
  to do all the work for you.

Integrate updating the submodules into git checkout, with the same safety promises that git-checkout has, i.e. not throw away data unless asked to. This is done by first checking if the submodule is at the same sha1 as it is recorded in the superproject. If there are changes we stop proceeding the checkout just like it is when checking out a file that has local changes.

The integration happens in the code that is also used in other commands such that it will be easier in the future to make other commands aware of submodule.

This also solves d/f conflicts in case you replace a file/directory with a submodule or vice versa.

The patches are still a bit rough, but the overall series seems promising enough to me that I want to put it out here.

Any review, specifically on the design level welcome!
Thanks,
Stefan
  
diff --git a/builtin/read-tree.c b/builtin/read-tree.c
index 2dc5cc06da..23e212ee8c 100644
--- a/builtin/read-tree.c
+++ b/builtin/read-tree.c
@@ -21,7 +21,7 @@
 static int nr_trees;
 static int read_empty;
 static struct tree *trees[MAX_UNPACK_TREES];
-int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
 
 static int list_tree(unsigned char *sha1)
 {
diff --git a/read-cache.c b/read-cache.c
index 9a2abacf7a..9054369dd0 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -18,8 +18,6 @@
 #include "varint.h"
 #include "split-index.h"
 #include "utf8.h"
-#include "submodule.h"
-#include "submodule-config.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
@@ -522,22 +520,6 @@ int remove_index_entry_at(struct index_state *istate, int pos)
 	return 1;
 }
 
-static void remove_submodule_according_to_strategy(const struct submodule *sub)
-{
-	switch (sub->update_strategy.type) {
-	case SM_UPDATE_UNSPECIFIED:
-	case SM_UPDATE_CHECKOUT:
-	case SM_UPDATE_REBASE:
-	case SM_UPDATE_MERGE:
-		submodule_move_head(sub->path, "HEAD", NULL, \
-				    SUBMODULE_MOVE_HEAD_FORCE);
-		break;
-	case SM_UPDATE_NONE:
-	case SM_UPDATE_COMMAND:
-		; /* Do not touch the submodule. */
-	}
-}
-
 /*
  * Remove all cache entries marked for removal, that is where
  * CE_REMOVE is set in ce_flags.  This is much more effective than
@@ -550,13 +532,8 @@ void remove_marked_cache_entries(struct index_state *istate)
 
 	for (i = j = 0; i < istate->cache_nr; i++) {
 		if (ce_array[i]->ce_flags & CE_REMOVE) {
-			const struct submodule *sub = submodule_from_ce(ce_array[i]);
-			if (sub) {
-				remove_submodule_according_to_strategy(sub);
-			} else {
-				remove_name_hash(istate, ce_array[i]);
-				save_or_free_index_entry(istate, ce_array[i]);
-			}
+			remove_name_hash(istate, ce_array[i]);
+			save_or_free_index_entry(istate, ce_array[i]);
 		}
 		else
 			ce_array[j++] = ce_array[i];
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 949ebd546c..e195b590a1 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -954,7 +954,7 @@ test_submodule_switch_recursing () {
 		)
 	'
 
-	# ... and ignored files are ignroed
+	# ... and ignored files are ignored
 	test_expect_success "$command: replace submodule with a file works ignores ignored files in submodule" '
 		test_when_finished "rm submodule_update/.git/modules/sub1/info/exclude" &&
 		prolog &&
Previous: Stefan BellerNext: Stefan Beller
Message 54 of 100 in “Checkout aware of Submodules!”
  1. 00/18 Checkout aware of Submodules!Stefan Beller, Mar 2, 2017
  2. 09/18 update submodules: add a config option to determine if submodules are updatedStefan Beller, Mar 2, 2017
  3. 02/18 lib-submodule-update.sh: do not use ./. as submodule remoteStefan Beller, Mar 2, 2017
  4. 10/18 submodules: introduce check to see whether to touch a submoduleStefan Beller, Mar 2, 2017
  5. 12/18 update submodules: add submodule_move_headStefan Beller, Mar 2, 2017
  6. 08/18 update submodules: add submodule config parsingStefan Beller, Mar 2, 2017
  7. 04/18 lib-submodule-update.sh: replace sha1 by hashStefan Beller, Mar 2, 2017
  8. 16/18 entry.c: update submodules when interestingStefan Beller, Mar 2, 2017
  9. 06/18 make is_submodule_populated gentlyStefan Beller, Mar 2, 2017
  10. 07/18 connect_work_tree_and_git_dir: safely create leading directoriesStefan Beller, Mar 2, 2017
  11. 17/18 builtin/checkout: add --recurse-submodules switchStefan Beller, Mar 2, 2017
  12. 18/18 builtin/read-tree: add --recurse-submodules switchStefan Beller, Mar 2, 2017
  13. 13/18 unpack-trees: pass old oid to verify_clean_submoduleStefan Beller, Mar 2, 2017
  14. 15/18 read-cache, remove_marked_cache_entries: wipe selected submodules.Stefan Beller, Mar 2, 2017
  15. 11/18 update submodules: move up prepare_submodule_repo_envStefan Beller, Mar 2, 2017
  16. 03/18 lib-submodule-update: teach test_submodule_content the -C <dir> flagStefan Beller, Mar 2, 2017
  17. Eric WongMar 2, 2017
  18. Stefan BellerMar 6, 2017
  19. Stefan BellerMar 6, 2017
  20. 05/18 lib-submodule-update.sh: define tests for recursing into submodulesStefan Beller, Mar 2, 2017
  21. 14/18 unpack-trees: check if we can perform the operation for submodulesStefan Beller, Mar 2, 2017
  22. 01/18 lib-submodule-update.sh: reorder create_lib_submodule_repoStefan Beller, Mar 2, 2017
  23. 00/18 Checkout aware of Submodules!Stefan Beller, Mar 6, 2017
  24. 03/18 lib-submodule-update: teach test_submodule_content the -C <dir> flagStefan Beller, Mar 6, 2017
  25. Junio C HamanoMar 6, 2017
  26. 10/18 submodules: introduce check to see whether to touch a submoduleStefan Beller, Mar 6, 2017
  27. 07/18 connect_work_tree_and_git_dir: safely create leading directoriesStefan Beller, Mar 6, 2017
  28. 02/18 lib-submodule-update.sh: do not use ./. as submodule remoteStefan Beller, Mar 6, 2017
  29. 17/18 builtin/checkout: add --recurse-submodules switchStefan Beller, Mar 6, 2017
  30. 01/18 lib-submodule-update.sh: reorder create_lib_submodule_repoStefan Beller, Mar 6, 2017
  31. 13/18 unpack-trees: pass old oid to verify_clean_submoduleStefan Beller, Mar 6, 2017
  32. 15/18 read-cache, remove_marked_cache_entries: wipe selected submodules.Stefan Beller, Mar 6, 2017
  33. Junio C HamanoMar 7, 2017
  34. Stefan BellerMar 7, 2017
  35. Junio C HamanoMar 8, 2017
  36. Stefan BellerMar 8, 2017
  37. Junio C HamanoMar 8, 2017
  38. 12/18 update submodules: add submodule_move_headStefan Beller, Mar 6, 2017
  39. 16/18 entry.c: update submodules when interestingStefan Beller, Mar 6, 2017
  40. Junio C HamanoMar 7, 2017
  41. Junio C HamanoMar 7, 2017
  42. Stefan BellerMar 7, 2017
  43. Junio C HamanoMar 8, 2017
  44. 11/18 update submodules: move up prepare_submodule_repo_envStefan Beller, Mar 6, 2017
  45. 04/18 lib-submodule-update.sh: replace sha1 by hashStefan Beller, Mar 6, 2017
  46. 14/18 unpack-trees: check if we can perform the operation for submodulesStefan Beller, Mar 6, 2017
  47. 05/18 lib-submodule-update.sh: define tests for recursing into submodulesStefan Beller, Mar 6, 2017
  48. Junio C HamanoMar 7, 2017
  49. 18/18 builtin/read-tree: add --recurse-submodules switchStefan Beller, Mar 6, 2017
  50. Junio C HamanoMar 7, 2017
  51. 08/18 update submodules: add submodule config parsingStefan Beller, Mar 6, 2017
  52. 06/18 make is_submodule_populated gentlyStefan Beller, Mar 6, 2017
  53. 09/18 update submodules: add a config option to determine if submodules are updatedStefan Beller, Mar 6, 2017
  54. 00/17 Checkout aware of Submodules!Stefan Beller, Mar 9, 2017
  55. 01/17 lib-submodule-update.sh: reorder create_lib_submodule_repoStefan Beller, Mar 9, 2017
  56. 08/17 update submodules: add submodule config parsingStefan Beller, Mar 9, 2017
  57. 03/17 lib-submodule-update: teach test_submodule_content the -C <dir> flagStefan Beller, Mar 9, 2017
  58. 10/17 submodules: introduce check to see whether to touch a submoduleStefan Beller, Mar 9, 2017
  59. 12/17 update submodules: add submodule_move_headStefan Beller, Mar 9, 2017
  60. Brandon WilliamsMar 9, 2017
  61. Junio C HamanoMar 12, 2017
  62. Brandon WilliamsMar 13, 2017
  63. Stefan BellerMar 13, 2017
  64. Brandon WilliamsMar 9, 2017
  65. Brandon WilliamsMar 9, 2017
  66. 07/17 connect_work_tree_and_git_dir: safely create leading directoriesStefan Beller, Mar 9, 2017
  67. Brandon WilliamsMar 9, 2017
  68. Junio C HamanoMar 12, 2017
  69. 05/17 lib-submodule-update.sh: define tests for recursing into submodulesStefan Beller, Mar 9, 2017
  70. 16/17 builtin/checkout: add --recurse-submodules switchStefan Beller, Mar 9, 2017
  71. 17/17 builtin/read-tree: add --recurse-submodules switchStefan Beller, Mar 9, 2017
  72. 15/17 entry.c: create submodules when interestingStefan Beller, Mar 9, 2017
  73. 09/17 update submodules: add a config option to determine if submodules are updatedStefan Beller, Mar 9, 2017
  74. 11/17 update submodules: move up prepare_submodule_repo_envStefan Beller, Mar 9, 2017
  75. 13/17 unpack-trees: pass old oid to verify_clean_submoduleStefan Beller, Mar 9, 2017
  76. 04/17 lib-submodule-update.sh: replace sha1 by hashStefan Beller, Mar 9, 2017
  77. 14/17 unpack-trees: check if we can perform the operation for submodulesStefan Beller, Mar 9, 2017
  78. 02/17 lib-submodule-update.sh: do not use ./. as submodule remoteStefan Beller, Mar 9, 2017
  79. 06/17 make is_submodule_populated gentlyStefan Beller, Mar 9, 2017
  80. 00/19 Checkout aware of Submodules!Stefan Beller, Mar 14, 2017
  81. 01/19 connect_work_tree_and_git_dir: safely create leading directoriesStefan Beller, Mar 14, 2017
  82. 02/19 submodule--helper.c: remove duplicate codeStefan Beller, Mar 14, 2017
  83. 04/19 lib-submodule-update.sh: do not use ./. as submodule remoteStefan Beller, Mar 14, 2017
  84. 07/19 lib-submodule-update.sh: define tests for recursing into submodulesStefan Beller, Mar 14, 2017
  85. 05/19 lib-submodule-update: teach test_submodule_content the -C <dir> flagStefan Beller, Mar 14, 2017
  86. 11/19 submodules: introduce check to see whether to touch a submoduleStefan Beller, Mar 14, 2017
  87. 17/19 entry.c: create submodules when interestingStefan Beller, Mar 14, 2017
  88. 14/19 update submodules: add submodule_move_headStefan Beller, Mar 14, 2017
  89. 18/19 builtin/checkout: add --recurse-submodules switchStefan Beller, Mar 14, 2017
  90. 16/19 unpack-trees: check if we can perform the operation for submodulesStefan Beller, Mar 14, 2017
  91. 09/19 update submodules: add submodule config parsingStefan Beller, Mar 14, 2017
  92. 19/19 builtin/read-tree: add --recurse-submodules switchStefan Beller, Mar 14, 2017
  93. 15/19 unpack-trees: pass old oid to verify_clean_submoduleStefan Beller, Mar 14, 2017
  94. 06/19 lib-submodule-update.sh: replace sha1 by hashStefan Beller, Mar 14, 2017
  95. 08/19 make is_submodule_populated gentlyStefan Beller, Mar 14, 2017
  96. 12/19 update submodules: move up prepare_submodule_repo_envStefan Beller, Mar 14, 2017
  97. 13/19 submodule.c: get_super_prefix_or_emptyStefan Beller, Mar 14, 2017
  98. 03/19 lib-submodule-update.sh: reorder create_lib_submodule_repoStefan Beller, Mar 14, 2017
  99. 10/19 update submodules: add a config option to determine if submodules are updatedStefan Beller, Mar 14, 2017
  100. Brandon WilliamsMar 15, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.