{"thread":{"id":"45144","subject":"[RFCv3 PATCH 00/14] Checkout aware of Submodules!","startedAt":"2017-02-15T00:35:11Z","lastAt":"2017-02-24T19:21:23Z","messageCount":94,"participants":["Stefan Beller","brian m. carlson","Brandon Williams","Junio C Hamano","Jacob Keller","Ramsay Jones"],"isPatch":true,"patchVersion":1,"patchTotal":14},"messages":[{"id":"311610","messageId":"20170215003423.20245-1-sbeller@google.com","threadId":"45144","inReplyTo":null,"subject":"[RFCv3 PATCH 00/14] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:09Z","receivedAt":"2017-02-15T00:35:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20161203003022.29797-1-sbeller@google.com/\n\nv3:\n * moved tests from t2013 to the generic submodule library.\n * factored out the refactoring patches to be up front\n * As I redid the complete implementation, I have the impression this time\n   it is cleaner than previous versions.\n \n I think we still have to fix the corner cases of directory/file/submodule \n conflicts before merging, but this serves as a status update on my current\n way of thinking how to implement the worktree commands being aware of\n submodules.\n \nThanks,\nStefan\n\nv2:\n* based on top of the series sent out an hour ago\n  \"[PATCHv4 0/5] submodule embedgitdirs\"\n* Try to embed a submodule if we need to remove it.\n* Strictly do not change behavior if not giving the new flag.\n* I think I missed some review comments from v1, but I'd like to get\n  the current state out over the weekend, as a lot has changed so far.\n  On Monday I'll go through the previous discussion with a comb to see\n  if I missed something.\n  \nv1:\nWhen working with submodules, nearly anytime after checking out\na different state of the projects, that has submodules changed\nyou'd run \"git submodule update\" with a current version of Git.\n\nThere are two problems with this approach:\n\n* The \"submodule update\" command is dangerous as it\n  doesn't check for work that may be lost in the submodule\n  (e.g. a dangling commit).\n* you may forget to run the command as checkout is supposed\n  to do all the work for you.\n\nIntegrate updating the submodules into git checkout, with the same\nsafety promises that git-checkout has, i.e. not throw away data unless\nasked to. This is done by first checking if the submodule is at the same\nsha1 as it is recorded in the superproject. If there are changes we stop\nproceeding the checkout just like it is when checking out a file that\nhas local changes.\n\nThe integration happens in the code that is also used in other commands\nsuch that it will be easier in the future to make other commands aware\nof submodule.\n\nThis also solves d/f conflicts in case you replace a file/directory\nwith a submodule or vice versa.\n\nThe patches are still a bit rough, but the overall series seems\npromising enough to me that I want to put it out here.\n\nAny review, specifically on the design level welcome!\n\nThanks,\nStefan\n\n\nStefan Beller (14):\n  lib-submodule-update.sh: reorder create_lib_submodule_repo\n  lib-submodule-update.sh: define tests for recursing into submodules\n  make is_submodule_populated gently\n  connect_work_tree_and_git_dir: safely create leading directories\n  update submodules: add submodule config parsing\n  update submodules: add a config option to determine if submodules are\n    updated\n  update submodules: introduce is_interesting_submodule\n  update submodules: move up prepare_submodule_repo_env\n  update submodules: add submodule_go_from_to\n  unpack-trees: pass old oid to verify_clean_submodule\n  unpack-trees: check if we can perform the operation for submodules\n  read-cache: remove_marked_cache_entries to wipe selected submodules.\n  entry.c: update submodules when interesting\n  builtin/checkout: add --recurse-submodules switch\n\n Documentation/git-checkout.txt |   7 +\n builtin/checkout.c             |  28 +++\n builtin/grep.c                 |   2 +-\n dir.c                          |   2 +\n entry.c                        |  28 +++\n read-cache.c                   |   3 +\n submodule-config.c             |  22 ++\n submodule-config.h             |  17 +-\n submodule.c                    | 216 +++++++++++++++--\n submodule.h                    |  16 +-\n t/lib-submodule-update.sh      | 534 +++++++++++++++++++++++++++++++++++++++--\n t/t2013-checkout-submodule.sh  |   5 +\n unpack-trees.c                 | 115 +++++++--\n unpack-trees.h                 |   1 +\n 14 files changed, 936 insertions(+), 60 deletions(-)\n\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311611","messageId":"20170215003423.20245-2-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 01/14] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:10Z","receivedAt":"2017-02-15T00:35:15Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand.  The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..61c54f2098 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n #   replace_file_with_sub1)\n #\n-#                   --O-----O\n-#                  /  ^     replace_directory_with_sub1\n-#                 /   replace_sub1_with_directory\n-#                /----O\n-#               /     ^\n-#              /      modify_sub1\n-#      O------O-------O\n-#      ^      ^\\      ^\n-#      |      | \\     remove_sub1\n-#      |      |  -----O-----O\n-#      |      |   \\   ^     replace_file_with_sub1\n-#      |      |    \\  replace_sub1_with_file\n-#      |   add_sub1 --O-----O\n-# no_submodule        ^     valid_sub1\n-#                     invalid_sub1\n+#                     ----O\n+#                    /    ^\n+#                   /     remove_sub1\n+#                  /\n+#       add_sub1  /-------O\n+#             |  /        ^\n+#             | /         modify_sub1\n+#             v/\n+#      O------O-----------O---------O\n+#      ^       \\          ^         replace_directory_with_sub1\n+#      |        \\         replace_sub1_with_directory\n+# no_submodule   \\\n+#                 --------O---------O\n+#                  \\      ^         replace_file_with_sub1\n+#                   \\     replace_sub1_with_file\n+#                    \\\n+#                     ----O---------O\n+#                         ^         valid_sub1\n+#                         invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1&&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311612","messageId":"20170215003423.20245-3-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 02/14] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:11Z","receivedAt":"2017-02-15T00:35:17Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to properly recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 474 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 472 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 61c54f2098..7c8c557572 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n #                    /    ^\n #                   /     remove_sub1\n #                  /\n-#       add_sub1  /-------O\n-#             |  /        ^\n+#       add_sub1  /-------O---------O\n+#             |  /        ^         modify_sub1_recursive\n #             | /         modify_sub1\n #             v/\n #      O------O-----------O---------O\n@@ -73,6 +74,14 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b modify_sub1_recursively modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ./. sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -139,6 +148,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -169,6 +187,18 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# also have it available:\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\tfi\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -684,3 +714,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+#  in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+#   git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 > .git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignroed\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively\n+\t\t\ttest_submodule_content sub1/sub2\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: > sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: > sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311613","messageId":"20170215003423.20245-8-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 07/14] update submodules: introduce is_interesting_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:16Z","receivedAt":"2017-02-15T00:35:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that pre checks if a submodule needs\nto be checked for an update.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 26 ++++++++++++++++++++++++++\n submodule.h |  8 ++++++++\n 2 files changed, 34 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex c0060c29f2..4c33374ae8 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -551,6 +551,32 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int submodules_interesting_for_update(void)\n+{\n+\t/*\n+\t * Update can't be \"none\", \"merge\" or \"rebase\",\n+\t * treat any value as OFF, except an explicit ON.\n+\t */\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+int is_interesting_submodule(const struct cache_entry *ce)\n+{\n+\tconst struct submodule *sub;\n+\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn 0;\n+\n+\tif (!submodules_interesting_for_update())\n+\t\treturn 0;\n+\n+\tsub = submodule_from_path(null_sha1, ce->name);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\treturn sub->update_strategy.type != SM_UPDATE_NONE;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t      int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex c4e1ac828e..84b67a7c4a 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -59,6 +59,14 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+\n+/*\n+ * Traditionally Git ignored changes made for submodules.\n+ * This function checks if we are interested in the given submodule\n+ * for any kind of operation.\n+ */\n+extern int submodules_interesting_for_update(void);\n+extern int is_interesting_submodule(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311614","messageId":"20170215003423.20245-7-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 06/14] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:15Z","receivedAt":"2017-02-15T00:35:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 9bbdd3ce7c..c0060c29f2 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -545,6 +546,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t      int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 689033e538..c4e1ac828e 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -58,6 +58,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311615","messageId":"20170215003423.20245-6-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 05/14] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:14Z","receivedAt":"2017-02-15T00:35:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule-config.c | 22 ++++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 31 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..93f01c4378 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,28 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (!strcmp(arg, \"checkout\"))\n+\t\t\treturn RECURSE_SUBMODULES_ON;\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t       int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t      unsigned char *gitmodules_sha1,\n \t\t\t\t      struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311616","messageId":"20170215003423.20245-12-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 11/14] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:20Z","receivedAt":"2017-02-15T00:35:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n unpack-trees.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++------\n unpack-trees.h |  1 +\n 2 files changed, 90 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..6805cb9549 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,7 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n \n /*\n  * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +46,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE */\n+\t\"Submodule '%s' cannot be deleted as it contains untracked files.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +165,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot be deleted as it contains untracked files.\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +246,44 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int submodule_check_from_to(const struct cache_entry *ce, const char *old_id, const char *new_id, struct unpack_trees_options *o)\n+{\n+\tif (submodule_go_from_to(ce->name, old_id,\n+\t\t\t\t new_id, 1, o->reset))\n+\t\treturn o->gently ? -1 :\n+\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE, ce->name);\n+\treturn 0;\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t   struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+\tgitmodules_config();\n+\tgit_config(submodule_config, NULL);\n+}\n+\n /*\n  * Unlink the last component and schedule the leading directories for\n  * removal, such that empty directories get removed.\n  */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tif (is_interesting_submodule(ce))\n+\t\tsubmodule_go_from_to(ce->name, \"HEAD\", NULL, 0, 1);\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +339,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (submodules_interesting_for_update() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1399,27 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (is_interesting_submodule(ce)) {\n+\t\t\tint r;\n+\t\t\tr = submodule_check_from_to(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index.  This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1463,12 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!is_interesting_submodule(ce))\n+\t\treturn 0;\n+\n+\treturn submodule_check_from_to(ce,\n+\t\t\t\t       old_sha1,\n+\t\t\t\t       oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1634,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (is_interesting_submodule(ce))\n+\t\t\t\tret = submodule_check_from_to(ce,\n+\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1650,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (is_interesting_submodule(ce))\n+\t\t\treturn submodule_check_from_to(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t       NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t  ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t  error_type, o);\n@@ -1643,6 +1709,16 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (is_interesting_submodule(ce)) {\n+\t\t\tint ret = submodule_check_from_to(ce,\n+\t\t\t\t\t\t\t  oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t  oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t  o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1739,10 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\t\tif (is_interesting_submodule(ce)) {\n+\t\t\tif (submodule_check_from_to(ce, oid_to_hex(&old->oid), oid_to_hex(&ce->oid), o))\n+\t\t\t\treturn -1;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..c0427ce082 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_UNTRACKED_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311617","messageId":"20170215003423.20245-11-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 10/14] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:19Z","receivedAt":"2017-02-15T00:35:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n  * Currently, git does not checkout subprojects during a superproject\n  * checkout, so it is not going to overwrite anything.\n  */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t  const struct cache_entry *ce,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t    resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t      ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311618","messageId":"20170215003423.20245-10-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 09/14] update submodules: add submodule_go_from_to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:18Z","receivedAt":"2017-02-15T00:35:35Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n  \"Is it safe to change the submodule to this new state?\"\n  e.g. is it overwriting untracked files or are there local\n  changes that would be overwritten?\n* supports a force flag that can be used for resetting\n  the tree.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h |   5 ++\n 2 files changed, 156 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex d3fc6c2a75..194cba9535 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1252,6 +1252,157 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tssize_t len;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tint ret = 0;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.out = -1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule %s\", sub->path);\n+\n+\tlen = strbuf_read(&buf, cp.out, 1024);\n+\tif (len > 2)\n+\t\tret = 1;\n+\n+\tclose(cp.out);\n+\tif (finish_command(&cp))\n+\t\tdie(\"could not recurse into submodule %s\", sub->path);\n+\n+\tstrbuf_release(&buf);\n+\treturn ret;\n+}\n+\n+void submodule_clean_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not clean submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ *\n+ * TODO: move dryrun and forced to flags.\n+ */\n+int submodule_go_from_to(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t int dry_run,\n+\t\t\t int force)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (!dry_run) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t    get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_clean_index(path);\n+\t\t}\n+\t}\n+\n+\tif (old && !force) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub)) {\n+\t\t\t/* print a thing here? */\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (!dry_run)\n+\t\targv_array_push(&cp.args, \"-u\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-n\");\n+\n+\tif (force)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!dry_run) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 84b67a7c4a..570953a351 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -91,6 +91,11 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+extern int submodule_go_from_to(const char *path,\n+\t\t\t\tconst char *old,\n+\t\t\t\tconst char *new,\n+\t\t\t\tint dry_run, int force);\n+\n /*\n  * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n  * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311619","messageId":"20170215003423.20245-5-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 04/14] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:13Z","receivedAt":"2017-02-15T00:35:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. Safely create\nthe directory first.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n dir.c | 2 ++\n 1 file changed, 2 insertions(+)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..69ca3d1411 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2735,6 +2735,8 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n \n \t/* Update gitfile */\n \tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n+\tif (safe_create_leading_directories_const(file_name.buf))\n+\t\tfprintf(stderr, \"could not create directories for %s\\n\", file_name.buf);\n \twrite_file(file_name.buf, \"gitdir: %s\",\n \t\t   relative_path(git_dir, work_tree, &rel_path));\n \n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311620","messageId":"20170215003423.20245-14-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 13/14] entry.c: update submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:22Z","receivedAt":"2017-02-15T00:35:40Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n entry.c | 28 ++++++++++++++++++++++++++++\n 1 file changed, 28 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..bc6295d41a 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t       const struct checkout *state)\n@@ -203,6 +204,13 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tif (is_interesting_submodule(ce))\n+\t\t\t\t/*\n+\t\t\t\t * force=1 is ok for any case as we did a dry\n+\t\t\t\t * run before with appropriate force setting\n+\t\t\t\t */\n+\t\t\t\treturn submodule_go_from_to(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid), 0, 1);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -260,6 +268,26 @@ int checkout_entry(struct cache_entry *ce,\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tif (is_interesting_submodule(ce)) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_go_from_to(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid), 0, 1);\n+\t\t\t} else\n+\t\t\t\treturn submodule_go_from_to(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), 0, 1);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311621","messageId":"20170215003423.20245-15-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 14/14] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:23Z","receivedAt":"2017-02-15T00:35:42Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n Documentation/git-checkout.txt |  7 +++++++\n builtin/checkout.c             | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh      | 33 ++++++++++++++++++++++++---------\n t/t2013-checkout-submodule.sh  |  5 +++++\n 4 files changed, 64 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..a0ea2c5651 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail until `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n <branch>::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..207ce09771 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [<options>] <branch>\"),\n \tN_(\"git checkout [<options>] [<branch>] -- <file>...\"),\n \tNULL,\n };\n \n+int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t    const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t    arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout <no-such-branch>'\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t    \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t    PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 7c8c557572..9c75a417d4 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -734,6 +734,11 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -843,7 +848,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -855,7 +860,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -883,7 +888,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -939,16 +944,21 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n+\t# This test fails, due to missing setup, we do not clone sub2 into\n+\t# submodule_update, because it doesn't exist in the 'add_sub1' version\n+\t#\n \ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n-\t\t\t$command modify_sub1_recursively &&\n-\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n-\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively\n-\t\t\ttest_submodule_content sub1/sub2\n+\t\t\ttest_must_fail $command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t\t# test_superproject_content origin/modify_sub1_recursively &&\n+\t\t\t# test_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\t# test_submodule_content sub1/sub2 no_submodule\n \t\t)\n \t'\n }\n@@ -958,6 +968,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1052,7 +1067,7 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -1103,7 +1118,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..aa35223369 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,11 @@ test_expect_success '\"checkout <submodule>\" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311622","messageId":"20170215003423.20245-13-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 12/14] read-cache: remove_marked_cache_entries to wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:21Z","receivedAt":"2017-02-15T00:35:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n read-cache.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 9054369dd0..c0776b93b0 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -18,6 +18,7 @@\n #include \"varint.h\"\n #include \"split-index.h\"\n #include \"utf8.h\"\n+#include \"submodule.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -532,6 +533,8 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n+\t\t\tif (is_interesting_submodule(ce_array[i]))\n+\t\t\t\tsubmodule_go_from_to(ce_array[i]->name, \"HEAD\", NULL, 0, 1);\n \t\t\tremove_name_hash(istate, ce_array[i]);\n \t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n \t\t}\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311623","messageId":"20170215003423.20245-4-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 03/14] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:12Z","receivedAt":"2017-02-15T00:35:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/grep.c | 2 +-\n submodule.c    | 4 ++--\n submodule.h    | 2 +-\n 3 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..9bbdd3ce7c 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -237,12 +237,12 @@ int is_submodule_initialized(const char *path)\n /*\n  * Determine if a submodule has been populated at a given 'path'\n  */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..689033e538 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,7 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311624","messageId":"20170215003423.20245-9-sbeller@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"[PATCH 08/14] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T00:34:17Z","receivedAt":"2017-02-15T00:35:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 4c33374ae8..d3fc6c2a75 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -359,6 +359,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n  * summary output. If it can locate the submodule objects directory it will\n  * attempt to lookup both the left and right commits and put them into the\n@@ -1403,18 +1420,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n  * Embeds a single submodules git directory into the superprojects git dir,\n  * non recursively.\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311631","messageId":"20170215014430.ngd5wtnvhe2nt6q5@genre.crustytoothpaste.net","threadId":"45144","inReplyTo":"20170215003423.20245-2-sbeller@google.com","subject":"Re: [PATCH 01/14] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2017-02-15T01:44:30Z","receivedAt":"2017-02-15T01:44:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 14, 2017 at 04:34:10PM -0800, Stefan Beller wrote:\n>  create_lib_submodule_repo () {\n>  \tgit init submodule_update_repo &&\n>  \t(\n> @@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n>  \t\tgit config submodule.sub1.ignore all &&\n>  \t\tgit add .gitmodules &&\n>  \t\tgit commit -m \"Add sub1\" &&\n> -\t\tgit checkout -b remove_sub1 &&\n> +\n> +\t\tgit checkout -b remove_sub1 add_sub1&&\n\nYou're missing a space before the \"&&\".\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"311633","messageId":"20170215020647.dn2drkep7o7qwqp7@genre.crustytoothpaste.net","threadId":"45144","inReplyTo":"20170215003423.20245-10-sbeller@google.com","subject":"Re: [PATCH 09/14] update submodules: add submodule_go_from_to","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2017-02-15T02:06:47Z","receivedAt":"2017-02-15T02:06:57Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 14, 2017 at 04:34:18PM -0800, Stefan Beller wrote:\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n> +\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n> +\n> +\tif (!dry_run)\n> +\t\targv_array_push(&cp.args, \"-u\");\n> +\telse\n> +\t\targv_array_push(&cp.args, \"-n\");\n\nI might write this as\n\n\tif (dry_run)\n\t\targv_array_push(&cp.args, \"-n\");\n\telse\n\t\targv_array_push(&cp.args, \"-u\");\n\nIn other words, avoiding the negation when you have an else branch.  I\ncan also see an argument for keeping the condition identical to the\nother branches, though.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"311634","messageId":"20170215020851.awwddgprd7nsdbuv@genre.crustytoothpaste.net","threadId":"45144","inReplyTo":"20170215003423.20245-15-sbeller@google.com","subject":"Re: [PATCH 14/14] builtin/checkout: add --recurse-submodules switch","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2017-02-15T02:08:51Z","receivedAt":"2017-02-15T02:09:00Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 14, 2017 at 04:34:23PM -0800, Stefan Beller wrote:\n> +--[no-]recurse-submodules::\n> +\tUsing --recurse-submodules will update the content of all initialized\n> +\tsubmodules according to the commit recorded in the superproject. If\n> +\tlocal modifications in a submodule would be overwritten the checkout\n> +\twill fail until `-f` is used. If nothing (or --no-recurse-submodules)\n\nI would say \"unless\" instead of \"until\".  \"Until\" implies an ongoing\nor repetitive action being interrupted, which isn't the case here.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"311635","messageId":"20170215021347.q6ko55eqj64roqcp@genre.crustytoothpaste.net","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"Re: [RFCv3 PATCH 00/14] Checkout aware of Submodules!","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2017-02-15T02:13:47Z","receivedAt":"2017-02-15T02:13:58Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Tue, Feb 14, 2017 at 04:34:09PM -0800, Stefan Beller wrote:\n> Integrate updating the submodules into git checkout, with the same\n> safety promises that git-checkout has, i.e. not throw away data unless\n> asked to. This is done by first checking if the submodule is at the same\n> sha1 as it is recorded in the superproject. If there are changes we stop\n> proceeding the checkout just like it is when checking out a file that\n> has local changes.\n> \n> The integration happens in the code that is also used in other commands\n> such that it will be easier in the future to make other commands aware\n> of submodule.\n> \n> This also solves d/f conflicts in case you replace a file/directory\n> with a submodule or vice versa.\n> \n> The patches are still a bit rough, but the overall series seems\n> promising enough to me that I want to put it out here.\n> \n> Any review, specifically on the design level welcome!\n\nOverall, I'm very pleased with this.  I don't really have any\ndesign-level comments at this point, only small nits.  I'm considering\nbuilding and testing it out at work, where this would be extremely\nuseful.  I'm therefore hoping to see how it works under real-world\nconditions shortly.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\n+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"311655","messageId":"20170215165107.GA29448@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-3-sbeller@google.com","subject":"Re: [PATCH 02/14] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-02-15T16:51:07Z","receivedAt":"2017-02-15T16:51:14Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/14, Stefan Beller wrote:\n> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\n> index 61c54f2098..7c8c557572 100755\n> --- a/t/lib-submodule-update.sh\n> +++ b/t/lib-submodule-update.sh\n> @@ -4,6 +4,7 @@\n>  # - New submodule (no_submodule => add_sub1)\n>  # - Removed submodule (add_sub1 => remove_sub1)\n>  # - Updated submodule (add_sub1 => modify_sub1)\n> +# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n>  # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n>  # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n>  # - Submodule replaced by tracked files in directory (add_sub1 =>\n> @@ -19,8 +20,8 @@\n>  #                    /    ^\n>  #                   /     remove_sub1\n>  #                  /\n> -#       add_sub1  /-------O\n> -#             |  /        ^\n> +#       add_sub1  /-------O---------O\n> +#             |  /        ^         modify_sub1_recursive\n>  #             | /         modify_sub1\n>  #             v/\n>  #      O------O-----------O---------O\n> @@ -73,6 +74,14 @@ create_lib_submodule_repo () {\n>  \t\tgit add sub1 &&\n>  \t\tgit commit -m \"Modify sub1\" &&\n>  \n> +\t\tgit checkout -b modify_sub1_recursively modify_sub1 &&\n> +\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n> +\t\tgit -C sub1 submodule add --branch no_submodule ./. sub2 &&\n\nI thought we were trying to avoid './.' when adding submodules?\n\n-- \nBrandon Williams\n"},{"id":"311656","messageId":"20170215170441.GB29448@google.com","threadId":"45144","inReplyTo":"20170215003423.20245-8-sbeller@google.com","subject":"Re: [PATCH 07/14] update submodules: introduce is_interesting_submodule","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-02-15T17:04:41Z","receivedAt":"2017-02-15T17:04:50Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/14, Stefan Beller wrote:\n> In later patches we introduce the --recurse-submodule flag for commands\n> that modify the working directory, e.g. git-checkout.\n> \n> It is potentially expensive to check if a submodule needs an update,\n> because a common theme to interact with submodules is to spawn a child\n> process for each interaction.\n> \n> So let's introduce a function that pre checks if a submodule needs\n> to be checked for an update.\n> \n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule.c | 26 ++++++++++++++++++++++++++\n>  submodule.h |  8 ++++++++\n>  2 files changed, 34 insertions(+)\n> \n> diff --git a/submodule.c b/submodule.c\n> index c0060c29f2..4c33374ae8 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -551,6 +551,32 @@ void set_config_update_recurse_submodules(int value)\n>  \tconfig_update_recurse_submodules = value;\n>  }\n>  \n> +int submodules_interesting_for_update(void)\n> +{\n> +\t/*\n> +\t * Update can't be \"none\", \"merge\" or \"rebase\",\n> +\t * treat any value as OFF, except an explicit ON.\n> +\t */\n> +\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n> +}\n> +\n> +int is_interesting_submodule(const struct cache_entry *ce)\n\nIs there perhaps a more descriptive function name we could use instead\nof \"is_interesting\"?  The problem is that its difficult to know why its\ninteresting or for what purpose it is interesting.\n\n> +{\n> +\tconst struct submodule *sub;\n> +\n> +\tif (!S_ISGITLINK(ce->ce_mode))\n> +\t\treturn 0;\n> +\n> +\tif (!submodules_interesting_for_update())\n> +\t\treturn 0;\n> +\n> +\tsub = submodule_from_path(null_sha1, ce->name);\n> +\tif (!sub)\n> +\t\treturn 0;\n> +\n> +\treturn sub->update_strategy.type != SM_UPDATE_NONE;\n> +}\n> +\n>  static int has_remote(const char *refname, const struct object_id *oid,\n>  \t\t      int flags, void *cb_data)\n>  {\n> diff --git a/submodule.h b/submodule.h\n> index c4e1ac828e..84b67a7c4a 100644\n> --- a/submodule.h\n> +++ b/submodule.h\n> @@ -59,6 +59,14 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n>  \t\tconst struct diff_options *opt);\n>  extern void set_config_fetch_recurse_submodules(int value);\n>  extern void set_config_update_recurse_submodules(int value);\n> +\n> +/*\n> + * Traditionally Git ignored changes made for submodules.\n> + * This function checks if we are interested in the given submodule\n> + * for any kind of operation.\n> + */\n> +extern int submodules_interesting_for_update(void);\n> +extern int is_interesting_submodule(const struct cache_entry *ce);\n>  extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n>  extern int fetch_populated_submodules(const struct argv_array *options,\n>  \t\t\t       const char *prefix, int command_line_option,\n> -- \n> 2.12.0.rc0.16.gd1691994b4.dirty\n> \n\n-- \nBrandon Williams\n"},{"id":"311657","messageId":"xmqqlgt7495z.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170215003423.20245-4-sbeller@google.com","subject":"Re: [PATCH 03/14] make is_submodule_populated gently","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-15T18:10:32Z","receivedAt":"2017-02-15T18:10:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> We need the gentle version in a later patch. As we have just one caller,\n> migrate the caller.\n\nOrdinarily, we keep the original helper implemented as a thin\nwrapper that passes NULL as retun_error_code, which causes it to\ndie() on error for existing callers.  But because we only have one\ncaller (and topics in-flight do not add new ones), we do not bother\nwith that.\n\nThe reasoning makes sense, at least to me.\n\nWe may want to add a comment about the behaviour upon error for the\nhelper function?  I see resolve_gitdir_gently() does not do so and\nthe readers have to follow the callflow down to read_gitfile_gently()\nwhich does have the comment, so perhaps we are OK without any.\n\nLooks good to me.\n\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  builtin/grep.c | 2 +-\n>  submodule.c    | 4 ++--\n>  submodule.h    | 2 +-\n>  3 files changed, 4 insertions(+), 4 deletions(-)\n>\n> diff --git a/builtin/grep.c b/builtin/grep.c\n> index 2c727ef499..b17835aed6 100644\n> --- a/builtin/grep.c\n> +++ b/builtin/grep.c\n> @@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n>  {\n>  \tif (!is_submodule_initialized(path))\n>  \t\treturn 0;\n> -\tif (!is_submodule_populated(path)) {\n> +\tif (!is_submodule_populated_gently(path, NULL)) {\n>  \t\t/*\n>  \t\t * If searching history, check for the presense of the\n>  \t\t * submodule's gitdir before skipping the submodule.\n> diff --git a/submodule.c b/submodule.c\n> index 3b98766a6b..9bbdd3ce7c 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -237,12 +237,12 @@ int is_submodule_initialized(const char *path)\n>  /*\n>   * Determine if a submodule has been populated at a given 'path'\n>   */\n> -int is_submodule_populated(const char *path)\n> +int is_submodule_populated_gently(const char *path, int *return_error_code)\n>  {\n>  \tint ret = 0;\n>  \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n>  \n> -\tif (resolve_gitdir(gitdir))\n> +\tif (resolve_gitdir_gently(gitdir, return_error_code))\n>  \t\tret = 1;\n>  \n>  \tfree(gitdir);\n> diff --git a/submodule.h b/submodule.h\n> index 05ab674f06..689033e538 100644\n> --- a/submodule.h\n> +++ b/submodule.h\n> @@ -41,7 +41,7 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n>  extern void gitmodules_config(void);\n>  extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n>  extern int is_submodule_initialized(const char *path);\n> -extern int is_submodule_populated(const char *path);\n> +extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n>  extern int parse_submodule_update_strategy(const char *value,\n>  \t\tstruct submodule_update_strategy *dst);\n>  extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n"},{"id":"311659","messageId":"xmqqh93v48mv.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170215003423.20245-5-sbeller@google.com","subject":"Re: [PATCH 04/14] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-15T18:22:00Z","receivedAt":"2017-02-15T18:22:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> In a later patch we'll use connect_work_tree_and_git_dir when the\n> directory for the gitlink file doesn't exist yet. Safely create\n> the directory first.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n\nAmong the existing two callers, the \"absorb\" logic in submodule.c\nhas safe-create-leading-directory (SCLD) immediately before the call\nto this function, which can now be lost with this change.  The other\none in cmd_mv() has the target directory as the result of moving the\noriginal directory, and I do not think there is any corresponding\ncall that can be lost from there after this change, but it is not an\nerror to call SCLD on a path that already exists, so all is OK.\n\nIs it sensible to let the code continue with just an fprintf() (not\neven warning() or error()) when SCLD fails?\n\n> ---\n>  dir.c | 2 ++\n>  1 file changed, 2 insertions(+)\n>\n> diff --git a/dir.c b/dir.c\n> index 4541f9e146..69ca3d1411 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -2735,6 +2735,8 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n>  \n>  \t/* Update gitfile */\n>  \tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n> +\tif (safe_create_leading_directories_const(file_name.buf))\n> +\t\tfprintf(stderr, \"could not create directories for %s\\n\", file_name.buf);\n\n>  \twrite_file(file_name.buf, \"gitdir: %s\",\n>  \t\t   relative_path(git_dir, work_tree, &rel_path));\n"},{"id":"311660","messageId":"xmqqd1ej48am.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170215003423.20245-6-sbeller@google.com","subject":"Re: [PATCH 05/14] update submodules: add submodule config parsing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-15T18:29:21Z","receivedAt":"2017-02-15T18:29:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Similar to b33a15b08 (push: add recurseSubmodules config option,\n> 2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\n> fetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\n> that is later used to parse whether we are interested in updating\n> submodules.\n>\n> We need the `die_on_error` parameter to be able to call this parsing\n> function for the config file as well, which if incorrect lets Git die.\n>\n> As we're just touching the header file, also mark all functions extern.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule-config.c | 22 ++++++++++++++++++++++\n>  submodule-config.h | 17 +++++++++--------\n>  2 files changed, 31 insertions(+), 8 deletions(-)\n>\n> diff --git a/submodule-config.c b/submodule-config.c\n> index 93453909cf..93f01c4378 100644\n> --- a/submodule-config.c\n> +++ b/submodule-config.c\n> @@ -234,6 +234,28 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n>  \treturn parse_fetch_recurse(opt, arg, 1);\n>  }\n>  \n> +static int parse_update_recurse(const char *opt, const char *arg,\n> +\t\t\t\tint die_on_error)\n> +{\n> +\tswitch (git_config_maybe_bool(opt, arg)) {\n> +\tcase 1:\n> +\t\treturn RECURSE_SUBMODULES_ON;\n> +\tcase 0:\n> +\t\treturn RECURSE_SUBMODULES_OFF;\n> +\tdefault:\n> +\t\tif (!strcmp(arg, \"checkout\"))\n> +\t\t\treturn RECURSE_SUBMODULES_ON;\n> +\t\tif (die_on_error)\n> +\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n> +\t\treturn RECURSE_SUBMODULES_ERROR;\n> +\t}\n> +}\n\nProliferation of similarly looking config parser made me briefly\nwonder if they can somehow be shared, but I think it is correct to\nview that update/fetch/push have conceptually different set of\nallowed modes, whose names happen to overlap, so keeping them\nseparate like this patch does (and its predecessors did to take us\ninto the shape of the current code) makes sense, at least to me.\n\n"},{"id":"311662","messageId":"xmqq8tp74823.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170215003423.20245-1-sbeller@google.com","subject":"Re: [RFCv3 PATCH 00/14] Checkout aware of Submodules!","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-15T18:34:28Z","receivedAt":"2017-02-15T18:34:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Integrate updating the submodules into git checkout, with the same\n> safety promises that git-checkout has, i.e. not throw away data unless\n> asked to. This is done by first checking if the submodule is at the same\n> sha1 as it is recorded in the superproject.\n\nI've so far read only the first half of this series (i.e. the\npreparatory part) and haven't reached the place where it starts to\nbecome really interesting, but what I've seen so far looked very\nsensible.  I also find that comments given by others so far all\nraise good points.\n\n"},{"id":"311664","messageId":"CAGZ79kbN4e4btgVzzW7yKyUuaf7htmYFutp7ALhtygsKpFsTyQ@mail.gmail.com","threadId":"45144","inReplyTo":"20170215170441.GB29448@google.com","subject":"Re: [PATCH 07/14] update submodules: introduce is_interesting_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T18:46:06Z","receivedAt":"2017-02-15T18:46:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 15, 2017 at 9:04 AM, Brandon Williams <bmwill@google.com> wrote:\n> On 02/14, Stefan Beller wrote:\n>> In later patches we introduce the --recurse-submodule flag for commands\n>> that modify the working directory, e.g. git-checkout.\n>>\n>> It is potentially expensive to check if a submodule needs an update,\n>> because a common theme to interact with submodules is to spawn a child\n>> process for each interaction.\n>>\n>> So let's introduce a function that pre checks if a submodule needs\n>> to be checked for an update.\n>>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>>  submodule.c | 26 ++++++++++++++++++++++++++\n>>  submodule.h |  8 ++++++++\n>>  2 files changed, 34 insertions(+)\n>>\n>> diff --git a/submodule.c b/submodule.c\n>> index c0060c29f2..4c33374ae8 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -551,6 +551,32 @@ void set_config_update_recurse_submodules(int value)\n>>       config_update_recurse_submodules = value;\n>>  }\n>>\n>> +int submodules_interesting_for_update(void)\n>> +{\n>> +     /*\n>> +      * Update can't be \"none\", \"merge\" or \"rebase\",\n>> +      * treat any value as OFF, except an explicit ON.\n>> +      */\n>> +     return config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n>> +}\n>> +\n>> +int is_interesting_submodule(const struct cache_entry *ce)\n>\n> Is there perhaps a more descriptive function name we could use instead\n> of \"is_interesting\"?  The problem is that its difficult to know why its\n> interesting or for what purpose it is interesting.\n\nI should finish the background story patch first. By 'is_interesting' I mean\n* it is active/initialized/\"The user expressed interested in the submodule by\n  setting submodule.<name>.URL\n* its submodule.<name>.update strategy is != NONE.\n\nThe second point is interesting, as that entertains the thought that we'll pay\nattention to the submodule.<name>.update strategy at all and\nwe may want to also implement rebase/merge eventually.\n\nSo I think we'd want to tighten that down to \"checkout\" only for now.\n\nThanks,\nStefan\n"},{"id":"311665","messageId":"CAGZ79kaYRHHas-qiAr+RJB2+TLwEgcsOcXUk-SQ65qiSPSWXOw@mail.gmail.com","threadId":"45144","inReplyTo":"20170215165107.GA29448@google.com","subject":"Re: [PATCH 02/14] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T18:52:42Z","receivedAt":"2017-02-15T18:52:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 15, 2017 at 8:51 AM, Brandon Williams <bmwill@google.com> wrote:\n> On 02/14, Stefan Beller wrote:\n>> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\n>> index 61c54f2098..7c8c557572 100755\n>> --- a/t/lib-submodule-update.sh\n>> +++ b/t/lib-submodule-update.sh\n>> @@ -4,6 +4,7 @@\n>>  # - New submodule (no_submodule => add_sub1)\n>>  # - Removed submodule (add_sub1 => remove_sub1)\n>>  # - Updated submodule (add_sub1 => modify_sub1)\n>> +# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n>>  # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n>>  # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n>>  # - Submodule replaced by tracked files in directory (add_sub1 =>\n>> @@ -19,8 +20,8 @@\n>>  #                    /    ^\n>>  #                   /     remove_sub1\n>>  #                  /\n>> -#       add_sub1  /-------O\n>> -#             |  /        ^\n>> +#       add_sub1  /-------O---------O\n>> +#             |  /        ^         modify_sub1_recursive\n>>  #             | /         modify_sub1\n>>  #             v/\n>>  #      O------O-----------O---------O\n>> @@ -73,6 +74,14 @@ create_lib_submodule_repo () {\n>>               git add sub1 &&\n>>               git commit -m \"Modify sub1\" &&\n>>\n>> +             git checkout -b modify_sub1_recursively modify_sub1 &&\n>> +             git -C sub1 checkout -b \"add_nested_sub\" &&\n>> +             git -C sub1 submodule add --branch no_submodule ./. sub2 &&\n>\n> I thought we were trying to avoid './.' when adding submodules?\n>\n\nYes we should; I'll fix that in a reroll.\nIt's also still on my long term fix list to remove the ./.\n$ git grep 'add \\./\\.'\nlib-submodule-update.sh:                git submodule add ./. sub1 &&\nt7001-mv.sh:    git submodule add ./. sub &&\nt7001-mv.sh:    git submodule add ./. deep/directory/hierarchy/sub &&\nt7507-commit-verbose.sh:        git submodule add ./. sub &&\nt7800-difftool.sh:      git submodule add ./. submod/ule &&\n"},{"id":"311691","messageId":"CAGZ79kYb_w62ObACPnrMtzDrsRgqk7HWfhCFvCe=1oeaQTx6RA@mail.gmail.com","threadId":"45144","inReplyTo":"xmqqlgt7495z.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 03/14] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T22:42:34Z","receivedAt":"2017-02-15T22:42:45Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 15, 2017 at 10:10 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> We need the gentle version in a later patch. As we have just one caller,\n>> migrate the caller.\n>\n> Ordinarily, we keep the original helper implemented as a thin\n> wrapper that passes NULL as retun_error_code, which causes it to\n> die() on error for existing callers.  But because we only have one\n> caller (and topics in-flight do not add new ones), we do not bother\n> with that.\n\nRight.\n\n>\n> The reasoning makes sense, at least to me.\n>\n> We may want to add a comment about the behaviour upon error for the\n> helper function?  I see resolve_gitdir_gently() does not do so and\n> the readers have to follow the callflow down to read_gitfile_gently()\n> which does have the comment, so perhaps we are OK without any.\n>\n> Looks good to me.\n\nWill do in a reroll.\n\nThanks,\nStefan\n"},{"id":"311694","messageId":"CAGZ79kaX1URNvzp8QuFEFnJDzSW9nE971+nvhPLOQQx4aSyBkA@mail.gmail.com","threadId":"45144","inReplyTo":"xmqqh93v48mv.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 04/14] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-15T22:52:50Z","receivedAt":"2017-02-15T22:53:02Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 15, 2017 at 10:22 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> In a later patch we'll use connect_work_tree_and_git_dir when the\n>> directory for the gitlink file doesn't exist yet. Safely create\n>> the directory first.\n>>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>\n> Among the existing two callers, the \"absorb\" logic in submodule.c\n> has safe-create-leading-directory (SCLD) immediately before the call\n> to this function, which can now be lost with this change.  The other\n> one in cmd_mv() has the target directory as the result of moving the\n> original directory, and I do not think there is any corresponding\n> call that can be lost from there after this change, but it is not an\n> error to call SCLD on a path that already exists, so all is OK.\n>\n> Is it sensible to let the code continue with just an fprintf() (not\n> even warning() or error()) when SCLD fails?\n\nI'll move the code from the absorbing here (i.e. lose the\nSCLD lines there) and make it a die(_(..)) instead of fprintf here.\n\nThanks,\nStefan\n"},{"id":"311698","messageId":"xmqqpoij11qr.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"CAGZ79kaX1URNvzp8QuFEFnJDzSW9nE971+nvhPLOQQx4aSyBkA@mail.gmail.com","subject":"Re: [PATCH 04/14] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-15T23:19:08Z","receivedAt":"2017-02-15T23:19:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> On Wed, Feb 15, 2017 at 10:22 AM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Stefan Beller <sbeller@google.com> writes:\n>>\n>>> In a later patch we'll use connect_work_tree_and_git_dir when the\n>>> directory for the gitlink file doesn't exist yet. Safely create\n>>> the directory first.\n>>>\n>>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>>\n>> Among the existing two callers, the \"absorb\" logic in submodule.c\n>> has safe-create-leading-directory (SCLD) immediately before the call\n>> to this function, which can now be lost with this change.  The other\n>> one in cmd_mv() has the target directory as the result of moving the\n>> original directory, and I do not think there is any corresponding\n>> call that can be lost from there after this change, but it is not an\n>> error to call SCLD on a path that already exists, so all is OK.\n>>\n>> Is it sensible to let the code continue with just an fprintf() (not\n>> even warning() or error()) when SCLD fails?\n>\n> I'll move the code from the absorbing here (i.e. lose the\n> SCLD lines there) and make it a die(_(..)) instead of fprintf here.\n\nOK.  I didn't actually meant to suggest the former (I meant that I\nexpect that would happen in the future steps of this series, or it\ncan be left as a follow-up patch after the series settles); the\nlatter may be worth doing.\n\nThanks.\n"},{"id":"311712","messageId":"CAGZ79kbT2Gh70_OsE7d63pCxNSDb-RJBZmXsPKvkuRD9x__b9A@mail.gmail.com","threadId":"45144","inReplyTo":"20170215020851.awwddgprd7nsdbuv@genre.crustytoothpaste.net","subject":"Re: [PATCH 14/14] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:33:56Z","receivedAt":"2017-02-16T00:34:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 14, 2017 at 6:08 PM, brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n> On Tue, Feb 14, 2017 at 04:34:23PM -0800, Stefan Beller wrote:\n>> +--[no-]recurse-submodules::\n>> +     Using --recurse-submodules will update the content of all initialized\n>> +     submodules according to the commit recorded in the superproject. If\n>> +     local modifications in a submodule would be overwritten the checkout\n>> +     will fail until `-f` is used. If nothing (or --no-recurse-submodules)\n>\n> I would say \"unless\" instead of \"until\".  \"Until\" implies an ongoing\n> or repetitive action being interrupted, which isn't the case here.\n\neh, of course.\n"},{"id":"311713","messageId":"20170216003811.18273-6-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 05/15] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:01Z","receivedAt":"2017-02-16T00:38:32Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. Safely create\nthe directory first.\n\nOne of the two users of 'connect_work_tree_and_git_dir' already checked\nfor the directory being there, so we can loose that check.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n dir.c       | 32 +++++++++++++++++++++-----------\n submodule.c | 11 ++---------\n 2 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..6f52af7abb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n /* Update gitfile and core.worktree setting to connect work tree and git dir */\n void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n {\n-\tstruct strbuf file_name = STRBUF_INIT;\n+\tstruct strbuf gitfile_sb = STRBUF_INIT;\n+\tstruct strbuf cfg_sb = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n-\tchar *git_dir = real_pathdup(git_dir_);\n-\tchar *work_tree = real_pathdup(work_tree_);\n+\tchar *git_dir, *work_tree;\n \n-\t/* Update gitfile */\n-\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n-\twrite_file(file_name.buf, \"gitdir: %s\",\n-\t\t   relative_path(git_dir, work_tree, &rel_path));\n+\t/* Prepare .git file */\n+\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n+\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n+\n+\t/* Prepare config file */\n+\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n+\tif (safe_create_leading_directories_const(cfg_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n \n+\tgit_dir = real_pathdup(git_dir_);\n+\twork_tree = real_pathdup(work_tree_);\n+\n+\t/* Write .git file */\n+\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n+\t\t   relative_path(git_dir, work_tree, &rel_path));\n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n+\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n \t\t\t       relative_path(work_tree, git_dir, &rel_path));\n \n-\tstrbuf_release(&file_name);\n+\tstrbuf_release(&gitfile_sb);\n+\tstrbuf_release(&cfg_sb);\n \tstrbuf_release(&rel_path);\n \tfree(work_tree);\n \tfree(git_dir);\ndiff --git a/submodule.c b/submodule.c\nindex 0e55372f37..04d185738f 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1442,8 +1442,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \n \t/* Not populated? */\n \tif (!sub_git_dir) {\n-\t\tchar *real_new_git_dir;\n-\t\tconst char *new_git_dir;\n \t\tconst struct submodule *sub;\n \n \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n@@ -1466,13 +1464,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tsub = submodule_from_path(null_sha1, path);\n \t\tif (!sub)\n \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n-\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n-\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n-\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n-\t\treal_new_git_dir = real_pathdup(new_git_dir);\n-\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n-\n-\t\tfree(real_new_git_dir);\n+\t\tconnect_work_tree_and_git_dir(path,\n+\t\t\tgit_path(\"modules/%s\", sub->name));\n \t} else {\n \t\t/* Is it already absorbed into the superprojects git dir? */\n \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311714","messageId":"20170216003811.18273-1-sbeller@google.com","threadId":"45144","inReplyTo":"xmqq8tp74823.fsf@gitster.mtv.corp.google.com","subject":"[RFCv4 PATCH 00/14] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:37:56Z","receivedAt":"2017-02-16T00:38:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20161203003022.29797-1-sbeller@google.com/\n\nv4:\n * addressed all comments of Brian, Junio and Brandon.\n Thanks!\n * one major point of change is the introduction of another patch\n   \"lib-submodule-update.sh: do not use ./. as submodule remote\",\n   as that took some time to track down the existing bug.\n \nv3:\n * moved tests from t2013 to the generic submodule library.\n * factored out the refactoring patches to be up front\n * As I redid the complete implementation, I have the impression this time\n   it is cleaner than previous versions.\n \n I think we still have to fix the corner cases of directory/file/submodule \n conflicts before merging, but this serves as a status update on my current\n way of thinking how to implement the worktree commands being aware of\n submodules.\n \nThanks,\nStefan\n\nv2:\n* based on top of the series sent out an hour ago\n  \"[PATCHv4 0/5] submodule embedgitdirs\"\n* Try to embed a submodule if we need to remove it.\n* Strictly do not change behavior if not giving the new flag.\n* I think I missed some review comments from v1, but I'd like to get\n  the current state out over the weekend, as a lot has changed so far.\n  On Monday I'll go through the previous discussion with a comb to see\n  if I missed something.\n  \nv1:\nWhen working with submodules, nearly anytime after checking out\na different state of the projects, that has submodules changed\nyou'd run \"git submodule update\" with a current version of Git.\n\nThere are two problems with this approach:\n\n* The \"submodule update\" command is dangerous as it\n  doesn't check for work that may be lost in the submodule\n  (e.g. a dangling commit).\n* you may forget to run the command as checkout is supposed\n  to do all the work for you.\n\nIntegrate updating the submodules into git checkout, with the same\nsafety promises that git-checkout has, i.e. not throw away data unless\nasked to. This is done by first checking if the submodule is at the same\nsha1 as it is recorded in the superproject. If there are changes we stop\nproceeding the checkout just like it is when checking out a file that\nhas local changes.\n\nThe integration happens in the code that is also used in other commands\nsuch that it will be easier in the future to make other commands aware\nof submodule.\n\nThis also solves d/f conflicts in case you replace a file/directory\nwith a submodule or vice versa.\n\nThe patches are still a bit rough, but the overall series seems\npromising enough to me that I want to put it out here.\n\nAny review, specifically on the design level welcome!\n\nThanks,\nStefan\n\n\nStefan Beller (14):\n  lib-submodule-update.sh: reorder create_lib_submodule_repo\n  lib-submodule-update.sh: define tests for recursing into submodules\n  make is_submodule_populated gently\n  connect_work_tree_and_git_dir: safely create leading directories\n  update submodules: add submodule config parsing\n  update submodules: add a config option to determine if submodules are\n    updated\n  update submodules: introduce is_interesting_submodule\n  update submodules: move up prepare_submodule_repo_env\n  update submodules: add submodule_go_from_to\n  unpack-trees: pass old oid to verify_clean_submodule\n  unpack-trees: check if we can perform the operation for submodules\n  read-cache: remove_marked_cache_entries to wipe selected submodules.\n  entry.c: update submodules when interesting\n  builtin/checkout: add --recurse-submodules switch\n\n Documentation/git-checkout.txt |   7 +\n builtin/checkout.c             |  28 +++\n builtin/grep.c                 |   2 +-\n dir.c                          |   2 +\n entry.c                        |  28 +++\n read-cache.c                   |   3 +\n submodule-config.c             |  22 ++\n submodule-config.h             |  17 +-\n submodule.c                    | 216 +++++++++++++++--\n submodule.h                    |  16 +-\n t/lib-submodule-update.sh      | 534 +++++++++++++++++++++++++++++++++++++++--\n t/t2013-checkout-submodule.sh  |   5 +\n unpack-trees.c                 | 115 +++++++--\n unpack-trees.h                 |   1 +\n 14 files changed, 936 insertions(+), 60 deletions(-)\n\n-- \n2.12.0.rc0.16.gd1691994b4.dirty\n\n"},{"id":"311715","messageId":"20170216003811.18273-8-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 07/15] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:03Z","receivedAt":"2017-02-16T00:38:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 04d185738f..591f4a694e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -542,6 +543,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t      int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 0b915bd3ac..b4e60c08d2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -64,6 +64,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311716","messageId":"20170216003811.18273-4-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 03/15] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:37:59Z","receivedAt":"2017-02-16T00:38:51Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to properly recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 485 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 483 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex c0d6325133..ea838df028 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n #                    /    ^\n #                   /     remove_sub1\n #                  /\n-#       add_sub1  /-------O\n-#             |  /        ^\n+#       add_sub1  /-------O---------O\n+#             |  /        ^         modify_sub1_recursive\n #             | /         modify_sub1\n #             v/\n #      O------O-----------O---------O\n@@ -48,6 +49,17 @@ create_lib_submodule_repo () {\n \t\tgit commit -m \"Base inside first submodule\" &&\n \t\tgit branch \"no_submodule\"\n \t) &&\n+\tgit init submodule_update_sub2 &&\n+\t(\n+\t\tcd submodule_update_sub2\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"nested submodule base\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -84,6 +96,14 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b modify_sub1_recursively modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -150,6 +170,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -180,6 +209,18 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# also have it available:\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\tfi\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -695,3 +736,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+#  in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+#   git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 > .git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignroed\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively\n+\t\t\ttest_submodule_content sub1/sub2\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: > sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: > sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311717","messageId":"20170216003811.18273-3-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 02/15] lib-submodule-update.sh: do not use ./. as submodule remote","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:37:58Z","receivedAt":"2017-02-16T00:38:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Adding the repository itself as a submodule does not make sense in the\nreal world. In our test suite we used to do that out of convenience in\nsome tests as the current repository has easiest access for setting up\n'just a submodule'.\n\nHowever this doesn't quite test the real world, so let's do not follow\nthis pattern any further and actually create an independent repository\nthat we can use as a submodule.\n\nWhen using './.' as the remote the superproject and submodule share the\nsame objects, such that testing if a given sha1 is a valid commit works\nin either repository.  As running commands in an unpopulated submodule\nfall back to the superproject, this happens in `reset_work_tree_to`\nto determine if we need to populate the submodule. Fix this bug by\nchecking in the actual remote now.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 15 +++++++++++++--\n 1 file changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5df528ea81..c0d6325133 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -37,6 +37,17 @@\n #\n \n create_lib_submodule_repo () {\n+\tgit init submodule_update_sub1 &&\n+\t(\n+\t\tcd submodule_update_sub1 &&\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"Base inside first submodule\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -49,7 +60,7 @@ create_lib_submodule_repo () {\n \t\tgit branch \"no_submodule\" &&\n \n \t\tgit checkout -b \"add_sub1\" &&\n-\t\tgit submodule add ./. sub1 &&\n+\t\tgit submodule add ../submodule_update_sub1 sub1 &&\n \t\tgit config -f .gitmodules submodule.sub1.ignore all &&\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n@@ -162,7 +173,7 @@ reset_work_tree_to () {\n \t\ttest_must_be_empty actual &&\n \t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n \t\tif test -n \"$sha1\" &&\n-\t\t   test $(cd \"sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\t   test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311718","messageId":"20170216003811.18273-9-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 08/15] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:04Z","receivedAt":"2017-02-16T00:38:54Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that checks if a submodule needs\nto be checked for an update before attempting the update.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 27 +++++++++++++++++++++++++++\n submodule.h | 13 +++++++++++++\n 2 files changed, 40 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 591f4a694e..2a37e03420 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -548,6 +548,33 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int touch_submodules_in_worktree(void)\n+{\n+\t/*\n+\t * Update can't be \"none\", \"merge\" or \"rebase\",\n+\t * treat any value as OFF, except an explicit ON.\n+\t */\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+int is_active_submodule_with_strategy(const struct cache_entry *ce,\n+\t\t\t\t      enum submodule_update_type strategy)\n+{\n+\tconst struct submodule *sub;\n+\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn 0;\n+\n+\tif (!touch_submodules_in_worktree())\n+\t\treturn 0;\n+\n+\tsub = submodule_from_path(null_sha1, ce->name);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\treturn sub->update_strategy.type == strategy;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t      int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex b4e60c08d2..46d9f0f293 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,6 +65,19 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+\n+/*\n+ * Traditionally Git ignored changes made for submodules.\n+ * This function checks if we are interested in the given submodule\n+ * for any kind of operation.\n+ */\n+extern int touch_submodules_in_worktree(void);\n+/*\n+ * Check if the given ce entry is a submodule with the given update\n+ * strategy configured.\n+ */\n+extern int is_active_submodule_with_strategy(const struct cache_entry *ce,\n+\t\t\t\t\t     enum submodule_update_type strategy);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311719","messageId":"20170216003811.18273-5-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 04/15] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:00Z","receivedAt":"2017-02-16T00:38:57Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/grep.c | 2 +-\n submodule.c    | 7 ++-----\n submodule.h    | 8 +++++++-\n 3 files changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..0e55372f37 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -234,15 +234,12 @@ int is_submodule_initialized(const char *path)\n \treturn ret;\n }\n \n-/*\n- * Determine if a submodule has been populated at a given 'path'\n- */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..0b915bd3ac 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,13 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+/*\n+ * Determine if a submodule has been populated at a given 'path' by checking if\n+ * the <path>/.git resolves to a valid git repository.\n+ * If return_error_code is NULL, die on error.\n+ * Otherwise the return error code is the same as of resolve_gitdir_gently.\n+ */\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311720","messageId":"20170216003811.18273-10-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 09/15] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:05Z","receivedAt":"2017-02-16T00:38:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 2a37e03420..b262c5b0ad 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -356,6 +356,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n  * summary output. If it can locate the submodule objects directory it will\n  * attempt to lookup both the left and right commits and put them into the\n@@ -1401,18 +1418,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n  * Embeds a single submodules git directory into the superprojects git dir,\n  * non recursively.\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311721","messageId":"20170216003811.18273-11-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 10/15] update submodules: add submodule_go_from_to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:06Z","receivedAt":"2017-02-16T00:38:59Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n  \"Is it safe to change the submodule to this new state?\"\n  e.g. is it overwriting untracked files or are there local\n  changes that would be overwritten?\n* supports a force flag that can be used for resetting\n  the tree.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h |   5 ++\n 2 files changed, 156 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex b262c5b0ad..84cc62f3bb 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1250,6 +1250,157 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tssize_t len;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tint ret = 0;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.out = -1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule %s\", sub->path);\n+\n+\tlen = strbuf_read(&buf, cp.out, 1024);\n+\tif (len > 2)\n+\t\tret = 1;\n+\n+\tclose(cp.out);\n+\tif (finish_command(&cp))\n+\t\tdie(\"could not recurse into submodule %s\", sub->path);\n+\n+\tstrbuf_release(&buf);\n+\treturn ret;\n+}\n+\n+void submodule_clean_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not clean submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ *\n+ * TODO: move dryrun and forced to flags.\n+ */\n+int submodule_go_from_to(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t int dry_run,\n+\t\t\t int force)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (!dry_run) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t    get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_clean_index(path);\n+\t\t}\n+\t}\n+\n+\tif (old && !force) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub)) {\n+\t\t\t/* print a thing here? */\n+\t\t\treturn -1;\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (dry_run)\n+\t\targv_array_push(&cp.args, \"-n\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-u\");\n+\n+\tif (force)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!dry_run) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 46d9f0f293..3336607bfc 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -102,6 +102,11 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+extern int submodule_go_from_to(const char *path,\n+\t\t\t\tconst char *old,\n+\t\t\t\tconst char *new,\n+\t\t\t\tint dry_run, int force);\n+\n /*\n  * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n  * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311722","messageId":"20170216003811.18273-7-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 06/15] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:02Z","receivedAt":"2017-02-16T00:39:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule-config.c | 22 ++++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 31 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..93f01c4378 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,28 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (!strcmp(arg, \"checkout\"))\n+\t\t\treturn RECURSE_SUBMODULES_ON;\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t       int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t      unsigned char *gitmodules_sha1,\n \t\t\t\t      struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311723","messageId":"20170216003811.18273-14-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 13/15] read-cache: remove_marked_cache_entries to wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:09Z","receivedAt":"2017-02-16T00:39:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n read-cache.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 9054369dd0..b78a7f02e3 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -18,6 +18,7 @@\n #include \"varint.h\"\n #include \"split-index.h\"\n #include \"utf8.h\"\n+#include \"submodule.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -532,6 +533,8 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n+\t\t\tif (is_active_submodule_with_strategy(ce_array[i], SM_UPDATE_UNSPECIFIED))\n+\t\t\t\tsubmodule_go_from_to(ce_array[i]->name, \"HEAD\", NULL, 0, 1);\n \t\t\tremove_name_hash(istate, ce_array[i]);\n \t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n \t\t}\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311724","messageId":"20170216003811.18273-16-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 15/15] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:11Z","receivedAt":"2017-02-16T00:39:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n Documentation/git-checkout.txt |  7 +++++++\n builtin/checkout.c             | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh      | 33 ++++++++++++++++++++++++---------\n t/t2013-checkout-submodule.sh  |  5 +++++\n 4 files changed, 64 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..d6399c0af8 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n <branch>::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..207ce09771 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [<options>] <branch>\"),\n \tN_(\"git checkout [<options>] [<branch>] -- <file>...\"),\n \tNULL,\n };\n \n+int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t    const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t    arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout <no-such-branch>'\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t    \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t    PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex ea838df028..4693ba7a7e 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -756,6 +756,11 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -865,7 +870,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -877,7 +882,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -905,7 +910,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -961,16 +966,21 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n+\t# This test fails, due to missing setup, we do not clone sub2 into\n+\t# submodule_update, because it doesn't exist in the 'add_sub1' version\n+\t#\n \ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n-\t\t\t$command modify_sub1_recursively &&\n-\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n-\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively\n-\t\t\ttest_submodule_content sub1/sub2\n+\t\t\ttest_must_fail $command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t\t# test_superproject_content origin/modify_sub1_recursively &&\n+\t\t\t# test_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\t# test_submodule_content sub1/sub2 no_submodule\n \t\t)\n \t'\n }\n@@ -980,6 +990,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1074,7 +1089,7 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -1125,7 +1140,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..aa35223369 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,11 @@ test_expect_success '\"checkout <submodule>\" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311725","messageId":"20170216003811.18273-12-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 11/15] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:07Z","receivedAt":"2017-02-16T00:39:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n  * Currently, git does not checkout subprojects during a superproject\n  * checkout, so it is not going to overwrite anything.\n  */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t  const struct cache_entry *ce,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t    resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t      ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311726","messageId":"20170216003811.18273-13-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:08Z","receivedAt":"2017-02-16T00:39:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n unpack-trees.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++------\n unpack-trees.h |  1 +\n 2 files changed, 90 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..40af8e9b5f 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,7 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n \n /*\n  * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +46,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE */\n+\t\"Submodule '%s' cannot be deleted as it contains untracked files.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +165,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot be deleted as it contains untracked files.\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +246,44 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int submodule_check_from_to(const struct cache_entry *ce, const char *old_id, const char *new_id, struct unpack_trees_options *o)\n+{\n+\tif (submodule_go_from_to(ce->name, old_id,\n+\t\t\t\t new_id, 1, o->reset))\n+\t\treturn o->gently ? -1 :\n+\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE, ce->name);\n+\treturn 0;\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t   struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+\tgitmodules_config();\n+\tgit_config(submodule_config, NULL);\n+}\n+\n /*\n  * Unlink the last component and schedule the leading directories for\n  * removal, such that empty directories get removed.\n  */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n+\t\tsubmodule_go_from_to(ce->name, \"HEAD\", NULL, 0, 1);\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +339,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (touch_submodules_in_worktree() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1399,27 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n+\t\t\tint r;\n+\t\t\tr = submodule_check_from_to(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index.  This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1463,12 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n+\t\treturn 0;\n+\n+\treturn submodule_check_from_to(ce,\n+\t\t\t\t       old_sha1,\n+\t\t\t\t       oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1634,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n+\t\t\t\tret = submodule_check_from_to(ce,\n+\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1650,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n+\t\t\treturn submodule_check_from_to(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t       NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t  ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t  error_type, o);\n@@ -1643,6 +1709,16 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n+\t\t\tint ret = submodule_check_from_to(ce,\n+\t\t\t\t\t\t\t  NULL,\n+\t\t\t\t\t\t\t  oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t  o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1739,10 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n+\t\t\tif (submodule_check_from_to(ce, oid_to_hex(&old->oid), oid_to_hex(&ce->oid), o))\n+\t\t\t\treturn -1;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..c0427ce082 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_UNTRACKED_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311727","messageId":"20170216003811.18273-15-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 14/15] entry.c: update submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:38:10Z","receivedAt":"2017-02-16T00:39:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n entry.c | 28 ++++++++++++++++++++++++++++\n 1 file changed, 28 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..ae40611c97 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t       const struct checkout *state)\n@@ -203,6 +204,13 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n+\t\t\t\t/*\n+\t\t\t\t * force=1 is ok for any case as we did a dry\n+\t\t\t\t * run before with appropriate force setting\n+\t\t\t\t */\n+\t\t\t\treturn submodule_go_from_to(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid), 0, 1);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -260,6 +268,26 @@ int checkout_entry(struct cache_entry *ce,\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_go_from_to(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid), 0, 1);\n+\t\t\t} else\n+\t\t\t\treturn submodule_go_from_to(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), 0, 1);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311728","messageId":"20170216003811.18273-2-sbeller@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"[PATCH 01/15] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T00:37:57Z","receivedAt":"2017-02-16T00:39:35Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand.  The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..5df528ea81 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n #   replace_file_with_sub1)\n #\n-#                   --O-----O\n-#                  /  ^     replace_directory_with_sub1\n-#                 /   replace_sub1_with_directory\n-#                /----O\n-#               /     ^\n-#              /      modify_sub1\n-#      O------O-------O\n-#      ^      ^\\      ^\n-#      |      | \\     remove_sub1\n-#      |      |  -----O-----O\n-#      |      |   \\   ^     replace_file_with_sub1\n-#      |      |    \\  replace_sub1_with_file\n-#      |   add_sub1 --O-----O\n-# no_submodule        ^     valid_sub1\n-#                     invalid_sub1\n+#                     ----O\n+#                    /    ^\n+#                   /     remove_sub1\n+#                  /\n+#       add_sub1  /-------O\n+#             |  /        ^\n+#             | /         modify_sub1\n+#             v/\n+#      O------O-----------O---------O\n+#      ^       \\          ^         replace_directory_with_sub1\n+#      |        \\         replace_sub1_with_directory\n+# no_submodule   \\\n+#                 --------O---------O\n+#                  \\      ^         replace_file_with_sub1\n+#                   \\     replace_sub1_with_file\n+#                    \\\n+#                     ----O---------O\n+#                         ^         valid_sub1\n+#                         invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"311784","messageId":"20170216180113.GA158636@google.com","threadId":"45144","inReplyTo":"20170216003811.18273-13-sbeller@google.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-02-16T18:01:13Z","receivedAt":"2017-02-16T18:01:26Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/15, Stefan Beller wrote:\n> +static void reload_gitmodules_file(struct index_state *index,\n> +\t\t\t\t   struct checkout *state)\n> +{\n> +\tint i;\n> +\tfor (i = 0; i < index->cache_nr; i++) {\n> +\t\tstruct cache_entry *ce = index->cache[i];\n> +\t\tif (ce->ce_flags & CE_UPDATE) {\n> +\n> +\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n> +\t\t\tif (r < 0)\n> +\t\t\t\tcontinue;\n> +\t\t\telse if (r == 0) {\n> +\t\t\t\tcheckout_entry(ce, state, NULL);\n> +\t\t\t} else\n> +\t\t\t\tbreak;\n> +\t\t}\n> +\t}\n> +\tgitmodules_config();\n> +\tgit_config(submodule_config, NULL);\n> +}\n\nIf we are reloading the gitmodules file do you think it would makes\nsense to add in a call to 'submodule_free()' to clear the cache used to\nstore the gitmodules config?\n\n-- \nBrandon Williams\n"},{"id":"311802","messageId":"xmqqlgt5x430.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-4-sbeller@google.com","subject":"Re: [PATCH 03/15] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T20:39:47Z","receivedAt":"2017-02-16T20:39:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Currently lib-submodule-update.sh provides 2 functions\n> test_submodule_switch and test_submodule_forced_switch that are used by a\n> variety of tests to ensure that submodules behave as expected. The current\n> expected behavior is that submodules are not touched at all (see\n> 42639d2317a for the exact setup).\n>\n> In the future we want to teach all these commands to properly recurse\n> into submodules. To do that, we'll add two testing functions to\n> submodule-update-lib.sh test_submodule_switch_recursing and\n> test_submodule_forced_switch_recursing.\n\nI'd remove \"properly\" and insert a colon after \"add two ... to\nsubmodule-update-lib.sh\" before the names of two functions.\n\n> +reset_work_tree_to_interested () {\n> +\treset_work_tree_to $1 &&\n> +\t# indicate we are interested in the submodule:\n> +\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n> +\t# also have it available:\n> +\tif ! test -d submodule_update/.git/modules/sub1\n> +\tthen\n> +\t\tmkdir submodule_update/.git/modules &&\n\nWould we want \"mkdir -p\" here to be safe?\n\n> +\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n\n... ahh, wouldn't matter that much, we checked that modules/sub1\ndoes not exist, and as long as nobody creates modules/ or modules/somethingelse\nwe are OK.\n\n> +\tfi\n> +}\n> +\n\n> @@ -695,3 +736,443 @@ test_submodule_forced_switch () {\n>  \t\t)\n>  \t'\n>  }\n> +\n> +# Test that submodule contents are correctly updated when switching\n> +# between commits that change a submodule.\n> +# Test that the following transitions are correctly handled:\n> +# (These tests are also above in the case where we expect no change\n> +#  in the submodule)\n> +# - Updated submodule\n> +# - New submodule\n> +# - Removed submodule\n> +# - Directory containing tracked files replaced by submodule\n> +# - Submodule replaced by tracked files in directory\n> +# - Submodule replaced by tracked file with the same name\n> +# - tracked file replaced by submodule\n\nThese should work without trouble only when the paths involved in\nthe operation in the working tree are clean, right?  Just double\nchecking.  If they are dirty we should expect a failure, instead of\nsilent loss of information.\n\n> +# New test cases\n> +# - Removing a submodule with a git directory absorbs the submodules\n> +#   git directory first into the superproject.\n> +\n> +test_submodule_switch_recursing () {\n> +\tcommand=\"$1\"\n\nThe dq-pair is not strictly needed on the RHS of the assignment, but\nit is a good way to signal that we considered that we might receive\nan argument with $IFS in it...\n\n> +\t######################### Appearing submodule #########################\n> +\t# Switching to a commit letting a submodule appear checks it out ...\n> +\ttest_expect_success \"$command: added submodule is checked out\" '\n> +\t\tprolog &&\n> +\t\treset_work_tree_to_interested no_submodule &&\n> +\t\t(\n> +\t\t\tcd submodule_update &&\n> +\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n> +\t\t\t$command add_sub1 &&\n\n... and after doing so, not quoting $command here signals that we\nexpect command line splitting to happen.  Am I reading it correctly?\nWithout an actual caller appearing in this step, it is rather hard\nto judge.\n\n> +\t\t\ttest_superproject_content origin/add_sub1 &&\n> +\t\t\ttest_submodule_content sub1 origin/add_sub1\n> +\t\t)\n> ...\n> +\t# ... but an ignored file is fine.\n> +\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n> +\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n> +\t\tprolog &&\n> +\t\treset_work_tree_to_interested no_submodule &&\n> +\t\t(\n> +\t\t\tcd submodule_update &&\n> +\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n> +\t\t\t: >sub1 &&\n> +\t\t\techo sub1 > .git/info/exclude\n\n    \">.git/info/exclude\"\n\n> +\t\t\t$command add_sub1 &&\n> +\t\t\ttest_superproject_content origin/add_sub1 &&\n> +\t\t\ttest_submodule_content sub1 origin/add_sub1\n> +\t\t)\n> +\t'\n\n"},{"id":"311803","messageId":"xmqqh93tx3f3.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-6-sbeller@google.com","subject":"Re: [PATCH 05/15] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T20:54:08Z","receivedAt":"2017-02-16T20:54:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> In a later patch we'll use connect_work_tree_and_git_dir when the\n> directory for the gitlink file doesn't exist yet. Safely create\n> the directory first.\n>\n> One of the two users of 'connect_work_tree_and_git_dir' already checked\n> for the directory being there, so we can loose that check.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  dir.c       | 32 +++++++++++++++++++++-----------\n>  submodule.c | 11 ++---------\n>  2 files changed, 23 insertions(+), 20 deletions(-)\n>\n> diff --git a/dir.c b/dir.c\n> index 4541f9e146..6f52af7abb 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n>  /* Update gitfile and core.worktree setting to connect work tree and git dir */\n>  void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n>  {\n> -\tstruct strbuf file_name = STRBUF_INIT;\n> +\tstruct strbuf gitfile_sb = STRBUF_INIT;\n> +\tstruct strbuf cfg_sb = STRBUF_INIT;\n>  \tstruct strbuf rel_path = STRBUF_INIT;\n> -\tchar *git_dir = real_pathdup(git_dir_);\n> -\tchar *work_tree = real_pathdup(work_tree_);\n> +\tchar *git_dir, *work_tree;\n>  \n> -\t/* Update gitfile */\n> -\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n> -\twrite_file(file_name.buf, \"gitdir: %s\",\n> -\t\t   relative_path(git_dir, work_tree, &rel_path));\n> +\t/* Prepare .git file */\n> +\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n> +\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n> +\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n> +\n> +\t/* Prepare config file */\n> +\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n> +\tif (safe_create_leading_directories_const(cfg_sb.buf))\n> +\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n>  \n> +\tgit_dir = real_pathdup(git_dir_);\n> +\twork_tree = real_pathdup(work_tree_);\n> +\n> +\t/* Write .git file */\n> +\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n> +\t\t   relative_path(git_dir, work_tree, &rel_path));\n\nThe above does somewhat more than advertised and was a bit hard to\ngrok.  Initially I thought the reason why pathdup()s were delayed\nwas perhaps because you pathdup() something potentially different\nfrom the given parameter to the function (i.e. new code before\npathdup() may tweak what is pathdup()ed).\n\nBut that is not what is happening.  I suspect that you did this to\navoid leaking allocated memory when the code calls die().\n\nIf the code was written like so from the beginning, I do not see a\nreason to move the pathdup() up to deliberately make it leak [*1*].\nBut as a part of this change, I found it distracting and getting in\nthe way of understanding the change.  If you really care, it is\nnicer to do it to reviewers as a separate preparatory clean-up step,\nor follow-up standalone clean-up patch after the series settles.\n\nThe comment \"prepare config file\" was misleading; it is preparing\nthe place the config file will be created (same for \"prepare .git\nfile\").\n\nIt is a good thing to do to make sure git_dir_ exists and it should\nbe mentioned in the log message.  Adding \"Do the same for the place\nwhere the per-repo config file is created\". at the end of the first\nparagraph should be sufficient.\n\nThanks.\n\n\n[Footnote]\n\n*1* it is arguable a small piece of unfreed memory before exit() or\n    die() is worth called \"leak\", though.\n"},{"id":"311805","messageId":"xmqqd1ehx30c.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-9-sbeller@google.com","subject":"Re: [PATCH 08/15] submodules: introduce check to see whether to touch a submodule","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T21:02:59Z","receivedAt":"2017-02-16T21:03:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> In later patches we introduce the --recurse-submodule flag for commands\n> that modify the working directory, e.g. git-checkout.\n>\n> It is potentially expensive to check if a submodule needs an update,\n> because a common theme to interact with submodules is to spawn a child\n> process for each interaction.\n>\n> So let's introduce a function that checks if a submodule needs\n> to be checked for an update before attempting the update.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule.c | 27 +++++++++++++++++++++++++++\n>  submodule.h | 13 +++++++++++++\n>  2 files changed, 40 insertions(+)\n>\n> diff --git a/submodule.c b/submodule.c\n> index 591f4a694e..2a37e03420 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -548,6 +548,33 @@ void set_config_update_recurse_submodules(int value)\n>  \tconfig_update_recurse_submodules = value;\n>  }\n>  \n> +int touch_submodules_in_worktree(void)\n> +{\n> +\t/*\n> +\t * Update can't be \"none\", \"merge\" or \"rebase\",\n> +\t * treat any value as OFF, except an explicit ON.\n> +\t */\n> +\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n> +}\n\nI somehow sense a somewhat misnamed function.\n\n> +int is_active_submodule_with_strategy(const struct cache_entry *ce,\n> +\t\t\t\t      enum submodule_update_type strategy)\n> +{\n> +\tconst struct submodule *sub;\n> +\n> +\tif (!S_ISGITLINK(ce->ce_mode))\n> +\t\treturn 0;\n> +\n> +\tif (!touch_submodules_in_worktree())\n> +\t\treturn 0;\n\nReading this caller alone, it is totally unclear what this !touch is\nabout.  \"We try to touch it by calling this function, and if the\nfunction successfullly touches it, we return without doing anything\nelse?\"\n\nWould it help to avoid confusion, if the helper function is named to\nbe clearly a boolean?  should_update_submodules_in_worktree() or\nsomething along those lines?\n\n> +\tsub = submodule_from_path(null_sha1, ce->name);\n> +\tif (!sub)\n> +\t\treturn 0;\n> +\n> +\treturn sub->update_strategy.type == strategy;\n> +}\n\nI am not sure if this is a good API design; if it were \"static int\"\ncontained inside the module I wouldn't care, but wouldn't it be more\nnatural for the caller of this function to say\n\n\tif (get_submodule_update_strategy(ce) == STRATEGY_I_WANT)\n\t\tdo something;\n\telse\n\t\tdo something else;\n\nrather than forced to say:\n\n\tif (is_active_submodule_with_strategy(ce, STRATEGY_I_WANT))\n\t\tdo something;\n\telse\n\t\tdo something else;\n\nno?  The caller can easily be extended to\n\n\tswitch (get_submodule_update_strategy(ce)) {\n        case STRATEGY_I_WANT:\n\tcase STRATEGY_I_TOLERATE:\n\t\tdo something; \n\t\tbreak;\n\tdefault:\n\t\tdo something else;\n\t\tbreak;\n\t}                \n\nif the function does not insist taking a single allowed strategy and\nreturn just yes/no as its answer.\n"},{"id":"311806","messageId":"xmqq7f4px2fi.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-11-sbeller@google.com","subject":"Re: [PATCH 10/15] update submodules: add submodule_go_from_to","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T21:15:29Z","receivedAt":"2017-02-16T21:15:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n[administrivia: I've been seeing \"unlisted-recipients:; (no To-header on input)\"\nfor all of your recent patches.  Can it be corrected on your end, please?]\n\n> In later patches we introduce the options and flag for commands\n> that modify the working directory, e.g. git-checkout.\n>\n> This piece of code will be used universally for\n> all these working tree modifications as it\n> * supports dry run to answer the question:\n>   \"Is it safe to change the submodule to this new state?\"\n>   e.g. is it overwriting untracked files or are there local\n>   changes that would be overwritten?\n> * supports a force flag that can be used for resetting\n>   the tree.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n>  submodule.h |   5 ++\n>  2 files changed, 156 insertions(+)\n>\n> diff --git a/submodule.c b/submodule.c\n> index b262c5b0ad..84cc62f3bb 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1250,6 +1250,157 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n>  \treturn ret;\n>  }\n>  \n> +static int submodule_has_dirty_index(const struct submodule *sub)\n> +{\n> +\tssize_t len;\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tstruct strbuf buf = STRBUF_INIT;\n> +\tint ret = 0;\n> +\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\targv_array_pushl(&cp.args, \"diff-index\", \"--cached\", \"HEAD\", NULL);\n\nWe'd want to use the QUICK optimization here, I suspect.  This\ncaller does not need to (or want to) learn which exact paths are\nmodified, right?\n\n> +void submodule_clean_index(const char *path)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n> +\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n> +\n> +\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n> +\n> +\tif (run_command(&cp))\n> +\t\tdie(\"could not clean submodule index\");\n> +}\n\nDo s/clean/reset/ everywhere above.\n\n> +/**\n> + * Moves a submodule at a given path from a given head to another new head.\n> + * For edge cases (a submodule coming into existence or removing a submodule)\n> + * pass NULL for old or new respectively.\n> + *\n> + * TODO: move dryrun and forced to flags.\n\nThe reason why this seeingly trivial thing is left as TODO is...???\n\n> + */\n> +int submodule_go_from_to(const char *path,\n> +\t\t\t const char *old,\n> +\t\t\t const char *new,\n> +\t\t\t int dry_run,\n> +\t\t\t int force)\n> +{\n\ngo-from-to does not tell me what it does, but my cursory read of the\nbody of the function tells me that this is doing a checkout of a\nbranch in the submodule?  The operation in builtin/checkout.c that\nconceptually correspond to this is called switch_branches(), I\nthink, so perhaps submodule_switch_branches() is a better name?\n\n> +\tint ret = 0;\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tconst struct submodule *sub;\n> +\n> +\tsub = submodule_from_path(null_sha1, path);\n> +\n> +\tif (!sub)\n> +\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n> +\n> +\tif (!dry_run) {\n> +\t\tif (old) {\n> +\t\t\tif (!submodule_uses_gitfile(path))\n> +\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n> +\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n> +\t\t} else {\n> +\t\t\tstruct strbuf sb = STRBUF_INIT;\n> +\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n> +\t\t\t\t    get_git_common_dir(), sub->name);\n> +\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n> +\t\t\tstrbuf_release(&sb);\n> +\n> +\t\t\t/* make sure the index is clean as well */\n> +\t\t\tsubmodule_clean_index(path);\n> +\t\t}\n> +\t}\n> +\n> +\tif (old && !force) {\n> +\t\t/* Check if the submodule has a dirty index. */\n> +\t\tif (submodule_has_dirty_index(sub)) {\n> +\t\t\t/* print a thing here? */\n> +\t\t\treturn -1;\n> +\t\t}\n\nIsn't it too late to do this here?  You already reset the index\nin the submodule, no?\n\nIs the idea that changes that are only in the submodule's working\ntree are noticed by later \"read-tree -u -m\" down there?  Not\ncomplaining but trying to understand.\n\n> +\t}\n> +\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n> +\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n> +\n> +\tif (dry_run)\n> +\t\targv_array_push(&cp.args, \"-n\");\n> +\telse\n> +\t\targv_array_push(&cp.args, \"-u\");\n> +\n> +\tif (force)\n> +\t\targv_array_push(&cp.args, \"--reset\");\n> +\telse\n> +\t\targv_array_push(&cp.args, \"-m\");\n> +\n> +\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n> +\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n> +\n> +\tif (run_command(&cp)) {\n> +\t\tret = -1;\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (!dry_run) {\n> +\t\tif (new) {\n> +\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n> +\t\t\t/* also set the HEAD accordingly */\n> +\t\t\tcp1.git_cmd = 1;\n> +\t\t\tcp1.no_stdin = 1;\n> +\t\t\tcp1.dir = path;\n> +\n> +\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n> +\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n> +\n> +\t\t\tif (run_command(&cp1)) {\n> +\t\t\t\tret = -1;\n> +\t\t\t\tgoto out;\n> ...\n"},{"id":"311807","messageId":"CAGZ79kbMmPQ1ZxtdaoFof36g1An-xsAx85etiskirJfY4tGHaQ@mail.gmail.com","threadId":"45144","inReplyTo":"xmqqh93tx3f3.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 05/15] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T21:16:34Z","receivedAt":"2017-02-16T21:16:41Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 16, 2017 at 12:54 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> In a later patch we'll use connect_work_tree_and_git_dir when the\n>> directory for the gitlink file doesn't exist yet. Safely create\n>> the directory first.\n>>\n>> One of the two users of 'connect_work_tree_and_git_dir' already checked\n>> for the directory being there, so we can loose that check.\n>>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>>  dir.c       | 32 +++++++++++++++++++++-----------\n>>  submodule.c | 11 ++---------\n>>  2 files changed, 23 insertions(+), 20 deletions(-)\n>>\n>> diff --git a/dir.c b/dir.c\n>> index 4541f9e146..6f52af7abb 100644\n>> --- a/dir.c\n>> +++ b/dir.c\n>> @@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n>>  /* Update gitfile and core.worktree setting to connect work tree and git dir */\n>>  void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n>>  {\n>> -     struct strbuf file_name = STRBUF_INIT;\n>> +     struct strbuf gitfile_sb = STRBUF_INIT;\n>> +     struct strbuf cfg_sb = STRBUF_INIT;\n>>       struct strbuf rel_path = STRBUF_INIT;\n>> -     char *git_dir = real_pathdup(git_dir_);\n>> -     char *work_tree = real_pathdup(work_tree_);\n>> +     char *git_dir, *work_tree;\n>>\n>> -     /* Update gitfile */\n>> -     strbuf_addf(&file_name, \"%s/.git\", work_tree);\n>> -     write_file(file_name.buf, \"gitdir: %s\",\n>> -                relative_path(git_dir, work_tree, &rel_path));\n>> +     /* Prepare .git file */\n>> +     strbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n>> +     if (safe_create_leading_directories_const(gitfile_sb.buf))\n>> +             die(_(\"could not create directories for %s\"), gitfile_sb.buf);\n>> +\n>> +     /* Prepare config file */\n>> +     strbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n>> +     if (safe_create_leading_directories_const(cfg_sb.buf))\n>> +             die(_(\"could not create directories for %s\"), cfg_sb.buf);\n>>\n>> +     git_dir = real_pathdup(git_dir_);\n>> +     work_tree = real_pathdup(work_tree_);\n>> +\n>> +     /* Write .git file */\n>> +     write_file(gitfile_sb.buf, \"gitdir: %s\",\n>> +                relative_path(git_dir, work_tree, &rel_path));\n>\n> The above does somewhat more than advertised and was a bit hard to\n> grok.  Initially I thought the reason why pathdup()s were delayed\n> was perhaps because you pathdup() something potentially different\n> from the given parameter to the function (i.e. new code before\n> pathdup() may tweak what is pathdup()ed).\n>\n> But that is not what is happening.  I suspect that you did this to\n> avoid leaking allocated memory when the code calls die().\n\nThat is not what is happening, either.\nAFAICT real_pathdup resolves a path to its real path. But the path\n*must* exist already (except the very last part, i.e. the file name).\nSo the SCLD must come before the real_pathdup, which has to come\nbefore its consumers, which are the relative_path calls for write_file\nas well  as git_config_set_in_file.\n\nSo structurally we need to:\n\n1a) construct $GIT_DIR/modules/<name>/config\n1b) SCLD 1a)\n1c) get absolute path of 1a)\n\n2a) construct <submodule path>/.git file\n2b) SCLD 2a)\n2c) get absolute path of 2a)\n\n3) compute relative path of 1c and 2c\n    both ways, write to appropriate places.\n\nI chose to structure it as:\n\n1a\n1b\n2a\n2b\n\n1c\n2c\n3\n\nbecause the a/b steps seemed very related and the order is valid.\n\n\n>\n> If the code was written like so from the beginning, I do not see a\n> reason to move the pathdup() up to deliberately make it leak [*1*].\n> But as a part of this change, I found it distracting and getting in\n> the way of understanding the change.  If you really care, it is\n> nicer to do it to reviewers as a separate preparatory clean-up step,\n> or follow-up standalone clean-up patch after the series settles.\n\nI considered doing it in multiple patches as we have different things\ngoing on here:\n1) as the commit claims have SCLD in connect_work_tree_and_git_dir\n2) keep the dependencies of SCLD and real_pathdup in order.  C.f.\n  the deleted lines of code in submodule.c\n\n> It is a good thing to do to make sure git_dir_ exists and it should\n> be mentioned in the log message.  Adding \"Do the same for the place\n> where the per-repo config file is created\". at the end of the first\n> paragraph should be sufficient.\n>\n\nwill do.\n\nThanks for thorough review!\nStefan\n"},{"id":"311808","messageId":"xmqq37fdx282.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-12-sbeller@google.com","subject":"Re: [PATCH 11/15] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T21:19:57Z","receivedAt":"2017-02-16T21:20:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> The check (which uses the old oid) is yet to be implemented, but this part\n> is just a refactor, so it can go separately first.\n\nIf this didn't pass an unused parameter, then the change is just a\nrefactor, and I do think such a \"just a refactor\" can be a good step\non its own to keep the future step to manageable complexity.\n\nWith an unused parameter being passed, I do not think it is a good\nlogical single step anymore, though.\n\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  unpack-trees.c | 17 ++++++++++-------\n>  1 file changed, 10 insertions(+), 7 deletions(-)\n>\n> diff --git a/unpack-trees.c b/unpack-trees.c\n> index 3a8ee19fe8..616a0ae4b2 100644\n> --- a/unpack-trees.c\n> +++ b/unpack-trees.c\n> @@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n>   * Currently, git does not checkout subprojects during a superproject\n>   * checkout, so it is not going to overwrite anything.\n>   */\n> -static int verify_clean_submodule(const struct cache_entry *ce,\n> +static int verify_clean_submodule(const char *old_sha1,\n> +\t\t\t\t  const struct cache_entry *ce,\n>  \t\t\t\t  enum unpack_trees_error_types error_type,\n>  \t\t\t\t  struct unpack_trees_options *o)\n>  {\n> @@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n>  \tstruct dir_struct d;\n>  \tchar *pathbuf;\n>  \tint cnt = 0;\n> -\tunsigned char sha1[20];\n>  \n> -\tif (S_ISGITLINK(ce->ce_mode) &&\n> -\t    resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n> -\t\t/* If we are not going to update the submodule, then\n> +\tif (S_ISGITLINK(ce->ce_mode)) {\n> +\t\tunsigned char sha1[20];\n> +\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n> +\t\t/*\n> +\t\t * If we are not going to update the submodule, then\n>  \t\t * we don't care.\n>  \t\t */\n> -\t\tif (!hashcmp(sha1, ce->oid.hash))\n> +\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n>  \t\t\treturn 0;\n> -\t\treturn verify_clean_submodule(ce, error_type, o);\n> +\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n> +\t\t\t\t\t      ce, error_type, o);\n>  \t}\n>  \n>  \t/*\n"},{"id":"311809","messageId":"xmqqy3x5vnh1.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-13-sbeller@google.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T21:23:54Z","receivedAt":"2017-02-16T21:24:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> +\n> +\t/* ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE */\n> +\t\"Submodule '%s' cannot be deleted as it contains untracked files.\",\n\nOK.\n\n> +\tmsgs[ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE] =\n> +\t\t_(\"Submodule '%s' cannot be deleted as it contains untracked files.\");\n\nOK again.\n\n> @@ -240,12 +246,44 @@ static void display_error_msgs(struct unpack_trees_options *o)\n>  \t\tfprintf(stderr, _(\"Aborting\\n\"));\n>  }\n>  \n> +static int submodule_check_from_to(const struct cache_entry *ce, const char *old_id, const char *new_id, struct unpack_trees_options *o)\n> +{\n> +\tif (submodule_go_from_to(ce->name, old_id,\n> +\t\t\t\t new_id, 1, o->reset))\n> +\t\treturn o->gently ? -1 :\n> +\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE, ce->name);\n\nIs potential loss of untracked paths the only reason\nsubmodule_go_from_to() would fail?  I somehow thought that it would\nnot even care about untracked paths but cared deeply about already\nadded changes.\n\n"},{"id":"311810","messageId":"xmqqtw7tvnea.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"CAGZ79kbMmPQ1ZxtdaoFof36g1An-xsAx85etiskirJfY4tGHaQ@mail.gmail.com","subject":"Re: [PATCH 05/15] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T21:25:33Z","receivedAt":"2017-02-16T21:25:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>> The above does somewhat more than advertised and was a bit hard to\n>> grok.  Initially I thought the reason why pathdup()s were delayed\n>> was perhaps because you pathdup() something potentially different\n>> from the given parameter to the function (i.e. new code before\n>> pathdup() may tweak what is pathdup()ed).\n>>\n>> But that is not what is happening.  I suspect that you did this to\n>> avoid leaking allocated memory when the code calls die().\n>\n> That is not what is happening, either.\n\nThat's a good sign that you need a bit more in the log message.\n"},{"id":"311813","messageId":"xmqqpoihvn3b.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-14-sbeller@google.com","subject":"Re: [PATCH 13/15] read-cache: remove_marked_cache_entries to wipe selected submodules.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T21:32:08Z","receivedAt":"2017-02-16T21:32:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\nThe title was ungrokkable to me, but after reading the code I think\nyou are teaching the normal codepath where removal of working tree\nis done to match what is done to the index that submodules are also\nin the working tree and need to be removed when the corresopnding ce\nis removed.\n\nWhich makes sense.\n\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  read-cache.c | 3 +++\n>  1 file changed, 3 insertions(+)\n>\n> diff --git a/read-cache.c b/read-cache.c\n> index 9054369dd0..b78a7f02e3 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -18,6 +18,7 @@\n>  #include \"varint.h\"\n>  #include \"split-index.h\"\n>  #include \"utf8.h\"\n> +#include \"submodule.h\"\n>  \n>  /* Mask for the name length in ce_flags in the on-disk index */\n>  \n> @@ -532,6 +533,8 @@ void remove_marked_cache_entries(struct index_state *istate)\n>  \n>  \tfor (i = j = 0; i < istate->cache_nr; i++) {\n>  \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n> +\t\t\tif (is_active_submodule_with_strategy(ce_array[i], SM_UPDATE_UNSPECIFIED))\n> +\t\t\t\tsubmodule_go_from_to(ce_array[i]->name, \"HEAD\", NULL, 0, 1);\n>  \t\t\tremove_name_hash(istate, ce_array[i]);\n>  \t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n>  \t\t}\n"},{"id":"311814","messageId":"CAGZ79kYqa4jX-q_GHLyCE80VipBOHq48RZ_f=bGAw1CkyyLCBw@mail.gmail.com","threadId":"45144","inReplyTo":"xmqq7f4px2fi.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 10/15] update submodules: add submodule_go_from_to","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T21:33:44Z","receivedAt":"2017-02-16T21:33:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 16, 2017 at 1:15 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> [administrivia: I've been seeing \"unlisted-recipients:; (no To-header on input)\"\n> for all of your recent patches.  Can it be corrected on your end, please?]\n\nI cc'd everyone and had no to field, actually. Maybe git-send-email should\ncomplain more loudly when I am holding it wrong.\n\n>> +     cp.git_cmd = 1;\n>> +     argv_array_pushl(&cp.args, \"diff-index\", \"--cached\", \"HEAD\", NULL);\n>\n> We'd want to use the QUICK optimization here, I suspect.  This\n> caller does not need to (or want to) learn which exact paths are\n> modified, right?\n\nok\n\n>> +     if (run_command(&cp))\n>> +             die(\"could not clean submodule index\");\n>> +}\n>\n> Do s/clean/reset/ everywhere above.\n\nok\n\n>\n>> +/**\n>> + * Moves a submodule at a given path from a given head to another new head.\n>> + * For edge cases (a submodule coming into existence or removing a submodule)\n>> + * pass NULL for old or new respectively.\n>> + *\n>> + * TODO: move dryrun and forced to flags.\n>\n> The reason why this seeingly trivial thing is left as TODO is...???\n\nwill do. The reason was to first get the grand design right before\nspending more time in the details.\n\n>\n>> + */\n>> +int submodule_go_from_to(const char *path,\n>> +                      const char *old,\n>> +                      const char *new,\n>> +                      int dry_run,\n>> +                      int force)\n>> +{\n>\n> go-from-to does not tell me what it does, but my cursory read of the\n> body of the function tells me that this is doing a checkout of a\n> branch in the submodule?  The operation in builtin/checkout.c that\n> conceptually correspond to this is called switch_branches(), I\n> think, so perhaps submodule_switch_branches() is a better name?\n\nWell as of now all submodule operations (submodule update mostly)\nend up with detached HEADs in the submodule. So it is rather\ngoing from one state (sha1) to another given sha1.\n\nI would rather compare it to checkout_entry/write_entry in entry.c\nexcept that there are more things to go wrong. A single file has no\nnotion of its own index or dirtiness.\n\n>\n>> +     int ret = 0;\n>> +     struct child_process cp = CHILD_PROCESS_INIT;\n>> +     const struct submodule *sub;\n>> +\n>> +     sub = submodule_from_path(null_sha1, path);\n>> +\n>> +     if (!sub)\n>> +             die(\"BUG: could not get submodule information for '%s'\", path);\n>> +\n>> +     if (!dry_run) {\n>> +             if (old) {\n>> +                     if (!submodule_uses_gitfile(path))\n>> +                             absorb_git_dir_into_superproject(\"\", path,\n>> +                                     ABSORB_GITDIR_RECURSE_SUBMODULES);\n>> +             } else {\n>> +                     struct strbuf sb = STRBUF_INIT;\n>> +                     strbuf_addf(&sb, \"%s/modules/%s\",\n>> +                                 get_git_common_dir(), sub->name);\n>> +                     connect_work_tree_and_git_dir(path, sb.buf);\n>> +                     strbuf_release(&sb);\n>> +\n>> +                     /* make sure the index is clean as well */\n>> +                     submodule_clean_index(path);\n>> +             }\n>> +     }\n>> +\n>> +     if (old && !force) {\n>> +             /* Check if the submodule has a dirty index. */\n>> +             if (submodule_has_dirty_index(sub)) {\n>> +                     /* print a thing here? */\n>> +                     return -1;\n>> +             }\n>\n> Isn't it too late to do this here?  You already reset the index\n> in the submodule, no?\n\nYes this is confusing.\nWe run this function first as a dry_run, and in a second pass\nas a real run. So the order inside the function is confusing\nas we would run this first in the dry run.\n\n>\n> Is the idea that changes that are only in the submodule's working\n> tree are noticed by later \"read-tree -u -m\" down there?  Not\n> complaining but trying to understand.\n\nI think (as a first step) we only want to allow a clean index in\nsubmodules, as then we have to implement less cases at first.\n"},{"id":"311816","messageId":"xmqqlgt5vlse.fsf@gitster.mtv.corp.google.com","threadId":"45144","inReplyTo":"20170216003811.18273-1-sbeller@google.com","subject":"Re: [RFCv4 PATCH 00/14] Checkout aware of Submodules!","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-16T22:00:17Z","receivedAt":"2017-02-16T22:00:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Integrate updating the submodules into git checkout,...\n\nIt was more or less a pleasant read, once I decided to pretend that\nI were a machine who uses identifiers only to identify locations in\nthe program ;-) IOW, for human consumption, the new names introduced\nwere sometimes quite confusing and went against helping understanding.\n\nI saw a few places where logic looked somewhat iffy, which I sent\nseparate comments on; I may spot more if the code used more\nunderstandable names and calling conventions, but that is OK.  It is\nan expected part of an iterative process.  \n\nI can feel that this topic is getting closer to where we eventually\nwant to go.\n\nThanks.\n"},{"id":"311817","messageId":"CA+P7+xr=2=ZvFvSjMMKsS_T9uPzVW3K3jtaCjc4DSaxNGgozOQ@mail.gmail.com","threadId":"45144","inReplyTo":"20170216003811.18273-9-sbeller@google.com","subject":"Re: [PATCH 08/15] submodules: introduce check to see whether to touch a submodule","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-16T22:34:31Z","receivedAt":"2017-02-16T22:35:02Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n> +int touch_submodules_in_worktree(void)\n> +{\n> +       /*\n> +        * Update can't be \"none\", \"merge\" or \"rebase\",\n> +        * treat any value as OFF, except an explicit ON.\n> +        */\n> +       return config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n> +}\n> +\n\nThis function doesn't and the comment don't make sense to me. What do\nyou mean update can't be \"none\", \"merge\", or \"rebase\"? I'm thinking\nthis means that the update_recurse_submodules checks whether it's ok\nfor doing recursive update on submodules but only when the update type\nis checkout? This appears to be connected directly to the previous\npatch that reads the config value somehow. This is pretty convoluted\nto me, and took me quite a while to understand. Is it possible to make\nthis more clear in the comments or in the name?\n"},{"id":"311819","messageId":"CA+P7+xpWqiWeMOMauYgtVaiUngd73_dUqRxyWcp0QPHvgOe+vg@mail.gmail.com","threadId":"45144","inReplyTo":"xmqqlgt5vlse.fsf@gitster.mtv.corp.google.com","subject":"Re: [RFCv4 PATCH 00/14] Checkout aware of Submodules!","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-16T22:54:50Z","receivedAt":"2017-02-16T22:55:18Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Feb 16, 2017 at 2:00 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> Integrate updating the submodules into git checkout,...\n>\n> It was more or less a pleasant read, once I decided to pretend that\n> I were a machine who uses identifiers only to identify locations in\n> the program ;-) IOW, for human consumption, the new names introduced\n> were sometimes quite confusing and went against helping understanding.\n>\n\nBased on my cursory reading, I agree. I had trouble understanding how\nthe process worked because the names were somewhat confusing. They\nstarted to make some sense as I read more. I think v4 had better names\nthan v3, but they were still somewhat confusing to me.\n\nRegards,\nJake\n"},{"id":"311821","messageId":"CAGZ79kYQf2SL-RVETv8=6NaZHhub2kQmKevK32X_xfFkr_0yyA@mail.gmail.com","threadId":"45144","inReplyTo":"CA+P7+xpWqiWeMOMauYgtVaiUngd73_dUqRxyWcp0QPHvgOe+vg@mail.gmail.com","subject":"Re: [RFCv4 PATCH 00/14] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-16T22:56:39Z","receivedAt":"2017-02-16T22:56:45Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 16, 2017 at 2:54 PM, Jacob Keller <jacob.keller@gmail.com> wrote:\n> On Thu, Feb 16, 2017 at 2:00 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Stefan Beller <sbeller@google.com> writes:\n>>\n>>> Integrate updating the submodules into git checkout,...\n>>\n>> It was more or less a pleasant read, once I decided to pretend that\n>> I were a machine who uses identifiers only to identify locations in\n>> the program ;-) IOW, for human consumption, the new names introduced\n>> were sometimes quite confusing and went against helping understanding.\n>>\n>\n> Based on my cursory reading, I agree. I had trouble understanding how\n> the process worked because the names were somewhat confusing. They\n> started to make some sense as I read more. I think v4 had better names\n> than v3, but they were still somewhat confusing to me.\n>\n\nNow if only you could tell me what names were better to understand. ;)\nI'll reply to the individual patch remarks and hopefully there we find\ngood names for these functions.\n\nThanks,\nStefan\n"},{"id":"311825","messageId":"CA+P7+xp-XbYRp-uxDgSHz9y+OrSQ-pd0t-UsQyT843-t4ArPeA@mail.gmail.com","threadId":"45144","inReplyTo":"CAGZ79kYQf2SL-RVETv8=6NaZHhub2kQmKevK32X_xfFkr_0yyA@mail.gmail.com","subject":"Re: [RFCv4 PATCH 00/14] Checkout aware of Submodules!","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-16T23:27:02Z","receivedAt":"2017-02-16T23:27:29Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Thu, Feb 16, 2017 at 2:56 PM, Stefan Beller <sbeller@google.com> wrote:\n> On Thu, Feb 16, 2017 at 2:54 PM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>> On Thu, Feb 16, 2017 at 2:00 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>>> Stefan Beller <sbeller@google.com> writes:\n>>>\n>>>> Integrate updating the submodules into git checkout,...\n>>>\n>>> It was more or less a pleasant read, once I decided to pretend that\n>>> I were a machine who uses identifiers only to identify locations in\n>>> the program ;-) IOW, for human consumption, the new names introduced\n>>> were sometimes quite confusing and went against helping understanding.\n>>>\n>>\n>> Based on my cursory reading, I agree. I had trouble understanding how\n>> the process worked because the names were somewhat confusing. They\n>> started to make some sense as I read more. I think v4 had better names\n>> than v3, but they were still somewhat confusing to me.\n>>\n>\n> Now if only you could tell me what names were better to understand. ;)\n> I'll reply to the individual patch remarks and hopefully there we find\n> good names for these functions.\n>\n> Thanks,\n> Stefan\n\nI'll try to read it again and see if I think of anything.\n\nThanks,\nJake\n"},{"id":"311917","messageId":"CA+P7+xozip8TuvyUe9vAPYLAg=QFieExhOyR7a0pgGFhiuO3jw@mail.gmail.com","threadId":"45144","inReplyTo":"20170216003811.18273-7-sbeller@google.com","subject":"Re: [PATCH 06/15] update submodules: add submodule config parsing","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-17T18:24:51Z","receivedAt":"2017-02-17T18:25:18Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n> Similar to b33a15b08 (push: add recurseSubmodules config option,\n> 2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\n> fetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\n> that is later used to parse whether we are interested in updating\n> submodules.\n>\n> We need the `die_on_error` parameter to be able to call this parsing\n> function for the config file as well, which if incorrect lets Git die.\n>\n> As we're just touching the header file, also mark all functions extern.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule-config.c | 22 ++++++++++++++++++++++\n>  submodule-config.h | 17 +++++++++--------\n>  2 files changed, 31 insertions(+), 8 deletions(-)\n>\n> diff --git a/submodule-config.c b/submodule-config.c\n> index 93453909cf..93f01c4378 100644\n> --- a/submodule-config.c\n> +++ b/submodule-config.c\n> @@ -234,6 +234,28 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n>         return parse_fetch_recurse(opt, arg, 1);\n>  }\n>\n> +static int parse_update_recurse(const char *opt, const char *arg,\n> +                               int die_on_error)\n> +{\n> +       switch (git_config_maybe_bool(opt, arg)) {\n> +       case 1:\n> +               return RECURSE_SUBMODULES_ON;\n> +       case 0:\n> +               return RECURSE_SUBMODULES_OFF;\n> +       default:\n> +               if (!strcmp(arg, \"checkout\"))\n> +                       return RECURSE_SUBMODULES_ON;\n> +               if (die_on_error)\n> +                       die(\"bad %s argument: %s\", opt, arg);\n> +               return RECURSE_SUBMODULES_ERROR;\n> +       }\n> +}\n\nOk so this function here reads a recurse submodules parameter which is\na boolean or it can be set to the word \"checkout\"? Why does checkout\nneed its own value separate from true? Just so that we have a synonym?\nor so that we can expand on it in the future?\n"},{"id":"311921","messageId":"CA+P7+xqpxTt7KibOrVr2ekjLy6Hva4KJ6nCaN22j-Qpspky3aQ@mail.gmail.com","threadId":"45144","inReplyTo":"20170216003811.18273-9-sbeller@google.com","subject":"Re: [PATCH 08/15] submodules: introduce check to see whether to touch a submodule","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-17T18:36:20Z","receivedAt":"2017-02-17T18:36:53Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n> In later patches we introduce the --recurse-submodule flag for commands\n> that modify the working directory, e.g. git-checkout.\n>\n> It is potentially expensive to check if a submodule needs an update,\n> because a common theme to interact with submodules is to spawn a child\n> process for each interaction.\n>\n> So let's introduce a function that checks if a submodule needs\n> to be checked for an update before attempting the update.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule.c | 27 +++++++++++++++++++++++++++\n>  submodule.h | 13 +++++++++++++\n>  2 files changed, 40 insertions(+)\n>\n> diff --git a/submodule.c b/submodule.c\n> index 591f4a694e..2a37e03420 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -548,6 +548,33 @@ void set_config_update_recurse_submodules(int value)\n>         config_update_recurse_submodules = value;\n>  }\n>\n> +int touch_submodules_in_worktree(void)\n> +{\n> +       /*\n> +        * Update can't be \"none\", \"merge\" or \"rebase\",\n> +        * treat any value as OFF, except an explicit ON.\n> +        */\n> +       return config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n> +}\n\nOk, so here, we're just checking whether the value is\nRECURSE_SUBMODULES_ON. The comment doesn't make sense to me at all.\nWhat is \"update\" and why \"can't\" it be those values? How is this code\ntreating thise values as OFF exept for an explicit ON?\n\nAt first I thought this comment was related to check in the previous\npatch. I think I see the patch is \"recurse submodules = true\" or\n\"recurse submodules = checkout\" as a specific strategy? Ie:\nrecurse-submodules=checkout\" means \"recurse into submodules and update\nthem using checkout strategy?\n\nOk this starts to make a bit more sense. However, it's still somewhat\nconfusing to me.\n\nMaybe this should be called something like\nrecurse_into_submodules_in_worktree() though that is pretty verbose.\n\nI'm not sure I have a good name really. But I wonder why we need this\nin the first place. Basically, we set RECURSE_SUBMODULES_* value which\ncould be OFF, ON, or even future extensions of CHECKOUT, REBASE,\nMERGE, etc?\n\nBut shouldn't we just return the mode, and let the later code decide\n\"oh. actually I don't support this mode\". For now, obviously we\nwouldn't set any of the new modes yet.\n\n> +\n> +int is_active_submodule_with_strategy(const struct cache_entry *ce,\n> +                                     enum submodule_update_type strategy)\n> +{\n> +       const struct submodule *sub;\n> +\n> +       if (!S_ISGITLINK(ce->ce_mode))\n> +               return 0;\n> +\n> +       if (!touch_submodules_in_worktree())\n> +               return 0;\n> +\n> +       sub = submodule_from_path(null_sha1, ce->name);\n> +       if (!sub)\n> +               return 0;\n> +\n> +       return sub->update_strategy.type == strategy;\n> +}\n> +\n\nI liked Junio's suggestion where this just returns the strategy that\nit can use, or 0 if it shouldn't be updated. Then, other code just\ndecides: Yes, I can use this strategy or no I cannot.\n\nShould this be tied in with the strategy used by the\nrecurse_submodules above? ie: when/if we support recursing using other\nstrategies, shouldn't we make these match here, so that if the recurse\nmode is \"checkout\" we don't checkout a submodule that was configured\nto be rebased? Or do you want to blindly apply checkout to all\nsubmodules even if they don't have strategy?\n\nI assume you do not, since you check here with passing a strategy\nvalue, and then see if it matches.\n\nthis could be named something like:\n\nget_active_submodule_strategy() and return the strategy directly. It\nwould then return 0 in any case where it shouldn't be updated. Later\ncode can then check the strategy.\n\n>  static int has_remote(const char *refname, const struct object_id *oid,\n>                       int flags, void *cb_data)\n>  {\n> diff --git a/submodule.h b/submodule.h\n> index b4e60c08d2..46d9f0f293 100644\n> --- a/submodule.h\n> +++ b/submodule.h\n> @@ -65,6 +65,19 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n>                 const struct diff_options *opt);\n>  extern void set_config_fetch_recurse_submodules(int value);\n>  extern void set_config_update_recurse_submodules(int value);\n> +\n> +/*\n> + * Traditionally Git ignored changes made for submodules.\n> + * This function checks if we are interested in the given submodule\n> + * for any kind of operation.\n\nThis comment seems a bit weird.\n\n> + */\n> +extern int touch_submodules_in_worktree(void);\n> +/*\n> + * Check if the given ce entry is a submodule with the given update\n> + * strategy configured.\n\nI like Junio's suggestion of this \"getting the current configured\nstrategy for a submodule. When we aren't set to recurse into\nsubmodules we (obviously) return that we have no strategy since we're\nnot going to update it at all.\n\n> + */\n> +extern int is_active_submodule_with_strategy(const struct cache_entry *ce,\n> +                                            enum submodule_update_type strategy);\n>  extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n>  extern int fetch_populated_submodules(const struct argv_array *options,\n>                                const char *prefix, int command_line_option,\n> --\n> 2.12.0.rc1.16.ge4278d41a0.dirty\n>\n"},{"id":"311922","messageId":"CA+P7+xrwxb9G3QVOascSZqnBa_vGEx95nv0jN9USmBqB5Q_Mzw@mail.gmail.com","threadId":"45144","inReplyTo":"20170216003811.18273-13-sbeller@google.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-17T18:42:06Z","receivedAt":"2017-02-17T18:42:36Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n> +       if (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n\nHere, and in other cases where we use\nis_active_submodule_with_strategy(), why do we only ever check\nSM_UPDATE_UNSPECIFIED? It seems really weird that we're only going to\ncheck submodules who's strategy is unspecified, when that defaults to\ncheckout if I recall correctly? Shouldn't we check both? This applies\nto pretty much everywhere that you call this function that I noticed,\nwhich is why I removed the context.\n\nThanks,\nJake\n"},{"id":"312218","messageId":"CAGZ79kZyFfC9Xx-p8dpoAFFpz48BqmftpMonuxeiKg1sV68iuQ@mail.gmail.com","threadId":"45144","inReplyTo":"CA+P7+xozip8TuvyUe9vAPYLAg=QFieExhOyR7a0pgGFhiuO3jw@mail.gmail.com","subject":"Re: [PATCH 06/15] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-21T19:42:23Z","receivedAt":"2017-02-21T19:42:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Feb 17, 2017 at 10:24 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>\n> Ok so this function here reads a recurse submodules parameter which is\n> a boolean or it can be set to the word \"checkout\"? Why does checkout\n> need its own value separate from true? Just so that we have a synonym?\n> or so that we can expand on it in the future?\n\nI think eventually we want all the commands that touch the worktree to\nbe able to cope with submodules.\n\n  Now what should e.g. git-revert --recurse-submodules do?\n  yes == \"checkout\" means we'd revert the superproject commit and\n  if that commit changed any submodule pointers we'd just \"checkout\"\n  those states in the submodule.\n\n  For revert you could also imagine to have\n  git-revert --recurse-submodules=revert-in-subs\n  that would not repoint the submodule pointer to the old state, but\n  would try to revert $OLD..$NEW in the submodule and take the newly\n  reverted state as the new submodule pointer.\n\nAs I want to focus on checkout first, I went with \"yes == checkout\"\nhere (or rather the other way round).\n"},{"id":"312224","messageId":"CA+P7+xoVOxBgubfZP1uUnbXvbVNrCN_w2je7_tcxJ97pXzL1VQ@mail.gmail.com","threadId":"45144","inReplyTo":"CAGZ79kZyFfC9Xx-p8dpoAFFpz48BqmftpMonuxeiKg1sV68iuQ@mail.gmail.com","subject":"Re: [PATCH 06/15] update submodules: add submodule config parsing","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-21T20:48:39Z","receivedAt":"2017-02-21T20:49:06Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Tue, Feb 21, 2017 at 11:42 AM, Stefan Beller <sbeller@google.com> wrote:\n> On Fri, Feb 17, 2017 at 10:24 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>>\n>> Ok so this function here reads a recurse submodules parameter which is\n>> a boolean or it can be set to the word \"checkout\"? Why does checkout\n>> need its own value separate from true? Just so that we have a synonym?\n>> or so that we can expand on it in the future?\n>\n> I think eventually we want all the commands that touch the worktree to\n> be able to cope with submodules.\n>\n>   Now what should e.g. git-revert --recurse-submodules do?\n>   yes == \"checkout\" means we'd revert the superproject commit and\n>   if that commit changed any submodule pointers we'd just \"checkout\"\n>   those states in the submodule.\n>\n>   For revert you could also imagine to have\n>   git-revert --recurse-submodules=revert-in-subs\n>   that would not repoint the submodule pointer to the old state, but\n>   would try to revert $OLD..$NEW in the submodule and take the newly\n>   reverted state as the new submodule pointer.\n>\n> As I want to focus on checkout first, I went with \"yes == checkout\"\n> here (or rather the other way round).\n\nOk I understand, but this seems like the variable could eventually\nstart to included more and more complex things? For now, \"checkout\"\nmeans \"when changing submodules prefer to check out contents\" right?\n\nI guess that sort of makes some sense.\n\nThanks,\nJake\n"},{"id":"312225","messageId":"CAGZ79kbCvHkooxwFyhixnGKG4pSp8AX8UjCKr9OJzJm_EJEV5Q@mail.gmail.com","threadId":"45144","inReplyTo":"CA+P7+xqpxTt7KibOrVr2ekjLy6Hva4KJ6nCaN22j-Qpspky3aQ@mail.gmail.com","subject":"Re: [PATCH 08/15] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-21T20:56:50Z","receivedAt":"2017-02-21T20:56:57Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Feb 17, 2017 at 10:36 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n> On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n>> In later patches we introduce the --recurse-submodule flag for commands\n>> that modify the working directory, e.g. git-checkout.\n>>\n>> It is potentially expensive to check if a submodule needs an update,\n>> because a common theme to interact with submodules is to spawn a child\n>> process for each interaction.\n>>\n>> So let's introduce a function that checks if a submodule needs\n>> to be checked for an update before attempting the update.\n>>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>>  submodule.c | 27 +++++++++++++++++++++++++++\n>>  submodule.h | 13 +++++++++++++\n>>  2 files changed, 40 insertions(+)\n>>\n>> diff --git a/submodule.c b/submodule.c\n>> index 591f4a694e..2a37e03420 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -548,6 +548,33 @@ void set_config_update_recurse_submodules(int value)\n>>         config_update_recurse_submodules = value;\n>>  }\n>>\n>> +int touch_submodules_in_worktree(void)\n>> +{\n>> +       /*\n>> +        * Update can't be \"none\", \"merge\" or \"rebase\",\n>> +        * treat any value as OFF, except an explicit ON.\n>> +        */\n>> +       return config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n>> +}\n>\n> Ok, so here, we're just checking whether the value is\n> RECURSE_SUBMODULES_ON. The comment doesn't make sense to me at all.\n\nYes the comment was not updated in the last round of patches and is\nnow out of context.\n\n> What is \"update\" and why \"can't\" it be those values? How is this code\n> treating thise values as OFF exept for an explicit ON?\n\nPlease disregard the comment; I'll remove it in the next reroll.\nThe submodule API is in such a way that\nconfig_update_recurse_submodules\n\n>\n> At first I thought this comment was related to check in the previous\n> patch. I think I see the patch is \"recurse submodules = true\" or\n> \"recurse submodules = checkout\" as a specific strategy? Ie:\n> recurse-submodules=checkout\" means \"recurse into submodules and update\n> them using checkout strategy?\n\nYes that is what I had in mind. See previous comment, in a later series\nwe could extend that to other strategies such as \"revert-in-submodules\"\nfor git-revert or \"rebase\", \"merge\" as we curreently have for\n\"git submodule update\".\n\n> Maybe this should be called something like\n> recurse_into_submodules_in_worktree() though that is pretty verbose.\n\nI like that. (It's less than double the number of characters, so it's\nfine, isn't it?)\nMaybe we can abbreviate worktree by \"wt\" ans \"submodules by subs:\n\n    /* recurse into submodules in the worktree? */\n    int rec_subs_wt;\n\nThat looks short enough to qualify as non-Java.\n\n> I'm not sure I have a good name really. But I wonder why we need this\n> in the first place. Basically, we set RECURSE_SUBMODULES_* value which\n> could be OFF, ON, or even future extensions of CHECKOUT, REBASE,\n> MERGE, etc?\n>\n> But shouldn't we just return the mode, and let the later code decide\n> \"oh. actually I don't support this mode\". For now, obviously we\n> wouldn't set any of the new modes yet.\n\nMh, makes sense. Maybe I tricked myself into premature optimization,\nbecause I'd expect most of the users not caring about submodules, such\nthat we want to have a *really* cheap way of saying \"no, not interesting in\nsubmodules\", which is what this method mainly offers.\n\nJunio also remarked this and the following\n\"is_active_submodule_with_strategy\" to be bad design.\n\nI'll redo those, such that the caller decides what to do with each strategy.\n\n>\n>> +\n>> +int is_active_submodule_with_strategy(const struct cache_entry *ce,\n>> +                                     enum submodule_update_type strategy)\n>> +{\n>> +       const struct submodule *sub;\n>> +\n>> +       if (!S_ISGITLINK(ce->ce_mode))\n>> +               return 0;\n>> +\n>> +       if (!touch_submodules_in_worktree())\n>> +               return 0;\n>> +\n>> +       sub = submodule_from_path(null_sha1, ce->name);\n>> +       if (!sub)\n>> +               return 0;\n>> +\n>> +       return sub->update_strategy.type == strategy;\n>> +}\n>> +\n>\n> I liked Junio's suggestion where this just returns the strategy that\n> it can use, or 0 if it shouldn't be updated. Then, other code just\n> decides: Yes, I can use this strategy or no I cannot.\n>\n> Should this be tied in with the strategy used by the\n> recurse_submodules above? ie: when/if we support recursing using other\n> strategies, shouldn't we make these match here, so that if the recurse\n> mode is \"checkout\" we don't checkout a submodule that was configured\n> to be rebased? Or do you want to blindly apply checkout to all\n> submodules even if they don't have strategy?\n>\n> I assume you do not, since you check here with passing a strategy\n> value, and then see if it matches.\n>\n> this could be named something like:\n>\n> get_active_submodule_strategy() and return the strategy directly. It\n> would then return 0 in any case where it shouldn't be updated. Later\n> code can then check the strategy.\n\n0 is already taken as SM_UPDATE_UNSPECIFIED,\nso maybe we'd introduce a new update command\nSM_UPDATE_IGNORE = -1 or rather use\nSM_UPDATE_NONE instead.\n\n>\n>>  static int has_remote(const char *refname, const struct object_id *oid,\n>>                       int flags, void *cb_data)\n>>  {\n>> diff --git a/submodule.h b/submodule.h\n>> index b4e60c08d2..46d9f0f293 100644\n>> --- a/submodule.h\n>> +++ b/submodule.h\n>> @@ -65,6 +65,19 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n>>                 const struct diff_options *opt);\n>>  extern void set_config_fetch_recurse_submodules(int value);\n>>  extern void set_config_update_recurse_submodules(int value);\n>> +\n>> +/*\n>> + * Traditionally Git ignored changes made for submodules.\n>> + * This function checks if we are interested in the given submodule\n>> + * for any kind of operation.\n>\n> This comment seems a bit weird.\n\ncorrect, I'll reword that.\n\n>\n>> + */\n>> +extern int touch_submodules_in_worktree(void);\n>> +/*\n>> + * Check if the given ce entry is a submodule with the given update\n>> + * strategy configured.\n>\n> I like Junio's suggestion of this \"getting the current configured\n> strategy for a submodule. When we aren't set to recurse into\n> submodules we (obviously) return that we have no strategy since we're\n> not going to update it at all.\n>\n>> + */\n>> +extern int is_active_submodule_with_strategy(const struct cache_entry *ce,\n>> +                                            enum submodule_update_type strategy);\n>>  extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n>>  extern int fetch_populated_submodules(const struct argv_array *options,\n>>                                const char *prefix, int command_line_option,\n>> --\n>> 2.12.0.rc1.16.ge4278d41a0.dirty\n>>\n"},{"id":"312230","messageId":"CAGZ79kahCN2dd9=CzqXJWSJKrkOfvd3HeQ-NORhyP=6B=KCrqg@mail.gmail.com","threadId":"45144","inReplyTo":"CA+P7+xrwxb9G3QVOascSZqnBa_vGEx95nv0jN9USmBqB5Q_Mzw@mail.gmail.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-21T22:16:24Z","receivedAt":"2017-02-21T22:16:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Feb 17, 2017 at 10:42 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n> On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n>> +       if (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n>\n> Here, and in other cases where we use\n> is_active_submodule_with_strategy(), why do we only ever check\n> SM_UPDATE_UNSPECIFIED? It seems really weird that we're only going to\n> check submodules who's strategy is unspecified, when that defaults to\n> checkout if I recall correctly? Shouldn't we check both? This applies\n> to pretty much everywhere that you call this function that I noticed,\n> which is why I removed the context.\n\nI am torn between this.\n\nsubmodule.<name>.update = {rebase, merge, checkout, none !command}\nis currently documented in GIT-CONFIG(1) as\n\n       submodule.<name>.update\n           The default update procedure for a submodule. This variable is\n           populated by git submodule init from the gitmodules(5) file. See\n           description of update command in git-submodule(1).\n\nand in GIT-SUBMODULE(1) as\n\n       update\n           [...] can be done in several ways\n           depending on command line options and the value of\n           submodule.<name>.update configuration variable. Supported update\n           procedures are:\n\n           checkout\n               [...] or no option is given, and\n               submodule.<name>.update is unset, or if it is set to checkout.\n\nSo the \"update\" config clearly only applies to the \"submodule update\"\ncommand, right?\n\nWell no, \"checkout --recurse-submodules\" is very similar\nto running \"submodule update\", except with a bit more checks, so you could\nthink that such an option applies to checkout as well. (and eventually\nrebase/merge etc. are supported as well.)\n\nSo initially I assumed both \"unspecified\" as well as \"checkout\"\nare good matches to support in the first round.\n\nThen I flip flopped to think that we should not interfere with these\nsettings at all (The checkout command does checkout and checkout only;\nno implicit rebase/merge ever in the future, because that would be\nconfusing). So ignoring that option seemed like the way to go.\n\nBut ignoring that option is also not the right approach.\nWhat if you have set it to \"none\" and really *expect* Git to not touch\nthat submodule?\n\nSo I dunno. Maybe it is a documentation issue, we need to spell out\nin the man page for checkout that --recurse-submodules is\nfollowing one of these models. Now which is the best default model here?\n\nThanks,\nStefan\n"},{"id":"312238","messageId":"CA+P7+xq3XF2=erRLiNjOUq9H66Sa5VCxNBzTF=RAycPYrPRFJg@mail.gmail.com","threadId":"45144","inReplyTo":"CAGZ79kahCN2dd9=CzqXJWSJKrkOfvd3HeQ-NORhyP=6B=KCrqg@mail.gmail.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-21T23:35:18Z","receivedAt":"2017-02-21T23:35:45Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Tue, Feb 21, 2017 at 2:16 PM, Stefan Beller <sbeller@google.com> wrote:\n> On Fri, Feb 17, 2017 at 10:42 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>> On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n>>> +       if (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n>>\n>> Here, and in other cases where we use\n>> is_active_submodule_with_strategy(), why do we only ever check\n>> SM_UPDATE_UNSPECIFIED? It seems really weird that we're only going to\n>> check submodules who's strategy is unspecified, when that defaults to\n>> checkout if I recall correctly? Shouldn't we check both? This applies\n>> to pretty much everywhere that you call this function that I noticed,\n>> which is why I removed the context.\n>\n> I am torn between this.\n>\n> submodule.<name>.update = {rebase, merge, checkout, none !command}\n> is currently documented in GIT-CONFIG(1) as\n>\n>        submodule.<name>.update\n>            The default update procedure for a submodule. This variable is\n>            populated by git submodule init from the gitmodules(5) file. See\n>            description of update command in git-submodule(1).\n>\n> and in GIT-SUBMODULE(1) as\n>\n>        update\n>            [...] can be done in several ways\n>            depending on command line options and the value of\n>            submodule.<name>.update configuration variable. Supported update\n>            procedures are:\n>\n>            checkout\n>                [...] or no option is given, and\n>                submodule.<name>.update is unset, or if it is set to checkout.\n>\n> So the \"update\" config clearly only applies to the \"submodule update\"\n> command, right?\n>\n> Well no, \"checkout --recurse-submodules\" is very similar\n> to running \"submodule update\", except with a bit more checks, so you could\n> think that such an option applies to checkout as well. (and eventually\n> rebase/merge etc. are supported as well.)\n>\n> So initially I assumed both \"unspecified\" as well as \"checkout\"\n> are good matches to support in the first round.\n>\n> Then I flip flopped to think that we should not interfere with these\n> settings at all (The checkout command does checkout and checkout only;\n> no implicit rebase/merge ever in the future, because that would be\n> confusing). So ignoring that option seemed like the way to go.\n\nHmm. So it's a bit complicated.\n\n>\n> But ignoring that option is also not the right approach.\n> What if you have set it to \"none\" and really *expect* Git to not touch\n> that submodule?\n\nOr set it to \"rebase\" and suddenly git-checkout is ignoring you and\njust checking things out anyways.\n\n>\n> So I dunno. Maybe it is a documentation issue, we need to spell out\n> in the man page for checkout that --recurse-submodules is\n> following one of these models. Now which is the best default model here?\n\nPersonally, I would go with that the config option sets the general\nstrategy used by the submodule whenever its updated, regardless of\nhow.\n\nSo, for example, setting it to none, means that recurse-submoduls will\nignore it when checking out. Setting it to rebase, or merge, and the\ncheckout will try to do those things?\n\nOr, if that's not really feasible, have the checkout go \"hey.. you\nasked me to recurse, but uhhh these submodules don't allow me to do\ncheckout, so I'm gonna fail\"? I think that's the best approach for\nnow.\n\n>\n> Thanks,\n> Stefan\n"},{"id":"312241","messageId":"CAGZ79kbVr5uqwJmzXJxUn0bpe=d_pgnW3_pYW-q0W0iO0KSRtw@mail.gmail.com","threadId":"45144","inReplyTo":"CA+P7+xq3XF2=erRLiNjOUq9H66Sa5VCxNBzTF=RAycPYrPRFJg@mail.gmail.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-21T23:44:02Z","receivedAt":"2017-02-21T23:44:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 21, 2017 at 3:35 PM, Jacob Keller <jacob.keller@gmail.com> wrote:\n> On Tue, Feb 21, 2017 at 2:16 PM, Stefan Beller <sbeller@google.com> wrote:\n>> On Fri, Feb 17, 2017 at 10:42 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>>> On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n>>>> +       if (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n>>>\n>>> Here, and in other cases where we use\n>>> is_active_submodule_with_strategy(), why do we only ever check\n>>> SM_UPDATE_UNSPECIFIED? It seems really weird that we're only going to\n>>> check submodules who's strategy is unspecified, when that defaults to\n>>> checkout if I recall correctly? Shouldn't we check both? This applies\n>>> to pretty much everywhere that you call this function that I noticed,\n>>> which is why I removed the context.\n>>\n>> I am torn between this.\n>>\n>> submodule.<name>.update = {rebase, merge, checkout, none !command}\n>> is currently documented in GIT-CONFIG(1) as\n>>\n>>        submodule.<name>.update\n>>            The default update procedure for a submodule. This variable is\n>>            populated by git submodule init from the gitmodules(5) file. See\n>>            description of update command in git-submodule(1).\n>>\n>> and in GIT-SUBMODULE(1) as\n>>\n>>        update\n>>            [...] can be done in several ways\n>>            depending on command line options and the value of\n>>            submodule.<name>.update configuration variable. Supported update\n>>            procedures are:\n>>\n>>            checkout\n>>                [...] or no option is given, and\n>>                submodule.<name>.update is unset, or if it is set to checkout.\n>>\n>> So the \"update\" config clearly only applies to the \"submodule update\"\n>> command, right?\n>>\n>> Well no, \"checkout --recurse-submodules\" is very similar\n>> to running \"submodule update\", except with a bit more checks, so you could\n>> think that such an option applies to checkout as well. (and eventually\n>> rebase/merge etc. are supported as well.)\n>>\n>> So initially I assumed both \"unspecified\" as well as \"checkout\"\n>> are good matches to support in the first round.\n>>\n>> Then I flip flopped to think that we should not interfere with these\n>> settings at all (The checkout command does checkout and checkout only;\n>> no implicit rebase/merge ever in the future, because that would be\n>> confusing). So ignoring that option seemed like the way to go.\n>\n> Hmm. So it's a bit complicated.\n>\n>>\n>> But ignoring that option is also not the right approach.\n>> What if you have set it to \"none\" and really *expect* Git to not touch\n>> that submodule?\n>\n> Or set it to \"rebase\" and suddenly git-checkout is ignoring you and\n> just checking things out anyways.\n>\n>>\n>> So I dunno. Maybe it is a documentation issue, we need to spell out\n>> in the man page for checkout that --recurse-submodules is\n>> following one of these models. Now which is the best default model here?\n>\n> Personally, I would go with that the config option sets the general\n> strategy used by the submodule whenever its updated, regardless of\n> how.\n>\n> So, for example, setting it to none, means that recurse-submoduls will\n> ignore it when checking out. Setting it to rebase, or merge, and the\n> checkout will try to do those things?\n\nThat is generally a sound idea when it comes to git-checkout.\n\nWhat about other future things like git-revert?\n(Ok I already brought up this example too many times; it should have\na revert-submodules as well switch, which is neither of the current strategies,\nso we'd have to invent a new strategy and make that the default for\nrevert. That strategy would make no sense in any other command though)\n\nWhat about \"git-rebase --recurse-submodules\"?\nShould git-rebase merge the submodules when it is configured to \"merge\"\nOr just \"checkout\" (the possibly non-fast-forward-y old sha1) ?\n\nThe only sane option IMO is \"rebase\" as well in the submodules, rewriting\nthe submodule pointers in the rebased commits in the superproject.\n\n>\n> Or, if that's not really feasible, have the checkout go \"hey.. you\n> asked me to recurse, but uhhh these submodules don't allow me to do\n> checkout, so I'm gonna fail\"? I think that's the best approach for\n> now.\n\nSo you'd propose to generally use the submodule.<name>.update\nstrategies with aggressive error-out but also keeping in mind\nthat the strategies might grow by a lot in the future (well only revert\ncomes to mind here).\n\nok, let's do that then.\n\nThanks,\nStefan\n"},{"id":"312265","messageId":"CA+P7+xqCvjErBgdUOp3kDKfWo5U60J1X8Af71rRFbDFsT0ODbg@mail.gmail.com","threadId":"45144","inReplyTo":"CAGZ79kbVr5uqwJmzXJxUn0bpe=d_pgnW3_pYW-q0W0iO0KSRtw@mail.gmail.com","subject":"Re: [PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Jacob Keller","fromEmail":"jacob.keller@gmail.com","sentAt":"2017-02-22T00:31:19Z","receivedAt":"2017-02-22T00:31:46Z","isPatch":true,"sender":{"key":"jacob.keller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/874719?v=4"},"body":"On Tue, Feb 21, 2017 at 3:44 PM, Stefan Beller <sbeller@google.com> wrote:\n> On Tue, Feb 21, 2017 at 3:35 PM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>> On Tue, Feb 21, 2017 at 2:16 PM, Stefan Beller <sbeller@google.com> wrote:\n>>> On Fri, Feb 17, 2017 at 10:42 AM, Jacob Keller <jacob.keller@gmail.com> wrote:\n>>>> On Wed, Feb 15, 2017 at 4:38 PM, Stefan Beller <sbeller@google.com> wrote:\n>>>>> +       if (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n>>>>\n>>>> Here, and in other cases where we use\n>>>> is_active_submodule_with_strategy(), why do we only ever check\n>>>> SM_UPDATE_UNSPECIFIED? It seems really weird that we're only going to\n>>>> check submodules who's strategy is unspecified, when that defaults to\n>>>> checkout if I recall correctly? Shouldn't we check both? This applies\n>>>> to pretty much everywhere that you call this function that I noticed,\n>>>> which is why I removed the context.\n>>>\n>>> I am torn between this.\n>>>\n>>> submodule.<name>.update = {rebase, merge, checkout, none !command}\n>>> is currently documented in GIT-CONFIG(1) as\n>>>\n>>>        submodule.<name>.update\n>>>            The default update procedure for a submodule. This variable is\n>>>            populated by git submodule init from the gitmodules(5) file. See\n>>>            description of update command in git-submodule(1).\n>>>\n>>> and in GIT-SUBMODULE(1) as\n>>>\n>>>        update\n>>>            [...] can be done in several ways\n>>>            depending on command line options and the value of\n>>>            submodule.<name>.update configuration variable. Supported update\n>>>            procedures are:\n>>>\n>>>            checkout\n>>>                [...] or no option is given, and\n>>>                submodule.<name>.update is unset, or if it is set to checkout.\n>>>\n>>> So the \"update\" config clearly only applies to the \"submodule update\"\n>>> command, right?\n>>>\n>>> Well no, \"checkout --recurse-submodules\" is very similar\n>>> to running \"submodule update\", except with a bit more checks, so you could\n>>> think that such an option applies to checkout as well. (and eventually\n>>> rebase/merge etc. are supported as well.)\n>>>\n>>> So initially I assumed both \"unspecified\" as well as \"checkout\"\n>>> are good matches to support in the first round.\n>>>\n>>> Then I flip flopped to think that we should not interfere with these\n>>> settings at all (The checkout command does checkout and checkout only;\n>>> no implicit rebase/merge ever in the future, because that would be\n>>> confusing). So ignoring that option seemed like the way to go.\n>>\n>> Hmm. So it's a bit complicated.\n>>\n>>>\n>>> But ignoring that option is also not the right approach.\n>>> What if you have set it to \"none\" and really *expect* Git to not touch\n>>> that submodule?\n>>\n>> Or set it to \"rebase\" and suddenly git-checkout is ignoring you and\n>> just checking things out anyways.\n>>\n>>>\n>>> So I dunno. Maybe it is a documentation issue, we need to spell out\n>>> in the man page for checkout that --recurse-submodules is\n>>> following one of these models. Now which is the best default model here?\n>>\n>> Personally, I would go with that the config option sets the general\n>> strategy used by the submodule whenever its updated, regardless of\n>> how.\n>>\n>> So, for example, setting it to none, means that recurse-submoduls will\n>> ignore it when checking out. Setting it to rebase, or merge, and the\n>> checkout will try to do those things?\n>\n> That is generally a sound idea when it comes to git-checkout.\n>\n> What about other future things like git-revert?\n> (Ok I already brought up this example too many times; it should have\n> a revert-submodules as well switch, which is neither of the current strategies,\n> so we'd have to invent a new strategy and make that the default for\n> revert. That strategy would make no sense in any other command though)\n>\n\nThis is where things get tricky, IMHO. The problem is that the\nstrategy now wants to encompass more things.\n\n> What about \"git-rebase --recurse-submodules\"?\n> Should git-rebase merge the submodules when it is configured to \"merge\"\n> Or just \"checkout\" (the possibly non-fast-forward-y old sha1) ?\n>\n> The only sane option IMO is \"rebase\" as well in the submodules, rewriting\n> the submodule pointers in the rebased commits in the superproject.\n>\n\nI'm not even really sure what rebase should do here at all. I assume\nby this you mean \"what would a git-rebase that also rebased submodules\ndo\"\n\nOfcourse the sane answer might be something like \"uhh you have to\ndecide that for yourself manually\" I think this is a really complex\nproblem to solve, and in this case I do not think rebase should even\nrely on the strategy. a \"recurse-submodules rebase\" would do something\nlike:\n\nrebase parent as normal, but if a commit changes the submodule, then\nit needs to re-create that submodule change using its own rebase\ninside  the submodule based on the (new) parent from the parent\nprojects history change, and then commit that as the committed change?\n\nBut I don't even know if that really makes sense in all cases either.\n\nI think you could check strategy, and then have rebase go \"uhhh here's\nwhat I found, you fix this manually\"?\n\nThat's quite complicated.\n\n>>\n>> Or, if that's not really feasible, have the checkout go \"hey.. you\n>> asked me to recurse, but uhhh these submodules don't allow me to do\n>> checkout, so I'm gonna fail\"? I think that's the best approach for\n>> now.\n>\n> So you'd propose to generally use the submodule.<name>.update\n> strategies with aggressive error-out but also keeping in mind\n> that the strategies might grow by a lot in the future (well only revert\n> comes to mind here).\n>\n> ok, let's do that then.\n>\n\nI think that's the safest option. If we add a new strategy, each\ncommand can decide what it should do for that strategy, and we can\ndecide that in the future. I'm not sure what users expect, but I think\nif we start by erroring out on things we can't support, and then if we\ndecide we can later ,it's not a backwards compatibility hurdle. Where\nas if we decide \"this works now\" but later discover that it cannot,\nthen,... we have to figure out a lot more backwards compat issues.\n\n> Thanks,\n> Stefan\n"},{"id":"312328","messageId":"CAGZ79kYhT4ikj2pfDJM_PQsh5XzGTVT2kByiE1moK8RUcusLcA@mail.gmail.com","threadId":"45144","inReplyTo":"xmqqlgt5x430.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 03/15] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-22T18:43:56Z","receivedAt":"2017-02-22T18:45:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 16, 2017 at 12:39 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> Currently lib-submodule-update.sh provides 2 functions\n>> test_submodule_switch and test_submodule_forced_switch that are used by a\n>> variety of tests to ensure that submodules behave as expected. The current\n>> expected behavior is that submodules are not touched at all (see\n>> 42639d2317a for the exact setup).\n>>\n>> In the future we want to teach all these commands to properly recurse\n>> into submodules. To do that, we'll add two testing functions to\n>> submodule-update-lib.sh test_submodule_switch_recursing and\n>> test_submodule_forced_switch_recursing.\n>\n> I'd remove \"properly\" and insert a colon after \"add two ... to\n> submodule-update-lib.sh\" before the names of two functions.\n\nok\n\n>\n>> +reset_work_tree_to_interested () {\n>> +     reset_work_tree_to $1 &&\n>> +     # indicate we are interested in the submodule:\n>> +     git -C submodule_update config submodule.sub1.url \"bogus\" &&\n>> +     # also have it available:\n>> +     if ! test -d submodule_update/.git/modules/sub1\n>> +     then\n>> +             mkdir submodule_update/.git/modules &&\n>\n> Would we want \"mkdir -p\" here to be safe?\n\nYes I cannot think of a downside of being overly cautious here.\n\n>\n>> +             cp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n>\n> ... ahh, wouldn't matter that much, we checked that modules/sub1\n> does not exist, and as long as nobody creates modules/ or modules/somethingelse\n> we are OK.\n\nWell, I'll add the -p\n\n>> +# Test that submodule contents are correctly updated when switching\n>> +# between commits that change a submodule.\n>> +# Test that the following transitions are correctly handled:\n>> +# (These tests are also above in the case where we expect no change\n>> +#  in the submodule)\n>> +# - Updated submodule\n>> +# - New submodule\n>> +# - Removed submodule\n>> +# - Directory containing tracked files replaced by submodule\n>> +# - Submodule replaced by tracked files in directory\n>> +# - Submodule replaced by tracked file with the same name\n>> +# - tracked file replaced by submodule\n>\n> These should work without trouble only when the paths involved in\n> the operation in the working tree are clean, right?  Just double\n> checking.  If they are dirty we should expect a failure, instead of\n> silent loss of information.\n\nyes, I'll go over the tests again and add those cases if missing.\n\n\n>> +     command=\"$1\"\n>\n> The dq-pair is not strictly needed on the RHS of the assignment, but\n> it is a good way to signal that we considered that we might receive\n> an argument with $IFS in it...\n>\n>> +                     $command add_sub1 &&\n>\n> ... and after doing so, not quoting $command here signals that we\n> expect command line splitting to happen.  Am I reading it correctly?\n> Without an actual caller appearing in this step, it is rather hard\n> to judge.\n>\n\nI followed the existing code without thinking about these points, but they are\nvalid and exactly how we'd expect the code to behave.\n$1 / $command will be e.g. \"git checkout --recurse-submodules\" in\nthis patch series; but later on we could also have functions.\nC.f.  t4137 which defines a function\n\n    apply_3way () {\n        git diff --ignore-submodules=dirty \"..$1\" | git apply --3way -\n    }\n    test_submodule_switch \"apply_3way\"\n\nWe'd want to have a similar thing for the recursing part, e.g.\n\n    apply_3way_recursing () {\n        git diff --submodule=diff \"..$1\" | git apply\n--recurse-submodules --3way -\n    }\n    test_submodule_switch_recursing \"apply_3way_recursing\"\n\n>> +                     echo sub1 > .git/info/exclude\n>\n>     \">.git/info/exclude\"\n\nok\n"},{"id":"312449","messageId":"20170223225735.10994-9-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 08/15] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:28Z","receivedAt":"2017-02-23T23:04:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that checks if a submodule needs\nto be checked for an update before attempting the update.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 16 ++++++++++++++++\n submodule.h |  7 +++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 591f4a694e..8b2c0212be 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -548,6 +548,22 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int should_update_submodules(void)\n+{\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n+{\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn NULL;\n+\n+\tif (!should_update_submodules())\n+\t\treturn NULL;\n+\n+\treturn submodule_from_path(null_sha1, ce->name);\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t      int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex b4e60c08d2..6f3fe85c7c 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,6 +65,13 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+/* Check if we want to update any submodule.*/\n+extern int should_update_submodules(void);\n+/*\n+ * Returns the submodule struct if the given ce entry is a submodule\n+ * and it should be updated. Returns NULL otherwise.\n+ */\n+extern const struct submodule *submodule_from_ce(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312450","messageId":"20170223225735.10994-10-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 09/15] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:29Z","receivedAt":"2017-02-23T23:04:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 8b2c0212be..0b2596e88a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -356,6 +356,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n  * summary output. If it can locate the submodule objects directory it will\n  * attempt to lookup both the left and right commits and put them into the\n@@ -1390,18 +1407,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n  * Embeds a single submodules git directory into the superprojects git dir,\n  * non recursively.\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312451","messageId":"20170223225735.10994-2-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 01/15] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:21Z","receivedAt":"2017-02-23T23:04:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand.  The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..5df528ea81 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n #   replace_file_with_sub1)\n #\n-#                   --O-----O\n-#                  /  ^     replace_directory_with_sub1\n-#                 /   replace_sub1_with_directory\n-#                /----O\n-#               /     ^\n-#              /      modify_sub1\n-#      O------O-------O\n-#      ^      ^\\      ^\n-#      |      | \\     remove_sub1\n-#      |      |  -----O-----O\n-#      |      |   \\   ^     replace_file_with_sub1\n-#      |      |    \\  replace_sub1_with_file\n-#      |   add_sub1 --O-----O\n-# no_submodule        ^     valid_sub1\n-#                     invalid_sub1\n+#                     ----O\n+#                    /    ^\n+#                   /     remove_sub1\n+#                  /\n+#       add_sub1  /-------O\n+#             |  /        ^\n+#             | /         modify_sub1\n+#             v/\n+#      O------O-----------O---------O\n+#      ^       \\          ^         replace_directory_with_sub1\n+#      |        \\         replace_sub1_with_directory\n+# no_submodule   \\\n+#                 --------O---------O\n+#                  \\      ^         replace_file_with_sub1\n+#                   \\     replace_sub1_with_file\n+#                    \\\n+#                     ----O---------O\n+#                         ^         valid_sub1\n+#                         invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312452","messageId":"20170223225735.10994-5-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 04/15] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:24Z","receivedAt":"2017-02-23T23:04:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/grep.c | 2 +-\n submodule.c    | 7 ++-----\n submodule.h    | 8 +++++++-\n 3 files changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..0e55372f37 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -234,15 +234,12 @@ int is_submodule_initialized(const char *path)\n \treturn ret;\n }\n \n-/*\n- * Determine if a submodule has been populated at a given 'path'\n- */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..0b915bd3ac 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,13 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+/*\n+ * Determine if a submodule has been populated at a given 'path' by checking if\n+ * the <path>/.git resolves to a valid git repository.\n+ * If return_error_code is NULL, die on error.\n+ * Otherwise the return error code is the same as of resolve_gitdir_gently.\n+ */\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312453","messageId":"20170223225735.10994-4-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 03/15] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:23Z","receivedAt":"2017-02-23T23:04:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh: test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 485 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 483 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex c0d6325133..0b26f0e20f 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n #                    /    ^\n #                   /     remove_sub1\n #                  /\n-#       add_sub1  /-------O\n-#             |  /        ^\n+#       add_sub1  /-------O---------O\n+#             |  /        ^         modify_sub1_recursive\n #             | /         modify_sub1\n #             v/\n #      O------O-----------O---------O\n@@ -48,6 +49,17 @@ create_lib_submodule_repo () {\n \t\tgit commit -m \"Base inside first submodule\" &&\n \t\tgit branch \"no_submodule\"\n \t) &&\n+\tgit init submodule_update_sub2 &&\n+\t(\n+\t\tcd submodule_update_sub2\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"nested submodule base\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -84,6 +96,14 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b modify_sub1_recursively modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -150,6 +170,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -180,6 +209,18 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# also have it available:\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\tfi\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -695,3 +736,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+#  in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+#   git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 >.git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignroed\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively\n+\t\t\ttest_submodule_content sub1/sub2\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312454","messageId":"20170223225735.10994-1-sbeller@google.com","threadId":"45144","inReplyTo":"xmqqlgt5vlse.fsf@gitster.mtv.corp.google.com","subject":"[RFCv5 PATCH 00/14] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:20Z","receivedAt":"2017-02-23T23:04:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20161203003022.29797-1-sbeller@google.com/\n\nv5:\n * as v4 was the first version queued by Junio, we do have an interdiff below!\n * renamed functions\n * changed the API, now the caller has to take care of the submodule strategy\n   themselves. (Note this can be different for different situations, e.g.\n   when a submodule is deleted, we can do that for any strategy except none and\n   !command. But for moving to a new state of the submodule we currently\n   only implement \"checkout\" [unspecified defaults to checkout]. warning about\n   the others, doing nothing there.)\n\nv4:\n * addressed all comments of Brian, Junio and Brandon.\n Thanks!\n * one major point of change is the introduction of another patch\n   \"lib-submodule-update.sh: do not use ./. as submodule remote\",\n   as that took some time to track down the existing bug.\n \nv3:\n * moved tests from t2013 to the generic submodule library.\n * factored out the refactoring patches to be up front\n * As I redid the complete implementation, I have the impression this time\n   it is cleaner than previous versions.\n \n I think we still have to fix the corner cases of directory/file/submodule \n conflicts before merging, but this serves as a status update on my current\n way of thinking how to implement the worktree commands being aware of\n submodules.\n \nThanks,\nStefan\n\nv2:\n* based on top of the series sent out an hour ago\n  \"[PATCHv4 0/5] submodule embedgitdirs\"\n* Try to embed a submodule if we need to remove it.\n* Strictly do not change behavior if not giving the new flag.\n* I think I missed some review comments from v1, but I'd like to get\n  the current state out over the weekend, as a lot has changed so far.\n  On Monday I'll go through the previous discussion with a comb to see\n  if I missed something.\n  \nv1:\nWhen working with submodules, nearly anytime after checking out\na different state of the projects, that has submodules changed\nyou'd run \"git submodule update\" with a current version of Git.\n\nThere are two problems with this approach:\n\n* The \"submodule update\" command is dangerous as it\n  doesn't check for work that may be lost in the submodule\n  (e.g. a dangling commit).\n* you may forget to run the command as checkout is supposed\n  to do all the work for you.\n\nIntegrate updating the submodules into git checkout, with the same\nsafety promises that git-checkout has, i.e. not throw away data unless\nasked to. This is done by first checking if the submodule is at the same\nsha1 as it is recorded in the superproject. If there are changes we stop\nproceeding the checkout just like it is when checking out a file that\nhas local changes.\n\nThe integration happens in the code that is also used in other commands\nsuch that it will be easier in the future to make other commands aware\nof submodule.\n\nThis also solves d/f conflicts in case you replace a file/directory\nwith a submodule or vice versa.\n\nThe patches are still a bit rough, but the overall series seems\npromising enough to me that I want to put it out here.\n\nAny review, specifically on the design level welcome!\n\nThanks,\nStefan\n\n\nStefan Beller (14):\n  lib-submodule-update.sh: reorder create_lib_submodule_repo\n  lib-submodule-update.sh: define tests for recursing into submodules\n  make is_submodule_populated gently\n  connect_work_tree_and_git_dir: safely create leading directories\n  update submodules: add submodule config parsing\n  update submodules: add a config option to determine if submodules are\n    updated\n  update submodules: introduce is_interesting_submodule\n  update submodules: move up prepare_submodule_repo_env\n  update submodules: add submodule_go_from_to\n  unpack-trees: pass old oid to verify_clean_submodule\n  unpack-trees: check if we can perform the operation for submodules\n  read-cache: remove_marked_cache_entries to wipe selected submodules.\n  entry.c: update submodules when interesting\n  builtin/checkout: add --recurse-submodules switch\n\ndiff --git a/entry.c b/entry.c\nindex ae40611c97..d2b512da90 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -147,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n \tunsigned long size;\n \tsize_t wrote, newsize = 0;\n \tstruct stat st;\n+\tconst struct submodule *sub;\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n@@ -204,13 +205,10 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n-\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n-\t\t\t\t/*\n-\t\t\t\t * force=1 is ok for any case as we did a dry\n-\t\t\t\t * run before with appropriate force setting\n-\t\t\t\t */\n-\t\t\t\treturn submodule_go_from_to(ce->name,\n-\t\t\t\t\tNULL, oid_to_hex(&ce->oid), 0, 1);\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub)\n+\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -267,12 +265,14 @@ int checkout_entry(struct cache_entry *ce,\n \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n+\t\tconst struct submodule *sub;\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n \t\t/*\n \t\t * Needs to be checked before !changed returns early,\n \t\t * as the possibly empty directory was not changed\n \t\t */\n-\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub) {\n \t\t\tint err;\n \t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n \t\t\t\tstruct stat sb;\n@@ -281,11 +281,13 @@ int checkout_entry(struct cache_entry *ce,\n \t\t\t\tif (!(st.st_mode & S_IFDIR))\n \t\t\t\t\tunlink_or_warn(ce->name);\n \n-\t\t\t\treturn submodule_go_from_to(ce->name,\n-\t\t\t\t\tNULL, oid_to_hex(&ce->oid), 0, 1);\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n \t\t\t} else\n-\t\t\t\treturn submodule_go_from_to(ce->name,\n-\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), 0, 1);\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n \t\t}\n \n \t\tif (!changed)\ndiff --git a/read-cache.c b/read-cache.c\nindex b78a7f02e3..9a2abacf7a 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -19,6 +19,7 @@\n #include \"split-index.h\"\n #include \"utf8.h\"\n #include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -521,6 +522,22 @@ int remove_index_entry_at(struct index_state *istate, int pos)\n \treturn 1;\n }\n \n+static void remove_submodule_according_to_strategy(const struct submodule *sub)\n+{\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\t\tsubmodule_move_head(sub->path, \"HEAD\", NULL, \\\n+\t\t\t\t    SUBMODULE_MOVE_HEAD_FORCE);\n+\t\tbreak;\n+\tcase SM_UPDATE_NONE:\n+\tcase SM_UPDATE_COMMAND:\n+\t\t; /* Do not touch the submodule. */\n+\t}\n+}\n+\n /*\n  * Remove all cache entries marked for removal, that is where\n  * CE_REMOVE is set in ce_flags.  This is much more effective than\n@@ -533,10 +550,13 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n-\t\t\tif (is_active_submodule_with_strategy(ce_array[i], SM_UPDATE_UNSPECIFIED))\n-\t\t\t\tsubmodule_go_from_to(ce_array[i]->name, \"HEAD\", NULL, 0, 1);\n-\t\t\tremove_name_hash(istate, ce_array[i]);\n-\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\tconst struct submodule *sub = submodule_from_ce(ce_array[i]);\n+\t\t\tif (sub) {\n+\t\t\t\tremove_submodule_according_to_strategy(sub);\n+\t\t\t} else {\n+\t\t\t\tremove_name_hash(istate, ce_array[i]);\n+\t\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\t}\n \t\t}\n \t\telse\n \t\t\tce_array[j++] = ce_array[i];\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93f01c4378..3e8e380d98 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -243,8 +243,6 @@ static int parse_update_recurse(const char *opt, const char *arg,\n \tcase 0:\n \t\treturn RECURSE_SUBMODULES_OFF;\n \tdefault:\n-\t\tif (!strcmp(arg, \"checkout\"))\n-\t\t\treturn RECURSE_SUBMODULES_ON;\n \t\tif (die_on_error)\n \t\t\tdie(\"bad %s argument: %s\", opt, arg);\n \t\treturn RECURSE_SUBMODULES_ERROR;\ndiff --git a/submodule.c b/submodule.c\nindex 84cc62f3bb..a2cf8c9376 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -565,31 +565,20 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n-int touch_submodules_in_worktree(void)\n+int should_update_submodules(void)\n {\n-\t/*\n-\t * Update can't be \"none\", \"merge\" or \"rebase\",\n-\t * treat any value as OFF, except an explicit ON.\n-\t */\n \treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n }\n \n-int is_active_submodule_with_strategy(const struct cache_entry *ce,\n-\t\t\t\t      enum submodule_update_type strategy)\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n {\n-\tconst struct submodule *sub;\n-\n \tif (!S_ISGITLINK(ce->ce_mode))\n-\t\treturn 0;\n-\n-\tif (!touch_submodules_in_worktree())\n-\t\treturn 0;\n+\t\treturn NULL;\n \n-\tsub = submodule_from_path(null_sha1, ce->name);\n-\tif (!sub)\n-\t\treturn 0;\n+\tif (!should_update_submodules())\n+\t\treturn NULL;\n \n-\treturn sub->update_strategy.type == strategy;\n+\treturn submodule_from_path(null_sha1, ce->name);\n }\n \n static int has_remote(const char *refname, const struct object_id *oid,\n@@ -1252,34 +1241,23 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \n static int submodule_has_dirty_index(const struct submodule *sub)\n {\n-\tssize_t len;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n-\tstruct strbuf buf = STRBUF_INIT;\n-\tint ret = 0;\n \n \tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n \n \tcp.git_cmd = 1;\n-\targv_array_pushl(&cp.args, \"diff-index\", \"--cached\", \"HEAD\", NULL);\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n+\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n \tcp.no_stdin = 1;\n-\tcp.out = -1;\n+\tcp.no_stdout = 1;\n \tcp.dir = sub->path;\n \tif (start_command(&cp))\n-\t\tdie(\"could not recurse into submodule %s\", sub->path);\n+\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n \n-\tlen = strbuf_read(&buf, cp.out, 1024);\n-\tif (len > 2)\n-\t\tret = 1;\n-\n-\tclose(cp.out);\n-\tif (finish_command(&cp))\n-\t\tdie(\"could not recurse into submodule %s\", sub->path);\n-\n-\tstrbuf_release(&buf);\n-\treturn ret;\n+\treturn finish_command(&cp);\n }\n \n-void submodule_clean_index(const char *path)\n+void submodule_reset_index(const char *path)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n@@ -1294,21 +1272,18 @@ void submodule_clean_index(const char *path)\n \targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n \n \tif (run_command(&cp))\n-\t\tdie(\"could not clean submodule index\");\n+\t\tdie(\"could not reset submodule index\");\n }\n \n /**\n  * Moves a submodule at a given path from a given head to another new head.\n  * For edge cases (a submodule coming into existence or removing a submodule)\n  * pass NULL for old or new respectively.\n- *\n- * TODO: move dryrun and forced to flags.\n  */\n-int submodule_go_from_to(const char *path,\n+int submodule_move_head(const char *path,\n \t\t\t const char *old,\n \t\t\t const char *new,\n-\t\t\t int dry_run,\n-\t\t\t int force)\n+\t\t\t unsigned flags)\n {\n \tint ret = 0;\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n@@ -1319,7 +1294,13 @@ int submodule_go_from_to(const char *path,\n \tif (!sub)\n \t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n \n-\tif (!dry_run) {\n+\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub))\n+\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n \t\tif (old) {\n \t\t\tif (!submodule_uses_gitfile(path))\n \t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n@@ -1332,15 +1313,7 @@ int submodule_go_from_to(const char *path,\n \t\t\tstrbuf_release(&sb);\n \n \t\t\t/* make sure the index is clean as well */\n-\t\t\tsubmodule_clean_index(path);\n-\t\t}\n-\t}\n-\n-\tif (old && !force) {\n-\t\t/* Check if the submodule has a dirty index. */\n-\t\tif (submodule_has_dirty_index(sub)) {\n-\t\t\t/* print a thing here? */\n-\t\t\treturn -1;\n+\t\t\tsubmodule_reset_index(path);\n \t\t}\n \t}\n \n@@ -1353,12 +1326,12 @@ int submodule_go_from_to(const char *path,\n \targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n \targv_array_pushl(&cp.args, \"read-tree\", NULL);\n \n-\tif (dry_run)\n+\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n \t\targv_array_push(&cp.args, \"-n\");\n \telse\n \t\targv_array_push(&cp.args, \"-u\");\n \n-\tif (force)\n+\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n \t\targv_array_push(&cp.args, \"--reset\");\n \telse\n \t\targv_array_push(&cp.args, \"-m\");\n@@ -1371,7 +1344,7 @@ int submodule_go_from_to(const char *path,\n \t\tgoto out;\n \t}\n \n-\tif (!dry_run) {\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n \t\tif (new) {\n \t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n \t\t\t/* also set the HEAD accordingly */\ndiff --git a/submodule.h b/submodule.h\nindex 3336607bfc..4cdf6445f7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,19 +65,13 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n-\n-/*\n- * Traditionally Git ignored changes made for submodules.\n- * This function checks if we are interested in the given submodule\n- * for any kind of operation.\n- */\n-extern int touch_submodules_in_worktree(void);\n+/* Check if we want to update any submodule.*/\n+extern int should_update_submodules(void);\n /*\n- * Check if the given ce entry is a submodule with the given update\n- * strategy configured.\n+ * Returns the submodule struct if the given ce entry is a submodule\n+ * and it should be updated. Returns NULL otherwise.\n  */\n-extern int is_active_submodule_with_strategy(const struct cache_entry *ce,\n-\t\t\t\t\t     enum submodule_update_type strategy);\n+extern const struct submodule *submodule_from_ce(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n@@ -102,10 +96,12 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n-extern int submodule_go_from_to(const char *path,\n-\t\t\t\tconst char *old,\n-\t\t\t\tconst char *new,\n-\t\t\t\tint dry_run, int force);\n+#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n+#define SUBMODULE_MOVE_HEAD_FORCE   (1<<1)\n+extern int submodule_move_head(const char *path,\n+\t\t\t       const char *old,\n+\t\t\t       const char *new,\n+\t\t\t       unsigned flags);\n \n /*\n  * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 4693ba7a7e..54cd8a6366 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -216,7 +216,7 @@ reset_work_tree_to_interested () {\n \t# also have it available:\n \tif ! test -d submodule_update/.git/modules/sub1\n \tthen\n-\t\tmkdir submodule_update/.git/modules &&\n+\t\tmkdir -p submodule_update/.git/modules &&\n \t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n \tfi\n }\n@@ -809,7 +809,7 @@ test_submodule_switch_recursing () {\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n \t\t\t: >sub1 &&\n-\t\t\techo sub1 > .git/info/exclude\n+\t\t\techo sub1 >.git/info/exclude\n \t\t\t$command add_sub1 &&\n \t\t\ttest_superproject_content origin/add_sub1 &&\n \t\t\ttest_submodule_content sub1 origin/add_sub1\n@@ -1133,7 +1133,7 @@ test_submodule_forced_switch_recursing () {\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n-\t\t\t: > sub1/expect &&\n+\t\t\t: >sub1/expect &&\n \t\t\t$command replace_sub1_with_file &&\n \t\t\ttest_superproject_content origin/replace_sub1_with_file\n \t\t)\n@@ -1146,7 +1146,7 @@ test_submodule_forced_switch_recursing () {\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n-\t\t\t: > sub1/untracked_file &&\n+\t\t\t: >sub1/untracked_file &&\n \t\t\ttest_must_fail $command replace_sub1_with_file &&\n \t\t\ttest_superproject_content origin/add_sub1 &&\n \t\t\ttest -f sub1/untracked_file\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 40af8e9b5f..8333da2cc9 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -11,6 +11,7 @@\n #include \"split-index.h\"\n #include \"dir.h\"\n #include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /*\n  * Error messages expected by scripts out of plumbing commands such as\n@@ -47,8 +48,8 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n \n-\t/* ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE */\n-\t\"Submodule '%s' cannot be deleted as it contains untracked files.\",\n+\t/* ERROR_WOULD_LOSE_SUBMODULE */\n+\t\"Submodule '%s' cannot checkout new HEAD.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -165,8 +166,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n-\tmsgs[ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE] =\n-\t\t_(\"Submodule '%s' cannot be deleted as it contains untracked files.\");\n+\tmsgs[ERROR_WOULD_LOSE_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot checkout new HEAD\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -246,13 +247,31 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n-static int submodule_check_from_to(const struct cache_entry *ce, const char *old_id, const char *new_id, struct unpack_trees_options *o)\n+static int check_submodule_move_head(const struct cache_entry *ce,\n+\t\t\t\t     const char *old_id,\n+\t\t\t\t     const char *new_id,\n+\t\t\t\t     struct unpack_trees_options *o)\n {\n-\tif (submodule_go_from_to(ce->name, old_id,\n-\t\t\t\t new_id, 1, o->reset))\n-\t\treturn o->gently ? -1 :\n-\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_UNTRACKED_SUBMODULE, ce->name);\n-\treturn 0;\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\t\tif (submodule_move_head(ce->name, old_id, new_id, SUBMODULE_MOVE_HEAD_DRY_RUN))\n+\t\t\treturn o->gently ? -1 :\n+\t\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);\n+\t\treturn 0;\n+\tcase SM_UPDATE_NONE:\n+\t\treturn 0;\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\tcase SM_UPDATE_COMMAND:\n+\tdefault:\n+\t\twarning(_(\"submodule update strategy not supported for submodule '%s'\"), ce->name);\n+\t\treturn -1;\n+\t}\n }\n \n static void reload_gitmodules_file(struct index_state *index,\n@@ -262,18 +281,18 @@ static void reload_gitmodules_file(struct index_state *index,\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \t\tif (ce->ce_flags & CE_UPDATE) {\n-\n \t\t\tint r = strcmp(ce->name, \".gitmodules\");\n \t\t\tif (r < 0)\n \t\t\t\tcontinue;\n \t\t\telse if (r == 0) {\n+\t\t\t\tsubmodule_free();\n \t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t\tgitmodules_config();\n+\t\t\t\tgit_config(submodule_config, NULL);\n \t\t\t} else\n \t\t\t\tbreak;\n \t\t}\n \t}\n-\tgitmodules_config();\n-\tgit_config(submodule_config, NULL);\n }\n \n /*\n@@ -282,8 +301,21 @@ static void reload_gitmodules_file(struct index_state *index,\n  */\n static void unlink_entry(const struct cache_entry *ce)\n {\n-\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n-\t\tsubmodule_go_from_to(ce->name, \"HEAD\", NULL, 0, 1);\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (sub) {\n+\t\tswitch (sub->update_strategy.type) {\n+\t\tcase SM_UPDATE_UNSPECIFIED:\n+\t\tcase SM_UPDATE_CHECKOUT:\n+\t\tcase SM_UPDATE_REBASE:\n+\t\tcase SM_UPDATE_MERGE:\n+\t\t\tsubmodule_move_head(ce->name, \"HEAD\", NULL,\n+\t\t\t\t\t    SUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\tbreak;\n+\t\tcase SM_UPDATE_NONE:\n+\t\tcase SM_UPDATE_COMMAND:\n+\t\t\treturn; /* Do not touch the submodule. */\n+\t\t}\n+\t}\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -339,7 +371,7 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n-\tif (touch_submodules_in_worktree() && o->update && !o->dry_run)\n+\tif (should_update_submodules() && o->update && !o->dry_run)\n \t\treload_gitmodules_file(index, &state);\n \n \tfor (i = 0; i < index->cache_nr; i++) {\n@@ -1400,9 +1432,8 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n \n-\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n-\t\t\tint r;\n-\t\t\tr = submodule_check_from_to(ce,\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint r = check_submodule_move_head(ce,\n \t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n \t\t\tif (r)\n \t\t\t\treturn o->gently ? -1 :\n@@ -1463,12 +1494,11 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n-\tif (!is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n+\tif (!submodule_from_ce(ce))\n \t\treturn 0;\n \n-\treturn submodule_check_from_to(ce,\n-\t\t\t\t       old_sha1,\n-\t\t\t\t       oid_to_hex(&ce->oid), o);\n+\treturn check_submodule_move_head(ce, old_sha1,\n+\t\t\t\t\t oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1635,10 +1665,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n \t\telse {\n-\t\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n-\t\t\t\tret = submodule_check_from_to(ce,\n-\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n-\t\t\t\t\t\t\tNULL, o);\n+\t\t\tif (submodule_from_ce(ce))\n+\t\t\t\tret = check_submodule_move_head(ce,\n+\t\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t\tNULL, o);\n \t\t\telse\n \t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n \t\t\t\t\t\t\t &st, error_type, o);\n@@ -1650,9 +1680,9 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n-\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED))\n-\t\t\treturn submodule_check_from_to(ce, oid_to_hex(&ce->oid),\n-\t\t\t\t\t\t       NULL, o);\n+\t\tif (submodule_from_ce(ce))\n+\t\t\treturn check_submodule_move_head(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t NULL, o);\n \n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t  ce_to_dtype(ce), ce, &st,\n@@ -1710,11 +1740,10 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n \n-\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n-\t\t\tint ret = submodule_check_from_to(ce,\n-\t\t\t\t\t\t\t  NULL,\n-\t\t\t\t\t\t\t  oid_to_hex(&ce->oid),\n-\t\t\t\t\t\t\t  o);\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, NULL,\n+\t\t\t\t\t\t\t    oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t    o);\n \t\t\tif (ret)\n \t\t\t\treturn ret;\n \t\t}\n@@ -1739,9 +1768,13 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n-\t\tif (is_active_submodule_with_strategy(ce, SM_UPDATE_UNSPECIFIED)) {\n-\t\t\tif (submodule_check_from_to(ce, oid_to_hex(&old->oid), oid_to_hex(&ce->oid), o))\n-\t\t\t\treturn -1;\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t    oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t    o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n \t\t}\n \t} else {\n \t\t/*\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex c0427ce082..6c48117b84 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,7 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n-\tERROR_WOULD_LOSE_UNTRACKED_SUBMODULE,\n+\tERROR_WOULD_LOSE_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n\n\n"},{"id":"312455","messageId":"20170223225735.10994-12-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 11/15] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:31Z","receivedAt":"2017-02-23T23:05:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n  * Currently, git does not checkout subprojects during a superproject\n  * checkout, so it is not going to overwrite anything.\n  */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t  const struct cache_entry *ce,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t    resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t      ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312456","messageId":"20170223225735.10994-3-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 02/15] lib-submodule-update.sh: do not use ./. as submodule remote","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:22Z","receivedAt":"2017-02-23T23:05:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Adding the repository itself as a submodule does not make sense in the\nreal world. In our test suite we used to do that out of convenience in\nsome tests as the current repository has easiest access for setting up\n'just a submodule'.\n\nHowever this doesn't quite test the real world, so let's do not follow\nthis pattern any further and actually create an independent repository\nthat we can use as a submodule.\n\nWhen using './.' as the remote the superproject and submodule share the\nsame objects, such that testing if a given sha1 is a valid commit works\nin either repository.  As running commands in an unpopulated submodule\nfall back to the superproject, this happens in `reset_work_tree_to`\nto determine if we need to populate the submodule. Fix this bug by\nchecking in the actual remote now.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n t/lib-submodule-update.sh | 15 +++++++++++++--\n 1 file changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5df528ea81..c0d6325133 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -37,6 +37,17 @@\n #\n \n create_lib_submodule_repo () {\n+\tgit init submodule_update_sub1 &&\n+\t(\n+\t\tcd submodule_update_sub1 &&\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"Base inside first submodule\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -49,7 +60,7 @@ create_lib_submodule_repo () {\n \t\tgit branch \"no_submodule\" &&\n \n \t\tgit checkout -b \"add_sub1\" &&\n-\t\tgit submodule add ./. sub1 &&\n+\t\tgit submodule add ../submodule_update_sub1 sub1 &&\n \t\tgit config -f .gitmodules submodule.sub1.ignore all &&\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n@@ -162,7 +173,7 @@ reset_work_tree_to () {\n \t\ttest_must_be_empty actual &&\n \t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n \t\tif test -n \"$sha1\" &&\n-\t\t   test $(cd \"sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\t   test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312457","messageId":"20170223225735.10994-7-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 06/15] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:26Z","receivedAt":"2017-02-23T23:05:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule-config.c | 20 ++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 29 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..3e8e380d98 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,26 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t       int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t      unsigned char *gitmodules_sha1,\n \t\t\t\t      struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312459","messageId":"20170223225735.10994-15-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 14/15] entry.c: update submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:34Z","receivedAt":"2017-02-23T23:05:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n entry.c | 30 ++++++++++++++++++++++++++++++\n 1 file changed, 30 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..d2b512da90 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t       const struct checkout *state)\n@@ -146,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n \tunsigned long size;\n \tsize_t wrote, newsize = 0;\n \tstruct stat st;\n+\tconst struct submodule *sub;\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n@@ -203,6 +205,10 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub)\n+\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -259,7 +265,31 @@ int checkout_entry(struct cache_entry *ce,\n \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n+\t\tconst struct submodule *sub;\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\t} else\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312461","messageId":"20170223225735.10994-13-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 12/15] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:32Z","receivedAt":"2017-02-23T23:05:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n unpack-trees.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++++++----\n unpack-trees.h |   1 +\n 2 files changed, 123 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..8333da2cc9 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,8 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /*\n  * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +47,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_SUBMODULE */\n+\t\"Submodule '%s' cannot checkout new HEAD.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +166,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot checkout new HEAD\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +247,75 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int check_submodule_move_head(const struct cache_entry *ce,\n+\t\t\t\t     const char *old_id,\n+\t\t\t\t     const char *new_id,\n+\t\t\t\t     struct unpack_trees_options *o)\n+{\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\t\tif (submodule_move_head(ce->name, old_id, new_id, SUBMODULE_MOVE_HEAD_DRY_RUN))\n+\t\t\treturn o->gently ? -1 :\n+\t\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);\n+\t\treturn 0;\n+\tcase SM_UPDATE_NONE:\n+\t\treturn 0;\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\tcase SM_UPDATE_COMMAND:\n+\tdefault:\n+\t\twarning(_(\"submodule update strategy not supported for submodule '%s'\"), ce->name);\n+\t\treturn -1;\n+\t}\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t   struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tsubmodule_free();\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t\tgitmodules_config();\n+\t\t\t\tgit_config(submodule_config, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n /*\n  * Unlink the last component and schedule the leading directories for\n  * removal, such that empty directories get removed.\n  */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (sub) {\n+\t\tswitch (sub->update_strategy.type) {\n+\t\tcase SM_UPDATE_UNSPECIFIED:\n+\t\tcase SM_UPDATE_CHECKOUT:\n+\t\tcase SM_UPDATE_REBASE:\n+\t\tcase SM_UPDATE_MERGE:\n+\t\t\tsubmodule_move_head(ce->name, \"HEAD\", NULL,\n+\t\t\t\t\t    SUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\tbreak;\n+\t\tcase SM_UPDATE_NONE:\n+\t\tcase SM_UPDATE_COMMAND:\n+\t\t\treturn; /* Do not touch the submodule. */\n+\t\t}\n+\t}\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +371,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (should_update_submodules() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1431,26 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint r = check_submodule_move_head(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index.  This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1494,11 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t  enum unpack_trees_error_types error_type,\n \t\t\t\t  struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!submodule_from_ce(ce))\n+\t\treturn 0;\n+\n+\treturn check_submodule_move_head(ce, old_sha1,\n+\t\t\t\t\t oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1664,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (submodule_from_ce(ce))\n+\t\t\t\tret = check_submodule_move_head(ce,\n+\t\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1680,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (submodule_from_ce(ce))\n+\t\t\treturn check_submodule_move_head(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t  ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t  error_type, o);\n@@ -1643,6 +1739,15 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, NULL,\n+\t\t\t\t\t\t\t    oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t    o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1768,14 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t    oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t    o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..6c48117b84 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312463","messageId":"20170223225735.10994-6-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 05/15] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:25Z","receivedAt":"2017-02-23T23:06:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. This patch makes\nconnect_work_tree_and_git_dir safe to use for both cases of\neither the git dir or the working dir missing.\n\nTo do so, we need to call safe_create_leading_directories[_const]\non both directories. However this has to happen before we construct\nthe absolute paths as real_pathdup assumes the directories to\nbe there already.\n\nSo for both the config file in the git dir as well as the .git link\nfile we need to\na) construct the name\nb) call SCLD\nc) get the absolute path\nd) once a-c is done for both we can consume the absolute path\n   to compute the relative path to each other and store those\n   relative paths.\n\nThe implementation provided here puts a) and b) for both cases first,\nand then performs c and d after.\n\nOne of the two users of 'connect_work_tree_and_git_dir' already checked\nfor the directory being there, so we can loose that check as\nconnect_work_tree_and_git_dir handles this functionality now.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n dir.c       | 32 +++++++++++++++++++++-----------\n submodule.c | 11 ++---------\n 2 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..6f52af7abb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n /* Update gitfile and core.worktree setting to connect work tree and git dir */\n void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n {\n-\tstruct strbuf file_name = STRBUF_INIT;\n+\tstruct strbuf gitfile_sb = STRBUF_INIT;\n+\tstruct strbuf cfg_sb = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n-\tchar *git_dir = real_pathdup(git_dir_);\n-\tchar *work_tree = real_pathdup(work_tree_);\n+\tchar *git_dir, *work_tree;\n \n-\t/* Update gitfile */\n-\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n-\twrite_file(file_name.buf, \"gitdir: %s\",\n-\t\t   relative_path(git_dir, work_tree, &rel_path));\n+\t/* Prepare .git file */\n+\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n+\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n+\n+\t/* Prepare config file */\n+\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n+\tif (safe_create_leading_directories_const(cfg_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n \n+\tgit_dir = real_pathdup(git_dir_);\n+\twork_tree = real_pathdup(work_tree_);\n+\n+\t/* Write .git file */\n+\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n+\t\t   relative_path(git_dir, work_tree, &rel_path));\n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n+\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n \t\t\t       relative_path(work_tree, git_dir, &rel_path));\n \n-\tstrbuf_release(&file_name);\n+\tstrbuf_release(&gitfile_sb);\n+\tstrbuf_release(&cfg_sb);\n \tstrbuf_release(&rel_path);\n \tfree(work_tree);\n \tfree(git_dir);\ndiff --git a/submodule.c b/submodule.c\nindex 0e55372f37..04d185738f 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1442,8 +1442,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \n \t/* Not populated? */\n \tif (!sub_git_dir) {\n-\t\tchar *real_new_git_dir;\n-\t\tconst char *new_git_dir;\n \t\tconst struct submodule *sub;\n \n \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n@@ -1466,13 +1464,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tsub = submodule_from_path(null_sha1, path);\n \t\tif (!sub)\n \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n-\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n-\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n-\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n-\t\treal_new_git_dir = real_pathdup(new_git_dir);\n-\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n-\n-\t\tfree(real_new_git_dir);\n+\t\tconnect_work_tree_and_git_dir(path,\n+\t\t\tgit_path(\"modules/%s\", sub->name));\n \t} else {\n \t\t/* Is it already absorbed into the superprojects git dir? */\n \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312464","messageId":"20170223225735.10994-8-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 07/15] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:27Z","receivedAt":"2017-02-23T23:06:37Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 04d185738f..591f4a694e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -542,6 +543,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t      int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 0b915bd3ac..b4e60c08d2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -64,6 +64,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t       const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312466","messageId":"20170223225735.10994-11-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 10/15] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:30Z","receivedAt":"2017-02-23T23:07:05Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n  \"Is it safe to change the submodule to this new state?\"\n  e.g. is it overwriting untracked files or are there local\n  changes that would be overwritten?\n* supports a force flag that can be used for resetting\n  the tree.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h |   7 ++++\n 2 files changed, 142 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 0b2596e88a..a2cf8c9376 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n+\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.no_stdout = 1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n+\n+\treturn finish_command(&cp);\n+}\n+\n+void submodule_reset_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not reset submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ */\n+int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub))\n+\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t    get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_reset_index(path);\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n+\t\targv_array_push(&cp.args, \"-n\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-u\");\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 6f3fe85c7c..4cdf6445f7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -96,6 +96,13 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n+#define SUBMODULE_MOVE_HEAD_FORCE   (1<<1)\n+extern int submodule_move_head(const char *path,\n+\t\t\t       const char *old,\n+\t\t\t       const char *new,\n+\t\t\t       unsigned flags);\n+\n /*\n  * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n  * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312467","messageId":"20170223225735.10994-14-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 13/15] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:33Z","receivedAt":"2017-02-23T23:07:25Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n read-cache.c | 27 +++++++++++++++++++++++++--\n 1 file changed, 25 insertions(+), 2 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 9054369dd0..9a2abacf7a 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -18,6 +18,8 @@\n #include \"varint.h\"\n #include \"split-index.h\"\n #include \"utf8.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -520,6 +522,22 @@ int remove_index_entry_at(struct index_state *istate, int pos)\n \treturn 1;\n }\n \n+static void remove_submodule_according_to_strategy(const struct submodule *sub)\n+{\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\t\tsubmodule_move_head(sub->path, \"HEAD\", NULL, \\\n+\t\t\t\t    SUBMODULE_MOVE_HEAD_FORCE);\n+\t\tbreak;\n+\tcase SM_UPDATE_NONE:\n+\tcase SM_UPDATE_COMMAND:\n+\t\t; /* Do not touch the submodule. */\n+\t}\n+}\n+\n /*\n  * Remove all cache entries marked for removal, that is where\n  * CE_REMOVE is set in ce_flags.  This is much more effective than\n@@ -532,8 +550,13 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n-\t\t\tremove_name_hash(istate, ce_array[i]);\n-\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\tconst struct submodule *sub = submodule_from_ce(ce_array[i]);\n+\t\t\tif (sub) {\n+\t\t\t\tremove_submodule_according_to_strategy(sub);\n+\t\t\t} else {\n+\t\t\t\tremove_name_hash(istate, ce_array[i]);\n+\t\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\t}\n \t\t}\n \t\telse\n \t\t\tce_array[j++] = ce_array[i];\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312468","messageId":"20170223225735.10994-16-sbeller@google.com","threadId":"45144","inReplyTo":"20170223225735.10994-1-sbeller@google.com","subject":"[PATCH 15/15] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-23T22:57:35Z","receivedAt":"2017-02-23T23:07:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n Documentation/git-checkout.txt |  7 +++++++\n builtin/checkout.c             | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh      | 33 ++++++++++++++++++++++++---------\n t/t2013-checkout-submodule.sh  |  5 +++++\n 4 files changed, 64 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..d6399c0af8 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n <branch>::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..207ce09771 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [<options>] <branch>\"),\n \tN_(\"git checkout [<options>] [<branch>] -- <file>...\"),\n \tNULL,\n };\n \n+int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t    const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t    arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout <no-such-branch>'\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t    \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t    PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 0b26f0e20f..54cd8a6366 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -756,6 +756,11 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -865,7 +870,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -877,7 +882,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -905,7 +910,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -961,16 +966,21 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n+\t# This test fails, due to missing setup, we do not clone sub2 into\n+\t# submodule_update, because it doesn't exist in the 'add_sub1' version\n+\t#\n \ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n-\t\t\t$command modify_sub1_recursively &&\n-\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n-\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively\n-\t\t\ttest_submodule_content sub1/sub2\n+\t\t\ttest_must_fail $command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t\t# test_superproject_content origin/modify_sub1_recursively &&\n+\t\t\t# test_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\t# test_submodule_content sub1/sub2 no_submodule\n \t\t)\n \t'\n }\n@@ -980,6 +990,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1074,7 +1089,7 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -1125,7 +1140,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..aa35223369 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,11 @@ test_expect_success '\"checkout <submodule>\" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc1.16.ge4278d41a0.dirty\n\n"},{"id":"312496","messageId":"7e920dd5-3638-57d9-0ca1-3f8aed4d700b@ramsayjones.plus.com","threadId":"45144","inReplyTo":"20170223225735.10994-11-sbeller@google.com","subject":"Re: [PATCH 10/15] update submodules: add submodule_move_head","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2017-02-24T01:21:43Z","receivedAt":"2017-02-24T01:21:51Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 23/02/17 22:57, Stefan Beller wrote:\n> In later patches we introduce the options and flag for commands\n> that modify the working directory, e.g. git-checkout.\n> \n> This piece of code will be used universally for\n> all these working tree modifications as it\n> * supports dry run to answer the question:\n>   \"Is it safe to change the submodule to this new state?\"\n>   e.g. is it overwriting untracked files or are there local\n>   changes that would be overwritten?\n> * supports a force flag that can be used for resetting\n>   the tree.\n> \n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n>  submodule.h |   7 ++++\n>  2 files changed, 142 insertions(+)\n> \n> diff --git a/submodule.c b/submodule.c\n> index 0b2596e88a..a2cf8c9376 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n>  \treturn ret;\n>  }\n>  \n> +static int submodule_has_dirty_index(const struct submodule *sub)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n> +\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n> +\tcp.no_stdin = 1;\n> +\tcp.no_stdout = 1;\n> +\tcp.dir = sub->path;\n> +\tif (start_command(&cp))\n> +\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n> +\n> +\treturn finish_command(&cp);\n> +}\n> +\n> +void submodule_reset_index(const char *path)\n\nI was just about to send a patch against the previous series\n(in pu branch last night), but since you have sent another\nversion ...\n\nIn the last series this was called 'submodule_clean_index()'\nand, since it is a file-local symbol, should be marked with\nstatic. I haven't applied these patches to check, but the\ninterdiff in the cover letter leads me to believe that this\nwill also apply to the renamed function.\n\n[The patch subject was also slightly different.]\n\nATB,\nRamsay Jones\n\n"},{"id":"312497","messageId":"db5cfde2-a769-d786-8846-a5a526194b69@ramsayjones.plus.com","threadId":"45144","inReplyTo":"20170223225735.10994-16-sbeller@google.com","subject":"Re: [PATCH 15/15] builtin/checkout: add --recurse-submodules switch","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2017-02-24T01:25:34Z","receivedAt":"2017-02-24T01:25:42Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 23/02/17 22:57, Stefan Beller wrote:\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  Documentation/git-checkout.txt |  7 +++++++\n>  builtin/checkout.c             | 28 ++++++++++++++++++++++++++++\n>  t/lib-submodule-update.sh      | 33 ++++++++++++++++++++++++---------\n>  t/t2013-checkout-submodule.sh  |  5 +++++\n>  4 files changed, 64 insertions(+), 9 deletions(-)\n> \n> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\n> index 8e2c0662dd..d6399c0af8 100644\n> --- a/Documentation/git-checkout.txt\n> +++ b/Documentation/git-checkout.txt\n> @@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n>  \tout anyway. In other words, the ref can be held by more than one\n>  \tworktree.\n>  \n> +--[no-]recurse-submodules::\n> +\tUsing --recurse-submodules will update the content of all initialized\n> +\tsubmodules according to the commit recorded in the superproject. If\n> +\tlocal modifications in a submodule would be overwritten the checkout\n> +\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n> +\tis used, the work trees of submodules will not be updated.\n> +\n>  <branch>::\n>  \tBranch to checkout; if it refers to a branch (i.e., a name that,\n>  \twhen prepended with \"refs/heads/\", is a valid ref), then that\n> diff --git a/builtin/checkout.c b/builtin/checkout.c\n> index f174f50303..207ce09771 100644\n> --- a/builtin/checkout.c\n> +++ b/builtin/checkout.c\n> @@ -21,12 +21,31 @@\n>  #include \"submodule-config.h\"\n>  #include \"submodule.h\"\n>  \n> +static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n> +\n>  static const char * const checkout_usage[] = {\n>  \tN_(\"git checkout [<options>] <branch>\"),\n>  \tN_(\"git checkout [<options>] [<branch>] -- <file>...\"),\n>  \tNULL,\n>  };\n>  \n> +int option_parse_recurse_submodules(const struct option *opt,\n> +\t\t\t\t    const char *arg, int unset)\n\nAgain, this function should be marked static.\n\n[I also noted _two_ other local functions with the same name\nin builtin/fetch.c and builtin/push.c]\n\nATB,\nRamsay Jones\n\n\n"},{"id":"312535","messageId":"CAGZ79kZX=ZmHDbBhgApHTKhyXz489V5XQXPvUhfLyRjYhTfVJA@mail.gmail.com","threadId":"45144","inReplyTo":"7e920dd5-3638-57d9-0ca1-3f8aed4d700b@ramsayjones.plus.com","subject":"Re: [PATCH 10/15] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-24T19:08:12Z","receivedAt":"2017-02-24T19:09:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 23, 2017 at 5:21 PM, Ramsay Jones\n<ramsay@ramsayjones.plus.com> wrote:\n>\n>\n> On 23/02/17 22:57, Stefan Beller wrote:\n>> In later patches we introduce the options and flag for commands\n>> that modify the working directory, e.g. git-checkout.\n>>\n>> This piece of code will be used universally for\n>> all these working tree modifications as it\n>> * supports dry run to answer the question:\n>>   \"Is it safe to change the submodule to this new state?\"\n>>   e.g. is it overwriting untracked files or are there local\n>>   changes that would be overwritten?\n>> * supports a force flag that can be used for resetting\n>>   the tree.\n>>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>>  submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n>>  submodule.h |   7 ++++\n>>  2 files changed, 142 insertions(+)\n>>\n>> diff --git a/submodule.c b/submodule.c\n>> index 0b2596e88a..a2cf8c9376 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n>>       return ret;\n>>  }\n>>\n>> +static int submodule_has_dirty_index(const struct submodule *sub)\n>> +{\n>> +     struct child_process cp = CHILD_PROCESS_INIT;\n>> +\n>> +     prepare_submodule_repo_env_no_git_dir(&cp.env_array);\n>> +\n>> +     cp.git_cmd = 1;\n>> +     argv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n>> +                                     \"--cached\", \"HEAD\", NULL);\n>> +     cp.no_stdin = 1;\n>> +     cp.no_stdout = 1;\n>> +     cp.dir = sub->path;\n>> +     if (start_command(&cp))\n>> +             die(\"could not recurse into submodule '%s'\", sub->path);\n>> +\n>> +     return finish_command(&cp);\n>> +}\n>> +\n>> +void submodule_reset_index(const char *path)\n>\n> I was just about to send a patch against the previous series\n> (in pu branch last night), but since you have sent another\n> version ...\n>\n> In the last series this was called 'submodule_clean_index()'\n> and, since it is a file-local symbol, should be marked with\n> static. I haven't applied these patches to check, but the\n> interdiff in the cover letter leads me to believe that this\n> will also apply to the renamed function.\n>\n> [The patch subject was also slightly different.]\n>\n\ngood catch. Yes submodule_reset_index\nought to be static.\n\nfixed in a reroll.\n"},{"id":"312537","messageId":"CAGZ79kbD8SBHURQJvrNKKP8Ycay8wZcwUGw7V4mmo6PmfmJ9=w@mail.gmail.com","threadId":"45144","inReplyTo":"db5cfde2-a769-d786-8846-a5a526194b69@ramsayjones.plus.com","subject":"Re: [PATCH 15/15] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-02-24T19:20:47Z","receivedAt":"2017-02-24T19:21:23Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 23, 2017 at 5:25 PM, Ramsay Jones\n<ramsay@ramsayjones.plus.com> wrote:\n>> +int option_parse_recurse_submodules(const struct option *opt,\n>> +                                 const char *arg, int unset)\n>\n> Again, this function should be marked static.\n>\n> [I also noted _two_ other local functions with the same name\n> in builtin/fetch.c and builtin/push.c]\n\nfixed in a reroll.\n\nYes there is a pattern here. But as both fetch and push accept different\noptions (not just boolean, but strings) these have to be different.\nI thought about unifying them, but I do not think we can do so easily.\n\nThanks,\nStefan\n"}]}