{"thread":{"id":"45260","subject":"[PATCH 09/18] update submodules: add a config option to determine if submodules are updated","startedAt":"2017-03-02T00:54:44Z","lastAt":"2017-03-15T00:42:04Z","messageCount":100,"participants":["Stefan Beller","Eric Wong","Junio C Hamano","Brandon Williams"],"isPatch":true,"patchVersion":1,"patchTotal":18},"messages":[{"id":"313012","messageId":"20170302004759.27852-10-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 09/18] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:50Z","receivedAt":"2017-03-02T00:54:44Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 04d185738f..591f4a694e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -542,6 +543,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 0b915bd3ac..b4e60c08d2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -64,6 +64,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313013","messageId":"20170302004759.27852-3-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 02/18] lib-submodule-update.sh: do not use ./. as submodule remote","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:43Z","receivedAt":"2017-03-02T00:54:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Adding the repository itself as a submodule does not make sense in the\nreal world. In our test suite we used to do that out of convenience in\nsome tests as the current repository has easiest access for setting up\n'just a submodule'.\n\nHowever this doesn't quite test the real world, so let's do not follow\nthis pattern any further and actually create an independent repository\nthat we can use as a submodule.\n\nWhen using './.' as the remote the superproject and submodule share the\nsame objects, such that testing if a given sha1 is a valid commit works\nin either repository. As running commands in an unpopulated submodule\nfall back to the superproject, this happens in `reset_work_tree_to`\nto determine if we need to populate the submodule. Fix this bug by\nchecking in the actual remote now.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 15 +++++++++++++--\n 1 file changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5df528ea81..c0d6325133 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -37,6 +37,17 @@\n #\n \n create_lib_submodule_repo () {\n+\tgit init submodule_update_sub1 &&\n+\t(\n+\t\tcd submodule_update_sub1 &&\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"Base inside first submodule\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -49,7 +60,7 @@ create_lib_submodule_repo () {\n \t\tgit branch \"no_submodule\" &&\n \n \t\tgit checkout -b \"add_sub1\" &&\n-\t\tgit submodule add ./. sub1 &&\n+\t\tgit submodule add ../submodule_update_sub1 sub1 &&\n \t\tgit config -f .gitmodules submodule.sub1.ignore all &&\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n@@ -162,7 +173,7 @@ reset_work_tree_to () {\n \t\ttest_must_be_empty actual &&\n \t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n \t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313014","messageId":"20170302004759.27852-11-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 10/18] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:51Z","receivedAt":"2017-03-02T00:55:01Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that checks if a submodule needs\nto be checked for an update before attempting the update.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 16 ++++++++++++++++\n submodule.h | 7 +++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 591f4a694e..8b2c0212be 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -548,6 +548,22 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int should_update_submodules(void)\n+{\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n+{\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn NULL;\n+\n+\tif (!should_update_submodules())\n+\t\treturn NULL;\n+\n+\treturn submodule_from_path(null_sha1, ce->name);\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex b4e60c08d2..6f3fe85c7c 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,6 +65,13 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+/* Check if we want to update any submodule.*/\n+extern int should_update_submodules(void);\n+/*\n+ * Returns the submodule struct if the given ce entry is a submodule\n+ * and it should be updated. Returns NULL otherwise.\n+ */\n+extern const struct submodule *submodule_from_ce(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313015","messageId":"20170302004759.27852-13-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 12/18] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:53Z","receivedAt":"2017-03-02T00:55:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n \"Is it safe to change the submodule to this new state?\"\n e.g. is it overwriting untracked files or are there local\n changes that would be overwritten?\n* supports a force flag that can be used for resetting\n the tree.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h | 7 ++++\n 2 files changed, 142 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 0b2596e88a..bc5fecf8c5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n+\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.no_stdout = 1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n+\n+\treturn finish_command(&cp);\n+}\n+\n+static void submodule_reset_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not reset submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ */\n+int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub))\n+\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_reset_index(path);\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n+\t\targv_array_push(&cp.args, \"-n\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-u\");\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 6f3fe85c7c..4cdf6445f7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -96,6 +96,13 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n+#define SUBMODULE_MOVE_HEAD_FORCE (1<<1)\n+extern int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags);\n+\n /*\n * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313016","messageId":"20170302004759.27852-9-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 08/18] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:49Z","receivedAt":"2017-03-02T00:56:54Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller \n---\n submodule-config.c | 20 ++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 29 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..3e8e380d98 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,26 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t unsigned char *gitmodules_sha1,\n \t\t\t\t struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313017","messageId":"20170302004759.27852-5-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 04/18] lib-submodule-update.sh: replace sha1 by hash","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:45Z","receivedAt":"2017-03-02T00:56:55Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Cleaning up code by generalising it.\nCurrently the mailing list discusses yet again how\nto migrate away from sha1.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex a906c92cfb..83202c54fc 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -171,9 +171,9 @@ reset_work_tree_to () {\n \t\tgit checkout -f \"$1\" &&\n \t\tgit status -u -s >actual &&\n \t\ttest_must_be_empty actual &&\n-\t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n-\t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\thash=$(git rev-parse --revs-only HEAD:sub1) &&\n+\t\tif test -n \"$hash\" &&\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$hash^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313018","messageId":"20170302004759.27852-17-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 16/18] entry.c: update submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:57Z","receivedAt":"2017-03-02T00:56:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n entry.c | 30 ++++++++++++++++++++++++++++++\n 1 file changed, 30 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..d2b512da90 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t const struct checkout *state)\n@@ -146,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n \tunsigned long size;\n \tsize_t wrote, newsize = 0;\n \tstruct stat st;\n+\tconst struct submodule *sub;\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n@@ -203,6 +205,10 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub)\n+\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -259,7 +265,31 @@ int checkout_entry(struct cache_entry *ce,\n \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n+\t\tconst struct submodule *sub;\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\t} else\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313019","messageId":"20170302004759.27852-7-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 06/18] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:47Z","receivedAt":"2017-03-02T00:57:05Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller \n---\n builtin/grep.c | 2 +-\n submodule.c | 7 ++-----\n submodule.h | 8 +++++++-\n 3 files changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..0e55372f37 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -234,15 +234,12 @@ int is_submodule_initialized(const char *path)\n \treturn ret;\n }\n \n-/*\n- * Determine if a submodule has been populated at a given 'path'\n- */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..0b915bd3ac 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,13 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+/*\n+ * Determine if a submodule has been populated at a given 'path' by checking if\n+ * the /.git resolves to a valid git repository.\n+ * If return_error_code is NULL, die on error.\n+ * Otherwise the return error code is the same as of resolve_gitdir_gently.\n+ */\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313020","messageId":"20170302004759.27852-8-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 07/18] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:48Z","receivedAt":"2017-03-02T00:57:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. This patch makes\nconnect_work_tree_and_git_dir safe to use for both cases of\neither the git dir or the working dir missing.\n\nTo do so, we need to call safe_create_leading_directories[_const]\non both directories. However this has to happen before we construct\nthe absolute paths as real_pathdup assumes the directories to\nbe there already.\n\nSo for both the config file in the git dir as well as the .git link\nfile we need to\na) construct the name\nb) call SCLD\nc) get the absolute path\nd) once a-c is done for both we can consume the absolute path\n to compute the relative path to each other and store those\n relative paths.\n\nThe implementation provided here puts a) and b) for both cases first,\nand then performs c and d after.\n\nOne of the two users of 'connect_work_tree_and_git_dir' already checked\nfor the directory being there, so we can loose that check as\nconnect_work_tree_and_git_dir handles this functionality now.\n\nSigned-off-by: Stefan Beller \n---\n dir.c | 32 +++++++++++++++++++++-----------\n submodule.c | 11 ++---------\n 2 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..6f52af7abb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n /* Update gitfile and core.worktree setting to connect work tree and git dir */\n void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n {\n-\tstruct strbuf file_name = STRBUF_INIT;\n+\tstruct strbuf gitfile_sb = STRBUF_INIT;\n+\tstruct strbuf cfg_sb = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n-\tchar *git_dir = real_pathdup(git_dir_);\n-\tchar *work_tree = real_pathdup(work_tree_);\n+\tchar *git_dir, *work_tree;\n \n-\t/* Update gitfile */\n-\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n-\twrite_file(file_name.buf, \"gitdir: %s\",\n-\t\t relative_path(git_dir, work_tree, &rel_path));\n+\t/* Prepare .git file */\n+\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n+\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n+\n+\t/* Prepare config file */\n+\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n+\tif (safe_create_leading_directories_const(cfg_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n \n+\tgit_dir = real_pathdup(git_dir_);\n+\twork_tree = real_pathdup(work_tree_);\n+\n+\t/* Write .git file */\n+\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n+\t\t relative_path(git_dir, work_tree, &rel_path));\n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n+\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n \t\t\t relative_path(work_tree, git_dir, &rel_path));\n \n-\tstrbuf_release(&file_name);\n+\tstrbuf_release(&gitfile_sb);\n+\tstrbuf_release(&cfg_sb);\n \tstrbuf_release(&rel_path);\n \tfree(work_tree);\n \tfree(git_dir);\ndiff --git a/submodule.c b/submodule.c\nindex 0e55372f37..04d185738f 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1442,8 +1442,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \n \t/* Not populated? */\n \tif (!sub_git_dir) {\n-\t\tchar *real_new_git_dir;\n-\t\tconst char *new_git_dir;\n \t\tconst struct submodule *sub;\n \n \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n@@ -1466,13 +1464,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tsub = submodule_from_path(null_sha1, path);\n \t\tif (!sub)\n \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n-\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n-\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n-\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n-\t\treal_new_git_dir = real_pathdup(new_git_dir);\n-\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n-\n-\t\tfree(real_new_git_dir);\n+\t\tconnect_work_tree_and_git_dir(path,\n+\t\t\tgit_path(\"modules/%s\", sub->name));\n \t} else {\n \t\t/* Is it already absorbed into the superprojects git dir? */\n \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313021","messageId":"20170302004759.27852-18-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 17/18] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:58Z","receivedAt":"2017-03-02T00:57:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n Documentation/git-checkout.txt | 7 +++++++\n builtin/checkout.c | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 23 +++++++++++++++++------\n t/t2013-checkout-submodule.sh | 5 +++++\n 4 files changed, 57 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..d6399c0af8 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n ::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..e9c5fcfaf8 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [] \"),\n \tN_(\"git checkout [] [] -- ...\"),\n \tNULL,\n };\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout '\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex aa196d4d42..d71972c3db 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -782,6 +782,11 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -891,7 +896,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -903,7 +908,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -931,7 +936,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -987,7 +992,8 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t# recursing deeper than one level doesn't work yet.\n+\ttest_expect_failure \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n@@ -1006,6 +1012,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1100,7 +1111,7 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -1151,7 +1162,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..aa35223369 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,11 @@ test_expect_success '\"checkout \" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313022","messageId":"20170302004759.27852-19-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 18/18] builtin/read-tree: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:59Z","receivedAt":"2017-03-02T00:57:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"A new known failure mode is introduced[1], which is actually not\na failure but a feature in read-tree. Unlike checkout for which\nthe recursive submodule tests were originally written, read-tree does\nwarn about ignored untracked files that would be overwritten.\nFor the sake of keeping the test library for submodules genric, just\nmark the test as a failure.\n\n[1] KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-read-tree.txt | 6 ++++++\n builtin/read-tree.c | 29 +++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 7 ++++++-\n t/t1013-read-tree-submodule.sh | 7 +++++++\n 4 files changed, 48 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt\nindex fa1d557e5b..ed9d63ef4a 100644\n--- a/Documentation/git-read-tree.txt\n+++ b/Documentation/git-read-tree.txt\n@@ -115,6 +115,12 @@ OPTIONS\n \tdirectories the index file and index output file are\n \tlocated in.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject by\n+\tcalling read-tree recursively, also setting the submodules HEAD to be\n+\tdetached at that commit.\n+\n --no-sparse-checkout::\n \tDisable sparse checkout support even if `core.sparseCheckout`\n \tis true.\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 8ba64bc785..2dc5cc06da 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -15,10 +15,13 @@\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"resolve-undo.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n+int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\n@@ -96,6 +99,23 @@ static int debug_merge(const struct cache_entry * const *stages,\n \treturn 0;\n }\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n static struct lock_file lock_file;\n \n int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n@@ -137,6 +157,9 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n \t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_END()\n \t};\n \n@@ -152,6 +175,12 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \n \thold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT) {\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tset_config_update_recurse_submodules(RECURSE_SUBMODULES_ON);\n+\t}\n+\n \tprefix_set = opts.prefix ? 1 : 0;\n \tif (1 < opts.merge + opts.reset + prefix_set)\n \t\tdie(\"Which one? -m, --reset, or --prefix?\");\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex d71972c3db..5caae06bc5 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -787,6 +787,11 @@ test_submodule_switch_recursing () {\n \tthen\n \t\tRESULT=failure\n \tfi\n+\tRESULT1=success\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n+\tthen\n+\t\tRESULT1=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -827,7 +832,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULT1 \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 20526aed34..7019d0a04f 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,13 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n+\n+test_submodule_switch_recursing \"git read-tree --recurse-submodules -u -m\"\n+\n+test_submodule_forced_switch_recursing \"git read-tree --recurse-submodules -u --reset\"\n+\n test_submodule_switch \"git read-tree -u -m\"\n \n test_submodule_forced_switch \"git read-tree -u --reset\"\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313024","messageId":"20170302004759.27852-14-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 13/18] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:54Z","receivedAt":"2017-03-02T01:18:32Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller \n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n * Currently, git does not checkout subprojects during a superproject\n * checkout, so it is not going to overwrite anything.\n */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t const struct cache_entry *ce,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313025","messageId":"20170302004759.27852-16-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:56Z","receivedAt":"2017-03-02T01:43:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n read-cache.c | 27 +++++++++++++++++++++++++--\n 1 file changed, 25 insertions(+), 2 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 9054369dd0..9a2abacf7a 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -18,6 +18,8 @@\n #include \"varint.h\"\n #include \"split-index.h\"\n #include \"utf8.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -520,6 +522,22 @@ int remove_index_entry_at(struct index_state *istate, int pos)\n \treturn 1;\n }\n \n+static void remove_submodule_according_to_strategy(const struct submodule *sub)\n+{\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\t\tsubmodule_move_head(sub->path, \"HEAD\", NULL, \\\n+\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n+\t\tbreak;\n+\tcase SM_UPDATE_NONE:\n+\tcase SM_UPDATE_COMMAND:\n+\t\t; /* Do not touch the submodule. */\n+\t}\n+}\n+\n /*\n * Remove all cache entries marked for removal, that is where\n * CE_REMOVE is set in ce_flags. This is much more effective than\n@@ -532,8 +550,13 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n-\t\t\tremove_name_hash(istate, ce_array[i]);\n-\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\tconst struct submodule *sub = submodule_from_ce(ce_array[i]);\n+\t\t\tif (sub) {\n+\t\t\t\tremove_submodule_according_to_strategy(sub);\n+\t\t\t} else {\n+\t\t\t\tremove_name_hash(istate, ce_array[i]);\n+\t\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\t}\n \t\t}\n \t\telse\n \t\t\tce_array[j++] = ce_array[i];\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313027","messageId":"20170302004759.27852-12-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 11/18] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:52Z","receivedAt":"2017-03-02T01:59:01Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 8b2c0212be..0b2596e88a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -356,6 +356,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n * summary output. If it can locate the submodule objects directory it will\n * attempt to lookup both the left and right commits and put them into the\n@@ -1390,18 +1407,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n * Embeds a single submodules git directory into the superprojects git dir,\n * non recursively.\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313028","messageId":"20170302004759.27852-4-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 03/18] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:44Z","receivedAt":"2017-03-02T02:00:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex c0d6325133..a906c92cfb 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -193,6 +193,11 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n+\tif test \"$1\" == \"-C\"\n+\tthen\n+\t\tcd \"$2\"\n+\t\tshift; shift;\n+\tfi\n \tif test $# != 2\n \tthen\n \t\techo \"test_submodule_content needs two arguments\"\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313029","messageId":"20170302021135.GA26453@starla","threadId":"45260","inReplyTo":"20170302004759.27852-4-sbeller@google.com","subject":"Re: [PATCH 03/18] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Eric Wong","fromEmail":"e@80x24.org","sentAt":"2017-03-02T02:11:35Z","receivedAt":"2017-03-02T02:36:25Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Stefan Beller wrote:\n> test_submodule_content () {\n> +\tif test \"$1\" == \"-C\"\n\nUse a single '=' for portability in sh. It's also a good idea\nto prefix variables with 'x' or some such, since \"$1\" could be\n\"-n\" or some other supported switch for test(1).\nSo, something like:\n\n\tif test x\"$1\" = \"x-C\"\n\n...or use a case statement.\n\nOn Debian systems, I use the \"checkbashisms\" from the\n\"devscripts\" package to find and avoid bash dependencies.\n\nThanks.\n"},{"id":"313031","messageId":"20170302004759.27852-1-sbeller@google.com","threadId":"45260","inReplyTo":"20170223225735.10994-1-sbeller@google.com/","subject":"[RFCv6 PATCH 00/18] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:41Z","receivedAt":"2017-03-02T03:34:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20170223225735.10994-1-sbeller@google.com/\nhttps://public-inbox.org/git/20161203003022.29797-1-sbeller@google.com/\n\nv6:\n * added support for read-tree (see last patch) to see how generic the\n code of the previous patches is. I am pretty pleased with that patch.\n * marked two functions static. Thanks Ramsay!\n * fixed the recursive test; it still fails but it is the code that fails,\n not the test. For this I had to change the setup code slightly.\n * 2 new patches adding tiny refactoring to the submodule test lib. \n * interdiff (to origin/sb/checkout-recurse-submodules, which is v5) below.\n\nv5:\n * as v4 was the first version queued by Junio, we do have an interdiff below!\n * renamed functions\n * changed the API, now the caller has to take care of the submodule strategy\n themselves. (Note this can be different for different situations, e.g.\n when a submodule is deleted, we can do that for any strategy except none and\n !command. But for moving to a new state of the submodule we currently\n only implement \"checkout\" [unspecified defaults to checkout]. warning about\n the others, doing nothing there.)\n\nv4:\n * addressed all comments of Brian, Junio and Brandon.\n Thanks!\n * one major point of change is the introduction of another patch\n \"lib-submodule-update.sh: do not use ./. as submodule remote\",\n as that took some time to track down the existing bug.\n \nv3:\n * moved tests from t2013 to the generic submodule library.\n * factored out the refactoring patches to be up front\n * As I redid the complete implementation, I have the impression this time\n it is cleaner than previous versions.\n \n I think we still have to fix the corner cases of directory/file/submodule \n conflicts before merging, but this serves as a status update on my current\n way of thinking how to implement the worktree commands being aware of\n submodules.\n \nThanks,\nStefan\n\nv2:\n* based on top of the series sent out an hour ago\n \"[PATCHv4 0/5] submodule embedgitdirs\"\n* Try to embed a submodule if we need to remove it.\n* Strictly do not change behavior if not giving the new flag.\n* I think I missed some review comments from v1, but I'd like to get\n the current state out over the weekend, as a lot has changed so far.\n On Monday I'll go through the previous discussion with a comb to see\n if I missed something.\n \nv1:\nWhen working with submodules, nearly anytime after checking out\na different state of the projects, that has submodules changed\nyou'd run \"git submodule update\" with a current version of Git.\n\nThere are two problems with this approach:\n\n* The \"submodule update\" command is dangerous as it\n doesn't check for work that may be lost in the submodule\n (e.g. a dangling commit).\n* you may forget to run the command as checkout is supposed\n to do all the work for you.\n\nIntegrate updating the submodules into git checkout, with the same\nsafety promises that git-checkout has, i.e. not throw away data unless\nasked to. This is done by first checking if the submodule is at the same\nsha1 as it is recorded in the superproject. If there are changes we stop\nproceeding the checkout just like it is when checking out a file that\nhas local changes.\n\nThe integration happens in the code that is also used in other commands\nsuch that it will be easier in the future to make other commands aware\nof submodule.\n\nThis also solves d/f conflicts in case you replace a file/directory\nwith a submodule or vice versa.\n\nThe patches are still a bit rough, but the overall series seems\npromising enough to me that I want to put it out here.\n\nAny review, specifically on the design level welcome!\n\nThanks,\nStefan\n\n\nStefan Beller (14):\n lib-submodule-update.sh: reorder create_lib_submodule_repo\n lib-submodule-update.sh: define tests for recursing into submodules\n make is_submodule_populated gently\n connect_work_tree_and_git_dir: safely create leading directories\n update submodules: add submodule config parsing\n update submodules: add a config option to determine if submodules are\n updated\n update submodules: introduce is_interesting_submodule\n update submodules: move up prepare_submodule_repo_env\n update submodules: add submodule_go_from_to\n unpack-trees: pass old oid to verify_clean_submodule\n unpack-trees: check if we can perform the operation for submodules\n read-cache: remove_marked_cache_entries to wipe selected submodules.\n entry.c: update submodules when interesting\n builtin/checkout: add --recurse-submodules switch\ndiff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt\nindex fa1d557e5b..ed9d63ef4a 100644\n--- a/Documentation/git-read-tree.txt\n+++ b/Documentation/git-read-tree.txt\n@@ -115,6 +115,12 @@ OPTIONS\n \tdirectories the index file and index output file are\n \tlocated in.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject by\n+\tcalling read-tree recursively, also setting the submodules HEAD to be\n+\tdetached at that commit.\n+\n --no-sparse-checkout::\n \tDisable sparse checkout support even if `core.sparseCheckout`\n \tis true.\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 207ce09771..e9c5fcfaf8 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -29,8 +29,8 @@ static const char * const checkout_usage[] = {\n \tNULL,\n };\n \n-int option_parse_recurse_submodules(const struct option *opt,\n-\t\t\t\t const char *arg, int unset)\n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n {\n \tif (unset) {\n \t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 8ba64bc785..2dc5cc06da 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -15,10 +15,13 @@\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"resolve-undo.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n+int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\n@@ -96,6 +99,23 @@ static int debug_merge(const struct cache_entry * const *stages,\n \treturn 0;\n }\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n static struct lock_file lock_file;\n \n int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n@@ -137,6 +157,9 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n \t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_END()\n \t};\n \n@@ -152,6 +175,12 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \n \thold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT) {\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tset_config_update_recurse_submodules(RECURSE_SUBMODULES_ON);\n+\t}\n+\n \tprefix_set = opts.prefix ? 1 : 0;\n \tif (1 < opts.merge + opts.reset + prefix_set)\n \t\tdie(\"Which one? -m, --reset, or --prefix?\");\ndiff --git a/submodule.c b/submodule.c\nindex a2cf8c9376..bc5fecf8c5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1257,7 +1257,7 @@ static int submodule_has_dirty_index(const struct submodule *sub)\n \treturn finish_command(&cp);\n }\n \n-void submodule_reset_index(const char *path)\n+static void submodule_reset_index(const char *path)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 54cd8a6366..429cd1041d 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,7 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n-# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n+# - Updated submodule recursively (add_nested_sub => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -20,8 +20,8 @@\n # / ^\n # / remove_sub1\n # /\n-# add_sub1 /-------O---------O\n-# | / ^ modify_sub1_recursive\n+# add_sub1 /-------O---------O--------O modify_sub1_recursively\n+# | / ^ add_nested_sub\n # | / modify_sub1\n # v/\n # O------O-----------O---------O\n@@ -96,12 +96,24 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b modify_sub1_recursively modify_sub1 &&\n+\t\tgit checkout -b add_nested_sub modify_sub1 &&\n \t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n \t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n \t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n \t\tgit add sub1 &&\n \t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1 checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 checkout -b modify_sub1_recursively &&\n+\t\techo change >sub1/sub2/file3 &&\n+\t\tgit -C sub1/sub2 add file3 &&\n+\t\tgit -C sub1/sub2 commit -m \"make a change in nested sub\" &&\n+\t\tgit -C sub1 add sub2 &&\n+\t\tgit -C sub1 commit -m \"update nested sub\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -m \"update sub1, that updates nested sub\" &&\n+\t\tgit -C sub1 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 push origin modify_sub1_recursively &&\n \t\tgit -C sub1 submodule deinit -f --all &&\n \n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n@@ -200,9 +212,9 @@ reset_work_tree_to () {\n \t\tgit checkout -f \"$1\" &&\n \t\tgit status -u -s >actual &&\n \t\ttest_must_be_empty actual &&\n-\t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n-\t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\thash=$(git rev-parse --revs-only HEAD:sub1) &&\n+\t\tif test -n \"$hash\" &&\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$hash^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n@@ -211,14 +223,23 @@ reset_work_tree_to () {\n \n reset_work_tree_to_interested () {\n \treset_work_tree_to $1 &&\n-\t# indicate we are interested in the submodule:\n-\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n-\t# also have it available:\n+\t# make the submodule git dirs available\n \tif ! test -d submodule_update/.git/modules/sub1\n \tthen\n \t\tmkdir -p submodule_update/.git/modules &&\n \t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n-\tfi\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree\n+\tfi &&\n+\tif ! test -d submodule_update/.git/modules/sub1/modules/sub2\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules/sub1/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree\n+\tfi &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# sub1 might not be checked out, so use the git dir\n+\tgit -C submodule_update/.git/modules/sub1 config submodule.sub2.url \"bogus\"\n }\n \n # Test that the superproject contains the content according to commit \"$1\"\n@@ -234,6 +255,11 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n+\tif test \"$1\" == \"-C\"\n+\tthen\n+\t\tcd \"$2\"\n+\t\tshift; shift;\n+\tfi\n \tif test $# != 2\n \tthen\n \t\techo \"test_submodule_content needs two arguments\"\n@@ -761,6 +787,11 @@ test_submodule_switch_recursing () {\n \tthen\n \t\tRESULT=failure\n \tfi\n+\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n+\tthen\n+\t\tRESULT1=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -801,7 +832,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULT1 \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\n@@ -966,21 +997,17 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\t# This test fails, due to missing setup, we do not clone sub2 into\n-\t# submodule_update, because it doesn't exist in the 'add_sub1' version\n-\t#\n-\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t# recursing deeper than one level doesn't work yet.\n+\ttest_expect_failure \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n-\t\treset_work_tree_to_interested add_sub1 &&\n+\t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n-\t\t\ttest_must_fail $command modify_sub1_recursively &&\n-\t\t\ttest_superproject_content origin/add_sub1 &&\n-\t\t\ttest_submodule_content sub1 origin/add_sub1\n-\t\t\t# test_superproject_content origin/modify_sub1_recursively &&\n-\t\t\t# test_submodule_content sub1 origin/modify_sub1_recursively &&\n-\t\t\t# test_submodule_content sub1/sub2 no_submodule\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content -C sub1 sub2 origin/modify_sub1_recursively\n \t\t)\n \t'\n }\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 20526aed34..7019d0a04f 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,13 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n+\n+test_submodule_switch_recursing \"git read-tree --recurse-submodules -u -m\"\n+\n+test_submodule_forced_switch_recursing \"git read-tree --recurse-submodules -u --reset\"\n+\n test_submodule_switch \"git read-tree -u -m\"\n \n test_submodule_forced_switch \"git read-tree -u --reset\"\n"},{"id":"313032","messageId":"20170302004759.27852-6-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 05/18] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:46Z","receivedAt":"2017-03-02T03:50:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh: test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 506 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 504 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 83202c54fc..aa196d4d42 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (add_nested_sub => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n # / ^\n # / remove_sub1\n # /\n-# add_sub1 /-------O\n-# | / ^\n+# add_sub1 /-------O---------O--------O modify_sub1_recursively\n+# | / ^ add_nested_sub\n # | / modify_sub1\n # v/\n # O------O-----------O---------O\n@@ -48,6 +49,17 @@ create_lib_submodule_repo () {\n \t\tgit commit -m \"Base inside first submodule\" &&\n \t\tgit branch \"no_submodule\"\n \t) &&\n+\tgit init submodule_update_sub2 &&\n+\t(\n+\t\tcd submodule_update_sub2\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"nested submodule base\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -84,6 +96,26 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b add_nested_sub modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1 checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 checkout -b modify_sub1_recursively &&\n+\t\techo change >sub1/sub2/file3 &&\n+\t\tgit -C sub1/sub2 add file3 &&\n+\t\tgit -C sub1/sub2 commit -m \"make a change in nested sub\" &&\n+\t\tgit -C sub1 add sub2 &&\n+\t\tgit -C sub1 commit -m \"update nested sub\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -m \"update sub1, that updates nested sub\" &&\n+\t\tgit -C sub1 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -150,6 +182,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -180,6 +221,27 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# make the submodule git dirs available\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree\n+\tfi &&\n+\tif ! test -d submodule_update/.git/modules/sub1/modules/sub2\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules/sub1/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree\n+\tfi &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# sub1 might not be checked out, so use the git dir\n+\tgit -C submodule_update/.git/modules/sub1 config submodule.sub2.url \"bogus\"\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -700,3 +762,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+# in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+# git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 >.git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignroed\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_nested_sub &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content -C sub1 sub2 origin/modify_sub1_recursively\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313033","messageId":"20170302004759.27852-15-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 14/18] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:55Z","receivedAt":"2017-03-02T03:58:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n unpack-trees.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++++++----\n unpack-trees.h | 1 +\n 2 files changed, 123 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..8333da2cc9 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,8 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /*\n * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +47,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_SUBMODULE */\n+\t\"Submodule '%s' cannot checkout new HEAD.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +166,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot checkout new HEAD\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +247,75 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int check_submodule_move_head(const struct cache_entry *ce,\n+\t\t\t\t const char *old_id,\n+\t\t\t\t const char *new_id,\n+\t\t\t\t struct unpack_trees_options *o)\n+{\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\t\tif (submodule_move_head(ce->name, old_id, new_id, SUBMODULE_MOVE_HEAD_DRY_RUN))\n+\t\t\treturn o->gently ? -1 :\n+\t\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);\n+\t\treturn 0;\n+\tcase SM_UPDATE_NONE:\n+\t\treturn 0;\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\tcase SM_UPDATE_COMMAND:\n+\tdefault:\n+\t\twarning(_(\"submodule update strategy not supported for submodule '%s'\"), ce->name);\n+\t\treturn -1;\n+\t}\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tsubmodule_free();\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t\tgitmodules_config();\n+\t\t\t\tgit_config(submodule_config, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n /*\n * Unlink the last component and schedule the leading directories for\n * removal, such that empty directories get removed.\n */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (sub) {\n+\t\tswitch (sub->update_strategy.type) {\n+\t\tcase SM_UPDATE_UNSPECIFIED:\n+\t\tcase SM_UPDATE_CHECKOUT:\n+\t\tcase SM_UPDATE_REBASE:\n+\t\tcase SM_UPDATE_MERGE:\n+\t\t\tsubmodule_move_head(ce->name, \"HEAD\", NULL,\n+\t\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\tbreak;\n+\t\tcase SM_UPDATE_NONE:\n+\t\tcase SM_UPDATE_COMMAND:\n+\t\t\treturn; /* Do not touch the submodule. */\n+\t\t}\n+\t}\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +371,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (should_update_submodules() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1431,26 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint r = check_submodule_move_head(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index. This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1494,11 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!submodule_from_ce(ce))\n+\t\treturn 0;\n+\n+\treturn check_submodule_move_head(ce, old_sha1,\n+\t\t\t\t\t oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1664,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (submodule_from_ce(ce))\n+\t\t\t\tret = check_submodule_move_head(ce,\n+\t\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1680,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (submodule_from_ce(ce))\n+\t\t\treturn check_submodule_move_head(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t error_type, o);\n@@ -1643,6 +1739,15 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, NULL,\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1768,14 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..6c48117b84 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313034","messageId":"20170302004759.27852-2-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[PATCH 01/18] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-02T00:47:42Z","receivedAt":"2017-03-02T03:58:47Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand. The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..5df528ea81 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n # replace_file_with_sub1)\n #\n-# --O-----O\n-# / ^ replace_directory_with_sub1\n-# / replace_sub1_with_directory\n-# /----O\n-# / ^\n-# / modify_sub1\n-# O------O-------O\n-# ^ ^\\ ^\n-# | | \\ remove_sub1\n-# | | -----O-----O\n-# | | \\ ^ replace_file_with_sub1\n-# | | \\ replace_sub1_with_file\n-# | add_sub1 --O-----O\n-# no_submodule ^ valid_sub1\n-# invalid_sub1\n+# ----O\n+# / ^\n+# / remove_sub1\n+# /\n+# add_sub1 /-------O\n+# | / ^\n+# | / modify_sub1\n+# v/\n+# O------O-----------O---------O\n+# ^ \\ ^ replace_directory_with_sub1\n+# | \\ replace_sub1_with_directory\n+# no_submodule \\\n+# --------O---------O\n+# \\ ^ replace_file_with_sub1\n+# \\ replace_sub1_with_file\n+# \\\n+# ----O---------O\n+# ^ valid_sub1\n+# invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313358","messageId":"CAGZ79kZ_yuT+O84dRuJRtec+87PH6vmSA4gqeg6hzJyLT4_N2Q@mail.gmail.com","threadId":"45260","inReplyTo":"CAGZ79kbffqxq-D_PBGLYQqz9M1jgighutp8WPYmeTBhdfkuKnw@mail.gmail.com","subject":"Re: [PATCH 03/18] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:35:48Z","receivedAt":"2017-03-06T20:45:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Mar 6, 2017 at 12:30 PM, Stefan Beller wrote:\n>\n> as well as another fix in git-submodule.sh\n>\n\nI spoke too early, that seems like a false positive.\n"},{"id":"313359","messageId":"20170306205919.9713-4-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 03/18] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:04Z","receivedAt":"2017-03-06T21:06:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex c0d6325133..00128f28b5 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -193,6 +193,11 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n+\tif test x\"$1\" = \"x-C\"\n+\tthen\n+\t\tcd \"$2\"\n+\t\tshift; shift;\n+\tfi\n \tif test $# != 2\n \tthen\n \t\techo \"test_submodule_content needs two arguments\"\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313360","messageId":"20170306205919.9713-11-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 10/18] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:11Z","receivedAt":"2017-03-06T21:07:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that checks if a submodule needs\nto be checked for an update before attempting the update.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 16 ++++++++++++++++\n submodule.h | 7 +++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 591f4a694e..8b2c0212be 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -548,6 +548,22 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int should_update_submodules(void)\n+{\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n+{\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn NULL;\n+\n+\tif (!should_update_submodules())\n+\t\treturn NULL;\n+\n+\treturn submodule_from_path(null_sha1, ce->name);\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex b4e60c08d2..6f3fe85c7c 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,6 +65,13 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+/* Check if we want to update any submodule.*/\n+extern int should_update_submodules(void);\n+/*\n+ * Returns the submodule struct if the given ce entry is a submodule\n+ * and it should be updated. Returns NULL otherwise.\n+ */\n+extern const struct submodule *submodule_from_ce(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313361","messageId":"20170306205919.9713-8-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 07/18] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:08Z","receivedAt":"2017-03-06T21:07:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. This patch makes\nconnect_work_tree_and_git_dir safe to use for both cases of\neither the git dir or the working dir missing.\n\nTo do so, we need to call safe_create_leading_directories[_const]\non both directories. However this has to happen before we construct\nthe absolute paths as real_pathdup assumes the directories to\nbe there already.\n\nSo for both the config file in the git dir as well as the .git link\nfile we need to\na) construct the name\nb) call SCLD\nc) get the absolute path\nd) once a-c is done for both we can consume the absolute path\n to compute the relative path to each other and store those\n relative paths.\n\nThe implementation provided here puts a) and b) for both cases first,\nand then performs c and d after.\n\nOne of the two users of 'connect_work_tree_and_git_dir' already checked\nfor the directory being there, so we can loose that check as\nconnect_work_tree_and_git_dir handles this functionality now.\n\nSigned-off-by: Stefan Beller \n---\n dir.c | 32 +++++++++++++++++++++-----------\n submodule.c | 11 ++---------\n 2 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..6f52af7abb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n /* Update gitfile and core.worktree setting to connect work tree and git dir */\n void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n {\n-\tstruct strbuf file_name = STRBUF_INIT;\n+\tstruct strbuf gitfile_sb = STRBUF_INIT;\n+\tstruct strbuf cfg_sb = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n-\tchar *git_dir = real_pathdup(git_dir_);\n-\tchar *work_tree = real_pathdup(work_tree_);\n+\tchar *git_dir, *work_tree;\n \n-\t/* Update gitfile */\n-\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n-\twrite_file(file_name.buf, \"gitdir: %s\",\n-\t\t relative_path(git_dir, work_tree, &rel_path));\n+\t/* Prepare .git file */\n+\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n+\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n+\n+\t/* Prepare config file */\n+\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n+\tif (safe_create_leading_directories_const(cfg_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n \n+\tgit_dir = real_pathdup(git_dir_);\n+\twork_tree = real_pathdup(work_tree_);\n+\n+\t/* Write .git file */\n+\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n+\t\t relative_path(git_dir, work_tree, &rel_path));\n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n+\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n \t\t\t relative_path(work_tree, git_dir, &rel_path));\n \n-\tstrbuf_release(&file_name);\n+\tstrbuf_release(&gitfile_sb);\n+\tstrbuf_release(&cfg_sb);\n \tstrbuf_release(&rel_path);\n \tfree(work_tree);\n \tfree(git_dir);\ndiff --git a/submodule.c b/submodule.c\nindex 0e55372f37..04d185738f 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1442,8 +1442,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \n \t/* Not populated? */\n \tif (!sub_git_dir) {\n-\t\tchar *real_new_git_dir;\n-\t\tconst char *new_git_dir;\n \t\tconst struct submodule *sub;\n \n \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n@@ -1466,13 +1464,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tsub = submodule_from_path(null_sha1, path);\n \t\tif (!sub)\n \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n-\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n-\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n-\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n-\t\treal_new_git_dir = real_pathdup(new_git_dir);\n-\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n-\n-\t\tfree(real_new_git_dir);\n+\t\tconnect_work_tree_and_git_dir(path,\n+\t\t\tgit_path(\"modules/%s\", sub->name));\n \t} else {\n \t\t/* Is it already absorbed into the superprojects git dir? */\n \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313362","messageId":"20170306205919.9713-3-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 02/18] lib-submodule-update.sh: do not use ./. as submodule remote","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:03Z","receivedAt":"2017-03-06T21:07:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Adding the repository itself as a submodule does not make sense in the\nreal world. In our test suite we used to do that out of convenience in\nsome tests as the current repository has easiest access for setting up\n'just a submodule'.\n\nHowever this doesn't quite test the real world, so let's do not follow\nthis pattern any further and actually create an independent repository\nthat we can use as a submodule.\n\nWhen using './.' as the remote the superproject and submodule share the\nsame objects, such that testing if a given sha1 is a valid commit works\nin either repository. As running commands in an unpopulated submodule\nfall back to the superproject, this happens in `reset_work_tree_to`\nto determine if we need to populate the submodule. Fix this bug by\nchecking in the actual remote now.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 15 +++++++++++++--\n 1 file changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5df528ea81..c0d6325133 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -37,6 +37,17 @@\n #\n \n create_lib_submodule_repo () {\n+\tgit init submodule_update_sub1 &&\n+\t(\n+\t\tcd submodule_update_sub1 &&\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"Base inside first submodule\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -49,7 +60,7 @@ create_lib_submodule_repo () {\n \t\tgit branch \"no_submodule\" &&\n \n \t\tgit checkout -b \"add_sub1\" &&\n-\t\tgit submodule add ./. sub1 &&\n+\t\tgit submodule add ../submodule_update_sub1 sub1 &&\n \t\tgit config -f .gitmodules submodule.sub1.ignore all &&\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n@@ -162,7 +173,7 @@ reset_work_tree_to () {\n \t\ttest_must_be_empty actual &&\n \t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n \t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313363","messageId":"20170306205919.9713-18-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 17/18] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:18Z","receivedAt":"2017-03-06T21:07:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This exposes a flag to recurse into submodules\nin builtin/checkout making use of the code implemented\nin prior patches.\n\nA new failure mode is introduced in the submodule\nupdate library, as the directory/submodule conflict\nis not solved in prior patches.\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-checkout.txt | 7 +++++++\n builtin/checkout.c | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 23 +++++++++++++++++------\n t/t2013-checkout-submodule.sh | 5 +++++\n 4 files changed, 57 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..d6399c0af8 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n ::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..e9c5fcfaf8 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [] \"),\n \tN_(\"git checkout [] [] -- ...\"),\n \tNULL,\n };\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout '\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5b885cfbf8..6a78139f90 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -782,6 +782,11 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -891,7 +896,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -903,7 +908,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -931,7 +936,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -987,7 +992,8 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t# recursing deeper than one level doesn't work yet.\n+\ttest_expect_failure \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n@@ -1006,6 +1012,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1100,7 +1111,7 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -1151,7 +1162,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..aa35223369 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,11 @@ test_expect_success '\"checkout \" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313364","messageId":"20170306205919.9713-2-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 01/18] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:02Z","receivedAt":"2017-03-06T21:07:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand. The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..5df528ea81 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n # replace_file_with_sub1)\n #\n-# --O-----O\n-# / ^ replace_directory_with_sub1\n-# / replace_sub1_with_directory\n-# /----O\n-# / ^\n-# / modify_sub1\n-# O------O-------O\n-# ^ ^\\ ^\n-# | | \\ remove_sub1\n-# | | -----O-----O\n-# | | \\ ^ replace_file_with_sub1\n-# | | \\ replace_sub1_with_file\n-# | add_sub1 --O-----O\n-# no_submodule ^ valid_sub1\n-# invalid_sub1\n+# ----O\n+# / ^\n+# / remove_sub1\n+# /\n+# add_sub1 /-------O\n+# | / ^\n+# | / modify_sub1\n+# v/\n+# O------O-----------O---------O\n+# ^ \\ ^ replace_directory_with_sub1\n+# | \\ replace_sub1_with_directory\n+# no_submodule \\\n+# --------O---------O\n+# \\ ^ replace_file_with_sub1\n+# \\ replace_sub1_with_file\n+# \\\n+# ----O---------O\n+# ^ valid_sub1\n+# invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313365","messageId":"20170306205919.9713-14-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 13/18] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:14Z","receivedAt":"2017-03-06T21:08:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller \n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n * Currently, git does not checkout subprojects during a superproject\n * checkout, so it is not going to overwrite anything.\n */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t const struct cache_entry *ce,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313366","messageId":"20170306205919.9713-16-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:16Z","receivedAt":"2017-03-06T21:08:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n read-cache.c | 27 +++++++++++++++++++++++++--\n 1 file changed, 25 insertions(+), 2 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 9054369dd0..9a2abacf7a 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -18,6 +18,8 @@\n #include \"varint.h\"\n #include \"split-index.h\"\n #include \"utf8.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -520,6 +522,22 @@ int remove_index_entry_at(struct index_state *istate, int pos)\n \treturn 1;\n }\n \n+static void remove_submodule_according_to_strategy(const struct submodule *sub)\n+{\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\t\tsubmodule_move_head(sub->path, \"HEAD\", NULL, \\\n+\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n+\t\tbreak;\n+\tcase SM_UPDATE_NONE:\n+\tcase SM_UPDATE_COMMAND:\n+\t\t; /* Do not touch the submodule. */\n+\t}\n+}\n+\n /*\n * Remove all cache entries marked for removal, that is where\n * CE_REMOVE is set in ce_flags. This is much more effective than\n@@ -532,8 +550,13 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n-\t\t\tremove_name_hash(istate, ce_array[i]);\n-\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\tconst struct submodule *sub = submodule_from_ce(ce_array[i]);\n+\t\t\tif (sub) {\n+\t\t\t\tremove_submodule_according_to_strategy(sub);\n+\t\t\t} else {\n+\t\t\t\tremove_name_hash(istate, ce_array[i]);\n+\t\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n+\t\t\t}\n \t\t}\n \t\telse\n \t\t\tce_array[j++] = ce_array[i];\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313367","messageId":"20170306205919.9713-13-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 12/18] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:13Z","receivedAt":"2017-03-06T21:08:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n \"Is it safe to change the submodule to this new state?\"\n e.g. is it overwriting untracked files or are there local\n changes that would be overwritten?\n* supports a force flag that can be used for resetting\n the tree.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h | 7 ++++\n 2 files changed, 142 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 0b2596e88a..bc5fecf8c5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n+\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.no_stdout = 1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n+\n+\treturn finish_command(&cp);\n+}\n+\n+static void submodule_reset_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not reset submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ */\n+int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub))\n+\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_reset_index(path);\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n+\t\targv_array_push(&cp.args, \"-n\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-u\");\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 6f3fe85c7c..4cdf6445f7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -96,6 +96,13 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n+#define SUBMODULE_MOVE_HEAD_FORCE (1<<1)\n+extern int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags);\n+\n /*\n * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313368","messageId":"20170306205919.9713-17-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 16/18] entry.c: update submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:17Z","receivedAt":"2017-03-06T21:08:55Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n entry.c | 30 ++++++++++++++++++++++++++++++\n 1 file changed, 30 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..d2b512da90 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t const struct checkout *state)\n@@ -146,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n \tunsigned long size;\n \tsize_t wrote, newsize = 0;\n \tstruct stat st;\n+\tconst struct submodule *sub;\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n@@ -203,6 +205,10 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub)\n+\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -259,7 +265,31 @@ int checkout_entry(struct cache_entry *ce,\n \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n+\t\tconst struct submodule *sub;\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\t} else\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313369","messageId":"20170306205919.9713-12-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 11/18] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:12Z","receivedAt":"2017-03-06T21:08:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 8b2c0212be..0b2596e88a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -356,6 +356,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n * summary output. If it can locate the submodule objects directory it will\n * attempt to lookup both the left and right commits and put them into the\n@@ -1390,18 +1407,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n * Embeds a single submodules git directory into the superprojects git dir,\n * non recursively.\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313370","messageId":"20170306205919.9713-5-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 04/18] lib-submodule-update.sh: replace sha1 by hash","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:05Z","receivedAt":"2017-03-06T21:08:57Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Cleaning up code by generalising it.\nCurrently the mailing list discusses yet again how\nto migrate away from sha1.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 00128f28b5..f52c49c838 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -171,9 +171,9 @@ reset_work_tree_to () {\n \t\tgit checkout -f \"$1\" &&\n \t\tgit status -u -s >actual &&\n \t\ttest_must_be_empty actual &&\n-\t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n-\t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\thash=$(git rev-parse --revs-only HEAD:sub1) &&\n+\t\tif test -n \"$hash\" &&\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$hash^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313371","messageId":"20170306205919.9713-15-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 14/18] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:15Z","receivedAt":"2017-03-06T21:09:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n unpack-trees.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++++++----\n unpack-trees.h | 1 +\n 2 files changed, 123 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..8333da2cc9 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,8 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /*\n * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +47,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_SUBMODULE */\n+\t\"Submodule '%s' cannot checkout new HEAD.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +166,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot checkout new HEAD\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +247,75 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int check_submodule_move_head(const struct cache_entry *ce,\n+\t\t\t\t const char *old_id,\n+\t\t\t\t const char *new_id,\n+\t\t\t\t struct unpack_trees_options *o)\n+{\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\t\tif (submodule_move_head(ce->name, old_id, new_id, SUBMODULE_MOVE_HEAD_DRY_RUN))\n+\t\t\treturn o->gently ? -1 :\n+\t\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);\n+\t\treturn 0;\n+\tcase SM_UPDATE_NONE:\n+\t\treturn 0;\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\tcase SM_UPDATE_COMMAND:\n+\tdefault:\n+\t\twarning(_(\"submodule update strategy not supported for submodule '%s'\"), ce->name);\n+\t\treturn -1;\n+\t}\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tsubmodule_free();\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t\tgitmodules_config();\n+\t\t\t\tgit_config(submodule_config, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n /*\n * Unlink the last component and schedule the leading directories for\n * removal, such that empty directories get removed.\n */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (sub) {\n+\t\tswitch (sub->update_strategy.type) {\n+\t\tcase SM_UPDATE_UNSPECIFIED:\n+\t\tcase SM_UPDATE_CHECKOUT:\n+\t\tcase SM_UPDATE_REBASE:\n+\t\tcase SM_UPDATE_MERGE:\n+\t\t\tsubmodule_move_head(ce->name, \"HEAD\", NULL,\n+\t\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\tbreak;\n+\t\tcase SM_UPDATE_NONE:\n+\t\tcase SM_UPDATE_COMMAND:\n+\t\t\treturn; /* Do not touch the submodule. */\n+\t\t}\n+\t}\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +371,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (should_update_submodules() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1431,26 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint r = check_submodule_move_head(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index. This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1494,11 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!submodule_from_ce(ce))\n+\t\treturn 0;\n+\n+\treturn check_submodule_move_head(ce, old_sha1,\n+\t\t\t\t\t oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1664,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (submodule_from_ce(ce))\n+\t\t\t\tret = check_submodule_move_head(ce,\n+\t\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1680,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (submodule_from_ce(ce))\n+\t\t\treturn check_submodule_move_head(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t error_type, o);\n@@ -1643,6 +1739,15 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, NULL,\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1768,14 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..6c48117b84 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313372","messageId":"20170306205919.9713-6-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 05/18] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:06Z","receivedAt":"2017-03-06T21:09:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh: test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 506 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 504 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex f52c49c838..5b885cfbf8 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (add_nested_sub => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n # / ^\n # / remove_sub1\n # /\n-# add_sub1 /-------O\n-# | / ^\n+# add_sub1 /-------O---------O--------O modify_sub1_recursively\n+# | / ^ add_nested_sub\n # | / modify_sub1\n # v/\n # O------O-----------O---------O\n@@ -48,6 +49,17 @@ create_lib_submodule_repo () {\n \t\tgit commit -m \"Base inside first submodule\" &&\n \t\tgit branch \"no_submodule\"\n \t) &&\n+\tgit init submodule_update_sub2 &&\n+\t(\n+\t\tcd submodule_update_sub2\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"nested submodule base\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -84,6 +96,26 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b add_nested_sub modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1 checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 checkout -b modify_sub1_recursively &&\n+\t\techo change >sub1/sub2/file3 &&\n+\t\tgit -C sub1/sub2 add file3 &&\n+\t\tgit -C sub1/sub2 commit -m \"make a change in nested sub\" &&\n+\t\tgit -C sub1 add sub2 &&\n+\t\tgit -C sub1 commit -m \"update nested sub\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -m \"update sub1, that updates nested sub\" &&\n+\t\tgit -C sub1 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -150,6 +182,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -180,6 +221,27 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# make the submodule git dirs available\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree\n+\tfi &&\n+\tif ! test -d submodule_update/.git/modules/sub1/modules/sub2\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules/sub1/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree\n+\tfi &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# sub1 might not be checked out, so use the git dir\n+\tgit -C submodule_update/.git/modules/sub1 config submodule.sub2.url \"bogus\"\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -700,3 +762,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+# in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+# git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 >.git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignroed\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_nested_sub &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content -C sub1 sub2 origin/modify_sub1_recursively\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313373","messageId":"20170306205919.9713-19-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 18/18] builtin/read-tree: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:19Z","receivedAt":"2017-03-06T21:25:17Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"A new known failure mode is introduced[1], which is actually not\na failure but a feature in read-tree. Unlike checkout for which\nthe recursive submodule tests were originally written, read-tree does\nwarn about ignored untracked files that would be overwritten.\nFor the sake of keeping the test library for submodules genric, just\nmark the test as a failure.\n\n[1] KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-read-tree.txt | 6 ++++++\n builtin/read-tree.c | 29 +++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 7 ++++++-\n t/t1013-read-tree-submodule.sh | 7 +++++++\n 4 files changed, 48 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt\nindex fa1d557e5b..ed9d63ef4a 100644\n--- a/Documentation/git-read-tree.txt\n+++ b/Documentation/git-read-tree.txt\n@@ -115,6 +115,12 @@ OPTIONS\n \tdirectories the index file and index output file are\n \tlocated in.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject by\n+\tcalling read-tree recursively, also setting the submodules HEAD to be\n+\tdetached at that commit.\n+\n --no-sparse-checkout::\n \tDisable sparse checkout support even if `core.sparseCheckout`\n \tis true.\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 8ba64bc785..2dc5cc06da 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -15,10 +15,13 @@\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"resolve-undo.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n+int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\n@@ -96,6 +99,23 @@ static int debug_merge(const struct cache_entry * const *stages,\n \treturn 0;\n }\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n static struct lock_file lock_file;\n \n int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n@@ -137,6 +157,9 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n \t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_END()\n \t};\n \n@@ -152,6 +175,12 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \n \thold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT) {\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tset_config_update_recurse_submodules(RECURSE_SUBMODULES_ON);\n+\t}\n+\n \tprefix_set = opts.prefix ? 1 : 0;\n \tif (1 < opts.merge + opts.reset + prefix_set)\n \t\tdie(\"Which one? -m, --reset, or --prefix?\");\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 6a78139f90..949ebd546c 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -787,6 +787,11 @@ test_submodule_switch_recursing () {\n \tthen\n \t\tRESULT=failure\n \tfi\n+\tRESULT1=success\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n+\tthen\n+\t\tRESULT1=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -827,7 +832,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULT1 \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 20526aed34..7019d0a04f 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,13 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n+\n+test_submodule_switch_recursing \"git read-tree --recurse-submodules -u -m\"\n+\n+test_submodule_forced_switch_recursing \"git read-tree --recurse-submodules -u --reset\"\n+\n test_submodule_switch \"git read-tree -u -m\"\n \n test_submodule_forced_switch \"git read-tree -u --reset\"\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313376","messageId":"20170306205919.9713-9-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 08/18] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:09Z","receivedAt":"2017-03-06T22:00:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller \n---\n submodule-config.c | 20 ++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 29 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..3e8e380d98 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,26 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t unsigned char *gitmodules_sha1,\n \t\t\t\t struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313378","messageId":"xmqqo9xem4gk.fsf@junio-linux.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170306205919.9713-4-sbeller@google.com","subject":"Re: [PATCH 03/18] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-06T22:21:31Z","receivedAt":"2017-03-06T22:22:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Signed-off-by: Stefan Beller \n> ---\n> t/lib-submodule-update.sh | 5 +++++\n> 1 file changed, 5 insertions(+)\n>\n> diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\n> index c0d6325133..00128f28b5 100755\n> --- a/t/lib-submodule-update.sh\n> +++ b/t/lib-submodule-update.sh\n> @@ -193,6 +193,11 @@ test_superproject_content () {\n> # Test that the given submodule at path \"$1\" contains the content according\n> # to the submodule commit recorded in the superproject's commit \"$2\"\n> test_submodule_content () {\n> +\tif test x\"$1\" = \"x-C\"\n> +\tthen\n> +\t\tcd \"$2\"\n> +\t\tshift; shift;\n\nThat's old fashoned like me ;-) \n\nIt seems \"shift []\" is already used elsewhere in our test scripts\nwithout getting complaints.\n\n'git-difftool-helper.sh' is the only one that uses number above 1 in\nthe scripted Porcelains, so it is possible that an esoteric platform\nwithout a shell that understands it may be built and used _without_\npassing our test suite, so I'd be wary of using it in the scripted\nPorcelains, but this is part of the test suite, so I would think it\nis OK.\n\n> +\tfi\n> \tif test $# != 2\n> \tthen\n> \t\techo \"test_submodule_content needs two arguments\"\n"},{"id":"313380","messageId":"CAGZ79kbffqxq-D_PBGLYQqz9M1jgighutp8WPYmeTBhdfkuKnw@mail.gmail.com","threadId":"45260","inReplyTo":"20170302021135.GA26453@starla","subject":"Re: [PATCH 03/18] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:30:53Z","receivedAt":"2017-03-06T22:32:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Mar 1, 2017 at 6:11 PM, Eric Wong wrote:\n> Stefan Beller wrote:\n>> test_submodule_content () {\n>> + if test \"$1\" == \"-C\"\n>\n> Use a single '=' for portability in sh. It's also a good idea\n> to prefix variables with 'x' or some such, since \"$1\" could be\n> \"-n\" or some other supported switch for test(1).\n> So, something like:\n>\n> if test x\"$1\" = \"x-C\"\n>\n> ...or use a case statement.\n>\n> On Debian systems, I use the \"checkbashisms\" from the\n> \"devscripts\" package to find and avoid bash dependencies.\n\nThanks for review as well as the tip for using\ncheckbashisms,\n\nI'll resend this series with this fix as well as another fix\nin git-submodule.sh\n\nThanks,\nStefan\n"},{"id":"313391","messageId":"20170306205919.9713-7-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 06/18] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:07Z","receivedAt":"2017-03-07T00:59:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller \n---\n builtin/grep.c | 2 +-\n submodule.c | 7 ++-----\n submodule.h | 8 +++++++-\n 3 files changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..0e55372f37 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -234,15 +234,12 @@ int is_submodule_initialized(const char *path)\n \treturn ret;\n }\n \n-/*\n- * Determine if a submodule has been populated at a given 'path'\n- */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..0b915bd3ac 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,13 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+/*\n+ * Determine if a submodule has been populated at a given 'path' by checking if\n+ * the /.git resolves to a valid git repository.\n+ * If return_error_code is NULL, die on error.\n+ * Otherwise the return error code is the same as of resolve_gitdir_gently.\n+ */\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313395","messageId":"20170306205919.9713-10-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[PATCH 09/18] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:10Z","receivedAt":"2017-03-07T03:19:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 04d185738f..591f4a694e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -542,6 +543,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 0b915bd3ac..b4e60c08d2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -64,6 +64,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.52.ge239d7e709.dirty\n\n"},{"id":"313396","messageId":"20170306205919.9713-1-sbeller@google.com","threadId":"45260","inReplyTo":"20170302004759.27852-1-sbeller@google.com","subject":"[RFCv7 PATCH 00/18] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-06T20:59:01Z","receivedAt":"2017-03-07T03:30:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20170302004759.27852-1-sbeller@google.com\nhttps://public-inbox.org/git/20161203003022.29797-1-sbeller@google.com/\n\nv7:\n * addressed Erics comment by fixing the bashism in t/lib-submodule-update.sh\ndiff to v6:\n diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\n index 5caae06bc5..949ebd546c 100755\n --- a/t/lib-submodule-update.sh\n +++ b/t/lib-submodule-update.sh\n @@ -255,7 +255,7 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n - if test \"$1\" == \"-C\"\n + if test x\"$1\" = \"x-C\"\n then\n cd \"$2\"\n shift; shift;\n * interdiff to v5 below\n \n\nv6:\n * added support for read-tree (see last patch) to see how generic the\n code of the previous patches is. I am pretty pleased with that patch.\n * marked two functions static. Thanks Ramsay!\n * fixed the recursive test; it still fails but it is the code that fails,\n not the test. For this I had to change the setup code slightly.\n * 2 new patches adding tiny refactoring to the submodule test lib. \n * interdiff (to origin/sb/checkout-recurse-submodules, which is v5) below.\n\nv5:\n * as v4 was the first version queued by Junio, we do have an interdiff below!\n * renamed functions\n * changed the API, now the caller has to take care of the submodule strategy\n themselves. (Note this can be different for different situations, e.g.\n when a submodule is deleted, we can do that for any strategy except none and\n !command. But for moving to a new state of the submodule we currently\n only implement \"checkout\" [unspecified defaults to checkout]. warning about\n the others, doing nothing there.)\n\nv4:\n * addressed all comments of Brian, Junio and Brandon.\n Thanks!\n * one major point of change is the introduction of another patch\n \"lib-submodule-update.sh: do not use ./. as submodule remote\",\n as that took some time to track down the existing bug.\n \nv3:\n * moved tests from t2013 to the generic submodule library.\n * factored out the refactoring patches to be up front\n * As I redid the complete implementation, I have the impression this time\n it is cleaner than previous versions.\n \n I think we still have to fix the corner cases of directory/file/submodule \n conflicts before merging, but this serves as a status update on my current\n way of thinking how to implement the worktree commands being aware of\n submodules.\n \nThanks,\nStefan\n\nv2:\n* based on top of the series sent out an hour ago\n \"[PATCHv4 0/5] submodule embedgitdirs\"\n* Try to embed a submodule if we need to remove it.\n* Strictly do not change behavior if not giving the new flag.\n* I think I missed some review comments from v1, but I'd like to get\n the current state out over the weekend, as a lot has changed so far.\n On Monday I'll go through the previous discussion with a comb to see\n if I missed something.\n \nv1:\nWhen working with submodules, nearly anytime after checking out\na different state of the projects, that has submodules changed\nyou'd run \"git submodule update\" with a current version of Git.\n\nThere are two problems with this approach:\n\n* The \"submodule update\" command is dangerous as it\n doesn't check for work that may be lost in the submodule\n (e.g. a dangling commit).\n* you may forget to run the command as checkout is supposed\n to do all the work for you.\n\nIntegrate updating the submodules into git checkout, with the same\nsafety promises that git-checkout has, i.e. not throw away data unless\nasked to. This is done by first checking if the submodule is at the same\nsha1 as it is recorded in the superproject. If there are changes we stop\nproceeding the checkout just like it is when checking out a file that\nhas local changes.\n\nThe integration happens in the code that is also used in other commands\nsuch that it will be easier in the future to make other commands aware\nof submodule.\n\nThis also solves d/f conflicts in case you replace a file/directory\nwith a submodule or vice versa.\n\nThe patches are still a bit rough, but the overall series seems\npromising enough to me that I want to put it out here.\n\nAny review, specifically on the design level welcome!\n\nThanks,\nStefan\n\n\nStefan Beller (14):\n lib-submodule-update.sh: reorder create_lib_submodule_repo\n lib-submodule-update.sh: define tests for recursing into submodules\n make is_submodule_populated gently\n connect_work_tree_and_git_dir: safely create leading directories\n update submodules: add submodule config parsing\n update submodules: add a config option to determine if submodules are\n updated\n update submodules: introduce is_interesting_submodule\n update submodules: move up prepare_submodule_repo_env\n update submodules: add submodule_go_from_to\n unpack-trees: pass old oid to verify_clean_submodule\n unpack-trees: check if we can perform the operation for submodules\n read-cache: remove_marked_cache_entries to wipe selected submodules.\n entry.c: update submodules when interesting\n builtin/checkout: add --recurse-submodules switch\ndiff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt\nindex fa1d557e5b..ed9d63ef4a 100644\n--- a/Documentation/git-read-tree.txt\n+++ b/Documentation/git-read-tree.txt\n@@ -115,6 +115,12 @@ OPTIONS\n \tdirectories the index file and index output file are\n \tlocated in.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject by\n+\tcalling read-tree recursively, also setting the submodules HEAD to be\n+\tdetached at that commit.\n+\n --no-sparse-checkout::\n \tDisable sparse checkout support even if `core.sparseCheckout`\n \tis true.\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 207ce09771..e9c5fcfaf8 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -29,8 +29,8 @@ static const char * const checkout_usage[] = {\n \tNULL,\n };\n \n-int option_parse_recurse_submodules(const struct option *opt,\n-\t\t\t\t const char *arg, int unset)\n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n {\n \tif (unset) {\n \t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 8ba64bc785..2dc5cc06da 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -15,10 +15,13 @@\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"resolve-undo.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n+int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\n@@ -96,6 +99,23 @@ static int debug_merge(const struct cache_entry * const *stages,\n \treturn 0;\n }\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n static struct lock_file lock_file;\n \n int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n@@ -137,6 +157,9 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n \t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_END()\n \t};\n \n@@ -152,6 +175,12 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \n \thold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT) {\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tset_config_update_recurse_submodules(RECURSE_SUBMODULES_ON);\n+\t}\n+\n \tprefix_set = opts.prefix ? 1 : 0;\n \tif (1 < opts.merge + opts.reset + prefix_set)\n \t\tdie(\"Which one? -m, --reset, or --prefix?\");\ndiff --git a/submodule.c b/submodule.c\nindex a2cf8c9376..bc5fecf8c5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1257,7 +1257,7 @@ static int submodule_has_dirty_index(const struct submodule *sub)\n \treturn finish_command(&cp);\n }\n \n-void submodule_reset_index(const char *path)\n+static void submodule_reset_index(const char *path)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 54cd8a6366..429cd1041d 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,7 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n-# - Updated submodule recursively (modify_sub1 => modify_sub1_recursively)\n+# - Updated submodule recursively (add_nested_sub => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -20,8 +20,8 @@\n # / ^\n # / remove_sub1\n # /\n-# add_sub1 /-------O---------O\n-# | / ^ modify_sub1_recursive\n+# add_sub1 /-------O---------O--------O modify_sub1_recursively\n+# | / ^ add_nested_sub\n # | / modify_sub1\n # v/\n # O------O-----------O---------O\n@@ -96,12 +96,24 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b modify_sub1_recursively modify_sub1 &&\n+\t\tgit checkout -b add_nested_sub modify_sub1 &&\n \t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n \t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n \t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n \t\tgit add sub1 &&\n \t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1 checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 checkout -b modify_sub1_recursively &&\n+\t\techo change >sub1/sub2/file3 &&\n+\t\tgit -C sub1/sub2 add file3 &&\n+\t\tgit -C sub1/sub2 commit -m \"make a change in nested sub\" &&\n+\t\tgit -C sub1 add sub2 &&\n+\t\tgit -C sub1 commit -m \"update nested sub\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -m \"update sub1, that updates nested sub\" &&\n+\t\tgit -C sub1 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 push origin modify_sub1_recursively &&\n \t\tgit -C sub1 submodule deinit -f --all &&\n \n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n@@ -200,9 +212,9 @@ reset_work_tree_to () {\n \t\tgit checkout -f \"$1\" &&\n \t\tgit status -u -s >actual &&\n \t\ttest_must_be_empty actual &&\n-\t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n-\t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\thash=$(git rev-parse --revs-only HEAD:sub1) &&\n+\t\tif test -n \"$hash\" &&\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$hash^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n@@ -211,14 +223,23 @@ reset_work_tree_to () {\n \n reset_work_tree_to_interested () {\n \treset_work_tree_to $1 &&\n-\t# indicate we are interested in the submodule:\n-\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n-\t# also have it available:\n+\t# make the submodule git dirs available\n \tif ! test -d submodule_update/.git/modules/sub1\n \tthen\n \t\tmkdir -p submodule_update/.git/modules &&\n \t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n-\tfi\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree\n+\tfi &&\n+\tif ! test -d submodule_update/.git/modules/sub1/modules/sub2\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules/sub1/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree\n+\tfi &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# sub1 might not be checked out, so use the git dir\n+\tgit -C submodule_update/.git/modules/sub1 config submodule.sub2.url \"bogus\"\n }\n \n # Test that the superproject contains the content according to commit \"$1\"\n@@ -234,6 +255,11 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n+\tif test x\"$1\" == \"x-C\"\n+\tthen\n+\t\tcd \"$2\"\n+\t\tshift; shift;\n+\tfi\n \tif test $# != 2\n \tthen\n \t\techo \"test_submodule_content needs two arguments\"\n@@ -761,6 +787,11 @@ test_submodule_switch_recursing () {\n \tthen\n \t\tRESULT=failure\n \tfi\n+\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n+\tthen\n+\t\tRESULT1=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -801,7 +832,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULT1 \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\n@@ -966,21 +997,17 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\t# This test fails, due to missing setup, we do not clone sub2 into\n-\t# submodule_update, because it doesn't exist in the 'add_sub1' version\n-\t#\n-\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t# recursing deeper than one level doesn't work yet.\n+\ttest_expect_failure \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n-\t\treset_work_tree_to_interested add_sub1 &&\n+\t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n-\t\t\ttest_must_fail $command modify_sub1_recursively &&\n-\t\t\ttest_superproject_content origin/add_sub1 &&\n-\t\t\ttest_submodule_content sub1 origin/add_sub1\n-\t\t\t# test_superproject_content origin/modify_sub1_recursively &&\n-\t\t\t# test_submodule_content sub1 origin/modify_sub1_recursively &&\n-\t\t\t# test_submodule_content sub1/sub2 no_submodule\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content -C sub1 sub2 origin/modify_sub1_recursively\n \t\t)\n \t'\n }\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 20526aed34..7019d0a04f 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,13 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n+\n+test_submodule_switch_recursing \"git read-tree --recurse-submodules -u -m\"\n+\n+test_submodule_forced_switch_recursing \"git read-tree --recurse-submodules -u --reset\"\n+\n test_submodule_switch \"git read-tree -u -m\"\n \n test_submodule_forced_switch \"git read-tree -u --reset\"\n"},{"id":"313455","messageId":"xmqqfuio674n.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170306205919.9713-16-sbeller@google.com","subject":"Re: [PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-07T22:42:48Z","receivedAt":"2017-03-07T22:42:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Signed-off-by: Stefan Beller \n> ---\n> read-cache.c | 27 +++++++++++++++++++++++++--\n> 1 file changed, 25 insertions(+), 2 deletions(-)\n>\n> diff --git a/read-cache.c b/read-cache.c\n> index 9054369dd0..9a2abacf7a 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -18,6 +18,8 @@\n> #include \"varint.h\"\n> #include \"split-index.h\"\n> #include \"utf8.h\"\n> +#include \"submodule.h\"\n> +#include \"submodule-config.h\"\n> \n> /* Mask for the name length in ce_flags in the on-disk index */\n> \n> @@ -520,6 +522,22 @@ int remove_index_entry_at(struct index_state *istate, int pos)\n> \treturn 1;\n> }\n> \n> +static void remove_submodule_according_to_strategy(const struct submodule *sub)\n> +{\n> +\tswitch (sub->update_strategy.type) {\n> +\tcase SM_UPDATE_UNSPECIFIED:\n> +\tcase SM_UPDATE_CHECKOUT:\n> +\tcase SM_UPDATE_REBASE:\n> +\tcase SM_UPDATE_MERGE:\n> +\t\tsubmodule_move_head(sub->path, \"HEAD\", NULL, \\\n\nWhat is this backslash doing here?\n\n> +\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n> +\t\tbreak;\n> +\tcase SM_UPDATE_NONE:\n> +\tcase SM_UPDATE_COMMAND:\n> +\t\t; /* Do not touch the submodule. */\n> +\t}\n> +}\n> +\n> /*\n> * Remove all cache entries marked for removal, that is where\n> * CE_REMOVE is set in ce_flags. This is much more effective than\n> @@ -532,8 +550,13 @@ void remove_marked_cache_entries(struct index_state *istate)\n> \n> \tfor (i = j = 0; i < istate->cache_nr; i++) {\n> \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n> -\t\t\tremove_name_hash(istate, ce_array[i]);\n> -\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n> +\t\t\tconst struct submodule *sub = submodule_from_ce(ce_array[i]);\n> +\t\t\tif (sub) {\n> +\t\t\t\tremove_submodule_according_to_strategy(sub);\n> +\t\t\t} else {\n> +\t\t\t\tremove_name_hash(istate, ce_array[i]);\n> +\t\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n> +\t\t\t}\n\nI cannot readily tell as the proposed log message is on the sketchy\nside to explain the reasoning behind this, but this behaviour change\nis done unconditionally (in other words, without introducing a flag\nthat is set by --recurse-submodules command line flag and switches\nbehaviour based on the flag) here. What is the end-user visible\neffect with this change? Can we have a new test in this same patch\nto demonstrate the desired behaviour introduced by this change (or\nthe bug this change fixes)?\n\n> \t\t}\n> \t\telse\n> \t\t\tce_array[j++] = ce_array[i];\n"},{"id":"313456","messageId":"xmqqtw746758.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170306205919.9713-17-sbeller@google.com","subject":"Re: [PATCH 16/18] entry.c: update submodules when interesting","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-07T22:42:27Z","receivedAt":"2017-03-07T22:43:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Signed-off-by: Stefan Beller \n> ---\n> entry.c | 30 ++++++++++++++++++++++++++++++\n> 1 file changed, 30 insertions(+)\n>\n> diff --git a/entry.c b/entry.c\n> index c6eea240b6..d2b512da90 100644\n> --- a/entry.c\n> +++ b/entry.c\n> @@ -2,6 +2,7 @@\n> #include \"blob.h\"\n> #include \"dir.h\"\n> #include \"streaming.h\"\n> +#include \"submodule.h\"\n> \n> static void create_directories(const char *path, int path_len,\n> \t\t\t const struct checkout *state)\n> @@ -146,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n> \tunsigned long size;\n> \tsize_t wrote, newsize = 0;\n> \tstruct stat st;\n> +\tconst struct submodule *sub;\n> \n> \tif (ce_mode_s_ifmt == S_IFREG) {\n> \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n> @@ -203,6 +205,10 @@ static int write_entry(struct cache_entry *ce,\n> \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n> \t\tif (mkdir(path, 0777) < 0)\n> \t\t\treturn error(\"cannot create submodule directory %s\", path);\n> +\t\tsub = submodule_from_ce(ce);\n> +\t\tif (sub)\n> +\t\t\treturn submodule_move_head(ce->name,\n> +\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n> \t\tbreak;\n> \tdefault:\n> \t\treturn error(\"unknown file mode for %s in index\", path);\n> @@ -259,7 +265,31 @@ int checkout_entry(struct cache_entry *ce,\n> \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n> \n> \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n> +\t\tconst struct submodule *sub;\n> \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n> +\t\t/*\n> +\t\t * Needs to be checked before !changed returns early,\n> +\t\t * as the possibly empty directory was not changed\n> +\t\t */\n> +\t\tsub = submodule_from_ce(ce);\n> +\t\tif (sub) {\n> +\t\t\tint err;\n> +\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n> +\t\t\t\tstruct stat sb;\n> +\t\t\t\tif (lstat(ce->name, &sb))\n> +\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n> +\t\t\t\tif (!(st.st_mode & S_IFDIR))\n> +\t\t\t\t\tunlink_or_warn(ce->name);\n\nWe found that the path ce->name is supposed to be a submodule that\nis checked out, we found that the submodule is not yet populated, we\ntried to make sure what is on that path, and its failure would cause\nus to die(). All that is sensible.\n\nThen we want to make sure the filesystem entity at the path\ncurrently is a directory and otherwise unlink (i.e. we may have an\nunrelated file that is not tracked there, or perhaps the user earlier\ndecided that replacing the submodule with a single file is a good\nidea, but then getting rid of the change with \"checkout -f\" before\ndoing \"git add\" on that file). That is also sensible.\n\nBut if that unlink fails, shouldn't we die, just like the case where\nwe cannot tell what is at the path ce->name?\n\nAnd if that unlink succeeds, instead of having an empty directory,\nwe start the \"move-head\" call to switch from NULL to ce->oid without\nhaving any directory. Wouldn't we want to mkdir() here (and remove\nmkdir() in submodule_move_head() if there is one---if there isn't\nthen I do not think this codepath would work)?\n\nIf we had a directory here, but if that directory is not empty,\nshould we proceed? I am assuming (I haven't carefully read\n\"move-head\") that it is OK because the \"run an equivalent of\n'checkout --detach ce->oid'\" done by \"move-head\" would notice a\npossible information loss to overwrite an untracked path (everything\nis \"untracked\" as the head moves from NULL to ce->oid in this case),\nand prevent it from happening.\n\nAm I reading the design correctly?\n\n> +\t\t\t\treturn submodule_move_head(ce->name,\n> +\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n> +\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n> +\t\t\t} else\n> +\t\t\t\treturn submodule_move_head(ce->name,\n> +\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n> +\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n> +\t\t}\n> +\n> \t\tif (!changed)\n> \t\t\treturn 0;\n> \t\tif (!state->force) {\n"},{"id":"313457","messageId":"xmqq4lz47mg3.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170306205919.9713-6-sbeller@google.com","subject":"Re: [PATCH 05/18] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-07T22:26:36Z","receivedAt":"2017-03-07T22:43:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> +\t# ... and ignored files are ignroed\n\nignored.\n"},{"id":"313458","messageId":"xmqqmvcw6750.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170306205919.9713-19-sbeller@google.com","subject":"Re: [PATCH 18/18] builtin/read-tree: add --recurse-submodules switch","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-07T22:42:35Z","receivedAt":"2017-03-07T22:43:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> A new known failure mode is introduced[1], which is actually not\n> a failure but a feature in read-tree. Unlike checkout for which\n> the recursive submodule tests were originally written, read-tree does\n> warn about ignored untracked files that would be overwritten.\n> For the sake of keeping the test library for submodules genric, just\n\ngeneric\n\n"},{"id":"313462","messageId":"xmqqy3wg4rko.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"xmqqtw746758.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 16/18] entry.c: update submodules when interesting","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-07T23:04:07Z","receivedAt":"2017-03-07T23:19:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n>> +\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n>> +\t\t\t\tstruct stat sb;\n>> +\t\t\t\tif (lstat(ce->name, &sb))\n>> +\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n>> +\t\t\t\tif (!(st.st_mode & S_IFDIR))\n>> +\t\t\t\t\tunlink_or_warn(ce->name);\n>\n> We found that the path ce->name is supposed to be a submodule that\n> is checked out, we found that the submodule is not yet populated, we\n> tried to make sure what is on that path, and its failure would cause\n> us to die(). All that is sensible.\n> ...\n> But if that unlink fails, shouldn't we die, just like the case where\n> we cannot tell what is at the path ce->name?\n>\n> And if that unlink succeeds, instead of having an empty directory,\n> we start the \"move-head\" call to switch from NULL to ce->oid without\n> having any directory. Wouldn't we want to mkdir() here (and remove\n> mkdir() in submodule_move_head() if there is one---if there isn't\n> then I do not think this codepath would work)?\n\nIn addition to mkdir(), would we also need the .git file that point\ninto superproject's .git/modules/ too?\n"},{"id":"313464","messageId":"CAGZ79kYmaAY-ASa672eXmQapxJns+a3dEC_BiEmH7ti==arLtw@mail.gmail.com","threadId":"45260","inReplyTo":"xmqqy3wg4rko.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 16/18] entry.c: update submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-07T23:08:21Z","receivedAt":"2017-03-07T23:42:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Mar 7, 2017 at 3:04 PM, Junio C Hamano wrote:\n> Junio C Hamano writes:\n>\n>>> + if (!is_submodule_populated_gently(ce->name, &err)) {\n>>> + struct stat sb;\n>>> + if (lstat(ce->name, &sb))\n>>> + die(_(\"could not stat file '%s'\"), ce->name);\n>>> + if (!(st.st_mode & S_IFDIR))\n>>> + unlink_or_warn(ce->name);\n>>\n>> We found that the path ce->name is supposed to be a submodule that\n>> is checked out, we found that the submodule is not yet populated, we\n>> tried to make sure what is on that path, and its failure would cause\n>> us to die(). All that is sensible.\n>> ...\n>> But if that unlink fails, shouldn't we die, just like the case where\n>> we cannot tell what is at the path ce->name?\n>>\n>> And if that unlink succeeds, instead of having an empty directory,\n>> we start the \"move-head\" call to switch from NULL to ce->oid without\n>> having any directory. Wouldn't we want to mkdir() here (and remove\n>> mkdir() in submodule_move_head() if there is one---if there isn't\n>> then I do not think this codepath would work)?\n>\n> In addition to mkdir(), would we also need the .git file that point\n> into superproject's .git/modules/ too?\n\nThe move_head function takes care of it\nBoth creation as well as deletion are handled in the move_head function,\nwhen either new or old is NULL.\n\nWe can drag that out of that function and have it at the appropriate places\nmanually.\n\nWhy do you think it is better design to have mkdir here and not in move_head?\n(For me having everything in move_head was easier to understand,\nmaybe it is not for others)\n\nThanks,\nStefan\n"},{"id":"313466","messageId":"CAGZ79kbdJa54YyKPHR9ycuJBVtyNY_2yaD7_5RPPu++Awiz5cA@mail.gmail.com","threadId":"45260","inReplyTo":"xmqqfuio674n.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-07T23:37:28Z","receivedAt":"2017-03-07T23:46:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Mar 7, 2017 at 2:42 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> Signed-off-by: Stefan Beller \n\n>> + submodule_move_head(sub->path, \"HEAD\", NULL, \\\n>\n> What is this backslash doing here?\n\nI am still bad at following coding conventions, apparently.\nwill remove.\n\n>> @@ -532,8 +550,13 @@ void remove_marked_cache_entries(struct index_state *istate)\n>>\n>> for (i = j = 0; i < istate->cache_nr; i++) {\n>> if (ce_array[i]->ce_flags & CE_REMOVE) {\n>> - remove_name_hash(istate, ce_array[i]);\n>> - save_or_free_index_entry(istate, ce_array[i]);\n>> + const struct submodule *sub = submodule_from_ce(ce_array[i]);\n>> + if (sub) {\n>> + remove_submodule_according_to_strategy(sub);\n>> + } else {\n>> + remove_name_hash(istate, ce_array[i]);\n>> + save_or_free_index_entry(istate, ce_array[i]);\n>> + }\n>\n> I cannot readily tell as the proposed log message is on the sketchy\n> side to explain the reasoning behind this, but this behaviour change\n> is done unconditionally (in other words, without introducing a flag\n> that is set by --recurse-submodules command line flag and switches\n> behaviour based on the flag) here. What is the end-user visible\n> effect with this change?\n\nsubmodule_from_ce returns always NULL, when such flag is not given.\nFrom 10/18:\n\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n+{\n+ if (!S_ISGITLINK(ce->ce_mode))\n+ return NULL;\n+\n+ if (!should_update_submodules())\n+ return NULL;\n+\n+ return submodule_from_path(null_sha1, ce->name);\n+}\n\nshould_update_submodules is always false if such a flag is not set,\nsuch that we end up in the else case which is literally the same as\nthe removed lines (they are just indented).\n\n> Can we have a new test in this same patch\n> to demonstrate the desired behaviour introduced by this change (or\n> the bug this change fixes)?\n\nThis doesn't fix a bug, but in case the flag is given (in patches 17,18)\nthis new code removes submodules that are no longer recorded in\nthe tree.\n"},{"id":"313474","messageId":"xmqqpohs4ltc.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"CAGZ79kYmaAY-ASa672eXmQapxJns+a3dEC_BiEmH7ti==arLtw@mail.gmail.com","subject":"Re: [PATCH 16/18] entry.c: update submodules when interesting","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-08T01:08:31Z","receivedAt":"2017-03-08T01:09:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n>> In addition to mkdir(), would we also need the .git file that point\n>> into superproject's .git/modules/ too?\n>\n> The move_head function takes care of it\n> Both creation as well as deletion are handled in the move_head function,\n> when either new or old is NULL.\n\nOh, if it does the creation of directory and adjusting of .git\ngitfile, and handles other anomalies, I have no problem with it.\n\nIt was just that this codepath tried to handle some kind of anomaly\n(namely, the user originally thought the submodule needs to be\nchanged to a regular file and then changed mind and wants to restore\nit from the index) and yet not doing the full anomaly handling (like\n\"not even an empty directory exists\" case) that confused me.\n\n"},{"id":"313476","messageId":"xmqqlgsg4lj4.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"CAGZ79kbdJa54YyKPHR9ycuJBVtyNY_2yaD7_5RPPu++Awiz5cA@mail.gmail.com","subject":"Re: [PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-08T01:14:39Z","receivedAt":"2017-03-08T01:20:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> submodule_from_ce returns always NULL, when such flag is not given.\n> From 10/18:\n>\n> +const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n> +{\n> + if (!S_ISGITLINK(ce->ce_mode))\n> + return NULL;\n> +\n> + if (!should_update_submodules())\n> + return NULL;\n> +\n> + return submodule_from_path(null_sha1, ce->name);\n> +}\n>\n> should_update_submodules is always false if such a flag is not set,\n> such that we end up in the else case which is literally the same as\n> the removed lines (they are just indented).\n\nI see. \n\nI didn't think a function this deep in the callchain that does not\ntake any parameter could possibly change the behaviour based on the\nend-user input. I was expecting that such a state (i.e. are we\nrecursive? are we allowed to forcibly update the working tree\nfiles?) would be kept part of something like \"struct checkout\" and\npassed around the callchain.\n\nThat was why I didn't look at how that function answers \"should\nupdate?\" question, and got puzzled. Because it would imply there is\nsome hidden state that is accessible by everybody--a global variable\nor something--which would point at a deeper design issue.\n"},{"id":"313574","messageId":"CAGZ79kYF63E17SNa9wt3X_28kbvjNujUBPoMct=RvDcyOeJm-w@mail.gmail.com","threadId":"45260","inReplyTo":"xmqqlgsg4lj4.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-08T22:39:21Z","receivedAt":"2017-03-08T23:36:36Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Mar 7, 2017 at 5:14 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> submodule_from_ce returns always NULL, when such flag is not given.\n>> From 10/18:\n>>\n>> +const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n>> +{\n>> + if (!S_ISGITLINK(ce->ce_mode))\n>> + return NULL;\n>> +\n>> + if (!should_update_submodules())\n>> + return NULL;\n>> +\n>> + return submodule_from_path(null_sha1, ce->name);\n>> +}\n>>\n>> should_update_submodules is always false if such a flag is not set,\n>> such that we end up in the else case which is literally the same as\n>> the removed lines (they are just indented).\n>\n> I see.\n>\n> I didn't think a function this deep in the callchain that does not\n> take any parameter could possibly change the behaviour based on the\n> end-user input. I was expecting that such a state (i.e. are we\n> recursive? are we allowed to forcibly update the working tree\n> files?) would be kept part of something like \"struct checkout\" and\n> passed around the callchain.\n>\n> That was why I didn't look at how that function answers \"should\n> update?\" question, and got puzzled. Because it would imply there is\n> some hidden state that is accessible by everybody--a global variable\n> or something--which would point at a deeper design issue.\n\nWell it is just as deep as e.g. reacting on some bits of\nstruct unpack_trees_options in unpack-trees.c ?\n\nBut I can see how this is an issue with design.\nI also think my previous answer was a straw man.\n\nWe only need to differentiate between 'ce' being a submodule\nor not, because it should not be marked CE_REMOVE in\nthe non-recursive state.\n\nSide-question:\nIs there some doc (commit message), that explains the difference\nbetween CE_REMOVE and CE_WT_REMOVE ?\n\nThanks,\nStefan\n"},{"id":"313575","messageId":"xmqq1su71gt8.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"CAGZ79kYF63E17SNa9wt3X_28kbvjNujUBPoMct=RvDcyOeJm-w@mail.gmail.com","subject":"Re: [PATCH 15/18] read-cache, remove_marked_cache_entries: wipe selected submodules.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-08T23:37:07Z","receivedAt":"2017-03-08T23:37:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Side-question:\n> Is there some doc (commit message), that explains the difference\n> between CE_REMOVE and CE_WT_REMOVE ?\n\nThat's something you need to ask Duy, I think, as it was introduced\nat e663db2f (\"unpack-trees(): add CE_WT_REMOVE to remove on worktree\nalone\", 2009-08-20) and was added for the sparse checkout stuff.\n\nDuring my review of the series that added the feature, I only had\ntime to make sure that the patches do not change the behaviour when\nit is not in use, ignoring the other side of \"if\" statement that\nchecked ce_skip_worktree(ce) ;-)\n"},{"id":"313688","messageId":"20170309221543.15897-2-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 01/17] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:27Z","receivedAt":"2017-03-09T22:16:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand. The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..5df528ea81 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n # replace_file_with_sub1)\n #\n-# --O-----O\n-# / ^ replace_directory_with_sub1\n-# / replace_sub1_with_directory\n-# /----O\n-# / ^\n-# / modify_sub1\n-# O------O-------O\n-# ^ ^\\ ^\n-# | | \\ remove_sub1\n-# | | -----O-----O\n-# | | \\ ^ replace_file_with_sub1\n-# | | \\ replace_sub1_with_file\n-# | add_sub1 --O-----O\n-# no_submodule ^ valid_sub1\n-# invalid_sub1\n+# ----O\n+# / ^\n+# / remove_sub1\n+# /\n+# add_sub1 /-------O\n+# | / ^\n+# | / modify_sub1\n+# v/\n+# O------O-----------O---------O\n+# ^ \\ ^ replace_directory_with_sub1\n+# | \\ replace_sub1_with_directory\n+# no_submodule \\\n+# --------O---------O\n+# \\ ^ replace_file_with_sub1\n+# \\ replace_sub1_with_file\n+# \\\n+# ----O---------O\n+# ^ valid_sub1\n+# invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313689","messageId":"20170309221543.15897-9-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 08/17] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:34Z","receivedAt":"2017-03-09T22:16:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller \n---\n submodule-config.c | 20 ++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 29 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..3e8e380d98 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,26 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t unsigned char *gitmodules_sha1,\n \t\t\t\t struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313690","messageId":"20170309221543.15897-4-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 03/17] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:29Z","receivedAt":"2017-03-09T22:16:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex c0d6325133..00128f28b5 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -193,6 +193,11 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n+\tif test x\"$1\" = \"x-C\"\n+\tthen\n+\t\tcd \"$2\"\n+\t\tshift; shift;\n+\tfi\n \tif test $# != 2\n \tthen\n \t\techo \"test_submodule_content needs two arguments\"\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313691","messageId":"20170309221543.15897-11-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 10/17] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:36Z","receivedAt":"2017-03-09T22:16:15Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that checks if a submodule needs\nto be checked for an update before attempting the update.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 16 ++++++++++++++++\n submodule.h | 7 +++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 591f4a694e..8b2c0212be 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -548,6 +548,22 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int should_update_submodules(void)\n+{\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n+{\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn NULL;\n+\n+\tif (!should_update_submodules())\n+\t\treturn NULL;\n+\n+\treturn submodule_from_path(null_sha1, ce->name);\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex b4e60c08d2..6f3fe85c7c 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,6 +65,13 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+/* Check if we want to update any submodule.*/\n+extern int should_update_submodules(void);\n+/*\n+ * Returns the submodule struct if the given ce entry is a submodule\n+ * and it should be updated. Returns NULL otherwise.\n+ */\n+extern const struct submodule *submodule_from_ce(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313692","messageId":"20170309221543.15897-13-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 12/17] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:38Z","receivedAt":"2017-03-09T22:16:20Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n \"Is it safe to change the submodule to this new state?\"\n e.g. is it overwriting untracked files or are there local\n changes that would be overwritten?\n* supports a force flag that can be used for resetting\n the tree.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h | 7 ++++\n 2 files changed, 142 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 0b2596e88a..bc5fecf8c5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n+\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.no_stdout = 1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n+\n+\treturn finish_command(&cp);\n+}\n+\n+static void submodule_reset_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not reset submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ */\n+int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub))\n+\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_reset_index(path);\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n+\t\targv_array_push(&cp.args, \"-n\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-u\");\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 6f3fe85c7c..4cdf6445f7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -96,6 +96,13 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n+#define SUBMODULE_MOVE_HEAD_FORCE (1<<1)\n+extern int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags);\n+\n /*\n * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313693","messageId":"20170309221543.15897-8-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 07/17] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:33Z","receivedAt":"2017-03-09T22:16:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. This patch makes\nconnect_work_tree_and_git_dir safe to use for both cases of\neither the git dir or the working dir missing.\n\nTo do so, we need to call safe_create_leading_directories[_const]\non both directories. However this has to happen before we construct\nthe absolute paths as real_pathdup assumes the directories to\nbe there already.\n\nSo for both the config file in the git dir as well as the .git link\nfile we need to\na) construct the name\nb) call SCLD\nc) get the absolute path\nd) once a-c is done for both we can consume the absolute path\n to compute the relative path to each other and store those\n relative paths.\n\nThe implementation provided here puts a) and b) for both cases first,\nand then performs c and d after.\n\nOne of the two users of 'connect_work_tree_and_git_dir' already checked\nfor the directory being there, so we can loose that check as\nconnect_work_tree_and_git_dir handles this functionality now.\n\nSigned-off-by: Stefan Beller \n---\n dir.c | 32 +++++++++++++++++++++-----------\n submodule.c | 11 ++---------\n 2 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..6f52af7abb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n /* Update gitfile and core.worktree setting to connect work tree and git dir */\n void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n {\n-\tstruct strbuf file_name = STRBUF_INIT;\n+\tstruct strbuf gitfile_sb = STRBUF_INIT;\n+\tstruct strbuf cfg_sb = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n-\tchar *git_dir = real_pathdup(git_dir_);\n-\tchar *work_tree = real_pathdup(work_tree_);\n+\tchar *git_dir, *work_tree;\n \n-\t/* Update gitfile */\n-\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n-\twrite_file(file_name.buf, \"gitdir: %s\",\n-\t\t relative_path(git_dir, work_tree, &rel_path));\n+\t/* Prepare .git file */\n+\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n+\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n+\n+\t/* Prepare config file */\n+\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n+\tif (safe_create_leading_directories_const(cfg_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n \n+\tgit_dir = real_pathdup(git_dir_);\n+\twork_tree = real_pathdup(work_tree_);\n+\n+\t/* Write .git file */\n+\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n+\t\t relative_path(git_dir, work_tree, &rel_path));\n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n+\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n \t\t\t relative_path(work_tree, git_dir, &rel_path));\n \n-\tstrbuf_release(&file_name);\n+\tstrbuf_release(&gitfile_sb);\n+\tstrbuf_release(&cfg_sb);\n \tstrbuf_release(&rel_path);\n \tfree(work_tree);\n \tfree(git_dir);\ndiff --git a/submodule.c b/submodule.c\nindex 0e55372f37..04d185738f 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1442,8 +1442,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \n \t/* Not populated? */\n \tif (!sub_git_dir) {\n-\t\tchar *real_new_git_dir;\n-\t\tconst char *new_git_dir;\n \t\tconst struct submodule *sub;\n \n \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n@@ -1466,13 +1464,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tsub = submodule_from_path(null_sha1, path);\n \t\tif (!sub)\n \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n-\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n-\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n-\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n-\t\treal_new_git_dir = real_pathdup(new_git_dir);\n-\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n-\n-\t\tfree(real_new_git_dir);\n+\t\tconnect_work_tree_and_git_dir(path,\n+\t\t\tgit_path(\"modules/%s\", sub->name));\n \t} else {\n \t\t/* Is it already absorbed into the superprojects git dir? */\n \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313694","messageId":"20170309221543.15897-6-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 05/17] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:31Z","receivedAt":"2017-03-09T22:16:26Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh: test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 506 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 504 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex f52c49c838..deba618bee 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (add_nested_sub => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n # / ^\n # / remove_sub1\n # /\n-# add_sub1 /-------O\n-# | / ^\n+# add_sub1 /-------O---------O--------O modify_sub1_recursively\n+# | / ^ add_nested_sub\n # | / modify_sub1\n # v/\n # O------O-----------O---------O\n@@ -48,6 +49,17 @@ create_lib_submodule_repo () {\n \t\tgit commit -m \"Base inside first submodule\" &&\n \t\tgit branch \"no_submodule\"\n \t) &&\n+\tgit init submodule_update_sub2 &&\n+\t(\n+\t\tcd submodule_update_sub2\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"nested submodule base\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -84,6 +96,26 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b add_nested_sub modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1 checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 checkout -b modify_sub1_recursively &&\n+\t\techo change >sub1/sub2/file3 &&\n+\t\tgit -C sub1/sub2 add file3 &&\n+\t\tgit -C sub1/sub2 commit -m \"make a change in nested sub\" &&\n+\t\tgit -C sub1 add sub2 &&\n+\t\tgit -C sub1 commit -m \"update nested sub\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -m \"update sub1, that updates nested sub\" &&\n+\t\tgit -C sub1 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -150,6 +182,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -180,6 +221,27 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# make the submodule git dirs available\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree\n+\tfi &&\n+\tif ! test -d submodule_update/.git/modules/sub1/modules/sub2\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules/sub1/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree\n+\tfi &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# sub1 might not be checked out, so use the git dir\n+\tgit -C submodule_update/.git/modules/sub1 config submodule.sub2.url \"bogus\"\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -700,3 +762,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+# in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+# git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 >.git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignored\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_nested_sub &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content -C sub1 sub2 origin/modify_sub1_recursively\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313695","messageId":"20170309221543.15897-17-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 16/17] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:42Z","receivedAt":"2017-03-09T22:16:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This exposes a flag to recurse into submodules\nin builtin/checkout making use of the code implemented\nin prior patches.\n\nA new failure mode is introduced in the submodule\nupdate library, as the directory/submodule conflict\nis not solved in prior patches.\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-checkout.txt | 7 +++++++\n builtin/checkout.c | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 23 +++++++++++++++++------\n t/t2013-checkout-submodule.sh | 5 +++++\n 4 files changed, 57 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..d6399c0af8 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n ::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..e9c5fcfaf8 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [] \"),\n \tN_(\"git checkout [] [] -- ...\"),\n \tNULL,\n };\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout '\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex deba618bee..0b659b530d 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -782,6 +782,11 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -891,7 +896,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -903,7 +908,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -931,7 +936,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -987,7 +992,8 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t# recursing deeper than one level doesn't work yet.\n+\ttest_expect_failure \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n@@ -1006,6 +1012,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1100,7 +1111,7 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -1151,7 +1162,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..aa35223369 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,11 @@ test_expect_success '\"checkout \" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313696","messageId":"20170309221543.15897-18-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 17/17] builtin/read-tree: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:43Z","receivedAt":"2017-03-09T22:16:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"A new known failure mode is introduced[1], which is actually not\na failure but a feature in read-tree. Unlike checkout for which\nthe recursive submodule tests were originally written, read-tree does\nwarn about ignored untracked files that would be overwritten.\nFor the sake of keeping the test library for submodules generic, just\nmark the test as a failure.\n\n[1] KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-read-tree.txt | 6 ++++++\n builtin/read-tree.c | 29 +++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 7 ++++++-\n t/t1013-read-tree-submodule.sh | 7 +++++++\n 4 files changed, 48 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt\nindex fa1d557e5b..ed9d63ef4a 100644\n--- a/Documentation/git-read-tree.txt\n+++ b/Documentation/git-read-tree.txt\n@@ -115,6 +115,12 @@ OPTIONS\n \tdirectories the index file and index output file are\n \tlocated in.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject by\n+\tcalling read-tree recursively, also setting the submodules HEAD to be\n+\tdetached at that commit.\n+\n --no-sparse-checkout::\n \tDisable sparse checkout support even if `core.sparseCheckout`\n \tis true.\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 8ba64bc785..23e212ee8c 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -15,10 +15,13 @@\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"resolve-undo.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\n@@ -96,6 +99,23 @@ static int debug_merge(const struct cache_entry * const *stages,\n \treturn 0;\n }\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n static struct lock_file lock_file;\n \n int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n@@ -137,6 +157,9 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n \t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_END()\n \t};\n \n@@ -152,6 +175,12 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \n \thold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT) {\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tset_config_update_recurse_submodules(RECURSE_SUBMODULES_ON);\n+\t}\n+\n \tprefix_set = opts.prefix ? 1 : 0;\n \tif (1 < opts.merge + opts.reset + prefix_set)\n \t\tdie(\"Which one? -m, --reset, or --prefix?\");\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 0b659b530d..e195b590a1 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -787,6 +787,11 @@ test_submodule_switch_recursing () {\n \tthen\n \t\tRESULT=failure\n \tfi\n+\tRESULT1=success\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n+\tthen\n+\t\tRESULT1=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -827,7 +832,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULT1 \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 20526aed34..7019d0a04f 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,13 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n+\n+test_submodule_switch_recursing \"git read-tree --recurse-submodules -u -m\"\n+\n+test_submodule_forced_switch_recursing \"git read-tree --recurse-submodules -u --reset\"\n+\n test_submodule_switch \"git read-tree -u -m\"\n \n test_submodule_forced_switch \"git read-tree -u --reset\"\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313697","messageId":"20170309221543.15897-16-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 15/17] entry.c: create submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:41Z","receivedAt":"2017-03-09T22:16:32Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a submodule is introduced with a new revision\nwe need to create the submodule in the worktree as well.\nAs 'submodule_move_head' handles edge cases, all we have\nto do is call it from within the function that creates\nnew files in the working tree for workingtree operations.\n\nSigned-off-by: Stefan Beller \n---\n entry.c | 30 ++++++++++++++++++++++++++++++\n 1 file changed, 30 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..d2b512da90 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t const struct checkout *state)\n@@ -146,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n \tunsigned long size;\n \tsize_t wrote, newsize = 0;\n \tstruct stat st;\n+\tconst struct submodule *sub;\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n@@ -203,6 +205,10 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub)\n+\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -259,7 +265,31 @@ int checkout_entry(struct cache_entry *ce,\n \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n+\t\tconst struct submodule *sub;\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\t} else\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313698","messageId":"20170309221543.15897-10-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 09/17] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:35Z","receivedAt":"2017-03-09T22:16:35Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 04d185738f..591f4a694e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -542,6 +543,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 0b915bd3ac..b4e60c08d2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -64,6 +64,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313699","messageId":"20170309221543.15897-12-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 11/17] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:37Z","receivedAt":"2017-03-09T22:16:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 8b2c0212be..0b2596e88a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -356,6 +356,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n * summary output. If it can locate the submodule objects directory it will\n * attempt to lookup both the left and right commits and put them into the\n@@ -1390,18 +1407,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n * Embeds a single submodules git directory into the superprojects git dir,\n * non recursively.\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313700","messageId":"20170309221543.15897-14-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 13/17] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:39Z","receivedAt":"2017-03-09T22:16:42Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller \n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n * Currently, git does not checkout subprojects during a superproject\n * checkout, so it is not going to overwrite anything.\n */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t const struct cache_entry *ce,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313701","messageId":"20170309221543.15897-5-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 04/17] lib-submodule-update.sh: replace sha1 by hash","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:30Z","receivedAt":"2017-03-09T22:16:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Cleaning up code by generalising it.\nCurrently the mailing list discusses yet again how\nto migrate away from sha1.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 00128f28b5..f52c49c838 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -171,9 +171,9 @@ reset_work_tree_to () {\n \t\tgit checkout -f \"$1\" &&\n \t\tgit status -u -s >actual &&\n \t\ttest_must_be_empty actual &&\n-\t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n-\t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\thash=$(git rev-parse --revs-only HEAD:sub1) &&\n+\t\tif test -n \"$hash\" &&\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$hash^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313702","messageId":"20170309221543.15897-15-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 14/17] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:40Z","receivedAt":"2017-03-09T22:16:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll support submodule entries to be\nin sync with the tree in working tree changing commands,\nsuch as checkout or read-tree.\n\nWhen a new submodule entry changes in the tree, we need to\ncheck if there are conflicts (directory/file conflicts)\nfor the tree. Add this check for submodules to be\nperformed before the working tree is touched.\n\nSigned-off-by: Stefan Beller \n---\n unpack-trees.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++++++----\n unpack-trees.h | 1 +\n 2 files changed, 123 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..8333da2cc9 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,8 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /*\n * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +47,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_SUBMODULE */\n+\t\"Submodule '%s' cannot checkout new HEAD.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +166,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot checkout new HEAD\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +247,75 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int check_submodule_move_head(const struct cache_entry *ce,\n+\t\t\t\t const char *old_id,\n+\t\t\t\t const char *new_id,\n+\t\t\t\t struct unpack_trees_options *o)\n+{\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\t\tif (submodule_move_head(ce->name, old_id, new_id, SUBMODULE_MOVE_HEAD_DRY_RUN))\n+\t\t\treturn o->gently ? -1 :\n+\t\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);\n+\t\treturn 0;\n+\tcase SM_UPDATE_NONE:\n+\t\treturn 0;\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\tcase SM_UPDATE_COMMAND:\n+\tdefault:\n+\t\twarning(_(\"submodule update strategy not supported for submodule '%s'\"), ce->name);\n+\t\treturn -1;\n+\t}\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tsubmodule_free();\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t\tgitmodules_config();\n+\t\t\t\tgit_config(submodule_config, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n /*\n * Unlink the last component and schedule the leading directories for\n * removal, such that empty directories get removed.\n */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (sub) {\n+\t\tswitch (sub->update_strategy.type) {\n+\t\tcase SM_UPDATE_UNSPECIFIED:\n+\t\tcase SM_UPDATE_CHECKOUT:\n+\t\tcase SM_UPDATE_REBASE:\n+\t\tcase SM_UPDATE_MERGE:\n+\t\t\tsubmodule_move_head(ce->name, \"HEAD\", NULL,\n+\t\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\tbreak;\n+\t\tcase SM_UPDATE_NONE:\n+\t\tcase SM_UPDATE_COMMAND:\n+\t\t\treturn; /* Do not touch the submodule. */\n+\t\t}\n+\t}\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +371,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (should_update_submodules() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1431,26 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint r = check_submodule_move_head(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index. This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1494,11 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!submodule_from_ce(ce))\n+\t\treturn 0;\n+\n+\treturn check_submodule_move_head(ce, old_sha1,\n+\t\t\t\t\t oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1664,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (submodule_from_ce(ce))\n+\t\t\t\tret = check_submodule_move_head(ce,\n+\t\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1680,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (submodule_from_ce(ce))\n+\t\t\treturn check_submodule_move_head(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t error_type, o);\n@@ -1643,6 +1739,15 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, NULL,\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1768,14 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..6c48117b84 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313703","messageId":"20170309221543.15897-3-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 02/17] lib-submodule-update.sh: do not use ./. as submodule remote","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:28Z","receivedAt":"2017-03-09T22:17:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Adding the repository itself as a submodule does not make sense in the\nreal world. In our test suite we used to do that out of convenience in\nsome tests as the current repository has easiest access for setting up\n'just a submodule'.\n\nHowever this doesn't quite test the real world, so let's do not follow\nthis pattern any further and actually create an independent repository\nthat we can use as a submodule.\n\nWhen using './.' as the remote the superproject and submodule share the\nsame objects, such that testing if a given sha1 is a valid commit works\nin either repository. As running commands in an unpopulated submodule\nfall back to the superproject, this happens in `reset_work_tree_to`\nto determine if we need to populate the submodule. Fix this bug by\nchecking in the actual remote now.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 15 +++++++++++++--\n 1 file changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5df528ea81..c0d6325133 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -37,6 +37,17 @@\n #\n \n create_lib_submodule_repo () {\n+\tgit init submodule_update_sub1 &&\n+\t(\n+\t\tcd submodule_update_sub1 &&\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"Base inside first submodule\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -49,7 +60,7 @@ create_lib_submodule_repo () {\n \t\tgit branch \"no_submodule\" &&\n \n \t\tgit checkout -b \"add_sub1\" &&\n-\t\tgit submodule add ./. sub1 &&\n+\t\tgit submodule add ../submodule_update_sub1 sub1 &&\n \t\tgit config -f .gitmodules submodule.sub1.ignore all &&\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n@@ -162,7 +173,7 @@ reset_work_tree_to () {\n \t\ttest_must_be_empty actual &&\n \t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n \t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313704","messageId":"20170309221543.15897-7-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCH 06/17] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:32Z","receivedAt":"2017-03-09T22:17:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller \n---\n builtin/grep.c | 2 +-\n submodule.c | 7 ++-----\n submodule.h | 8 +++++++-\n 3 files changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..0e55372f37 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -234,15 +234,12 @@ int is_submodule_initialized(const char *path)\n \treturn ret;\n }\n \n-/*\n- * Determine if a submodule has been populated at a given 'path'\n- */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..0b915bd3ac 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,13 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+/*\n+ * Determine if a submodule has been populated at a given 'path' by checking if\n+ * the /.git resolves to a valid git repository.\n+ * If return_error_code is NULL, die on error.\n+ * Otherwise the return error code is the same as of resolve_gitdir_gently.\n+ */\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc1.45.g207f5fbb2b\n\n"},{"id":"313705","messageId":"20170309221543.15897-1-sbeller@google.com","threadId":"45260","inReplyTo":"20170306205919.9713-1-sbeller@google.com","subject":"[RFCv8 PATCH 00/17] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-09T22:15:26Z","receivedAt":"2017-03-09T22:17:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20170306205919.9713-1-sbeller@google.com/\n\nv8:\n * Thanks Ramsay; another variable is static now!\n * Thanks Junio; fixed typos (in comments and commit messages)!\n * dropped the patch touching read-cache.c; I realized it is bogus after\n inspecting it and trying to write a commit message.\n * All commits have commit messages now.\n * diff to Junios version below.\n \nv7:\n * addressed Erics comment by fixing the bashism in t/lib-submodule-update.sh\ndiff to v6:\n diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\n index 5caae06bc5..949ebd546c 100755\n --- a/t/lib-submodule-update.sh\n +++ b/t/lib-submodule-update.sh\n @@ -255,7 +255,7 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n - if test \"$1\" == \"-C\"\n + if test x\"$1\" = \"x-C\"\n then\n cd \"$2\"\n shift; shift;\n * interdiff to v5 below\n \n\nv6:\n * added support for read-tree (see last patch) to see how generic the\n code of the previous patches is. I am pretty pleased with that patch.\n * marked two functions static. Thanks Ramsay!\n * fixed the recursive test; it still fails but it is the code that fails,\n not the test. For this I had to change the setup code slightly.\n * 2 new patches adding tiny refactoring to the submodule test lib. \n * interdiff (to origin/sb/checkout-recurse-submodules, which is v5) below.\n\nv5:\n * as v4 was the first version queued by Junio, we do have an interdiff below!\n * renamed functions\n * changed the API, now the caller has to take care of the submodule strategy\n themselves. (Note this can be different for different situations, e.g.\n when a submodule is deleted, we can do that for any strategy except none and\n !command. But for moving to a new state of the submodule we currently\n only implement \"checkout\" [unspecified defaults to checkout]. warning about\n the others, doing nothing there.)\n\nv4:\n * addressed all comments of Brian, Junio and Brandon.\n Thanks!\n * one major point of change is the introduction of another patch\n \"lib-submodule-update.sh: do not use ./. as submodule remote\",\n as that took some time to track down the existing bug.\n \nv3:\n * moved tests from t2013 to the generic submodule library.\n * factored out the refactoring patches to be up front\n * As I redid the complete implementation, I have the impression this time\n it is cleaner than previous versions.\n \n I think we still have to fix the corner cases of directory/file/submodule \n conflicts before merging, but this serves as a status update on my current\n way of thinking how to implement the worktree commands being aware of\n submodules.\n \nThanks,\nStefan\n\nv2:\n* based on top of the series sent out an hour ago\n \"[PATCHv4 0/5] submodule embedgitdirs\"\n* Try to embed a submodule if we need to remove it.\n* Strictly do not change behavior if not giving the new flag.\n* I think I missed some review comments from v1, but I'd like to get\n the current state out over the weekend, as a lot has changed so far.\n On Monday I'll go through the previous discussion with a comb to see\n if I missed something.\n \nv1:\nWhen working with submodules, nearly anytime after checking out\na different state of the projects, that has submodules changed\nyou'd run \"git submodule update\" with a current version of Git.\n\nThere are two problems with this approach:\n\n* The \"submodule update\" command is dangerous as it\n doesn't check for work that may be lost in the submodule\n (e.g. a dangling commit).\n* you may forget to run the command as checkout is supposed\n to do all the work for you.\n\nIntegrate updating the submodules into git checkout, with the same\nsafety promises that git-checkout has, i.e. not throw away data unless\nasked to. This is done by first checking if the submodule is at the same\nsha1 as it is recorded in the superproject. If there are changes we stop\nproceeding the checkout just like it is when checking out a file that\nhas local changes.\n\nThe integration happens in the code that is also used in other commands\nsuch that it will be easier in the future to make other commands aware\nof submodule.\n\nThis also solves d/f conflicts in case you replace a file/directory\nwith a submodule or vice versa.\n\nThe patches are still a bit rough, but the overall series seems\npromising enough to me that I want to put it out here.\n\nAny review, specifically on the design level welcome!\n\nThanks,\nStefan\n \ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 2dc5cc06da..23e212ee8c 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -21,7 +21,7 @@\n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n-int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\ndiff --git a/read-cache.c b/read-cache.c\nindex 9a2abacf7a..9054369dd0 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -18,8 +18,6 @@\n #include \"varint.h\"\n #include \"split-index.h\"\n #include \"utf8.h\"\n-#include \"submodule.h\"\n-#include \"submodule-config.h\"\n \n /* Mask for the name length in ce_flags in the on-disk index */\n \n@@ -522,22 +520,6 @@ int remove_index_entry_at(struct index_state *istate, int pos)\n \treturn 1;\n }\n \n-static void remove_submodule_according_to_strategy(const struct submodule *sub)\n-{\n-\tswitch (sub->update_strategy.type) {\n-\tcase SM_UPDATE_UNSPECIFIED:\n-\tcase SM_UPDATE_CHECKOUT:\n-\tcase SM_UPDATE_REBASE:\n-\tcase SM_UPDATE_MERGE:\n-\t\tsubmodule_move_head(sub->path, \"HEAD\", NULL, \\\n-\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n-\t\tbreak;\n-\tcase SM_UPDATE_NONE:\n-\tcase SM_UPDATE_COMMAND:\n-\t\t; /* Do not touch the submodule. */\n-\t}\n-}\n-\n /*\n * Remove all cache entries marked for removal, that is where\n * CE_REMOVE is set in ce_flags. This is much more effective than\n@@ -550,13 +532,8 @@ void remove_marked_cache_entries(struct index_state *istate)\n \n \tfor (i = j = 0; i < istate->cache_nr; i++) {\n \t\tif (ce_array[i]->ce_flags & CE_REMOVE) {\n-\t\t\tconst struct submodule *sub = submodule_from_ce(ce_array[i]);\n-\t\t\tif (sub) {\n-\t\t\t\tremove_submodule_according_to_strategy(sub);\n-\t\t\t} else {\n-\t\t\t\tremove_name_hash(istate, ce_array[i]);\n-\t\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n-\t\t\t}\n+\t\t\tremove_name_hash(istate, ce_array[i]);\n+\t\t\tsave_or_free_index_entry(istate, ce_array[i]);\n \t\t}\n \t\telse\n \t\t\tce_array[j++] = ce_array[i];\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 949ebd546c..e195b590a1 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -954,7 +954,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\t# ... and ignored files are ignroed\n+\t# ... and ignored files are ignored\n \ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n \t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n \t\tprolog &&\n"},{"id":"313721","messageId":"20170309232940.GC52558@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-8-sbeller@google.com","subject":"Re: [PATCH 07/17] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-03-09T23:29:40Z","receivedAt":"2017-03-09T23:29:49Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 03/09, Stefan Beller wrote:\n> In a later patch we'll use connect_work_tree_and_git_dir when the\n> directory for the gitlink file doesn't exist yet. This patch makes\n> connect_work_tree_and_git_dir safe to use for both cases of\n> either the git dir or the working dir missing.\n> \n> To do so, we need to call safe_create_leading_directories[_const]\n> on both directories. However this has to happen before we construct\n> the absolute paths as real_pathdup assumes the directories to\n> be there already.\n> \n> So for both the config file in the git dir as well as the .git link\n> file we need to\n> a) construct the name\n> b) call SCLD\n> c) get the absolute path\n> d) once a-c is done for both we can consume the absolute path\n> to compute the relative path to each other and store those\n> relative paths.\n> \n> The implementation provided here puts a) and b) for both cases first,\n> and then performs c and d after.\n> \n> One of the two users of 'connect_work_tree_and_git_dir' already checked\n> for the directory being there, so we can loose that check as\n> connect_work_tree_and_git_dir handles this functionality now.\n> \n> Signed-off-by: Stefan Beller \n> ---\n> dir.c | 32 +++++++++++++++++++++-----------\n> submodule.c | 11 ++---------\n> 2 files changed, 23 insertions(+), 20 deletions(-)\n> \n> diff --git a/dir.c b/dir.c\n> index 4541f9e146..6f52af7abb 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n> /* Update gitfile and core.worktree setting to connect work tree and git dir */\n> void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n> {\n> -\tstruct strbuf file_name = STRBUF_INIT;\n> +\tstruct strbuf gitfile_sb = STRBUF_INIT;\n> +\tstruct strbuf cfg_sb = STRBUF_INIT;\n> \tstruct strbuf rel_path = STRBUF_INIT;\n> -\tchar *git_dir = real_pathdup(git_dir_);\n> -\tchar *work_tree = real_pathdup(work_tree_);\n> +\tchar *git_dir, *work_tree;\n> \n> -\t/* Update gitfile */\n> -\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n> -\twrite_file(file_name.buf, \"gitdir: %s\",\n> -\t\t relative_path(git_dir, work_tree, &rel_path));\n> +\t/* Prepare .git file */\n> +\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n> +\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n> +\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n> +\n> +\t/* Prepare config file */\n> +\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n> +\tif (safe_create_leading_directories_const(cfg_sb.buf))\n> +\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n> \n> +\tgit_dir = real_pathdup(git_dir_);\n> +\twork_tree = real_pathdup(work_tree_);\n\nJust a note that this is a spot that'll be affected by the change to\nreal_pathdup() which adds a 'die_on_error' parameter to correct bad\nbehaviour I introduced.\n\n> +\n> +\t/* Write .git file */\n> +\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n> +\t\t relative_path(git_dir, work_tree, &rel_path));\n> \t/* Update core.worktree setting */\n> -\tstrbuf_reset(&file_name);\n> -\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n> -\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n> +\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n> \t\t\t relative_path(work_tree, git_dir, &rel_path));\n> \n> -\tstrbuf_release(&file_name);\n> +\tstrbuf_release(&gitfile_sb);\n> +\tstrbuf_release(&cfg_sb);\n> \tstrbuf_release(&rel_path);\n> \tfree(work_tree);\n> \tfree(git_dir);\n> diff --git a/submodule.c b/submodule.c\n> index 0e55372f37..04d185738f 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1442,8 +1442,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n> \n> \t/* Not populated? */\n> \tif (!sub_git_dir) {\n> -\t\tchar *real_new_git_dir;\n> -\t\tconst char *new_git_dir;\n> \t\tconst struct submodule *sub;\n> \n> \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n> @@ -1466,13 +1464,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n> \t\tsub = submodule_from_path(null_sha1, path);\n> \t\tif (!sub)\n> \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n> -\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n> -\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n> -\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n> -\t\treal_new_git_dir = real_pathdup(new_git_dir);\n> -\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n> -\n> -\t\tfree(real_new_git_dir);\n> +\t\tconnect_work_tree_and_git_dir(path,\n> +\t\t\tgit_path(\"modules/%s\", sub->name));\n> \t} else {\n> \t\t/* Is it already absorbed into the superprojects git dir? */\n> \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n> -- \n> 2.12.0.rc1.45.g207f5fbb2b\n> \n\n-- \nBrandon Williams\n"},{"id":"313722","messageId":"20170309233709.GD52558@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-13-sbeller@google.com","subject":"Re: [PATCH 12/17] update submodules: add submodule_move_head","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-03-09T23:37:09Z","receivedAt":"2017-03-09T23:37:33Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 03/09, Stefan Beller wrote:\n> In later patches we introduce the options and flag for commands\n> that modify the working directory, e.g. git-checkout.\n> \n> This piece of code will be used universally for\n> all these working tree modifications as it\n> * supports dry run to answer the question:\n> \"Is it safe to change the submodule to this new state?\"\n> e.g. is it overwriting untracked files or are there local\n> changes that would be overwritten?\n> * supports a force flag that can be used for resetting\n> the tree.\n> \n> Signed-off-by: Stefan Beller \n> ---\n> submodule.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n> submodule.h | 7 ++++\n> 2 files changed, 142 insertions(+)\n> \n> diff --git a/submodule.c b/submodule.c\n> index 0b2596e88a..bc5fecf8c5 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n> \treturn ret;\n> }\n> \n> +static int submodule_has_dirty_index(const struct submodule *sub)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n> +\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n\nThe formatting of this line is a little odd. Also you can drop the\nbackslash.\n\nMostly because I haven't dug too deep yet/can't remember: Does\ndiff-index do the correct thing with nested submodules? If I remember\ncorrectly it does so this shouldn't be a problem.\n\n> +\tcp.no_stdin = 1;\n> +\tcp.no_stdout = 1;\n> +\tcp.dir = sub->path;\n> +\tif (start_command(&cp))\n> +\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n> +\n> +\treturn finish_command(&cp);\n> +}\n> +\n> +static void submodule_reset_index(const char *path)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n> +\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n> +\n> +\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n> +\n> +\tif (run_command(&cp))\n> +\t\tdie(\"could not reset submodule index\");\n> +}\n> +\n> +/**\n> + * Moves a submodule at a given path from a given head to another new head.\n> + * For edge cases (a submodule coming into existence or removing a submodule)\n> + * pass NULL for old or new respectively.\n> + */\n> +int submodule_move_head(const char *path,\n> +\t\t\t const char *old,\n> +\t\t\t const char *new,\n> +\t\t\t unsigned flags)\n> +{\n> +\tint ret = 0;\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tconst struct submodule *sub;\n> +\n> +\tsub = submodule_from_path(null_sha1, path);\n> +\n> +\tif (!sub)\n> +\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n> +\n> +\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n> +\t\t/* Check if the submodule has a dirty index. */\n> +\t\tif (submodule_has_dirty_index(sub))\n> +\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n> +\t}\n> +\n> +\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n> +\t\tif (old) {\n> +\t\t\tif (!submodule_uses_gitfile(path))\n> +\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n> +\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n> +\t\t} else {\n> +\t\t\tstruct strbuf sb = STRBUF_INIT;\n> +\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n> +\t\t\t\t get_git_common_dir(), sub->name);\n> +\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n> +\t\t\tstrbuf_release(&sb);\n> +\n> +\t\t\t/* make sure the index is clean as well */\n> +\t\t\tsubmodule_reset_index(path);\n> +\t\t}\n> +\t}\n> +\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n> +\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n> +\n> +\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n> +\t\targv_array_push(&cp.args, \"-n\");\n> +\telse\n> +\t\targv_array_push(&cp.args, \"-u\");\n> +\n> +\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n> +\t\targv_array_push(&cp.args, \"--reset\");\n> +\telse\n> +\t\targv_array_push(&cp.args, \"-m\");\n> +\n> +\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n> +\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n> +\n> +\tif (run_command(&cp)) {\n> +\t\tret = -1;\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n> +\t\tif (new) {\n> +\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n> +\t\t\t/* also set the HEAD accordingly */\n> +\t\t\tcp1.git_cmd = 1;\n> +\t\t\tcp1.no_stdin = 1;\n> +\t\t\tcp1.dir = path;\n> +\n> +\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n> +\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n> +\n> +\t\t\tif (run_command(&cp1)) {\n> +\t\t\t\tret = -1;\n> +\t\t\t\tgoto out;\n> +\t\t\t}\n> +\t\t} else {\n> +\t\t\tstruct strbuf sb = STRBUF_INIT;\n> +\n> +\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n> +\t\t\tunlink_or_warn(sb.buf);\n> +\t\t\tstrbuf_release(&sb);\n> +\n> +\t\t\tif (is_empty_dir(path))\n> +\t\t\t\trmdir_or_warn(path);\n> +\t\t}\n> +\t}\n> +out:\n> +\treturn ret;\n> +}\n> +\n> static int find_first_merges(struct object_array *result, const char *path,\n> \t\tstruct commit *a, struct commit *b)\n> {\n> diff --git a/submodule.h b/submodule.h\n> index 6f3fe85c7c..4cdf6445f7 100644\n> --- a/submodule.h\n> +++ b/submodule.h\n> @@ -96,6 +96,13 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n> extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n> extern int parallel_submodules(void);\n> \n> +#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n> +#define SUBMODULE_MOVE_HEAD_FORCE (1<<1)\n> +extern int submodule_move_head(const char *path,\n> +\t\t\t const char *old,\n> +\t\t\t const char *new,\n> +\t\t\t unsigned flags);\n> +\n> /*\n> * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n> * a submodule by clearing any repo-specific envirionment variables, but\n> -- \n> 2.12.0.rc1.45.g207f5fbb2b\n> \n\n-- \nBrandon Williams\n"},{"id":"313723","messageId":"20170309234317.GF52558@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-13-sbeller@google.com","subject":"Re: [PATCH 12/17] update submodules: add submodule_move_head","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-03-09T23:43:17Z","receivedAt":"2017-03-09T23:43:25Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 03/09, Stefan Beller wrote:\n> +/**\n> + * Moves a submodule at a given path from a given head to another new head.\n> + * For edge cases (a submodule coming into existence or removing a submodule)\n> + * pass NULL for old or new respectively.\n> + */\n> +int submodule_move_head(const char *path,\n> +\t\t\t const char *old,\n> +\t\t\t const char *new,\n> +\t\t\t unsigned flags)\n> +{\n> +\tint ret = 0;\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tconst struct submodule *sub;\n> +\n> +\tsub = submodule_from_path(null_sha1, path);\n> +\n> +\tif (!sub)\n> +\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n> +\n> +\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n> +\t\t/* Check if the submodule has a dirty index. */\n> +\t\tif (submodule_has_dirty_index(sub))\n> +\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n> +\t}\n> +\n> +\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n> +\t\tif (old) {\n> +\t\t\tif (!submodule_uses_gitfile(path))\n> +\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n> +\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n> +\t\t} else {\n> +\t\t\tstruct strbuf sb = STRBUF_INIT;\n> +\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n> +\t\t\t\t get_git_common_dir(), sub->name);\n> +\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n> +\t\t\tstrbuf_release(&sb);\n> +\n> +\t\t\t/* make sure the index is clean as well */\n> +\t\t\tsubmodule_reset_index(path);\n> +\t\t}\n> +\t}\n> +\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n\nMissed this one too. Same question as the other spot.\n\n\n-- \nBrandon Williams\n"},{"id":"313724","messageId":"20170309234040.GE52558@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-13-sbeller@google.com","subject":"Re: [PATCH 12/17] update submodules: add submodule_move_head","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-03-09T23:40:40Z","receivedAt":"2017-03-09T23:48:26Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 03/09, Stefan Beller wrote:\n> +static void submodule_reset_index(const char *path)\n> +{\n> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> +\n> +\tcp.git_cmd = 1;\n> +\tcp.no_stdin = 1;\n> +\tcp.dir = path;\n> +\n> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n\nWhat happens with nested submodules? As in can we reach this code path\nwith super_prefix already being set? If so we would need to include\nthat as part of the super_prefix being passed to the child. Just\nlooking for some clarification.\n\n> +\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n> +\n> +\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n> +\n> +\tif (run_command(&cp))\n> +\t\tdie(\"could not reset submodule index\");\n> +}\n> +\n\n-- \nBrandon Williams\n"},{"id":"313843","messageId":"xmqqshmjug7l.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170309232940.GC52558@google.com","subject":"Re: [PATCH 07/17] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-12T07:06:22Z","receivedAt":"2017-03-12T07:06:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n>> +\tgit_dir = real_pathdup(git_dir_);\n>> +\twork_tree = real_pathdup(work_tree_);\n>\n> Just a note that this is a spot that'll be affected by the change to\n> real_pathdup() which adds a 'die_on_error' parameter to correct bad\n> behaviour I introduced.\n\nThanks for a reminder. I _think_ I resolved this correctly when I\nreplaced the previous one with this round to re-queue on 'pu'.\n"},{"id":"313844","messageId":"xmqqo9x7ug2j.fsf@gitster.mtv.corp.google.com","threadId":"45260","inReplyTo":"20170309233709.GD52558@google.com","subject":"Re: [PATCH 12/17] update submodules: add submodule_move_head","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-12T07:09:24Z","receivedAt":"2017-03-12T07:09:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n>> diff --git a/submodule.c b/submodule.c\n>> index 0b2596e88a..bc5fecf8c5 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n>> \treturn ret;\n>> }\n>> \n>> +static int submodule_has_dirty_index(const struct submodule *sub)\n>> +{\n>> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n>> +\n>> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n>> +\n>> +\tcp.git_cmd = 1;\n>> +\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n>> +\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n>\n> The formatting of this line is a little odd. Also you can drop the\n> backslash.\n\nHeh. I think I saw and pointed out the same during the review of\nthe previous round X-<. It is a bit disappointing.\n\n>> +static void submodule_reset_index(const char *path)\n>> +{\n>> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n>> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n>> +\n>> +\tcp.git_cmd = 1;\n>> +\tcp.no_stdin = 1;\n>> +\tcp.dir = path;\n>> +\n>> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n>> +\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n>> +\n>> +\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n\nSomewhat related; will this use of --super-prefix be affected when\nwe split it into two for \"adjust pathspec\" prefix and \"adjust\noutput\" prefix?\n"},{"id":"313905","messageId":"20170313172421.GA66601@google.com","threadId":"45260","inReplyTo":"xmqqo9x7ug2j.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 12/17] update submodules: add submodule_move_head","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-03-13T17:24:21Z","receivedAt":"2017-03-13T17:24:29Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 03/11, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> >> diff --git a/submodule.c b/submodule.c\n> >> index 0b2596e88a..bc5fecf8c5 100644\n> >> --- a/submodule.c\n> >> +++ b/submodule.c\n> >> @@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n> >> \treturn ret;\n> >> }\n> >> \n> >> +static int submodule_has_dirty_index(const struct submodule *sub)\n> >> +{\n> >> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> >> +\n> >> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> >> +\n> >> +\tcp.git_cmd = 1;\n> >> +\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n> >> +\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n> >\n> > The formatting of this line is a little odd. Also you can drop the\n> > backslash.\n> \n> Heh. I think I saw and pointed out the same during the review of\n> the previous round X-<. It is a bit disappointing.\n> \n> >> +static void submodule_reset_index(const char *path)\n> >> +{\n> >> +\tstruct child_process cp = CHILD_PROCESS_INIT;\n> >> +\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n> >> +\n> >> +\tcp.git_cmd = 1;\n> >> +\tcp.no_stdin = 1;\n> >> +\tcp.dir = path;\n> >> +\n> >> +\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n> >> +\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n> >> +\n> >> +\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n> \n> Somewhat related; will this use of --super-prefix be affected when\n> we split it into two for \"adjust pathspec\" prefix and \"adjust\n> output\" prefix?\n\nMaybe? From what we discussed we can have 'super_prefix' maintain its\ncurrent meaning (Path from root of parent down to the submodule) while\nadding a new env var or option to pass the 'prefix' variable to the\nchild process (Path from root of parent to the directory the original\ncmd was executed in).\n\nSo this shouldn't be affected unless we decide to change the semantics\nof 'super_prefix'.\n\nAnd I'm always torn on how best to do naming for the newer\nfunctionality. I think the easiest thing to do would be to have a\n--prefix option to pass the 'prefix' from one process to the next, but\nI'm sure there are other opinions on the matter.\n\n-- \nBrandon Williams\n"},{"id":"313966","messageId":"CAGZ79kYGdafOc4W6_aUwvCDs5aqXKf5fwa8msQ2gq4swND69ng@mail.gmail.com","threadId":"45260","inReplyTo":"xmqqo9x7ug2j.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 12/17] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-13T20:50:24Z","receivedAt":"2017-03-13T20:50:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Sat, Mar 11, 2017 at 11:09 PM, Junio C Hamano wrote:\n> Brandon Williams writes:\n>\n>>> diff --git a/submodule.c b/submodule.c\n>>> index 0b2596e88a..bc5fecf8c5 100644\n>>> --- a/submodule.c\n>>> +++ b/submodule.c\n>>> @@ -1239,6 +1239,141 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n>>> return ret;\n>>> }\n>>>\n>>> +static int submodule_has_dirty_index(const struct submodule *sub)\n>>> +{\n>>> + struct child_process cp = CHILD_PROCESS_INIT;\n>>> +\n>>> + prepare_submodule_repo_env_no_git_dir(&cp.env_array);\n>>> +\n>>> + cp.git_cmd = 1;\n>>> + argv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n>>> + \"--cached\", \"HEAD\", NULL);\n>>\n>> The formatting of this line is a little odd. Also you can drop the\n>> backslash.\n>\n> Heh. I think I saw and pointed out the same during the review of\n> the previous round X-<. It is a bit disappointing.\n\nYes. I remember having a local mixup of branches, such that I\nfear I may have missed other feedback of the previous round as\nwell then. I do remember removing a backslash, so maybe I\ngot confused with another backslash as well.\n\nI'll reroll with this fixed.\n\n>\n>>> +static void submodule_reset_index(const char *path)\n>>> +{\n>>> + struct child_process cp = CHILD_PROCESS_INIT;\n>>> + prepare_submodule_repo_env_no_git_dir(&cp.env_array);\n>>> +\n>>> + cp.git_cmd = 1;\n>>> + cp.no_stdin = 1;\n>>> + cp.dir = path;\n>>> +\n>>> + argv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n>>> + argv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n>>> +\n>>> + argv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n>\n> Somewhat related; will this use of --super-prefix be affected when\n> we split it into two for \"adjust pathspec\" prefix and \"adjust\n> output\" prefix?\n\nLet's see how the superproject prefixing evolves and what Brandon\nsends out. When the superprefix changes its meaning I'll adjust here.\n"},{"id":"314065","messageId":"20170314214642.7701-1-sbeller@google.com","threadId":"45260","inReplyTo":"20170309221543.15897-1-sbeller@google.com","subject":"[PATCHv9 00/19] Checkout aware of Submodules!","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:23Z","receivedAt":"2017-03-14T21:46:56Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"previous work:\nhttps://public-inbox.org/git/20170309221543.15897-1-sbeller@google.com/\n\nv9:\n * inderdiff to current origin/sb/checkout-recurse-submodules below\n * fixed a '\\' that was violating the style guide.\n * reordered the patches, such that\n -> the first two patches are Valerys series and could go on its own as\n a cleanup \n -> added a new patch \"submodule.c: get_super_prefix_or_empty\" to\n have cleaner code.\n * split up the error conditions in the test lib for readability\n\nv1..v8: see description in link to previous work above.\n\nThanks,\nStefan\n\ndiff --git a/submodule.c b/submodule.c\nindex bc5fecf8c5..929fc7bf04 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1239,6 +1239,14 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static const char *get_super_prefix_or_empty()\n+{\n+\tconst char *s = get_super_prefix();\n+\tif (!s)\n+\t\ts = \"\";\n+\treturn s;\n+}\n+\n static int submodule_has_dirty_index(const struct submodule *sub)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n@@ -1246,8 +1254,8 @@ static int submodule_has_dirty_index(const struct submodule *sub)\n \tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n \n \tcp.git_cmd = 1;\n-\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\", \\\n-\t\t\t\t\t\"--cached\", \"HEAD\", NULL);\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\",\n+\t\t\t\t \"--cached\", \"HEAD\", NULL);\n \tcp.no_stdin = 1;\n \tcp.no_stdout = 1;\n \tcp.dir = sub->path;\n@@ -1266,7 +1274,8 @@ static void submodule_reset_index(const char *path)\n \tcp.no_stdin = 1;\n \tcp.dir = path;\n \n-\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\t\t get_super_prefix_or_empty(), path);\n \targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n \n \targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n@@ -1323,7 +1332,8 @@ int submodule_move_head(const char *path,\n \tcp.no_stdin = 1;\n \tcp.dir = path;\n \n-\targv_array_pushf(&cp.args, \"--super-prefix=%s/\", path);\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\tget_super_prefix_or_empty(), path);\n \targv_array_pushl(&cp.args, \"read-tree\", NULL);\n \n \tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n@@ -1573,11 +1583,8 @@ static void relocate_single_git_dir_into_superproject(const char *prefix,\n \t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n \treal_new_git_dir = real_pathdup(new_git_dir);\n \n-\tif (!prefix)\n-\t\tprefix = get_super_prefix();\n-\n \tfprintf(stderr, _(\"Migrating git directory of '%s%s' from\\n'%s' to\\n'%s'\\n\"),\n-\t\tprefix ? prefix : \"\", path,\n+\t\tget_super_prefix_or_empty(), path,\n \t\treal_old_git_dir, real_new_git_dir);\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n@@ -1648,8 +1655,7 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tif (flags & ~ABSORB_GITDIR_RECURSE_SUBMODULES)\n \t\t\tdie(\"BUG: we don't know how to pass the flags down?\");\n \n-\t\tif (get_super_prefix())\n-\t\t\tstrbuf_addstr(&sb, get_super_prefix());\n+\t\tstrbuf_addstr(&sb, get_super_prefix_or_empty());\n \t\tstrbuf_addstr(&sb, path);\n \t\tstrbuf_addch(&sb, '/');\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex e195b590a1..fb4f7b014e 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -782,15 +782,20 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n-\tRESULT=success\n+\tRESULTDS=success\n \tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n \tthen\n-\t\tRESULT=failure\n+\t\tRESULTDS=failure\n+\tfi\n+\tRESULTR=success\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_RECURSIVE_NESTED\" = 1\n+\tthen\n+\t\tRESULTR=failure\n \tfi\n-\tRESULT1=success\n+\tRESULTOI=success\n \tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n \tthen\n-\t\tRESULT1=failure\n+\t\tRESULTOI=failure\n \tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n@@ -832,7 +837,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_$RESULT1 \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULTOI \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\n@@ -901,7 +906,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULTDS \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -913,7 +918,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_$RESULT \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULTDS \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -941,7 +946,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_$RESULT \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULTDS \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -998,7 +1003,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# recursing deeper than one level doesn't work yet.\n-\ttest_expect_failure \"$command: modified submodule updates submodule recursively\" '\n+\ttest_expect_$RESULTR \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n@@ -1116,13 +1121,13 @@ test_submodule_forced_switch_recursing () {\n \t\t)\n \t'\n \t# Replacing a submodule with files in a directory ...\n-\ttest_expect_$RESULT \"$command: replace submodule with a directory\" '\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n \t\t\tcd submodule_update &&\n \t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n-\t\t\ttest_must_fail $command replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n \t\t\ttest_superproject_content origin/replace_sub1_with_directory\n \t\t)\n \t'\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 7019d0a04f..de1ba02dc5 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,7 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_SUBMODULE_RECURSIVE_NESTED=1\n KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n \ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex aa35223369..e8f70b806f 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -64,6 +64,7 @@ test_expect_success '\"checkout \" honors submodule.*.ignore from .git/\n '\n \n KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_RECURSIVE_NESTED=1\n test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n \n test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n"},{"id":"314066","messageId":"20170314214642.7701-2-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 01/19] connect_work_tree_and_git_dir: safely create leading directories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:24Z","receivedAt":"2017-03-14T21:46:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll use connect_work_tree_and_git_dir when the\ndirectory for the gitlink file doesn't exist yet. This patch makes\nconnect_work_tree_and_git_dir safe to use for both cases of\neither the git dir or the working dir missing.\n\nTo do so, we need to call safe_create_leading_directories[_const]\non both directories. However this has to happen before we construct\nthe absolute paths as real_pathdup assumes the directories to\nbe there already.\n\nSo for both the config file in the git dir as well as the .git link\nfile we need to\na) construct the name\nb) call SCLD\nc) get the absolute path\nd) once a-c is done for both we can consume the absolute path\n to compute the relative path to each other and store those\n relative paths.\n\nThe implementation provided here puts a) and b) for both cases first,\nand then performs c and d after.\n\nOne of the two users of 'connect_work_tree_and_git_dir' already checked\nfor the directory being there, so we can loose that check as\nconnect_work_tree_and_git_dir handles this functionality now.\n\nSigned-off-by: Stefan Beller \n---\n dir.c | 32 +++++++++++++++++++++-----------\n submodule.c | 11 ++---------\n 2 files changed, 23 insertions(+), 20 deletions(-)\n\ndiff --git a/dir.c b/dir.c\nindex 4541f9e146..6f52af7abb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -2728,23 +2728,33 @@ void untracked_cache_add_to_index(struct index_state *istate,\n /* Update gitfile and core.worktree setting to connect work tree and git dir */\n void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)\n {\n-\tstruct strbuf file_name = STRBUF_INIT;\n+\tstruct strbuf gitfile_sb = STRBUF_INIT;\n+\tstruct strbuf cfg_sb = STRBUF_INIT;\n \tstruct strbuf rel_path = STRBUF_INIT;\n-\tchar *git_dir = real_pathdup(git_dir_);\n-\tchar *work_tree = real_pathdup(work_tree_);\n+\tchar *git_dir, *work_tree;\n \n-\t/* Update gitfile */\n-\tstrbuf_addf(&file_name, \"%s/.git\", work_tree);\n-\twrite_file(file_name.buf, \"gitdir: %s\",\n-\t\t relative_path(git_dir, work_tree, &rel_path));\n+\t/* Prepare .git file */\n+\tstrbuf_addf(&gitfile_sb, \"%s/.git\", work_tree_);\n+\tif (safe_create_leading_directories_const(gitfile_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), gitfile_sb.buf);\n+\n+\t/* Prepare config file */\n+\tstrbuf_addf(&cfg_sb, \"%s/config\", git_dir_);\n+\tif (safe_create_leading_directories_const(cfg_sb.buf))\n+\t\tdie(_(\"could not create directories for %s\"), cfg_sb.buf);\n \n+\tgit_dir = real_pathdup(git_dir_);\n+\twork_tree = real_pathdup(work_tree_);\n+\n+\t/* Write .git file */\n+\twrite_file(gitfile_sb.buf, \"gitdir: %s\",\n+\t\t relative_path(git_dir, work_tree, &rel_path));\n \t/* Update core.worktree setting */\n-\tstrbuf_reset(&file_name);\n-\tstrbuf_addf(&file_name, \"%s/config\", git_dir);\n-\tgit_config_set_in_file(file_name.buf, \"core.worktree\",\n+\tgit_config_set_in_file(cfg_sb.buf, \"core.worktree\",\n \t\t\t relative_path(work_tree, git_dir, &rel_path));\n \n-\tstrbuf_release(&file_name);\n+\tstrbuf_release(&gitfile_sb);\n+\tstrbuf_release(&cfg_sb);\n \tstrbuf_release(&rel_path);\n \tfree(work_tree);\n \tfree(git_dir);\ndiff --git a/submodule.c b/submodule.c\nindex 3b98766a6b..45e93a1d55 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1445,8 +1445,6 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \n \t/* Not populated? */\n \tif (!sub_git_dir) {\n-\t\tchar *real_new_git_dir;\n-\t\tconst char *new_git_dir;\n \t\tconst struct submodule *sub;\n \n \t\tif (err_code == READ_GITFILE_ERR_STAT_FAILED) {\n@@ -1469,13 +1467,8 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tsub = submodule_from_path(null_sha1, path);\n \t\tif (!sub)\n \t\t\tdie(_(\"could not lookup name for submodule '%s'\"), path);\n-\t\tnew_git_dir = git_path(\"modules/%s\", sub->name);\n-\t\tif (safe_create_leading_directories_const(new_git_dir) < 0)\n-\t\t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n-\t\treal_new_git_dir = real_pathdup(new_git_dir);\n-\t\tconnect_work_tree_and_git_dir(path, real_new_git_dir);\n-\n-\t\tfree(real_new_git_dir);\n+\t\tconnect_work_tree_and_git_dir(path,\n+\t\t\tgit_path(\"modules/%s\", sub->name));\n \t} else {\n \t\t/* Is it already absorbed into the superprojects git dir? */\n \t\tchar *real_sub_git_dir = real_pathdup(sub_git_dir);\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314067","messageId":"20170314214642.7701-3-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 02/19] submodule--helper.c: remove duplicate code","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:25Z","receivedAt":"2017-03-14T21:47:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Valery Tolstov \n\nRemove code fragment from module_clone that duplicates functionality\nof connect_work_tree_and_git_dir in dir.c\n\nSigned-off-by: Valery Tolstov \nReviewed-by: Brandon Williams \nSigned-off-by: Junio C Hamano \nSigned-off-by: Stefan Beller \n---\n builtin/submodule--helper.c | 22 ++--------------------\n 1 file changed, 2 insertions(+), 20 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 899dc334e3..86bafe1660 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -579,9 +579,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \tconst char *name = NULL, *url = NULL, *depth = NULL;\n \tint quiet = 0;\n \tint progress = 0;\n-\tFILE *submodule_dot_git;\n \tchar *p, *path = NULL, *sm_gitdir;\n-\tstruct strbuf rel_path = STRBUF_INIT;\n \tstruct strbuf sb = STRBUF_INIT;\n \tstruct string_list reference = STRING_LIST_INIT_NODUP;\n \tchar *sm_alternate = NULL, *error_strategy = NULL;\n@@ -653,27 +651,12 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \t\tstrbuf_reset(&sb);\n \t}\n \n-\t/* Write a .git file in the submodule to redirect to the superproject. */\n-\tstrbuf_addf(&sb, \"%s/.git\", path);\n-\tif (safe_create_leading_directories_const(sb.buf) < 0)\n-\t\tdie(_(\"could not create leading directories of '%s'\"), sb.buf);\n-\tsubmodule_dot_git = fopen(sb.buf, \"w\");\n-\tif (!submodule_dot_git)\n-\t\tdie_errno(_(\"cannot open file '%s'\"), sb.buf);\n-\n-\tfprintf_or_die(submodule_dot_git, \"gitdir: %s\\n\",\n-\t\t relative_path(sm_gitdir, path, &rel_path));\n-\tif (fclose(submodule_dot_git))\n-\t\tdie(_(\"could not close file %s\"), sb.buf);\n-\tstrbuf_reset(&sb);\n-\tstrbuf_reset(&rel_path);\n+\t/* Connect module worktree and git dir */\n+\tconnect_work_tree_and_git_dir(path, sm_gitdir);\n \n-\t/* Redirect the worktree of the submodule in the superproject's config */\n \tp = git_pathdup_submodule(path, \"config\");\n \tif (!p)\n \t\tdie(_(\"could not get submodule directory for '%s'\"), path);\n-\tgit_config_set_in_file(p, \"core.worktree\",\n-\t\t\t relative_path(path, sm_gitdir, &rel_path));\n \n \t/* setup alternateLocation and alternateErrorStrategy in the cloned submodule if needed */\n \tgit_config_get_string(\"submodule.alternateLocation\", &sm_alternate);\n@@ -689,7 +672,6 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \tfree(error_strategy);\n \n \tstrbuf_release(&sb);\n-\tstrbuf_release(&rel_path);\n \tfree(sm_gitdir);\n \tfree(path);\n \tfree(p);\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314068","messageId":"20170314214642.7701-5-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 04/19] lib-submodule-update.sh: do not use ./. as submodule remote","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:27Z","receivedAt":"2017-03-14T21:47:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Adding the repository itself as a submodule does not make sense in the\nreal world. In our test suite we used to do that out of convenience in\nsome tests as the current repository has easiest access for setting up\n'just a submodule'.\n\nHowever this doesn't quite test the real world, so let's do not follow\nthis pattern any further and actually create an independent repository\nthat we can use as a submodule.\n\nWhen using './.' as the remote the superproject and submodule share the\nsame objects, such that testing if a given sha1 is a valid commit works\nin either repository. As running commands in an unpopulated submodule\nfall back to the superproject, this happens in `reset_work_tree_to`\nto determine if we need to populate the submodule. Fix this bug by\nchecking in the actual remote now.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 15 +++++++++++++--\n 1 file changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 5df528ea81..c0d6325133 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -37,6 +37,17 @@\n #\n \n create_lib_submodule_repo () {\n+\tgit init submodule_update_sub1 &&\n+\t(\n+\t\tcd submodule_update_sub1 &&\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"Base inside first submodule\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -49,7 +60,7 @@ create_lib_submodule_repo () {\n \t\tgit branch \"no_submodule\" &&\n \n \t\tgit checkout -b \"add_sub1\" &&\n-\t\tgit submodule add ./. sub1 &&\n+\t\tgit submodule add ../submodule_update_sub1 sub1 &&\n \t\tgit config -f .gitmodules submodule.sub1.ignore all &&\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n@@ -162,7 +173,7 @@ reset_work_tree_to () {\n \t\ttest_must_be_empty actual &&\n \t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n \t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314069","messageId":"20170314214642.7701-8-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 07/19] lib-submodule-update.sh: define tests for recursing into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:30Z","receivedAt":"2017-03-14T21:47:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Currently lib-submodule-update.sh provides 2 functions\ntest_submodule_switch and test_submodule_forced_switch that are used by a\nvariety of tests to ensure that submodules behave as expected. The current\nexpected behavior is that submodules are not touched at all (see\n42639d2317a for the exact setup).\n\nIn the future we want to teach all these commands to recurse\ninto submodules. To do that, we'll add two testing functions to\nsubmodule-update-lib.sh: test_submodule_switch_recursing and\ntest_submodule_forced_switch_recursing.\n\nThese two functions behave in analogy to the already existing functions\njust with a different expectation on submodule behavior. The submodule\nin the working tree is expected to be updated to the recorded submodule\nversion. The behavior is analogous to e.g. the behavior of files in a\nnested directory in the working tree, where a change to the working tree\nhandles any arising directory/file conflicts just fine.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 506 +++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 504 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex f52c49c838..ae560dc95c 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -4,6 +4,7 @@\n # - New submodule (no_submodule => add_sub1)\n # - Removed submodule (add_sub1 => remove_sub1)\n # - Updated submodule (add_sub1 => modify_sub1)\n+# - Updated submodule recursively (add_nested_sub => modify_sub1_recursively)\n # - Submodule updated to invalid commit (add_sub1 => invalid_sub1)\n # - Submodule updated from invalid commit (invalid_sub1 => valid_sub1)\n # - Submodule replaced by tracked files in directory (add_sub1 =>\n@@ -19,8 +20,8 @@\n # / ^\n # / remove_sub1\n # /\n-# add_sub1 /-------O\n-# | / ^\n+# add_sub1 /-------O---------O--------O modify_sub1_recursively\n+# | / ^ add_nested_sub\n # | / modify_sub1\n # v/\n # O------O-----------O---------O\n@@ -48,6 +49,17 @@ create_lib_submodule_repo () {\n \t\tgit commit -m \"Base inside first submodule\" &&\n \t\tgit branch \"no_submodule\"\n \t) &&\n+\tgit init submodule_update_sub2 &&\n+\t(\n+\t\tcd submodule_update_sub2\n+\t\techo \"expect\" >>.gitignore &&\n+\t\techo \"actual\" >>.gitignore &&\n+\t\techo \"x\" >file1 &&\n+\t\techo \"y\" >file2 &&\n+\t\tgit add .gitignore file1 file2 &&\n+\t\tgit commit -m \"nested submodule base\" &&\n+\t\tgit branch \"no_submodule\"\n+\t) &&\n \tgit init submodule_update_repo &&\n \t(\n \t\tcd submodule_update_repo &&\n@@ -84,6 +96,26 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n+\t\tgit checkout -b add_nested_sub modify_sub1 &&\n+\t\tgit -C sub1 checkout -b \"add_nested_sub\" &&\n+\t\tgit -C sub1 submodule add --branch no_submodule ../submodule_update_sub2 sub2 &&\n+\t\tgit -C sub1 commit -a -m \"add a nested submodule\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -a -m \"update submodule, that updates a nested submodule\" &&\n+\t\tgit checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1 checkout -b modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 checkout -b modify_sub1_recursively &&\n+\t\techo change >sub1/sub2/file3 &&\n+\t\tgit -C sub1/sub2 add file3 &&\n+\t\tgit -C sub1/sub2 commit -m \"make a change in nested sub\" &&\n+\t\tgit -C sub1 add sub2 &&\n+\t\tgit -C sub1 commit -m \"update nested sub\" &&\n+\t\tgit add sub1 &&\n+\t\tgit commit -m \"update sub1, that updates nested sub\" &&\n+\t\tgit -C sub1 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1/sub2 push origin modify_sub1_recursively &&\n+\t\tgit -C sub1 submodule deinit -f --all &&\n+\n \t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n@@ -150,6 +182,15 @@ test_git_directory_is_unchanged () {\n \t)\n }\n \n+test_git_directory_exists() {\n+\ttest -e \".git/modules/$1\" &&\n+\tif test -f sub1/.git\n+\tthen\n+\t\t# does core.worktree point at the right place?\n+\t\ttest \"$(git -C .git/modules/$1 config core.worktree)\" = \"../../../$1\"\n+\tfi\n+}\n+\n # Helper function to be executed at the start of every test below, it sets up\n # the submodule repo if it doesn't exist and configures the most problematic\n # settings for diff.ignoreSubmodules.\n@@ -180,6 +221,27 @@ reset_work_tree_to () {\n \t)\n }\n \n+reset_work_tree_to_interested () {\n+\treset_work_tree_to $1 &&\n+\t# make the submodule git dirs available\n+\tif ! test -d submodule_update/.git/modules/sub1\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1 submodule_update/.git/modules/sub1\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1 config --unset core.worktree\n+\tfi &&\n+\tif ! test -d submodule_update/.git/modules/sub1/modules/sub2\n+\tthen\n+\t\tmkdir -p submodule_update/.git/modules/sub1/modules &&\n+\t\tcp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2\n+\t\tGIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree\n+\tfi &&\n+\t# indicate we are interested in the submodule:\n+\tgit -C submodule_update config submodule.sub1.url \"bogus\" &&\n+\t# sub1 might not be checked out, so use the git dir\n+\tgit -C submodule_update/.git/modules/sub1 config submodule.sub2.url \"bogus\"\n+}\n+\n # Test that the superproject contains the content according to commit \"$1\"\n # (the work tree must match the index for everything but submodules but the\n # index must exactly match the given commit including any submodule SHA-1s).\n@@ -700,3 +762,443 @@ test_submodule_forced_switch () {\n \t\t)\n \t'\n }\n+\n+# Test that submodule contents are correctly updated when switching\n+# between commits that change a submodule.\n+# Test that the following transitions are correctly handled:\n+# (These tests are also above in the case where we expect no change\n+# in the submodule)\n+# - Updated submodule\n+# - New submodule\n+# - Removed submodule\n+# - Directory containing tracked files replaced by submodule\n+# - Submodule replaced by tracked files in directory\n+# - Submodule replaced by tracked file with the same name\n+# - tracked file replaced by submodule\n+#\n+# New test cases\n+# - Removing a submodule with a git directory absorbs the submodules\n+# git directory first into the superproject.\n+\n+test_submodule_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear checks it out ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... ignoring an empty existing directory ...\n+\ttest_expect_success \"$command: added submodule is checked out in empty dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tmkdir sub1 &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... unless there is an untracked file in its place.\n+\ttest_expect_success \"$command: added submodule doesn't remove untracked file with same name\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\ttest_must_fail $command add_sub1 &&\n+\t\t\ttest_superproject_content origin/no_submodule &&\n+\t\t\ttest_must_be_empty sub1\n+\t\t)\n+\t'\n+\t# ... but an ignored file is fine.\n+\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t: >sub1 &&\n+\t\t\techo sub1 >.git/info/exclude\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule produces a checked out submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule checks out submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule removes its work tree ...\n+\ttest_expect_success \"$command: removed submodule removes submodules working tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory along the way.\n+\ttest_expect_success \"$command: removed submodule absorbs submodules .git directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1 &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory must succeeds\n+\t# when the submodule is clean\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\n+\t# Replacing it with a file ...\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t# ... must check its local work tree for untracked files\n+\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untrackedfile &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\t# ... and ignored files are ignored\n+\ttest_expect_success \"$command: replace submodule with a file works ignores ignored files in submodule\" '\n+\t\ttest_when_finished \"rm submodule_update/.git/modules/sub1/info/exclude\" &&\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/ignored &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file &&\n+\t\t\ttest -f sub1\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# superproject nor the submodule's work tree.\n+\ttest_expect_success \"$command: updating to a missing submodule commit fails\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\n+\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_nested_sub &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1_recursively origin/modify_sub1_recursively &&\n+\t\t\t$command modify_sub1_recursively &&\n+\t\t\ttest_superproject_content origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1_recursively &&\n+\t\t\ttest_submodule_content -C sub1 sub2 origin/modify_sub1_recursively\n+\t\t)\n+\t'\n+}\n+\n+# Test that submodule contents are updated when switching between commits\n+# that change a submodule, but throwing away local changes in\n+# the superproject as well as the submodule is allowed.\n+test_submodule_forced_switch_recursing () {\n+\tcommand=\"$1\"\n+\t######################### Appearing submodule #########################\n+\t# Switching to a commit letting a submodule appear creates empty dir ...\n+\ttest_expect_success \"$command: added submodule is checked out\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... and doesn't care if it already exists ...\n+\ttest_expect_success \"$command: added submodule ignores empty directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\tmkdir sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# ... not caring about an untracked file either\n+\ttest_expect_success \"$command: added submodule does remove untracked unignored file with same name when forced\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested no_submodule &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t add_sub1 origin/add_sub1 &&\n+\t\t\t>sub1 &&\n+\t\t\t$command add_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Replacing a tracked file with a submodule checks out the submodule\n+\ttest_expect_success \"$command: replace tracked file with submodule populates the submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_file &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_file_with_sub1 origin/replace_file_with_sub1 &&\n+\t\t\t$command replace_file_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_file_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_file_with_sub1\n+\t\t)\n+\t'\n+\t# ... as does removing a directory with tracked files with a\n+\t# submodule.\n+\ttest_expect_success \"$command: replace directory with submodule\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested replace_sub1_with_directory &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_directory_with_sub1 origin/replace_directory_with_sub1 &&\n+\t\t\t$command replace_directory_with_sub1 &&\n+\t\t\ttest_superproject_content origin/replace_directory_with_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/replace_directory_with_sub1\n+\t\t)\n+\t'\n+\n+\t######################## Disappearing submodule #######################\n+\t# Removing a submodule doesn't remove its work tree ...\n+\ttest_expect_success \"$command: removed submodule leaves submodule directory and its contents in place\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# ... especially when it contains a .git directory.\n+\ttest_expect_success \"$command: removed submodule leaves submodule containing a .git directory alone\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t remove_sub1 origin/remove_sub1 &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command remove_sub1 &&\n+\t\t\ttest_superproject_content origin/remove_sub1 &&\n+\t\t\ttest_git_directory_exists sub1 &&\n+\t\t\t! test -e sub1\n+\t\t)\n+\t'\n+\t# Replacing a submodule with files in a directory ...\n+\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory\n+\t\t)\n+\t'\n+\t# ... absorbing a .git directory.\n+\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must fail\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_directory origin/replace_sub1_with_directory &&\n+\t\t\treplace_gitfile_with_git_dir sub1 &&\n+\t\t\trm -rf .git/modules/sub1 &&\n+\t\t\t$command replace_sub1_with_directory &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_directory &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t\ttest_git_directory_exists sub1\n+\t\t)\n+\t'\n+\t# Replacing it with a file\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... even if the submodule contains ignored files\n+\ttest_expect_success \"$command: replace submodule with a file ignoring ignored files\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/expect &&\n+\t\t\t$command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/replace_sub1_with_file\n+\t\t)\n+\t'\n+\n+\t# ... but stops for untracked files that would be lost\n+\ttest_expect_success \"$command: replace submodule with a file\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t replace_sub1_with_file origin/replace_sub1_with_file &&\n+\t\t\t: >sub1/untracked_file &&\n+\t\t\ttest_must_fail $command replace_sub1_with_file &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest -f sub1/untracked_file\n+\t\t)\n+\t'\n+\n+\t########################## Modified submodule #########################\n+\t# Updating a submodule sha1 updates the submodule's work tree\n+\ttest_expect_success \"$command: modified submodule updates submodule work tree\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t modify_sub1 origin/modify_sub1 &&\n+\t\t\t$command modify_sub1 &&\n+\t\t\ttest_superproject_content origin/modify_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/modify_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule to an invalid sha1 doesn't update the\n+\t# submodule's work tree, subsequent update will fail\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree to invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested add_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t invalid_sub1 origin/invalid_sub1 &&\n+\t\t\ttest_must_fail $command invalid_sub1 &&\n+\t\t\ttest_superproject_content origin/add_sub1 &&\n+\t\t\ttest_submodule_content sub1 origin/add_sub1\n+\t\t)\n+\t'\n+\t# Updating a submodule from an invalid sha1 updates\n+\ttest_expect_success \"$command: modified submodule does not update submodule work tree from invalid commit\" '\n+\t\tprolog &&\n+\t\treset_work_tree_to_interested invalid_sub1 &&\n+\t\t(\n+\t\t\tcd submodule_update &&\n+\t\t\tgit branch -t valid_sub1 origin/valid_sub1 &&\n+\t\t\ttest_must_fail $command valid_sub1 &&\n+\t\t\ttest_superproject_content origin/invalid_sub1\n+\t\t)\n+\t'\n+}\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314070","messageId":"20170314214642.7701-6-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 05/19] lib-submodule-update: teach test_submodule_content the -C flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:28Z","receivedAt":"2017-03-14T21:47:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex c0d6325133..00128f28b5 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -193,6 +193,11 @@ test_superproject_content () {\n # Test that the given submodule at path \"$1\" contains the content according\n # to the submodule commit recorded in the superproject's commit \"$2\"\n test_submodule_content () {\n+\tif test x\"$1\" = \"x-C\"\n+\tthen\n+\t\tcd \"$2\"\n+\t\tshift; shift;\n+\tfi\n \tif test $# != 2\n \tthen\n \t\techo \"test_submodule_content needs two arguments\"\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314071","messageId":"20170314214642.7701-12-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 11/19] submodules: introduce check to see whether to touch a submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:34Z","receivedAt":"2017-03-14T21:47:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the --recurse-submodule flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nIt is potentially expensive to check if a submodule needs an update,\nbecause a common theme to interact with submodules is to spawn a child\nprocess for each interaction.\n\nSo let's introduce a function that checks if a submodule needs\nto be checked for an update before attempting the update.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 16 ++++++++++++++++\n submodule.h | 7 +++++++\n 2 files changed, 23 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 591f4a694e..8b2c0212be 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -548,6 +548,22 @@ void set_config_update_recurse_submodules(int value)\n \tconfig_update_recurse_submodules = value;\n }\n \n+int should_update_submodules(void)\n+{\n+\treturn config_update_recurse_submodules == RECURSE_SUBMODULES_ON;\n+}\n+\n+const struct submodule *submodule_from_ce(const struct cache_entry *ce)\n+{\n+\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\treturn NULL;\n+\n+\tif (!should_update_submodules())\n+\t\treturn NULL;\n+\n+\treturn submodule_from_path(null_sha1, ce->name);\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex b4e60c08d2..6f3fe85c7c 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -65,6 +65,13 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n extern void set_config_update_recurse_submodules(int value);\n+/* Check if we want to update any submodule.*/\n+extern int should_update_submodules(void);\n+/*\n+ * Returns the submodule struct if the given ce entry is a submodule\n+ * and it should be updated. Returns NULL otherwise.\n+ */\n+extern const struct submodule *submodule_from_ce(const struct cache_entry *ce);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314072","messageId":"20170314214642.7701-18-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 17/19] entry.c: create submodules when interesting","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:40Z","receivedAt":"2017-03-14T21:47:23Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When a submodule is introduced with a new revision\nwe need to create the submodule in the worktree as well.\nAs 'submodule_move_head' handles edge cases, all we have\nto do is call it from within the function that creates\nnew files in the working tree for workingtree operations.\n\nSigned-off-by: Stefan Beller \n---\n entry.c | 30 ++++++++++++++++++++++++++++++\n 1 file changed, 30 insertions(+)\n\ndiff --git a/entry.c b/entry.c\nindex c6eea240b6..d2b512da90 100644\n--- a/entry.c\n+++ b/entry.c\n@@ -2,6 +2,7 @@\n #include \"blob.h\"\n #include \"dir.h\"\n #include \"streaming.h\"\n+#include \"submodule.h\"\n \n static void create_directories(const char *path, int path_len,\n \t\t\t const struct checkout *state)\n@@ -146,6 +147,7 @@ static int write_entry(struct cache_entry *ce,\n \tunsigned long size;\n \tsize_t wrote, newsize = 0;\n \tstruct stat st;\n+\tconst struct submodule *sub;\n \n \tif (ce_mode_s_ifmt == S_IFREG) {\n \t\tstruct stream_filter *filter = get_stream_filter(ce->name,\n@@ -203,6 +205,10 @@ static int write_entry(struct cache_entry *ce,\n \t\t\treturn error(\"cannot create temporary submodule %s\", path);\n \t\tif (mkdir(path, 0777) < 0)\n \t\t\treturn error(\"cannot create submodule directory %s\", path);\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub)\n+\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\tNULL, oid_to_hex(&ce->oid), SUBMODULE_MOVE_HEAD_FORCE);\n \t\tbreak;\n \tdefault:\n \t\treturn error(\"unknown file mode for %s in index\", path);\n@@ -259,7 +265,31 @@ int checkout_entry(struct cache_entry *ce,\n \tstrbuf_add(&path, ce->name, ce_namelen(ce));\n \n \tif (!check_path(path.buf, path.len, &st, state->base_dir_len)) {\n+\t\tconst struct submodule *sub;\n \t\tunsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);\n+\t\t/*\n+\t\t * Needs to be checked before !changed returns early,\n+\t\t * as the possibly empty directory was not changed\n+\t\t */\n+\t\tsub = submodule_from_ce(ce);\n+\t\tif (sub) {\n+\t\t\tint err;\n+\t\t\tif (!is_submodule_populated_gently(ce->name, &err)) {\n+\t\t\t\tstruct stat sb;\n+\t\t\t\tif (lstat(ce->name, &sb))\n+\t\t\t\t\tdie(_(\"could not stat file '%s'\"), ce->name);\n+\t\t\t\tif (!(st.st_mode & S_IFDIR))\n+\t\t\t\t\tunlink_or_warn(ce->name);\n+\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\tNULL, oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\t} else\n+\t\t\t\treturn submodule_move_head(ce->name,\n+\t\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid),\n+\t\t\t\t\tSUBMODULE_MOVE_HEAD_FORCE);\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\tif (!state->force) {\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314073","messageId":"20170314214642.7701-15-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 14/19] update submodules: add submodule_move_head","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:37Z","receivedAt":"2017-03-14T21:47:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nThis piece of code will be used universally for\nall these working tree modifications as it\n* supports dry run to answer the question:\n \"Is it safe to change the submodule to this new state?\"\n e.g. is it overwriting untracked files or are there local\n changes that would be overwritten?\n* supports a force flag that can be used for resetting\n the tree.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 137 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n submodule.h | 7 ++++\n 2 files changed, 144 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 3501497a78..929fc7bf04 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1247,6 +1247,143 @@ static const char *get_super_prefix_or_empty()\n \treturn s;\n }\n \n+static int submodule_has_dirty_index(const struct submodule *sub)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\targv_array_pushl(&cp.args, \"diff-index\", \"--quiet\",\n+\t\t\t\t \"--cached\", \"HEAD\", NULL);\n+\tcp.no_stdin = 1;\n+\tcp.no_stdout = 1;\n+\tcp.dir = sub->path;\n+\tif (start_command(&cp))\n+\t\tdie(\"could not recurse into submodule '%s'\", sub->path);\n+\n+\treturn finish_command(&cp);\n+}\n+\n+static void submodule_reset_index(const char *path)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\t\t get_super_prefix_or_empty(), path);\n+\targv_array_pushl(&cp.args, \"read-tree\", \"-u\", \"--reset\", NULL);\n+\n+\targv_array_push(&cp.args, EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp))\n+\t\tdie(\"could not reset submodule index\");\n+}\n+\n+/**\n+ * Moves a submodule at a given path from a given head to another new head.\n+ * For edge cases (a submodule coming into existence or removing a submodule)\n+ * pass NULL for old or new respectively.\n+ */\n+int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags)\n+{\n+\tint ret = 0;\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tconst struct submodule *sub;\n+\n+\tsub = submodule_from_path(null_sha1, path);\n+\n+\tif (!sub)\n+\t\tdie(\"BUG: could not get submodule information for '%s'\", path);\n+\n+\tif (old && !(flags & SUBMODULE_MOVE_HEAD_FORCE)) {\n+\t\t/* Check if the submodule has a dirty index. */\n+\t\tif (submodule_has_dirty_index(sub))\n+\t\t\treturn error(_(\"submodule '%s' has dirty index\"), path);\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (old) {\n+\t\t\tif (!submodule_uses_gitfile(path))\n+\t\t\t\tabsorb_git_dir_into_superproject(\"\", path,\n+\t\t\t\t\tABSORB_GITDIR_RECURSE_SUBMODULES);\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\t\tstrbuf_addf(&sb, \"%s/modules/%s\",\n+\t\t\t\t get_git_common_dir(), sub->name);\n+\t\t\tconnect_work_tree_and_git_dir(path, sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\t/* make sure the index is clean as well */\n+\t\t\tsubmodule_reset_index(path);\n+\t\t}\n+\t}\n+\n+\tprepare_submodule_repo_env_no_git_dir(&cp.env_array);\n+\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.dir = path;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\tget_super_prefix_or_empty(), path);\n+\targv_array_pushl(&cp.args, \"read-tree\", NULL);\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_DRY_RUN)\n+\t\targv_array_push(&cp.args, \"-n\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-u\");\n+\n+\tif (flags & SUBMODULE_MOVE_HEAD_FORCE)\n+\t\targv_array_push(&cp.args, \"--reset\");\n+\telse\n+\t\targv_array_push(&cp.args, \"-m\");\n+\n+\targv_array_push(&cp.args, old ? old : EMPTY_TREE_SHA1_HEX);\n+\targv_array_push(&cp.args, new ? new : EMPTY_TREE_SHA1_HEX);\n+\n+\tif (run_command(&cp)) {\n+\t\tret = -1;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) {\n+\t\tif (new) {\n+\t\t\tstruct child_process cp1 = CHILD_PROCESS_INIT;\n+\t\t\t/* also set the HEAD accordingly */\n+\t\t\tcp1.git_cmd = 1;\n+\t\t\tcp1.no_stdin = 1;\n+\t\t\tcp1.dir = path;\n+\n+\t\t\targv_array_pushl(&cp1.args, \"update-ref\", \"HEAD\",\n+\t\t\t\t\t new ? new : EMPTY_TREE_SHA1_HEX, NULL);\n+\n+\t\t\tif (run_command(&cp1)) {\n+\t\t\t\tret = -1;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct strbuf sb = STRBUF_INIT;\n+\n+\t\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\t\t\tunlink_or_warn(sb.buf);\n+\t\t\tstrbuf_release(&sb);\n+\n+\t\t\tif (is_empty_dir(path))\n+\t\t\t\trmdir_or_warn(path);\n+\t\t}\n+\t}\n+out:\n+\treturn ret;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 6f3fe85c7c..4cdf6445f7 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -96,6 +96,13 @@ extern int push_unpushed_submodules(struct sha1_array *commits,\n extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);\n extern int parallel_submodules(void);\n \n+#define SUBMODULE_MOVE_HEAD_DRY_RUN (1<<0)\n+#define SUBMODULE_MOVE_HEAD_FORCE (1<<1)\n+extern int submodule_move_head(const char *path,\n+\t\t\t const char *old,\n+\t\t\t const char *new,\n+\t\t\t unsigned flags);\n+\n /*\n * Prepare the \"env_array\" parameter of a \"struct child_process\" for executing\n * a submodule by clearing any repo-specific envirionment variables, but\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314074","messageId":"20170314214642.7701-19-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 18/19] builtin/checkout: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:41Z","receivedAt":"2017-03-14T21:47:26Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This exposes a flag to recurse into submodules\nin builtin/checkout making use of the code implemented\nin prior patches.\n\nA new failure mode is introduced in the submodule\nupdate library, as the directory/submodule conflict\nis not solved in prior patches.\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-checkout.txt | 7 +++++++\n builtin/checkout.c | 28 ++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 26 +++++++++++++++++++++-----\n t/t2013-checkout-submodule.sh | 6 ++++++\n 4 files changed, 62 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt\nindex 8e2c0662dd..d6399c0af8 100644\n--- a/Documentation/git-checkout.txt\n+++ b/Documentation/git-checkout.txt\n@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.\n \tout anyway. In other words, the ref can be held by more than one\n \tworktree.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject. If\n+\tlocal modifications in a submodule would be overwritten the checkout\n+\twill fail unless `-f` is used. If nothing (or --no-recurse-submodules)\n+\tis used, the work trees of submodules will not be updated.\n+\n ::\n \tBranch to checkout; if it refers to a branch (i.e., a name that,\n \twhen prepended with \"refs/heads/\", is a valid ref), then that\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex f174f50303..e9c5fcfaf8 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -21,12 +21,31 @@\n #include \"submodule-config.h\"\n #include \"submodule.h\"\n \n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n+\n static const char * const checkout_usage[] = {\n \tN_(\"git checkout [] \"),\n \tN_(\"git checkout [] [] -- ...\"),\n \tNULL,\n };\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n struct checkout_opts {\n \tint patch_mode;\n \tint quiet;\n@@ -1163,6 +1182,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\t\t\tN_(\"second guess 'git checkout '\")),\n \t\tOPT_BOOL(0, \"ignore-other-worktrees\", &opts.ignore_other_worktrees,\n \t\t\t N_(\"do not check if another worktree is holding the given ref\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_BOOL(0, \"progress\", &opts.show_progress, N_(\"force progress reporting\")),\n \t\tOPT_END(),\n \t};\n@@ -1193,6 +1215,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)\n \t\tgit_xmerge_config(\"merge.conflictstyle\", conflict_style, NULL);\n \t}\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_OFF) {\n+\t\tgit_config(submodule_config, NULL);\n+\t\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)\n+\t\t\tset_config_update_recurse_submodules(recurse_submodules);\n+\t}\n+\n \tif ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)\n \t\tdie(_(\"-b, -B and --orphan are mutually exclusive\"));\n \ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex ae560dc95c..4e775a38b6 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -782,6 +782,16 @@ test_submodule_forced_switch () {\n \n test_submodule_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULTDS=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULTDS=failure\n+\tfi\n+\tRESULTR=success\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_RECURSIVE_NESTED\" = 1\n+\tthen\n+\t\tRESULTR=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -891,7 +901,7 @@ test_submodule_switch_recursing () {\n \t'\n \t# Replacing a submodule with files in a directory must succeeds\n \t# when the submodule is clean\n-\ttest_expect_success \"$command: replace submodule with a directory\" '\n+\ttest_expect_$RESULTDS \"$command: replace submodule with a directory\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -903,7 +913,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... absorbing a .git directory.\n-\ttest_expect_success \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n+\ttest_expect_$RESULTDS \"$command: replace submodule containing a .git directory with a directory must absorb the git dir\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -931,7 +941,7 @@ test_submodule_switch_recursing () {\n \t'\n \n \t# ... must check its local work tree for untracked files\n-\ttest_expect_success \"$command: replace submodule with a file must fail with untracked files\" '\n+\ttest_expect_$RESULTDS \"$command: replace submodule with a file must fail with untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\n@@ -987,7 +997,8 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \n-\ttest_expect_success \"$command: modified submodule updates submodule recursively\" '\n+\t# recursing deeper than one level doesn't work yet.\n+\ttest_expect_$RESULTR \"$command: modified submodule updates submodule recursively\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_nested_sub &&\n \t\t(\n@@ -1006,6 +1017,11 @@ test_submodule_switch_recursing () {\n # the superproject as well as the submodule is allowed.\n test_submodule_forced_switch_recursing () {\n \tcommand=\"$1\"\n+\tRESULT=success\n+\tif test \"$KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS\" = 1\n+\tthen\n+\t\tRESULT=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear creates empty dir ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -1151,7 +1167,7 @@ test_submodule_forced_switch_recursing () {\n \t'\n \n \t# ... but stops for untracked files that would be lost\n-\ttest_expect_success \"$command: replace submodule with a file\" '\n+\ttest_expect_$RESULT \"$command: replace submodule with a file stops for untracked files\" '\n \t\tprolog &&\n \t\treset_work_tree_to_interested add_sub1 &&\n \t\t(\ndiff --git a/t/t2013-checkout-submodule.sh b/t/t2013-checkout-submodule.sh\nindex 6847f75822..e8f70b806f 100755\n--- a/t/t2013-checkout-submodule.sh\n+++ b/t/t2013-checkout-submodule.sh\n@@ -63,6 +63,12 @@ test_expect_success '\"checkout \" honors submodule.*.ignore from .git/\n \t! test -s actual\n '\n \n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_RECURSIVE_NESTED=1\n+test_submodule_switch_recursing \"git checkout --recurse-submodules\"\n+\n+test_submodule_forced_switch_recursing \"git checkout -f --recurse-submodules\"\n+\n test_submodule_switch \"git checkout\"\n \n test_submodule_forced_switch \"git checkout -f\"\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314075","messageId":"20170314214642.7701-17-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 16/19] unpack-trees: check if we can perform the operation for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:39Z","receivedAt":"2017-03-14T21:47:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we'll support submodule entries to be\nin sync with the tree in working tree changing commands,\nsuch as checkout or read-tree.\n\nWhen a new submodule entry changes in the tree, we need to\ncheck if there are conflicts (directory/file conflicts)\nfor the tree. Add this check for submodules to be\nperformed before the working tree is touched.\n\nSigned-off-by: Stefan Beller \n---\n unpack-trees.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++++++----\n unpack-trees.h | 1 +\n 2 files changed, 123 insertions(+), 9 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 616a0ae4b2..8333da2cc9 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -10,6 +10,8 @@\n #include \"attr.h\"\n #include \"split-index.h\"\n #include \"dir.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n /*\n * Error messages expected by scripts out of plumbing commands such as\n@@ -45,6 +47,9 @@ static const char *unpack_plumbing_errors[NB_UNPACK_TREES_ERROR_TYPES] = {\n \n \t/* ERROR_WOULD_LOSE_ORPHANED_REMOVED */\n \t\"Working tree file '%s' would be removed by sparse checkout update.\",\n+\n+\t/* ERROR_WOULD_LOSE_SUBMODULE */\n+\t\"Submodule '%s' cannot checkout new HEAD.\",\n };\n \n #define ERRORMSG(o,type) \\\n@@ -161,6 +166,8 @@ void setup_unpack_trees_porcelain(struct unpack_trees_options *opts,\n \t\t_(\"The following working tree files would be overwritten by sparse checkout update:\\n%s\");\n \tmsgs[ERROR_WOULD_LOSE_ORPHANED_REMOVED] =\n \t\t_(\"The following working tree files would be removed by sparse checkout update:\\n%s\");\n+\tmsgs[ERROR_WOULD_LOSE_SUBMODULE] =\n+\t\t_(\"Submodule '%s' cannot checkout new HEAD\");\n \n \topts->show_all_errors = 1;\n \t/* rejected paths may not have a static buffer */\n@@ -240,12 +247,75 @@ static void display_error_msgs(struct unpack_trees_options *o)\n \t\tfprintf(stderr, _(\"Aborting\\n\"));\n }\n \n+static int check_submodule_move_head(const struct cache_entry *ce,\n+\t\t\t\t const char *old_id,\n+\t\t\t\t const char *new_id,\n+\t\t\t\t struct unpack_trees_options *o)\n+{\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (!sub)\n+\t\treturn 0;\n+\n+\tswitch (sub->update_strategy.type) {\n+\tcase SM_UPDATE_UNSPECIFIED:\n+\tcase SM_UPDATE_CHECKOUT:\n+\t\tif (submodule_move_head(ce->name, old_id, new_id, SUBMODULE_MOVE_HEAD_DRY_RUN))\n+\t\t\treturn o->gently ? -1 :\n+\t\t\t\tadd_rejected_path(o, ERROR_WOULD_LOSE_SUBMODULE, ce->name);\n+\t\treturn 0;\n+\tcase SM_UPDATE_NONE:\n+\t\treturn 0;\n+\tcase SM_UPDATE_REBASE:\n+\tcase SM_UPDATE_MERGE:\n+\tcase SM_UPDATE_COMMAND:\n+\tdefault:\n+\t\twarning(_(\"submodule update strategy not supported for submodule '%s'\"), ce->name);\n+\t\treturn -1;\n+\t}\n+}\n+\n+static void reload_gitmodules_file(struct index_state *index,\n+\t\t\t\t struct checkout *state)\n+{\n+\tint i;\n+\tfor (i = 0; i < index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tif (ce->ce_flags & CE_UPDATE) {\n+\t\t\tint r = strcmp(ce->name, \".gitmodules\");\n+\t\t\tif (r < 0)\n+\t\t\t\tcontinue;\n+\t\t\telse if (r == 0) {\n+\t\t\t\tsubmodule_free();\n+\t\t\t\tcheckout_entry(ce, state, NULL);\n+\t\t\t\tgitmodules_config();\n+\t\t\t\tgit_config(submodule_config, NULL);\n+\t\t\t} else\n+\t\t\t\tbreak;\n+\t\t}\n+\t}\n+}\n+\n /*\n * Unlink the last component and schedule the leading directories for\n * removal, such that empty directories get removed.\n */\n static void unlink_entry(const struct cache_entry *ce)\n {\n+\tconst struct submodule *sub = submodule_from_ce(ce);\n+\tif (sub) {\n+\t\tswitch (sub->update_strategy.type) {\n+\t\tcase SM_UPDATE_UNSPECIFIED:\n+\t\tcase SM_UPDATE_CHECKOUT:\n+\t\tcase SM_UPDATE_REBASE:\n+\t\tcase SM_UPDATE_MERGE:\n+\t\t\tsubmodule_move_head(ce->name, \"HEAD\", NULL,\n+\t\t\t\t\t SUBMODULE_MOVE_HEAD_FORCE);\n+\t\t\tbreak;\n+\t\tcase SM_UPDATE_NONE:\n+\t\tcase SM_UPDATE_COMMAND:\n+\t\t\treturn; /* Do not touch the submodule. */\n+\t\t}\n+\t}\n \tif (!check_leading_path(ce->name, ce_namelen(ce)))\n \t\treturn;\n \tif (remove_or_warn(ce->ce_mode, ce->name))\n@@ -301,6 +371,9 @@ static int check_updates(struct unpack_trees_options *o)\n \tremove_marked_cache_entries(index);\n \tremove_scheduled_dirs();\n \n+\tif (should_update_submodules() && o->update && !o->dry_run)\n+\t\treload_gitmodules_file(index, &state);\n+\n \tfor (i = 0; i < index->cache_nr; i++) {\n \t\tstruct cache_entry *ce = index->cache[i];\n \n@@ -1358,17 +1431,26 @@ static int verify_uptodate_1(const struct cache_entry *ce,\n \tif (!lstat(ce->name, &st)) {\n \t\tint flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;\n \t\tunsigned changed = ie_match_stat(o->src_index, ce, &st, flags);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint r = check_submodule_move_head(ce,\n+\t\t\t\t\"HEAD\", oid_to_hex(&ce->oid), o);\n+\t\t\tif (r)\n+\t\t\t\treturn o->gently ? -1 :\n+\t\t\t\t\tadd_rejected_path(o, error_type, ce->name);\n+\t\t\treturn 0;\n+\t\t}\n+\n \t\tif (!changed)\n \t\t\treturn 0;\n \t\t/*\n-\t\t * NEEDSWORK: the current default policy is to allow\n-\t\t * submodule to be out of sync wrt the superproject\n-\t\t * index. This needs to be tightened later for\n-\t\t * submodules that are marked to be automatically\n-\t\t * checked out.\n+\t\t * Historic default policy was to allow submodule to be out\n+\t\t * of sync wrt the superproject index. If the submodule was\n+\t\t * not considered interesting above, we don't care here.\n \t\t */\n \t\tif (S_ISGITLINK(ce->ce_mode))\n \t\t\treturn 0;\n+\n \t\terrno = 0;\n \t}\n \tif (errno == ENOENT)\n@@ -1412,7 +1494,11 @@ static int verify_clean_submodule(const char *old_sha1,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n-\treturn 0;\n+\tif (!submodule_from_ce(ce))\n+\t\treturn 0;\n+\n+\treturn check_submodule_move_head(ce, old_sha1,\n+\t\t\t\t\t oid_to_hex(&ce->oid), o);\n }\n \n static int verify_clean_subdirectory(const struct cache_entry *ce,\n@@ -1578,9 +1664,15 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\tpath = xmemdupz(ce->name, len);\n \t\tif (lstat(path, &st))\n \t\t\tret = error_errno(\"cannot stat '%s'\", path);\n-\t\telse\n-\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n-\t\t\t\t\t\t &st, error_type, o);\n+\t\telse {\n+\t\t\tif (submodule_from_ce(ce))\n+\t\t\t\tret = check_submodule_move_head(ce,\n+\t\t\t\t\t\t\t\toid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t\tNULL, o);\n+\t\t\telse\n+\t\t\t\tret = check_ok_to_remove(path, len, DT_UNKNOWN, NULL,\n+\t\t\t\t\t\t\t &st, error_type, o);\n+\t\t}\n \t\tfree(path);\n \t\treturn ret;\n \t} else if (lstat(ce->name, &st)) {\n@@ -1588,6 +1680,10 @@ static int verify_absent_1(const struct cache_entry *ce,\n \t\t\treturn error_errno(\"cannot stat '%s'\", ce->name);\n \t\treturn 0;\n \t} else {\n+\t\tif (submodule_from_ce(ce))\n+\t\t\treturn check_submodule_move_head(ce, oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t NULL, o);\n+\n \t\treturn check_ok_to_remove(ce->name, ce_namelen(ce),\n \t\t\t\t\t ce_to_dtype(ce), ce, &st,\n \t\t\t\t\t error_type, o);\n@@ -1643,6 +1739,15 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\treturn -1;\n \t\t}\n \t\tinvalidate_ce_path(merge, o);\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, NULL,\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n+\n \t} else if (!(old->ce_flags & CE_CONFLICTED)) {\n \t\t/*\n \t\t * See if we can re-use the old CE directly?\n@@ -1663,6 +1768,14 @@ static int merged_entry(const struct cache_entry *ce,\n \t\t\tupdate |= old->ce_flags & (CE_SKIP_WORKTREE | CE_NEW_SKIP_WORKTREE);\n \t\t\tinvalidate_ce_path(old, o);\n \t\t}\n+\n+\t\tif (submodule_from_ce(ce)) {\n+\t\t\tint ret = check_submodule_move_head(ce, oid_to_hex(&old->oid),\n+\t\t\t\t\t\t\t oid_to_hex(&ce->oid),\n+\t\t\t\t\t\t\t o);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\t\t}\n \t} else {\n \t\t/*\n \t\t * Previously unmerged entry left as an existence\ndiff --git a/unpack-trees.h b/unpack-trees.h\nindex 36a73a6d00..6c48117b84 100644\n--- a/unpack-trees.h\n+++ b/unpack-trees.h\n@@ -21,6 +21,7 @@ enum unpack_trees_error_types {\n \tERROR_SPARSE_NOT_UPTODATE_FILE,\n \tERROR_WOULD_LOSE_ORPHANED_OVERWRITTEN,\n \tERROR_WOULD_LOSE_ORPHANED_REMOVED,\n+\tERROR_WOULD_LOSE_SUBMODULE,\n \tNB_UNPACK_TREES_ERROR_TYPES\n };\n \n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314076","messageId":"20170314214642.7701-10-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 09/19] update submodules: add submodule config parsing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:32Z","receivedAt":"2017-03-14T21:47:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Similar to b33a15b08 (push: add recurseSubmodules config option,\n2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the\nfetchRecurseSubmodules option, 2015-08-17), we add submodule-config code\nthat is later used to parse whether we are interested in updating\nsubmodules.\n\nWe need the `die_on_error` parameter to be able to call this parsing\nfunction for the config file as well, which if incorrect lets Git die.\n\nAs we're just touching the header file, also mark all functions extern.\n\nSigned-off-by: Stefan Beller \n---\n submodule-config.c | 20 ++++++++++++++++++++\n submodule-config.h | 17 +++++++++--------\n 2 files changed, 29 insertions(+), 8 deletions(-)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 93453909cf..3e8e380d98 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -234,6 +234,26 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n \treturn parse_fetch_recurse(opt, arg, 1);\n }\n \n+static int parse_update_recurse(const char *opt, const char *arg,\n+\t\t\t\tint die_on_error)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (die_on_error)\n+\t\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t\treturn RECURSE_SUBMODULES_ERROR;\n+\t}\n+}\n+\n+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\treturn parse_update_recurse(opt, arg, 1);\n+}\n+\n static int parse_push_recurse(const char *opt, const char *arg,\n \t\t\t int die_on_error)\n {\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 70f19363fd..d434ecdb45 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -22,16 +22,17 @@ struct submodule {\n \tint recommend_shallow;\n };\n \n-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n-int parse_submodule_config_option(const char *var, const char *value);\n-const struct submodule *submodule_from_name(const unsigned char *commit_or_tree,\n-\t\tconst char *name);\n-const struct submodule *submodule_from_path(const unsigned char *commit_or_tree,\n-\t\tconst char *path);\n+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);\n+extern int parse_submodule_config_option(const char *var, const char *value);\n+extern const struct submodule *submodule_from_name(\n+\t\tconst unsigned char *commit_or_tree, const char *name);\n+extern const struct submodule *submodule_from_path(\n+\t\tconst unsigned char *commit_or_tree, const char *path);\n extern int gitmodule_sha1_from_commit(const unsigned char *commit_sha1,\n \t\t\t\t unsigned char *gitmodules_sha1,\n \t\t\t\t struct strbuf *rev);\n-void submodule_free(void);\n+extern void submodule_free(void);\n \n #endif /* SUBMODULE_CONFIG_H */\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314077","messageId":"20170314214642.7701-20-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 19/19] builtin/read-tree: add --recurse-submodules switch","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:42Z","receivedAt":"2017-03-14T21:47:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"A new known failure mode is introduced[1], which is actually not\na failure but a feature in read-tree. Unlike checkout for which\nthe recursive submodule tests were originally written, read-tree does\nwarn about ignored untracked files that would be overwritten.\nFor the sake of keeping the test library for submodules generic, just\nmark the test as a failure.\n\n[1] KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\n\nSigned-off-by: Stefan Beller \n---\n Documentation/git-read-tree.txt | 6 ++++++\n builtin/read-tree.c | 29 +++++++++++++++++++++++++++++\n t/lib-submodule-update.sh | 7 ++++++-\n t/t1013-read-tree-submodule.sh | 8 ++++++++\n 4 files changed, 49 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt\nindex fa1d557e5b..ed9d63ef4a 100644\n--- a/Documentation/git-read-tree.txt\n+++ b/Documentation/git-read-tree.txt\n@@ -115,6 +115,12 @@ OPTIONS\n \tdirectories the index file and index output file are\n \tlocated in.\n \n+--[no-]recurse-submodules::\n+\tUsing --recurse-submodules will update the content of all initialized\n+\tsubmodules according to the commit recorded in the superproject by\n+\tcalling read-tree recursively, also setting the submodules HEAD to be\n+\tdetached at that commit.\n+\n --no-sparse-checkout::\n \tDisable sparse checkout support even if `core.sparseCheckout`\n \tis true.\ndiff --git a/builtin/read-tree.c b/builtin/read-tree.c\nindex 8ba64bc785..23e212ee8c 100644\n--- a/builtin/read-tree.c\n+++ b/builtin/read-tree.c\n@@ -15,10 +15,13 @@\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"resolve-undo.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n \n static int nr_trees;\n static int read_empty;\n static struct tree *trees[MAX_UNPACK_TREES];\n+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n \n static int list_tree(unsigned char *sha1)\n {\n@@ -96,6 +99,23 @@ static int debug_merge(const struct cache_entry * const *stages,\n \treturn 0;\n }\n \n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t\t const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t\treturn 0;\n+\t}\n+\tif (arg)\n+\t\trecurse_submodules =\n+\t\t\tparse_update_recurse_submodules_arg(opt->long_name,\n+\t\t\t\t\t\t\t arg);\n+\telse\n+\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\n+\treturn 0;\n+}\n+\n static struct lock_file lock_file;\n \n int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n@@ -137,6 +157,9 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \t\t\t N_(\"skip applying sparse checkout filter\")),\n \t\tOPT_BOOL(0, \"debug-unpack\", &opts.debug_unpack,\n \t\t\t N_(\"debug unpack-trees\")),\n+\t\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\t \"checkout\", \"control recursive updating of submodules\",\n+\t\t\t PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \t\tOPT_END()\n \t};\n \n@@ -152,6 +175,12 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)\n \n \thold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);\n \n+\tif (recurse_submodules != RECURSE_SUBMODULES_DEFAULT) {\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tset_config_update_recurse_submodules(RECURSE_SUBMODULES_ON);\n+\t}\n+\n \tprefix_set = opts.prefix ? 1 : 0;\n \tif (1 < opts.merge + opts.reset + prefix_set)\n \t\tdie(\"Which one? -m, --reset, or --prefix?\");\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 4e775a38b6..fb4f7b014e 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -792,6 +792,11 @@ test_submodule_switch_recursing () {\n \tthen\n \t\tRESULTR=failure\n \tfi\n+\tRESULTOI=success\n+\tif test \"$KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED\" = 1\n+\tthen\n+\t\tRESULTOI=failure\n+\tfi\n \t######################### Appearing submodule #########################\n \t# Switching to a commit letting a submodule appear checks it out ...\n \ttest_expect_success \"$command: added submodule is checked out\" '\n@@ -832,7 +837,7 @@ test_submodule_switch_recursing () {\n \t\t)\n \t'\n \t# ... but an ignored file is fine.\n-\ttest_expect_success \"$command: added submodule removes an untracked ignored file\" '\n+\ttest_expect_$RESULTOI \"$command: added submodule removes an untracked ignored file\" '\n \t\ttest_when_finished \"rm submodule_update/.git/info/exclude\" &&\n \t\tprolog &&\n \t\treset_work_tree_to_interested no_submodule &&\ndiff --git a/t/t1013-read-tree-submodule.sh b/t/t1013-read-tree-submodule.sh\nindex 20526aed34..de1ba02dc5 100755\n--- a/t/t1013-read-tree-submodule.sh\n+++ b/t/t1013-read-tree-submodule.sh\n@@ -5,6 +5,14 @@ test_description='read-tree can handle submodules'\n . ./test-lib.sh\n . \"$TEST_DIRECTORY\"/lib-submodule-update.sh\n \n+KNOWN_FAILURE_SUBMODULE_RECURSIVE_NESTED=1\n+KNOWN_FAILURE_DIRECTORY_SUBMODULE_CONFLICTS=1\n+KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED=1\n+\n+test_submodule_switch_recursing \"git read-tree --recurse-submodules -u -m\"\n+\n+test_submodule_forced_switch_recursing \"git read-tree --recurse-submodules -u --reset\"\n+\n test_submodule_switch \"git read-tree -u -m\"\n \n test_submodule_forced_switch \"git read-tree -u --reset\"\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314078","messageId":"20170314214642.7701-16-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 15/19] unpack-trees: pass old oid to verify_clean_submodule","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:38Z","receivedAt":"2017-03-14T21:47:40Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The check (which uses the old oid) is yet to be implemented, but this part\nis just a refactor, so it can go separately first.\n\nSigned-off-by: Stefan Beller \n---\n unpack-trees.c | 17 ++++++++++-------\n 1 file changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 3a8ee19fe8..616a0ae4b2 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -1407,7 +1407,8 @@ static void invalidate_ce_path(const struct cache_entry *ce,\n * Currently, git does not checkout subprojects during a superproject\n * checkout, so it is not going to overwrite anything.\n */\n-static int verify_clean_submodule(const struct cache_entry *ce,\n+static int verify_clean_submodule(const char *old_sha1,\n+\t\t\t\t const struct cache_entry *ce,\n \t\t\t\t enum unpack_trees_error_types error_type,\n \t\t\t\t struct unpack_trees_options *o)\n {\n@@ -1427,16 +1428,18 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,\n \tstruct dir_struct d;\n \tchar *pathbuf;\n \tint cnt = 0;\n-\tunsigned char sha1[20];\n \n-\tif (S_ISGITLINK(ce->ce_mode) &&\n-\t resolve_gitlink_ref(ce->name, \"HEAD\", sha1) == 0) {\n-\t\t/* If we are not going to update the submodule, then\n+\tif (S_ISGITLINK(ce->ce_mode)) {\n+\t\tunsigned char sha1[20];\n+\t\tint sub_head = resolve_gitlink_ref(ce->name, \"HEAD\", sha1);\n+\t\t/*\n+\t\t * If we are not going to update the submodule, then\n \t\t * we don't care.\n \t\t */\n-\t\tif (!hashcmp(sha1, ce->oid.hash))\n+\t\tif (!sub_head && !hashcmp(sha1, ce->oid.hash))\n \t\t\treturn 0;\n-\t\treturn verify_clean_submodule(ce, error_type, o);\n+\t\treturn verify_clean_submodule(sub_head ? NULL : sha1_to_hex(sha1),\n+\t\t\t\t\t ce, error_type, o);\n \t}\n \n \t/*\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314079","messageId":"20170314214642.7701-7-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 06/19] lib-submodule-update.sh: replace sha1 by hash","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:29Z","receivedAt":"2017-03-14T21:47:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Cleaning up code by generalising it.\nCurrently the mailing list discusses yet again how\nto migrate away from sha1.\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 00128f28b5..f52c49c838 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -171,9 +171,9 @@ reset_work_tree_to () {\n \t\tgit checkout -f \"$1\" &&\n \t\tgit status -u -s >actual &&\n \t\ttest_must_be_empty actual &&\n-\t\tsha1=$(git rev-parse --revs-only HEAD:sub1) &&\n-\t\tif test -n \"$sha1\" &&\n-\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$sha1^{commit}\")\n+\t\thash=$(git rev-parse --revs-only HEAD:sub1) &&\n+\t\tif test -n \"$hash\" &&\n+\t\t test $(cd \"../submodule_update_sub1\" && git rev-parse --verify \"$hash^{commit}\")\n \t\tthen\n \t\t\tgit submodule update --init --recursive \"sub1\"\n \t\tfi\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314080","messageId":"20170314214642.7701-9-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 08/19] make is_submodule_populated gently","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:31Z","receivedAt":"2017-03-14T21:47:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the gentle version in a later patch. As we have just one caller,\nmigrate the caller.\n\nSigned-off-by: Stefan Beller \n---\n builtin/grep.c | 2 +-\n submodule.c | 7 ++-----\n submodule.h | 8 +++++++-\n 3 files changed, 10 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 2c727ef499..b17835aed6 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -616,7 +616,7 @@ static int grep_submodule(struct grep_opt *opt, const unsigned char *sha1,\n {\n \tif (!is_submodule_initialized(path))\n \t\treturn 0;\n-\tif (!is_submodule_populated(path)) {\n+\tif (!is_submodule_populated_gently(path, NULL)) {\n \t\t/*\n \t\t * If searching history, check for the presense of the\n \t\t * submodule's gitdir before skipping the submodule.\ndiff --git a/submodule.c b/submodule.c\nindex 45e93a1d55..04d185738f 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -234,15 +234,12 @@ int is_submodule_initialized(const char *path)\n \treturn ret;\n }\n \n-/*\n- * Determine if a submodule has been populated at a given 'path'\n- */\n-int is_submodule_populated(const char *path)\n+int is_submodule_populated_gently(const char *path, int *return_error_code)\n {\n \tint ret = 0;\n \tchar *gitdir = xstrfmt(\"%s/.git\", path);\n \n-\tif (resolve_gitdir(gitdir))\n+\tif (resolve_gitdir_gently(gitdir, return_error_code))\n \t\tret = 1;\n \n \tfree(gitdir);\ndiff --git a/submodule.h b/submodule.h\nindex 05ab674f06..0b915bd3ac 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -41,7 +41,13 @@ extern int submodule_config(const char *var, const char *value, void *cb);\n extern void gitmodules_config(void);\n extern void gitmodules_config_sha1(const unsigned char *commit_sha1);\n extern int is_submodule_initialized(const char *path);\n-extern int is_submodule_populated(const char *path);\n+/*\n+ * Determine if a submodule has been populated at a given 'path' by checking if\n+ * the /.git resolves to a valid git repository.\n+ * If return_error_code is NULL, die on error.\n+ * Otherwise the return error code is the same as of resolve_gitdir_gently.\n+ */\n+extern int is_submodule_populated_gently(const char *path, int *return_error_code);\n extern int parse_submodule_update_strategy(const char *value,\n \t\tstruct submodule_update_strategy *dst);\n extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314081","messageId":"20170314214642.7701-13-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 12/19] update submodules: move up prepare_submodule_repo_env","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:35Z","receivedAt":"2017-03-14T21:47:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to prepare the submodule environment with\nanother git directory, so split up the function.\n\nAlso move it up in the file such that we do not need to declare the\nfunction later before using it.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 29 +++++++++++++++++------------\n 1 file changed, 17 insertions(+), 12 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 8b2c0212be..0b2596e88a 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -356,6 +356,23 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,\n \tstrbuf_release(&sb);\n }\n \n+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)\n+{\n+\tconst char * const *var;\n+\n+\tfor (var = local_repo_env; *var; var++) {\n+\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n+\t\t\targv_array_push(out, *var);\n+\t}\n+}\n+\n+void prepare_submodule_repo_env(struct argv_array *out)\n+{\n+\tprepare_submodule_repo_env_no_git_dir(out);\n+\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n+\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n+}\n+\n /* Helper function to display the submodule header line prior to the full\n * summary output. If it can locate the submodule objects directory it will\n * attempt to lookup both the left and right commits and put them into the\n@@ -1390,18 +1407,6 @@ int parallel_submodules(void)\n \treturn parallel_jobs;\n }\n \n-void prepare_submodule_repo_env(struct argv_array *out)\n-{\n-\tconst char * const *var;\n-\n-\tfor (var = local_repo_env; *var; var++) {\n-\t\tif (strcmp(*var, CONFIG_DATA_ENVIRONMENT))\n-\t\t\targv_array_push(out, *var);\n-\t}\n-\targv_array_pushf(out, \"%s=%s\", GIT_DIR_ENVIRONMENT,\n-\t\t\t DEFAULT_GIT_DIR_ENVIRONMENT);\n-}\n-\n /*\n * Embeds a single submodules git directory into the superprojects git dir,\n * non recursively.\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314082","messageId":"20170314214642.7701-14-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 13/19] submodule.c: get_super_prefix_or_empty","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:36Z","receivedAt":"2017-03-14T21:47:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a later patch we need to use the super_prefix, and\nin case it is NULL we can just assume it is empty.\nCreate a helper function for this.\n\nWe already have some use cases for this helper function,\nconvert them, too.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 16 ++++++++++------\n 1 file changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 0b2596e88a..3501497a78 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -1239,6 +1239,14 @@ int bad_to_remove_submodule(const char *path, unsigned flags)\n \treturn ret;\n }\n \n+static const char *get_super_prefix_or_empty()\n+{\n+\tconst char *s = get_super_prefix();\n+\tif (!s)\n+\t\ts = \"\";\n+\treturn s;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\n@@ -1438,11 +1446,8 @@ static void relocate_single_git_dir_into_superproject(const char *prefix,\n \t\tdie(_(\"could not create directory '%s'\"), new_git_dir);\n \treal_new_git_dir = real_pathdup(new_git_dir);\n \n-\tif (!prefix)\n-\t\tprefix = get_super_prefix();\n-\n \tfprintf(stderr, _(\"Migrating git directory of '%s%s' from\\n'%s' to\\n'%s'\\n\"),\n-\t\tprefix ? prefix : \"\", path,\n+\t\tget_super_prefix_or_empty(), path,\n \t\treal_old_git_dir, real_new_git_dir);\n \n \trelocate_gitdir(path, real_old_git_dir, real_new_git_dir);\n@@ -1513,8 +1518,7 @@ void absorb_git_dir_into_superproject(const char *prefix,\n \t\tif (flags & ~ABSORB_GITDIR_RECURSE_SUBMODULES)\n \t\t\tdie(\"BUG: we don't know how to pass the flags down?\");\n \n-\t\tif (get_super_prefix())\n-\t\t\tstrbuf_addstr(&sb, get_super_prefix());\n+\t\tstrbuf_addstr(&sb, get_super_prefix_or_empty());\n \t\tstrbuf_addstr(&sb, path);\n \t\tstrbuf_addch(&sb, '/');\n \n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314083","messageId":"20170314214642.7701-4-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 03/19] lib-submodule-update.sh: reorder create_lib_submodule_repo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:26Z","receivedAt":"2017-03-14T21:48:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Redraw the ASCII art describing the setup using more space, such that\nit is easier to understand. The leaf commits are now ordered the same\nway the actual code is ordered.\n\nAdd empty lines to the setup code separating each of the leaf commits,\neach starting with a \"checkout -b\".\n\nSigned-off-by: Stefan Beller \n---\n t/lib-submodule-update.sh | 49 ++++++++++++++++++++++++++++-------------------\n 1 file changed, 29 insertions(+), 20 deletions(-)\n\ndiff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh\nindex 915eb4a7c6..5df528ea81 100755\n--- a/t/lib-submodule-update.sh\n+++ b/t/lib-submodule-update.sh\n@@ -15,22 +15,27 @@\n # - Tracked file replaced by submodule (replace_sub1_with_file =>\n # replace_file_with_sub1)\n #\n-# --O-----O\n-# / ^ replace_directory_with_sub1\n-# / replace_sub1_with_directory\n-# /----O\n-# / ^\n-# / modify_sub1\n-# O------O-------O\n-# ^ ^\\ ^\n-# | | \\ remove_sub1\n-# | | -----O-----O\n-# | | \\ ^ replace_file_with_sub1\n-# | | \\ replace_sub1_with_file\n-# | add_sub1 --O-----O\n-# no_submodule ^ valid_sub1\n-# invalid_sub1\n+# ----O\n+# / ^\n+# / remove_sub1\n+# /\n+# add_sub1 /-------O\n+# | / ^\n+# | / modify_sub1\n+# v/\n+# O------O-----------O---------O\n+# ^ \\ ^ replace_directory_with_sub1\n+# | \\ replace_sub1_with_directory\n+# no_submodule \\\n+# --------O---------O\n+# \\ ^ replace_file_with_sub1\n+# \\ replace_sub1_with_file\n+# \\\n+# ----O---------O\n+# ^ valid_sub1\n+# invalid_sub1\n #\n+\n create_lib_submodule_repo () {\n \tgit init submodule_update_repo &&\n \t(\n@@ -49,10 +54,11 @@ create_lib_submodule_repo () {\n \t\tgit config submodule.sub1.ignore all &&\n \t\tgit add .gitmodules &&\n \t\tgit commit -m \"Add sub1\" &&\n-\t\tgit checkout -b remove_sub1 &&\n+\n+\t\tgit checkout -b remove_sub1 add_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"modify_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b modify_sub1 add_sub1 &&\n \t\tgit submodule update &&\n \t\t(\n \t\t\tcd sub1 &&\n@@ -67,7 +73,7 @@ create_lib_submodule_repo () {\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Modify sub1\" &&\n \n-\t\tgit checkout -b \"replace_sub1_with_directory\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_directory add_sub1 &&\n \t\tgit submodule update &&\n \t\tgit -C sub1 checkout modifications &&\n \t\tgit rm --cached sub1 &&\n@@ -75,22 +81,25 @@ create_lib_submodule_repo () {\n \t\tgit config -f .gitmodules --remove-section \"submodule.sub1\" &&\n \t\tgit add .gitmodules sub1/* &&\n \t\tgit commit -m \"Replace sub1 with directory\" &&\n+\n \t\tgit checkout -b replace_directory_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"replace_sub1_with_file\" \"add_sub1\" &&\n+\t\tgit checkout -b replace_sub1_with_file add_sub1 &&\n \t\tgit rm sub1 &&\n \t\techo \"content\" >sub1 &&\n \t\tgit add sub1 &&\n \t\tgit commit -m \"Replace sub1 with file\" &&\n+\n \t\tgit checkout -b replace_file_with_sub1 &&\n \t\tgit revert HEAD &&\n \n-\t\tgit checkout -b \"invalid_sub1\" \"add_sub1\" &&\n+\t\tgit checkout -b invalid_sub1 add_sub1 &&\n \t\tgit update-index --cacheinfo 160000 0123456789012345678901234567890123456789 sub1 &&\n \t\tgit commit -m \"Invalid sub1 commit\" &&\n \t\tgit checkout -b valid_sub1 &&\n \t\tgit revert HEAD &&\n+\n \t\tgit checkout master\n \t)\n }\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314084","messageId":"20170314214642.7701-11-sbeller@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"[PATCH 10/19] update submodules: add a config option to determine if submodules are updated","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-14T21:46:33Z","receivedAt":"2017-03-14T21:48:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In later patches we introduce the options and flag for commands\nthat modify the working directory, e.g. git-checkout.\n\nHave a central place to store such settings whether we want to update\na submodule.\n\nSigned-off-by: Stefan Beller \n---\n submodule.c | 6 ++++++\n submodule.h | 1 +\n 2 files changed, 7 insertions(+)\n\ndiff --git a/submodule.c b/submodule.c\nindex 04d185738f..591f4a694e 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -17,6 +17,7 @@\n #include \"worktree.h\"\n \n static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int parallel_jobs = 1;\n static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;\n static int initialized_fetch_ref_tips;\n@@ -542,6 +543,11 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n \n+void set_config_update_recurse_submodules(int value)\n+{\n+\tconfig_update_recurse_submodules = value;\n+}\n+\n static int has_remote(const char *refname, const struct object_id *oid,\n \t\t int flags, void *cb_data)\n {\ndiff --git a/submodule.h b/submodule.h\nindex 0b915bd3ac..b4e60c08d2 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -64,6 +64,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,\n \t\tconst char *del, const char *add, const char *reset,\n \t\tconst struct diff_options *opt);\n extern void set_config_fetch_recurse_submodules(int value);\n+extern void set_config_update_recurse_submodules(int value);\n extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n extern int fetch_populated_submodules(const struct argv_array *options,\n \t\t\t const char *prefix, int command_line_option,\n-- \n2.12.0.rc1.49.g0cfd38c5f6.dirty\n\n"},{"id":"314115","messageId":"20170315004156.GI168037@google.com","threadId":"45260","inReplyTo":"20170314214642.7701-1-sbeller@google.com","subject":"Re: [PATCHv9 00/19] Checkout aware of Submodules!","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-03-15T00:41:56Z","receivedAt":"2017-03-15T00:42:04Z","isPatch":false,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 03/14, Stefan Beller wrote:\n> previous work:\n> https://public-inbox.org/git/20170309221543.15897-1-sbeller@google.com/\n> \n> v9:\n> * inderdiff to current origin/sb/checkout-recurse-submodules below\n> * fixed a '\\' that was violating the style guide.\n> * reordered the patches, such that\n> -> the first two patches are Valerys series and could go on its own as\n> a cleanup \n> -> added a new patch \"submodule.c: get_super_prefix_or_empty\" to\n> have cleaner code.\n> * split up the error conditions in the test lib for readability\n\nChanges looks sensible to me.\n\n-- \nBrandon Williams\n"}]}