{"thread":{"id":"31366","subject":"[RFC v2 PATCH] Teach rm to remove submodules unless they contain a git directory","startedAt":"2012-08-27T19:42:09Z","lastAt":"2012-08-28T18:29:29Z","messageCount":3,"participants":["Jens Lehmann","Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"197930","messageId":"503BCD91.6080107@web.de","threadId":"31366","inReplyTo":null,"subject":"[RFC v2 PATCH] Teach rm to remove submodules unless they contain a git directory","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-08-27T19:42:09Z","receivedAt":"2012-08-27T19:42:09Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Currently using \"git rm\" on a submodule - populated or not - fails with\nthis error:\n\tfatal: git rm: '<submodule path>': Is a directory\nThis made sense in the past as there was no way to remove a submodule\nwithout possibly removing unpushed parts of the submodule's history\ncontained in its .git directory too, so erroring out here protected the\nuser from possible loss of data.\n\nBut submodules cloned with a recent git version do not contain the .git\ndirectory anymore, they use a gitfile to point to their git directory\nwhich is safely stored inside the superproject's .git directory. The work\ntree of these submodules can safely be removed without loosing history, so\nlet's teach git to do so.\n\nUsing rm on an unpopulated submodule now removes the empty directory from\nthe work tree and the gitlink from the index. If the submodule's directory\nis missing from the work tree, it will still be removed from the index.\n\nUsing rm on a populated submodule using a gitfile will apply the usual\nchecks for work tree modification adapted to submodules (unless forced).\nFor a submodule that means that the HEAD is the same as recorded in the\nindex, no tracked files are modified and no untracked files that aren't\nignored are present in the submodules work tree (Ignored files are deemed\nexpendable and won't stop a submodule's work tree from being removed).\nThat logic has to be applied in all nested submodules too.\n\nUsing rm on a submodule which has its .git directory inside the work trees\ntop level directory will just error out like it did before, forced or not.\nIn the future git could either provide a message informing the user to\nconvert the submodule to use a gitfile or even attempt to do the\nconversion itself, but that is not part of this change.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n\n\nThis is the reroll of the \"Teach rm to better handle submodules\" series\n($gmane/201015). It does not attempt to convert submodules that still\ncontain their git directory (by moving their git directory into\n\".git/modules/<name>\" and replacing it with a gitfile pointing there).\nThat will be subject to a future patch, as I'm not sure yet if \"git rm\"\nshould do that automagically or rather tell the user to use a (still\nto be added) \"git submodule to-gitfile <path>\" invocation to achieve\nthat.\n\nIn a follow up patch I'll teach \"git rm submod/\" to not barf about the\ntrailing '/', as that is added by TAB completion.\n\n\n Documentation/git-rm.txt |  15 ++++\n builtin/rm.c             |  95 ++++++++++++++++++---\n submodule.c              |  80 ++++++++++++++++++\n submodule.h              |   2 +\n t/t3600-rm.sh            | 210 +++++++++++++++++++++++++++++++++++++++++++++++\n 5 files changed, 389 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt\nindex 5d31860..3c76f9c 100644\n--- a/Documentation/git-rm.txt\n+++ b/Documentation/git-rm.txt\n@@ -107,6 +107,21 @@ as well as modifications of existing paths.\n Typically you would first remove all tracked files from the working\n tree using this command:\n\n+Submodules\n+~~~~~~~~~~~~~~~~~~~~\n+Only submodules using a gitfile (which means they were cloned\n+with a git version 1.7.8 or newer) will be removed from the work\n+tree, as their repository lives inside the .git directory of the\n+superproject. If a submodule (or one of those nested inside it)\n+still use a .git directory, `git rm` will fail - no matter if forced\n+or not - to protect the submodules history.\n+\n+A submodule is considered up-to-date when the HEAD is the same as\n+recorded in the index, no tracked files are modified and no untracked\n+files that aren't ignored are present in the submodules work tree.\n+Ignored files are deemed expendable and won't stop a submodule's work\n+tree from being removed.\n+\n ----------------\n git ls-files -z | xargs -0 rm -f\n ----------------\ndiff --git a/builtin/rm.c b/builtin/rm.c\nindex 90c8a50..cb927a8 100644\n--- a/builtin/rm.c\n+++ b/builtin/rm.c\n@@ -9,6 +9,7 @@\n #include \"cache-tree.h\"\n #include \"tree-walk.h\"\n #include \"parse-options.h\"\n+#include \"submodule.h\"\n\n static const char * const builtin_rm_usage[] = {\n \t\"git rm [options] [--] <file>...\",\n@@ -17,9 +18,43 @@ static const char * const builtin_rm_usage[] = {\n\n static struct {\n \tint nr, alloc;\n-\tconst char **name;\n+\tstruct {\n+\t\tconst char *name;\n+\t\tchar is_submodule;\n+\t} *entry;\n } list;\n\n+static int check_submodules_use_gitfiles()\n+{\n+\tint i;\n+\tint errs = 0;\n+\n+\tfor (i = 0; i < list.nr; i++) {\n+\t\tconst char *name = list.entry[i].name;\n+\t\tint pos;\n+\t\tstruct cache_entry *ce;\n+\t\tstruct stat st;\n+\n+\t\tpos = cache_name_pos(name, strlen(name));\n+\t\tif (pos < 0)\n+\t\t\tcontinue; /* ignore unmerged entry */\n+\t\tce = active_cache[pos];\n+\n+\t\tif (!S_ISGITLINK(ce->ce_mode) ||\n+\t\t    (lstat(ce->name, &st) < 0) ||\n+\t\t    is_empty_dir(name))\n+\t\t\tcontinue;\n+\n+\t\tif (!submodule_uses_gitfile(name))\n+\t\t\terrs = error(_(\"submodule '%s' (or one of its nested \"\n+\t\t\t\t     \"submodules) uses a .git directory\\n\"\n+\t\t\t\t     \"(use 'rm -rf' if you really want to remove \"\n+\t\t\t\t     \"it including all of its history)\"), name);\n+\t}\n+\n+\treturn errs;\n+}\n+\n static int check_local_mod(unsigned char *head, int index_only)\n {\n \t/*\n@@ -37,7 +72,7 @@ static int check_local_mod(unsigned char *head, int index_only)\n \t\tstruct stat st;\n \t\tint pos;\n \t\tstruct cache_entry *ce;\n-\t\tconst char *name = list.name[i];\n+\t\tconst char *name = list.entry[i].name;\n \t\tunsigned char sha1[20];\n \t\tunsigned mode;\n \t\tint local_changes = 0;\n@@ -58,9 +93,10 @@ static int check_local_mod(unsigned char *head, int index_only)\n \t\t\t/* if a file was removed and it is now a\n \t\t\t * directory, that is the same as ENOENT as\n \t\t\t * far as git is concerned; we do not track\n-\t\t\t * directories.\n+\t\t\t * directories unless they are submodules.\n \t\t\t */\n-\t\t\tcontinue;\n+\t\t\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\t\t\tcontinue;\n \t\t}\n\n \t\t/*\n@@ -80,8 +116,11 @@ static int check_local_mod(unsigned char *head, int index_only)\n\n \t\t/*\n \t\t * Is the index different from the file in the work tree?\n+\t\t * If it's a submodule, is its work tree modified?\n \t\t */\n-\t\tif (ce_match_stat(ce, &st, 0))\n+\t\tif (ce_match_stat(ce, &st, 0) ||\n+\t\t    (S_ISGITLINK(ce->ce_mode) &&\n+\t\t     !ok_to_remove_submodule(ce->name)))\n \t\t\tlocal_changes = 1;\n\n \t\t/*\n@@ -115,10 +154,18 @@ static int check_local_mod(unsigned char *head, int index_only)\n \t\t\t\terrs = error(_(\"'%s' has changes staged in the index\\n\"\n \t\t\t\t\t     \"(use --cached to keep the file, \"\n \t\t\t\t\t     \"or -f to force removal)\"), name);\n-\t\t\tif (local_changes)\n-\t\t\t\terrs = error(_(\"'%s' has local modifications\\n\"\n-\t\t\t\t\t     \"(use --cached to keep the file, \"\n-\t\t\t\t\t     \"or -f to force removal)\"), name);\n+\t\t\tif (local_changes) {\n+\t\t\t\tif (S_ISGITLINK(ce->ce_mode) &&\n+\t\t\t\t    !submodule_uses_gitfile(name)) {\n+\t\t\t\t\terrs = error(_(\"submodule '%s' (or one of its nested \"\n+\t\t\t\t\t\t     \"submodules) uses a .git directory\\n\"\n+\t\t\t\t\t\t     \"(use 'rm -rf' if you really want to remove \"\n+\t\t\t\t\t\t     \"it including all of its history)\"), name);\n+\t\t\t\t} else\n+\t\t\t\t\terrs = error(_(\"'%s' has local modifications\\n\"\n+\t\t\t\t\t\t     \"(use --cached to keep the file, \"\n+\t\t\t\t\t\t     \"or -f to force removal)\"), name);\n+\t\t\t}\n \t\t}\n \t}\n \treturn errs;\n@@ -173,8 +220,9 @@ int cmd_rm(int argc, const char **argv, const char *prefix)\n \t\tstruct cache_entry *ce = active_cache[i];\n \t\tif (!match_pathspec(pathspec, ce->name, ce_namelen(ce), 0, seen))\n \t\t\tcontinue;\n-\t\tALLOC_GROW(list.name, list.nr + 1, list.alloc);\n-\t\tlist.name[list.nr++] = ce->name;\n+\t\tALLOC_GROW(list.entry, list.nr + 1, list.alloc);\n+\t\tlist.entry[list.nr].name = ce->name;\n+\t\tlist.entry[list.nr++].is_submodule = S_ISGITLINK(ce->ce_mode);\n \t}\n\n \tif (pathspec) {\n@@ -215,6 +263,9 @@ int cmd_rm(int argc, const char **argv, const char *prefix)\n \t\t\thashclr(sha1);\n \t\tif (check_local_mod(sha1, index_only))\n \t\t\texit(1);\n+\t} else if (!index_only) {\n+\t\tif (check_submodules_use_gitfiles())\n+\t\t\texit(1);\n \t}\n\n \t/*\n@@ -222,7 +273,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)\n \t * the index unless all of them succeed.\n \t */\n \tfor (i = 0; i < list.nr; i++) {\n-\t\tconst char *path = list.name[i];\n+\t\tconst char *path = list.entry[i].name;\n \t\tif (!quiet)\n \t\t\tprintf(\"rm '%s'\\n\", path);\n\n@@ -244,7 +295,25 @@ int cmd_rm(int argc, const char **argv, const char *prefix)\n \tif (!index_only) {\n \t\tint removed = 0;\n \t\tfor (i = 0; i < list.nr; i++) {\n-\t\t\tconst char *path = list.name[i];\n+\t\t\tconst char *path = list.entry[i].name;\n+\t\t\tif (list.entry[i].is_submodule) {\n+\t\t\t\tif (is_empty_dir(path)) {\n+\t\t\t\t\tif (!rmdir(path)) {\n+\t\t\t\t\t\tremoved = 1;\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\t\t\t\t} else {\n+\t\t\t\t\tstruct strbuf buf = STRBUF_INIT;\n+\t\t\t\t\tstrbuf_addstr(&buf, path);\n+\t\t\t\t\tif (!remove_dir_recursively(&buf, 0)) {\n+\t\t\t\t\t\tremoved = 1;\n+\t\t\t\t\t\tstrbuf_release(&buf);\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\t}\n+\t\t\t\t\tstrbuf_release(&buf);\n+\t\t\t\t\t/* Fallthrough and let remove_path() fail. */\n+\t\t\t\t}\n+\t\t\t}\n \t\t\tif (!remove_path(path)) {\n \t\t\t\tremoved = 1;\n \t\t\t\tcontinue;\ndiff --git a/submodule.c b/submodule.c\nindex 19dc6a6..acb2fe0 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -758,6 +758,86 @@ unsigned is_submodule_modified(const char *path, int ignore_untracked)\n \treturn dirty_submodule;\n }\n\n+int submodule_uses_gitfile(const char *path)\n+{\n+\tstruct child_process cp;\n+\tconst char *argv[] = {\n+\t\t\"submodule\",\n+\t\t\"foreach\",\n+\t\t\"--quiet\",\n+\t\t\"--recursive\",\n+\t\t\"test -f .git\",\n+\t\tNULL,\n+\t};\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tconst char *git_dir;\n+\n+\tstrbuf_addf(&buf, \"%s/.git\", path);\n+\tgit_dir = read_gitfile(buf.buf);\n+\tif (!git_dir) {\n+\t\tstrbuf_release(&buf);\n+\t\treturn 0;\n+\t}\n+\tstrbuf_release(&buf);\n+\n+\t/* Now test that all nested submodules use a gitfile too */\n+\tmemset(&cp, 0, sizeof(cp));\n+\tcp.argv = argv;\n+\tcp.env = local_repo_env;\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.no_stderr = 1;\n+\tcp.no_stdout = 1;\n+\tcp.dir = path;\n+\tif (run_command(&cp))\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n+int ok_to_remove_submodule(const char *path)\n+{\n+\tstruct stat st;\n+\tssize_t len;\n+\tstruct child_process cp;\n+\tconst char *argv[] = {\n+\t\t\"status\",\n+\t\t\"--porcelain\",\n+\t\t\"-u\",\n+\t\t\"--ignore-submodules=none\",\n+\t\tNULL,\n+\t};\n+\tstruct strbuf buf = STRBUF_INIT;\n+\tint ok_to_remove = 1;\n+\n+\tif ((lstat(path, &st) < 0) || is_empty_dir(path))\n+\t\treturn 1;\n+\n+\tif (!submodule_uses_gitfile(path))\n+\t\treturn 0;\n+\n+\tmemset(&cp, 0, sizeof(cp));\n+\tcp.argv = argv;\n+\tcp.env = local_repo_env;\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\tcp.out = -1;\n+\tcp.dir = path;\n+\tif (start_command(&cp))\n+\t\tdie(\"Could not run 'git status --porcelain -uall --ignore-submodules=none' in submodule %s\", path);\n+\n+\tlen = strbuf_read(&buf, cp.out, 1024);\n+\tif (len > 2)\n+\t\tok_to_remove = 0;\n+\tclose(cp.out);\n+\n+\tif (finish_command(&cp))\n+\t\tdie(\"'git status --porcelain -uall --ignore-submodules=none' failed in submodule %s\", path);\n+\n+\tstrbuf_release(&buf);\n+\treturn ok_to_remove;\n+}\n+\n static int find_first_merges(struct object_array *result, const char *path,\n \t\tstruct commit *a, struct commit *b)\n {\ndiff --git a/submodule.h b/submodule.h\nindex e105b0e..9c0f6a4 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -27,6 +27,8 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\t       const char *prefix, int command_line_option,\n \t\t\t       int quiet);\n unsigned is_submodule_modified(const char *path, int ignore_untracked);\n+int submodule_uses_gitfile(const char *path);\n+int ok_to_remove_submodule(const char *path);\n int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],\n \t\t    const unsigned char a[20], const unsigned char b[20], int search);\n int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,\ndiff --git a/t/t3600-rm.sh b/t/t3600-rm.sh\nindex 9fd28bc..d55f2fb 100755\n--- a/t/t3600-rm.sh\n+++ b/t/t3600-rm.sh\n@@ -262,4 +262,214 @@ test_expect_success 'rm removes subdirectories recursively' '\n \t! test -d dir\n '\n\n+cat >expect <<EOF\n+D  submod\n+EOF\n+\n+cat >expect.modified <<EOF\n+ M submod\n+EOF\n+\n+test_expect_success 'rm removes empty submodules from work tree' '\n+\tmkdir submod &&\n+\tgit update-index --add --cacheinfo 160000 $(git rev-parse HEAD) submod &&\n+\tgit config -f .gitmodules submodule.sub.url ./. &&\n+\tgit config -f .gitmodules submodule.sub.path submod &&\n+\tgit submodule init &&\n+\tgit add .gitmodules &&\n+\tgit commit -m \"add submodule\" &&\n+\tgit rm submod &&\n+\ttest ! -e submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm removes removed submodule from index' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\trm -rf submod &&\n+\tgit rm submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm removes work tree of unmodified submodules' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\tgit rm submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated submodule with different HEAD fails unless forced' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\t(cd submod &&\n+\t\tgit checkout HEAD^\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -f submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect.modified actual &&\n+\tgit rm -f submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated submodule with modifications fails unless forced' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\t(cd submod &&\n+\t\techo X >empty\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -f submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect.modified actual &&\n+\tgit rm -f submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated submodule with untracked files fails unless forced' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\t(cd submod &&\n+\t\techo X >untracked\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -f submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect.modified actual &&\n+\tgit rm -f submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated submodule with a .git directory fails even when forced' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\t(cd submod &&\n+\t\trm .git &&\n+\t\tcp -a ../.git/modules/sub .git &&\n+\t\tGIT_WORK_TREE=. git config --unset core.worktree\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -d submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\t! test -s actual &&\n+\ttest_must_fail git rm -f submod &&\n+\ttest -d submod &&\n+\ttest -d submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\t! test -s actual &&\n+\trm -rf submod\n+'\n+\n+cat >expect.deepmodified <<EOF\n+ M submod/subsubmod\n+EOF\n+\n+test_expect_success 'setup subsubmodule' '\n+\tgit reset --hard &&\n+\tgit submodule update &&\n+\t(cd submod &&\n+\t\tgit update-index --add --cacheinfo 160000 $(git rev-parse HEAD) subsubmod &&\n+\t\tgit config -f .gitmodules submodule.sub.url ../. &&\n+\t\tgit config -f .gitmodules submodule.sub.path subsubmod &&\n+\t\tgit submodule init &&\n+\t\tgit add .gitmodules &&\n+\t\tgit commit -m \"add subsubmodule\" &&\n+\t\tgit submodule update subsubmod\n+\t) &&\n+\tgit commit -a -m \"added deep submodule\"\n+'\n+\n+test_expect_success 'rm recursively removes work tree of unmodified submodules' '\n+\tgit rm submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated nested submodule with different nested HEAD fails unless forced' '\n+\tgit reset --hard &&\n+\tgit submodule update --recursive &&\n+\t(cd submod/subsubmod &&\n+\t\tgit checkout HEAD^\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -f submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect.modified actual &&\n+\tgit rm -f submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated nested submodule with nested modifications fails unless forced' '\n+\tgit reset --hard &&\n+\tgit submodule update --recursive &&\n+\t(cd submod/subsubmod &&\n+\t\techo X >empty\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -f submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect.modified actual &&\n+\tgit rm -f submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated nested submodule with nested untracked files fails unless forced' '\n+\tgit reset --hard &&\n+\tgit submodule update --recursive &&\n+\t(cd submod/subsubmod &&\n+\t\techo X >untracked\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -f submod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect.modified actual &&\n+\tgit rm -f submod &&\n+\ttest ! -d submod &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'rm of a populated nested submodule with a nested .git directory fails even when forced' '\n+\tgit reset --hard &&\n+\tgit submodule update --recursive &&\n+\t(cd submod/subsubmod &&\n+\t\trm .git &&\n+\t\tcp -a ../../.git/modules/sub/modules/sub .git &&\n+\t\tGIT_WORK_TREE=. git config --unset core.worktree\n+\t) &&\n+\ttest_must_fail git rm submod &&\n+\ttest -d submod &&\n+\ttest -d submod/subsubmod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\t! test -s actual &&\n+\ttest_must_fail git rm -f submod &&\n+\ttest -d submod &&\n+\ttest -d submod/subsubmod/.git &&\n+\tgit status -s -uno --ignore-submodules=none > actual &&\n+\t! test -s actual &&\n+\trm -rf submod\n+'\n+\n test_done\n-- \n1.7.12.85.g08e971b\n"},{"id":"197932","messageId":"7v1uisqcef.fsf@alter.siamese.dyndns.org","threadId":"31366","inReplyTo":"503BCD91.6080107@web.de","subject":"Re: [RFC v2 PATCH] Teach rm to remove submodules unless they contain a git directory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-08-27T20:59:36Z","receivedAt":"2012-08-27T20:59:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann <Jens.Lehmann@web.de> writes:\n\n> diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt\n> index 5d31860..3c76f9c 100644\n> --- a/Documentation/git-rm.txt\n> +++ b/Documentation/git-rm.txt\n> @@ -107,6 +107,21 @@ as well as modifications of existing paths.\n>  Typically you would first remove all tracked files from the working\n>  tree using this command:\n>\n> +Submodules\n> +~~~~~~~~~~~~~~~~~~~~\n\nYou need to match the underline to the text if you want to make this\na heading.\n\n> diff --git a/builtin/rm.c b/builtin/rm.c\n> index 90c8a50..cb927a8 100644\n> --- a/builtin/rm.c\n> +++ b/builtin/rm.c\n> @@ -9,6 +9,7 @@\n>  #include \"cache-tree.h\"\n>  #include \"tree-walk.h\"\n>  #include \"parse-options.h\"\n> +#include \"submodule.h\"\n>\n>  static const char * const builtin_rm_usage[] = {\n>  \t\"git rm [options] [--] <file>...\",\n> @@ -17,9 +18,43 @@ static const char * const builtin_rm_usage[] = {\n>\n>  static struct {\n>  \tint nr, alloc;\n> -\tconst char **name;\n> +\tstruct {\n> +\t\tconst char *name;\n> +\t\tchar is_submodule;\n> +\t} *entry;\n>  } list;\n>\n> +static int check_submodules_use_gitfiles()\n\nstatic int check_submodules_use_gitfiles(void)\n\n> +{\n> +\tint i;\n> +\tint errs = 0;\n> +\n> +\tfor (i = 0; i < list.nr; i++) {\n> +\t\tconst char *name = list.entry[i].name;\n> +\t\tint pos;\n> +\t\tstruct cache_entry *ce;\n> +\t\tstruct stat st;\n> +\n> +\t\tpos = cache_name_pos(name, strlen(name));\n> +\t\tif (pos < 0)\n> +\t\t\tcontinue; /* ignore unmerged entry */\n\nWould this cause \"git rm -f path\" for an unmerged submodule bypass\nthe safety check?\n\nWith or without this patch, check_local_mod() will allow you to\nremove unmerged entry and the file in the working tree, and that is\nperfectly fine for a regular file or a symlink (as the path is\ninvolved in a conflicted merge (or other mergy operation), and its\nchange from the HEAD can only come from that merge, because we would\nnot let merge touch a path and leave its index entry unmerged if the\npath has local changes in the first place).  Resolving the merge as\na removal at the index level for a submodule is also fine in such a\ncase, but don't you want to still keep the submodule working tree if\nit has its sole copy of the repository?  And as far as I can tell,\nthis function is the only thing that protects the user in such a\nsituation.\n\n> +\t\tce = active_cache[pos];\n> +\n> +\t\tif (!S_ISGITLINK(ce->ce_mode) ||\n> +\t\t    (lstat(ce->name, &st) < 0) ||\n> +\t\t    is_empty_dir(name))\n> +\t\t\tcontinue;\n> +\n> +\t\tif (!submodule_uses_gitfile(name))\n> +\t\t\terrs = error(_(\"submodule '%s' (or one of its nested \"\n> +\t\t\t\t     \"submodules) uses a .git directory\\n\"\n> +\t\t\t\t     \"(use 'rm -rf' if you really want to remove \"\n> +\t\t\t\t     \"it including all of its history)\"), name);\n> +\t}\n> +\n> +\treturn errs;\n> +}\n\nThe call to this function comes at the very end and gives us yes/no\nfor the entire set of paths.  After getting this error for one\nsubmodule and bunch of other non-submodule paths, what is the\nprocedure for the user to remove it that we want to recommend in our\ndocumentation?  Would it go like this?\n\n\t$ git rm path1 path2 sub path3\n\t... get the above error ...\n\t$ git submodule --to-gitfile sub\n        $ rm -fr sub\n        $ git rm sub\n        ... then finally ...\n        $ git rm path1 path2 path3\n\nThis is not a complaint about the error message above, by the way.\n\n> @@ -37,7 +72,7 @@ static int check_local_mod(unsigned char *head, int index_only)\n>  \t\tstruct stat st;\n>  \t\tint pos;\n>  \t\tstruct cache_entry *ce;\n> -\t\tconst char *name = list.name[i];\n> +\t\tconst char *name = list.entry[i].name;\n>  \t\tunsigned char sha1[20];\n>  \t\tunsigned mode;\n>  \t\tint local_changes = 0;\n> @@ -58,9 +93,10 @@ static int check_local_mod(unsigned char *head, int index_only)\n>  \t\t\t/* if a file was removed and it is now a\n>  \t\t\t * directory, that is the same as ENOENT as\n>  \t\t\t * far as git is concerned; we do not track\n> -\t\t\t * directories.\n> +\t\t\t * directories unless they are submodules.\n>  \t\t\t */\n> -\t\t\tcontinue;\n> +\t\t\tif (!S_ISGITLINK(ce->ce_mode))\n> +\t\t\t\tcontinue;\n>  \t\t}\n>\n>  \t\t/*\n> @@ -80,8 +116,11 @@ static int check_local_mod(unsigned char *head, int index_only)\n>\n>  \t\t/*\n>  \t\t * Is the index different from the file in the work tree?\n> +\t\t * If it's a submodule, is its work tree modified?\n>  \t\t */\n> -\t\tif (ce_match_stat(ce, &st, 0))\n> +\t\tif (ce_match_stat(ce, &st, 0) ||\n> +\t\t    (S_ISGITLINK(ce->ce_mode) &&\n> +\t\t     !ok_to_remove_submodule(ce->name)))\n>  \t\t\tlocal_changes = 1;\n\nAs noted before, because we also skip these \"does it match the\nindex?  does it match the HEAD?\" checks for unmerged paths in this\nfunction, a submodule that has local changes or new files is\neligible for removal during a conflicted merge.  I have a feeling\nthat this should be tightened a bit; wouldn't we want to check at\nleast in the checked out version (i.e. stage #2 in the index) if the\npath were a submodule, even if we are in the middle of a conflicted\nmerge?  After all, the top level merge shouldn't have touched the\nsubmodule working tree, so the local modes and new files must have\ncome from the end user action that was done _before_ the conflicted\nmerge started, and not expendable, no?\n\n\n\n-- >8 --\n\n Documentation/git-rm.txt | 6 +++---\n 1 file changed, 3 insertions(+), 3 deletions(-)\n\ndiff --git i/Documentation/git-rm.txt w/Documentation/git-rm.txt\nindex 3c76f9c..882cb11 100644\n--- i/Documentation/git-rm.txt\n+++ w/Documentation/git-rm.txt\n@@ -108,13 +108,13 @@ Typically you would first remove all tracked files from the working\n tree using this command:\n \n Submodules\n-~~~~~~~~~~~~~~~~~~~~\n+~~~~~~~~~~\n Only submodules using a gitfile (which means they were cloned\n with a git version 1.7.8 or newer) will be removed from the work\n tree, as their repository lives inside the .git directory of the\n superproject. If a submodule (or one of those nested inside it)\n-still use a .git directory, `git rm` will fail - no matter if forced\n-or not - to protect the submodules history.\n+still uses a .git directory, `git rm` will fail - no matter if forced\n+or not - to protect the submodule's history.\n \n A submodule is considered up-to-date when the HEAD is the same as\n recorded in the index, no tracked files are modified and no untracked\n"},{"id":"198008","messageId":"503D0E09.8010605@web.de","threadId":"31366","inReplyTo":"7v1uisqcef.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC v2 PATCH] Teach rm to remove submodules unless they contain a git directory","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-08-28T18:29:29Z","receivedAt":"2012-08-28T18:29:29Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 27.08.2012 22:59, schrieb Junio C Hamano:\n> Jens Lehmann <Jens.Lehmann@web.de> writes:\n>> +{\n>> +\tint i;\n>> +\tint errs = 0;\n>> +\n>> +\tfor (i = 0; i < list.nr; i++) {\n>> +\t\tconst char *name = list.entry[i].name;\n>> +\t\tint pos;\n>> +\t\tstruct cache_entry *ce;\n>> +\t\tstruct stat st;\n>> +\n>> +\t\tpos = cache_name_pos(name, strlen(name));\n>> +\t\tif (pos < 0)\n>> +\t\t\tcontinue; /* ignore unmerged entry */\n> \n> Would this cause \"git rm -f path\" for an unmerged submodule bypass\n> the safety check?\n\nOops, thanks for spotting that. So replacing the \"continue;\" with\n\"pos = -pos-1;\" should do the trick here, right? Will add some\ntests for unmerged submodules ...\n\n>> +\t\tce = active_cache[pos];\n>> +\n>> +\t\tif (!S_ISGITLINK(ce->ce_mode) ||\n>> +\t\t    (lstat(ce->name, &st) < 0) ||\n>> +\t\t    is_empty_dir(name))\n>> +\t\t\tcontinue;\n>> +\n>> +\t\tif (!submodule_uses_gitfile(name))\n>> +\t\t\terrs = error(_(\"submodule '%s' (or one of its nested \"\n>> +\t\t\t\t     \"submodules) uses a .git directory\\n\"\n>> +\t\t\t\t     \"(use 'rm -rf' if you really want to remove \"\n>> +\t\t\t\t     \"it including all of its history)\"), name);\n>> +\t}\n>> +\n>> +\treturn errs;\n>> +}\n> \n> The call to this function comes at the very end and gives us yes/no\n> for the entire set of paths.  After getting this error for one\n> submodule and bunch of other non-submodule paths, what is the\n> procedure for the user to remove it that we want to recommend in our\n> documentation?  Would it go like this?\n> \n> \t$ git rm path1 path2 sub path3\n> \t... get the above error ...\n> \t$ git submodule --to-gitfile sub\n>         $ rm -fr sub\n>         $ git rm sub\n>         ... then finally ...\n>         $ git rm path1 path2 path3\n\nWith current git I'd recommend:\n\n \t$ git rm path1 path2 sub path3\n \t... get the above error ...\n        $ rm -fr sub\n \t... try again ...\n        $ git rm path1 path2 sub path3\n\nMaybe I should add the hint to repeat the git rm after removing the\nsubmodule to the error output?\n\nOnce we implemented \"git submodule --to-gitfile\" it could be used\ninstead of \"rm -fr sub\" to preserve the submodule's repo if the user\nwants to.\n\nBTW: I added the same message twice, here for the forced case and in\ncheck_local_mod() when not forced. Is there a recommended way to assign\na localized message to a static variable, so I could define it only once\nand reuse it?\n\n>> @@ -80,8 +116,11 @@ static int check_local_mod(unsigned char *head, int index_only)\n>>\n>>  \t\t/*\n>>  \t\t * Is the index different from the file in the work tree?\n>> +\t\t * If it's a submodule, is its work tree modified?\n>>  \t\t */\n>> -\t\tif (ce_match_stat(ce, &st, 0))\n>> +\t\tif (ce_match_stat(ce, &st, 0) ||\n>> +\t\t    (S_ISGITLINK(ce->ce_mode) &&\n>> +\t\t     !ok_to_remove_submodule(ce->name)))\n>>  \t\t\tlocal_changes = 1;\n> \n> As noted before, because we also skip these \"does it match the\n> index?  does it match the HEAD?\" checks for unmerged paths in this\n> function, a submodule that has local changes or new files is\n> eligible for removal during a conflicted merge.  I have a feeling\n> that this should be tightened a bit; wouldn't we want to check at\n> least in the checked out version (i.e. stage #2 in the index) if the\n> path were a submodule, even if we are in the middle of a conflicted\n> merge?  After all, the top level merge shouldn't have touched the\n> submodule working tree, so the local modes and new files must have\n> come from the end user action that was done _before_ the conflicted\n> merge started, and not expendable, no?\n\nRight, I'll change that.\n"}]}