{"thread":{"id":"58187","subject":"[PATCH v1 1/7] t7002: add tests for moving from in-cone to out-of-cone","startedAt":"2022-07-19T14:10:28Z","lastAt":"2022-08-16T15:52:09Z","messageCount":61,"participants":["Shaoxuan Yuan","Ævar Arnfjörð Bjarmason","Derrick Stolee","Victoria Dye","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":7},"messages":[{"id":"459358","messageId":"20220719132809.409247-2-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 1/7] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:03Z","receivedAt":"2022-07-19T14:10:28Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test that user can move an in-cone \nto out-of-cone when --sparse is supplied.\n\nSuch can be either clean or dirty, and moving it results in\ndifferent behaviors:\n\nA clean move should move the to the , both in the\nworking tree and the index, then remove the resulted path from the\nworking tree, and turn on its CE_SKIP_WORKTREE bit.\n\nA dirty move should move the to the , both in the\nworking tree and the index, but should *not* remove the resulted path\nfrom the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\nInstead advise user to \"git add\" this path and run \"git sparse-checkout\nreapply\" to re-sparsify that path.\n\nAlso make sure that if exists in the index (existing\ncheck for if is in the worktree is not enough in\nin-to-out moves), warn user against the overwrite. And Git should force\nthe overwrite when supplied with -f or --force.\n\nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 148 +++++++++++++++++++++++++++++++++-\n 1 file changed, 147 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 71fe29690f..c27fcdbfd0 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -28,12 +28,25 @@ test_expect_success 'setup' \"\n \tupdated in the index:\n \tEOF\n \n-\tcat >sparse_hint <<-EOF\n+\tcat >sparse_hint <<-EOF &&\n \thint: If you intend to update such entries, try one of the following:\n \thint: * Use the --sparse option.\n \thint: * Disable or modify the sparsity rules.\n \thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n \tEOF\n+\n+\tcat >dirty_error_header <<-EOF &&\n+\tThe following paths have been moved outside the\n+\tsparse-checkout definition but are not sparse due to local\n+\tmodifications.\n+\tEOF\n+\n+\tcat >dirty_hint <<-EOF\n+\thint: To correct the sparsity of these paths, do the following:\n+\thint: * Use \\\"git add --sparse \\\" to update the index\n+\thint: * Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\n+\thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n+\tEOF\n \"\n \n test_expect_success 'mv refuses to move sparse-to-sparse' '\n@@ -290,4 +303,137 @@ test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n \n+test_expect_failure 'move clean path from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv sub/d folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/d folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/d &&\n+\ttest_path_is_missing folder1/d &&\n+\tgit ls-files -t >actual &&\n+\t! grep -x \"H sub/d\" actual &&\n+\tgrep -x \"S folder1/d\" actual\n+'\n+\n+test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"sub/file1 overwrite\" >sub/file1 &&\n+\tgit add sub/file1 &&\n+\n+\ttest_must_fail git mv sub/file1 folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/file1\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_must_fail git mv --sparse sub/file1 folder1 2>stderr &&\n+\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n+\t>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse -f sub/file1 folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/file1 &&\n+\ttest_path_is_missing folder1/file1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep -x \"H sub/file1\" actual &&\n+\tgrep -x \"S folder1/file1\" actual &&\n+\n+\t# compare file content before move and after move\n+\techo \"sub/file1 overwrite\" >expect &&\n+\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n+\tgit cat-file blob $(cat oid) >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"modified\" >>sub/d &&\n+\n+\ttest_must_fail git mv sub/d folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/d folder1 2>stderr &&\n+\tcat dirty_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat dirty_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_path_is_missing sub/d &&\n+\ttest_path_is_file folder1/d &&\n+\tgit ls-files -t >actual &&\n+\t! grep -x \"H sub/d\" actual &&\n+\tgrep -x \"H folder1/d\" actual\n+'\n+\n+test_expect_failure 'move dir from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/dir folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/dir &&\n+\ttest_path_is_missing folder1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep -x \"H sub/dir/e\" actual &&\n+\tgrep -x \"S folder1/dir/e\" actual\n+'\n+\n+test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\ttouch sub/dir/e2 sub/dir/e3 &&\n+\tgit add sub/dir/e2 sub/dir/e3 &&\n+\techo \"modified\" >>sub/dir/e2 &&\n+\techo \"modified\" >>sub/dir/e3 &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\techo \"folder1/dir/e2\" >>expect &&\n+\techo \"folder1/dir/e3\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/dir folder1 2>stderr &&\n+\tcat dirty_error_header >expect &&\n+\techo \"folder1/dir/e2\" >>expect &&\n+\techo \"folder1/dir/e3\" >>expect &&\n+\tcat dirty_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_path_is_missing sub/dir &&\n+\ttest_path_is_missing folder1/dir/e &&\n+\ttest_path_is_file folder1/dir/e2 &&\n+\ttest_path_is_file folder1/dir/e3 &&\n+\tgit ls-files -t >actual &&\n+\t! grep -x \"H sub/dir/e\" actual &&\n+\t! grep -x \"H sub/dir/e2\" actual &&\n+\t! grep -x \"H sub/dir/e3\" actual &&\n+\tgrep -x \"S folder1/dir/e\" actual &&\n+\tgrep -x \"H folder1/dir/e2\" actual &&\n+\tgrep -x \"H folder1/dir/e3\" actual\n+'\n+\n test_done\n-- \n2.37.0\n\n"},{"id":"459359","messageId":"20220719132809.409247-3-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 2/7] mv: add documentation for check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:04Z","receivedAt":"2022-07-19T14:10:31Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Using check_dir_in_index without checking if the directory is on-disk\ncould get a false positive for partially sparsified directory.\n\nAdd a note in the documentation to warn potential user.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 4729bb1a1a..c8b9069db8 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -132,6 +132,11 @@ static int index_range_of_same_dir(const char *src, int length,\n * Return 0 if such directory exist (i.e. with any of its contained files not\n * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n * Return 1 otherwise.\n+ *\n+ * Note: *always* check the directory is not on-disk before this function\n+ * (i.e. using lstat());\n+ * otherwise it may return a false positive for a partially sparsified\n+ * directory.\n */\n static int check_dir_in_index(const char *name)\n {\n-- \n2.37.0\n\n"},{"id":"459360","messageId":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":null,"subject":"[PATCH v1 0/7] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:02Z","receivedAt":"2022-07-19T14:10:32Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"This is a sister series to the previous \"from out-of-cone to in-cone\" [1]\nseries. This series is trying to make the opposite operation possible for\n'mv', namely move , which is in-cone, to , which is\nout-of-cone.\n\nOther than the main task, there are also some minor fixes done.\n\n[1] https://lore.kernel.org/git/20220331091755.385961-1-shaoxuan.yuan02@gmail.com/\n\nShaoxuan Yuan (7):\n t7002: add tests for moving from in-cone to out-of-cone\n mv: add documentation for check_dir_in_index()\n mv: free the *with_slash in check_dir_in_index()\n mv: check if is a SKIP_WORKTREE_DIR\n mv: remove BOTH from enum update_mode\n mv: from in-cone to out-of-cone\n mv: check overwrite for in-to-out move\n\n advice.c | 19 +++++\n advice.h | 1 +\n builtin/mv.c | 100 +++++++++++++++++++----\n t/t7002-mv-sparse-checkout.sh | 148 +++++++++++++++++++++++++++++++++-\n 4 files changed, 250 insertions(+), 18 deletions(-)\n\n\nbase-commit: 71a8fab31b70c417e8f5b5f716581f89955a7082\n-- \n2.37.0\n\n"},{"id":"459361","messageId":"20220719132809.409247-4-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 3/7] mv: free the *with_slash in check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:05Z","receivedAt":"2022-07-19T14:10:35Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"*with_slash may be a malloc'd pointer, and when it is, free it.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 13 +++++++++----\n 1 file changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex c8b9069db8..23a297d6b8 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -140,6 +140,7 @@ static int index_range_of_same_dir(const char *src, int length,\n */\n static int check_dir_in_index(const char *name)\n {\n+\tint ret = 1;\n \tconst char *with_slash = add_slash(name);\n \tint length = strlen(with_slash);\n \n@@ -149,14 +150,18 @@ static int check_dir_in_index(const char *name)\n \tif (pos < 0) {\n \t\tpos = -pos - 1;\n \t\tif (pos >= the_index.cache_nr)\n-\t\t\treturn 1;\n+\t\t\tgoto free_return;\n \t\tce = active_cache[pos];\n \t\tif (strncmp(with_slash, ce->name, length))\n-\t\t\treturn 1;\n+\t\t\tgoto free_return;\n \t\tif (ce_skip_worktree(ce))\n-\t\t\treturn 0;\n+\t\t\tret = 0;\n \t}\n-\treturn 1;\n+\n+free_return:\n+\tif (with_slash != name)\n+\t\tfree((char *)with_slash);\n+\treturn ret;\n }\n \n int cmd_mv(int argc, const char **argv, const char *prefix)\n-- \n2.37.0\n\n"},{"id":"459362","messageId":"20220719132809.409247-6-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 5/7] mv: remove BOTH from enum update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:07Z","receivedAt":"2022-07-19T14:10:37Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Since BOTH is not practically used anywhere in the code and its meaning\nis unclear, remove it.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 2e9d577227..d05914a233 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -21,7 +21,6 @@ static const char * const builtin_mv_usage[] = {\n };\n \n enum update_mode {\n-\tBOTH = 0,\n \tWORKING_DIRECTORY = (1 << 1),\n \tINDEX = (1 << 2),\n \tSPARSE = (1 << 3),\n-- \n2.37.0\n\n"},{"id":"459363","messageId":"20220719132809.409247-5-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 4/7] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:06Z","receivedAt":"2022-07-19T14:10:38Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, is assumed to be in the working tree. If it is\nnot found as a directory, then it is determined to be either a regular file\npath, or error out if used under the second form (move into a directory)\nof 'git-mv'. Such behavior is not ideal, mainly because Git does not\nlook into the index for , which could potentially be a\nSKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\nin-cone to out-of-cone\" patch.\n\nChange the logic so that Git first check if is a directory\nwith all its contents sparsified (a SKIP_WORKTREE_DIR). If yes,\nthen treat as a directory exists in the working tree, and\nthus using the second form of 'git-mv', i.e. move into this\n, and mark as a SKIP_WORKTREE_DIR.\nIf no, continue the original checking logic.\n\nAlso add a `dst_w_slash` to reuse the result from `add_slash()`, which\nwas everywhere and can be simplified.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 19 ++++++++++++++-----\n 1 file changed, 14 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 23a297d6b8..2e9d577227 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -178,7 +178,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n-\tenum update_mode *modes;\n+\tconst char *dst_w_slash;\n+\tenum update_mode *modes, dst_mode = 0;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n@@ -208,6 +209,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tif (argc == 1 && is_directory(argv[0]) && !is_directory(argv[1]))\n \t\tflags = 0;\n \tdest_path = internal_prefix_pathspec(prefix, argv + argc, 1, flags);\n+\tdst_w_slash = add_slash(dest_path[0]);\n \tsubmodule_gitfile = xcalloc(argc, sizeof(char *));\n \n \tif (dest_path[0][0] == '\\0')\n@@ -215,13 +217,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n \telse if (!lstat(dest_path[0], &st) &&\n \t\t\tS_ISDIR(st.st_mode)) {\n-\t\tdest_path[0] = add_slash(dest_path[0]);\n-\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n+\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n \t} else {\n-\t\tif (argc != 1)\n+\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n+\t\t !check_dir_in_index(dst_w_slash)) {\n+\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n+\t\t\tdst_mode |= SKIP_WORKTREE_DIR;\n+\t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n-\t\tdestination = dest_path;\n+\t\t} else {\n+\t\t\tdestination = dest_path;\n+\t\t}\n \t}\n+\tif (dst_w_slash != dest_path[0])\n+\t\tfree((char *)dst_w_slash);\n \n \t/* Checking */\n \tfor (i = 0; i < argc; i++) {\n-- \n2.37.0\n\n"},{"id":"459364","messageId":"20220719132809.409247-7-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 6/7] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:08Z","receivedAt":"2022-07-19T14:10:39Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving an in-cone to an out-of-cone \nwas not possible, mainly because such is a directory that\nis not present in the working tree.\n\nChange the behavior so that we can move an in-cone to\nout-of-cone when --sparse is supplied.\n\nSuch can be either clean or dirty, and moving it results in\ndifferent behaviors:\n\nA clean move should move the to the , both in the\nworking tree and the index, then remove the resulted path from the\nworking tree, and turn on its CE_SKIP_WORKTREE bit.\n\nA dirty move should move the to the , both in the\nworking tree and the index, but should *not* remove the resulted path\nfrom the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\nInstead advise user to \"git add\" this path and run \"git sparse-checkout\nreapply\" to re-sparsify that path.\n\nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n advice.c | 19 +++++++++++++\n advice.h | 1 +\n builtin/mv.c | 50 ++++++++++++++++++++++++++++++-----\n t/t7002-mv-sparse-checkout.sh | 8 +++---\n 4 files changed, 67 insertions(+), 11 deletions(-)\n\ndiff --git a/advice.c b/advice.c\nindex 6fda9edbc2..fd18968943 100644\n--- a/advice.c\n+++ b/advice.c\n@@ -261,3 +261,22 @@ void detach_advice(const char *new_name)\n \n \tfprintf(stderr, fmt, new_name);\n }\n+\n+void advise_on_moving_dirty_path(struct string_list *pathspec_list)\n+{\n+\tstruct string_list_item *item;\n+\n+\tif (!pathspec_list->nr)\n+\t\treturn;\n+\n+\tfprintf(stderr, _(\"The following paths have been moved outside the\\n\"\n+\t\t\t \"sparse-checkout definition but are not sparse due to local\\n\"\n+\t\t\t \"modifications.\\n\"));\n+\tfor_each_string_list_item(item, pathspec_list)\n+\t\tfprintf(stderr, \"%s\\n\", item->string);\n+\n+\tadvise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,\n+\t\t\t _(\"To correct the sparsity of these paths, do the following:\\n\"\n+\t\t\t \"* Use \\\"git add --sparse \\\" to update the index\\n\"\n+\t\t\t \"* Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\"));\n+}\ndiff --git a/advice.h b/advice.h\nindex 7ddc6cbc1a..07e0f76833 100644\n--- a/advice.h\n+++ b/advice.h\n@@ -74,5 +74,6 @@ void NORETURN die_conclude_merge(void);\n void NORETURN die_ff_impossible(void);\n void advise_on_updating_sparse_paths(struct string_list *pathspec_list);\n void detach_advice(const char *new_name);\n+void advise_on_moving_dirty_path(struct string_list *pathspec_list);\n \n #endif /* ADVICE_H */\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex d05914a233..d35994c443 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -184,6 +184,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tstruct lock_file lock_file = LOCK_INIT;\n \tstruct cache_entry *ce;\n \tstruct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;\n+\tstruct string_list dirty_paths = STRING_LIST_INIT_NODUP;\n \n \tgit_config(git_default_config, NULL);\n \n@@ -414,6 +415,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tint up_to_date = 0;\n \t\tstruct checkout state = CHECKOUT_INIT;\n \t\tstate.istate = &the_index;\n \n@@ -424,6 +426,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (show_only)\n \t\t\tcontinue;\n \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n+\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n \t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n@@ -443,20 +446,52 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n+\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n+\t\t\tup_to_date = !ce_modified(active_cache[pos], &st, 0);\n \t\trename_cache_entry_at(pos, dst);\n \n-\t\tif ((mode & SPARSE) &&\n-\t\t (path_in_sparse_checkout(dst, &the_index))) {\n-\t\t\tint dst_pos;\n+\t\tif (ignore_sparse &&\n+\t\t core_apply_sparse_checkout &&\n+\t\t core_sparse_checkout_cone) {\n \n-\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n-\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n+\t\t\t/* from out-of-cone to in-cone */\n+\t\t\tif ((mode & SPARSE) &&\n+\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n \n-\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n-\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n+\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n+\n+\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n+\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\t/* from in-cone to out-of-cone */\n+\t\t\tif ((dst_mode & SKIP_WORKTREE_DIR) &&\n+\t\t\t !(mode & SPARSE) &&\n+\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n+\t\t\t\tchar *src_dir = dirname(xstrdup(src));\n+\n+\t\t\t\tif (up_to_date) {\n+\t\t\t\t\tdst_ce->ce_flags |= CE_SKIP_WORKTREE;\n+\t\t\t\t\tunlink_or_warn(src);\n+\t\t\t\t} else {\n+\t\t\t\t\tstring_list_append(&dirty_paths, dst);\n+\t\t\t\t\tsafe_create_leading_directories(xstrdup(dst));\n+\t\t\t\t\trename(src, dst);\n+\t\t\t\t}\n+\t\t\t\tif ((mode & INDEX) && is_empty_dir(src_dir))\n+\t\t\t\t\trmdir_or_warn(src_dir);\n+\t\t\t}\n \t\t}\n \t}\n \n+\tif (dirty_paths.nr)\n+\t\tadvise_on_moving_dirty_path(&dirty_paths);\n+\n \tif (gitmodules_modified)\n \t\tstage_updated_gitmodules(&the_index);\n \n@@ -465,6 +500,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdie(_(\"Unable to write new index file\"));\n \n \tstring_list_clear(&src_for_dst, 0);\n+\tstring_list_clear(&dirty_paths, 0);\n \tUNLEAK(source);\n \tUNLEAK(dest_path);\n \tfree(submodule_gitfile);\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex c27fcdbfd0..dafe15b9cf 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -303,7 +303,7 @@ test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n \n-test_expect_failure 'move clean path from in-cone to out-of-cone' '\n+test_expect_success 'move clean path from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -356,7 +356,7 @@ test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n+test_expect_success 'move dirty path from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"modified\" >>sub/d &&\n@@ -380,7 +380,7 @@ test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n \tgrep -x \"H folder1/d\" actual\n '\n \n-test_expect_failure 'move dir from in-cone to out-of-cone' '\n+test_expect_success 'move dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -400,7 +400,7 @@ test_expect_failure 'move dir from in-cone to out-of-cone' '\n \tgrep -x \"S folder1/dir/e\" actual\n '\n \n-test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n+test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \ttouch sub/dir/e2 sub/dir/e3 &&\n-- \n2.37.0\n\n"},{"id":"459365","messageId":"20220719132809.409247-8-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v1 7/7] mv: check overwrite for in-to-out move","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-19T13:28:09Z","receivedAt":"2022-07-19T14:10:41Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add checking logic for overwritng when moving from in-cone to\nout-of-cone. It is the index version of the original overwrite logic.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 12 ++++++++++++\n t/t7002-mv-sparse-checkout.sh | 2 +-\n 2 files changed, 13 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex d35994c443..5ed3bd3431 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -365,6 +365,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tgoto act_on_entry;\n \t\t}\n \n+\t\tif (ignore_sparse &&\n+\t\t (dst_mode & SKIP_WORKTREE_DIR) &&\n+\t\t index_entry_exists(&the_index, dst, strlen(dst))) {\n+\t\t\tbad = _(\"destination exists in the index\");\n+\t\t\tif (force) {\n+\t\t\t\tif (verbose)\n+\t\t\t\t\twarning(_(\"overwriting '%s'\"), dst);\n+\t\t\t\tbad = NULL;\n+\t\t\t} else {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n \t\t/*\n \t\t * We check if the paths are in the sparse-checkout\n \t\t * definition as a very final check, since that\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex dafe15b9cf..5d810f3af0 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -323,7 +323,7 @@ test_expect_success 'move clean path from in-cone to out-of-cone' '\n \tgrep -x \"S folder1/d\" actual\n '\n \n-test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n+test_expect_success 'move clean path from in-cone to out-of-cone overwrite' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"sub/file1 overwrite\" >sub/file1 &&\n-- \n2.37.0\n\n"},{"id":"459368","messageId":"220719.86pmi1up9d.gmgdl@evledraar.gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-2-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 1/7] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-07-19T14:52:20Z","receivedAt":"2022-07-19T14:54:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Jul 19 2022, Shaoxuan Yuan wrote:\n\n> +\t! grep -x \"H sub/d\" actual &&\n> +\tgrep -x \"S folder1/d\" actual\n\nThis *might* be a portability problem, but probably not. It's listed in\nPOSIX:\nhttps://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html\n\nBut it appears we have no \"grep\" use of the \"-x\" option in-tree, at\nleast from what I found with grepping..\n"},{"id":"459392","messageId":"d914bd30-0f53-864e-b640-0173d168bcb5@github.com","threadId":"58187","inReplyTo":"220719.86pmi1up9d.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v1 1/7] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T17:36:46Z","receivedAt":"2022-07-19T17:36:53Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 10:52 AM, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Tue, Jul 19 2022, Shaoxuan Yuan wrote:\n> \n>> +\t! grep -x \"H sub/d\" actual &&\n>> +\tgrep -x \"S folder1/d\" actual\n> \n> This *might* be a portability problem, but probably not. It's listed in\n> POSIX:\n> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html\n> \n> But it appears we have no \"grep\" use of the \"-x\" option in-tree, at\n> least from what I found with grepping..\n\nIt is helpful to recommend a working alternative when pointing out\nan issue like this.\n\nHere, the '-x' option means to match the entire line.\n\nWould using ^ and $ help here?\n\n\t! grep \"^H sub/d\\$\" actual &&\n\tgrep \"^S folder1/d\\$\" actual\n\nI haven't tested this, but hopefully the idea is sound. If not,\nthen you could make the names be sufficiently unique that\nwhole-lines are the only way they could match. We do have some\nexpectation about what lines are possible in this index, since\nwe constructed the example repo from scratch.\n\nThanks,\n-Stolee\n"},{"id":"459393","messageId":"228ad533-477c-f16e-220d-61e52d9aee26@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-3-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 2/7] mv: add documentation for check_dir_in_index()","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T17:43:00Z","receivedAt":"2022-07-19T17:43:06Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n> Using check_dir_in_index without checking if the directory is on-disk\n> could get a false positive for partially sparsified directory.\n\nIt can be helpful to point out that this is a relatively new\nmethod, introduced in b91a2b6594 (mv: add check_dir_in_index()\nand solve general dir check issue, 2022-06-30).\n\n> + *\n> + * Note: *always* check the directory is not on-disk before this function\n> + * (i.e. using lstat());\n> + * otherwise it may return a false positive for a partially sparsified\n> + * directory.\n\nI'm not sure what you mean by a \"false positive\" in this case.\nThe directory exists in the index, which is what the method is\ndefined as checking. This does not say anything about the\nworktree.\n\nPerhaps that's the real problem? Someone might interpret this\nas meaning the directory does not exist in the worktree? That\nwould mean that this doc update needs to be changed significantly\nto say \"Note that this does not imply anything about the state\nof the worktree\" or something.\n\nBut I think I'd rather just see this patch be dropped, unless I\nam missing something important.\n\nThanks,\n-Stolee\n"},{"id":"459394","messageId":"0a93bea5-2d5a-298d-f501-ec4bb9a9c45f@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-4-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 3/7] mv: free the *with_slash in check_dir_in_index()","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T17:46:44Z","receivedAt":"2022-07-19T17:46:50Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:> *with_slash may be a malloc'd pointer, and when it is, free it.\nIt might be worth mentioning that you are reorganizing the\nmethod to use gotos.\n\n> static int check_dir_in_index(const char *name)\n> {\n> +\tint ret = 1;\n\nInteresting to \"assume error\", but that works since there is only one\nplace where the success is guaranteed.\n\nShould we use -1 here?\n\n> \t\tif (ce_skip_worktree(ce))\n> -\t\t\treturn 0;\n> +\t\t\tret = 0;\n\nThis could be\n\n\tret = ce_skip_worktree(ce);\n\nunless you really rely on the \"1\" value.\n\n> \t}\n> -\treturn 1;\n> +\n> +free_return:\n> +\tif (with_slash != name)\n> +\t\tfree((char *)with_slash);\n\nGood check to not clear 'name'. It's unfortunate that we need to cast here,\nbut it makes sense.\n\nThanks,\n-Stolee\n"},{"id":"459395","messageId":"95263c88-6894-87bb-4d52-84d8d1cc5671@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-5-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 4/7] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T17:59:27Z","receivedAt":"2022-07-19T17:59:33Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n> Originally, is assumed to be in the working tree. If it is\n> not found as a directory, then it is determined to be either a regular file\n> path, or error out if used under the second form (move into a directory)\n> of 'git-mv'. Such behavior is not ideal, mainly because Git does not\n> look into the index for , which could potentially be a\n> SKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\n> in-cone to out-of-cone\" patch.\n> \n> Change the logic so that Git first check if is a directory\n> with all its contents sparsified (a SKIP_WORKTREE_DIR). If yes,\n> then treat as a directory exists in the working tree, and\n\n\"treat as a directory exists in the working tree\" is a bit\nakward, and the rest of the sentence struggles with that. Starting with\n\"If yes,\" we could rewrite as follows:\n\n If is such a sparse directory, then we should modify the\n index the same way as we would if this were a non-sparse directory. We\n must be careful to ensure that the is marked with \n SKIP_WORKTREE_DIR.\n\n(Note that I don't equate this as doing the same thing, just operating on\nthe index.)\n\n> If no, continue the original checking logic.\n\nI think this part doesn't need to be there, but I don't feel strongly\nabout it.\n\n> Also add a `dst_w_slash` to reuse the result from `add_slash()`, which\n> was everywhere and can be simplified.\n\n> \telse if (!lstat(dest_path[0], &st) &&\n> \t\t\tS_ISDIR(st.st_mode)) {\n> -\t\tdest_path[0] = add_slash(dest_path[0]);\n> -\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> +\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n\nHm. I find it interesting how this works even if the directory is _not_\npresent in the index. Is there a test that checks this kind of setup?\n\n\tgit init &&\n\t>file &&\n\tgit add file &&\n\tgit commit -m init &&\n\tmkdir dir &&\n\tgit mv file dir/\n\nLocally, this indeed works with the following 'git status' detail:\n\n renamed: file -> dir/file\n\n> \t} else {\n> -\t\tif (argc != 1)\n> +\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n> +\t\t !check_dir_in_index(dst_w_slash)) {\n> +\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n> +\t\t\tdst_mode |= SKIP_WORKTREE_DIR;\n\nLooks like we are assigning dst_mode here, but not using it. I wonder if\nyou'll get a compiler error if you build this patch with DEVELOPER=1.\n\nYou can test all of the commits in your series using this command:\n\n git rebase -x \"make -j12 DEVELOPER=1\" \n\n> +\tif (dst_w_slash != dest_path[0])\n> +\t\tfree((char *)dst_w_slash);\n\nGood that you are freeing this here. You might also want to set it to NULL\njust in case.\n\nThanks,\n-Stolee\n"},{"id":"459396","messageId":"4996a896-cf92-7fc7-40d3-dc3e74e804a4@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-6-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 5/7] mv: remove BOTH from enum update_mode","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T18:00:44Z","receivedAt":"2022-07-19T18:00:50Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n> Since BOTH is not practically used anywhere in the code and its meaning\n> is unclear, remove it.\n\nIt's literally not used any where. (I would just drop \"practically\".)\n\nAlso, now that enum update_mode is a flag-based enum, the 0 value cannot\nhave any meaning to it except \"none of the other flags are on\".\n \nThanks,\n-Stolee\n\n"},{"id":"459397","messageId":"6188fab7-df5e-b3fb-92a2-47039254ac45@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-3-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 2/7] mv: add documentation for check_dir_in_index()","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-07-19T18:01:21Z","receivedAt":"2022-07-19T18:01:27Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Using check_dir_in_index without checking if the directory is on-disk\n> could get a false positive for partially sparsified directory.\n> \n> Add a note in the documentation to warn potential user.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 5 +++++\n> 1 file changed, 5 insertions(+)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 4729bb1a1a..c8b9069db8 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -132,6 +132,11 @@ static int index_range_of_same_dir(const char *src, int length,\n> * Return 0 if such directory exist (i.e. with any of its contained files not\n> * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n> * Return 1 otherwise.\n> + *\n> + * Note: *always* check the directory is not on-disk before this function\n> + * (i.e. using lstat());\n> + * otherwise it may return a false positive for a partially sparsified\n> + * directory.\n\nTo me, a comment like this indicates that either the function is not doing\nwhat it should be doing, or its name doesn't properly describe the\nfunction's behavior.\n\nPer the function description:\n\n\tCheck if an out-of-cone directory should be in the index. Imagine\n\tthis case that all the files under a directory are marked with\n\t'CE_SKIP_WORKTREE' bit and thus the directory is sparsified.\n\nBut neither the name of the function ('check_dir_in_index') nor the\nfunction's behavior (hence the comment you're adding here) match that\ndescription.\n\nSince this function is intended to verify that a directory 1) exists in the\nindex, and 2) is *entirely* sparse, I have two suggestions:\n\n* Change the description to specify that the non-existence of the directory\n on disk is an *assumption*, not an opportunity for a \"false positive.\"\n It's a subtle distinction, but it clarifies that the function should be\n used only when the caller already knows the directory is empty. For\n example:\n\n\t/*\n\t * Given the path of a directory that does not exist on-disk, check whether the\n\t * directory contains any entries in the index with the SKIP_WORKTREE flag \n\t * enabled.\n\t *\n\t * Return 1 if such index entries exist.\n\t * Return 0 otherwise.\n\t */\n\n* 'check_dir_in_index' doesn't reflect the \"is not on disk\" prerequisite, so\n the function name can be updated to clarify that (e.g.,\n 'empty_dir_has_sparse_contents')\n\n> */\n> static int check_dir_in_index(const char *name)\n> {\n\n"},{"id":"459398","messageId":"d5889658-77f5-25ff-7469-3f00a41857da@github.com","threadId":"58187","inReplyTo":"6188fab7-df5e-b3fb-92a2-47039254ac45@github.com","subject":"Re: [PATCH v1 2/7] mv: add documentation for check_dir_in_index()","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-07-19T18:10:56Z","receivedAt":"2022-07-19T18:11:43Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> Using check_dir_in_index without checking if the directory is on-disk\n>> could get a false positive for partially sparsified directory.\n>>\n>> Add a note in the documentation to warn potential user.\n>>\n>> Signed-off-by: Shaoxuan Yuan \n>> ---\n>> builtin/mv.c | 5 +++++\n>> 1 file changed, 5 insertions(+)\n>>\n>> diff --git a/builtin/mv.c b/builtin/mv.c\n>> index 4729bb1a1a..c8b9069db8 100644\n>> --- a/builtin/mv.c\n>> +++ b/builtin/mv.c\n>> @@ -132,6 +132,11 @@ static int index_range_of_same_dir(const char *src, int length,\n>> * Return 0 if such directory exist (i.e. with any of its contained files not\n>> * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n>> * Return 1 otherwise.\n>> + *\n>> + * Note: *always* check the directory is not on-disk before this function\n>> + * (i.e. using lstat());\n>> + * otherwise it may return a false positive for a partially sparsified\n>> + * directory.\n> \n> To me, a comment like this indicates that either the function is not doing\n> what it should be doing, or its name doesn't properly describe the\n> function's behavior.\n> \n> Per the function description:\n> \n> \tCheck if an out-of-cone directory should be in the index. Imagine\n> \tthis case that all the files under a directory are marked with\n> \t'CE_SKIP_WORKTREE' bit and thus the directory is sparsified.\n> \n> But neither the name of the function ('check_dir_in_index') nor the\n> function's behavior (hence the comment you're adding here) match that\n> description.\n> \n> Since this function is intended to verify that a directory 1) exists in the\n> index, and 2) is *entirely* sparse, I have two suggestions:\n> \n> * Change the description to specify that the non-existence of the directory\n> on disk is an *assumption*, not an opportunity for a \"false positive.\"\n> It's a subtle distinction, but it clarifies that the function should be\n> used only when the caller already knows the directory is empty. For\n> example:\n> \n> \t/*\n> \t * Given the path of a directory that does not exist on-disk, check whether the\n> \t * directory contains any entries in the index with the SKIP_WORKTREE flag \n> \t * enabled.\n> \t *\n> \t * Return 1 if such index entries exist.\n> \t * Return 0 otherwise.\n> \t */\n\nWhoops, I mixed up the return values (I assumed this returned a boolean\nbased on the 'check' in the function name). In that case...\n> \n> * 'check_dir_in_index' doesn't reflect the \"is not on disk\" prerequisite, so\n> the function name can be updated to clarify that (e.g.,\n> 'empty_dir_has_sparse_contents')\n\n...there are two options. Either you can use a more \"boolean-y\" name (like\nthe one I suggest above) and flip the return values (where \"1\" means \"the\nempty dir has sparse contents\"), or change the name to something that\nexplicitly *doesn't* imply boolean. I'm personally in favor of the former\n(I'm really struggling to come up with a descriptive, non-boolean name for\nthis function), but I'm fine with either if you can come up with a good\nfunction name.\n\n> \n>> */\n>> static int check_dir_in_index(const char *name)\n>> {\n> \n\n"},{"id":"459399","messageId":"d30a358e-c9f0-55fa-8c8b-37f0cb0d7eb3@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-7-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 6/7] mv: from in-cone to out-of-cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T18:14:28Z","receivedAt":"2022-07-19T18:14:38Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n> Originally, moving an in-cone to an out-of-cone \n> was not possible, mainly because such is a directory that\n> is not present in the working tree.\n> \n> Change the behavior so that we can move an in-cone to\n> out-of-cone when --sparse is supplied.\n> \n> Such can be either clean or dirty, and moving it results in\n> different behaviors:\n> \n> A clean move should move the to the , both in the\n> working tree and the index, then remove the resulted path from the\n> working tree, and turn on its CE_SKIP_WORKTREE bit.\n> \n> A dirty move should move the to the , both in the\n> working tree and the index, but should *not* remove the resulted path\n> from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n> Instead advise user to \"git add\" this path and run \"git sparse-checkout\n> reapply\" to re-sparsify that path.\n\nI feel like this patch should be two, instead: you can have one that\nmakes the commands succeed or fail properly, and another that outputs the\nadvice. As it is, it's a bit muddled as to what is necessary for the\nmovement of the index entry versus representing the error message.\n\nThis might mean that you want to pull the advice messages out of the tests\nthat are added in patch 1 and only apply those to the test as you implement\nthe advice in that later patch. Such a split of the test implementation\nwould allow this patch to still switch a bunch of test_expect_failure tests\nto test_expect_success.\n\n> @@ -424,6 +426,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tif (show_only)\n> \t\t\tcontinue;\n> \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> +\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n\nHere is a use of dst_mode that might be helpful to put with the change in\npatch 4. Alternatively, you could delay declaring dst_mode until now.\n\n> +\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n> +\t\t\tup_to_date = !ce_modified(active_cache[pos], &st, 0);\n\nHere, you are only checking for a dirty file if (mode & SPARSE) and the\nfile exists.\n\nPerhaps it would be helpful to negate this and rename the variable?\n\n\tsparse_and_dirty = ce_modified(active_cache[pos], &st, 0);\n\nThis makes it clear that we can't use the variable except in this\nparticular case.\n\n> -\t\tif ((mode & SPARSE) &&\n> -\t\t (path_in_sparse_checkout(dst, &the_index))) {\n> -\t\t\tint dst_pos;\n> +\t\tif (ignore_sparse &&\n> +\t\t core_apply_sparse_checkout &&\n> +\t\t core_sparse_checkout_cone) {\n> \n> -\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n> -\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n> +\t\t\t/* from out-of-cone to in-cone */\n> +\t\t\tif ((mode & SPARSE) &&\n> +\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n> +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n> \n> -\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n> -\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n> +\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n> +\n> +\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n> +\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n> +\t\t\t\tcontinue;\n> +\t\t\t}\n\nHere, it helps to ignore whitespace changes. This out to in was already\nhandled by the existing implementation.\n\n> +\t\t\t/* from in-cone to out-of-cone */\n> +\t\t\tif ((dst_mode & SKIP_WORKTREE_DIR) &&\n\nThis is disjoint from the other case (because of !path_in_sparse_checkout()),\nso maybe we could short-circuit with an \"else if\" here? You could put your\ncomments about the in-to-out or out-to-in inside the if blocks.\n\n> +\t\t\t !(mode & SPARSE) &&\n> +\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n> +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n\n(It took me a while to realize that dst_ce is the right entry because we\nalready called rename_cache_entry_at() earlier.)\n\n> +\t\t\t\tchar *src_dir = dirname(xstrdup(src));\n\nThe xstrdup(src) return string is being lost here.\n\n> +\n> +\t\t\t\tif (up_to_date) {\n\nBased on my recommendation earlier, this would become\n\n\tif (!sparse_and_dirty) {\n\n> +\t\t\t\t\tdst_ce->ce_flags |= CE_SKIP_WORKTREE;\n> +\t\t\t\t\tunlink_or_warn(src);\n> +\t\t\t\t} else {\n> +\t\t\t\t\tstring_list_append(&dirty_paths, dst);\n> +\t\t\t\t\tsafe_create_leading_directories(xstrdup(dst));\n> +\t\t\t\t\trename(src, dst);\n\nOk, still doing the file rename, but leaving it unstaged.\n\n> +\t\t\t\t}\n\nPlease provide some whitespace between the close of an if/else chain before\nstarting the next if.\n\n> +\t\t\t\tif ((mode & INDEX) && is_empty_dir(src_dir))\n> +\t\t\t\t\trmdir_or_warn(src_dir);\n\nThis is an interesting cleanup of the first-level directory. Should it be\nrecursive and clean up an entire chain of paths?\n\nThanks,\n-Stolee\n"},{"id":"459400","messageId":"97586f49-1d06-636a-200f-00b68930ed5e@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-8-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 7/7] mv: check overwrite for in-to-out move","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T18:15:27Z","receivedAt":"2022-07-19T18:15:32Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n> Add checking logic for overwritng when moving from in-cone to\n\ns/overwritng/overwriting/\n\n> out-of-cone. It is the index version of the original overwrite logic.\n\nCode looks good to me.\n\nThanks,\n-Stolee\n"},{"id":"459401","messageId":"16b9fd86-ba06-47b6-84f2-38a6e53e45fc@github.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v1 0/7] mv: from in-cone to out-of-cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-19T18:16:54Z","receivedAt":"2022-07-19T18:16:59Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n> This is a sister series to the previous \"from out-of-cone to in-cone\" [1]\n> series. This series is trying to make the opposite operation possible for\n> 'mv', namely move , which is in-cone, to , which is\n> out-of-cone.\n> \n> Other than the main task, there are also some minor fixes done.\n\nI'm happy to see that the main task is less complicated than the previous\ncase, because of your hard work refactoring some things.\n\nI still have some ideas about how to break up the change to be a bit more\nreadable. Most of my comments on this v1 are nits, though. Good work!\n\nThanks,\n-Stolee\n"},{"id":"459405","messageId":"xmqq4jzdaran.fsf@gitster.g","threadId":"58187","inReplyTo":"d914bd30-0f53-864e-b640-0173d168bcb5@github.com","subject":"Re: [PATCH v1 1/7] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-19T18:30:40Z","receivedAt":"2022-07-19T18:30:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 7/19/2022 10:52 AM, Ævar Arnfjörð Bjarmason wrote:\n>> \n>> On Tue, Jul 19 2022, Shaoxuan Yuan wrote:\n>> \n>>> +\t! grep -x \"H sub/d\" actual &&\n>>> +\tgrep -x \"S folder1/d\" actual\n>> \n>> This *might* be a portability problem, but probably not. It's listed in\n>> POSIX:\n>> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html\n>> \n>> But it appears we have no \"grep\" use of the \"-x\" option in-tree, at\n>> least from what I found with grepping..\n>\n> It is helpful to recommend a working alternative when pointing out\n> an issue like this.\n>\n> Here, the '-x' option means to match the entire line.\n>\n> Would using ^ and $ help here?\n\nYup, that would be quite reasonable and removes the need to use,\nlearn about, or have your readers wonder about the \"-x\" option; I do\nnot think I saw a grep that did not understand the option, though.\n\nThanks.\n"},{"id":"459621","messageId":"0af9c29a-2071-6aa3-28ce-9b9127789644@gmail.com","threadId":"58187","inReplyTo":"228ad533-477c-f16e-220d-61e52d9aee26@github.com","subject":"Re: [PATCH v1 2/7] mv: add documentation for check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-21T13:58:23Z","receivedAt":"2022-07-21T13:58:35Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 7/20/2022 1:43 AM, Derrick Stolee wrote:\n >> + *\n >> + * Note: *always* check the directory is not on-disk before this \nfunction\n >> + * (i.e. using lstat());\n >> + * otherwise it may return a false positive for a partially sparsified\n >> + * directory.\n >\n > I'm not sure what you mean by a \"false positive\" in this case.\n > The directory exists in the index, which is what the method is\n > defined as checking. This does not say anything about the\n > worktree.\n >\n > Perhaps that's the real problem? Someone might interpret this\n > as meaning the directory does not exist in the worktree? That\n > would mean that this doc update needs to be changed significantly\n > to say \"Note that this does not imply anything about the state\n > of the worktree\" or something.\n\nThis method assumes that the directory being checking does not exist\nin the working tree, but the method itself does not check this. And\nif the user does not make sure the directory is absent from the\nworktree, this method may return a success for a partially sparsified\ndirectory, which is not intended.\n\n > But I think I'd rather just see this patch be dropped, unless I\n > am missing something important.\n\nI found Victoria's paraphrase [1] makes my point much clearer.\n\n--\nThanks,\nShaoxuan\n\n"},{"id":"459626","messageId":"ab96cbe0-f256-ea5e-8356-db11d2b773f4@gmail.com","threadId":"58187","inReplyTo":"95263c88-6894-87bb-4d52-84d8d1cc5671@github.com","subject":"Re: [PATCH v1 4/7] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-21T14:13:44Z","receivedAt":"2022-07-21T14:13:58Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\n\nOn 7/20/2022 1:59 AM, Derrick Stolee wrote:\n > On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n >> Originally, is assumed to be in the working tree. If it is\n >> not found as a directory, then it is determined to be either a \nregular file\n >> path, or error out if used under the second form (move into a directory)\n >> of 'git-mv'. Such behavior is not ideal, mainly because Git does not\n >> look into the index for , which could potentially be a\n >> SKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\n >> in-cone to out-of-cone\" patch.\n >>\n >> Change the logic so that Git first check if is a directory\n >> with all its contents sparsified (a SKIP_WORKTREE_DIR). If yes,\n >> then treat as a directory exists in the working tree, and\n >\n > \"treat as a directory exists in the working tree\" is a bit\n > akward, and the rest of the sentence struggles with that. Starting with\n > \"If yes,\" we could rewrite as follows:\n >\n >   If is such a sparse directory, then we should modify the\n >   index the same way as we would if this were a non-sparse directory. We\n >   must be careful to ensure that the is marked with\n >   SKIP_WORKTREE_DIR.\n >\n > (Note that I don't equate this as doing the same thing, just operating on\n > the index.)\n\nThis wording sounds more natural, thanks for the rewrite!\n\n >> If no, continue the original checking logic.\n >\n > I think this part doesn't need to be there, but I don't feel strongly\n > about it.\n >\n >> Also add a `dst_w_slash` to reuse the result from `add_slash()`, which\n >> was everywhere and can be simplified.\n >\n >>      else if (!lstat(dest_path[0], &st) &&\n >>              S_ISDIR(st.st_mode)) {\n >> -        dest_path[0] = add_slash(dest_path[0]);\n >> -        destination = internal_prefix_pathspec(dest_path[0], argv, \nargc, DUP_BASENAME);\n >> +        destination = internal_prefix_pathspec(dst_w_slash, argv, \nargc, DUP_BASENAME);\n >\n > Hm. I find it interesting how this works even if the directory is _not_\n > present in the index. Is there a test that checks this kind of setup?\n >\n >     git init &&\n >     >file &&\n >     git add file &&\n >     git commit -m init &&\n >     mkdir dir &&\n >     git mv file dir/\n >\n > Locally, this indeed works with the following 'git status' detail:\n >\n >         renamed:    file -> dir/file\n\nIn my impression, 'git-mv' does not seem to care about whether the\n directory is in index? Given that `rename()` works so\nlong as the directory is in the working tree, and `rename_index_entry_at()`\ncares even less about the dir?\n\n >>      } else {\n >> -        if (argc != 1)\n >> +        if (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n >> +            !check_dir_in_index(dst_w_slash)) {\n >> +            destination = internal_prefix_pathspec(dst_w_slash, \nargv, argc, DUP_BASENAME);\n >> +            dst_mode |= SKIP_WORKTREE_DIR;\n >\n > Looks like we are assigning dst_mode here, but not using it. I wonder if\n > you'll get a compiler error if you build this patch with DEVELOPER=1.\n >\n > You can test all of the commits in your series using this command:\n >\n >   git rebase -x \"make -j12 DEVELOPER=1\" \n\nOops, it seems that I was doing the wrong way. I only run test on the\ntip of the branch, without actually testing individual commits.\nWill do this.\n\n >> +    if (dst_w_slash != dest_path[0])\n >> +        free((char *)dst_w_slash);\n >\n > Good that you are freeing this here. You might also want to set it to \nNULL\n > just in case.\n\nI was using the `FREE_AND_NULL()` macro, but I wasn't sure since other\nplaces in 'git-mv' only use `free()`. Though I think it is better to\n`FREE_AND_NULL()`.\n\n--\nThanks,\nShaoxuan\n\n"},{"id":"459628","messageId":"191bcdca-c7fd-665c-78ce-ef4c5843d13d@gmail.com","threadId":"58187","inReplyTo":"6188fab7-df5e-b3fb-92a2-47039254ac45@github.com","subject":"Re: [PATCH v1 2/7] mv: add documentation for check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-07-21T14:20:25Z","receivedAt":"2022-07-21T14:20:35Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\n\nOn 7/20/2022 2:01 AM, Victoria Dye wrote:\n > Shaoxuan Yuan wrote:\n >> Using check_dir_in_index without checking if the directory is on-disk\n >> could get a false positive for partially sparsified directory.\n >>\n >> Add a note in the documentation to warn potential user.\n >>\n >> Signed-off-by: Shaoxuan Yuan \n >> ---\n >>  builtin/mv.c | 5 +++++\n >>  1 file changed, 5 insertions(+)\n >>\n >> diff --git a/builtin/mv.c b/builtin/mv.c\n >> index 4729bb1a1a..c8b9069db8 100644\n >> --- a/builtin/mv.c\n >> +++ b/builtin/mv.c\n >> @@ -132,6 +132,11 @@ static int index_range_of_same_dir(const char \n*src, int length,\n >>   * Return 0 if such directory exist (i.e. with any of its contained \nfiles not\n >>   * marked with CE_SKIP_WORKTREE, the directory would be present in \nworking tree).\n >>   * Return 1 otherwise.\n >> + *\n >> + * Note: *always* check the directory is not on-disk before this \nfunction\n >> + * (i.e. using lstat());\n >> + * otherwise it may return a false positive for a partially sparsified\n >> + * directory.\n >\n > To me, a comment like this indicates that either the function is not \ndoing\n > what it should be doing, or its name doesn't properly describe the\n > function's behavior.\n >\n > Per the function description:\n >\n >     Check if an out-of-cone directory should be in the index. Imagine\n >     this case that all the files under a directory are marked with\n >     'CE_SKIP_WORKTREE' bit and thus the directory is sparsified.\n >\n > But neither the name of the function ('check_dir_in_index') nor the\n > function's behavior (hence the comment you're adding here) match that\n > description.\n >\n > Since this function is intended to verify that a directory 1) exists \nin the\n > index, and 2) is *entirely* sparse, I have two suggestions:\n >\n > * Change the description to specify that the non-existence of the \ndirectory\n >   on disk is an *assumption*, not an opportunity for a \"false positive.\"\n >   It's a subtle distinction, but it clarifies that the function should be\n >   used only when the caller already knows the directory is empty. For\n >   example:\n >\n >     /*\n >      * Given the path of a directory that does not exist on-disk, \ncheck whether the\n >      * directory contains any entries in the index with the \nSKIP_WORKTREE flag\n >      * enabled.\n >      *\n >      * Return 1 if such index entries exist.\n >      * Return 0 otherwise.\n >      */\n >\n > * 'check_dir_in_index' doesn't reflect the \"is not on disk\" \nprerequisite, so\n >   the function name can be updated to clarify that (e.g.,\n >   'empty_dir_has_sparse_contents')\n >\n\nI really breathed a sigh of relief after seeing these two points :-)\nThey word things out much better than the original ones.\n\n--\nThanks,\nShaoxuan\n\n"},{"id":"459773","messageId":"25ca0c62-e3b4-e96a-4c44-285bcfef2143@github.com","threadId":"58187","inReplyTo":"ab96cbe0-f256-ea5e-8356-db11d2b773f4@gmail.com","subject":"Re: [PATCH v1 4/7] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-22T12:48:08Z","receivedAt":"2022-07-22T12:48:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/21/2022 10:13 AM, Shaoxuan Yuan wrote:\n> On 7/20/2022 1:59 AM, Derrick Stolee wrote:\n>> On 7/19/2022 9:28 AM, Shaoxuan Yuan wrote:\n>>> Also add a `dst_w_slash` to reuse the result from `add_slash()`, which\n>>> was everywhere and can be simplified.\n>>\n>>>      else if (!lstat(dest_path[0], &st) &&\n>>>              S_ISDIR(st.st_mode)) {\n>>> -        dest_path[0] = add_slash(dest_path[0]);\n>>> -        destination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n>>> +        destination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n>>\n>> Hm. I find it interesting how this works even if the directory is _not_\n>> present in the index. Is there a test that checks this kind of setup?\n>>\n>>     git init &&\n>>     >file &&\n>>     git add file &&\n>>     git commit -m init &&\n>>     mkdir dir &&\n>>     git mv file dir/\n>>\n>> Locally, this indeed works with the following 'git status' detail:\n>>\n>>         renamed:    file -> dir/file\n> \n> In my impression, 'git-mv' does not seem to care about whether the\n> directory is in index? Given that `rename()` works so\n> long as the directory is in the working tree, and `rename_index_entry_at()`\n> cares even less about the dir?\n\nRight. Instead of changing the behavior, I'm asking you to double-check that\nthis behavior is tested. If not, then please add a test.\n\n>>> +    if (dst_w_slash != dest_path[0])\n>>> +        free((char *)dst_w_slash);\n>>\n>> Good that you are freeing this here. You might also want to set it to NULL\n>> just in case.\n> \n> I was using the `FREE_AND_NULL()` macro, but I wasn't sure since other\n> places in 'git-mv' only use `free()`. Though I think it is better to\n> `FREE_AND_NULL()`.\n\nfree() is generally the way to go if it is clear that the variable\nis about to go out-of-scope and could not possibly be referenced\nagain. Since there is a lot more of the current code block to go,\nnulling the variable is good defensive programming.\n\nThanks,\n-Stolee\n"},{"id":"459784","messageId":"xmqqczdxc7of.fsf@gitster.g","threadId":"58187","inReplyTo":"25ca0c62-e3b4-e96a-4c44-285bcfef2143@github.com","subject":"Re: [PATCH v1 4/7] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-22T18:40:32Z","receivedAt":"2022-07-22T18:40:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n>>> Good that you are freeing this here. You might also want to set it to NULL\n>>> just in case.\n>> \n>> I was using the `FREE_AND_NULL()` macro, but I wasn't sure since other\n>> places in 'git-mv' only use `free()`. Though I think it is better to\n>> `FREE_AND_NULL()`.\n>\n> free() is generally the way to go if it is clear that the variable\n> is about to go out-of-scope and could not possibly be referenced\n> again. Since there is a lot more of the current code block to go,\n> nulling the variable is good defensive programming.\n\nNULLing it out is better when a potential misuse of the pointer\nafter it got freed will be caught by dereferencing NULL.\n\nThere however are pointer members of structures wher they represent\noptional data. Access to such a member goes like so:\n\n\tif (structure->optinal_member)\n\t\tdo_things(structure->optional_member);\n\nWhen you are done using such a structure and clearing it, after\nreleasing the resource held by the member, it is better to leave it\ndangling than assigning NULL to it. If somebody reuses that\nstructure and the control enters a codepath like the above one to\nuse the \"optional\" pointer, uncleared dangling pointer will likely\nbe caught at runtime; setting it to NULL will paper over it. We've\nseen many bugs caused by a premature releasing of a member that was\nhidden exactly by such a use of FREE_AND_NULL() few relases ago.\n\nThanks.\n"},{"id":"460540","messageId":"9a568923-0b2c-8c41-e774-bcf8e6e9d9ea@gmail.com","threadId":"58187","inReplyTo":"d30a358e-c9f0-55fa-8c8b-37f0cb0d7eb3@github.com","subject":"Re: [PATCH v1 6/7] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-03T11:50:05Z","receivedAt":"2022-08-03T11:50:19Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 7/20/2022 2:14 AM, Derrick Stolee wrote:\n >> -        if ((mode & SPARSE) &&\n >> -            (path_in_sparse_checkout(dst, &the_index))) {\n >> -            int dst_pos;\n >> +        if (ignore_sparse &&\n >> +            core_apply_sparse_checkout &&\n >> +            core_sparse_checkout_cone) {\n >>\n >> -            dst_pos = cache_name_pos(dst, strlen(dst));\n >> -            active_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n >> +            /* from out-of-cone to in-cone */\n >> +            if ((mode & SPARSE) &&\n >> +                path_in_sparse_checkout(dst, &the_index)) {\n >> +                int dst_pos = cache_name_pos(dst, strlen(dst));\n >> +                struct cache_entry *dst_ce = active_cache[dst_pos];\n >>\n >> -            if (checkout_entry(active_cache[dst_pos], &state, NULL, \nNULL))\n >> -                die(_(\"cannot checkout %s\"), \nactive_cache[dst_pos]->name);\n >> +                dst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n >> +\n >> +                if (checkout_entry(dst_ce, &state, NULL, NULL))\n >> +                    die(_(\"cannot checkout %s\"), dst_ce->name);\n >> +                continue;\n >> +            }\n >\n > Here, it helps to ignore whitespace changes. This out to in was already\n > handled by the existing implementation.\n\nYes, I think it would be better to let `diff` ignore the existing\nimplementation. Are you suggesting the `-w` (--ignore-all-space) option\nof `diff`? I tried this option and it does not work. But another reason\nis that there *are* some changes that are different from the original\nout-to-in implementation, so even though it looks a bit messy, I think\nit makes sense.\n\n >> +            /* from in-cone to out-of-cone */\n >> +            if ((dst_mode & SKIP_WORKTREE_DIR) &&\n >\n > This is disjoint from the other case (because of \n!path_in_sparse_checkout()),\n > so maybe we could short-circuit with an \"else if\" here? You could put \nyour\n > comments about the in-to-out or out-to-in inside the if blocks.\n\nI tried an else-if but it does clutter the code a bit. I think I'll\nleave it as-is. Or do you mind show me a diff of your approach? To be\nhonest, this disjoint here looks logically cleaner to me.\n\n--\nThanks,\nShaoxuan\n\n"},{"id":"460547","messageId":"1db0d83d-0239-5a5f-3390-822ee780172e@github.com","threadId":"58187","inReplyTo":"9a568923-0b2c-8c41-e774-bcf8e6e9d9ea@gmail.com","subject":"Re: [PATCH v1 6/7] mv: from in-cone to out-of-cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-08-03T14:30:20Z","receivedAt":"2022-08-03T14:30:28Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/3/2022 7:50 AM, Shaoxuan Yuan wrote:\n> On 7/20/2022 2:14 AM, Derrick Stolee wrote:\n>>> -        if ((mode & SPARSE) &&\n>>> -            (path_in_sparse_checkout(dst, &the_index))) {\n>>> -            int dst_pos;\n>>> +        if (ignore_sparse &&\n>>> +            core_apply_sparse_checkout &&\n>>> +            core_sparse_checkout_cone) {\n>>>\n>>> -            dst_pos = cache_name_pos(dst, strlen(dst));\n>>> -            active_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n>>> +            /* from out-of-cone to in-cone */\n>>> +            if ((mode & SPARSE) &&\n>>> +                path_in_sparse_checkout(dst, &the_index)) {\n>>> +                int dst_pos = cache_name_pos(dst, strlen(dst));\n>>> +                struct cache_entry *dst_ce = active_cache[dst_pos];\n>>>\n>>> -            if (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n>>> -                die(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n>>> +                dst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n>>> +\n>>> +                if (checkout_entry(dst_ce, &state, NULL, NULL))\n>>> +                    die(_(\"cannot checkout %s\"), dst_ce->name);\n>>> +                continue;\n>>> +            }\n>>\n>> Here, it helps to ignore whitespace changes. This out to in was already\n>> handled by the existing implementation.\n> \n> Yes, I think it would be better to let `diff` ignore the existing\n> implementation. Are you suggesting the `-w` (--ignore-all-space) option\n> of `diff`? I tried this option and it does not work. But another reason\n> is that there *are* some changes that are different from the original\n> out-to-in implementation, so even though it looks a bit messy, I think\n> it makes sense.\n\nI'm just making a note that I looked at this not in its patch form,\nbut as a commit diff where I could use the '-w' option to get a nice\nview. It's not possible to do that in the patch.\n \n>>> +            /* from in-cone to out-of-cone */\n>>> +            if ((dst_mode & SKIP_WORKTREE_DIR) &&\n>>\n>> This is disjoint from the other case (because of !path_in_sparse_checkout()),\n>> so maybe we could short-circuit with an \"else if\" here? You could put your\n>> comments about the in-to-out or out-to-in inside the if blocks.\n> \n> I tried an else-if but it does clutter the code a bit. I think I'll\n> leave it as-is. Or do you mind show me a diff of your approach? To be\n> honest, this disjoint here looks logically cleaner to me.\n\nHere's the diff I had in mind:\n\n--- >8 ---\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex df1f69f1a7..111aafb69a 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -455,10 +455,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (ignore_sparse &&\n \t\t core_apply_sparse_checkout &&\n \t\t core_sparse_checkout_cone) {\n-\n-\t\t\t/* from out-of-cone to in-cone */\n \t\t\tif ((mode & SPARSE) &&\n \t\t\t path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\t/* from out-of-cone to in-cone */\n \t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n \t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n \n@@ -466,13 +465,10 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n \t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n-\t\t\t\tcontinue;\n-\t\t\t}\n-\n-\t\t\t/* from in-cone to out-of-cone */\n-\t\t\tif ((dst_mode & SKIP_WORKTREE_DIR) &&\n-\t\t\t !(mode & SPARSE) &&\n-\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n+\t\t\t\t !(mode & SPARSE) &&\n+\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\t/* from in-cone to out-of-cone */\n \t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n \t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n \t\t\t\tchar *src_dir = dirname(xstrdup(src));\n\n--- >8 ---\n\nI agree with you that the whitespace breaking the two cases is nice,\nbut relying on that \"continue;\" to keep these cases disjoint is easy\nto miss and I'd rather let the code be clearer about the cases.\n\nThanks,\n-Stolee\n"},{"id":"460610","messageId":"1fb0b736-280f-01bf-bb7c-48e8c9b060cb@gmail.com","threadId":"58187","inReplyTo":"d30a358e-c9f0-55fa-8c8b-37f0cb0d7eb3@github.com","subject":"Re: [PATCH v1 6/7] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-04T08:40:13Z","receivedAt":"2022-08-04T08:40:25Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 7/20/2022 2:14 AM, Derrick Stolee wrote:\n >> +                if ((mode & INDEX) && is_empty_dir(src_dir))\n >> +                    rmdir_or_warn(src_dir);\n >\n > This is an interesting cleanup of the first-level directory. Should it be\n > recursive and clean up an entire chain of paths?\n\nIndeed. I'm planning to use an array to record the possible `src_dir`,\ni.e. WORKING_DIRECTORYs in `git-mv`, and use a loop to cleanup the\nempty directories recursively.\n\n--\nThanks,\nShaoxuan\n"},{"id":"460685","messageId":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 0/9] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:19Z","receivedAt":"2022-08-05T03:05:56Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"## Changes since PATCH v1 ##\n\n1. Change \"grep -x\" to ^ and $ in t7002 test, and remove some\n useless \"-x\" (lines that are *not* ambiguous).\n\n2. Rename check_dir_in_index() to empty_dir_has_sparse_contents()\n and add more precise documentation.\n\n3. Reverse return values from empty_dir_has_sparse_contents() to\n comply with the method's name.\n\n4. Make some commit messages more natural/fluent/without typos.\n\n5. Split commit \"mv: from in-cone to out-of-cone\" to two commits,\n one does in-to-out functionality, the other one does advice.\n\n6. Take care a few memory leaks (`xstrdup`s).\n\n7. Add a new way to recursively cleanup empty WORKING_DIRECTORY.\n\n## leftover bits ##\n\nI decided *not* to solve these bits in v2, but in a future v3,\nso things can be handled one at a time, not exceeding my capability.\n\n1. When trying to move from-in-to-out, the mechanism assumes that\n the is out-of-cone, and by cone-mode definition,\n is basically an invisible directory \n (SKIP_WORKTREE_DIR). However, the dirty move mechanism materializes\n the moved file to make sure the information is not lost, and this\n mechanism brings the invisible directory back into the worktree.\n Hence, if we want to perform a second move from-in-to-out, the\n assumption that is not on-disk is broken, and\n everything follows breaks too. \n\n2. A logic loophole introduced in the previous out-to-in patch,\n especially in b91a2b6594 (mv: add check_dir_in_index() and solve \n general dir check issue). Please detach your head to b91a2b6594\n for context. Copy and paste: \n \n git switch b91a2b6594\n \n When moving from out-to-in, is an invisible \n SKIP_WORKTREE_DIR. Line 226 uses `lstat()` to make\n sure the is not on-disk; then line 233-237 checks if\n is a SKIP_WORKTREE_DIR. If the check passes, line 236\n jump to line 276 and try to verify is a directory using\n `S_ISDIR`. However, because the prerequisite is that the line 226\n `lstat()` fails so we can go through the steps said above; and when\n it fails, the `st` stat, especially the `st_mode` member, is either\n an uninitialized chunk of garbage, or the result from previous\n `lstat()`. In this case, `st_mode` *is* from a previous `lstat()`,\n on line 205. This `lstat()` is testing the , which,\n under the out-to-in situation, is always an on-disk directory.\n Thus, by a series of coincidence, the `S_ISDIR()` on line 276\n succeed and everything *looks* OK test-wise. But clearly the `st`\n at this point is an impostor: it does not reflect the actual stat\n situation of and it luckily slips through.\n\n This needs to be fixed.\n\n## PATCH v1 info ##\n\nThis is a sister series to the previous \"from out-of-cone to in-cone\" [1]\nseries. This series is trying to make the opposite operation possible for\n'mv', namely move , which is in-cone, to , which is\nout-of-cone.\n\nOther than the main task, there are also some minor fixes done.\n\n[1] https://lore.kernel.org/git/20220331091755.385961-1-shaoxuan.yuan02@gmail.com/\n\nShaoxuan Yuan (9):\n t7002: add tests for moving from in-cone to out-of-cone\n mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()\n mv: free the *with_slash in check_dir_in_index()\n mv: check if is a SKIP_WORKTREE_DIR\n mv: remove BOTH from enum update_mode\n mv: from in-cone to out-of-cone\n mv: cleanup empty WORKING_DIRECTORY\n advice.h: add advise_on_moving_dirty_path()\n mv: check overwrite for in-to-out move\n\n advice.c | 19 +++++\n advice.h | 1 +\n builtin/mv.c | 143 ++++++++++++++++++++++++++------\n t/t7002-mv-sparse-checkout.sh | 150 +++++++++++++++++++++++++++++++++-\n 4 files changed, 286 insertions(+), 27 deletions(-)\n\nRange-diff against v1:\n 1: 31d1136d40 ! 1: 5de0a2585f t7002: add tests for moving from in-cone to out-of-cone\n @@ Commit message\n A dirty move should move the to the , both in the\n working tree and the index, but should *not* remove the resulted path\n from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n - Instead advise user to \"git add\" this path and run \"git sparse-checkout\n - reapply\" to re-sparsify that path.\n \n Also make sure that if exists in the index (existing\n check for if is in the worktree is not enough in\n in-to-out moves), warn user against the overwrite. And Git should force\n the overwrite when supplied with -f or --force.\n \n + Helped-by: Derrick Stolee \n + Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n \n ## t/t7002-mv-sparse-checkout.sh ##\n -@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'setup' \"\n - \tupdated in the index:\n - \tEOF\n - \n --\tcat >sparse_hint <<-EOF\n -+\tcat >sparse_hint <<-EOF &&\n - \thint: If you intend to update such entries, try one of the following:\n - \thint: * Use the --sparse option.\n - \thint: * Disable or modify the sparsity rules.\n - \thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n - \tEOF\n -+\n -+\tcat >dirty_error_header <<-EOF &&\n -+\tThe following paths have been moved outside the\n -+\tsparse-checkout definition but are not sparse due to local\n -+\tmodifications.\n -+\tEOF\n -+\n -+\tcat >dirty_hint <<-EOF\n -+\thint: To correct the sparsity of these paths, do the following:\n -+\thint: * Use \\\"git add --sparse \\\" to update the index\n -+\thint: * Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\n -+\thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n -+\tEOF\n - \"\n - \n - test_expect_success 'mv refuses to move sparse-to-sparse' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n +\ttest_path_is_missing sub/d &&\n +\ttest_path_is_missing folder1/d &&\n +\tgit ls-files -t >actual &&\n -+\t! grep -x \"H sub/d\" actual &&\n -+\tgrep -x \"S folder1/d\" actual\n ++\t! grep \"^H sub/d\\$\" actual &&\n ++\tgrep \"S folder1/d\" actual\n +'\n +\n +test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n +\ttest_path_is_missing sub/file1 &&\n +\ttest_path_is_missing folder1/file1 &&\n +\tgit ls-files -t >actual &&\n -+\t! grep -x \"H sub/file1\" actual &&\n -+\tgrep -x \"S folder1/file1\" actual &&\n ++\t! grep \"H sub/file1\" actual &&\n ++\tgrep \"S folder1/file1\" actual &&\n +\n +\t# compare file content before move and after move\n +\techo \"sub/file1 overwrite\" >expect &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n +\ttest_cmp expect stderr &&\n +\n +\tgit mv --sparse sub/d folder1 2>stderr &&\n -+\tcat dirty_error_header >expect &&\n -+\techo \"folder1/d\" >>expect &&\n -+\tcat dirty_hint >>expect &&\n -+\ttest_cmp expect stderr &&\n +\n +\ttest_path_is_missing sub/d &&\n +\ttest_path_is_file folder1/d &&\n +\tgit ls-files -t >actual &&\n -+\t! grep -x \"H sub/d\" actual &&\n -+\tgrep -x \"H folder1/d\" actual\n ++\t! grep \"^H sub/d\\$\" actual &&\n ++\tgrep \"H folder1/d\" actual\n +'\n +\n +test_expect_failure 'move dir from in-cone to out-of-cone' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n +\tgit mv --sparse sub/dir folder1 2>stderr &&\n +\ttest_must_be_empty stderr &&\n +\n -+\ttest_path_is_missing sub/dir &&\n +\ttest_path_is_missing folder1 &&\n +\tgit ls-files -t >actual &&\n -+\t! grep -x \"H sub/dir/e\" actual &&\n -+\tgrep -x \"S folder1/dir/e\" actual\n ++\t! grep \"H sub/dir/e\" actual &&\n ++\tgrep \"S folder1/dir/e\" actual\n +'\n +\n +test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n +\ttest_cmp expect stderr &&\n +\n +\tgit mv --sparse sub/dir folder1 2>stderr &&\n -+\tcat dirty_error_header >expect &&\n -+\techo \"folder1/dir/e2\" >>expect &&\n -+\techo \"folder1/dir/e3\" >>expect &&\n -+\tcat dirty_hint >>expect &&\n -+\ttest_cmp expect stderr &&\n +\n -+\ttest_path_is_missing sub/dir &&\n +\ttest_path_is_missing folder1/dir/e &&\n +\ttest_path_is_file folder1/dir/e2 &&\n +\ttest_path_is_file folder1/dir/e3 &&\n +\tgit ls-files -t >actual &&\n -+\t! grep -x \"H sub/dir/e\" actual &&\n -+\t! grep -x \"H sub/dir/e2\" actual &&\n -+\t! grep -x \"H sub/dir/e3\" actual &&\n -+\tgrep -x \"S folder1/dir/e\" actual &&\n -+\tgrep -x \"H folder1/dir/e2\" actual &&\n -+\tgrep -x \"H folder1/dir/e3\" actual\n ++\t! grep \"H sub/dir/e\" actual &&\n ++\t! grep \"H sub/dir/e2\" actual &&\n ++\t! grep \"H sub/dir/e3\" actual &&\n ++\tgrep \"S folder1/dir/e\" actual &&\n ++\tgrep \"H folder1/dir/e2\" actual &&\n ++\tgrep \"H folder1/dir/e3\" actual\n +'\n +\n test_done\n 2: df17173219 < -: ---------- mv: add documentation for check_dir_in_index()\n -: ---------- > 2: ebf8382f15 mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()\n 3: 61af1a2546 ! 3: 26871b1070 mv: free the *with_slash in check_dir_in_index()\n @@ Commit message\n \n *with_slash may be a malloc'd pointer, and when it is, free it.\n \n + Helped-by: Derrick Stolee \n + Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n */\n - static int check_dir_in_index(const char *name)\n + static int empty_dir_has_sparse_contents(const char *name)\n {\n -+\tint ret = 1;\n ++\tint ret = 0;\n \tconst char *with_slash = add_slash(name);\n \tint length = strlen(with_slash);\n \n -@@ builtin/mv.c: static int check_dir_in_index(const char *name)\n +@@ builtin/mv.c: static int empty_dir_has_sparse_contents(const char *name)\n \tif (pos < 0) {\n \t\tpos = -pos - 1;\n \t\tif (pos >= the_index.cache_nr)\n --\t\t\treturn 1;\n +-\t\t\treturn 0;\n +\t\t\tgoto free_return;\n \t\tce = active_cache[pos];\n \t\tif (strncmp(with_slash, ce->name, length))\n --\t\t\treturn 1;\n +-\t\t\treturn 0;\n +\t\t\tgoto free_return;\n \t\tif (ce_skip_worktree(ce))\n --\t\t\treturn 0;\n -+\t\t\tret = 0;\n +-\t\t\treturn 1;\n ++\t\t\tret = 1;\n \t}\n --\treturn 1;\n +-\treturn 0;\n +\n +free_return:\n +\tif (with_slash != name)\n 4: c83fa51784 ! 4: 872b3ebdce mv: check if is a SKIP_WORKTREE_DIR\n @@ Commit message\n in-cone to out-of-cone\" patch.\n \n Change the logic so that Git first check if is a directory\n - with all its contents sparsified (a SKIP_WORKTREE_DIR). If yes,\n - then treat as a directory exists in the working tree, and\n - thus using the second form of 'git-mv', i.e. move into this\n - , and mark as a SKIP_WORKTREE_DIR.\n - If no, continue the original checking logic.\n + with all its contents sparsified (a SKIP_WORKTREE_DIR).\n +\n + If is such a sparse directory, then we should modify the\n + index the same way as we would if this were a non-sparse directory. We\n + must be careful to ensure that the is marked with\n + SKIP_WORKTREE_DIR.\n \n Also add a `dst_w_slash` to reuse the result from `add_slash()`, which\n was everywhere and can be simplified.\n \n + Helped-by: Derrick Stolee \n + Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n --\tenum update_mode *modes;\n +\tconst char *dst_w_slash;\n -+\tenum update_mode *modes, dst_mode = 0;\n + \tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n - \tstruct lock_file lock_file = LOCK_INIT;\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \tif (argc == 1 && is_directory(argv[0]) && !is_directory(argv[1]))\n \t\tflags = 0;\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t} else {\n -\t\tif (argc != 1)\n +\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n -+\t\t !check_dir_in_index(dst_w_slash)) {\n ++\t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n +\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n -+\t\t\tdst_mode |= SKIP_WORKTREE_DIR;\n +\t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n -\t\tdestination = dest_path;\n +\t\t} else {\n +\t\t\tdestination = dest_path;\n +\t\t}\n - \t}\n -+\tif (dst_w_slash != dest_path[0])\n ++\t}\n ++\tif (dst_w_slash != dest_path[0]) {\n +\t\tfree((char *)dst_w_slash);\n ++\t\tdst_w_slash = NULL;\n + \t}\n \n \t/* Checking */\n - \tfor (i = 0; i < argc; i++) {\n 5: e50ff529c5 ! 5: df625cf852 mv: remove BOTH from enum update_mode\n @@ Metadata\n ## Commit message ##\n mv: remove BOTH from enum update_mode\n \n - Since BOTH is not practically used anywhere in the code and its meaning\n - is unclear, remove it.\n + Since BOTH is not used anywhere in the code and its meaning is unclear,\n + remove it.\n \n + Helped-by: Derrick Stolee \n + Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n 6: 1ae2e26237 ! 6: e4a4742d8b mv: from in-cone to out-of-cone\n @@ Commit message\n A dirty move should move the to the , both in the\n working tree and the index, but should *not* remove the resulted path\n from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n - Instead advise user to \"git add\" this path and run \"git sparse-checkout\n - reapply\" to re-sparsify that path.\n \n + Helped-by: Derrick Stolee \n Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n \n - ## advice.c ##\n -@@ advice.c: void detach_advice(const char *new_name)\n - \n - \tfprintf(stderr, fmt, new_name);\n - }\n -+\n -+void advise_on_moving_dirty_path(struct string_list *pathspec_list)\n -+{\n -+\tstruct string_list_item *item;\n -+\n -+\tif (!pathspec_list->nr)\n -+\t\treturn;\n -+\n -+\tfprintf(stderr, _(\"The following paths have been moved outside the\\n\"\n -+\t\t\t \"sparse-checkout definition but are not sparse due to local\\n\"\n -+\t\t\t \"modifications.\\n\"));\n -+\tfor_each_string_list_item(item, pathspec_list)\n -+\t\tfprintf(stderr, \"%s\\n\", item->string);\n -+\n -+\tadvise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,\n -+\t\t\t _(\"To correct the sparsity of these paths, do the following:\\n\"\n -+\t\t\t \"* Use \\\"git add --sparse \\\" to update the index\\n\"\n -+\t\t\t \"* Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\"));\n -+}\n -\n - ## advice.h ##\n -@@ advice.h: void NORETURN die_conclude_merge(void);\n - void NORETURN die_ff_impossible(void);\n - void advise_on_updating_sparse_paths(struct string_list *pathspec_list);\n - void detach_advice(const char *new_name);\n -+void advise_on_moving_dirty_path(struct string_list *pathspec_list);\n - \n - #endif /* ADVICE_H */\n -\n ## builtin/mv.c ##\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n + \t};\n + \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n + \tconst char *dst_w_slash;\n +-\tenum update_mode *modes;\n ++\tenum update_mode *modes, dst_mode = 0;\n + \tstruct stat st;\n + \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n \tstruct cache_entry *ce;\n \tstruct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \tgit_config(git_default_config, NULL);\n \n +@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n + \t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n + \t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n + \t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n ++\t\t\tdst_mode = SKIP_WORKTREE_DIR;\n + \t\t} else if (argc != 1) {\n + \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n + \t\t} else {\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n -+\t\tint up_to_date = 0;\n ++\t\tint sparse_and_dirty = 0;\n \t\tstruct checkout state = CHECKOUT_INIT;\n \t\tstate.istate = &the_index;\n \n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n +\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n -+\t\t\tup_to_date = !ce_modified(active_cache[pos], &st, 0);\n ++\t\t\tsparse_and_dirty = ce_modified(active_cache[pos], &st, 0);\n \t\trename_cache_entry_at(pos, dst);\n \n -\t\tif ((mode & SPARSE) &&\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n +\t\tif (ignore_sparse &&\n +\t\t core_apply_sparse_checkout &&\n +\t\t core_sparse_checkout_cone) {\n - \n --\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n --\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n -+\t\t\t/* from out-of-cone to in-cone */\n +\t\t\tif ((mode & SPARSE) &&\n +\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n ++\t\t\t\t/* from out-of-cone to in-cone */\n +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n - \n --\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n --\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n ++\n +\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n +\n +\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n +\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n -+\t\t\t\tcontinue;\n -+\t\t\t}\n -+\n -+\t\t\t/* from in-cone to out-of-cone */\n -+\t\t\tif ((dst_mode & SKIP_WORKTREE_DIR) &&\n -+\t\t\t !(mode & SPARSE) &&\n -+\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n ++\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n ++\t\t\t\t !(mode & SPARSE) &&\n ++\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n ++\t\t\t\t/* from in-cone to out-of-cone */\n +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n -+\t\t\t\tchar *src_dir = dirname(xstrdup(src));\n -+\n -+\t\t\t\tif (up_to_date) {\n + \n +-\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n +-\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n +-\n +-\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n +-\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n ++\t\t\t\t/*\n ++\t\t\t\t * if src is clean, it will suffice to remove it\n ++\t\t\t\t */\n ++\t\t\t\tif (!sparse_and_dirty) {\n +\t\t\t\t\tdst_ce->ce_flags |= CE_SKIP_WORKTREE;\n +\t\t\t\t\tunlink_or_warn(src);\n +\t\t\t\t} else {\n ++\t\t\t\t\t/*\n ++\t\t\t\t\t * if src is dirty, move it to the\n ++\t\t\t\t\t * destination and create leading\n ++\t\t\t\t\t * dirs if necessary\n ++\t\t\t\t\t */\n ++\t\t\t\t\tchar *dst_dup = xstrdup(dst);\n +\t\t\t\t\tstring_list_append(&dirty_paths, dst);\n -+\t\t\t\t\tsafe_create_leading_directories(xstrdup(dst));\n ++\t\t\t\t\tsafe_create_leading_directories(dst_dup);\n ++\t\t\t\t\tFREE_AND_NULL(dst_dup);\n +\t\t\t\t\trename(src, dst);\n +\t\t\t\t}\n -+\t\t\t\tif ((mode & INDEX) && is_empty_dir(src_dir))\n -+\t\t\t\t\trmdir_or_warn(src_dir);\n +\t\t\t}\n \t\t}\n \t}\n \n -+\tif (dirty_paths.nr)\n -+\t\tadvise_on_moving_dirty_path(&dirty_paths);\n -+\n - \tif (gitmodules_modified)\n - \t\tstage_updated_gitmodules(&the_index);\n - \n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdie(_(\"Unable to write new index file\"));\n \n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone\n \tsetup_sparse_checkout &&\n \techo \"modified\" >>sub/d &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n - \tgrep -x \"H folder1/d\" actual\n + \tgrep \"H folder1/d\" actual\n '\n \n -test_expect_failure 'move dir from in-cone to out-of-cone' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move dirty path from in-cone\n \tsetup_sparse_checkout &&\n \n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move dir from in-cone to out-of-cone' '\n - \tgrep -x \"S folder1/dir/e\" actual\n + \tgrep \"S folder1/dir/e\" actual\n '\n \n -test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n -: ---------- > 7: fec084aaae mv: cleanup empty WORKING_DIRECTORY\n -: ---------- > 8: ac5345c013 advice.h: add advise_on_moving_dirty_path()\n 7: 672b4053cf ! 9: 70d5a67731 mv: check overwrite for in-to-out move\n @@ Metadata\n ## Commit message ##\n mv: check overwrite for in-to-out move\n \n - Add checking logic for overwritng when moving from in-cone to\n + Add checking logic for overwriting when moving from in-cone to\n out-of-cone. It is the index version of the original overwrite logic.\n \n + Helped-by: Derrick Stolee \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \n ## t/t7002-mv-sparse-checkout.sh ##\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move clean path from in-cone to out-of-cone' '\n - \tgrep -x \"S folder1/d\" actual\n + \tgrep \"S folder1/d\" actual\n '\n \n -test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n\nbase-commit: 4af7188bc97f70277d0f10d56d5373022b1fa385\n-- \n2.37.0\n\n"},{"id":"460686","messageId":"20220805030528.1535376-2-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 1/9] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:20Z","receivedAt":"2022-08-05T03:05:59Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test that user can move an in-cone \nto out-of-cone when --sparse is supplied.\n\nSuch can be either clean or dirty, and moving it results in\ndifferent behaviors:\n\nA clean move should move the to the , both in the\nworking tree and the index, then remove the resulted path from the\nworking tree, and turn on its CE_SKIP_WORKTREE bit.\n\nA dirty move should move the to the , both in the\nworking tree and the index, but should *not* remove the resulted path\nfrom the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n\nAlso make sure that if exists in the index (existing\ncheck for if is in the worktree is not enough in\nin-to-out moves), warn user against the overwrite. And Git should force\nthe overwrite when supplied with -f or --force.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 122 ++++++++++++++++++++++++++++++++++\n 1 file changed, 122 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 71fe29690f..9b3a9ab4c3 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -290,4 +290,126 @@ test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n \n+test_expect_failure 'move clean path from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv sub/d folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/d folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/d &&\n+\ttest_path_is_missing folder1/d &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"^H sub/d\\$\" actual &&\n+\tgrep \"S folder1/d\" actual\n+'\n+\n+test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"sub/file1 overwrite\" >sub/file1 &&\n+\tgit add sub/file1 &&\n+\n+\ttest_must_fail git mv sub/file1 folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/file1\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_must_fail git mv --sparse sub/file1 folder1 2>stderr &&\n+\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n+\t>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse -f sub/file1 folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/file1 &&\n+\ttest_path_is_missing folder1/file1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/file1\" actual &&\n+\tgrep \"S folder1/file1\" actual &&\n+\n+\t# compare file content before move and after move\n+\techo \"sub/file1 overwrite\" >expect &&\n+\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n+\tgit cat-file blob $(cat oid) >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"modified\" >>sub/d &&\n+\n+\ttest_must_fail git mv sub/d folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/d folder1 2>stderr &&\n+\n+\ttest_path_is_missing sub/d &&\n+\ttest_path_is_file folder1/d &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"^H sub/d\\$\" actual &&\n+\tgrep \"H folder1/d\" actual\n+'\n+\n+test_expect_failure 'move dir from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/dir folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing folder1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/dir/e\" actual &&\n+\tgrep \"S folder1/dir/e\" actual\n+'\n+\n+test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\ttouch sub/dir/e2 sub/dir/e3 &&\n+\tgit add sub/dir/e2 sub/dir/e3 &&\n+\techo \"modified\" >>sub/dir/e2 &&\n+\techo \"modified\" >>sub/dir/e3 &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\techo \"folder1/dir/e2\" >>expect &&\n+\techo \"folder1/dir/e3\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/dir folder1 2>stderr &&\n+\n+\ttest_path_is_missing folder1/dir/e &&\n+\ttest_path_is_file folder1/dir/e2 &&\n+\ttest_path_is_file folder1/dir/e3 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/dir/e\" actual &&\n+\t! grep \"H sub/dir/e2\" actual &&\n+\t! grep \"H sub/dir/e3\" actual &&\n+\tgrep \"S folder1/dir/e\" actual &&\n+\tgrep \"H folder1/dir/e2\" actual &&\n+\tgrep \"H folder1/dir/e3\" actual\n+'\n+\n test_done\n-- \n2.37.0\n\n"},{"id":"460687","messageId":"20220805030528.1535376-3-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 2/9] mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:21Z","receivedAt":"2022-08-05T03:06:03Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Method check_dir_in_index() introduced in b91a2b6594 (mv: add\ncheck_dir_in_index() and solve general dir check issue, 2022-06-30)\ndoes not describe its intent and behavior well.\n\nChange its name to empty_dir_has_sparse_contents(), which more\nprecisely describes its purpose.\n\nReverse the return values, check_dir_in_index() return 0 for success\nand 1 for failure; reverse the values so empty_dir_has_sparse_contents()\nreturn 1 for success and 0 for failure. These values are more intuitive\nbecause 1 usually means \"has\" and 0 means \"not found\".\n\nAlso modify the documentation to better align with the method's\nintent and behavior.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 24 +++++++++++-------------\n 1 file changed, 11 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 4729bb1a1a..7c11b8f995 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -125,15 +125,13 @@ static int index_range_of_same_dir(const char *src, int length,\n }\n \n /*\n- * Check if an out-of-cone directory should be in the index. Imagine this case\n- * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n- * and thus the directory is sparsified.\n- *\n- * Return 0 if such directory exist (i.e. with any of its contained files not\n- * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n- * Return 1 otherwise.\n+ * Given the path of a directory that does not exist on-disk, check whether the\n+ * directory contains any entries in the index with the SKIP_WORKTREE flag\n+ * enabled.\n+ * Return 1 if such index entries exist.\n+ * Return 0 otherwise.\n */\n-static int check_dir_in_index(const char *name)\n+static int empty_dir_has_sparse_contents(const char *name)\n {\n \tconst char *with_slash = add_slash(name);\n \tint length = strlen(with_slash);\n@@ -144,14 +142,14 @@ static int check_dir_in_index(const char *name)\n \tif (pos < 0) {\n \t\tpos = -pos - 1;\n \t\tif (pos >= the_index.cache_nr)\n-\t\t\treturn 1;\n+\t\t\treturn 0;\n \t\tce = active_cache[pos];\n \t\tif (strncmp(with_slash, ce->name, length))\n-\t\t\treturn 1;\n-\t\tif (ce_skip_worktree(ce))\n \t\t\treturn 0;\n+\t\tif (ce_skip_worktree(ce))\n+\t\t\treturn 1;\n \t}\n-\treturn 1;\n+\treturn 0;\n }\n \n int cmd_mv(int argc, const char **argv, const char *prefix)\n@@ -232,7 +230,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tif (pos < 0) {\n \t\t\t\tconst char *src_w_slash = add_slash(src);\n \t\t\t\tif (!path_in_sparse_checkout(src_w_slash, &the_index) &&\n-\t\t\t\t !check_dir_in_index(src)) {\n+\t\t\t\t empty_dir_has_sparse_contents(src)) {\n \t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n \t\t\t\t\tgoto dir_check;\n \t\t\t\t}\n-- \n2.37.0\n\n"},{"id":"460688","messageId":"20220805030528.1535376-4-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 3/9] mv: free the *with_slash in check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:22Z","receivedAt":"2022-08-05T03:06:04Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"*with_slash may be a malloc'd pointer, and when it is, free it.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 13 +++++++++----\n 1 file changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 7c11b8f995..0a999640c9 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -133,6 +133,7 @@ static int index_range_of_same_dir(const char *src, int length,\n */\n static int empty_dir_has_sparse_contents(const char *name)\n {\n+\tint ret = 0;\n \tconst char *with_slash = add_slash(name);\n \tint length = strlen(with_slash);\n \n@@ -142,14 +143,18 @@ static int empty_dir_has_sparse_contents(const char *name)\n \tif (pos < 0) {\n \t\tpos = -pos - 1;\n \t\tif (pos >= the_index.cache_nr)\n-\t\t\treturn 0;\n+\t\t\tgoto free_return;\n \t\tce = active_cache[pos];\n \t\tif (strncmp(with_slash, ce->name, length))\n-\t\t\treturn 0;\n+\t\t\tgoto free_return;\n \t\tif (ce_skip_worktree(ce))\n-\t\t\treturn 1;\n+\t\t\tret = 1;\n \t}\n-\treturn 0;\n+\n+free_return:\n+\tif (with_slash != name)\n+\t\tfree((char *)with_slash);\n+\treturn ret;\n }\n \n int cmd_mv(int argc, const char **argv, const char *prefix)\n-- \n2.37.0\n\n"},{"id":"460689","messageId":"20220805030528.1535376-5-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 4/9] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:23Z","receivedAt":"2022-08-05T03:06:15Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, is assumed to be in the working tree. If it is\nnot found as a directory, then it is determined to be either a regular file\npath, or error out if used under the second form (move into a directory)\nof 'git-mv'. Such behavior is not ideal, mainly because Git does not\nlook into the index for , which could potentially be a\nSKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\nin-cone to out-of-cone\" patch.\n\nChange the logic so that Git first check if is a directory\nwith all its contents sparsified (a SKIP_WORKTREE_DIR).\n\nIf is such a sparse directory, then we should modify the\nindex the same way as we would if this were a non-sparse directory. We\nmust be careful to ensure that the is marked with\nSKIP_WORKTREE_DIR.\n\nAlso add a `dst_w_slash` to reuse the result from `add_slash()`, which\nwas everywhere and can be simplified.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 18 ++++++++++++++----\n 1 file changed, 14 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 0a999640c9..f213a92bf6 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -171,6 +171,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n+\tconst char *dst_w_slash;\n \tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n@@ -201,6 +202,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tif (argc == 1 && is_directory(argv[0]) && !is_directory(argv[1]))\n \t\tflags = 0;\n \tdest_path = internal_prefix_pathspec(prefix, argv + argc, 1, flags);\n+\tdst_w_slash = add_slash(dest_path[0]);\n \tsubmodule_gitfile = xcalloc(argc, sizeof(char *));\n \n \tif (dest_path[0][0] == '\\0')\n@@ -208,12 +210,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n \telse if (!lstat(dest_path[0], &st) &&\n \t\t\tS_ISDIR(st.st_mode)) {\n-\t\tdest_path[0] = add_slash(dest_path[0]);\n-\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n+\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n \t} else {\n-\t\tif (argc != 1)\n+\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n+\t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n+\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n+\t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n-\t\tdestination = dest_path;\n+\t\t} else {\n+\t\t\tdestination = dest_path;\n+\t\t}\n+\t}\n+\tif (dst_w_slash != dest_path[0]) {\n+\t\tfree((char *)dst_w_slash);\n+\t\tdst_w_slash = NULL;\n \t}\n \n \t/* Checking */\n-- \n2.37.0\n\n"},{"id":"460690","messageId":"20220805030528.1535376-6-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 5/9] mv: remove BOTH from enum update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:24Z","receivedAt":"2022-08-05T03:06:35Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Since BOTH is not used anywhere in the code and its meaning is unclear,\nremove it.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex f213a92bf6..1dc55153ed 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -21,7 +21,6 @@ static const char * const builtin_mv_usage[] = {\n };\n \n enum update_mode {\n-\tBOTH = 0,\n \tWORKING_DIRECTORY = (1 << 1),\n \tINDEX = (1 << 2),\n \tSPARSE = (1 << 3),\n-- \n2.37.0\n\n"},{"id":"460691","messageId":"20220805030528.1535376-7-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 6/9] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:25Z","receivedAt":"2022-08-05T03:06:37Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving an in-cone to an out-of-cone \nwas not possible, mainly because such is a directory that\nis not present in the working tree.\n\nChange the behavior so that we can move an in-cone to\nout-of-cone when --sparse is supplied.\n\nSuch can be either clean or dirty, and moving it results in\ndifferent behaviors:\n\nA clean move should move the to the , both in the\nworking tree and the index, then remove the resulted path from the\nworking tree, and turn on its CE_SKIP_WORKTREE bit.\n\nA dirty move should move the to the , both in the\nworking tree and the index, but should *not* remove the resulted path\nfrom the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 55 +++++++++++++++++++++++++++++------\n t/t7002-mv-sparse-checkout.sh | 8 ++---\n 2 files changed, 50 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 1dc55153ed..e4a14aea2d 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -171,12 +171,13 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n \tconst char *dst_w_slash;\n-\tenum update_mode *modes;\n+\tenum update_mode *modes, dst_mode = 0;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n \tstruct cache_entry *ce;\n \tstruct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;\n+\tstruct string_list dirty_paths = STRING_LIST_INIT_NODUP;\n \n \tgit_config(git_default_config, NULL);\n \n@@ -214,6 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n \t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n \t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n+\t\t\tdst_mode = SKIP_WORKTREE_DIR;\n \t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n \t\t} else {\n@@ -408,6 +410,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tint sparse_and_dirty = 0;\n \t\tstruct checkout state = CHECKOUT_INIT;\n \t\tstate.istate = &the_index;\n \n@@ -418,6 +421,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (show_only)\n \t\t\tcontinue;\n \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n+\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n \t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n@@ -437,17 +441,49 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n+\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n+\t\t\tsparse_and_dirty = ce_modified(active_cache[pos], &st, 0);\n \t\trename_cache_entry_at(pos, dst);\n \n-\t\tif ((mode & SPARSE) &&\n-\t\t (path_in_sparse_checkout(dst, &the_index))) {\n-\t\t\tint dst_pos;\n+\t\tif (ignore_sparse &&\n+\t\t core_apply_sparse_checkout &&\n+\t\t core_sparse_checkout_cone) {\n+\t\t\tif ((mode & SPARSE) &&\n+\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\t/* from out-of-cone to in-cone */\n+\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n+\n+\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n+\n+\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n+\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n+\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n+\t\t\t\t !(mode & SPARSE) &&\n+\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\t/* from in-cone to out-of-cone */\n+\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n \n-\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n-\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n-\n-\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n-\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n+\t\t\t\t/*\n+\t\t\t\t * if src is clean, it will suffice to remove it\n+\t\t\t\t */\n+\t\t\t\tif (!sparse_and_dirty) {\n+\t\t\t\t\tdst_ce->ce_flags |= CE_SKIP_WORKTREE;\n+\t\t\t\t\tunlink_or_warn(src);\n+\t\t\t\t} else {\n+\t\t\t\t\t/*\n+\t\t\t\t\t * if src is dirty, move it to the\n+\t\t\t\t\t * destination and create leading\n+\t\t\t\t\t * dirs if necessary\n+\t\t\t\t\t */\n+\t\t\t\t\tchar *dst_dup = xstrdup(dst);\n+\t\t\t\t\tstring_list_append(&dirty_paths, dst);\n+\t\t\t\t\tsafe_create_leading_directories(dst_dup);\n+\t\t\t\t\tFREE_AND_NULL(dst_dup);\n+\t\t\t\t\trename(src, dst);\n+\t\t\t\t}\n+\t\t\t}\n \t\t}\n \t}\n \n@@ -459,6 +495,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdie(_(\"Unable to write new index file\"));\n \n \tstring_list_clear(&src_for_dst, 0);\n+\tstring_list_clear(&dirty_paths, 0);\n \tUNLEAK(source);\n \tUNLEAK(dest_path);\n \tfree(submodule_gitfile);\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 9b3a9ab4c3..fc9577b2a6 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -290,7 +290,7 @@ test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n \n-test_expect_failure 'move clean path from in-cone to out-of-cone' '\n+test_expect_success 'move clean path from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -343,7 +343,7 @@ test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n+test_expect_success 'move dirty path from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"modified\" >>sub/d &&\n@@ -363,7 +363,7 @@ test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n \tgrep \"H folder1/d\" actual\n '\n \n-test_expect_failure 'move dir from in-cone to out-of-cone' '\n+test_expect_success 'move dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -382,7 +382,7 @@ test_expect_failure 'move dir from in-cone to out-of-cone' '\n \tgrep \"S folder1/dir/e\" actual\n '\n \n-test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n+test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \ttouch sub/dir/e2 sub/dir/e3 &&\n-- \n2.37.0\n\n"},{"id":"460692","messageId":"20220805030528.1535376-8-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 7/9] mv: cleanup empty WORKING_DIRECTORY","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:26Z","receivedAt":"2022-08-05T03:06:43Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving from-in-to-out may leave an empty \ndirectory on-disk (this kind of directory is marked as\nWORKING_DIRECTORY).\n\nCleanup such directories if they are empty (don't have any entries\nunder them).\n\nModify two tests that take as WORKING_DIRECTORY to test\nthis behavior.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 27 +++++++++++++++++++++++++++\n t/t7002-mv-sparse-checkout.sh | 4 ++++\n 2 files changed, 31 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex e4a14aea2d..a58387a986 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -171,6 +171,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n \tconst char *dst_w_slash;\n+\tconst char **src_dir = NULL;\n+\tint src_dir_nr = 0, src_dir_alloc = 0;\n+\tstruct strbuf a_src_dir = STRBUF_INIT;\n \tenum update_mode *modes, dst_mode = 0;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n@@ -304,6 +307,10 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\t\t/* last - first >= 1 */\n \t\t\tmodes[i] |= WORKING_DIRECTORY;\n+\n+\t\t\tALLOC_GROW(src_dir, src_dir_nr + 1, src_dir_alloc);\n+\t\t\tsrc_dir[src_dir_nr++] = src;\n+\n \t\t\tn = argc + last - first;\n \t\t\tREALLOC_ARRAY(source, n);\n \t\t\tREALLOC_ARRAY(destination, n);\n@@ -487,6 +494,26 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t}\n \t}\n \n+\t/*\n+\t * cleanup the empty src_dirs\n+\t */\n+\tfor (i = 0; i < src_dir_nr; i++) {\n+\t\tint dummy;\n+\t\tstrbuf_addstr(&a_src_dir, src_dir[i]);\n+\t\t/*\n+\t\t * if entries under a_src_dir are all moved away,\n+\t\t * recursively remove a_src_dir to cleanup\n+\t\t */\n+\t\tif (index_range_of_same_dir(a_src_dir.buf, a_src_dir.len,\n+\t\t\t\t\t &dummy, &dummy) < 1) {\n+\t\t\tremove_dir_recursively(&a_src_dir, 0);\n+\t\t}\n+\t\tstrbuf_reset(&a_src_dir);\n+\t}\n+\n+\tstrbuf_release(&a_src_dir);\n+\tfree(src_dir);\n+\n \tif (gitmodules_modified)\n \t\tstage_updated_gitmodules(&the_index);\n \ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex fc9577b2a6..23d0c4cec6 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -366,6 +366,7 @@ test_expect_success 'move dirty path from in-cone to out-of-cone' '\n test_expect_success 'move dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n+\tmkdir sub/dir/deep &&\n \n \ttest_must_fail git mv sub/dir folder1 2>stderr &&\n \tcat sparse_error_header >expect &&\n@@ -376,6 +377,7 @@ test_expect_success 'move dir from in-cone to out-of-cone' '\n \tgit mv --sparse sub/dir folder1 2>stderr &&\n \ttest_must_be_empty stderr &&\n \n+\ttest_path_is_missing sub/dir &&\n \ttest_path_is_missing folder1 &&\n \tgit ls-files -t >actual &&\n \t! grep \"H sub/dir/e\" actual &&\n@@ -385,6 +387,7 @@ test_expect_success 'move dir from in-cone to out-of-cone' '\n test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n+\tmkdir sub/dir/deep &&\n \ttouch sub/dir/e2 sub/dir/e3 &&\n \tgit add sub/dir/e2 sub/dir/e3 &&\n \techo \"modified\" >>sub/dir/e2 &&\n@@ -400,6 +403,7 @@ test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \n \tgit mv --sparse sub/dir folder1 2>stderr &&\n \n+\ttest_path_is_missing sub/dir &&\n \ttest_path_is_missing folder1/dir/e &&\n \ttest_path_is_file folder1/dir/e2 &&\n \ttest_path_is_file folder1/dir/e3 &&\n-- \n2.37.0\n\n"},{"id":"460693","messageId":"20220805030528.1535376-9-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 8/9] advice.h: add advise_on_moving_dirty_path()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:27Z","receivedAt":"2022-08-05T03:06:48Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add an advice.\n\nWhen the user use `git mv --sparse `, Git\nwill warn the user to use `git add --sparse ` then use\n`git sparse-checkout reapply` to apply the sparsity rules.\n\nAdd a few lines to previous \"move dirty path\" tests so we can test\nthis new advice is working.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n advice.c | 19 +++++++++++++++++++\n advice.h | 1 +\n builtin/mv.c | 3 +++\n t/t7002-mv-sparse-checkout.sh | 24 +++++++++++++++++++++++-\n 4 files changed, 46 insertions(+), 1 deletion(-)\n\ndiff --git a/advice.c b/advice.c\nindex 6fda9edbc2..fd18968943 100644\n--- a/advice.c\n+++ b/advice.c\n@@ -261,3 +261,22 @@ void detach_advice(const char *new_name)\n \n \tfprintf(stderr, fmt, new_name);\n }\n+\n+void advise_on_moving_dirty_path(struct string_list *pathspec_list)\n+{\n+\tstruct string_list_item *item;\n+\n+\tif (!pathspec_list->nr)\n+\t\treturn;\n+\n+\tfprintf(stderr, _(\"The following paths have been moved outside the\\n\"\n+\t\t\t \"sparse-checkout definition but are not sparse due to local\\n\"\n+\t\t\t \"modifications.\\n\"));\n+\tfor_each_string_list_item(item, pathspec_list)\n+\t\tfprintf(stderr, \"%s\\n\", item->string);\n+\n+\tadvise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,\n+\t\t\t _(\"To correct the sparsity of these paths, do the following:\\n\"\n+\t\t\t \"* Use \\\"git add --sparse \\\" to update the index\\n\"\n+\t\t\t \"* Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\"));\n+}\ndiff --git a/advice.h b/advice.h\nindex 7ddc6cbc1a..07e0f76833 100644\n--- a/advice.h\n+++ b/advice.h\n@@ -74,5 +74,6 @@ void NORETURN die_conclude_merge(void);\n void NORETURN die_ff_impossible(void);\n void advise_on_updating_sparse_paths(struct string_list *pathspec_list);\n void detach_advice(const char *new_name);\n+void advise_on_moving_dirty_path(struct string_list *pathspec_list);\n \n #endif /* ADVICE_H */\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex a58387a986..765a1e8eb5 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -514,6 +514,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tstrbuf_release(&a_src_dir);\n \tfree(src_dir);\n \n+\tif (dirty_paths.nr)\n+\t\tadvise_on_moving_dirty_path(&dirty_paths);\n+\n \tif (gitmodules_modified)\n \t\tstage_updated_gitmodules(&the_index);\n \ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 23d0c4cec6..f0b32a2f70 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -28,12 +28,25 @@ test_expect_success 'setup' \"\n \tupdated in the index:\n \tEOF\n \n-\tcat >sparse_hint <<-EOF\n+\tcat >sparse_hint <<-EOF &&\n \thint: If you intend to update such entries, try one of the following:\n \thint: * Use the --sparse option.\n \thint: * Disable or modify the sparsity rules.\n \thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n \tEOF\n+\n+\tcat >dirty_error_header <<-EOF &&\n+\tThe following paths have been moved outside the\n+\tsparse-checkout definition but are not sparse due to local\n+\tmodifications.\n+\tEOF\n+\n+\tcat >dirty_hint <<-EOF\n+\thint: To correct the sparsity of these paths, do the following:\n+\thint: * Use \\\"git add --sparse \\\" to update the index\n+\thint: * Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\n+\thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n+\tEOF\n \"\n \n test_expect_success 'mv refuses to move sparse-to-sparse' '\n@@ -355,6 +368,10 @@ test_expect_success 'move dirty path from in-cone to out-of-cone' '\n \ttest_cmp expect stderr &&\n \n \tgit mv --sparse sub/d folder1 2>stderr &&\n+\tcat dirty_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat dirty_hint >>expect &&\n+\ttest_cmp expect stderr &&\n \n \ttest_path_is_missing sub/d &&\n \ttest_path_is_file folder1/d &&\n@@ -402,6 +419,11 @@ test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_cmp expect stderr &&\n \n \tgit mv --sparse sub/dir folder1 2>stderr &&\n+\tcat dirty_error_header >expect &&\n+\techo \"folder1/dir/e2\" >>expect &&\n+\techo \"folder1/dir/e3\" >>expect &&\n+\tcat dirty_hint >>expect &&\n+\ttest_cmp expect stderr &&\n \n \ttest_path_is_missing sub/dir &&\n \ttest_path_is_missing folder1/dir/e &&\n-- \n2.37.0\n\n"},{"id":"460694","messageId":"20220805030528.1535376-10-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220805030528.1535376-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v2 9/9] mv: check overwrite for in-to-out move","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-05T03:05:28Z","receivedAt":"2022-08-05T03:06:58Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add checking logic for overwriting when moving from in-cone to\nout-of-cone. It is the index version of the original overwrite logic.\n\nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 12 ++++++++++++\n t/t7002-mv-sparse-checkout.sh | 2 +-\n 2 files changed, 13 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 765a1e8eb5..70996d582f 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -367,6 +367,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tgoto act_on_entry;\n \t\t}\n \n+\t\tif (ignore_sparse &&\n+\t\t (dst_mode & SKIP_WORKTREE_DIR) &&\n+\t\t index_entry_exists(&the_index, dst, strlen(dst))) {\n+\t\t\tbad = _(\"destination exists in the index\");\n+\t\t\tif (force) {\n+\t\t\t\tif (verbose)\n+\t\t\t\t\twarning(_(\"overwriting '%s'\"), dst);\n+\t\t\t\tbad = NULL;\n+\t\t\t} else {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n \t\t/*\n \t\t * We check if the paths are in the sparse-checkout\n \t\t * definition as a very final check, since that\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex f0b32a2f70..50bcca583c 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -323,7 +323,7 @@ test_expect_success 'move clean path from in-cone to out-of-cone' '\n \tgrep \"S folder1/d\" actual\n '\n \n-test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n+test_expect_success 'move clean path from in-cone to out-of-cone overwrite' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"sub/file1 overwrite\" >sub/file1 &&\n-- \n2.37.0\n\n"},{"id":"460855","messageId":"698b86d0-c906-9a9a-839e-71806ae41f2c@github.com","threadId":"58187","inReplyTo":"20220805030528.1535376-5-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v2 4/9] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-08T23:41:26Z","receivedAt":"2022-08-08T23:41:32Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, is assumed to be in the working tree. If it is\n> not found as a directory, then it is determined to be either a regular file\n> path, or error out if used under the second form (move into a directory)\n> of 'git-mv'. Such behavior is not ideal, mainly because Git does not\n> look into the index for , which could potentially be a\n> SKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\n> in-cone to out-of-cone\" patch.\n> \n> Change the logic so that Git first check if is a directory\n> with all its contents sparsified (a SKIP_WORKTREE_DIR).\n> \n> If is such a sparse directory, then we should modify the\n> index the same way as we would if this were a non-sparse directory. We\n> must be careful to ensure that the is marked with\n> SKIP_WORKTREE_DIR.\n> \n> Also add a `dst_w_slash` to reuse the result from `add_slash()`, which\n> was everywhere and can be simplified.\n\nThis all makes sense. Stepping through the code...\n\n> \n> Helped-by: Derrick Stolee \n> Helped-by: Victoria Dye \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 18 ++++++++++++++----\n> 1 file changed, 14 insertions(+), 4 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 0a999640c9..f213a92bf6 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -171,6 +171,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tOPT_END(),\n> \t};\n> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n> +\tconst char *dst_w_slash;\n> \tenum update_mode *modes;\n> \tstruct stat st;\n> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n> @@ -201,6 +202,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \tif (argc == 1 && is_directory(argv[0]) && !is_directory(argv[1]))\n> \t\tflags = 0;\n> \tdest_path = internal_prefix_pathspec(prefix, argv + argc, 1, flags);\n> +\tdst_w_slash = add_slash(dest_path[0]);\n\n...you pre-compute a reusable 'dst_w_slash' here...\n\n> \tsubmodule_gitfile = xcalloc(argc, sizeof(char *));\n> \n> \tif (dest_path[0][0] == '\\0')\n> @@ -208,12 +210,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> \telse if (!lstat(dest_path[0], &st) &&\n> \t\t\tS_ISDIR(st.st_mode)) {\n> -\t\tdest_path[0] = add_slash(dest_path[0]);\n> -\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> +\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n\n...then remove the in-place 'add_slash()' of 'dest_path[0]' and use\n'dst_w_slash' in 'internal_prefix_pathspec()'. Makes sense.\n\n> \t} else {\n\nThen, this block is reached if 'dest_path' is not '.' and it is not a\ndirectory that exists on disk.\n\nPreviously, reaching this point meant that 'dest_path' *must* refer to a\nfile, not a directory. However, you want to add handling for the case where\n'dst_w_slash' doesn't exist on disk because all of its contents are sparse:\n\n> -\t\tif (argc != 1)\n> +\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n> +\t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n> +\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n\nso the above condition identifies whether 'dest_path[0]' is non-empty in the\nindex, and sets 'destination' accordingly. \n\nIt took me some time to understand what all of these (nested) conditions are\ndoing; one suggestion I have (feel free to ignore it, since it's really just\na matter of stylistic preference) is reduce some duplicate code/simplify the\nchange a bit by moving the sparse directory check into the main \"if-else\"\nblock:\n\n------------->8------------->8------------->8------------->8------------->8-------------\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 4729bb1a1a..1c1b9559f6 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -203,10 +203,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tif (dest_path[0][0] == '\\0')\n \t\t/* special case: \".\" was normalized to \"\" */\n \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n-\telse if (!lstat(dest_path[0], &st) &&\n-\t\t\tS_ISDIR(st.st_mode)) {\n-\t\tdest_path[0] = add_slash(dest_path[0]);\n-\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n+\telse if ((!lstat(dest_path[0], &st) && S_ISDIR(st.st_mode)) ||\n+\t\t (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n+\t\t empty_dir_has_sparse_contents(dst_w_slash))) {\n+\t\t/* directory dest_path[0] exists on-disk or in the index */\n+\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n \t} else {\n \t\tif (argc != 1)\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n\n-------------8<-------------8<-------------8<-------------8<-------------8<-------------\n\nIt doesn't make for the prettiest condition (so your current approach might\nbe better in terms of readability) but, to me, it creates a clearer\ndistinction between the \"if\" and \"else if\" blocks (which handle the case\nwhere 'dest_path[0]' is a directory), and the \"else\" block (which handles\nthe case where 'dest_path[0]' is a file).\n\n> +\t\t} else if (argc != 1) {\n> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n> -\t\tdestination = dest_path;\n> +\t\t} else {\n> +\t\t\tdestination = dest_path;\n> +\t\t}\n> +\t}\n> +\tif (dst_w_slash != dest_path[0]) {\n> +\t\tfree((char *)dst_w_slash);\n> +\t\tdst_w_slash = NULL;\n\nLooks good.\n\n> \t}\n> \n> \t/* Checking */\n\n"},{"id":"460856","messageId":"69039db2-e7b2-f006-63da-e699d302c8ce@github.com","threadId":"58187","inReplyTo":"20220805030528.1535376-4-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v2 3/9] mv: free the *with_slash in check_dir_in_index()","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-08T23:41:49Z","receivedAt":"2022-08-08T23:41:53Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> *with_slash may be a malloc'd pointer, and when it is, free it.\n\nSuper-nit: technically, `with_slash` (no `*`) is how you'd refer to the\npointer. `*` is the dereference operator [1], so `*with_slash` has type\n`const char` and refers to the first character in the `with_slash` string.\n\n[1] https://en.wikipedia.org/wiki/Dereference_operator\n\n> \n> Helped-by: Derrick Stolee \n> Helped-by: Victoria Dye \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 13 +++++++++----\n> 1 file changed, 9 insertions(+), 4 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 7c11b8f995..0a999640c9 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -133,6 +133,7 @@ static int index_range_of_same_dir(const char *src, int length,\n> */\n> static int empty_dir_has_sparse_contents(const char *name)\n> {\n> +\tint ret = 0;\n> \tconst char *with_slash = add_slash(name);\n> \tint length = strlen(with_slash);\n> \n> @@ -142,14 +143,18 @@ static int empty_dir_has_sparse_contents(const char *name)\n> \tif (pos < 0) {\n> \t\tpos = -pos - 1;\n> \t\tif (pos >= the_index.cache_nr)\n> -\t\t\treturn 0;\n> +\t\t\tgoto free_return;\n> \t\tce = active_cache[pos];\n> \t\tif (strncmp(with_slash, ce->name, length))\n> -\t\t\treturn 0;\n> +\t\t\tgoto free_return;\n> \t\tif (ce_skip_worktree(ce))\n> -\t\t\treturn 1;\n> +\t\t\tret = 1;\n> \t}\n> -\treturn 0;\n> +\n> +free_return:\n> +\tif (with_slash != name)\n> +\t\tfree((char *)with_slash);\n> +\treturn ret;\n> }\n> \n> int cmd_mv(int argc, const char **argv, const char *prefix)\n\nThe rest of this looks good, nice catch on the potential memory leak.\n"},{"id":"460857","messageId":"8b5801a3-fa3a-acf4-0c34-a753ab7770ac@github.com","threadId":"58187","inReplyTo":"20220805030528.1535376-10-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v2 9/9] mv: check overwrite for in-to-out move","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-08T23:53:54Z","receivedAt":"2022-08-08T23:54:09Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Add checking logic for overwriting when moving from in-cone to\n> out-of-cone. It is the index version of the original overwrite logic.\n> \n> Helped-by: Derrick Stolee \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 12 ++++++++++++\n> t/t7002-mv-sparse-checkout.sh | 2 +-\n> 2 files changed, 13 insertions(+), 1 deletion(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 765a1e8eb5..70996d582f 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -367,6 +367,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\tgoto act_on_entry;\n> \t\t}\n> \n> +\t\tif (ignore_sparse &&\n\nIf '--sparse' is specified...\n\n> +\t\t (dst_mode & SKIP_WORKTREE_DIR) &&\n\n...and the destination's parent directory is outside the sparse cone...\n\n> +\t\t index_entry_exists(&the_index, dst, strlen(dst))) {\n\n...and the destination file exists in the index, then we're going to be\noverwriting an existing index entry. \n\n> +\t\t\tbad = _(\"destination exists in the index\");\n> +\t\t\tif (force) {\n> +\t\t\t\tif (verbose)\n> +\t\t\t\t\twarning(_(\"overwriting '%s'\"), dst);\n> +\t\t\t\tbad = NULL;\n> +\t\t\t} else {\n> +\t\t\t\tgoto act_on_entry;\n> +\t\t\t}\n> +\t\t}\n\nThe rest of this aligns with what's done for a normal (exists on-disk)\noverwrite. There's not much in the original overwrite-handling logic that\ncan be reused here (it's checking for overwrite based on 'stat' rather than\nthe contents of the index), so code structure-wise this makes sense as a\nstandalone check.\n\nLooking good!\n\n> \t\t/*\n> \t\t * We check if the paths are in the sparse-checkout\n> \t\t * definition as a very final check, since that\n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index f0b32a2f70..50bcca583c 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -323,7 +323,7 @@ test_expect_success 'move clean path from in-cone to out-of-cone' '\n> \tgrep \"S folder1/d\" actual\n> '\n> \n> -test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n> +test_expect_success 'move clean path from in-cone to out-of-cone overwrite' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \techo \"sub/file1 overwrite\" >sub/file1 &&\n\n"},{"id":"460859","messageId":"cb6e6a7b-f5a5-eadb-3de0-a98737932ea8@github.com","threadId":"58187","inReplyTo":"698b86d0-c906-9a9a-839e-71806ae41f2c@github.com","subject":"Re: [PATCH v2 4/9] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-09T00:23:44Z","receivedAt":"2022-08-09T00:23:50Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> Originally, is assumed to be in the working tree. If it is\n>> not found as a directory, then it is determined to be either a regular file\n>> path, or error out if used under the second form (move into a directory)\n>> of 'git-mv'. Such behavior is not ideal, mainly because Git does not\n>> look into the index for , which could potentially be a\n>> SKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\n>> in-cone to out-of-cone\" patch.\n>>\n>> Change the logic so that Git first check if is a directory\n>> with all its contents sparsified (a SKIP_WORKTREE_DIR).\n>>\n>> If is such a sparse directory, then we should modify the\n>> index the same way as we would if this were a non-sparse directory. We\n>> must be careful to ensure that the is marked with\n>> SKIP_WORKTREE_DIR.\n>>\n>> Also add a `dst_w_slash` to reuse the result from `add_slash()`, which\n>> was everywhere and can be simplified.\n> \n> This all makes sense. Stepping through the code...\n> \n>>\n>> Helped-by: Derrick Stolee \n>> Helped-by: Victoria Dye \n>> Signed-off-by: Shaoxuan Yuan \n>> ---\n>> builtin/mv.c | 18 ++++++++++++++----\n>> 1 file changed, 14 insertions(+), 4 deletions(-)\n>>\n>> diff --git a/builtin/mv.c b/builtin/mv.c\n>> index 0a999640c9..f213a92bf6 100644\n>> --- a/builtin/mv.c\n>> +++ b/builtin/mv.c\n>> @@ -171,6 +171,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\tOPT_END(),\n>> \t};\n>> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n>> +\tconst char *dst_w_slash;\n>> \tenum update_mode *modes;\n>> \tstruct stat st;\n>> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n>> @@ -201,6 +202,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \tif (argc == 1 && is_directory(argv[0]) && !is_directory(argv[1]))\n>> \t\tflags = 0;\n>> \tdest_path = internal_prefix_pathspec(prefix, argv + argc, 1, flags);\n>> +\tdst_w_slash = add_slash(dest_path[0]);\n> \n> ...you pre-compute a reusable 'dst_w_slash' here...\n> \n>> \tsubmodule_gitfile = xcalloc(argc, sizeof(char *));\n>> \n>> \tif (dest_path[0][0] == '\\0')\n>> @@ -208,12 +210,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n>> \telse if (!lstat(dest_path[0], &st) &&\n>> \t\t\tS_ISDIR(st.st_mode)) {\n>> -\t\tdest_path[0] = add_slash(dest_path[0]);\n>> -\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n>> +\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n> \n> ...then remove the in-place 'add_slash()' of 'dest_path[0]' and use\n> 'dst_w_slash' in 'internal_prefix_pathspec()'. Makes sense.\n> \n>> \t} else {\n> \n> Then, this block is reached if 'dest_path' is not '.' and it is not a\n> directory that exists on disk.\n> \n> Previously, reaching this point meant that 'dest_path' *must* refer to a\n> file, not a directory. However, you want to add handling for the case where\n> 'dst_w_slash' doesn't exist on disk because all of its contents are sparse:\n> \n>> -\t\tif (argc != 1)\n>> +\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n>> +\t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n>> +\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n> \n> so the above condition identifies whether 'dest_path[0]' is non-empty in the\n> index, and sets 'destination' accordingly. \n> \n> It took me some time to understand what all of these (nested) conditions are\n> doing; one suggestion I have (feel free to ignore it, since it's really just\n> a matter of stylistic preference) is reduce some duplicate code/simplify the\n> change a bit by moving the sparse directory check into the main \"if-else\"\n> block:\n> \n> ------------->8------------->8------------->8------------->8------------->8-------------\n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 4729bb1a1a..1c1b9559f6 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -203,10 +203,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \tif (dest_path[0][0] == '\\0')\n> \t\t/* special case: \".\" was normalized to \"\" */\n> \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> -\telse if (!lstat(dest_path[0], &st) &&\n> -\t\t\tS_ISDIR(st.st_mode)) {\n> -\t\tdest_path[0] = add_slash(dest_path[0]);\n> -\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> +\telse if ((!lstat(dest_path[0], &st) && S_ISDIR(st.st_mode)) ||\n> +\t\t (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n> +\t\t empty_dir_has_sparse_contents(dst_w_slash))) {\n> +\t\t/* directory dest_path[0] exists on-disk or in the index */\n> +\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n> \t} else {\n> \t\tif (argc != 1)\n> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n> \n> -------------8<-------------8<-------------8<-------------8<-------------8<-------------\n> \n> It doesn't make for the prettiest condition (so your current approach might\n> be better in terms of readability) but, to me, it creates a clearer\n> distinction between the \"if\" and \"else if\" blocks (which handle the case\n> where 'dest_path[0]' is a directory), and the \"else\" block (which handles\n> the case where 'dest_path[0]' is a file).\n\nNow that I've read patch 6 [1], I can see that you need the \"sparse\ndirectory\" condition block to stand alone. I think it might still help to\nput that block in the top-level condition:\n\n------------->8------------->8------------->8------------->8------------->8-------------\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 4729bb1a1a..4a16a5e602 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -205,8 +205,10 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n \telse if (!lstat(dest_path[0], &st) &&\n \t\t\tS_ISDIR(st.st_mode)) {\n-\t\tdest_path[0] = add_slash(dest_path[0]);\n-\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n+\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n+\t} else if (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n+\t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n+\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n \t} else {\n \t\tif (argc != 1)\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n-------------8<-------------8<-------------8<-------------8<-------------8<---------\n\n...but, as before, I'm happy with whichever approach you decide to take.\n\n[1] https://lore.kernel.org/git/20220805030528.1535376-7-shaoxuan.yuan02@gmail.com/\n\n> \n>> +\t\t} else if (argc != 1) {\n>> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n>> -\t\tdestination = dest_path;\n>> +\t\t} else {\n>> +\t\t\tdestination = dest_path;\n>> +\t\t}\n>> +\t}\n>> +\tif (dst_w_slash != dest_path[0]) {\n>> +\t\tfree((char *)dst_w_slash);\n>> +\t\tdst_w_slash = NULL;\n> \n> Looks good.\n> \n>> \t}\n>> \n>> \t/* Checking */\n> \n\n"},{"id":"460868","messageId":"bd80881d-b2a3-c220-8f2d-a07a46e14207@github.com","threadId":"58187","inReplyTo":"20220805030528.1535376-2-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v2 1/9] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-09T00:51:24Z","receivedAt":"2022-08-09T00:51:32Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Add corresponding tests to test that user can move an in-cone \n> to out-of-cone when --sparse is supplied.\n> \n> Such can be either clean or dirty, and moving it results in\n> different behaviors:\n> \n> A clean move should move the to the , both in the\n> working tree and the index, then remove the resulted path from the\n> working tree, and turn on its CE_SKIP_WORKTREE bit.\n\nLooking ahead to patch 6, I think the part about \"move it in the working\ntree, then delete from the working tree\" doesn't quite match your\nimplementation. Instead, if I'm not mistaken, what happens is:\n\n1. Move to in the index (do *not* create\n in the worktree)\n2. Delete from the working tree\n\n> \n> A dirty move should move the to the , both in the\n> working tree and the index, but should *not* remove the resulted path\n> from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n\nMakes sense.\n\n> \n> Also make sure that if exists in the index (existing\n> check for if is in the worktree is not enough in\n> in-to-out moves), warn user against the overwrite. And Git should force\n> the overwrite when supplied with -f or --force.\n\nAlso makes sense. \n\n> \n> Helped-by: Derrick Stolee \n> Helped-by: Victoria Dye \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> t/t7002-mv-sparse-checkout.sh | 122 ++++++++++++++++++++++++++++++++++\n> 1 file changed, 122 insertions(+)\n> \n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 71fe29690f..9b3a9ab4c3 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -290,4 +290,126 @@ test_expect_success 'move sparse file to existing destination with --force and -\n> \ttest_cmp expect sub/file1\n> '\n> \n> +test_expect_failure 'move clean path from in-cone to out-of-cone' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\n> +\ttest_must_fail git mv sub/d folder1 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo \"folder1/d\" >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr &&\n> +\n> +\tgit mv --sparse sub/d folder1 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\ttest_path_is_missing sub/d &&\n> +\ttest_path_is_missing folder1/d &&\n> +\tgit ls-files -t >actual &&\n> +\t! grep \"^H sub/d\\$\" actual &&\n> +\tgrep \"S folder1/d\" actual\n> +'\n> +\n> +test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\techo \"sub/file1 overwrite\" >sub/file1 &&\n> +\tgit add sub/file1 &&\n> +\n> +\ttest_must_fail git mv sub/file1 folder1 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo \"folder1/file1\" >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr &&\n> +\n> +\ttest_must_fail git mv --sparse sub/file1 folder1 2>stderr &&\n> +\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n> +\t>expect &&\n> +\ttest_cmp expect stderr &&\n> +\n> +\tgit mv --sparse -f sub/file1 folder1 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\ttest_path_is_missing sub/file1 &&\n> +\ttest_path_is_missing folder1/file1 &&\n> +\tgit ls-files -t >actual &&\n> +\t! grep \"H sub/file1\" actual &&\n> +\tgrep \"S folder1/file1\" actual &&\n> +\n> +\t# compare file content before move and after move\n> +\techo \"sub/file1 overwrite\" >expect &&\n> +\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n> +\tgit cat-file blob $(cat oid) >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n> +test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\techo \"modified\" >>sub/d &&\n> +\n> +\ttest_must_fail git mv sub/d folder1 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo \"folder1/d\" >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr &&\n> +\n> +\tgit mv --sparse sub/d folder1 2>stderr &&\n> +\n> +\ttest_path_is_missing sub/d &&\n> +\ttest_path_is_file folder1/d &&\n> +\tgit ls-files -t >actual &&\n> +\t! grep \"^H sub/d\\$\" actual &&\n> +\tgrep \"H folder1/d\" actual\n> +'\n> +\n> +test_expect_failure 'move dir from in-cone to out-of-cone' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\n> +\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo \"folder1/dir/e\" >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr &&\n> +\n> +\tgit mv --sparse sub/dir folder1 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\ttest_path_is_missing folder1 &&\n> +\tgit ls-files -t >actual &&\n> +\t! grep \"H sub/dir/e\" actual &&\n> +\tgrep \"S folder1/dir/e\" actual\n> +'\n> +\n> +test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\ttouch sub/dir/e2 sub/dir/e3 &&\n> +\tgit add sub/dir/e2 sub/dir/e3 &&\n> +\techo \"modified\" >>sub/dir/e2 &&\n> +\techo \"modified\" >>sub/dir/e3 &&\n> +\n> +\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo \"folder1/dir/e\" >>expect &&\n> +\techo \"folder1/dir/e2\" >>expect &&\n> +\techo \"folder1/dir/e3\" >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr &&\n> +\n> +\tgit mv --sparse sub/dir folder1 2>stderr &&\n> +\n> +\ttest_path_is_missing folder1/dir/e &&\n> +\ttest_path_is_file folder1/dir/e2 &&\n> +\ttest_path_is_file folder1/dir/e3 &&\n> +\tgit ls-files -t >actual &&\n> +\t! grep \"H sub/dir/e\" actual &&\n> +\t! grep \"H sub/dir/e2\" actual &&\n> +\t! grep \"H sub/dir/e3\" actual &&\n> +\tgrep \"S folder1/dir/e\" actual &&\n> +\tgrep \"H folder1/dir/e2\" actual &&\n> +\tgrep \"H folder1/dir/e3\" actual\n> +'\n> +\n\nThere are two other test cases I'd be interested in seeing:\n\n1. Move a (clean or dirty) in-cone source file to an out-of-cone destination\n *file*. For example:\n\n\techo test >sub/dir/file1 && \n\tgit add sub/dir/file1 && \n\tgit mv --sparse sub/dir/file1 folder1/file1\n\n I'm assuming this should behave the same way as show in 'move clean path\n from in-cone to out-of-cone overwrite'. \n\n2. Move a (clean or dirty) in-cone source directory to an out-of-cone\n destination where one or more files in overwrite files in .\n For example, something like:\n\n\techo test >sub/dir/file1 &&\n\tgit add sub/dir/file1 &&\n\tgit mv --sparse sub/dir folder1\n\n I don't have a strong opinion on the behavior (does it fail the whole\n 'mv' operation? move everything except the files that overwrite\n something?), but it would help to have it documented via test here.\n\n> test_done\n\n"},{"id":"460869","messageId":"9718ab8a-5a1a-d93c-ae8f-aa06f6822577@github.com","threadId":"58187","inReplyTo":"20220805030528.1535376-7-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v2 6/9] mv: from in-cone to out-of-cone","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-09T00:53:25Z","receivedAt":"2022-08-09T00:53:30Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, moving an in-cone to an out-of-cone \n> was not possible, mainly because such is a directory that\n> is not present in the working tree.\n> \n> Change the behavior so that we can move an in-cone to\n> out-of-cone when --sparse is supplied.\n> \n> Such can be either clean or dirty, and moving it results in\n> different behaviors:\n> \n> A clean move should move the to the , both in the\n> working tree and the index, then remove the resulted path from the\n> working tree, and turn on its CE_SKIP_WORKTREE bit.\n\nIt looks like this description is the same as what you wrote in patch 1 [1].\nThat's fine with me but, as noted in [2], I wanted to double-check whether\nthe \"move to in worktree, then remove from worktree\" is an\naccurate description of what's happening.\n\n[1] https://lore.kernel.org/git/20220805030528.1535376-2-shaoxuan.yuan02@gmail.com/\n[2] https://lore.kernel.org/git/bd80881d-b2a3-c220-8f2d-a07a46e14207@github.com/\n\n> \n> A dirty move should move the to the , both in the\n> working tree and the index, but should *not* remove the resulted path\n> from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.> \n> Helped-by: Derrick Stolee \n> Helped-by: Victoria Dye \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 55 +++++++++++++++++++++++++++++------\n> t/t7002-mv-sparse-checkout.sh | 8 ++---\n> 2 files changed, 50 insertions(+), 13 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 1dc55153ed..e4a14aea2d 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -171,12 +171,13 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t};\n> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n> \tconst char *dst_w_slash;\n> -\tenum update_mode *modes;\n> +\tenum update_mode *modes, dst_mode = 0;\n> \tstruct stat st;\n> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n> \tstruct lock_file lock_file = LOCK_INIT;\n> \tstruct cache_entry *ce;\n> \tstruct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;\n> +\tstruct string_list dirty_paths = STRING_LIST_INIT_NODUP;\n> \n> \tgit_config(git_default_config, NULL);\n> \n> @@ -214,6 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n> \t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n> \t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n> +\t\t\tdst_mode = SKIP_WORKTREE_DIR;\n> \t\t} else if (argc != 1) {\n> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n> \t\t} else {\n> @@ -408,6 +410,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tconst char *src = source[i], *dst = destination[i];\n> \t\tenum update_mode mode = modes[i];\n> \t\tint pos;\n> +\t\tint sparse_and_dirty = 0;\n> \t\tstruct checkout state = CHECKOUT_INIT;\n> \t\tstate.istate = &the_index;\n> \n> @@ -418,6 +421,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tif (show_only)\n> \t\t\tcontinue;\n> \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> +\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n> \t\t rename(src, dst) < 0) {\n> \t\t\tif (ignore_errors)\n> \t\t\t\tcontinue;\n> @@ -437,17 +441,49 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \n> \t\tpos = cache_name_pos(src, strlen(src));\n> \t\tassert(pos >= 0);\n> +\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n> +\t\t\tsparse_and_dirty = ce_modified(active_cache[pos], &st, 0);\n> \t\trename_cache_entry_at(pos, dst);\n> \n> -\t\tif ((mode & SPARSE) &&\n> -\t\t (path_in_sparse_checkout(dst, &the_index))) {\n> -\t\t\tint dst_pos;\n> +\t\tif (ignore_sparse &&\n> +\t\t core_apply_sparse_checkout &&\n> +\t\t core_sparse_checkout_cone) {\n> +\t\t\tif ((mode & SPARSE) &&\n> +\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\t/* from out-of-cone to in-cone */\n> +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n> +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n> +\n> +\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n> +\n> +\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n> +\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n> +\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n> +\t\t\t\t !(mode & SPARSE) &&\n> +\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\t/* from in-cone to out-of-cone */\n> +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n> +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n\nIt looks like the above conditions assume \"out-of-cone \" and\n\"out-of-cone \" are mutually exclusive. Have you checked what happens\nwhen you try a move like that? \n\nIf the behavior is sensible, it would be nice to add a test (in 't7002'?)\nestablishing that. Otherwise, changing that behavior is reasonably outside\nthe scope of this series, so it's fine with me if you add a either\n\"test_expect_failure\" test, an \"unsupported\" warning message earlier in\n'mv', or even just a \"NEEDSWORK\" comment somewhere around this code.\n\n> \n> -\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n> -\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n> -\n> -\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n> -\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n> +\t\t\t\t/*\n> +\t\t\t\t * if src is clean, it will suffice to remove it\n> +\t\t\t\t */\n> +\t\t\t\tif (!sparse_and_dirty) {\n> +\t\t\t\t\tdst_ce->ce_flags |= CE_SKIP_WORKTREE;\n> +\t\t\t\t\tunlink_or_warn(src);\n> +\t\t\t\t} else {\n> +\t\t\t\t\t/*\n> +\t\t\t\t\t * if src is dirty, move it to the\n> +\t\t\t\t\t * destination and create leading\n> +\t\t\t\t\t * dirs if necessary\n> +\t\t\t\t\t */\n> +\t\t\t\t\tchar *dst_dup = xstrdup(dst);\n> +\t\t\t\t\tstring_list_append(&dirty_paths, dst);\n> +\t\t\t\t\tsafe_create_leading_directories(dst_dup);\n> +\t\t\t\t\tFREE_AND_NULL(dst_dup);\n> +\t\t\t\t\trename(src, dst);\n> +\t\t\t\t}\n> +\t\t\t}\n\nThe rest of this is clearly organized and (as far as I can tell) matches the\nbehavior you established with the tests in patch 1. Nice work!\n\n> \t\t}\n> \t}\n> \n> @@ -459,6 +495,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tdie(_(\"Unable to write new index file\"));\n> \n> \tstring_list_clear(&src_for_dst, 0);\n> +\tstring_list_clear(&dirty_paths, 0);\n> \tUNLEAK(source);\n> \tUNLEAK(dest_path);\n> \tfree(submodule_gitfile);\n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 9b3a9ab4c3..fc9577b2a6 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -290,7 +290,7 @@ test_expect_success 'move sparse file to existing destination with --force and -\n> \ttest_cmp expect sub/file1\n> '\n> \n> -test_expect_failure 'move clean path from in-cone to out-of-cone' '\n> +test_expect_success 'move clean path from in-cone to out-of-cone' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \n> @@ -343,7 +343,7 @@ test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n> \ttest_cmp expect actual\n> '\n> \n> -test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n> +test_expect_success 'move dirty path from in-cone to out-of-cone' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \techo \"modified\" >>sub/d &&\n> @@ -363,7 +363,7 @@ test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n> \tgrep \"H folder1/d\" actual\n> '\n> \n> -test_expect_failure 'move dir from in-cone to out-of-cone' '\n> +test_expect_success 'move dir from in-cone to out-of-cone' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \n> @@ -382,7 +382,7 @@ test_expect_failure 'move dir from in-cone to out-of-cone' '\n> \tgrep \"S folder1/dir/e\" actual\n> '\n> \n> -test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n> +test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \ttouch sub/dir/e2 sub/dir/e3 &&\n\n"},{"id":"460872","messageId":"542b40f5-0c7f-5f3d-eb81-6b8e1ab8924d@gmail.com","threadId":"58187","inReplyTo":"698b86d0-c906-9a9a-839e-71806ae41f2c@github.com","subject":"Re: [PATCH v2 4/9] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T02:31:29Z","receivedAt":"2022-08-09T02:31:37Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\n\nOn 8/9/2022 7:41 AM, Victoria Dye wrote:\n...truncated...\n> It took me some time to understand what all of these (nested) conditions are\n> doing; one suggestion I have (feel free to ignore it, since it's really just\n> a matter of stylistic preference) is reduce some duplicate code/simplify the\n> change a bit by moving the sparse directory check into the main \"if-else\"\n> block:\nYes, I acknowledge this part is cluttered slightly ;)\n> ------------->8------------->8------------->8------------->8------------->8-------------\n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 4729bb1a1a..1c1b9559f6 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -203,10 +203,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \tif (dest_path[0][0] == '\\0')\n> \t\t/* special case: \".\" was normalized to \"\" */\n> \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> -\telse if (!lstat(dest_path[0], &st) &&\n> -\t\t\tS_ISDIR(st.st_mode)) {\n> -\t\tdest_path[0] = add_slash(dest_path[0]);\n> -\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n> +\telse if ((!lstat(dest_path[0], &st) && S_ISDIR(st.st_mode)) ||\n> +\t\t (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n> +\t\t empty_dir_has_sparse_contents(dst_w_slash))) {\n> +\t\t/* directory dest_path[0] exists on-disk or in the index */\n> +\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n> \t} else {\n> \t\tif (argc != 1)\n> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n>\n> -------------8<-------------8<-------------8<-------------8<-------------8<-------------\n>\n> It doesn't make for the prettiest condition (so your current approach might\n> be better in terms of readability) but, to me, it creates a clearer\n> distinction between the \"if\" and \"else if\" blocks (which handle the case\n> where 'dest_path[0]' is a directory), and the \"else\" block (which handles\n> the case where 'dest_path[0]' is a file).\n\nI also find this way clearer! Thanks for the suggestion!\n\n"},{"id":"460873","messageId":"1c8cbb48-cf62-8105-6ef9-19e9eabf1b80@gmail.com","threadId":"58187","inReplyTo":"69039db2-e7b2-f006-63da-e699d302c8ce@github.com","subject":"Re: [PATCH v2 3/9] mv: free the *with_slash in check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T02:33:42Z","receivedAt":"2022-08-09T02:33:49Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\n\nOn 8/9/2022 7:41 AM, Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> *with_slash may be a malloc'd pointer, and when it is, free it.\n> Super-nit: technically, `with_slash` (no `*`) is how you'd refer to the\n> pointer. `*` is the dereference operator [1], so `*with_slash` has type\n> `const char` and refers to the first character in the `with_slash` string.\n>\n> [1] https://en.wikipedia.org/wiki/Dereference_operator\nOh! Thanks, I was completely unaware. Will fix.\n\n--\nThanks,\nShaoxuan\n"},{"id":"460874","messageId":"651d89e2-5282-2cf8-ffc3-8650a023c80a@gmail.com","threadId":"58187","inReplyTo":"bd80881d-b2a3-c220-8f2d-a07a46e14207@github.com","subject":"Re: [PATCH v2 1/9] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T02:55:31Z","receivedAt":"2022-08-09T02:55:42Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\n\nOn 8/9/2022 8:51 AM, Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> Add corresponding tests to test that user can move an in-cone \n>> to out-of-cone when --sparse is supplied.\n>>\n>> Such can be either clean or dirty, and moving it results in\n>> different behaviors:\n>>\n>> A clean move should move the to the , both in the\n>> working tree and the index, then remove the resulted path from the\n>> working tree, and turn on its CE_SKIP_WORKTREE bit.\n> Looking ahead to patch 6, I think the part about \"move it in the working\n> tree, then delete from the working tree\" doesn't quite match your\n> implementation. Instead, if I'm not mistaken, what happens is:\n>\n> 1. Move to in the index (do *not* create\n> in the worktree)\n> 2. Delete from the working tree\nYou are right. The description here seems to be my first in-mind \nimplementation, which\nI wrote down for record. Later I changed the implementation to the one \nyou mentioned\nhere, but somehow forgot to change the description here.\n\nThanks for the catch!\n>> A dirty move should move the to the , both in the\n>> working tree and the index, but should *not* remove the resulted path\n>> from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n> Makes sense.\n>\n>> Also make sure that if exists in the index (existing\n>> check for if is in the worktree is not enough in\n>> in-to-out moves), warn user against the overwrite. And Git should force\n>> the overwrite when supplied with -f or --force.\n> Also makes sense.\n>\n>> Helped-by: Derrick Stolee \n>> Helped-by: Victoria Dye \n>> Signed-off-by: Shaoxuan Yuan \n>> ---\n>> t/t7002-mv-sparse-checkout.sh | 122 ++++++++++++++++++++++++++++++++++\n>> 1 file changed, 122 insertions(+)\n>>\n>> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n>> index 71fe29690f..9b3a9ab4c3 100755\n>> --- a/t/t7002-mv-sparse-checkout.sh\n>> +++ b/t/t7002-mv-sparse-checkout.sh\n>> @@ -290,4 +290,126 @@ test_expect_success 'move sparse file to existing destination with --force and -\n>> \ttest_cmp expect sub/file1\n>> '\n>> \n>> +test_expect_failure 'move clean path from in-cone to out-of-cone' '\n>> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n>> +\tsetup_sparse_checkout &&\n>> +\n>> +\ttest_must_fail git mv sub/d folder1 2>stderr &&\n>> +\tcat sparse_error_header >expect &&\n>> +\techo \"folder1/d\" >>expect &&\n>> +\tcat sparse_hint >>expect &&\n>> +\ttest_cmp expect stderr &&\n>> +\n>> +\tgit mv --sparse sub/d folder1 2>stderr &&\n>> +\ttest_must_be_empty stderr &&\n>> +\n>> +\ttest_path_is_missing sub/d &&\n>> +\ttest_path_is_missing folder1/d &&\n>> +\tgit ls-files -t >actual &&\n>> +\t! grep \"^H sub/d\\$\" actual &&\n>> +\tgrep \"S folder1/d\" actual\n>> +'\n>> +\n>> +test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n>> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n>> +\tsetup_sparse_checkout &&\n>> +\techo \"sub/file1 overwrite\" >sub/file1 &&\n>> +\tgit add sub/file1 &&\n>> +\n>> +\ttest_must_fail git mv sub/file1 folder1 2>stderr &&\n>> +\tcat sparse_error_header >expect &&\n>> +\techo \"folder1/file1\" >>expect &&\n>> +\tcat sparse_hint >>expect &&\n>> +\ttest_cmp expect stderr &&\n>> +\n>> +\ttest_must_fail git mv --sparse sub/file1 folder1 2>stderr &&\n>> +\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n>> +\t>expect &&\n>> +\ttest_cmp expect stderr &&\n>> +\n>> +\tgit mv --sparse -f sub/file1 folder1 2>stderr &&\n>> +\ttest_must_be_empty stderr &&\n>> +\n>> +\ttest_path_is_missing sub/file1 &&\n>> +\ttest_path_is_missing folder1/file1 &&\n>> +\tgit ls-files -t >actual &&\n>> +\t! grep \"H sub/file1\" actual &&\n>> +\tgrep \"S folder1/file1\" actual &&\n>> +\n>> +\t# compare file content before move and after move\n>> +\techo \"sub/file1 overwrite\" >expect &&\n>> +\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n>> +\tgit cat-file blob $(cat oid) >actual &&\n>> +\ttest_cmp expect actual\n>> +'\n>> +\n>> +test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n>> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n>> +\tsetup_sparse_checkout &&\n>> +\techo \"modified\" >>sub/d &&\n>> +\n>> +\ttest_must_fail git mv sub/d folder1 2>stderr &&\n>> +\tcat sparse_error_header >expect &&\n>> +\techo \"folder1/d\" >>expect &&\n>> +\tcat sparse_hint >>expect &&\n>> +\ttest_cmp expect stderr &&\n>> +\n>> +\tgit mv --sparse sub/d folder1 2>stderr &&\n>> +\n>> +\ttest_path_is_missing sub/d &&\n>> +\ttest_path_is_file folder1/d &&\n>> +\tgit ls-files -t >actual &&\n>> +\t! grep \"^H sub/d\\$\" actual &&\n>> +\tgrep \"H folder1/d\" actual\n>> +'\n>> +\n>> +test_expect_failure 'move dir from in-cone to out-of-cone' '\n>> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n>> +\tsetup_sparse_checkout &&\n>> +\n>> +\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n>> +\tcat sparse_error_header >expect &&\n>> +\techo \"folder1/dir/e\" >>expect &&\n>> +\tcat sparse_hint >>expect &&\n>> +\ttest_cmp expect stderr &&\n>> +\n>> +\tgit mv --sparse sub/dir folder1 2>stderr &&\n>> +\ttest_must_be_empty stderr &&\n>> +\n>> +\ttest_path_is_missing folder1 &&\n>> +\tgit ls-files -t >actual &&\n>> +\t! grep \"H sub/dir/e\" actual &&\n>> +\tgrep \"S folder1/dir/e\" actual\n>> +'\n>> +\n>> +test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n>> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n>> +\tsetup_sparse_checkout &&\n>> +\ttouch sub/dir/e2 sub/dir/e3 &&\n>> +\tgit add sub/dir/e2 sub/dir/e3 &&\n>> +\techo \"modified\" >>sub/dir/e2 &&\n>> +\techo \"modified\" >>sub/dir/e3 &&\n>> +\n>> +\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n>> +\tcat sparse_error_header >expect &&\n>> +\techo \"folder1/dir/e\" >>expect &&\n>> +\techo \"folder1/dir/e2\" >>expect &&\n>> +\techo \"folder1/dir/e3\" >>expect &&\n>> +\tcat sparse_hint >>expect &&\n>> +\ttest_cmp expect stderr &&\n>> +\n>> +\tgit mv --sparse sub/dir folder1 2>stderr &&\n>> +\n>> +\ttest_path_is_missing folder1/dir/e &&\n>> +\ttest_path_is_file folder1/dir/e2 &&\n>> +\ttest_path_is_file folder1/dir/e3 &&\n>> +\tgit ls-files -t >actual &&\n>> +\t! grep \"H sub/dir/e\" actual &&\n>> +\t! grep \"H sub/dir/e2\" actual &&\n>> +\t! grep \"H sub/dir/e3\" actual &&\n>> +\tgrep \"S folder1/dir/e\" actual &&\n>> +\tgrep \"H folder1/dir/e2\" actual &&\n>> +\tgrep \"H folder1/dir/e3\" actual\n>> +'\n>> +\n> There are two other test cases I'd be interested in seeing:\n>\n> 1. Move a (clean or dirty) in-cone source file to an out-of-cone destination\n> *file*. For example:\n>\n> \techo test >sub/dir/file1 &&\n> \tgit add sub/dir/file1 &&\n> \tgit mv --sparse sub/dir/file1 folder1/file1\n>\n> I'm assuming this should behave the same way as show in 'move clean path\n> from in-cone to out-of-cone overwrite'.\nOK.\n>\n> 2. Move a (clean or dirty) in-cone source directory to an out-of-cone\n> destination where one or more files in overwrite files in .\n> For example, something like:\n>\n> \techo test >sub/dir/file1 &&\n> \tgit add sub/dir/file1 &&\n> \tgit mv --sparse sub/dir folder1\n>\n> I don't have a strong opinion on the behavior (does it fail the whole\n> 'mv' operation? move everything except the files that overwrite\n> something?), but it would help to have it documented via test here.\nOK. I think it will fail the whole `mv` operation. The program will \nreport unspecified\noverwrite during the early checking phase. The actual \"moving\" phase \nwon't be touched\nat all because Git complains early.\n\n--\nThanks,\nShaoxuan\n\n\n"},{"id":"460875","messageId":"c6888e45-79d6-40b4-dc1c-0454486fc6f1@gmail.com","threadId":"58187","inReplyTo":"9718ab8a-5a1a-d93c-ae8f-aa06f6822577@github.com","subject":"Re: [PATCH v2 6/9] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T03:16:03Z","receivedAt":"2022-08-09T03:16:14Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 8/9/2022 8:53 AM, Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> Originally, moving an in-cone to an out-of-cone \n>> was not possible, mainly because such is a directory that\n>> is not present in the working tree.\n>>\n>> Change the behavior so that we can move an in-cone to\n>> out-of-cone when --sparse is supplied.\n>>\n>> Such can be either clean or dirty, and moving it results in\n>> different behaviors:\n>>\n>> A clean move should move the to the , both in the\n>> working tree and the index, then remove the resulted path from the\n>> working tree, and turn on its CE_SKIP_WORKTREE bit.\n> It looks like this description is the same as what you wrote in patch 1 [1].\n> That's fine with me but, as noted in [2], I wanted to double-check whether\n> the \"move to in worktree, then remove from worktree\" is an\n> accurate description of what's happening.\n>\n> [1] https://lore.kernel.org/git/20220805030528.1535376-2-shaoxuan.yuan02@gmail.com/\n> [2] https://lore.kernel.org/git/bd80881d-b2a3-c220-8f2d-a07a46e14207@github.com/\n\nThis description is incorrect, as I mentioned in a previous email [3] :)\n[3] \nhttps://lore.kernel.org/git/651d89e2-5282-2cf8-ffc3-8650a023c80a@gmail.com/\n\n>> A dirty move should move the to the , both in the\n>> working tree and the index, but should *not* remove the resulted path\n>> from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.>\n>> Helped-by: Derrick Stolee \n>> Helped-by: Victoria Dye \n>> Signed-off-by: Shaoxuan Yuan \n>> ---\n>> builtin/mv.c | 55 +++++++++++++++++++++++++++++------\n>> t/t7002-mv-sparse-checkout.sh | 8 ++---\n>> 2 files changed, 50 insertions(+), 13 deletions(-)\n>>\n>> diff --git a/builtin/mv.c b/builtin/mv.c\n>> index 1dc55153ed..e4a14aea2d 100644\n>> --- a/builtin/mv.c\n>> +++ b/builtin/mv.c\n>> @@ -171,12 +171,13 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t};\n>> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n>> \tconst char *dst_w_slash;\n>> -\tenum update_mode *modes;\n>> +\tenum update_mode *modes, dst_mode = 0;\n>> \tstruct stat st;\n>> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n>> \tstruct lock_file lock_file = LOCK_INIT;\n>> \tstruct cache_entry *ce;\n>> \tstruct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;\n>> +\tstruct string_list dirty_paths = STRING_LIST_INIT_NODUP;\n>> \n>> \tgit_config(git_default_config, NULL);\n>> \n>> @@ -214,6 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n>> \t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n>> \t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n>> +\t\t\tdst_mode = SKIP_WORKTREE_DIR;\n>> \t\t} else if (argc != 1) {\n>> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n>> \t\t} else {\n>> @@ -408,6 +410,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\tconst char *src = source[i], *dst = destination[i];\n>> \t\tenum update_mode mode = modes[i];\n>> \t\tint pos;\n>> +\t\tint sparse_and_dirty = 0;\n>> \t\tstruct checkout state = CHECKOUT_INIT;\n>> \t\tstate.istate = &the_index;\n>> \n>> @@ -418,6 +421,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\tif (show_only)\n>> \t\t\tcontinue;\n>> \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n>> +\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n>> \t\t rename(src, dst) < 0) {\n>> \t\t\tif (ignore_errors)\n>> \t\t\t\tcontinue;\n>> @@ -437,17 +441,49 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \n>> \t\tpos = cache_name_pos(src, strlen(src));\n>> \t\tassert(pos >= 0);\n>> +\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n>> +\t\t\tsparse_and_dirty = ce_modified(active_cache[pos], &st, 0);\n>> \t\trename_cache_entry_at(pos, dst);\n>> \n>> -\t\tif ((mode & SPARSE) &&\n>> -\t\t (path_in_sparse_checkout(dst, &the_index))) {\n>> -\t\t\tint dst_pos;\n>> +\t\tif (ignore_sparse &&\n>> +\t\t core_apply_sparse_checkout &&\n>> +\t\t core_sparse_checkout_cone) {\n>> +\t\t\tif ((mode & SPARSE) &&\n>> +\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n>> +\t\t\t\t/* from out-of-cone to in-cone */\n>> +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n>> +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n>> +\n>> +\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n>> +\n>> +\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n>> +\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n>> +\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n>> +\t\t\t\t !(mode & SPARSE) &&\n>> +\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n>> +\t\t\t\t/* from in-cone to out-of-cone */\n>> +\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n>> +\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n> It looks like the above conditions assume \"out-of-cone \" and\n> \"out-of-cone \" are mutually exclusive. Have you checked what happens\n> when you try a move like that?\n\nDo you mean can   and both be out-of-cone (out-to-out move)?\nIf that's your question, I'm not sure about the answer. As for now, `mv` has\naddressed the following conditions:\n\n1. in-to-in\n2. in-to-out\n3. out-to-in\n\nBut out-to-out is something mysterious at this point.\n\n> If the behavior is sensible, it would be nice to add a test (in 't7002'?)\n> establishing that. Otherwise, changing that behavior is reasonably outside\n> the scope of this series, so it's fine with me if you add a either\n> \"test_expect_failure\" test, an \"unsupported\" warning message earlier in\n> 'mv', or even just a \"NEEDSWORK\" comment somewhere around this code.\n\nYes, I will try test this mode and see where it goes. Based on the test \nresult I'll\ndecide if that's an easy fix or should be marked as \"NEEDSWORK\". I \nprefer to leave\nit to a future patch, though :)\n\nThanks for the catch!\n\n\n"},{"id":"460878","messageId":"399f56dd-b3a7-9510-a45c-9b10a52dea09@gmail.com","threadId":"58187","inReplyTo":"bd80881d-b2a3-c220-8f2d-a07a46e14207@github.com","subject":"Re: [PATCH v2 1/9] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T07:53:45Z","receivedAt":"2022-08-09T07:53:55Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 8/9/2022 8:51 AM, Victoria Dye wrote:\n > There are two other test cases I'd be interested in seeing:\n >\n > 1. Move a (clean or dirty) in-cone source file to an out-of-cone \ndestination\n >    *file*. For example:\n >\n >     echo test >sub/dir/file1 &&\n >     git add sub/dir/file1 &&\n >     git mv --sparse sub/dir/file1 folder1/file1\n >\n >    I'm assuming this should behave the same way as show in 'move \nclean path\n >    from in-cone to out-of-cone overwrite'.\n\nIt's interesting that this test covers an aspect that was not properly\nconsidered. When designing in-to-out, I only thought to\nbe a sparse directory, rather than a possible sparse file, which is\na totally valid argument. Hence, all the logics and mechanisms about\nin-to-out are skipped because Git is completely blind to this file\n.\n\nIt seems like an easy fix, but I was completely unaware.\nThanks for catching this!\n\n"},{"id":"460889","messageId":"3c4ee06e-1145-8efc-ddb9-eeaed81079f5@gmail.com","threadId":"58187","inReplyTo":"651d89e2-5282-2cf8-ffc3-8650a023c80a@gmail.com","subject":"Re: [PATCH v2 1/9] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T11:24:40Z","receivedAt":"2022-08-09T11:24:50Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 8/9/2022 10:55 AM, Shaoxuan Yuan wrote:\n >> 2. Move a (clean or dirty) in-cone source directory to an out-of-cone\n >>     destination where one or more files in overwrite files in \n.\n >>     For example, something like:\n >>\n >>     echo test >sub/dir/file1 &&\n >>     git add sub/dir/file1 &&\n >>     git mv --sparse sub/dir folder1\n >>\n >>     I don't have a strong opinion on the behavior (does it fail the \nwhole\n >>     'mv' operation? move everything except the files that overwrite\n >>     something?), but it would help to have it documented via test here.\n > OK. I think it will fail the whole `mv` operation. The program will \nreport unspecified\n > overwrite during the early checking phase. The actual \"moving\" phase \nwon't be touched\n > at all because Git complains early.\n\nNo, actually this example does not fail anything. Because\n\"sub/dir/file1\" after the move will be \"folder1/dir/file1\", and this\nmove does not involve overwrite at all.\n\nI'm still adding a test case that precisely reflects the main idea\nhere, though. Thanks for the suggestion :)\n\n--\nThanks,\nShaoxuan\n\n"},{"id":"460892","messageId":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220719132809.409247-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 0/9] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:01Z","receivedAt":"2022-08-09T12:09:38Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"## Changes since PATCH v2 ##\n\n1. Rephrase the descriptions for clean moves to match the actual\n implementation.\n\n2. Add test for moving to a file path with overwrite.\n\n3. Add test for moving a directory as , with one of its file\n overwrites file entry in .\n\n4. *with_slash -> with_slash, remove the dereference sign.\n\n5. Take care of in-to-out where is a file path, rather\n than a directory, this is what point 2 is testing.\n\n6. Add a NEEDSWORK doc to address \"out-to-out\" moves.\n\n## Changes since PATCH v1 ##\n\n1. Change \"grep -x\" to ^ and $ in t7002 test, and remove some\n useless \"-x\" (lines that are *not* ambiguous).\n\n2. Rename check_dir_in_index() to empty_dir_has_sparse_contents()\n and add more precise documentation.\n\n3. Reverse return values from empty_dir_has_sparse_contents() to\n comply with the method's name.\n\n4. Make some commit messages more natural/fluent/without typos.\n\n5. Split commit \"mv: from in-cone to out-of-cone\" to two commits,\n one does in-to-out functionality, the other one does advice.\n\n6. Take care a few memory leaks (`xstrdup`s).\n\n7. Add a new way to recursively cleanup empty WORKING_DIRECTORY.\n\n## leftover bits ##\n\nI decided *not* to solve these bits in this series but in a future \none, so things can be handled one at a time without exceeding my\ncapability.\n\n1. When trying to move from-in-to-out, the mechanism assumes that\n the is out-of-cone, and by cone-mode definition,\n is basically an invisible directory \n (SKIP_WORKTREE_DIR). However, the dirty move mechanism materializes\n the moved file to make sure the information is not lost, and this\n mechanism brings the invisible directory back into the worktree.\n Hence, if we want to perform a second move from-in-to-out, the\n assumption that is not on-disk is broken, and\n everything follows breaks too. \n\n2. A logic loophole introduced in the previous out-to-in patch,\n especially in b91a2b6594 (mv: add check_dir_in_index() and solve \n general dir check issue). Please detach your head to b91a2b6594\n for context. Copy and paste: \n \n git switch b91a2b6594\n \n When moving from out-to-in, is an invisible \n SKIP_WORKTREE_DIR. Line 226 uses `lstat()` to make\n sure the is not on-disk; then line 233-237 checks if\n is a SKIP_WORKTREE_DIR. If the check passes, line 236\n jump to line 276 and try to verify is a directory using\n `S_ISDIR`. However, because the prerequisite is that the line 226\n `lstat()` fails so we can go through the steps said above; and when\n it fails, the `st` stat, especially the `st_mode` member, is either\n an uninitialized chunk of garbage, or the result from previous\n `lstat()`. In this case, `st_mode` *is* from a previous `lstat()`,\n on line 205. This `lstat()` is testing the , which,\n under the out-to-in situation, is always an on-disk directory.\n Thus, by a series of coincidence, the `S_ISDIR()` on line 276\n succeed and everything *looks* OK test-wise. But clearly the `st`\n at this point is an impostor: it does not reflect the actual stat\n situation of and it luckily slips through.\n\n This needs to be fixed.\n\n3. A NEEDSWORK added to address \"out-to-out\" move.\n\n## PATCH v1 info ##\n\nThis is a sister series to the previous \"from out-of-cone to in-cone\" [1]\nseries. This series is trying to make the opposite operation possible for\n'mv', namely move , which is in-cone, to , which is\nout-of-cone.\n\nOther than the main task, there are also some minor fixes done.\n\n[1] https://lore.kernel.org/git/20220331091755.385961-1-shaoxuan.yuan02@gmail.com/\n\nShaoxuan Yuan (9):\n t7002: add tests for moving from in-cone to out-of-cone\n mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()\n mv: free the with_slash in check_dir_in_index()\n mv: check if is a SKIP_WORKTREE_DIR\n mv: remove BOTH from enum update_mode\n mv: from in-cone to out-of-cone\n mv: cleanup empty WORKING_DIRECTORY\n advice.h: add advise_on_moving_dirty_path()\n mv: check overwrite for in-to-out move\n\n advice.c | 19 +++\n advice.h | 1 +\n builtin/mv.c | 159 ++++++++++++++++++++----\n t/t7002-mv-sparse-checkout.sh | 226 +++++++++++++++++++++++++++++++++-\n 4 files changed, 378 insertions(+), 27 deletions(-)\n\nRange-diff against v2:\n 1: a8c360c083 ! 1: 8134fa5990 t7002: add tests for moving from in-cone to out-of-cone\n @@ Commit message\n Such can be either clean or dirty, and moving it results in\n different behaviors:\n \n - A clean move should move the to the , both in the\n - working tree and the index, then remove the resulted path from the\n - working tree, and turn on its CE_SKIP_WORKTREE bit.\n + A clean move should move to in the index (do\n + *not* create in the worktree), then delete from\n + the worktree.\n \n A dirty move should move the to the , both in the\n working tree and the index, but should *not* remove the resulted path\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n +\ttest_cmp expect actual\n +'\n +\n ++# This test is testing the same behavior as the\n ++# \"move clean path from in-cone to out-of-cone overwrite\" above.\n ++# The only difference is the changes from \"folder1\" to \"folder1/file1\"\n ++test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite' '\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n ++\tsetup_sparse_checkout &&\n ++\techo \"sub/file1 overwrite\" >sub/file1 &&\n ++\tgit add sub/file1 &&\n ++\n ++\ttest_must_fail git mv sub/file1 folder1/file1 2>stderr &&\n ++\tcat sparse_error_header >expect &&\n ++\techo \"folder1/file1\" >>expect &&\n ++\tcat sparse_hint >>expect &&\n ++\ttest_cmp expect stderr &&\n ++\n ++\ttest_must_fail git mv --sparse sub/file1 folder1/file1 2>stderr &&\n ++\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n ++\t>expect &&\n ++\ttest_cmp expect stderr &&\n ++\n ++\tgit mv --sparse -f sub/file1 folder1/file1 2>stderr &&\n ++\ttest_must_be_empty stderr &&\n ++\n ++\ttest_path_is_missing sub/file1 &&\n ++\ttest_path_is_missing folder1/file1 &&\n ++\tgit ls-files -t >actual &&\n ++\t! grep \"H sub/file1\" actual &&\n ++\tgrep \"S folder1/file1\" actual &&\n ++\n ++\t# compare file content before move and after move\n ++\techo \"sub/file1 overwrite\" >expect &&\n ++\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n ++\tgit cat-file blob $(cat oid) >actual &&\n ++\ttest_cmp expect actual\n ++'\n ++\n ++test_expect_failure 'move directory with one of the files overwrite' '\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n ++\tmkdir -p folder1/dir &&\n ++\ttouch folder1/dir/file1 &&\n ++\tgit add folder1 &&\n ++\tgit sparse-checkout set --cone sub &&\n ++\n ++\techo test >sub/dir/file1 &&\n ++\tgit add sub/dir/file1 &&\n ++\n ++\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n ++\tcat sparse_error_header >expect &&\n ++\techo \"folder1/dir/e\" >>expect &&\n ++\techo \"folder1/dir/file1\" >>expect &&\n ++\tcat sparse_hint >>expect &&\n ++\ttest_cmp expect stderr &&\n ++\n ++\ttest_must_fail git mv --sparse sub/dir folder1 2>stderr &&\n ++\techo \"fatal: destination exists in the index, source=sub/dir/file1, destination=folder1/dir/file1\" \\\n ++\t>expect &&\n ++\ttest_cmp expect stderr &&\n ++\n ++\tgit mv --sparse -f sub/dir folder1 2>stderr &&\n ++\ttest_must_be_empty stderr &&\n ++\n ++\ttest_path_is_missing sub/dir/file1 &&\n ++\ttest_path_is_missing sub/dir/e &&\n ++\ttest_path_is_missing folder1/file1 &&\n ++\tgit ls-files -t >actual &&\n ++\t! grep \"H sub/dir/file1\" actual &&\n ++\t! grep \"H sub/dir/e\" actual &&\n ++\tgrep \"S folder1/dir/file1\" actual &&\n ++\n ++\t# compare file content before move and after move\n ++\techo test >expect &&\n ++\tgit ls-files -s -- folder1/dir/file1 | awk \"{print \\$2}\" >oid &&\n ++\tgit cat-file blob $(cat oid) >actual &&\n ++\ttest_cmp expect actual\n ++'\n ++\n +test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n +\tsetup_sparse_checkout &&\n 2: 725a83c575 = 2: cc5f2770de mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()\n 3: 1d4a0c16a6 ! 3: 1780f36825 mv: free the *with_slash in check_dir_in_index()\n @@ Metadata\n Author: Shaoxuan Yuan \n \n ## Commit message ##\n - mv: free the *with_slash in check_dir_in_index()\n + mv: free the with_slash in check_dir_in_index()\n \n - *with_slash may be a malloc'd pointer, and when it is, free it.\n + with_slash may be a malloc'd pointer, and when it is, free it.\n \n Helped-by: Derrick Stolee \n Helped-by: Victoria Dye \n 4: 1f35f0eb34 = 4: d935bd8d7a mv: check if is a SKIP_WORKTREE_DIR\n 5: 17a871a06b = 5: 26f29df8c8 mv: remove BOTH from enum update_mode\n 6: 32b9f85aa1 ! 6: 2169b873f7 mv: from in-cone to out-of-cone\n @@ Commit message\n Change the behavior so that we can move an in-cone to\n out-of-cone when --sparse is supplied.\n \n + Notice that can also be an out-of-cone file path, rather\n + than a directory.\n +\n Such can be either clean or dirty, and moving it results in\n different behaviors:\n \n - A clean move should move the to the , both in the\n - working tree and the index, then remove the resulted path from the\n - working tree, and turn on its CE_SKIP_WORKTREE bit.\n + A clean move should move to in the index (do\n + *not* create in the worktree), then delete from\n + the worktree.\n \n A dirty move should move the to the , both in the\n working tree and the index, but should *not* remove the resulted path\n from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n \n + Optional reading\n + ================\n +\n + We are strict about cone mode when is a file path.\n + The reason is that some of the previous tests that use no-cone mode in\n + t7002 are keep breaking, mainly because the `dst_mode = SPARSE;` line\n + added in this patch.\n +\n + Most features developed in both \"from-out-to-in\" and \"from-in-to-out\"\n + only care about cone mode situation, as no-cone mode is becoming\n + irrelevant. And because assigning `SPARSE` to `dst_mode` when the\n + repo is in no-cone mode causes miscellaneous bugs, we should just leave\n + this new functionality to be exclusive cone mode and save some time.\n +\n Helped-by: Derrick Stolee \n Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n \t\t} else {\n + \t\t\tdestination = dest_path;\n ++\t\t\t/*\n ++\t\t\t * is a file outside of sparse-checkout\n ++\t\t\t * cone. Insist on cone mode here for backward\n ++\t\t\t * compatibility. We don't want dst_mode to be assigned\n ++\t\t\t * for a file when the repo is using no-cone mode (which\n ++\t\t\t * is deprecated at this point) sparse-checkout. As\n ++\t\t\t * SPARSE here is only considering cone-mode situation.\n ++\t\t\t */\n ++\t\t\tif (!path_in_cone_mode_sparse_checkout(destination[0], &the_index))\n ++\t\t\t\tdst_mode = SPARSE;\n + \t\t}\n + \t}\n + \tif (dst_w_slash != dest_path[0]) {\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (show_only)\n \t\t\tcontinue;\n \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n -+\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n ++\t\t !(dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n \t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n +\t\tif (ignore_sparse &&\n +\t\t core_apply_sparse_checkout &&\n +\t\t core_sparse_checkout_cone) {\n ++\t\t\t/*\n ++\t\t\t * NEEDSWORK: we are *not* paying attention to\n ++\t\t\t * \"out-to-out\" move ( is out-of-cone and\n ++\t\t\t * is out-of-cone) at this point. It\n ++\t\t\t * should be added in a future patch.\n ++\t\t\t */\n +\t\t\tif ((mode & SPARSE) &&\n +\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n +\t\t\t\t/* from out-of-cone to in-cone */\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n +\n +\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n +\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n -+\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n ++\t\t\t} else if ((dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n +\t\t\t\t !(mode & SPARSE) &&\n +\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n +\t\t\t\t/* from in-cone to out-of-cone */\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n -@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move directory with one of the files overwrite' '\n \ttest_cmp expect actual\n '\n \n 7: 449dba3853 = 7: 78a55e2a46 mv: cleanup empty WORKING_DIRECTORY\n 8: 875756480e = 8: 43ce1c07ec advice.h: add advise_on_moving_dirty_path()\n 9: cd5d30505b ! 9: 2e7c6a33c2 mv: check overwrite for in-to-out move\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t}\n \n +\t\tif (ignore_sparse &&\n -+\t\t (dst_mode & SKIP_WORKTREE_DIR) &&\n ++\t\t (dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n +\t\t index_entry_exists(&the_index, dst, strlen(dst))) {\n +\t\t\tbad = _(\"destination exists in the index\");\n +\t\t\tif (force) {\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move clean path from in-cone\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"sub/file1 overwrite\" >sub/file1 &&\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n + # This test is testing the same behavior as the\n + # \"move clean path from in-cone to out-of-cone overwrite\" above.\n + # The only difference is the changes from \"folder1\" to \"folder1/file1\"\n +-test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite' '\n ++test_expect_success 'move clean path from in-cone to out-of-cone file overwrite' '\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tsetup_sparse_checkout &&\n + \techo \"sub/file1 overwrite\" >sub/file1 &&\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite'\n + \ttest_cmp expect actual\n + '\n + \n +-test_expect_failure 'move directory with one of the files overwrite' '\n ++test_expect_success 'move directory with one of the files overwrite' '\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tmkdir -p folder1/dir &&\n + \ttouch folder1/dir/file1 &&\n\nbase-commit: c50926e1f48891e2671e1830dbcd2912a4563450\n-- \n2.37.0\n\n"},{"id":"460893","messageId":"20220809120910.2021413-2-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 1/9] t7002: add tests for moving from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:02Z","receivedAt":"2022-08-09T12:09:39Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test that user can move an in-cone \nto out-of-cone when --sparse is supplied.\n\nSuch can be either clean or dirty, and moving it results in\ndifferent behaviors:\n\nA clean move should move to in the index (do\n*not* create in the worktree), then delete from\nthe worktree.\n\nA dirty move should move the to the , both in the\nworking tree and the index, but should *not* remove the resulted path\nfrom the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n\nAlso make sure that if exists in the index (existing\ncheck for if is in the worktree is not enough in\nin-to-out moves), warn user against the overwrite. And Git should force\nthe overwrite when supplied with -f or --force.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 198 ++++++++++++++++++++++++++++++++++\n 1 file changed, 198 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 71fe29690f..1ac78edde6 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -290,4 +290,202 @@ test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n \n+test_expect_failure 'move clean path from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv sub/d folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/d folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/d &&\n+\ttest_path_is_missing folder1/d &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"^H sub/d\\$\" actual &&\n+\tgrep \"S folder1/d\" actual\n+'\n+\n+test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"sub/file1 overwrite\" >sub/file1 &&\n+\tgit add sub/file1 &&\n+\n+\ttest_must_fail git mv sub/file1 folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/file1\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_must_fail git mv --sparse sub/file1 folder1 2>stderr &&\n+\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n+\t>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse -f sub/file1 folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/file1 &&\n+\ttest_path_is_missing folder1/file1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/file1\" actual &&\n+\tgrep \"S folder1/file1\" actual &&\n+\n+\t# compare file content before move and after move\n+\techo \"sub/file1 overwrite\" >expect &&\n+\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n+\tgit cat-file blob $(cat oid) >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+# This test is testing the same behavior as the\n+# \"move clean path from in-cone to out-of-cone overwrite\" above.\n+# The only difference is the changes from \"folder1\" to \"folder1/file1\"\n+test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"sub/file1 overwrite\" >sub/file1 &&\n+\tgit add sub/file1 &&\n+\n+\ttest_must_fail git mv sub/file1 folder1/file1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/file1\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_must_fail git mv --sparse sub/file1 folder1/file1 2>stderr &&\n+\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n+\t>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse -f sub/file1 folder1/file1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/file1 &&\n+\ttest_path_is_missing folder1/file1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/file1\" actual &&\n+\tgrep \"S folder1/file1\" actual &&\n+\n+\t# compare file content before move and after move\n+\techo \"sub/file1 overwrite\" >expect &&\n+\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n+\tgit cat-file blob $(cat oid) >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'move directory with one of the files overwrite' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir -p folder1/dir &&\n+\ttouch folder1/dir/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\techo test >sub/dir/file1 &&\n+\tgit add sub/dir/file1 &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\techo \"folder1/dir/file1\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\ttest_must_fail git mv --sparse sub/dir folder1 2>stderr &&\n+\techo \"fatal: destination exists in the index, source=sub/dir/file1, destination=folder1/dir/file1\" \\\n+\t>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse -f sub/dir folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing sub/dir/file1 &&\n+\ttest_path_is_missing sub/dir/e &&\n+\ttest_path_is_missing folder1/file1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/dir/file1\" actual &&\n+\t! grep \"H sub/dir/e\" actual &&\n+\tgrep \"S folder1/dir/file1\" actual &&\n+\n+\t# compare file content before move and after move\n+\techo test >expect &&\n+\tgit ls-files -s -- folder1/dir/file1 | awk \"{print \\$2}\" >oid &&\n+\tgit cat-file blob $(cat oid) >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\techo \"modified\" >>sub/d &&\n+\n+\ttest_must_fail git mv sub/d folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/d folder1 2>stderr &&\n+\n+\ttest_path_is_missing sub/d &&\n+\ttest_path_is_file folder1/d &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"^H sub/d\\$\" actual &&\n+\tgrep \"H folder1/d\" actual\n+'\n+\n+test_expect_failure 'move dir from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/dir folder1 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_missing folder1 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/dir/e\" actual &&\n+\tgrep \"S folder1/dir/e\" actual\n+'\n+\n+test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\ttouch sub/dir/e2 sub/dir/e3 &&\n+\tgit add sub/dir/e2 sub/dir/e3 &&\n+\techo \"modified\" >>sub/dir/e2 &&\n+\techo \"modified\" >>sub/dir/e3 &&\n+\n+\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo \"folder1/dir/e\" >>expect &&\n+\techo \"folder1/dir/e2\" >>expect &&\n+\techo \"folder1/dir/e3\" >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr &&\n+\n+\tgit mv --sparse sub/dir folder1 2>stderr &&\n+\n+\ttest_path_is_missing folder1/dir/e &&\n+\ttest_path_is_file folder1/dir/e2 &&\n+\ttest_path_is_file folder1/dir/e3 &&\n+\tgit ls-files -t >actual &&\n+\t! grep \"H sub/dir/e\" actual &&\n+\t! grep \"H sub/dir/e2\" actual &&\n+\t! grep \"H sub/dir/e3\" actual &&\n+\tgrep \"S folder1/dir/e\" actual &&\n+\tgrep \"H folder1/dir/e2\" actual &&\n+\tgrep \"H folder1/dir/e3\" actual\n+'\n+\n test_done\n-- \n2.37.0\n\n"},{"id":"460894","messageId":"20220809120910.2021413-3-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 2/9] mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:03Z","receivedAt":"2022-08-09T12:09:43Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Method check_dir_in_index() introduced in b91a2b6594 (mv: add\ncheck_dir_in_index() and solve general dir check issue, 2022-06-30)\ndoes not describe its intent and behavior well.\n\nChange its name to empty_dir_has_sparse_contents(), which more\nprecisely describes its purpose.\n\nReverse the return values, check_dir_in_index() return 0 for success\nand 1 for failure; reverse the values so empty_dir_has_sparse_contents()\nreturn 1 for success and 0 for failure. These values are more intuitive\nbecause 1 usually means \"has\" and 0 means \"not found\".\n\nAlso modify the documentation to better align with the method's\nintent and behavior.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 24 +++++++++++-------------\n 1 file changed, 11 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 4729bb1a1a..7c11b8f995 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -125,15 +125,13 @@ static int index_range_of_same_dir(const char *src, int length,\n }\n \n /*\n- * Check if an out-of-cone directory should be in the index. Imagine this case\n- * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n- * and thus the directory is sparsified.\n- *\n- * Return 0 if such directory exist (i.e. with any of its contained files not\n- * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n- * Return 1 otherwise.\n+ * Given the path of a directory that does not exist on-disk, check whether the\n+ * directory contains any entries in the index with the SKIP_WORKTREE flag\n+ * enabled.\n+ * Return 1 if such index entries exist.\n+ * Return 0 otherwise.\n */\n-static int check_dir_in_index(const char *name)\n+static int empty_dir_has_sparse_contents(const char *name)\n {\n \tconst char *with_slash = add_slash(name);\n \tint length = strlen(with_slash);\n@@ -144,14 +142,14 @@ static int check_dir_in_index(const char *name)\n \tif (pos < 0) {\n \t\tpos = -pos - 1;\n \t\tif (pos >= the_index.cache_nr)\n-\t\t\treturn 1;\n+\t\t\treturn 0;\n \t\tce = active_cache[pos];\n \t\tif (strncmp(with_slash, ce->name, length))\n-\t\t\treturn 1;\n-\t\tif (ce_skip_worktree(ce))\n \t\t\treturn 0;\n+\t\tif (ce_skip_worktree(ce))\n+\t\t\treturn 1;\n \t}\n-\treturn 1;\n+\treturn 0;\n }\n \n int cmd_mv(int argc, const char **argv, const char *prefix)\n@@ -232,7 +230,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tif (pos < 0) {\n \t\t\t\tconst char *src_w_slash = add_slash(src);\n \t\t\t\tif (!path_in_sparse_checkout(src_w_slash, &the_index) &&\n-\t\t\t\t !check_dir_in_index(src)) {\n+\t\t\t\t empty_dir_has_sparse_contents(src)) {\n \t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n \t\t\t\t\tgoto dir_check;\n \t\t\t\t}\n-- \n2.37.0\n\n"},{"id":"460895","messageId":"20220809120910.2021413-4-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 3/9] mv: free the with_slash in check_dir_in_index()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:04Z","receivedAt":"2022-08-09T12:09:55Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"with_slash may be a malloc'd pointer, and when it is, free it.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 13 +++++++++----\n 1 file changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 7c11b8f995..0a999640c9 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -133,6 +133,7 @@ static int index_range_of_same_dir(const char *src, int length,\n */\n static int empty_dir_has_sparse_contents(const char *name)\n {\n+\tint ret = 0;\n \tconst char *with_slash = add_slash(name);\n \tint length = strlen(with_slash);\n \n@@ -142,14 +143,18 @@ static int empty_dir_has_sparse_contents(const char *name)\n \tif (pos < 0) {\n \t\tpos = -pos - 1;\n \t\tif (pos >= the_index.cache_nr)\n-\t\t\treturn 0;\n+\t\t\tgoto free_return;\n \t\tce = active_cache[pos];\n \t\tif (strncmp(with_slash, ce->name, length))\n-\t\t\treturn 0;\n+\t\t\tgoto free_return;\n \t\tif (ce_skip_worktree(ce))\n-\t\t\treturn 1;\n+\t\t\tret = 1;\n \t}\n-\treturn 0;\n+\n+free_return:\n+\tif (with_slash != name)\n+\t\tfree((char *)with_slash);\n+\treturn ret;\n }\n \n int cmd_mv(int argc, const char **argv, const char *prefix)\n-- \n2.37.0\n\n"},{"id":"460896","messageId":"20220809120910.2021413-6-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 5/9] mv: remove BOTH from enum update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:06Z","receivedAt":"2022-08-09T12:09:58Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Since BOTH is not used anywhere in the code and its meaning is unclear,\nremove it.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex f213a92bf6..1dc55153ed 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -21,7 +21,6 @@ static const char * const builtin_mv_usage[] = {\n };\n \n enum update_mode {\n-\tBOTH = 0,\n \tWORKING_DIRECTORY = (1 << 1),\n \tINDEX = (1 << 2),\n \tSPARSE = (1 << 3),\n-- \n2.37.0\n\n"},{"id":"460897","messageId":"20220809120910.2021413-7-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 6/9] mv: from in-cone to out-of-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:07Z","receivedAt":"2022-08-09T12:09:59Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving an in-cone to an out-of-cone \nwas not possible, mainly because such is a directory that\nis not present in the working tree.\n\nChange the behavior so that we can move an in-cone to\nout-of-cone when --sparse is supplied.\n\nNotice that can also be an out-of-cone file path, rather\nthan a directory.\n\nSuch can be either clean or dirty, and moving it results in\ndifferent behaviors:\n\nA clean move should move to in the index (do\n*not* create in the worktree), then delete from\nthe worktree.\n\nA dirty move should move the to the , both in the\nworking tree and the index, but should *not* remove the resulted path\nfrom the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n\nOptional reading\n================\n\nWe are strict about cone mode when is a file path.\nThe reason is that some of the previous tests that use no-cone mode in\nt7002 are keep breaking, mainly because the `dst_mode = SPARSE;` line\nadded in this patch.\n\nMost features developed in both \"from-out-to-in\" and \"from-in-to-out\"\nonly care about cone mode situation, as no-cone mode is becoming\nirrelevant. And because assigning `SPARSE` to `dst_mode` when the\nrepo is in no-cone mode causes miscellaneous bugs, we should just leave\nthis new functionality to be exclusive cone mode and save some time.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 71 ++++++++++++++++++++++++++++++-----\n t/t7002-mv-sparse-checkout.sh | 8 ++--\n 2 files changed, 66 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 1dc55153ed..6a6420c984 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -171,12 +171,13 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n \tconst char *dst_w_slash;\n-\tenum update_mode *modes;\n+\tenum update_mode *modes, dst_mode = 0;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n \tstruct cache_entry *ce;\n \tstruct string_list only_match_skip_worktree = STRING_LIST_INIT_NODUP;\n+\tstruct string_list dirty_paths = STRING_LIST_INIT_NODUP;\n \n \tgit_config(git_default_config, NULL);\n \n@@ -214,10 +215,21 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n \t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n \t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n+\t\t\tdst_mode = SKIP_WORKTREE_DIR;\n \t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n \t\t} else {\n \t\t\tdestination = dest_path;\n+\t\t\t/*\n+\t\t\t * is a file outside of sparse-checkout\n+\t\t\t * cone. Insist on cone mode here for backward\n+\t\t\t * compatibility. We don't want dst_mode to be assigned\n+\t\t\t * for a file when the repo is using no-cone mode (which\n+\t\t\t * is deprecated at this point) sparse-checkout. As\n+\t\t\t * SPARSE here is only considering cone-mode situation.\n+\t\t\t */\n+\t\t\tif (!path_in_cone_mode_sparse_checkout(destination[0], &the_index))\n+\t\t\t\tdst_mode = SPARSE;\n \t\t}\n \t}\n \tif (dst_w_slash != dest_path[0]) {\n@@ -408,6 +420,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tint sparse_and_dirty = 0;\n \t\tstruct checkout state = CHECKOUT_INIT;\n \t\tstate.istate = &the_index;\n \n@@ -418,6 +431,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (show_only)\n \t\t\tcontinue;\n \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n+\t\t !(dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n \t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n@@ -437,17 +451,55 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n+\t\tif (!(mode & SPARSE) && !lstat(src, &st))\n+\t\t\tsparse_and_dirty = ce_modified(active_cache[pos], &st, 0);\n \t\trename_cache_entry_at(pos, dst);\n \n-\t\tif ((mode & SPARSE) &&\n-\t\t (path_in_sparse_checkout(dst, &the_index))) {\n-\t\t\tint dst_pos;\n+\t\tif (ignore_sparse &&\n+\t\t core_apply_sparse_checkout &&\n+\t\t core_sparse_checkout_cone) {\n+\t\t\t/*\n+\t\t\t * NEEDSWORK: we are *not* paying attention to\n+\t\t\t * \"out-to-out\" move ( is out-of-cone and\n+\t\t\t * is out-of-cone) at this point. It\n+\t\t\t * should be added in a future patch.\n+\t\t\t */\n+\t\t\tif ((mode & SPARSE) &&\n+\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\t/* from out-of-cone to in-cone */\n+\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n+\n+\t\t\t\tdst_ce->ce_flags &= ~CE_SKIP_WORKTREE;\n+\n+\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n+\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n+\t\t\t} else if ((dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n+\t\t\t\t !(mode & SPARSE) &&\n+\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\t/* from in-cone to out-of-cone */\n+\t\t\t\tint dst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tstruct cache_entry *dst_ce = active_cache[dst_pos];\n \n-\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n-\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n-\n-\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n-\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n+\t\t\t\t/*\n+\t\t\t\t * if src is clean, it will suffice to remove it\n+\t\t\t\t */\n+\t\t\t\tif (!sparse_and_dirty) {\n+\t\t\t\t\tdst_ce->ce_flags |= CE_SKIP_WORKTREE;\n+\t\t\t\t\tunlink_or_warn(src);\n+\t\t\t\t} else {\n+\t\t\t\t\t/*\n+\t\t\t\t\t * if src is dirty, move it to the\n+\t\t\t\t\t * destination and create leading\n+\t\t\t\t\t * dirs if necessary\n+\t\t\t\t\t */\n+\t\t\t\t\tchar *dst_dup = xstrdup(dst);\n+\t\t\t\t\tstring_list_append(&dirty_paths, dst);\n+\t\t\t\t\tsafe_create_leading_directories(dst_dup);\n+\t\t\t\t\tFREE_AND_NULL(dst_dup);\n+\t\t\t\t\trename(src, dst);\n+\t\t\t\t}\n+\t\t\t}\n \t\t}\n \t}\n \n@@ -459,6 +511,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdie(_(\"Unable to write new index file\"));\n \n \tstring_list_clear(&src_for_dst, 0);\n+\tstring_list_clear(&dirty_paths, 0);\n \tUNLEAK(source);\n \tUNLEAK(dest_path);\n \tfree(submodule_gitfile);\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 1ac78edde6..d875f492dd 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -290,7 +290,7 @@ test_expect_success 'move sparse file to existing destination with --force and -\n \ttest_cmp expect sub/file1\n '\n \n-test_expect_failure 'move clean path from in-cone to out-of-cone' '\n+test_expect_success 'move clean path from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -419,7 +419,7 @@ test_expect_failure 'move directory with one of the files overwrite' '\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n+test_expect_success 'move dirty path from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"modified\" >>sub/d &&\n@@ -439,7 +439,7 @@ test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n \tgrep \"H folder1/d\" actual\n '\n \n-test_expect_failure 'move dir from in-cone to out-of-cone' '\n+test_expect_success 'move dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -458,7 +458,7 @@ test_expect_failure 'move dir from in-cone to out-of-cone' '\n \tgrep \"S folder1/dir/e\" actual\n '\n \n-test_expect_failure 'move partially-dirty dir from in-cone to out-of-cone' '\n+test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \ttouch sub/dir/e2 sub/dir/e3 &&\n-- \n2.37.0\n\n"},{"id":"460898","messageId":"20220809120910.2021413-8-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 7/9] mv: cleanup empty WORKING_DIRECTORY","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:08Z","receivedAt":"2022-08-09T12:10:00Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving from-in-to-out may leave an empty \ndirectory on-disk (this kind of directory is marked as\nWORKING_DIRECTORY).\n\nCleanup such directories if they are empty (don't have any entries\nunder them).\n\nModify two tests that take as WORKING_DIRECTORY to test\nthis behavior.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 27 +++++++++++++++++++++++++++\n t/t7002-mv-sparse-checkout.sh | 4 ++++\n 2 files changed, 31 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 6a6420c984..32e7ac4896 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -171,6 +171,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n \tconst char *dst_w_slash;\n+\tconst char **src_dir = NULL;\n+\tint src_dir_nr = 0, src_dir_alloc = 0;\n+\tstruct strbuf a_src_dir = STRBUF_INIT;\n \tenum update_mode *modes, dst_mode = 0;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n@@ -314,6 +317,10 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\t\t/* last - first >= 1 */\n \t\t\tmodes[i] |= WORKING_DIRECTORY;\n+\n+\t\t\tALLOC_GROW(src_dir, src_dir_nr + 1, src_dir_alloc);\n+\t\t\tsrc_dir[src_dir_nr++] = src;\n+\n \t\t\tn = argc + last - first;\n \t\t\tREALLOC_ARRAY(source, n);\n \t\t\tREALLOC_ARRAY(destination, n);\n@@ -503,6 +510,26 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t}\n \t}\n \n+\t/*\n+\t * cleanup the empty src_dirs\n+\t */\n+\tfor (i = 0; i < src_dir_nr; i++) {\n+\t\tint dummy;\n+\t\tstrbuf_addstr(&a_src_dir, src_dir[i]);\n+\t\t/*\n+\t\t * if entries under a_src_dir are all moved away,\n+\t\t * recursively remove a_src_dir to cleanup\n+\t\t */\n+\t\tif (index_range_of_same_dir(a_src_dir.buf, a_src_dir.len,\n+\t\t\t\t\t &dummy, &dummy) < 1) {\n+\t\t\tremove_dir_recursively(&a_src_dir, 0);\n+\t\t}\n+\t\tstrbuf_reset(&a_src_dir);\n+\t}\n+\n+\tstrbuf_release(&a_src_dir);\n+\tfree(src_dir);\n+\n \tif (gitmodules_modified)\n \t\tstage_updated_gitmodules(&the_index);\n \ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex d875f492dd..df8c0fa572 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -442,6 +442,7 @@ test_expect_success 'move dirty path from in-cone to out-of-cone' '\n test_expect_success 'move dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n+\tmkdir sub/dir/deep &&\n \n \ttest_must_fail git mv sub/dir folder1 2>stderr &&\n \tcat sparse_error_header >expect &&\n@@ -452,6 +453,7 @@ test_expect_success 'move dir from in-cone to out-of-cone' '\n \tgit mv --sparse sub/dir folder1 2>stderr &&\n \ttest_must_be_empty stderr &&\n \n+\ttest_path_is_missing sub/dir &&\n \ttest_path_is_missing folder1 &&\n \tgit ls-files -t >actual &&\n \t! grep \"H sub/dir/e\" actual &&\n@@ -461,6 +463,7 @@ test_expect_success 'move dir from in-cone to out-of-cone' '\n test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n+\tmkdir sub/dir/deep &&\n \ttouch sub/dir/e2 sub/dir/e3 &&\n \tgit add sub/dir/e2 sub/dir/e3 &&\n \techo \"modified\" >>sub/dir/e2 &&\n@@ -476,6 +479,7 @@ test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \n \tgit mv --sparse sub/dir folder1 2>stderr &&\n \n+\ttest_path_is_missing sub/dir &&\n \ttest_path_is_missing folder1/dir/e &&\n \ttest_path_is_file folder1/dir/e2 &&\n \ttest_path_is_file folder1/dir/e3 &&\n-- \n2.37.0\n\n"},{"id":"460899","messageId":"20220809120910.2021413-9-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 8/9] advice.h: add advise_on_moving_dirty_path()","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:09Z","receivedAt":"2022-08-09T12:10:14Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add an advice.\n\nWhen the user use `git mv --sparse `, Git\nwill warn the user to use `git add --sparse ` then use\n`git sparse-checkout reapply` to apply the sparsity rules.\n\nAdd a few lines to previous \"move dirty path\" tests so we can test\nthis new advice is working.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n advice.c | 19 +++++++++++++++++++\n advice.h | 1 +\n builtin/mv.c | 3 +++\n t/t7002-mv-sparse-checkout.sh | 24 +++++++++++++++++++++++-\n 4 files changed, 46 insertions(+), 1 deletion(-)\n\ndiff --git a/advice.c b/advice.c\nindex 6fda9edbc2..fd18968943 100644\n--- a/advice.c\n+++ b/advice.c\n@@ -261,3 +261,22 @@ void detach_advice(const char *new_name)\n \n \tfprintf(stderr, fmt, new_name);\n }\n+\n+void advise_on_moving_dirty_path(struct string_list *pathspec_list)\n+{\n+\tstruct string_list_item *item;\n+\n+\tif (!pathspec_list->nr)\n+\t\treturn;\n+\n+\tfprintf(stderr, _(\"The following paths have been moved outside the\\n\"\n+\t\t\t \"sparse-checkout definition but are not sparse due to local\\n\"\n+\t\t\t \"modifications.\\n\"));\n+\tfor_each_string_list_item(item, pathspec_list)\n+\t\tfprintf(stderr, \"%s\\n\", item->string);\n+\n+\tadvise_if_enabled(ADVICE_UPDATE_SPARSE_PATH,\n+\t\t\t _(\"To correct the sparsity of these paths, do the following:\\n\"\n+\t\t\t \"* Use \\\"git add --sparse \\\" to update the index\\n\"\n+\t\t\t \"* Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\"));\n+}\ndiff --git a/advice.h b/advice.h\nindex 7ddc6cbc1a..07e0f76833 100644\n--- a/advice.h\n+++ b/advice.h\n@@ -74,5 +74,6 @@ void NORETURN die_conclude_merge(void);\n void NORETURN die_ff_impossible(void);\n void advise_on_updating_sparse_paths(struct string_list *pathspec_list);\n void detach_advice(const char *new_name);\n+void advise_on_moving_dirty_path(struct string_list *pathspec_list);\n \n #endif /* ADVICE_H */\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 32e7ac4896..a396a030c7 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -530,6 +530,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tstrbuf_release(&a_src_dir);\n \tfree(src_dir);\n \n+\tif (dirty_paths.nr)\n+\t\tadvise_on_moving_dirty_path(&dirty_paths);\n+\n \tif (gitmodules_modified)\n \t\tstage_updated_gitmodules(&the_index);\n \ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex df8c0fa572..5e5eb70e7a 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -28,12 +28,25 @@ test_expect_success 'setup' \"\n \tupdated in the index:\n \tEOF\n \n-\tcat >sparse_hint <<-EOF\n+\tcat >sparse_hint <<-EOF &&\n \thint: If you intend to update such entries, try one of the following:\n \thint: * Use the --sparse option.\n \thint: * Disable or modify the sparsity rules.\n \thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n \tEOF\n+\n+\tcat >dirty_error_header <<-EOF &&\n+\tThe following paths have been moved outside the\n+\tsparse-checkout definition but are not sparse due to local\n+\tmodifications.\n+\tEOF\n+\n+\tcat >dirty_hint <<-EOF\n+\thint: To correct the sparsity of these paths, do the following:\n+\thint: * Use \\\"git add --sparse \\\" to update the index\n+\thint: * Use \\\"git sparse-checkout reapply\\\" to apply the sparsity rules\n+\thint: Disable this message with \\\"git config advice.updateSparsePath false\\\"\n+\tEOF\n \"\n \n test_expect_success 'mv refuses to move sparse-to-sparse' '\n@@ -431,6 +444,10 @@ test_expect_success 'move dirty path from in-cone to out-of-cone' '\n \ttest_cmp expect stderr &&\n \n \tgit mv --sparse sub/d folder1 2>stderr &&\n+\tcat dirty_error_header >expect &&\n+\techo \"folder1/d\" >>expect &&\n+\tcat dirty_hint >>expect &&\n+\ttest_cmp expect stderr &&\n \n \ttest_path_is_missing sub/d &&\n \ttest_path_is_file folder1/d &&\n@@ -478,6 +495,11 @@ test_expect_success 'move partially-dirty dir from in-cone to out-of-cone' '\n \ttest_cmp expect stderr &&\n \n \tgit mv --sparse sub/dir folder1 2>stderr &&\n+\tcat dirty_error_header >expect &&\n+\techo \"folder1/dir/e2\" >>expect &&\n+\techo \"folder1/dir/e3\" >>expect &&\n+\tcat dirty_hint >>expect &&\n+\ttest_cmp expect stderr &&\n \n \ttest_path_is_missing sub/dir &&\n \ttest_path_is_missing folder1/dir/e &&\n-- \n2.37.0\n\n"},{"id":"460900","messageId":"20220809120910.2021413-10-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 9/9] mv: check overwrite for in-to-out move","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:10Z","receivedAt":"2022-08-09T12:10:16Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add checking logic for overwriting when moving from in-cone to\nout-of-cone. It is the index version of the original overwrite logic.\n\nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 12 ++++++++++++\n t/t7002-mv-sparse-checkout.sh | 6 +++---\n 2 files changed, 15 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex a396a030c7..2d64c1e80f 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -377,6 +377,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tgoto act_on_entry;\n \t\t}\n \n+\t\tif (ignore_sparse &&\n+\t\t (dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n+\t\t index_entry_exists(&the_index, dst, strlen(dst))) {\n+\t\t\tbad = _(\"destination exists in the index\");\n+\t\t\tif (force) {\n+\t\t\t\tif (verbose)\n+\t\t\t\t\twarning(_(\"overwriting '%s'\"), dst);\n+\t\t\t\tbad = NULL;\n+\t\t\t} else {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n \t\t/*\n \t\t * We check if the paths are in the sparse-checkout\n \t\t * definition as a very final check, since that\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 5e5eb70e7a..26582ae4e5 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -323,7 +323,7 @@ test_expect_success 'move clean path from in-cone to out-of-cone' '\n \tgrep \"S folder1/d\" actual\n '\n \n-test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n+test_expect_success 'move clean path from in-cone to out-of-cone overwrite' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"sub/file1 overwrite\" >sub/file1 &&\n@@ -359,7 +359,7 @@ test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n # This test is testing the same behavior as the\n # \"move clean path from in-cone to out-of-cone overwrite\" above.\n # The only difference is the changes from \"folder1\" to \"folder1/file1\"\n-test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite' '\n+test_expect_success 'move clean path from in-cone to out-of-cone file overwrite' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \techo \"sub/file1 overwrite\" >sub/file1 &&\n@@ -392,7 +392,7 @@ test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite'\n \ttest_cmp expect actual\n '\n \n-test_expect_failure 'move directory with one of the files overwrite' '\n+test_expect_success 'move directory with one of the files overwrite' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir -p folder1/dir &&\n \ttouch folder1/dir/file1 &&\n-- \n2.37.0\n\n"},{"id":"460901","messageId":"20220809120910.2021413-5-shaoxuan.yuan02@gmail.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v3 4/9] mv: check if is a SKIP_WORKTREE_DIR","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-08-09T12:09:05Z","receivedAt":"2022-08-09T12:10:17Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, is assumed to be in the working tree. If it is\nnot found as a directory, then it is determined to be either a regular file\npath, or error out if used under the second form (move into a directory)\nof 'git-mv'. Such behavior is not ideal, mainly because Git does not\nlook into the index for , which could potentially be a\nSKIP_WORKTREE_DIR, which we need to determine for the later \"moving from\nin-cone to out-of-cone\" patch.\n\nChange the logic so that Git first check if is a directory\nwith all its contents sparsified (a SKIP_WORKTREE_DIR).\n\nIf is such a sparse directory, then we should modify the\nindex the same way as we would if this were a non-sparse directory. We\nmust be careful to ensure that the is marked with\nSKIP_WORKTREE_DIR.\n\nAlso add a `dst_w_slash` to reuse the result from `add_slash()`, which\nwas everywhere and can be simplified.\n\nHelped-by: Derrick Stolee \nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 18 ++++++++++++++----\n 1 file changed, 14 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 0a999640c9..f213a92bf6 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -171,6 +171,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n+\tconst char *dst_w_slash;\n \tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n@@ -201,6 +202,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \tif (argc == 1 && is_directory(argv[0]) && !is_directory(argv[1]))\n \t\tflags = 0;\n \tdest_path = internal_prefix_pathspec(prefix, argv + argc, 1, flags);\n+\tdst_w_slash = add_slash(dest_path[0]);\n \tsubmodule_gitfile = xcalloc(argc, sizeof(char *));\n \n \tif (dest_path[0][0] == '\\0')\n@@ -208,12 +210,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n \telse if (!lstat(dest_path[0], &st) &&\n \t\t\tS_ISDIR(st.st_mode)) {\n-\t\tdest_path[0] = add_slash(dest_path[0]);\n-\t\tdestination = internal_prefix_pathspec(dest_path[0], argv, argc, DUP_BASENAME);\n+\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n \t} else {\n-\t\tif (argc != 1)\n+\t\tif (!path_in_sparse_checkout(dst_w_slash, &the_index) &&\n+\t\t empty_dir_has_sparse_contents(dst_w_slash)) {\n+\t\t\tdestination = internal_prefix_pathspec(dst_w_slash, argv, argc, DUP_BASENAME);\n+\t\t} else if (argc != 1) {\n \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n-\t\tdestination = dest_path;\n+\t\t} else {\n+\t\t\tdestination = dest_path;\n+\t\t}\n+\t}\n+\tif (dst_w_slash != dest_path[0]) {\n+\t\tfree((char *)dst_w_slash);\n+\t\tdst_w_slash = NULL;\n \t}\n \n \t/* Checking */\n-- \n2.37.0\n\n"},{"id":"461290","messageId":"52b67242-5d91-5406-9c95-9d8ddc49c079@github.com","threadId":"58187","inReplyTo":"20220809120910.2021413-1-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v3 0/9] mv: from in-cone to out-of-cone","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-08-16T15:48:29Z","receivedAt":"2022-08-16T15:52:09Z","isPatch":true,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> ## Changes since PATCH v2 ##\n> \n> 1. Rephrase the descriptions for clean moves to match the actual\n> implementation.\n> \n> 2. Add test for moving to a file path with overwrite.\n> \n> 3. Add test for moving a directory as , with one of its file\n> overwrites file entry in .\n> \n> 4. *with_slash -> with_slash, remove the dereference sign.\n> \n> 5. Take care of in-to-out where is a file path, rather\n> than a directory, this is what point 2 is testing.\n> \n> 6. Add a NEEDSWORK doc to address \"out-to-out\" moves.\n> \n> ## Changes since PATCH v1 ##\n> \n> 1. Change \"grep -x\" to ^ and $ in t7002 test, and remove some\n> useless \"-x\" (lines that are *not* ambiguous).\n> \n> 2. Rename check_dir_in_index() to empty_dir_has_sparse_contents()\n> and add more precise documentation.\n> \n> 3. Reverse return values from empty_dir_has_sparse_contents() to\n> comply with the method's name.\n> \n> 4. Make some commit messages more natural/fluent/without typos.\n> \n> 5. Split commit \"mv: from in-cone to out-of-cone\" to two commits,\n> one does in-to-out functionality, the other one does advice.\n> \n> 6. Take care a few memory leaks (`xstrdup`s).\n> \n> 7. Add a new way to recursively cleanup empty WORKING_DIRECTORY.\n> \n> ## leftover bits ##\n> \n> I decided *not* to solve these bits in this series but in a future \n> one, so things can be handled one at a time without exceeding my\n> capability.\n> \n> 1. When trying to move from-in-to-out, the mechanism assumes that\n> the is out-of-cone, and by cone-mode definition,\n> is basically an invisible directory \n> (SKIP_WORKTREE_DIR). However, the dirty move mechanism materializes\n> the moved file to make sure the information is not lost, and this\n> mechanism brings the invisible directory back into the worktree.\n> Hence, if we want to perform a second move from-in-to-out, the\n> assumption that is not on-disk is broken, and\n> everything follows breaks too. \n> \n> 2. A logic loophole introduced in the previous out-to-in patch,\n> especially in b91a2b6594 (mv: add check_dir_in_index() and solve \n> general dir check issue). Please detach your head to b91a2b6594\n> for context. Copy and paste: \n> \n> git switch b91a2b6594\n> \n> When moving from out-to-in, is an invisible \n> SKIP_WORKTREE_DIR. Line 226 uses `lstat()` to make\n> sure the is not on-disk; then line 233-237 checks if\n> is a SKIP_WORKTREE_DIR. If the check passes, line 236\n> jump to line 276 and try to verify is a directory using\n> `S_ISDIR`. However, because the prerequisite is that the line 226\n> `lstat()` fails so we can go through the steps said above; and when\n> it fails, the `st` stat, especially the `st_mode` member, is either\n> an uninitialized chunk of garbage, or the result from previous\n> `lstat()`. In this case, `st_mode` *is* from a previous `lstat()`,\n> on line 205. This `lstat()` is testing the , which,\n> under the out-to-in situation, is always an on-disk directory.\n> Thus, by a series of coincidence, the `S_ISDIR()` on line 276\n> succeed and everything *looks* OK test-wise. But clearly the `st`\n> at this point is an impostor: it does not reflect the actual stat\n> situation of and it luckily slips through.\n> \n> This needs to be fixed.\n> \n> 3. A NEEDSWORK added to address \"out-to-out\" move.\n> \n> ## PATCH v1 info ##\n> \n> This is a sister series to the previous \"from out-of-cone to in-cone\" [1]\n> series. This series is trying to make the opposite operation possible for\n> 'mv', namely move , which is in-cone, to , which is\n> out-of-cone.\n> \n> Other than the main task, there are also some minor fixes done.\n> \n> [1] https://lore.kernel.org/git/20220331091755.385961-1-shaoxuan.yuan02@gmail.com/\n> \n\n(Sorry for the slow response) \n\nThe range-diff below looks good to me (w.r.t feedback on the last version).\nI do still find patch 4 [1] a bit cluttered, but it's a minor point and\notherwise everything looks functional; I think this is probably ready for\n'next'.\n\nThanks!\n\n> Shaoxuan Yuan (9):\n> t7002: add tests for moving from in-cone to out-of-cone\n> mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()\n> mv: free the with_slash in check_dir_in_index()\n> mv: check if is a SKIP_WORKTREE_DIR\n> mv: remove BOTH from enum update_mode\n> mv: from in-cone to out-of-cone\n> mv: cleanup empty WORKING_DIRECTORY\n> advice.h: add advise_on_moving_dirty_path()\n> mv: check overwrite for in-to-out move\n> \n> advice.c | 19 +++\n> advice.h | 1 +\n> builtin/mv.c | 159 ++++++++++++++++++++----\n> t/t7002-mv-sparse-checkout.sh | 226 +++++++++++++++++++++++++++++++++-\n> 4 files changed, 378 insertions(+), 27 deletions(-)\n> \n> Range-diff against v2:\n> 1: a8c360c083 ! 1: 8134fa5990 t7002: add tests for moving from in-cone to out-of-cone\n> @@ Commit message\n> Such can be either clean or dirty, and moving it results in\n> different behaviors:\n> \n> - A clean move should move the to the , both in the\n> - working tree and the index, then remove the resulted path from the\n> - working tree, and turn on its CE_SKIP_WORKTREE bit.\n> + A clean move should move to in the index (do\n> + *not* create in the worktree), then delete from\n> + the worktree.\n> \n> A dirty move should move the to the , both in the\n> working tree and the index, but should *not* remove the resulted path\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n> +\ttest_cmp expect actual\n> +'\n> +\n> ++# This test is testing the same behavior as the\n> ++# \"move clean path from in-cone to out-of-cone overwrite\" above.\n> ++# The only difference is the changes from \"folder1\" to \"folder1/file1\"\n> ++test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite' '\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> ++\tsetup_sparse_checkout &&\n> ++\techo \"sub/file1 overwrite\" >sub/file1 &&\n> ++\tgit add sub/file1 &&\n> ++\n> ++\ttest_must_fail git mv sub/file1 folder1/file1 2>stderr &&\n> ++\tcat sparse_error_header >expect &&\n> ++\techo \"folder1/file1\" >>expect &&\n> ++\tcat sparse_hint >>expect &&\n> ++\ttest_cmp expect stderr &&\n> ++\n> ++\ttest_must_fail git mv --sparse sub/file1 folder1/file1 2>stderr &&\n> ++\techo \"fatal: destination exists in the index, source=sub/file1, destination=folder1/file1\" \\\n> ++\t>expect &&\n> ++\ttest_cmp expect stderr &&\n> ++\n> ++\tgit mv --sparse -f sub/file1 folder1/file1 2>stderr &&\n> ++\ttest_must_be_empty stderr &&\n> ++\n> ++\ttest_path_is_missing sub/file1 &&\n> ++\ttest_path_is_missing folder1/file1 &&\n> ++\tgit ls-files -t >actual &&\n> ++\t! grep \"H sub/file1\" actual &&\n> ++\tgrep \"S folder1/file1\" actual &&\n> ++\n> ++\t# compare file content before move and after move\n> ++\techo \"sub/file1 overwrite\" >expect &&\n> ++\tgit ls-files -s -- folder1/file1 | awk \"{print \\$2}\" >oid &&\n> ++\tgit cat-file blob $(cat oid) >actual &&\n> ++\ttest_cmp expect actual\n> ++'\n> ++\n> ++test_expect_failure 'move directory with one of the files overwrite' '\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> ++\tmkdir -p folder1/dir &&\n> ++\ttouch folder1/dir/file1 &&\n> ++\tgit add folder1 &&\n> ++\tgit sparse-checkout set --cone sub &&\n> ++\n> ++\techo test >sub/dir/file1 &&\n> ++\tgit add sub/dir/file1 &&\n> ++\n> ++\ttest_must_fail git mv sub/dir folder1 2>stderr &&\n> ++\tcat sparse_error_header >expect &&\n> ++\techo \"folder1/dir/e\" >>expect &&\n> ++\techo \"folder1/dir/file1\" >>expect &&\n> ++\tcat sparse_hint >>expect &&\n> ++\ttest_cmp expect stderr &&\n> ++\n> ++\ttest_must_fail git mv --sparse sub/dir folder1 2>stderr &&\n> ++\techo \"fatal: destination exists in the index, source=sub/dir/file1, destination=folder1/dir/file1\" \\\n> ++\t>expect &&\n> ++\ttest_cmp expect stderr &&\n> ++\n> ++\tgit mv --sparse -f sub/dir folder1 2>stderr &&\n> ++\ttest_must_be_empty stderr &&\n> ++\n> ++\ttest_path_is_missing sub/dir/file1 &&\n> ++\ttest_path_is_missing sub/dir/e &&\n> ++\ttest_path_is_missing folder1/file1 &&\n> ++\tgit ls-files -t >actual &&\n> ++\t! grep \"H sub/dir/file1\" actual &&\n> ++\t! grep \"H sub/dir/e\" actual &&\n> ++\tgrep \"S folder1/dir/file1\" actual &&\n> ++\n> ++\t# compare file content before move and after move\n> ++\techo test >expect &&\n> ++\tgit ls-files -s -- folder1/dir/file1 | awk \"{print \\$2}\" >oid &&\n> ++\tgit cat-file blob $(cat oid) >actual &&\n> ++\ttest_cmp expect actual\n> ++'\n> ++\n> +test_expect_failure 'move dirty path from in-cone to out-of-cone' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> 2: 725a83c575 = 2: cc5f2770de mv: rename check_dir_in_index() to empty_dir_has_sparse_contents()\n> 3: 1d4a0c16a6 ! 3: 1780f36825 mv: free the *with_slash in check_dir_in_index()\n> @@ Metadata\n> Author: Shaoxuan Yuan \n> \n> ## Commit message ##\n> - mv: free the *with_slash in check_dir_in_index()\n> + mv: free the with_slash in check_dir_in_index()\n> \n> - *with_slash may be a malloc'd pointer, and when it is, free it.\n> + with_slash may be a malloc'd pointer, and when it is, free it.\n> \n> Helped-by: Derrick Stolee \n> Helped-by: Victoria Dye \n> 4: 1f35f0eb34 = 4: d935bd8d7a mv: check if is a SKIP_WORKTREE_DIR\n> 5: 17a871a06b = 5: 26f29df8c8 mv: remove BOTH from enum update_mode\n> 6: 32b9f85aa1 ! 6: 2169b873f7 mv: from in-cone to out-of-cone\n> @@ Commit message\n> Change the behavior so that we can move an in-cone to\n> out-of-cone when --sparse is supplied.\n> \n> + Notice that can also be an out-of-cone file path, rather\n> + than a directory.\n> +\n> Such can be either clean or dirty, and moving it results in\n> different behaviors:\n> \n> - A clean move should move the to the , both in the\n> - working tree and the index, then remove the resulted path from the\n> - working tree, and turn on its CE_SKIP_WORKTREE bit.\n> + A clean move should move to in the index (do\n> + *not* create in the worktree), then delete from\n> + the worktree.\n> \n> A dirty move should move the to the , both in the\n> working tree and the index, but should *not* remove the resulted path\n> from the working tree and should *not* turn on its CE_SKIP_WORKTREE bit.\n> \n> + Optional reading\n> + ================\n> +\n> + We are strict about cone mode when is a file path.\n> + The reason is that some of the previous tests that use no-cone mode in\n> + t7002 are keep breaking, mainly because the `dst_mode = SPARSE;` line\n> + added in this patch.\n> +\n> + Most features developed in both \"from-out-to-in\" and \"from-in-to-out\"\n> + only care about cone mode situation, as no-cone mode is becoming\n> + irrelevant. And because assigning `SPARSE` to `dst_mode` when the\n> + repo is in no-cone mode causes miscellaneous bugs, we should just leave\n> + this new functionality to be exclusive cone mode and save some time.\n> +\n> Helped-by: Derrick Stolee \n> Helped-by: Victoria Dye \n> Signed-off-by: Shaoxuan Yuan \n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t} else if (argc != 1) {\n> \t\t\tdie(_(\"destination '%s' is not a directory\"), dest_path[0]);\n> \t\t} else {\n> + \t\t\tdestination = dest_path;\n> ++\t\t\t/*\n> ++\t\t\t * is a file outside of sparse-checkout\n> ++\t\t\t * cone. Insist on cone mode here for backward\n> ++\t\t\t * compatibility. We don't want dst_mode to be assigned\n> ++\t\t\t * for a file when the repo is using no-cone mode (which\n> ++\t\t\t * is deprecated at this point) sparse-checkout. As\n> ++\t\t\t * SPARSE here is only considering cone-mode situation.\n> ++\t\t\t */\n> ++\t\t\tif (!path_in_cone_mode_sparse_checkout(destination[0], &the_index))\n> ++\t\t\t\tdst_mode = SPARSE;\n> + \t\t}\n> + \t}\n> + \tif (dst_w_slash != dest_path[0]) {\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tconst char *src = source[i], *dst = destination[i];\n> \t\tenum update_mode mode = modes[i];\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tif (show_only)\n> \t\t\tcontinue;\n> \t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> -+\t\t !(dst_mode & SKIP_WORKTREE_DIR) &&\n> ++\t\t !(dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n> \t\t rename(src, dst) < 0) {\n> \t\t\tif (ignore_errors)\n> \t\t\t\tcontinue;\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> +\t\tif (ignore_sparse &&\n> +\t\t core_apply_sparse_checkout &&\n> +\t\t core_sparse_checkout_cone) {\n> ++\t\t\t/*\n> ++\t\t\t * NEEDSWORK: we are *not* paying attention to\n> ++\t\t\t * \"out-to-out\" move ( is out-of-cone and\n> ++\t\t\t * is out-of-cone) at this point. It\n> ++\t\t\t * should be added in a future patch.\n> ++\t\t\t */\n> +\t\t\tif ((mode & SPARSE) &&\n> +\t\t\t path_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\t/* from out-of-cone to in-cone */\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> +\n> +\t\t\t\tif (checkout_entry(dst_ce, &state, NULL, NULL))\n> +\t\t\t\t\tdie(_(\"cannot checkout %s\"), dst_ce->name);\n> -+\t\t\t} else if ((dst_mode & SKIP_WORKTREE_DIR) &&\n> ++\t\t\t} else if ((dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n> +\t\t\t\t !(mode & SPARSE) &&\n> +\t\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\t/* from in-cone to out-of-cone */\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move sparse file to existing\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \n> -@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n> +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move directory with one of the files overwrite' '\n> \ttest_cmp expect actual\n> '\n> \n> 7: 449dba3853 = 7: 78a55e2a46 mv: cleanup empty WORKING_DIRECTORY\n> 8: 875756480e = 8: 43ce1c07ec advice.h: add advise_on_moving_dirty_path()\n> 9: cd5d30505b ! 9: 2e7c6a33c2 mv: check overwrite for in-to-out move\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t}\n> \n> +\t\tif (ignore_sparse &&\n> -+\t\t (dst_mode & SKIP_WORKTREE_DIR) &&\n> ++\t\t (dst_mode & (SKIP_WORKTREE_DIR | SPARSE)) &&\n> +\t\t index_entry_exists(&the_index, dst, strlen(dst))) {\n> +\t\t\tbad = _(\"destination exists in the index\");\n> +\t\t\tif (force) {\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'move clean path from in-cone\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \techo \"sub/file1 overwrite\" >sub/file1 &&\n> +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone to out-of-cone overwrite' '\n> + # This test is testing the same behavior as the\n> + # \"move clean path from in-cone to out-of-cone overwrite\" above.\n> + # The only difference is the changes from \"folder1\" to \"folder1/file1\"\n> +-test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite' '\n> ++test_expect_success 'move clean path from in-cone to out-of-cone file overwrite' '\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tsetup_sparse_checkout &&\n> + \techo \"sub/file1 overwrite\" >sub/file1 &&\n> +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'move clean path from in-cone to out-of-cone file overwrite'\n> + \ttest_cmp expect actual\n> + '\n> + \n> +-test_expect_failure 'move directory with one of the files overwrite' '\n> ++test_expect_success 'move directory with one of the files overwrite' '\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tmkdir -p folder1/dir &&\n> + \ttouch folder1/dir/file1 &&\n> \n> base-commit: c50926e1f48891e2671e1830dbcd2912a4563450\n\n"}]}