{"thread":{"id":"57642","subject":"[WIP v1 0/4] mv: fix out-of-cone file/directory move logic","startedAt":"2022-03-31T09:20:20Z","lastAt":"2022-07-01T21:50:32Z","messageCount":95,"participants":["Shaoxuan Yuan","Ævar Arnfjörð Bjarmason","Victoria Dye","Derrick Stolee","Eric Sunshine","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"452774","messageId":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":null,"subject":"[WIP v1 0/4] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-03-31T09:17:51Z","receivedAt":"2022-03-31T09:20:20Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Before integrating 'mv' with sparse-index, I still find some possibly buggy\nUX when 'mv' is interacting with 'sparse-checkout'. \n\nSo I kept sparse-index off in order to sort things out without a sparse index.\nWe can proceed to integrate with sparse-index once these changes are solid.\n\nNote that this patch is tentative, and still have known glitches, but it \nillustrates a general approach that I intended to harmonize 'mv' \nwith 'sparse-checkout'.\n\nShaoxuan Yuan (4):\n mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n mv: add check_dir_in_index() and solve general dir check issue\n mv: add advise_to_reapply hint for moving file into cone\n t7002: add tests for moving out-of-cone file/directory\n\n builtin/mv.c | 76 ++++++++++++++++++++++++++++++++---\n t/t7002-mv-sparse-checkout.sh | 72 +++++++++++++++++++++++++++++++++\n 2 files changed, 142 insertions(+), 6 deletions(-)\n\n\nbase-commit: 805e0a68082a217f0112db9ee86a022227a9c81b\n-- \n2.35.1\n\n"},{"id":"452775","messageId":"20220331091755.385961-2-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v1 1/4] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-03-31T09:17:52Z","receivedAt":"2022-03-31T09:20:25Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a file which is not on-disk but exists in\nindex as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\nout with \"bad source\".\n\nChange the checking logic, so that such \nfile makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\nI found a new problem introduced by this patch, it is written in the TODO.\nI still haven't found a better way to reconcile this conflict. Please enlighten\nme on this :-)\n\n builtin/mv.c | 26 +++++++++++++++++++++++++-\n 1 file changed, 25 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba83..32ad4d5682 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -185,8 +185,32 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n+\t\t\t/*\n+\t\t\t * TODO: for now, when you try to overwrite a \n+\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n+\t\t\t * flag, then the action will be done without providing \"--force\"\n+\t\t\t * and no warning.\n+\t\t\t *\n+\t\t\t * This is mainly because the sparse \n+\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n+\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n+\t\t\t */\n+\n+\t\t\tint pos = cache_name_pos(src, length);\n+\t\t\tif (pos >= 0) {\n+\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n+\n+\t\t\t\tif (ce_skip_worktree(ce)) {\n+\t\t\t\t\tif (!ignore_sparse)\n+\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\t\t\telse\n+\t\t\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\t}\n+\t\t\t\telse\n+\t\t\t\t\tbad = _(\"bad source\");\n+\t\t\t}\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\telse if (modes[i] != SPARSE)\n \t\t\t\tbad = _(\"bad source\");\n \t\t} else if (!strncmp(src, dst, length) &&\n \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n-- \n2.35.1\n\n"},{"id":"452776","messageId":"20220331091755.385961-3-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-03-31T09:17:53Z","receivedAt":"2022-03-31T09:20:27Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a directory which is not on-disk due\nto its existence outside of sparse-checkout cone, \"giv mv\" command\nerrors out with \"bad source\".\n\nAdd a helper check_dir_in_index() function to see if a directory\nname exists in the index. Also add a SPARSE_DIRECTORY bit to mark\nsuch directories.\n\nChange the checking logic, so that such directory makes\n\"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\nSince I'm so new to C language (not an acquaintance until this patch), \nthe \"check_dir_in_index()\" function I added might not be ideal (in terms of \nsafety and correctness?). I have digging into the APIs provided in the codebase \nbut I haven't found anything to do this very job: find out if a directory is \nin the index (am I missing something?). \nProbably because contents are stored in the index as blobs and \nthey all represent regular files. So I came up with this dull solution...\n\n builtin/mv.c | 41 ++++++++++++++++++++++++++++++++++++-----\n 1 file changed, 36 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 32ad4d5682..9da9205e01 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -115,6 +115,25 @@ static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n \n+static int check_dir_in_index(const char *dir)\n+{\n+\tint ret = 0;\n+\tint length = sizeof(dir) + 1;\n+\tchar *substr = malloc(length);\n+\n+\tfor (int i = 0; i < the_index.cache_nr; i++) {\n+\t\tmemcpy(substr, the_index.cache[i]->name, length);\n+\t\tmemset(substr + length - 1, 0, 1);\n+\n+\t\tif (strcmp(dir, substr) == 0) {\n+\t\t\tret = 1;\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\tfree(substr);\n+\treturn ret;\n+}\n+\n int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n@@ -129,7 +148,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n-\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n+\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE,\n+\tSPARSE_DIRECTORY } *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n@@ -197,6 +217,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t */\n \n \t\t\tint pos = cache_name_pos(src, length);\n+\t\t\tconst char *src_w_slash = add_slash(src);\n+\n \t\t\tif (pos >= 0) {\n \t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n \n@@ -209,6 +231,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\telse\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t}\n+\t\t\telse if (check_dir_in_index(src_w_slash) &&\n+\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n+\t\t\t\tmodes[i] = SPARSE_DIRECTORY;\n+\t\t\t\tgoto dir_check;\n+\t\t\t}\n \t\t\t/* only error if existence is expected. */\n \t\t\telse if (modes[i] != SPARSE)\n \t\t\t\tbad = _(\"bad source\");\n@@ -219,7 +246,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t&& lstat(dst, &st) == 0)\n \t\t\tbad = _(\"cannot move directory over file\");\n \t\telse if (src_is_dir) {\n-\t\t\tint first = cache_name_pos(src, length), last;\n+\t\t\tint first, last;\n+dir_check:\n+\t\t\tfirst = cache_name_pos(src, length);\n \n \t\t\tif (first >= 0)\n \t\t\t\tprepare_move_submodule(src, first,\n@@ -230,7 +259,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\telse { /* last - first >= 1 */\n \t\t\t\tint j, dst_len, n;\n \n-\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\t\tif (!modes[i])\n+\t\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n \t\t\t\tn = argc + last - first;\n \t\t\t\tREALLOC_ARRAY(source, n);\n \t\t\t\tREALLOC_ARRAY(destination, n);\n@@ -332,7 +362,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n+\t\tif (mode != INDEX && mode != SPARSE && mode != SPARSE_DIRECTORY &&\n+\t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n@@ -346,7 +377,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode == WORKING_DIRECTORY)\n+\t\tif (mode == WORKING_DIRECTORY || mode == SPARSE_DIRECTORY)\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\n-- \n2.35.1\n\n"},{"id":"452777","messageId":"20220331091755.385961-4-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-03-31T09:17:54Z","receivedAt":"2022-03-31T09:20:28Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, the SKIP_WORKTREE bit is not removed when moving an out-of-cone\nfile into sparse cone, thus the moved file does not show up in the working tree.\nHint the user to use \"git sparse-checkout reapply\" to reapply the sparsity rules\nto the working tree, by which the SKIP_WORKTREE bit is removed.\n\nSigned-off-by: Shaoxuan Yuan \n---\nI offered this solution becasue I'm not sure how to turn a cache_entry's \nce_flags back to a non-sparse state. I tried directly set it to 0 like this:\n\n\tce->ce_flags = 0;\n\nBut the behavior after this seems undefined. The file still won't show up\nin the working tree.\n\nAnd I found that \"git sparse-checkout reapply\" seems to be a nice fit for the\njob. But I guess if there is a way (there must be but I don't know) to do it\ndirecly in the code, that could also be nice.\n\n builtin/mv.c | 9 +++++++++\n 1 file changed, 9 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 9da9205e01..5f511fb8da 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -138,6 +138,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n+\tint advise_to_reapply = 0;\n \tstruct option builtin_mv_options[] = {\n \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n@@ -383,6 +384,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n \t\trename_cache_entry_at(pos, dst);\n+\t\tif (!advise_to_reapply &&\n+\t\t\t!path_in_sparse_checkout(src, &the_index) &&\n+\t\t\tpath_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\tadvise_to_reapply = 1;\n+\t\t\t}\n \t}\n \n \tif (gitmodules_modified)\n@@ -392,6 +398,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n \t\tdie(_(\"Unable to write new index file\"));\n \n+\tif (advise_to_reapply)\n+\t\tprintf(_(\"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\\n\"));\n+\n \tstring_list_clear(&src_for_dst, 0);\n \tUNLEAK(source);\n \tUNLEAK(dest_path);\n-- \n2.35.1\n\n"},{"id":"452778","messageId":"20220331091755.385961-5-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v1 4/4] t7002: add tests for moving out-of-cone file/directory","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-03-31T09:17:55Z","receivedAt":"2022-03-31T09:20:29Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test following situations:\n\n* 'refuse to move out-of-cone directory without --sparse'\n* 'can move out-of-cone directory with --sparse'\n* 'refuse to move out-of-cone file without --sparse'\n* 'can move out-of-cone file with --sparse'\n\nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 72 +++++++++++++++++++++++++++++++++++\n 1 file changed, 72 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 1d3d2aca21..efb260d015 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -206,4 +206,76 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n+test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\ttest_must_fail git mv folder1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_success 'can move out-of-cone directory with --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n+\t>expect &&\n+\ttest_cmp actual expect &&\n+\n+\tgit sparse-checkout reapply &&\n+\ttest_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/folder1/file1\n+'\n+\n+test_expect_success 'refuse to move out-of-cone file without --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_success 'can move out-of-cone file with --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n+\t>expect &&\n+\ttest_cmp actual expect &&\n+\n+\tgit sparse-checkout reapply &&\n+\t! test_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/file1\n+'\n+\n test_done\n-- \n2.35.1\n\n"},{"id":"452779","messageId":"CAJyCBOQaojt8u_KaobDuQS7hWRC0Bgf82c_geZj4hOw_HB4ZNA@mail.gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 0/4] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-03-31T09:28:18Z","receivedAt":"2022-03-31T09:29:18Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Thu, Mar 31, 2022 at 5:20 PM Shaoxuan Yuan wrote:\n>\n> Before integrating 'mv' with sparse-index, I still find some possibly buggy\n> UX when 'mv' is interacting with 'sparse-checkout'.\n>\n> So I kept sparse-index off in order to sort things out without a sparse index.\n> We can proceed to integrate with sparse-index once these changes are solid.\n>\n> Note that this patch is tentative, and still have known glitches, but it\n> illustrates a general approach that I intended to harmonize 'mv'\n> with 'sparse-checkout'.\n>\n> Shaoxuan Yuan (4):\n> mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n> mv: add check_dir_in_index() and solve general dir check issue\n> mv: add advise_to_reapply hint for moving file into cone\n> t7002: add tests for moving out-of-cone file/directory\n>\n> builtin/mv.c | 76 ++++++++++++++++++++++++++++++++---\n> t/t7002-mv-sparse-checkout.sh | 72 +++++++++++++++++++++++++++++++++\n> 2 files changed, 142 insertions(+), 6 deletions(-)\n>\n>\n> base-commit: 805e0a68082a217f0112db9ee86a022227a9c81b\n> --\n> 2.35.1\n>\n\nThe original related RFC patch is [1], and this patch should be\n--in-reply-to [2].\n\n[1] https://lore.kernel.org/git/20220315100145.214054-1-shaoxuan.yuan02@gmail.com/\n[2] https://lore.kernel.org/git/97a665fe-07c9-c4f6-4ab6-b6c0e1397c31@github.com/\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"452786","messageId":"220331.86mth6qus1.gmgdl@evledraar.gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-3-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-31T10:25:36Z","receivedAt":"2022-03-31T10:30:14Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Mar 31 2022, Shaoxuan Yuan wrote:\n\n> +static int check_dir_in_index(const char *dir)\n> +{\n> +\tint ret = 0;\n> +\tint length = sizeof(dir) + 1;\n> +\tchar *substr = malloc(length);\n> +\n> +\tfor (int i = 0; i < the_index.cache_nr; i++) {\n\nSee https://lore.kernel.org/git/xmqqy20r3rv7.fsf@gitster.g/ for how\nwe're not quite using this syntax yet.\n\nThis should also be \"unsigned int\" to go with the \"cache_nr\" member.\n\n> +\t\tmemcpy(substr, the_index.cache[i]->name, length);\n> +\t\tmemset(substr + length - 1, 0, 1);\n> +\n> +\t\tif (strcmp(dir, substr) == 0) {\n\nStyle: don't compare against == 0, or == NULL, use !, see CodingGuidelines.\n\n> +\t\t\telse if (check_dir_in_index(src_w_slash) &&\n> +\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n\nFunny indentation, the ! should be aligned with \"(\".\n\n> -\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n> +\t\t\t\tif (!modes[i])\n> +\t\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n\nThis works, but assuming things about enum values (even if 0) always\nseems a bit nasty, can this be a comparison to BOTH instead of !? May or\nmay not be better...\n\nBut then again we do xcalloc() to allocate it, so we assume that\nalready, nevermind... :)\n\n(there were also indentation issues below)\n"},{"id":"452787","messageId":"220331.86ilruqup6.gmgdl@evledraar.gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-4-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-31T10:30:25Z","receivedAt":"2022-03-31T10:31:55Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Mar 31 2022, Shaoxuan Yuan wrote:\n\n> Originally, the SKIP_WORKTREE bit is not removed when moving an out-of-cone\n> file into sparse cone, thus the moved file does not show up in the working tree.\n> Hint the user to use \"git sparse-checkout reapply\" to reapply the sparsity rules\n> to the working tree, by which the SKIP_WORKTREE bit is removed.\n>\n> Signed-off-by: Shaoxuan Yuan \n> ---\n> I offered this solution becasue I'm not sure how to turn a cache_entry's \n> ce_flags back to a non-sparse state. I tried directly set it to 0 like this:\n>\n> \tce->ce_flags = 0;\n>\n> But the behavior after this seems undefined. The file still won't show up\n> in the working tree.\n>\n> And I found that \"git sparse-checkout reapply\" seems to be a nice fit for the\n> job. But I guess if there is a way (there must be but I don't know) to do it\n> direcly in the code, that could also be nice.\n>\n> builtin/mv.c | 9 +++++++++\n> 1 file changed, 9 insertions(+)\n>\n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 9da9205e01..5f511fb8da 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -138,6 +138,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> {\n> \tint i, flags, gitmodules_modified = 0;\n> \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n> +\tint advise_to_reapply = 0;\n> \tstruct option builtin_mv_options[] = {\n> \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n> \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n> @@ -383,6 +384,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tpos = cache_name_pos(src, strlen(src));\n> \t\tassert(pos >= 0);\n> \t\trename_cache_entry_at(pos, dst);\n> +\t\tif (!advise_to_reapply &&\n> +\t\t\t!path_in_sparse_checkout(src, &the_index) &&\n> +\t\t\tpath_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\tadvise_to_reapply = 1;\n> +\t\t\t}\n\nMore odd indentation, and the braces aren't needed.\n\n> \t}\n> \n> \tif (gitmodules_modified)\n> @@ -392,6 +398,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> \t\tdie(_(\"Unable to write new index file\"));\n> \n> +\tif (advise_to_reapply)\n> +\t\tprintf(_(\"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\\n\"));\n> +\n\nPlease see 93026558512 (tracking branches: add advice to ambiguous\nrefspec error, 2022-03-28) (the OID may change after I send this, as\nit's in \"seen\") for how to add new advise, i.e. we use advise(), add an\nenum field, config var etc.\n"},{"id":"452788","messageId":"220331.86ee2iqukd.gmgdl@evledraar.gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-5-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 4/4] t7002: add tests for moving out-of-cone file/directory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-03-31T10:33:29Z","receivedAt":"2022-03-31T10:34:47Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Mar 31 2022, Shaoxuan Yuan wrote:\n\n> +\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n> +\t>expect &&\n\nstyle: space before that \\ at the end, but I think it's much better to put this on one line.\n\nThis looks at a glance like it's creating an empty file, until you\nnotice the \\ instead of && at the end..\n\n> +\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n> +\t>expect &&\n\n.. so if you want to keep it wrapped we usually tab-indent the +1th line\nto make it clear that it's continuing the above command\n"},{"id":"452805","messageId":"180efaaf-7bb5-6ed7-2fc6-3c5d5f1304db@github.com","threadId":"57642","inReplyTo":"20220331091755.385961-2-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 1/4] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-03-31T16:39:56Z","receivedAt":"2022-03-31T16:40:03Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, moving a file which is not on-disk but exists in\n> index as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\n> out with \"bad source\".\n> \n> Change the checking logic, so that such \n> file makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n> this operation can be carried out successfully.\n> \n\nGood commit message, this clearly explains your changes!\n\n> Signed-off-by: Shaoxuan Yuan \n> ---\n> I found a new problem introduced by this patch, it is written in the TODO.\n> I still haven't found a better way to reconcile this conflict. Please enlighten\n> me on this :-)\n> \n> builtin/mv.c | 26 +++++++++++++++++++++++++-\n> 1 file changed, 25 insertions(+), 1 deletion(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 83a465ba83..32ad4d5682 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -185,8 +185,32 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \n> \t\tlength = strlen(src);\n> \t\tif (lstat(src, &st) < 0) {\n> +\t\t\t/*\n> +\t\t\t * TODO: for now, when you try to overwrite a \n> +\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n> +\t\t\t * flag, then the action will be done without providing \"--force\"\n> +\t\t\t * and no warning.\n> +\t\t\t *\n> +\t\t\t * This is mainly because the sparse \n> +\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n> +\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n> +\t\t\t */\n> +\n\nI can clarify this a bit. 'mv' is done in two steps: first the file-on-disk\nrename (in the call to 'rename()'), then the index entry (in\n'rename_cache_entry_at()'). In the case of a sparse file, you're only\ndealing with the latter. However, 'rename_cache_entry_at()' moves the index\nentry with the flag 'ADD_CACHE_OK_TO_REPLACE', since it leaves it up to\n'cmd_mv()' to enforce the \"no overwrite\" rule. \n\nSo, in the case of moving *to* a SKIP_WORKTREE entry (where a file being\npresent won't trigger the failure), you'll want to check that the\ndestination *index entry* doesn't exist in addition to the 'lstat()' check.\nIt might require some rearranging of if-statements in this block, but I\nthink it can be done in 'cmd_mv'. \n\n> +\t\t\tint pos = cache_name_pos(src, length);\n> +\t\t\tif (pos >= 0) {\n> +\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n> +\n> +\t\t\t\tif (ce_skip_worktree(ce)) {\n> +\t\t\t\t\tif (!ignore_sparse)\n> +\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> +\t\t\t\t\telse\n> +\t\t\t\t\t\tmodes[i] = SPARSE;\n> +\t\t\t\t}\n> +\t\t\t\telse\n> +\t\t\t\t\tbad = _(\"bad source\");\n\nThis block is good. At first, I thought it was mishandling the\n'!ignore_sparse' case (i.e., that case should have included the \"bad source\"\nassignment), but using the 'only_match_skip_worktree' list is the\nappropriate way to handle it.\n\n> +\t\t\t}\n> \t\t\t/* only error if existence is expected. */\n> -\t\t\tif (modes[i] != SPARSE)\n> +\t\t\telse if (modes[i] != SPARSE)\n> \t\t\t\tbad = _(\"bad source\");\n> \t\t} else if (!strncmp(src, dst, length) &&\n> \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n\nFor a change like this, it would be really helpful to include the tests\nshowing how sparse file moves should now be treated in this commit. I see\nthat you've added some in patch 4 - could you move the ones related to this\nchange into this commit?\n\nAnother way you could do this is to put your \"add tests\" commit first in\nthis series, changing the condition on the ones that are fixed later in the\nseries to \"test_expect_failure\". Then, in each commit that \"fixes\" a test's\nbehavior, change that test to \"test_expect_success\". This approach had the\nadded benefit of showing that, before this series, the tests would fail and\nthat this series explicitly fixes those scenarios.\n"},{"id":"452838","messageId":"52784078-c748-a4a0-68b4-db3c133dfa80@github.com","threadId":"57642","inReplyTo":"20220331091755.385961-3-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-03-31T21:28:17Z","receivedAt":"2022-03-31T21:28:25Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, moving a directory which is not on-disk due\n> to its existence outside of sparse-checkout cone, \"giv mv\" command\n> errors out with \"bad source\".\n> \n> Add a helper check_dir_in_index() function to see if a directory\n> name exists in the index. Also add a SPARSE_DIRECTORY bit to mark\n> such directories.\n> \n\nHmm, I think this patch would fit better in your eventual \"sparse index\nintegration\" series than this \"prerequisite fixes to sparse-checkout\"\nseries. Sparse directories *only* appear when you're using a sparse index\nso, theoretically, this shouldn't ever come up (and thus isn't testable)\nuntil you're using a sparse index. \n\nSince it's here, though, I'm happy to review what you have (even if you\neventually move it to a later series)!\n\n> Change the checking logic, so that such directory makes\n> \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n> this operation can be carried out successfully.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> Since I'm so new to C language (not an acquaintance until this patch), \n> the \"check_dir_in_index()\" function I added might not be ideal (in terms of \n> safety and correctness?). I have digging into the APIs provided in the codebase \n> but I haven't found anything to do this very job: find out if a directory is \n> in the index (am I missing something?). \n> Probably because contents are stored in the index as blobs and \n> they all represent regular files. So I came up with this dull solution...\n> \n> builtin/mv.c | 41 ++++++++++++++++++++++++++++++++++++-----\n> 1 file changed, 36 insertions(+), 5 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 32ad4d5682..9da9205e01 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -115,6 +115,25 @@ static int index_range_of_same_dir(const char *src, int length,\n> \treturn last - first;\n> }\n> \n> +static int check_dir_in_index(const char *dir)\n> +{\n\nThis function can be made a lot simpler - you can use `cache_name_pos()` to\nfind the index entry - if it's found, the directory exists as a sparse\ndirectory. And, because 'add_slash()' enforces the trailing slash later on,\nyou don't need to worry about adjusting the name before you look for the\nentry.\n\n> +\tint ret = 0;\n> +\tint length = sizeof(dir) + 1;\n> +\tchar *substr = malloc(length);\n> +\n> +\tfor (int i = 0; i < the_index.cache_nr; i++) {\n> +\t\tmemcpy(substr, the_index.cache[i]->name, length);\n> +\t\tmemset(substr + length - 1, 0, 1);\n> +\n> +\t\tif (strcmp(dir, substr) == 0) {\n> +\t\t\tret = 1;\n> +\t\t\treturn ret;\n> +\t\t}\n> +\t}\n> +\tfree(substr);\n> +\treturn ret;\n> +}\n> +\n> int cmd_mv(int argc, const char **argv, const char *prefix)\n> {\n> \tint i, flags, gitmodules_modified = 0;\n> @@ -129,7 +148,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tOPT_END(),\n> \t};\n> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n> -\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n> +\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE,\n> +\tSPARSE_DIRECTORY } *modes;\n> \tstruct stat st;\n> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n> \tstruct lock_file lock_file = LOCK_INIT;\n> @@ -197,6 +217,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t */\n> \n> \t\t\tint pos = cache_name_pos(src, length);\n> +\t\t\tconst char *src_w_slash = add_slash(src);\n> +\n> \t\t\tif (pos >= 0) {\n> \t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n> \n> @@ -209,6 +231,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\telse\n> \t\t\t\t\tbad = _(\"bad source\");\n> \t\t\t}\n> +\t\t\telse if (check_dir_in_index(src_w_slash) &&\n> +\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n> +\t\t\t\tmodes[i] = SPARSE_DIRECTORY;\n> +\t\t\t\tgoto dir_check;\n> +\t\t\t}\n\nIn if-statements like this, you'll want to line up the statements in\nparentheses on subsequent lines, like:\n\n\telse if (check_dir_in_index(src_w_slash) &&\n\t\t !path_in_sparse_checkout(src_w_slash, &the_index)) {\n\n...where the second line is indented 1 (8-space-sized) tab + 1 space. \n\nIn general, if you're trying to align code (in this repository), align first\nwith as many tabs as possible, then the \"remainder\" with spaces. Note that\nthis isn't 100% consistent throughout the repository - older lines might not\nhave been aligned properly - but you should go for this styling on any new\nlines that you add.\n\n> \t\t\t/* only error if existence is expected. */\n> \t\t\telse if (modes[i] != SPARSE)\n> \t\t\t\tbad = _(\"bad source\");\n> @@ -219,7 +246,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\t&& lstat(dst, &st) == 0)\n> \t\t\tbad = _(\"cannot move directory over file\");\n> \t\telse if (src_is_dir) {\n> -\t\t\tint first = cache_name_pos(src, length), last;\n> +\t\t\tint first, last;\n> +dir_check:\n> +\t\t\tfirst = cache_name_pos(src, length);\n> \n> \t\t\tif (first >= 0)\n> \t\t\t\tprepare_move_submodule(src, first,\n> @@ -230,7 +259,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\telse { /* last - first >= 1 */\n> \t\t\t\tint j, dst_len, n;\n> \n> -\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n> +\t\t\t\tif (!modes[i])\n> +\t\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n> \t\t\t\tn = argc + last - first;\n> \t\t\t\tREALLOC_ARRAY(source, n);\n> \t\t\t\tREALLOC_ARRAY(destination, n);\n> @@ -332,7 +362,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> \t\tif (show_only)\n> \t\t\tcontinue;\n> -\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n> +\t\tif (mode != INDEX && mode != SPARSE && mode != SPARSE_DIRECTORY &&\n> +\t\t rename(src, dst) < 0) {\n> \t\t\tif (ignore_errors)\n> \t\t\t\tcontinue;\n> \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n> @@ -346,7 +377,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\t\t\t\t 1);\n> \t\t}\n> \n> -\t\tif (mode == WORKING_DIRECTORY)\n> +\t\tif (mode == WORKING_DIRECTORY || mode == SPARSE_DIRECTORY)\n\nI'm a bit confused - doesn't this mean the sparse dir move will be skipped?\nIn your commit description, you mention that this 'mv' succeeds with the\n'--sparse' option, but I don't see any place where the sparse directory\nwould be moved. \n\n> \t\t\tcontinue;\n> \n> \t\tpos = cache_name_pos(src, strlen(src));\n\n"},{"id":"452840","messageId":"c483c964-3e99-4f43-3739-ca6de6670182@github.com","threadId":"57642","inReplyTo":"20220331091755.385961-4-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-03-31T21:56:50Z","receivedAt":"2022-03-31T21:57:08Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, the SKIP_WORKTREE bit is not removed when moving an out-of-cone\n> file into sparse cone, thus the moved file does not show up in the working tree.\n> Hint the user to use \"git sparse-checkout reapply\" to reapply the sparsity rules\n> to the working tree, by which the SKIP_WORKTREE bit is removed.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> I offered this solution becasue I'm not sure how to turn a cache_entry's \n> ce_flags back to a non-sparse state. I tried directly set it to 0 like this:\n> \n> \tce->ce_flags = 0;\n> \n> But the behavior after this seems undefined. The file still won't show up\n> in the working tree.\n> \n\nWhat (I think) you're looking for is something like this:\n\n\tce->ce_flags &= ~CE_SKIP_WORKTREE;\n\nThis disables only the SKIP_WORKTREE flag on the entry (leaving the others\nunchanged). Similarly, you can enable SKIP_WORKTREE with:\n\n\tce->ce_flags |= CE_SKIP_WORKTREE;\n\n> And I found that \"git sparse-checkout reapply\" seems to be a nice fit for the\n> job. But I guess if there is a way (there must be but I don't know) to do it\n> direcly in the code, that could also be nice.\n> \n\nThis brings up an interesting point - what *do* we want to do when you move\nan entry and, based on sparse checkout patterns, the SKIP_WORKTREE status\nchanges? In the case of sparse-checkout, I like your approach: disable\nSKIP_WORKTREE if you're moving inside the sparse checkout definition. And,\nthis only happens if you're using the '--sparse' flag, so a user will have\nto acknowledge that they're moving something sparse to do any of this. \n\nThe other situation to consider is when you're moving something *out* of the\nsparse cone; your approach right now is to move it out *without* enabling\nSKIP_WORKTREE. I think this is also probably valid - you might not want to\nassume a user wants to remove the file from their worktree *just* yet.\nHowever, it does create a situation where a user has a \"sparse\" file active\nin their repo, so *this* might be a situation where you want to advise the\nuser to call 'git sparse-checkout reapply' to \"sparsify\" that file.\n\nIn any case, you'll only want to do this if the global variable\n'core_apply_sparse_checkout' is non-zero. 'SKIP_WORKTREE' can be used when\nnot in a sparse checkout, so you can't always change the flag based on any\nsparse patterns (because there might not be any!). \n\n> builtin/mv.c | 9 +++++++++\n> 1 file changed, 9 insertions(+)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 9da9205e01..5f511fb8da 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -138,6 +138,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> {\n> \tint i, flags, gitmodules_modified = 0;\n> \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n> +\tint advise_to_reapply = 0;\n> \tstruct option builtin_mv_options[] = {\n> \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n> \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n> @@ -383,6 +384,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tpos = cache_name_pos(src, strlen(src));\n> \t\tassert(pos >= 0);\n> \t\trename_cache_entry_at(pos, dst);\n> +\t\tif (!advise_to_reapply &&\n> +\t\t\t!path_in_sparse_checkout(src, &the_index) &&\n> +\t\t\tpath_in_sparse_checkout(dst, &the_index)) {\n> +\t\t\t\tadvise_to_reapply = 1;\n> +\t\t\t}\n> \t}\n> \n> \tif (gitmodules_modified)\n> @@ -392,6 +398,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> \t\tdie(_(\"Unable to write new index file\"));\n> \n> +\tif (advise_to_reapply)\n> +\t\tprintf(_(\"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\\n\"));\n\nI know you're hoping to change your implementation so that you don't need\nthis advice. But, if you *do* end up needing it (or some other advice)\nsomewhere else, you can implement it using the 'advise()' API. For an\nexample on how that's used, see how 'ADVICE_SKIPPED_CHERRY_PICKS' is used in\n'sequencer.c'. \n\nIn your case, you could have something like 'ADVICE_REAPPLY_SPARSE_PATTERNS'\nthat, if enabled, prints a message like the one you have here.\n\n> +\n> \tstring_list_clear(&src_for_dst, 0);\n> \tUNLEAK(source);\n> \tUNLEAK(dest_path);\n\nAs with patch 1, it would help paint a clear picture of what this patch does\nif you could incorporate the tests from patch 4 into this one.\n"},{"id":"452843","messageId":"3be083bb-7bc8-3aa2-400e-774b9a1524e0@github.com","threadId":"57642","inReplyTo":"20220331091755.385961-5-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 4/4] t7002: add tests for moving out-of-cone file/directory","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-03-31T22:11:33Z","receivedAt":"2022-03-31T22:11:38Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Add corresponding tests to test following situations:\n> \n> * 'refuse to move out-of-cone directory without --sparse'\n> * 'can move out-of-cone directory with --sparse'\n> * 'refuse to move out-of-cone file without --sparse'\n> * 'can move out-of-cone file with --sparse'\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> t/t7002-mv-sparse-checkout.sh | 72 +++++++++++++++++++++++++++++++++++\n> 1 file changed, 72 insertions(+)\n> \n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 1d3d2aca21..efb260d015 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -206,4 +206,76 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> \ttest_cmp expect stderr\n> '\n> \n> +test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\ttest_must_fail git mv folder1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_success 'can move out-of-cone directory with --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n> +\t>expect &&\n> +\ttest_cmp actual expect &&\n> +\n> +\tgit sparse-checkout reapply &&\n> +\ttest_path_is_dir sub/folder1 &&\n> +\ttest_path_is_file sub/folder1/file1\n> +'\n> +\n> +test_expect_success 'refuse to move out-of-cone file without --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_success 'can move out-of-cone file with --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n> +\t>expect &&\n> +\ttest_cmp actual expect &&\n> +\n> +\tgit sparse-checkout reapply &&\n> +\t! test_path_is_dir sub/folder1 &&\n> +\ttest_path_is_file sub/file1\n> +'\n> +\n> test_done\n\nOther than my earlier comments about moving the tests to another point in\nthe series, the content of the tests looks great!\n"},{"id":"452844","messageId":"3c6a562b-d55f-d73e-3547-36ce752fcc6f@github.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 0/4] mv: fix out-of-cone file/directory move logic","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-03-31T22:21:05Z","receivedAt":"2022-03-31T22:21:12Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Before integrating 'mv' with sparse-index, I still find some possibly buggy\n> UX when 'mv' is interacting with 'sparse-checkout'. \n> \n> So I kept sparse-index off in order to sort things out without a sparse index.\n> We can proceed to integrate with sparse-index once these changes are solid.\n> \n> Note that this patch is tentative, and still have known glitches, but it \n> illustrates a general approach that I intended to harmonize 'mv' \n> with 'sparse-checkout'.\n> \n\nThanks for working out some ways to make 'mv' behave more nicely with sparse\ncheckouts! I did my best to address some of the specific implementation\nquestions you had in your commit messages. Beyond that, my main points of\nfeedback (beyond some formatting nits and implementation questions) are:\n\n* Patch 2 deals with sparse directories, which won't show up until you\n enable sparse index; since you can't test that yet, you should save the\n patch for your \"sparse index integration\" series.\n* Patch 4 should either be moved to the beginning of the series (with the\n tests flagged with 'test_expect_failure' until the patch that fixes the\n associated behavior), or split up with the tests associated with a change\n moved into the patch that makes that change.\n\nAnd, as always, I'm happy to answer any questions and/or clarify weird\nbehavior you encounter while making changes to this (or subsequent) series!\n\n> Shaoxuan Yuan (4):\n> mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n> mv: add check_dir_in_index() and solve general dir check issue\n> mv: add advise_to_reapply hint for moving file into cone\n> t7002: add tests for moving out-of-cone file/directory\n> \n> builtin/mv.c | 76 ++++++++++++++++++++++++++++++++---\n> t/t7002-mv-sparse-checkout.sh | 72 +++++++++++++++++++++++++++++++++\n> 2 files changed, 142 insertions(+), 6 deletions(-)\n> \n> \n> base-commit: 805e0a68082a217f0112db9ee86a022227a9c81b\n\n"},{"id":"452849","messageId":"CAJyCBOThDykT9dn09CutAdaSwWXapVn3St3XnnLYmORfGfq+=w@mail.gmail.com","threadId":"57642","inReplyTo":"220331.86mth6qus1.gmgdl@evledraar.gmail.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-01T03:51:15Z","receivedAt":"2022-04-01T03:51:33Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Thu, Mar 31, 2022 at 6:30 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> On Thu, Mar 31 2022, Shaoxuan Yuan wrote:\n>\n> > +static int check_dir_in_index(const char *dir)\n> > +{\n> > + int ret = 0;\n> > + int length = sizeof(dir) + 1;\n> > + char *substr = malloc(length);\n> > +\n> > + for (int i = 0; i < the_index.cache_nr; i++) {\n>\n> See https://lore.kernel.org/git/xmqqy20r3rv7.fsf@gitster.g/ for how\n> we're not quite using this syntax yet.\n>\n> This should also be \"unsigned int\" to go with the \"cache_nr\" member.\n>\n> > + memcpy(substr, the_index.cache[i]->name, length);\n> > + memset(substr + length - 1, 0, 1);\n> > +\n> > + if (strcmp(dir, substr) == 0) {\n>\n> Style: don't compare against == 0, or == NULL, use !, see CodingGuidelines.\n>\n> > + else if (check_dir_in_index(src_w_slash) &&\n> > + !path_in_sparse_checkout(src_w_slash, &the_index)) {\n>\n> Funny indentation, the ! should be aligned with \"(\".\n>\n> > - modes[i] = WORKING_DIRECTORY;\n> > + if (!modes[i])\n> > + modes[i] = WORKING_DIRECTORY;\n>\n> This works, but assuming things about enum values (even if 0) always\n> seems a bit nasty, can this be a comparison to BOTH instead of !? May or\n> may not be better...\n>\n> But then again we do xcalloc() to allocate it, so we assume that\n> already, nevermind... :)\n>\n> (there were also indentation issues below)\n\nThanks for the styling reminders! I should go back and reread CodingGuidelines\nmore often... Things just slipped off my mind since I'm still trying to remember\nthe guideline...\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"452850","messageId":"CAJyCBORNQEN4WNfuYbO264qz0W6d-rbT6z=T_-AZF5jY36c1yQ@mail.gmail.com","threadId":"57642","inReplyTo":"220331.86ilruqup6.gmgdl@evledraar.gmail.com","subject":"Re: [WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-01T04:00:24Z","receivedAt":"2022-04-01T04:00:41Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Thu, Mar 31, 2022 at 6:31 PM Ævar Arnfjörð Bjarmason\n wrote:\n>\n> More odd indentation, and the braces aren't needed.\n\nGot me again :-( Will make a change.\n\n> > }\n> >\n> > if (gitmodules_modified)\n> > @@ -392,6 +398,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> > die(_(\"Unable to write new index file\"));\n> >\n> > + if (advise_to_reapply)\n> > + printf(_(\"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\\n\"));\n> > +\n>\n> Please see 93026558512 (tracking branches: add advice to ambiguous\n> refspec error, 2022-03-28) (the OID may change after I send this, as\n> it's in \"seen\") for how to add new advise, i.e. we use advise(), add an\n> enum field, config var etc.\n\nI actually did use advise(), but I noticed that it prints to stderr\nand ... nevermind,\nI realized that printing to stderr is OK. But can I print to stdout\nsince I think users should\nbe \"reminded\" instead of \"warned\"?\n\nAnyway, I think using advice() is probably better.\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"452853","messageId":"220401.867d89p6sp.gmgdl@evledraar.gmail.com","threadId":"57642","inReplyTo":"CAJyCBORNQEN4WNfuYbO264qz0W6d-rbT6z=T_-AZF5jY36c1yQ@mail.gmail.com","subject":"Re: [WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-01T08:02:56Z","receivedAt":"2022-04-01T08:05:51Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Apr 01 2022, Shaoxuan Yuan wrote:\n\n> On Thu, Mar 31, 2022 at 6:31 PM Ævar Arnfjörð Bjarmason\n> wrote:\n>>\n>> More odd indentation, and the braces aren't needed.\n>\n> Got me again :-( Will make a change.\n>\n>> > }\n>> >\n>> > if (gitmodules_modified)\n>> > @@ -392,6 +398,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> > COMMIT_LOCK | SKIP_IF_UNCHANGED))\n>> > die(_(\"Unable to write new index file\"));\n>> >\n>> > + if (advise_to_reapply)\n>> > + printf(_(\"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\\n\"));\n>> > +\n>>\n>> Please see 93026558512 (tracking branches: add advice to ambiguous\n>> refspec error, 2022-03-28) (the OID may change after I send this, as\n>> it's in \"seen\") for how to add new advise, i.e. we use advise(), add an\n>> enum field, config var etc.\n>\n> I actually did use advise(), but I noticed that it prints to stderr\n> and ... nevermind,\n> I realized that printing to stderr is OK. But can I print to stdout\n> since I think users should\n> be \"reminded\" instead of \"warned\"?\n>\n> Anyway, I think using advice() is probably better.\n\nWe've typically used stderr in git not to mean \"error\", but to\ndistinguish \"chatty\" and non-primary output from non-chatty.\n\nSo (leaving aside that we're unlikely to add advice to plumbing) if you\nemitted a warning() or advice from git-ls-tree you should be able to run\nsomething like:\n\n git ls-tree -r HEAD >output-for-a-script\n\nAnd get the advise() on stderr, while the \"primary\" output is on stdout.\n\nThere's a recent-ish (last year or so) thread where I think Jeff King\nexplained this better than I'm doing here, but I couldn't find it with a\nquick search.\n\nIn other words, you can just use advise() here, don't worry about it\nwriting to stderr.\n\n\n"},{"id":"452866","messageId":"CAJyCBORjR1C1cs9aUYPsyEJoBaRsBo9ATHXBopFYPqrKQAbXxA@mail.gmail.com","threadId":"57642","inReplyTo":"3c6a562b-d55f-d73e-3547-36ce752fcc6f@github.com","subject":"Re: [WIP v1 0/4] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-01T12:18:44Z","receivedAt":"2022-04-01T12:19:01Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Fri, Apr 1, 2022 at 6:21 AM Victoria Dye wrote:\n> Thanks for working out some ways to make 'mv' behave more nicely with sparse\n> checkouts! I did my best to address some of the specific implementation\n> questions you had in your commit messages. Beyond that, my main points of\n> feedback (beyond some formatting nits and implementation questions) are:\n>\n> * Patch 2 deals with sparse directories, which won't show up until you\n> enable sparse index; since you can't test that yet, you should save the\n> patch for your \"sparse index integration\" series.\n> * Patch 4 should either be moved to the beginning of the series (with the\n> tests flagged with 'test_expect_failure' until the patch that fixes the\n> associated behavior), or split up with the tests associated with a change\n> moved into the patch that makes that change.\n>\n> And, as always, I'm happy to answer any questions and/or clarify weird\n> behavior you encounter while making changes to this (or subsequent) series!\n\nHi Victoria,\n\nThanks a lot for the detailed and informative feedback, they are\nincredibly helpful\nto guide me through this first attempt! I have read them all and am preparing\ncorresponding fixes :-)\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"452867","messageId":"CAJyCBOQmUYe53ahpEXQZAWMoers0o7b1xuCYu_k-LrfvKTkV-g@mail.gmail.com","threadId":"57642","inReplyTo":"52784078-c748-a4a0-68b4-db3c133dfa80@github.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-01T12:49:10Z","receivedAt":"2022-04-01T12:49:27Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Fri, Apr 1, 2022 at 5:28 AM Victoria Dye wrote:\n>\n> Shaoxuan Yuan wrote:\n> > Originally, moving a directory which is not on-disk due\n> > to its existence outside of sparse-checkout cone, \"giv mv\" command\n> > errors out with \"bad source\".\n> >\n> > Add a helper check_dir_in_index() function to see if a directory\n> > name exists in the index. Also add a SPARSE_DIRECTORY bit to mark\n> > such directories.\n> >\n>\n> Hmm, I think this patch would fit better in your eventual \"sparse index\n> integration\" series than this \"prerequisite fixes to sparse-checkout\"\n> series. Sparse directories *only* appear when you're using a sparse index\n> so, theoretically, this shouldn't ever come up (and thus isn't testable)\n> until you're using a sparse index.\n\nAfter reading your feedback, I realized that I totally misused\nthe phrase \"sparse directory\". Clearly, this patch series does not\ndeal with sparse-\nindex yet, as \"sparse directory\" is a cache entry that points to a\ntree, if sparse-index\nis enabled. Silly me ;)\n\nWhat I was *actually* trying to say is: I want to change the checking\nlogic of moving\na \"directory that exists outside of sparse-checkout cone\", and I\napparently misused\n\"sparse directory\" to reference such a thing.\n\n> Since it's here, though, I'm happy to review what you have (even if you\n> eventually move it to a later series)!\n\nThanks!\n\n> > diff --git a/builtin/mv.c b/builtin/mv.c\n> > index 32ad4d5682..9da9205e01 100644\n> > --- a/builtin/mv.c\n> > +++ b/builtin/mv.c\n> > @@ -115,6 +115,25 @@ static int index_range_of_same_dir(const char *src, int length,\n> > return last - first;\n> > }\n> >\n> > +static int check_dir_in_index(const char *dir)\n> > +{\n>\n> This function can be made a lot simpler - you can use `cache_name_pos()` to\n> find the index entry - if it's found, the directory exists as a sparse\n> directory. And, because 'add_slash()' enforces the trailing slash later on,\n> you don't need to worry about adjusting the name before you look for the\n> entry.\n\nYes, if I correctly used the phrase \"sparse directory\", but I did not...\nI think I can use 'cache_name_pos()' to\ncheck a directory *iff* it is a legit sparse directory when using sparse-index?\n\nIn my case, I just want to check a regular directory that is not in\nthe worktree,\nsince the cone pattern excludes it. And in a non-sparse index, cache\nentry points only\nto blobs, not trees, and that's the reason I wrote this weird function\nto look into the\nindex. I understand that sounds not compatible with how git manages\nindex, but all\nI want to know is \"does this directory exist in the index?\" (this\nquestion is also quasi-correct).\n\nI tried to find an existing API for this job, but I failed to find\nany. Though I have a hunch\nthat there must be something to do it...\n\n> > + int ret = 0;\n> > + int length = sizeof(dir) + 1;\n> > + char *substr = malloc(length);\n> > +\n> > + for (int i = 0; i < the_index.cache_nr; i++) {\n> > + memcpy(substr, the_index.cache[i]->name, length);\n> > + memset(substr + length - 1, 0, 1);\n> > +\n> > + if (strcmp(dir, substr) == 0) {\n> > + ret = 1;\n> > + return ret;\n> > + }\n> > + }\n> > + free(substr);\n> > + return ret;\n> > +}\n> > +\n> > int cmd_mv(int argc, const char **argv, const char *prefix)\n> > {\n> > int i, flags, gitmodules_modified = 0;\n> > @@ -129,7 +148,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > OPT_END(),\n> > };\n> > const char **source, **destination, **dest_path, **submodule_gitfile;\n> > - enum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n> > + enum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE,\n> > + SPARSE_DIRECTORY } *modes;\n> > struct stat st;\n> > struct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n> > struct lock_file lock_file = LOCK_INIT;\n> > @@ -197,6 +217,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > */\n> >\n> > int pos = cache_name_pos(src, length);\n> > + const char *src_w_slash = add_slash(src);\n> > +\n> > if (pos >= 0) {\n> > const struct cache_entry *ce = active_cache[pos];\n> >\n> > @@ -209,6 +231,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > else\n> > bad = _(\"bad source\");\n> > }\n> > + else if (check_dir_in_index(src_w_slash) &&\n> > + !path_in_sparse_checkout(src_w_slash, &the_index)) {\n> > + modes[i] = SPARSE_DIRECTORY;\n> > + goto dir_check;\n> > + }\n>\n> In if-statements like this, you'll want to line up the statements in\n> parentheses on subsequent lines, like:\n>\n> else if (check_dir_in_index(src_w_slash) &&\n> !path_in_sparse_checkout(src_w_slash, &the_index)) {\n>\n> ...where the second line is indented 1 (8-space-sized) tab + 1 space.\n>\n> In general, if you're trying to align code (in this repository), align first\n> with as many tabs as possible, then the \"remainder\" with spaces. Note that\n> this isn't 100% consistent throughout the repository - older lines might not\n> have been aligned properly - but you should go for this styling on any new\n> lines that you add.\n\nWill do.\n\n>\n> > /* only error if existence is expected. */\n> > else if (modes[i] != SPARSE)\n> > bad = _(\"bad source\");\n> > @@ -219,7 +246,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > && lstat(dst, &st) == 0)\n> > bad = _(\"cannot move directory over file\");\n> > else if (src_is_dir) {\n> > - int first = cache_name_pos(src, length), last;\n> > + int first, last;\n> > +dir_check:\n> > + first = cache_name_pos(src, length);\n> >\n> > if (first >= 0)\n> > prepare_move_submodule(src, first,\n> > @@ -230,7 +259,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > else { /* last - first >= 1 */\n> > int j, dst_len, n;\n> >\n> > - modes[i] = WORKING_DIRECTORY;\n> > + if (!modes[i])\n> > + modes[i] = WORKING_DIRECTORY;\n> > n = argc + last - first;\n> > REALLOC_ARRAY(source, n);\n> > REALLOC_ARRAY(destination, n);\n> > @@ -332,7 +362,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > printf(_(\"Renaming %s to %s\\n\"), src, dst);\n> > if (show_only)\n> > continue;\n> > - if (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n> > + if (mode != INDEX && mode != SPARSE && mode != SPARSE_DIRECTORY &&\n> > + rename(src, dst) < 0) {\n> > if (ignore_errors)\n> > continue;\n> > die_errno(_(\"renaming '%s' failed\"), src);\n> > @@ -346,7 +377,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > 1);\n> > }\n> >\n> > - if (mode == WORKING_DIRECTORY)\n> > + if (mode == WORKING_DIRECTORY || mode == SPARSE_DIRECTORY)\n>\n> I'm a bit confused - doesn't this mean the sparse dir move will be skipped?\n> In your commit description, you mention that this 'mv' succeeds with the\n> '--sparse' option, but I don't see any place where the sparse directory\n> would be moved.\n\nWell, you know the drill, I did not use \"sparse directory\" correctly, let alone\n'SPARSE_DIRECTORY' enum bit in this hunk. I think it makes some sense\nif you apply my actual meaning of 'SPARSE_DIRECTORY' here (it should be\nsomething like OUT_OF_CONE_WORKING_DIRECTORY)? Because such\ndirectory is not on disk, it cannot be \"rename()\"d, and should also skip the\n\"rename_cache_entry_at()\" function. If all the files under the directory are\nmoved/renamed, then (in my opinion) the directory is both moved to the\ndestination,\nboth in the worktree and in the index.\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"452882","messageId":"edbbd81e-2117-c9b9-76a5-4713e6326d2f@github.com","threadId":"57642","inReplyTo":"180efaaf-7bb5-6ed7-2fc6-3c5d5f1304db@github.com","subject":"Re: [WIP v1 1/4] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-04-01T14:30:46Z","receivedAt":"2022-04-01T14:37:21Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/31/2022 12:39 PM, Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n\n>> \t\tif (lstat(src, &st) < 0) {\n>> +\t\t\t/*\n>> +\t\t\t * TODO: for now, when you try to overwrite a \n>> +\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n>> +\t\t\t * flag, then the action will be done without providing \"--force\"\n>> +\t\t\t * and no warning.\n>> +\t\t\t *\n>> +\t\t\t * This is mainly because the sparse \n>> +\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n>> +\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n>> +\t\t\t */\n>> +\n> \n> I can clarify this a bit. 'mv' is done in two steps: first the file-on-disk\n> rename (in the call to 'rename()'), then the index entry (in\n> 'rename_cache_entry_at()'). In the case of a sparse file, you're only\n> dealing with the latter. However, 'rename_cache_entry_at()' moves the index\n> entry with the flag 'ADD_CACHE_OK_TO_REPLACE', since it leaves it up to\n> 'cmd_mv()' to enforce the \"no overwrite\" rule. \n> \n> So, in the case of moving *to* a SKIP_WORKTREE entry (where a file being\n> present won't trigger the failure), you'll want to check that the\n> destination *index entry* doesn't exist in addition to the 'lstat()' check.\n> It might require some rearranging of if-statements in this block, but I\n> think it can be done in 'cmd_mv'. \n\nThis also explains the issue when going from sparse to non-sparse: the\nfile move is the expected way to populate the end-result, but we skip that\npart in the sparse case. We need to do an extra step to populate the file\nfrom the version in the index (after moving the cache entry).\n\nRelated to this chain of if/else if/else blocks, it might be worth\nrefactoring them to be sequential \"if ()\" blocks where we jump to a\n\"cleanup:\" label via a 'goto' if we know that we are in a failure mode.\n\nThe previous organization made sense because any of the if () or else if\n() conditions were a failure mode. However, it might be better to\nrearrange things to be clearer about the situation.\n\nHere is a diff from what I was playing with. It's... unclear if this is a\nbetter arrangement, but I thought it worth discussing.\n\n--- >8 ---\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba831..683a412a3fc 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -186,15 +186,22 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\tif (modes[i] != SPARSE) {\n \t\t\t\tbad = _(\"bad source\");\n-\t\t} else if (!strncmp(src, dst, length) &&\n+\t\t\t\tgoto checked_move;\n+\t\t\t}\n+\t\t}\n+\t\tif (!strncmp(src, dst, length) &&\n \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n \t\t\tbad = _(\"can not move directory into itself\");\n-\t\t} else if ((src_is_dir = S_ISDIR(st.st_mode))\n-\t\t\t\t&& lstat(dst, &st) == 0)\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\t\t\t&& lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n-\t\telse if (src_is_dir) {\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif (src_is_dir) {\n \t\t\tint first = cache_name_pos(src, length), last;\n \n \t\t\tif (first >= 0)\n@@ -227,11 +234,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t}\n \t\t\t\targc += last - first;\n \t\t\t}\n-\t\t} else if (!(ce = cache_file_exists(src, length, 0))) {\n+\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif (!(ce = cache_file_exists(src, length, 0))) {\n \t\t\tbad = _(\"not under version control\");\n-\t\t} else if (ce_stage(ce)) {\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif (ce_stage(ce)) {\n \t\t\tbad = _(\"conflicted\");\n-\t\t} else if (lstat(dst, &st) == 0 &&\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif (lstat(dst, &st) == 0 &&\n \t\t\t (!ignore_case || strcasecmp(src, dst))) {\n \t\t\tbad = _(\"destination exists\");\n \t\t\tif (force) {\n@@ -246,34 +260,40 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t} else\n \t\t\t\t\tbad = _(\"Cannot overwrite\");\n \t\t\t}\n-\t\t} else if (string_list_has_string(&src_for_dst, dst))\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif (string_list_has_string(&src_for_dst, dst)) {\n \t\t\tbad = _(\"multiple sources for the same target\");\n-\t\telse if (is_dir_sep(dst[strlen(dst) - 1]))\n+\t\t\tgoto checked_move;\n+\t\t}\n+\t\tif (is_dir_sep(dst[strlen(dst) - 1])) {\n \t\t\tbad = _(\"destination directory does not exist\");\n-\t\telse {\n-\t\t\t/*\n-\t\t\t * We check if the paths are in the sparse-checkout\n-\t\t\t * definition as a very final check, since that\n-\t\t\t * allows us to point the user to the --sparse\n-\t\t\t * option as a way to have a successful run.\n-\t\t\t */\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(src, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\n-\t\t\tif (skip_sparse)\n-\t\t\t\tgoto remove_entry;\n+\t\t\tgoto checked_move;\n+\t\t}\n \n-\t\t\tstring_list_insert(&src_for_dst, dst);\n+\t\t/*\n+\t\t * We check if the paths are in the sparse-checkout\n+\t\t * definition as a very final check, since that\n+\t\t * allows us to point the user to the --sparse\n+\t\t * option as a way to have a successful run.\n+\t\t */\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(src, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\tskip_sparse = 1;\n+\t\t}\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n+\t\t\tskip_sparse = 1;\n \t\t}\n \n+\t\tif (skip_sparse)\n+\t\t\tgoto remove_entry;\n+\n+\t\tstring_list_insert(&src_for_dst, dst);\n+\n+checked_move:\n \t\tif (!bad)\n \t\t\tcontinue;\n \t\tif (!ignore_errors)\n\n--- >8 --- \n>> +\t\t\t}\n>> \t\t\t/* only error if existence is expected. */\n>> -\t\t\tif (modes[i] != SPARSE)\n>> +\t\t\telse if (modes[i] != SPARSE)\n>> \t\t\t\tbad = _(\"bad source\");\n>> \t\t} else if (!strncmp(src, dst, length) &&\n>> \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n> \n> For a change like this, it would be really helpful to include the tests\n> showing how sparse file moves should now be treated in this commit. I see\n> that you've added some in patch 4 - could you move the ones related tothis\n> change into this commit?\n\nI completely agree: it's nice to see how behavior is intended to change\nnext to your code change.\n\n> Another way you could do this is to put your \"add tests\" commit first in\n> this series, changing the condition on the ones that are fixed later in the\n> series to \"test_expect_failure\". Then, in each commit that \"fixes\" a test's\n> behavior, change that test to \"test_expect_success\". This approach had the\n> added benefit of showing that, before this series, the tests would fail and\n> that this series explicitly fixes those scenarios.\n\nAnd this would be easier to adapt your current patch structure to this model:\nmove the last commit to be first, but flip the expectation. Then modify the\nexpectation for the tests that pass as you go.\n\nThis only works as long as you can make an entire test pass with each change.\nIf multiple changes are needed to make any one test pass, then we don't get\nthe benefit we're looking for. In that case, your test might be covering too\nmuch behavior in a single test, so it would be worth rewriting the tests to\ncheck a smaller part of the behavior.\n\nThanks,\n-Stolee\n"},{"id":"452889","messageId":"22aadea2-9330-aa9e-7b6a-834585189144@github.com","threadId":"57642","inReplyTo":"CAJyCBOQmUYe53ahpEXQZAWMoers0o7b1xuCYu_k-LrfvKTkV-g@mail.gmail.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-04-01T14:49:24Z","receivedAt":"2022-04-01T15:03:55Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 4/1/2022 8:49 AM, Shaoxuan Yuan wrote:> On Fri, Apr 1, 2022 at 5:28 AM Victoria Dye wrote:\n>>\n>> Shaoxuan Yuan wrote:\n>>> Originally, moving a directory which is not on-disk due\n>>> to its existence outside of sparse-checkout cone, \"giv mv\" command\n>>> errors out with \"bad source\".\n>>>\n>>> Add a helper check_dir_in_index() function to see if a directory\n>>> name exists in the index. Also add a SPARSE_DIRECTORY bit to mark\n>>> such directories.\n>>>\n>>\n>> Hmm, I think this patch would fit better in your eventual \"sparse index\n>> integration\" series than this \"prerequisite fixes to sparse-checkout\"\n>> series. Sparse directories *only* appear when you're using a sparse index\n>> so, theoretically, this shouldn't ever come up (and thus isn't testable)\n>> until you're using a sparse index.\n> \n> After reading your feedback, I realized that I totally misused\n> the phrase \"sparse directory\". Clearly, this patch series does not\n> deal with sparse-\n> index yet, as \"sparse directory\" is a cache entry that points to a\n> tree, if sparse-index\n> is enabled. Silly me ;)\n> \n> What I was *actually* trying to say is: I want to change the checking\n> logic of moving\n> a \"directory that exists outside of sparse-checkout cone\", and I\n> apparently misused\n> \"sparse directory\" to reference such a thing.\n\nIn the case of a full index (or an expanded sparse index, which is\ncurrently always the case for `git mv`), you detect a sparse directory\nby looking for the directory in the index, _not_ finding it, and then\nseeing if the cache entry at the position where the directory _would_\nexist is marked with the SKIP_WORKTREE bit.\n\nThis works in cone mode and the old mode because I assume you've already\nchecked for the existence of the directory, so if there _was_ any\nnon-SKIP_WORKTREE cache entry within the directory, then the directory\nwould exist in the worktree.\n\n(These are good details to include in the commit message.)\n\n>>> +static int check_dir_in_index(const char *dir)\n>>> +{\n>>\n>> This function can be made a lot simpler - you can use `cache_name_pos()` to\n>> find the index entry - if it's found, the directory exists as a sparse\n>> directory. And, because 'add_slash()' enforces the trailing slash later on,\n>> you don't need to worry about adjusting the name before you look for the\n>> entry.\n> \n> Yes, if I correctly used the phrase \"sparse directory\", but I did not...\n> I think I can use 'cache_name_pos()' to\n> check a directory *iff* it is a legit sparse directory when using sparse-index?\n> \n> In my case, I just want to check a regular directory that is not in\n> the worktree,\n> since the cone pattern excludes it. And in a non-sparse index, cache\n> entry points only\n> to blobs, not trees, and that's the reason I wrote this weird function\n> to look into the\n> index. I understand that sounds not compatible with how git manages\n> index, but all\n> I want to know is \"does this directory exist in the index?\" (this\n> question is also quasi-correct).\n> \n> I tried to find an existing API for this job, but I failed to find\n> any. Though I have a hunch\n> that there must be something to do it...\n\nYou can still use cache_name_pos() and if the resulting value is negative,\nthen you can \"flip it\" (pos = -1 - pos) to get the array index where the\ndirectory _would_ be inserted.\n\nFor example, here is a case in unpack-trees.c (that uses the synonym\nindex_name_pos()):\n\nstatic int locate_in_src_index(const struct cache_entry *ce,\n\t\t\t struct unpack_trees_options *o)\n{\n\tstruct index_state *index = o->src_index;\n\tint len = ce_namelen(ce);\n\tint pos = index_name_pos(index, ce->name, len);\n\tif (pos < 0)\n\t\tpos = -1 - pos;\n\treturn pos;\n}\n\nThis uses a binary search inside the method, so it will be much faster\nthan the loop you wrote here.\n\nIf you have this helper, then you can integrate with the sparse index later\nby checking for a sparse directory entry when pos is non-negative. But that\ncan wait for the next series.\n\n>>> /* only error if existence is expected. */\n>>> else if (modes[i] != SPARSE)\n>>> bad = _(\"bad source\");\n>>> @@ -219,7 +246,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>>> && lstat(dst, &st) == 0)\n>>> bad = _(\"cannot move directory over file\");\n>>> else if (src_is_dir) {\n>>> - int first = cache_name_pos(src, length), last;\n>>> + int first, last;\n>>> +dir_check:\n>>> + first = cache_name_pos(src, length);\n>>>\n>>> if (first >= 0)\n>>> prepare_move_submodule(src, first,\n>>> @@ -230,7 +259,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>>> else { /* last - first >= 1 */\n>>> int j, dst_len, n;\n>>>\n>>> - modes[i] = WORKING_DIRECTORY;\n>>> + if (!modes[i])\n>>> + modes[i] = WORKING_DIRECTORY;\n\nThis is curious that we could get here with an existing mode. I wonder if\nit would be worthwhile to make the enum using a \"flags\" mode (each state\nis a different bit in the word) so instead of\n\n\tmodes[i] = WORKING_DIRECTORY;\n\nwe would write\n\n\tmodes[i] |= WORKING_DIRECTORY;\n\n>>> n = argc + last - first;\n>>> REALLOC_ARRAY(source, n);\n>>> REALLOC_ARRAY(destination, n);\n>>> @@ -332,7 +362,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>>> printf(_(\"Renaming %s to %s\\n\"), src, dst);\n>>> if (show_only)\n>>> continue;\n>>> - if (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n>>> + if (mode != INDEX && mode != SPARSE && mode != SPARSE_DIRECTORY &&\n\nAnd here we would write something like\n\n\tif (!(mode & (INDEX | SPARSE | SPARSE_DIRECTORY)) &&\n\n>>> + rename(src, dst) < 0) {\n>>> if (ignore_errors)\n>>> continue;\n>>> die_errno(_(\"renaming '%s' failed\"), src);\n>>> @@ -346,7 +377,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>>> 1);\n>>> }\n>>>\n>>> - if (mode == WORKING_DIRECTORY)\n>>> + if (mode == WORKING_DIRECTORY || mode == SPARSE_DIRECTORY)\n\nand here:\n\n\tif (mode & (WORKING_DIRECTORY | SPARSE_DIRECTORY))\n\nThis requires changing your enum definition. It got lost in the previous\nquoting, it seems, but here it is again:\n\n>>> @@ -129,7 +148,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>>> \t\tOPT_END(),\n>>> \t};\n>>> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n>>> -\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n>>> +\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE,\n>>> +\tSPARSE_DIRECTORY } *modes;\n>>> \tstruct stat st;\n>>> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n>>> \tstruct lock_file lock_file = LOCK_INIT;\n\nI think it is time to split out \"enum update_mode\" to the top of the file\ninstead of defining it inline here. Here is what it could look like:\n\nenum update_mode {\n\tBOTH = 0,\n\tWORKING_DIRECTORY = (1 << 1),\n\tINDEX = (1 << 2),\n\tSPARSE = (1 << 3),\n};\n\n(This is how it would look before adding your new value.)\n\nI can imagine making a new commit that does the following:\n\n* Move update_mode to the top and set the values to be independent bits.\n* Change \"mode[i] =\" to \"mode[i] |=\"\n* Change \"mode ==\" checks to \"mode &\" checks.\n\nThink about it.\n\n>> I'm a bit confused - doesn't this mean the sparse dir move will be skipped?\n>> In your commit description, you mention that this 'mv' succeeds with the\n>> '--sparse' option, but I don't see any place where the sparse directory\n>> would be moved.\n> \n> Well, you know the drill, I did not use \"sparse directory\" correctly, let alone\n> 'SPARSE_DIRECTORY' enum bit in this hunk. I think it makes some sense\n> if you apply my actual meaning of 'SPARSE_DIRECTORY' here (it should be\n> something like OUT_OF_CONE_WORKING_DIRECTORY)? Because such\n> directory is not on disk, it cannot be \"rename()\"d, and should also skip the\n> \"rename_cache_entry_at()\" function. If all the files under the directory are\n> moved/renamed, then (in my opinion) the directory is both moved to the\n> destination,\n> both in the worktree and in the index.\n\nPerhaps a better name would be SKIP_WORKTREE_DIR.\n\nBut yes, we need to make sure that all cache entries under the directory\nhave their SKIP_WORKTREE bits re-checked and any that lose the bit need to\nbe written to the worktree.\n\nI wonder if it is as simple as marking a boolean that says \"I moved at\nleast one sparse entry\" and then calling update_sparsity() at the end of\ncmd_mv() if that boolean is true.\n\nThanks,\n-Stolee\n"},{"id":"452899","messageId":"c97cfe34-50d3-f7dd-bfc3-428d4518f308@github.com","threadId":"57642","inReplyTo":"20220331091755.385961-4-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-04-01T14:55:56Z","receivedAt":"2022-04-01T15:55:13Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 3/31/2022 5:17 AM, Shaoxuan Yuan wrote:\n> Originally, the SKIP_WORKTREE bit is not removed when moving an out-of-cone\n> file into sparse cone, thus the moved file does not show up in the working tree.\n> Hint the user to use \"git sparse-checkout reapply\" to reapply the sparsity rules\n> to the working tree, by which the SKIP_WORKTREE bit is removed.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> I offered this solution becasue I'm not sure how to turn a cache_entry's \n> ce_flags back to a non-sparse state. I tried directly set it to 0 like this:\n> \n> \tce->ce_flags = 0;\n> \n> But the behavior after this seems undefined. The file still won't show up\n> in the working tree.\n> \n> And I found that \"git sparse-checkout reapply\" seems to be a nice fit for the\n> job. But I guess if there is a way (there must be but I don't know) to do it\n> direcly in the code, that could also be nice.\n\nI point out lower down the API that the 'reapply' command uses to gain\nthis functionality.\n\n> \tif (gitmodules_modified)\n> @@ -392,6 +398,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t COMMIT_LOCK | SKIP_IF_UNCHANGED))\n> \t\tdie(_(\"Unable to write new index file\"));\n> \n> +\tif (advise_to_reapply)\n> +\t\tprintf(_(\"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\\n\"));\n> +\n\nI think we can skip the advice here and instead run update_sparsity()\nfrom unpack-trees.c (see update_working_directory() in builtin/sparse-checkout.c\nfor an example using it). Something along those lines would double-check the\nnew cache entries against the sparse-checkout patterns and update the worktree\nto match expectation.\n\nI originally thought we wouldn't want to run update_sparsity() unless we\ninteracted with a cache entry with SKIP_WORKTREE, but this could be important\nfor moving things from inside the cone to outside. In addition, in non-cone\nmode there is no way to predict how the new cache entry could match the\npatterns once its name changes.\n\nHopefully this gives you something to try, especially because it should be\nsimple to check that a file exists or not after the 'git mv' call, making\ntesting relatively easy.\n\nThanks,\n-Stolee\n"},{"id":"452970","messageId":"CAPig+cRg09zG+mRKXE+EKgX-quP85yBi+RZcW=xiqfVs9Z=yAQ@mail.gmail.com","threadId":"57642","inReplyTo":"220401.867d89p6sp.gmgdl@evledraar.gmail.com","subject":"Re: [WIP v1 3/4] mv: add advise_to_reapply hint for moving file into cone","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-04-03T02:01:48Z","receivedAt":"2022-04-03T02:02:05Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Apr 2, 2022 at 11:15 AM Ævar Arnfjörð Bjarmason\n wrote:\n> On Fri, Apr 01 2022, Shaoxuan Yuan wrote:\n> > I actually did use advise(), but I noticed that it prints to stderr\n> > and ... nevermind,\n>\n> We've typically used stderr in git not to mean \"error\", but to\n> distinguish \"chatty\" and non-primary output from non-chatty.\n>\n> So (leaving aside that we're unlikely to add advice to plumbing) if you\n> emitted a warning() or advice from git-ls-tree you should be able to run\n> something like:\n>\n> git ls-tree -r HEAD >output-for-a-script\n>\n> And get the advise() on stderr, while the \"primary\" output is on stdout.\n>\n> There's a recent-ish (last year or so) thread where I think Jeff King\n> explained this better than I'm doing here, but I couldn't find it with a\n> quick search.\n\nYou're probably thinking of [1] in which Peff referenced an earlier\nemail[2] in which he stated his thoughts on the subject. The result of\n[1] was that git-workree was changed[3] to use stderr rather than\nstdout for its chatty messages, and (significant to this discussion)\nCodingGuidelines was updated[4] to cover this topic, so we can now\nrefer people to CodingGuidelines.\n\n[1]: https://lore.kernel.org/git/YaaN0pibKWgjcVk3@coredump.intra.peff.net/\n[2]: https://lore.kernel.org/git/20110907215716.GJ13364@sigill.intra.peff.net/\n[3]: https://lore.kernel.org/git/20211203034420.47447-2-sunshine@sunshineco.com/\n[4]: https://lore.kernel.org/git/20211202223110.22062-1-sunshine@sunshineco.com/\n"},{"id":"453028","messageId":"CAJyCBOSQVA5oEW7_iKZW895ycQ4T=XAx422s4m+6VFDSDVeNMA@mail.gmail.com","threadId":"57642","inReplyTo":"22aadea2-9330-aa9e-7b6a-834585189144@github.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-04T07:25:23Z","receivedAt":"2022-04-04T07:25:38Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Fri, Apr 1, 2022 at 10:49 PM Derrick Stolee wrote:\n>\n> On 4/1/2022 8:49 AM, Shaoxuan Yuan wrote:> On Fri, Apr 1, 2022 at 5:28 AM Victoria Dye wrote:\n> >>\n> >> Shaoxuan Yuan wrote:\n> >>> Originally, moving a directory which is not on-disk due\n> >>> to its existence outside of sparse-checkout cone, \"giv mv\" command\n> >>> errors out with \"bad source\".\n> >>>\n> >>> Add a helper check_dir_in_index() function to see if a directory\n> >>> name exists in the index. Also add a SPARSE_DIRECTORY bit to mark\n> >>> such directories.\n> >>>\n> >>\n> >> Hmm, I think this patch would fit better in your eventual \"sparse index\n> >> integration\" series than this \"prerequisite fixes to sparse-checkout\"\n> >> series. Sparse directories *only* appear when you're using a sparse index\n> >> so, theoretically, this shouldn't ever come up (and thus isn't testable)\n> >> until you're using a sparse index.\n> >\n> > After reading your feedback, I realized that I totally misused\n> > the phrase \"sparse directory\". Clearly, this patch series does not\n> > deal with sparse-\n> > index yet, as \"sparse directory\" is a cache entry that points to a\n> > tree, if sparse-index\n> > is enabled. Silly me ;)\n> >\n> > What I was *actually* trying to say is: I want to change the checking\n> > logic of moving\n> > a \"directory that exists outside of sparse-checkout cone\", and I\n> > apparently misused\n> > \"sparse directory\" to reference such a thing.\n>\n> In the case of a full index (or an expanded sparse index, which is\n> currently always the case for `git mv`), you detect a sparse directory\n> by looking for the directory in the index, _not_ finding it, and then\n> seeing if the cache entry at the position where the directory _would_\n> exist is marked with the SKIP_WORKTREE bit.\n>\n> This works in cone mode and the old mode because I assume you've already\n> checked for the existence of the directory, so if there _was_ any\n> non-SKIP_WORKTREE cache entry within the directory, then the directory\n> would exist in the worktree.\n>\n> (These are good details to include in the commit message.)\n\nI read and think about this part a few times, but I'm still confused.\n\nAs Victoria pointed out earlier, and I quote, \"Sparse directories *only* appear\nwhen you're using a sparse index, so, theoretically, this shouldn't ever\ncome up (and thus isn't testable) until you're using a sparse index.\"\nSo I'm not so sure what do you mean by putting \"full index\" and \"sparse\ndirectory\" together.\n\nThus, I go ahead and try to detect a directory that is outside of\nsparse-checkout cone, without sparse-index enabled.\n\nI found a problem that if you use cache_name_pos() to do this\ndetection, I imagined the following example (I'm trying to imitate an\noutput of \"git ls-files -t\"):\n\nH a\nH b\nS d/file1\nH e/file1\n\nSo in this index, I use cache_name_pos() to find a directory \"c/\". I imagine the\nthe value returned would be -3, which indicates this directory would be inserted\nat index position 2. However, the cache entry at position 2 is\n\"d/file1\", which is\nmarked with SKIP_WORKTREE, and this fact cannot guarantee that \"c/\" is\na sparse directory, since ''c/\" is not in the index per se.\n\nProbably I'm missing something, or I'm just dumb.\n\n> Thanks,\n> -Stolee\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"453029","messageId":"CAJyCBOREzrGSwzjAz_89q1+O2X-qqa-qB2ywPVGU=q37O-QdzQ@mail.gmail.com","threadId":"57642","inReplyTo":"CAJyCBOSQVA5oEW7_iKZW895ycQ4T=XAx422s4m+6VFDSDVeNMA@mail.gmail.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-04T07:49:43Z","receivedAt":"2022-04-04T07:50:00Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Mon, Apr 4, 2022 at 3:25 PM Shaoxuan Yuan wrote:\n> I read and think about this part a few times, but I'm still confused.\n>\n> As Victoria pointed out earlier, and I quote, \"Sparse directories *only* appear\n> when you're using a sparse index, so, theoretically, this shouldn't ever\n> come up (and thus isn't testable) until you're using a sparse index.\"\n> So I'm not so sure what do you mean by putting \"full index\" and \"sparse\n> directory\" together.\n>\n> Thus, I go ahead and try to detect a directory that is outside of\n> sparse-checkout cone, without sparse-index enabled.\n>\n> I found a problem that if you use cache_name_pos() to do this\n> detection, I imagined the following example (I'm trying to imitate an\n> output of \"git ls-files -t\"):\n>\n> H a\n> H b\n> S d/file1\n> H e/file1\n>\n> So in this index, I use cache_name_pos() to find a directory \"c/\". I imagine the\n> the value returned would be -3, which indicates this directory would be inserted\n> at index position 2. However, the cache entry at position 2 is\n> \"d/file1\", which is\n> marked with SKIP_WORKTREE, and this fact cannot guarantee that \"c/\" is\n> a sparse directory, since ''c/\" is not in the index per se.\n>\n> Probably I'm missing something, or I'm just dumb.\n\nThough I think doing a strncmp() after the cache_name_pos()\ncan get the job done :)\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"453034","messageId":"68e1703d-7a4a-cc83-f344-494533297e3d@github.com","threadId":"57642","inReplyTo":"CAJyCBOREzrGSwzjAz_89q1+O2X-qqa-qB2ywPVGU=q37O-QdzQ@mail.gmail.com","subject":"Re: [WIP v1 2/4] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-04-04T12:43:44Z","receivedAt":"2022-04-04T12:43:54Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 4/4/2022 3:49 AM, Shaoxuan Yuan wrote:\n> On Mon, Apr 4, 2022 at 3:25 PM Shaoxuan Yuan wrote:\n>> I read and think about this part a few times, but I'm still confused.\n>>\n>> As Victoria pointed out earlier, and I quote, \"Sparse directories *only* appear\n>> when you're using a sparse index, so, theoretically, this shouldn't ever\n>> come up (and thus isn't testable) until you're using a sparse index.\"\n>> So I'm not so sure what do you mean by putting \"full index\" and \"sparse\n>> directory\" together.\n>>\n>> Thus, I go ahead and try to detect a directory that is outside of\n>> sparse-checkout cone, without sparse-index enabled.\n>>\n>> I found a problem that if you use cache_name_pos() to do this\n>> detection, I imagined the following example (I'm trying to imitate an\n>> output of \"git ls-files -t\"):\n>>\n>> H a\n>> H b\n>> S d/file1\n>> H e/file1\n>>\n>> So in this index, I use cache_name_pos() to find a directory \"c/\". I imagine the\n>> the value returned would be -3, which indicates this directory would be inserted\n>> at index position 2. However, the cache entry at position 2 is\n>> \"d/file1\", which is\n>> marked with SKIP_WORKTREE, and this fact cannot guarantee that \"c/\" is\n>> a sparse directory, since ''c/\" is not in the index per se.\n\nI was thinking more about the case where you find a tracked directory that\nhas all contained files marked with SKIP_WORTKREE.\n\nIf you search for \"d/\" you will also get -3. Then, you will see that at\nposition 2 the cache entry d/file1 has SKIP_WORKTREE. The directory \"d/\"\nshould exist on disk if there are any entries starting with \"d/\" that do\nnot have the SKIP_WORKTREE bit.\n\nInteresting things happen if you are in the scenario where d/f is in the\ncone-mode sparse-checkout definition and you see this list of cache entries:\n\nS d/a\nH d/f/b\nS d/g\n\nAgain, d/f/b should imply the existence of the directory d/ in the worktree.\n>> Probably I'm missing something, or I'm just dumb.\n> \n> Though I think doing a strncmp() after the cache_name_pos()\n> can get the job done :)\n \nI think this is the way to do it, including index_range_of_same_dir() in\nbuiltin/mv.c.\n\nThanks,\n-Stolee\n"},{"id":"453331","messageId":"CAJyCBOQz-ns2hHUQ-Kg-UOMfutV7H=MbQMcdO_=oxYKrXfjzjA@mail.gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v1 0/4] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-04-08T12:22:42Z","receivedAt":"2022-04-08T12:22:58Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Thu, Mar 31, 2022 at 5:20 PM Shaoxuan Yuan wrote:\n>\n> Before integrating 'mv' with sparse-index, I still find some possibly buggy\n> UX when 'mv' is interacting with 'sparse-checkout'.\n>\n> So I kept sparse-index off in order to sort things out without a sparse index.\n> We can proceed to integrate with sparse-index once these changes are solid.\n>\n> Note that this patch is tentative, and still have known glitches, but it\n> illustrates a general approach that I intended to harmonize 'mv'\n> with 'sparse-checkout'.\n>\n> Shaoxuan Yuan (4):\n> mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n> mv: add check_dir_in_index() and solve general dir check issue\n> mv: add advise_to_reapply hint for moving file into cone\n> t7002: add tests for moving out-of-cone file/directory\n>\n> builtin/mv.c | 76 ++++++++++++++++++++++++++++++++---\n> t/t7002-mv-sparse-checkout.sh | 72 +++++++++++++++++++++++++++++++++\n> 2 files changed, 142 insertions(+), 6 deletions(-)\n>\n>\n> base-commit: 805e0a68082a217f0112db9ee86a022227a9c81b\n> --\n> 2.35.1\n>\n\nHi, to whom it may concern,\n\nI'm writing to say that I'm making useful (possibly) progress on this topic.\n\nI've been busy composing a GSoC proposal last week, so the progress\npaused for a while. And I'm going to have a trip for around a week, starting\nnext week, so I may not have enough time next week to push forward on\nthis topic.\n\nBut I will always be available through email ;-)\n\n\n--\nThanks & Regards,\nShaoxuan\n"},{"id":"456273","messageId":"20220527100804.209890-1-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v2 0/5] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-27T10:07:59Z","receivedAt":"2022-05-27T10:08:19Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"## Changes since WIP v1 ##\n\n1. Move t7002 tests to the front and turn corresponding tests to \n test_expect_success along with corresponding commits.\n\n2. Add two tests to t7002.\n\n3. Update check_dir_in_index() and added corresponding documentation.\n\n4. Turn update_mode into enum flags.\n\n5. Use update_sparsity() to replace advise*() function after touching\n sparse contents (this change is INCOMPLETE, NEED FIX).\n\n6. Fix some format issues.\n\n## Limitations ##\n\nThis series has not considered moving file from in-cone area to out-of-cone\narea *yet*. Moving from in-cone to out-of-cone has not been covered/tested. The\nplan is to add the \"in-cone to out-of-cone\" functionality later, since this \nseries is WIP for now.\n\nShaoxuan Yuan (5):\n t7002: add tests for moving out-of-cone file/directory\n mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n mv: check if exists in index to handle overwriting\n mv: add check_dir_in_index() and solve general dir check issue\n mv: use update_sparsity() after touching sparse contents\n\n builtin/mv.c | 104 +++++++++++++++++++++++++++++--\n t/t7002-mv-sparse-checkout.sh | 114 ++++++++++++++++++++++++++++++++++\n 2 files changed, 212 insertions(+), 6 deletions(-)\n\nRange-diff against v1:\n4: 1dd2fcb234 ! 1: 485d1e9102 t7002: add tests for moving out-of-cone file/directory\n @@ Commit message\n * 'can move out-of-cone directory with --sparse'\n * 'refuse to move out-of-cone file without --sparse'\n * 'can move out-of-cone file with --sparse'\n + * 'refuse to move sparse file to existing destination'\n + * 'move sparse file to existing destination with --force and --sparse'\n \n Signed-off-by: Shaoxuan Yuan \n \n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n \ttest_cmp expect stderr\n '\n \n -+test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n ++test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n +\tgit sparse-checkout disable &&\n +\tgit reset --hard &&\n +\tmkdir folder1 &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +\ttest_cmp expect stderr\n +'\n +\n -+test_expect_success 'can move out-of-cone directory with --sparse' '\n ++test_expect_failure 'can move out-of-cone directory with --sparse' '\n +\tgit sparse-checkout disable &&\n +\tgit reset --hard &&\n +\tmkdir folder1 &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +\n +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n +\ttest_must_be_empty stderr &&\n -+\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n -+\t>expect &&\n -+\ttest_cmp actual expect &&\n +\n +\tgit sparse-checkout reapply &&\n +\ttest_path_is_dir sub/folder1 &&\n +\ttest_path_is_file sub/folder1/file1\n +'\n +\n -+test_expect_success 'refuse to move out-of-cone file without --sparse' '\n ++test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n +\tgit sparse-checkout disable &&\n +\tgit reset --hard &&\n +\tmkdir folder1 &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +\ttest_cmp expect stderr\n +'\n +\n -+test_expect_success 'can move out-of-cone file with --sparse' '\n ++test_expect_failure 'can move out-of-cone file with --sparse' '\n +\tgit sparse-checkout disable &&\n +\tgit reset --hard &&\n +\tmkdir folder1 &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +\n +\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n +\ttest_must_be_empty stderr &&\n -+\techo \"Please use \\\"git sparse-checkout reapply\\\" to reapply the sparsity.\"\\\n -+\t>expect &&\n -+\ttest_cmp actual expect &&\n +\n +\tgit sparse-checkout reapply &&\n +\t! test_path_is_dir sub/folder1 &&\n +\ttest_path_is_file sub/file1\n +'\n ++\n ++test_expect_failure 'refuse to move sparse file to existing destination' '\n ++\tgit sparse-checkout disable &&\n ++\tgit reset --hard &&\n ++\tmkdir folder1 &&\n ++\ttouch folder1/file1 &&\n ++\ttouch sub/file1 &&\n ++\tgit add folder1 sub/file1 &&\n ++\tgit sparse-checkout init --cone &&\n ++\tgit sparse-checkout set sub &&\n ++\n ++\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n ++\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n ++\ttest_cmp expect stderr\n ++'\n ++\n ++test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n ++\tgit sparse-checkout disable &&\n ++\tgit reset --hard &&\n ++\tmkdir folder1 &&\n ++\ttouch folder1/file1 &&\n ++\ttouch sub/file1 &&\n ++\techo \"overwrite\" >folder1/file1 &&\n ++\tgit add folder1 sub/file1 &&\n ++\tgit sparse-checkout init --cone &&\n ++\tgit sparse-checkout set sub &&\n ++\n ++\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n ++\ttest_must_be_empty stderr &&\n ++\techo \"overwrite\" >expect &&\n ++\ttest_cmp expect sub/file1\n ++'\n +\n test_done\n1: 5cf6b860e3 ! 2: c99df4fc1a mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tbad = _(\"bad source\");\n \t\t} else if (!strncmp(src, dst, length) &&\n \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n +\n + ## t/t7002-mv-sparse-checkout.sh ##\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'can move out-of-cone directory with --sparse' '\n + \ttest_path_is_file sub/folder1/file1\n + '\n + \n +-test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n ++test_expect_success 'refuse to move out-of-cone file without --sparse' '\n + \tgit sparse-checkout disable &&\n + \tgit reset --hard &&\n + \tmkdir folder1 &&\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n + \ttest_cmp expect stderr\n + '\n + \n +-test_expect_failure 'can move out-of-cone file with --sparse' '\n ++test_expect_success 'can move out-of-cone file with --sparse' '\n + \tgit sparse-checkout disable &&\n + \tgit reset --hard &&\n + \tmkdir folder1 &&\n-: ---------- > 3: 8f1193188b mv: check if exists in index to handle overwriting\n2: 7b3c931f3f ! 4: e195bfbc73 mv: add check_dir_in_index() and solve general dir check issue\n @@ Commit message\n errors out with \"bad source\".\n \n Add a helper check_dir_in_index() function to see if a directory\n - name exists in the index. Also add a SPARSE_DIRECTORY bit to mark\n + name exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\n such directories.\n \n Change the checking logic, so that such directory makes\n @@ Commit message\n instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n this operation can be carried out successfully.\n \n + Also, as suggested by Derrick [1],\n + move the in-line definition of \"enum update_mode\" to the top\n + of the file and make it use \"flags\" mode (each state is a different\n + bit in the word).\n +\n + [1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n +\n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n +@@ builtin/mv.c: static const char * const builtin_mv_usage[] = {\n + \tNULL\n + };\n + \n ++enum update_mode {\n ++\tBOTH = 0,\n ++\tWORKING_DIRECTORY = (1 << 1),\n ++\tINDEX = (1 << 2),\n ++\tSPARSE = (1 << 3),\n ++\tSKIP_WORKTREE_DIR = (1 << 4),\n ++};\n ++\n + #define DUP_BASENAME 1\n + #define KEEP_TRAILING_SLASH 2\n + \n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n \n -+static int check_dir_in_index(const char *dir)\n ++/*\n ++ * Check if an out-of-cone directory should be in the index. Imagine this case\n ++ * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n ++ * and thus the directory is sparsified.\n ++ *\n ++ * Return 0 if such directory exist (i.e. with any of its contained files not\n ++ * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n ++ * Return 1 otherwise.\n ++ */\n ++static int check_dir_in_index(const char *name, int namelen)\n +{\n -+\tint ret = 0;\n -+\tint length = sizeof(dir) + 1;\n -+\tchar *substr = malloc(length);\n ++\tint ret = 1;\n ++\tconst char *with_slash = add_slash(name);\n ++\tint length = namelen + 1;\n +\n -+\tfor (int i = 0; i < the_index.cache_nr; i++) {\n -+\t\tmemcpy(substr, the_index.cache[i]->name, length);\n -+\t\tmemset(substr + length - 1, 0, 1);\n ++\tint pos = cache_name_pos(with_slash, length);\n ++\tconst struct cache_entry *ce;\n +\n -+\t\tif (strcmp(dir, substr) == 0) {\n -+\t\t\tret = 1;\n ++\tif (pos < 0) {\n ++\t\tpos = -pos - 1;\n ++\t\tif (pos >= the_index.cache_nr)\n +\t\t\treturn ret;\n -+\t\t}\n ++\t\tce = active_cache[pos];\n ++\t\tif (strncmp(with_slash, ce->name, length))\n ++\t\t\treturn ret;\n ++\t\tif (ce_skip_worktree(ce))\n ++\t\t\treturn ret = 0;\n +\t}\n -+\tfree(substr);\n +\treturn ret;\n +}\n +\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n -\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n -+\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE,\n -+\tSPARSE_DIRECTORY } *modes;\n ++\tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\t */\n + \t\tif (lstat(src, &st) < 0) {\n \n \t\t\tint pos = cache_name_pos(src, length);\n +\t\t\tconst char *src_w_slash = add_slash(src);\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\telse\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t}\n -+\t\t\telse if (check_dir_in_index(src_w_slash) &&\n -+\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n -+\t\t\t\tmodes[i] = SPARSE_DIRECTORY;\n ++\t\t\telse if (!check_dir_in_index(src, length) &&\n ++\t\t\t\t\t !path_in_sparse_checkout(src_w_slash, &the_index)) {\n ++\t\t\t\tmodes[i] = SKIP_WORKTREE_DIR;\n +\t\t\t\tgoto dir_check;\n +\t\t\t}\n \t\t\t/* only error if existence is expected. */\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \n -\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n +\t\t\t\tif (!modes[i])\n -+\t\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n ++\t\t\t\t\tmodes[i] |= WORKING_DIRECTORY;\n \t\t\t\tn = argc + last - first;\n \t\t\t\tREALLOC_ARRAY(source, n);\n \t\t\t\tREALLOC_ARRAY(destination, n);\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (show_only)\n \t\t\tcontinue;\n -\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n -+\t\tif (mode != INDEX && mode != SPARSE && mode != SPARSE_DIRECTORY &&\n -+\t\t rename(src, dst) < 0) {\n ++\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n ++\t\t \trename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t}\n \n -\t\tif (mode == WORKING_DIRECTORY)\n -+\t\tif (mode == WORKING_DIRECTORY || mode == SPARSE_DIRECTORY)\n ++\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\n +\n + ## t/t7002-mv-sparse-checkout.sh ##\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n + \ttest_cmp expect stderr\n + '\n + \n +-test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n ++test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n + \tgit sparse-checkout disable &&\n + \tgit reset --hard &&\n + \tmkdir folder1 &&\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n + \ttest_cmp expect stderr\n + '\n + \n +-test_expect_failure 'can move out-of-cone directory with --sparse' '\n ++test_expect_success 'can move out-of-cone directory with --sparse' '\n + \tgit sparse-checkout disable &&\n + \tgit reset --hard &&\n + \tmkdir folder1 &&\n3: 4be4c4f34d < -: ---------- mv: add advise_to_reapply hint for moving file into cone\n-: ---------- > 5: aa82ba56b0 mv: use update_sparsity() after touching sparse contents\n-- \n2.35.1\n\n"},{"id":"456274","messageId":"20220527100804.209890-2-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v2 1/5] t7002: add tests for moving out-of-cone file/directory","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-27T10:08:00Z","receivedAt":"2022-05-27T10:08:22Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test following situations:\n\n* 'refuse to move out-of-cone directory without --sparse'\n* 'can move out-of-cone directory with --sparse'\n* 'refuse to move out-of-cone file without --sparse'\n* 'can move out-of-cone file with --sparse'\n* 'refuse to move sparse file to existing destination'\n* 'move sparse file to existing destination with --force and --sparse'\n\nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 98 +++++++++++++++++++++++++++++++++++\n 1 file changed, 98 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 1d3d2aca21..963cb512e2 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -206,4 +206,102 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n+test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\ttest_must_fail git mv folder1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone directory with --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\tgit sparse-checkout reapply &&\n+\ttest_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/folder1/file1\n+'\n+\n+test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone file with --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\tgit sparse-checkout reapply &&\n+\t! test_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/file1\n+'\n+\n+test_expect_failure 'refuse to move sparse file to existing destination' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n+\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\techo \"overwrite\" >folder1/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout init --cone &&\n+\tgit sparse-checkout set sub &&\n+\n+\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\techo \"overwrite\" >expect &&\n+\ttest_cmp expect sub/file1\n+'\n+\n test_done\n-- \n2.35.1\n\n"},{"id":"456275","messageId":"20220527100804.209890-3-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v2 2/5] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-27T10:08:01Z","receivedAt":"2022-05-27T10:08:26Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a file which is not on-disk but exists in\nindex as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\nout with \"bad source\".\n\nChange the checking logic, so that such \nfile makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 26 +++++++++++++++++++++++++-\n t/t7002-mv-sparse-checkout.sh | 4 ++--\n 2 files changed, 27 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba83..32ad4d5682 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -185,8 +185,32 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n+\t\t\t/*\n+\t\t\t * TODO: for now, when you try to overwrite a \n+\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n+\t\t\t * flag, then the action will be done without providing \"--force\"\n+\t\t\t * and no warning.\n+\t\t\t *\n+\t\t\t * This is mainly because the sparse \n+\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n+\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n+\t\t\t */\n+\n+\t\t\tint pos = cache_name_pos(src, length);\n+\t\t\tif (pos >= 0) {\n+\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n+\n+\t\t\t\tif (ce_skip_worktree(ce)) {\n+\t\t\t\t\tif (!ignore_sparse)\n+\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\t\t\telse\n+\t\t\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\t}\n+\t\t\t\telse\n+\t\t\t\t\tbad = _(\"bad source\");\n+\t\t\t}\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\telse if (modes[i] != SPARSE)\n \t\t\t\tbad = _(\"bad source\");\n \t\t} else if (!strncmp(src, dst, length) &&\n \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 963cb512e2..581ef4c0f6 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -239,7 +239,7 @@ test_expect_failure 'can move out-of-cone directory with --sparse' '\n \ttest_path_is_file sub/folder1/file1\n '\n \n-test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+test_expect_success 'refuse to move out-of-cone file without --sparse' '\n \tgit sparse-checkout disable &&\n \tgit reset --hard &&\n \tmkdir folder1 &&\n@@ -255,7 +255,7 @@ test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone file with --sparse' '\n+test_expect_success 'can move out-of-cone file with --sparse' '\n \tgit sparse-checkout disable &&\n \tgit reset --hard &&\n \tmkdir folder1 &&\n-- \n2.35.1\n\n"},{"id":"456276","messageId":"20220527100804.209890-4-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v2 3/5] mv: check if exists in index to handle overwriting","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-27T10:08:02Z","receivedAt":"2022-05-27T10:08:28Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a sparse file into cone can result in unwarned\noverwrite of existing entry. The expected behavior is that if the\n exists in the entry, user should be prompted to supply\na [-f|--force] to carry out the operation, or the operation should\nfail.\n\nAdd a check mechanism to do that.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 23 +++++++++++------------\n t/t7002-mv-sparse-checkout.sh | 2 +-\n 2 files changed, 12 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 32ad4d5682..62284e3f86 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -185,16 +185,6 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n-\t\t\t/*\n-\t\t\t * TODO: for now, when you try to overwrite a \n-\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n-\t\t\t * flag, then the action will be done without providing \"--force\"\n-\t\t\t * and no warning.\n-\t\t\t *\n-\t\t\t * This is mainly because the sparse \n-\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n-\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n-\t\t\t */\n \n \t\t\tint pos = cache_name_pos(src, length);\n \t\t\tif (pos >= 0) {\n@@ -203,8 +193,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tif (ce_skip_worktree(ce)) {\n \t\t\t\t\tif (!ignore_sparse)\n \t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\t\t\telse\n-\t\t\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\t\telse {\n+\t\t\t\t\t\t/* Check if dst exists in index */\n+\t\t\t\t\t\tif (cache_name_pos(dst, strlen(dst)) >= 0) {\n+\t\t\t\t\t\t\tif (force)\n+\t\t\t\t\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\t\t\t\telse\n+\t\t\t\t\t\t\t\tbad = _(\"destination exists\");\n+\t\t\t\t\t\t}\n+\t\t\t\t\t\telse\n+\t\t\t\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\t\t}\n \t\t\t\t}\n \t\t\t\telse\n \t\t\t\t\tbad = _(\"bad source\");\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 581ef4c0f6..2c9008573a 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -272,7 +272,7 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_path_is_file sub/file1\n '\n \n-test_expect_failure 'refuse to move sparse file to existing destination' '\n+test_expect_success 'refuse to move sparse file to existing destination' '\n \tgit sparse-checkout disable &&\n \tgit reset --hard &&\n \tmkdir folder1 &&\n-- \n2.35.1\n\n"},{"id":"456277","messageId":"20220527100804.209890-5-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v2 4/5] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-27T10:08:03Z","receivedAt":"2022-05-27T10:08:36Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a directory which is not on-disk due\nto its existence outside of sparse-checkout cone, \"giv mv\" command\nerrors out with \"bad source\".\n\nAdd a helper check_dir_in_index() function to see if a directory\nname exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\nsuch directories.\n\nChange the checking logic, so that such directory makes\n\"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nAlso, as suggested by Derrick [1],\nmove the in-line definition of \"enum update_mode\" to the top\nof the file and make it use \"flags\" mode (each state is a different\nbit in the word).\n\n[1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 60 ++++++++++++++++++++++++++++++++---\n t/t7002-mv-sparse-checkout.sh | 4 +--\n 2 files changed, 57 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 62284e3f86..e64f251a69 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -19,6 +19,14 @@ static const char * const builtin_mv_usage[] = {\n \tNULL\n };\n \n+enum update_mode {\n+\tBOTH = 0,\n+\tWORKING_DIRECTORY = (1 << 1),\n+\tINDEX = (1 << 2),\n+\tSPARSE = (1 << 3),\n+\tSKIP_WORKTREE_DIR = (1 << 4),\n+};\n+\n #define DUP_BASENAME 1\n #define KEEP_TRAILING_SLASH 2\n \n@@ -115,6 +123,37 @@ static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n \n+/*\n+ * Check if an out-of-cone directory should be in the index. Imagine this case\n+ * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n+ * and thus the directory is sparsified.\n+ *\n+ * Return 0 if such directory exist (i.e. with any of its contained files not\n+ * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n+ * Return 1 otherwise.\n+ */\n+static int check_dir_in_index(const char *name, int namelen)\n+{\n+\tint ret = 1;\n+\tconst char *with_slash = add_slash(name);\n+\tint length = namelen + 1;\n+\n+\tint pos = cache_name_pos(with_slash, length);\n+\tconst struct cache_entry *ce;\n+\n+\tif (pos < 0) {\n+\t\tpos = -pos - 1;\n+\t\tif (pos >= the_index.cache_nr)\n+\t\t\treturn ret;\n+\t\tce = active_cache[pos];\n+\t\tif (strncmp(with_slash, ce->name, length))\n+\t\t\treturn ret;\n+\t\tif (ce_skip_worktree(ce))\n+\t\t\treturn ret = 0;\n+\t}\n+\treturn ret;\n+}\n+\n int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n@@ -129,7 +168,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n-\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n+\tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n@@ -187,6 +226,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (lstat(src, &st) < 0) {\n \n \t\t\tint pos = cache_name_pos(src, length);\n+\t\t\tconst char *src_w_slash = add_slash(src);\n+\n \t\t\tif (pos >= 0) {\n \t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n \n@@ -208,6 +249,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\telse\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t}\n+\t\t\telse if (!check_dir_in_index(src, length) &&\n+\t\t\t\t\t !path_in_sparse_checkout(src_w_slash, &the_index)) {\n+\t\t\t\tmodes[i] = SKIP_WORKTREE_DIR;\n+\t\t\t\tgoto dir_check;\n+\t\t\t}\n \t\t\t/* only error if existence is expected. */\n \t\t\telse if (modes[i] != SPARSE)\n \t\t\t\tbad = _(\"bad source\");\n@@ -218,7 +264,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t&& lstat(dst, &st) == 0)\n \t\t\tbad = _(\"cannot move directory over file\");\n \t\telse if (src_is_dir) {\n-\t\t\tint first = cache_name_pos(src, length), last;\n+\t\t\tint first, last;\n+dir_check:\n+\t\t\tfirst = cache_name_pos(src, length);\n \n \t\t\tif (first >= 0)\n \t\t\t\tprepare_move_submodule(src, first,\n@@ -229,7 +277,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\telse { /* last - first >= 1 */\n \t\t\t\tint j, dst_len, n;\n \n-\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\t\tif (!modes[i])\n+\t\t\t\t\tmodes[i] |= WORKING_DIRECTORY;\n \t\t\t\tn = argc + last - first;\n \t\t\t\tREALLOC_ARRAY(source, n);\n \t\t\t\tREALLOC_ARRAY(destination, n);\n@@ -331,7 +380,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n+\t\t \trename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n@@ -345,7 +395,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode == WORKING_DIRECTORY)\n+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 2c9008573a..cf2f5dc46f 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -206,7 +206,7 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n \tgit sparse-checkout disable &&\n \tgit reset --hard &&\n \tmkdir folder1 &&\n@@ -222,7 +222,7 @@ test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone directory with --sparse' '\n+test_expect_success 'can move out-of-cone directory with --sparse' '\n \tgit sparse-checkout disable &&\n \tgit reset --hard &&\n \tmkdir folder1 &&\n-- \n2.35.1\n\n"},{"id":"456278","messageId":"20220527100804.209890-6-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-27T10:08:04Z","receivedAt":"2022-05-27T10:08:49Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, \"git mv\" a sparse file/directory from out/in-cone to\nin/out-cone does not update the sparsity following the sparse-checkout\npatterns.\n\nUse update_sparsity() after touching sparse contents, so the sparsity\nwill be updated after the move.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 19 +++++++++++++++++++\n t/t7002-mv-sparse-checkout.sh | 16 ++++++++++++++++\n 2 files changed, 35 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex e64f251a69..2c02120941 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -13,6 +13,7 @@\n #include \"string-list.h\"\n #include \"parse-options.h\"\n #include \"submodule.h\"\n+#include \"unpack-trees.h\"\n \n static const char * const builtin_mv_usage[] = {\n \tN_(\"git mv [] ... \"),\n@@ -158,6 +159,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n+\tint sparse_moved = 0;\n \tstruct option builtin_mv_options[] = {\n \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n@@ -376,6 +378,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tif (!sparse_moved && mode & (SPARSE | SKIP_WORKTREE_DIR))\n+\t\t\tsparse_moved = 1;\n \t\tif (show_only || verbose)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n@@ -403,6 +407,21 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\trename_cache_entry_at(pos, dst);\n \t}\n \n+\tif (sparse_moved) {\n+\t\tstruct unpack_trees_options o;\n+\t\tmemset(&o, 0, sizeof(o));\n+\t\to.verbose_update = isatty(2);\n+\t\to.update = 1;\n+\t\to.head_idx = -1;\n+\t\to.src_index = &the_index;\n+\t\to.dst_index = &the_index;\n+\t\to.skip_sparse_checkout = 0;\n+\t\to.pl = the_index.sparse_checkout_patterns;\n+\t\tsetup_unpack_trees_porcelain(&o, \"mv\");\n+\t\tupdate_sparsity(&o);\n+\t\tclear_unpack_trees_porcelain(&o);\n+\t}\n+\n \tif (gitmodules_modified)\n \t\tstage_updated_gitmodules(&the_index);\n \ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex cf2f5dc46f..1fd3e3c0fc 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -287,6 +287,22 @@ test_expect_success 'refuse to move sparse file to existing destination' '\n \ttest_cmp expect stderr\n '\n \n+# Need fix.\n+#\n+# The *expected* behavior:\n+#\n+# Using --sparse to accept a sparse file, --force to overwrite the destination.\n+# The folder1/file1 should replace the sub/file1 without error.\n+#\n+# The *actual* behavior:\n+#\n+# It emits a warning:\n+#\n+# warning: Path ' sub/file1\n+# ' already present; will not overwrite with sparse update.\n+# After fixing the above paths, you may want to run `git sparse-checkout\n+# reapply`.\n+\n test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n \tgit sparse-checkout disable &&\n \tgit reset --hard &&\n-- \n2.35.1\n\n"},{"id":"456280","messageId":"220527.86wne72n55.gmgdl@evledraar.gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-2-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 1/5] t7002: add tests for moving out-of-cone file/directory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-05-27T12:07:46Z","receivedAt":"2022-05-27T12:30:51Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, May 27 2022, Shaoxuan Yuan wrote:\n\n> +test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n\nWe don't usually use \"touch file\", don't you mean just \">file\" here and\nbelow?\n\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\ttest_must_fail git mv folder1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone directory with --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n\nuse e.g. \"out\" and \"err\" instead of \"actual\" and \"stderr\". I.e. when we\ntest both we use something like that usually.\n"},{"id":"456281","messageId":"220527.86sfov2n26.gmgdl@evledraar.gmail.com","threadId":"57642","inReplyTo":"20220527100804.209890-6-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-05-27T12:10:00Z","receivedAt":"2022-05-27T12:31:47Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, May 27 2022, Shaoxuan Yuan wrote:\n\n> +\tif (sparse_moved) {\n> +\t\tstruct unpack_trees_options o;\n> +\t\tmemset(&o, 0, sizeof(o));\n> +\t\to.verbose_update = isatty(2);\n> +\t\to.update = 1;\n> +\t\to.head_idx = -1;\n> +\t\to.src_index = &the_index;\n> +\t\to.dst_index = &the_index;\n> +\t\to.skip_sparse_checkout = 0;\n> +\t\to.pl = the_index.sparse_checkout_patterns;\n\nYou can drop the memset here and use the designated init syntax\ninstead. I.e. \"struct x o = { .verbose_update .... };\"\n"},{"id":"456292","messageId":"8ced7008-5d52-01a6-4ebc-467343a2f11d@github.com","threadId":"57642","inReplyTo":"20220527100804.209890-2-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 1/5] t7002: add tests for moving out-of-cone file/directory","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-05-27T14:48:05Z","receivedAt":"2022-05-27T14:48:10Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 5/27/2022 6:08 AM, Shaoxuan Yuan wrote:\n> Add corresponding tests to test following situations:\n> \n> * 'refuse to move out-of-cone directory without --sparse'\n> * 'can move out-of-cone directory with --sparse'\n> * 'refuse to move out-of-cone file without --sparse'\n> * 'can move out-of-cone file with --sparse'\n> * 'refuse to move sparse file to existing destination'\n> * 'move sparse file to existing destination with --force and --sparse'\n\nStyle nit: bulleted lists like this don't add too much value\non top of reading the patch. You can use prose to describe how\nyou decided that these tests are the ones to write. Something\nlike:\n\n We do not have sufficient coverage of moving files outside\n of a sparse-checkout cone. Create new tests covering this\n behavior, keeping in mind that the user can include --sparse\n (or not), move a file or directory, and the destination can\n already exist in the index.\n\nThanks,\n-Stolee\n"},{"id":"456294","messageId":"0884b97b-0745-5cad-3034-a679be5d6c3a@github.com","threadId":"57642","inReplyTo":"20220527100804.209890-3-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 2/5] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-05-27T15:13:35Z","receivedAt":"2022-05-27T15:13:41Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"\n\nOn 5/27/2022 6:08 AM, Shaoxuan Yuan wrote:\n> Originally, moving a file which is not on-disk but exists in\n> index as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\n> out with \"bad source\".\n> \n> Change the checking logic, so that such \n> file makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n> this operation can be carried out successfully.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 26 +++++++++++++++++++++++++-\n> t/t7002-mv-sparse-checkout.sh | 4 ++--\n> 2 files changed, 27 insertions(+), 3 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 83a465ba83..32ad4d5682 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -185,8 +185,32 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \n> \t\tlength = strlen(src);\n> \t\tif (lstat(src, &st) < 0) {\n> +\t\t\t/*\n> +\t\t\t * TODO: for now, when you try to overwrite a \n> +\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n> +\t\t\t * flag, then the action will be done without providing \"--force\"\n> +\t\t\t * and no warning.\n> +\t\t\t *\n> +\t\t\t * This is mainly because the sparse \n> +\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n> +\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n> +\t\t\t */\n\nI wonder if this is worth the comment here, or if we'd rather see\nthe mention in the commit message. You have documented tests that\nfail in this case, so we already have something that marks this\nas \"TODO\" in a more discoverable place.\n\n> +\t\t\tint pos = cache_name_pos(src, length);\n> +\t\t\tif (pos >= 0) {\n> +\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n> +\n> +\t\t\t\tif (ce_skip_worktree(ce)) {\n> +\t\t\t\t\tif (!ignore_sparse)\n> +\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> +\t\t\t\t\telse\n> +\t\t\t\t\t\tmodes[i] = SPARSE;\n\n\n> +\t\t\t\t}\n> +\t\t\t\telse\n> +\t\t\t\t\tbad = _(\"bad source\");\n\nstyle nit:\n\n\t} else {\n\t\tbad = _(\"bad source\");\n\t}\n\n> +\t\t\t}\n> \t\t\t/* only error if existence is expected. */\n> -\t\t\tif (modes[i] != SPARSE)\n> +\t\t\telse if (modes[i] != SPARSE)\n> \t\t\t\tbad = _(\"bad source\");\n\nFor this one, the comment makes it difficult to connect the 'else\nif' to its corresponding 'if'. Perhaps:\n\n\t} else if (modes[i] != SPARSE) {\n\t\t/* only error if existence is expected. */\n\t\tbad = _(\"bad source\");\n\t}\n\n> \t\t} else if (!strncmp(src, dst, length) &&\n> \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n\nIn general, I found this if/else-if chain hard to grok, and\na lot of it is because we have \"simple\" cases at the end\nand the complicated parts have ever-increasing nesting. This\nis mostly due to the existing if/else-if chain in this method.\n\nHere is a diff that replaces that if/else-if chain with a\n'goto' trick to jump ahead, allowing some code to decrease in\ntabbing:\n\n---- >8 ----\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba831..1ca2c21da89 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -186,53 +186,68 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\tif (modes[i] != SPARSE) {\n \t\t\t\tbad = _(\"bad source\");\n-\t\t} else if (!strncmp(src, dst, length) &&\n-\t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n+\t\tif (!strncmp(src, dst, length) &&\n+\t\t (dst[length] == 0 || dst[length] == '/')) {\n \t\t\tbad = _(\"can not move directory into itself\");\n-\t\t} else if ((src_is_dir = S_ISDIR(st.st_mode))\n-\t\t\t\t&& lstat(dst, &st) == 0)\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n-\t\telse if (src_is_dir) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (src_is_dir) {\n+\t\t\tint j, dst_len, n;\n \t\t\tint first = cache_name_pos(src, length), last;\n \n-\t\t\tif (first >= 0)\n+\t\t\tif (first >= 0) {\n \t\t\t\tprepare_move_submodule(src, first,\n \t\t\t\t\t\t submodule_gitfile + i);\n-\t\t\telse if (index_range_of_same_dir(src, length,\n-\t\t\t\t\t\t\t &first, &last) < 1)\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t} else if (index_range_of_same_dir(src, length,\n+\t\t\t\t\t\t\t &first, &last) < 1) {\n \t\t\t\tbad = _(\"source directory is empty\");\n-\t\t\telse { /* last - first >= 1 */\n-\t\t\t\tint j, dst_len, n;\n-\n-\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n-\t\t\t\tn = argc + last - first;\n-\t\t\t\tREALLOC_ARRAY(source, n);\n-\t\t\t\tREALLOC_ARRAY(destination, n);\n-\t\t\t\tREALLOC_ARRAY(modes, n);\n-\t\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n-\n-\t\t\t\tdst = add_slash(dst);\n-\t\t\t\tdst_len = strlen(dst);\n-\n-\t\t\t\tfor (j = 0; j < last - first; j++) {\n-\t\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n-\t\t\t\t\tconst char *path = ce->name;\n-\t\t\t\t\tsource[argc + j] = path;\n-\t\t\t\t\tdestination[argc + j] =\n-\t\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n-\t\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n-\t\t\t\t}\n-\t\t\t\targc += last - first;\n+\t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t} else if (!(ce = cache_file_exists(src, length, 0))) {\n+\n+\t\t\t/* last - first >= 1 */\n+\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tn = argc + last - first;\n+\t\t\tREALLOC_ARRAY(source, n);\n+\t\t\tREALLOC_ARRAY(destination, n);\n+\t\t\tREALLOC_ARRAY(modes, n);\n+\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n+\n+\t\t\tdst = add_slash(dst);\n+\t\t\tdst_len = strlen(dst);\n+\n+\t\t\tfor (j = 0; j < last - first; j++) {\n+\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n+\t\t\t\tconst char *path = ce->name;\n+\t\t\t\tsource[argc + j] = path;\n+\t\t\t\tdestination[argc + j] =\n+\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n+\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n+\t\t\t}\n+\t\t\targc += last - first;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (!(ce = cache_file_exists(src, length, 0))) {\n \t\t\tbad = _(\"not under version control\");\n-\t\t} else if (ce_stage(ce)) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (ce_stage(ce)) {\n \t\t\tbad = _(\"conflicted\");\n-\t\t} else if (lstat(dst, &st) == 0 &&\n-\t\t\t (!ignore_case || strcasecmp(src, dst))) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (lstat(dst, &st) == 0 &&\n+\t\t (!ignore_case || strcasecmp(src, dst))) {\n \t\t\tbad = _(\"destination exists\");\n \t\t\tif (force) {\n \t\t\t\t/*\n@@ -246,34 +261,40 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t} else\n \t\t\t\t\tbad = _(\"Cannot overwrite\");\n \t\t\t}\n-\t\t} else if (string_list_has_string(&src_for_dst, dst))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (string_list_has_string(&src_for_dst, dst)) {\n \t\t\tbad = _(\"multiple sources for the same target\");\n-\t\telse if (is_dir_sep(dst[strlen(dst) - 1]))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (is_dir_sep(dst[strlen(dst) - 1])) {\n \t\t\tbad = _(\"destination directory does not exist\");\n-\t\telse {\n-\t\t\t/*\n-\t\t\t * We check if the paths are in the sparse-checkout\n-\t\t\t * definition as a very final check, since that\n-\t\t\t * allows us to point the user to the --sparse\n-\t\t\t * option as a way to have a successful run.\n-\t\t\t */\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(src, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\n-\t\t\tif (skip_sparse)\n-\t\t\t\tgoto remove_entry;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n \n-\t\t\tstring_list_insert(&src_for_dst, dst);\n+\t\t/*\n+\t\t * We check if the paths are in the sparse-checkout\n+\t\t * definition as a very final check, since that\n+\t\t * allows us to point the user to the --sparse\n+\t\t * option as a way to have a successful run.\n+\t\t */\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(src, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\tskip_sparse = 1;\n+\t\t}\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n+\t\t\tskip_sparse = 1;\n \t\t}\n \n+\t\tif (skip_sparse)\n+\t\t\tgoto remove_entry;\n+\n+\t\tstring_list_insert(&src_for_dst, dst);\n+\n+act_on_entry:\n \t\tif (!bad)\n \t\t\tcontinue;\n \t\tif (!ignore_errors)\n\n---- >8 ----\n\nBut mostly the reason for this refactor is that the following\ndiff should be equivalent to yours:\n\n---- >8 ----\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex d8b5c24fb5..add48e23b4 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -185,11 +185,28 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n-\t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE) {\n+\t\t\tint pos;\n+\t\t\tconst struct cache_entry *ce;\n+\n+\t\t\tpos = cache_name_pos(src, length);\n+\t\t\tif (pos < 0) {\n+\t\t\t\t/* only error if existence is expected. */\n+\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\t\tbad = _(\"bad source\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\n+\t\t\tce = active_cache[pos];\n+\t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n+\n+\t\t\tif (!ignore_sparse)\n+\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\telse\n+\t\t\t\tmodes[i] = SPARSE;\n+\t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n \t\t (dst[length] == 0 || dst[length] == '/')) {\n---- >8 ---\n\nTo me, this is a bit easier to parse, since we find the error\ncases and jump to the action before continuing on the \"happy\npath\". It does involve that first big refactor first, so I'd\nlike to hear opinions of other contributors before you jump to\ntaking this suggestion.\n\nThanks,\n-Stolee\n"},{"id":"456296","messageId":"bc51f198-629f-0b68-a8e4-8135f61c0d03@github.com","threadId":"57642","inReplyTo":"20220527100804.209890-5-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 4/5] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-05-27T15:27:25Z","receivedAt":"2022-05-27T15:27:31Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 5/27/2022 6:08 AM, Shaoxuan Yuan wrote:\n> +/*\n> + * Check if an out-of-cone directory should be in the index. Imagine this case\n> + * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n> + * and thus the directory is sparsified.\n> + *\n> + * Return 0 if such directory exist (i.e. with any of its contained files not\n> + * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n> + * Return 1 otherwise.\n> + */\n> +static int check_dir_in_index(const char *name, int namelen)\n> +{\n> +\tint ret = 1;\n> +\tconst char *with_slash = add_slash(name);\n> +\tint length = namelen + 1;\n> +\n> +\tint pos = cache_name_pos(with_slash, length);\n> +\tconst struct cache_entry *ce;\n> +\n> +\tif (pos < 0) {\n> +\t\tpos = -pos - 1;\n> +\t\tif (pos >= the_index.cache_nr)\n> +\t\t\treturn ret;\n> +\t\tce = active_cache[pos];\n> +\t\tif (strncmp(with_slash, ce->name, length))\n> +\t\t\treturn ret;\n> +\t\tif (ce_skip_worktree(ce))\n> +\t\t\treturn ret = 0;\n\nThis appears to check if the _first_ entry under the directory\nis sparse, but not if _all_ entries are sparse. These are not\nthe same thing, even in cone-mode sparse-checkout. The t1092\ntest directory has files like \"folder1/0/0/a\" but if\n\"folder1/1\" is in the sparse-checkout cone, then that first\nentry has the skip-worktree bit, but \"folder1/1/a\" and \"folder1/a\"\ndo not.\n\n> +\t}\n> +\treturn ret;\n\nAt the moment, it doesn't seem like we need 'ret' since the\nonly place you set it is in \"return ret = 0;\" (which could\njust be \"return 0;\" while the others are \"return 1;\"). But,\nperhaps you intended to create a loop over 'pos' while\nwith_slash is a prefix of the cache entry?\n\n> +\t\t\telse if (!check_dir_in_index(src, length) &&\n> +\t\t\t\t\t !path_in_sparse_checkout(src_w_slash, &the_index)) {\n\nstyle-nit: You'll want to align the different parts of your\nlogical statement to agree with the end of the \"else if (\",\n\n\telse if (A &&\n\t\t B) {\n\n\n> +\t\t\t\tmodes[i] = SKIP_WORKTREE_DIR;\n\nIf we are moving to a flags-based model, should we convert all\n\"modes[i] =\" to \"modes[i] |=\" as a first step (before adding the\nSKIP_WORTKREE_DIR flag)?\n\n> +\t\t\t\tgoto dir_check;\n\nHm. While I did recommend using 'goto' to jump to a common end\nplace in the loop body, I'm not sure about jumping into another\nelse-if statement. This might be a good time to extract the\ncode from \"else if (src_is_dir)\" below into a helper method that\ncan be used in both places.\n\n> +\t\t\t}\n> \t\t\t/* only error if existence is expected. */\n> \t\t\telse if (modes[i] != SPARSE)\n> \t\t\t\tbad = _(\"bad source\");\n> @@ -218,7 +264,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\t&& lstat(dst, &st) == 0)\n> \t\t\tbad = _(\"cannot move directory over file\");\n> \t\telse if (src_is_dir) {\n> -\t\t\tint first = cache_name_pos(src, length), last;\n> +\t\t\tint first, last;\n> +dir_check:\n> +\t\t\tfirst = cache_name_pos(src, length);\n> \n> \t\t\tif (first >= 0)\n> \t\t\t\tprepare_move_submodule(src, first,\n> @@ -229,7 +277,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\telse { /* last - first >= 1 */\n> \t\t\t\tint j, dst_len, n;\n> \n> -\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n> +\t\t\t\tif (!modes[i])\n> +\t\t\t\t\tmodes[i] |= WORKING_DIRECTORY;\n\nThis appears to only add the WORKING_DIRECTORY flag if modes[i] is\nalready zero. This maybe implies that we wouldn't understand\n\"WORKING_DIRECTORY | SKIP_WORKTREE_DIR\" as a value.\n\n> \t\t\t\tn = argc + last - first;\n> \t\t\t\tREALLOC_ARRAY(source, n);\n> \t\t\t\tREALLOC_ARRAY(destination, n);\n> @@ -331,7 +380,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> \t\tif (show_only)\n> \t\t\tcontinue;\n> -\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n> +\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> +\t\t \trename(src, dst) < 0) {\n\nstyle-nit: align your logical statements.\n\n> \t\t\tif (ignore_errors)\n> \t\t\t\tcontinue;\n> \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n> @@ -345,7 +395,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\t\t\t\t 1);\n> \t\t}\n> \n> -\t\tif (mode == WORKING_DIRECTORY)\n> +\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n> \t\t\tcontinue;\n\nOk, here you check if _either_ mode is enabled, which is good. Maybe\nyou don't need the \"if (!mode[i])\" part above.\n\nThanks,\n-Stolee\n"},{"id":"456298","messageId":"bd079b1b-d5a6-3191-3517-cb6329cc1e55@github.com","threadId":"57642","inReplyTo":"20220527100804.209890-2-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 1/5] t7002: add tests for moving out-of-cone file/directory","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-05-27T15:51:41Z","receivedAt":"2022-05-27T15:52:05Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Add corresponding tests to test following situations:\n> \n> * 'refuse to move out-of-cone directory without --sparse'\n> * 'can move out-of-cone directory with --sparse'\n> * 'refuse to move out-of-cone file without --sparse'\n> * 'can move out-of-cone file with --sparse'\n> * 'refuse to move sparse file to existing destination'\n> * 'move sparse file to existing destination with --force and --sparse'\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> t/t7002-mv-sparse-checkout.sh | 98 +++++++++++++++++++++++++++++++++++\n> 1 file changed, 98 insertions(+)\n> \n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 1d3d2aca21..963cb512e2 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -206,4 +206,102 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> \ttest_cmp expect stderr\n> '\n> \n\nApologies in advance for adding more comments (after I said the last version\nlooked good)! I'm always learning things about Git, so hopefully my\nsuggestions are at least better than in my last review. :) \n\n> +test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n\nNote that 'init' is now deprecated [1] (I think that happened between your\nv1 and now, FWIW). You can use 'git sparse-checkout set --cone sub', to do\nthe same thing as this + the subsequent line.\n\n[1] https://lore.kernel.org/git/9d96da855ea70e7e8a54bb68e710cc60a2f50376.1639454952.git.gitgitgadget@gmail.com/\n\n> +\tgit sparse-checkout set sub &&\n> +\n\nWhile the tests don't automatically \"reset\" between them (and therefore, you\ndon't need to disable & re-enable sparse-checkout), I like that you're\nexplicitly clearing & re-establishing the sparse-checkout state! It makes\nselectively running tests *much* easier and generally avoids hand-to-see\nside effects from prior tests.\n\nAs for test content, this particular setup block is (almost) identically\nrepeated in all of the tests. You could pull that into functions to reduce\nduplication, e.g.:\n\nsetup_sparse_checkout () {\n\tmkdir folder1 &&\n\ttouch folder1 &&\n\tgit add folder1 &&\n\tgit sparse-checkout set --cone sub\n}\n\ncleanup_sparse_checkout () {\n\tgit sparse-checkout disable &&\n\tgit reset --hard\n}\n\nYou may want to consider using \"test_when_finished \"\nto clean up each one *at the end* of the test, rather than the beginning of\nthe next one. E.g.:\n\ntest_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n\ttest_when_finished \"cleanup_sparse_checkout\" &&\n\tsetup_sparse_checkout &&\n\n\t# ...the rest of the test\n'\n\n> +\ttest_must_fail git mv folder1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone directory with --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\tgit sparse-checkout reapply &&\n> +\ttest_path_is_dir sub/folder1 &&\n> +\ttest_path_is_file sub/folder1/file1\n> +'\n> +\n> +test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone file with --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\tgit sparse-checkout reapply &&\n> +\t! test_path_is_dir sub/folder1 &&\n> +\ttest_path_is_file sub/file1\n> +'\n> +\n> +test_expect_failure 'refuse to move sparse file to existing destination' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\ttouch sub/file1 &&\n> +\tgit add folder1 sub/file1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n> +\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\ttouch sub/file1 &&\n> +\techo \"overwrite\" >folder1/file1 &&\n> +\tgit add folder1 sub/file1 &&\n> +\tgit sparse-checkout init --cone &&\n> +\tgit sparse-checkout set sub &&\n> +\n> +\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\techo \"overwrite\" >expect &&\n> +\ttest_cmp expect sub/file1\n> +'\n> +\n> test_done\n\nThese tests clearly establish the behavior you want to implement for 'git\nmv'. As in V1, nice work!\n"},{"id":"456313","messageId":"077a0579-903e-32ad-029c-48572d471c84@github.com","threadId":"57642","inReplyTo":"20220527100804.209890-6-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-05-27T19:36:13Z","receivedAt":"2022-05-27T19:36:20Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, \"git mv\" a sparse file/directory from out/in-cone to\n> in/out-cone does not update the sparsity following the sparse-checkout\n> patterns.\n> \n\nI generally agree with the intent here - that, if you move a non-sparse file\nout-of-cone, it should become sparse (and vice versa). However, that result\ncan be reached by simply flipping the 'SKIP_WORKTREE' bit(s) on the\nresultant index entry/entries (which you already have, since they're renamed\nwith 'rename_cache_entry_at()' below). \n\nNote that you'll also probably need to check out the file(s) (if moving into\nthe cone) or remove them from disk (if moving out of cone). If you don't,\nfiles moved into cone will appear \"deleted\" on-disk, and files moved\nout-of-cone that still appear on disk will have 'SKIP_WORKTREE'\nautomatically disabled (see [1]).\n\nFor reference, I'd advise against reapplying the sparsity patterns - as you\ndo below - because involves a much more expensive traversal of the entire\nrepository. It also has the possibly unwanted side effect of resetting the\n'SKIP_WORKTREE' bit to match the sparse patterns on *all* files, not just\nthe one(s) you moved. \n\n[1] https://lore.kernel.org/git/11d46a399d26c913787b704d2b7169cafc28d639.1642175983.git.gitgitgadget@gmail.com/\n\n> Use update_sparsity() after touching sparse contents, so the sparsity\n> will be updated after the move.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 19 +++++++++++++++++++\n> t/t7002-mv-sparse-checkout.sh | 16 ++++++++++++++++\n> 2 files changed, 35 insertions(+)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index e64f251a69..2c02120941 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -13,6 +13,7 @@\n> #include \"string-list.h\"\n> #include \"parse-options.h\"\n> #include \"submodule.h\"\n> +#include \"unpack-trees.h\"\n> \n> static const char * const builtin_mv_usage[] = {\n> \tN_(\"git mv [] ... \"),\n> @@ -158,6 +159,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> {\n> \tint i, flags, gitmodules_modified = 0;\n> \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n> +\tint sparse_moved = 0;\n> \tstruct option builtin_mv_options[] = {\n> \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n> \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n> @@ -376,6 +378,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tconst char *src = source[i], *dst = destination[i];\n> \t\tenum update_mode mode = modes[i];\n> \t\tint pos;\n> +\t\tif (!sparse_moved && mode & (SPARSE | SKIP_WORKTREE_DIR))\n> +\t\t\tsparse_moved = 1;\n> \t\tif (show_only || verbose)\n> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> \t\tif (show_only)\n> @@ -403,6 +407,21 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\trename_cache_entry_at(pos, dst);\n> \t}\n> \n> +\tif (sparse_moved) {\n> +\t\tstruct unpack_trees_options o;\n> +\t\tmemset(&o, 0, sizeof(o));\n> +\t\to.verbose_update = isatty(2);\n> +\t\to.update = 1;\n> +\t\to.head_idx = -1;\n> +\t\to.src_index = &the_index;\n> +\t\to.dst_index = &the_index;\n> +\t\to.skip_sparse_checkout = 0;\n> +\t\to.pl = the_index.sparse_checkout_patterns;\n> +\t\tsetup_unpack_trees_porcelain(&o, \"mv\");\n> +\t\tupdate_sparsity(&o);\n> +\t\tclear_unpack_trees_porcelain(&o);\n> +\t}\n> +\n> \tif (gitmodules_modified)\n> \t\tstage_updated_gitmodules(&the_index);\n> \n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index cf2f5dc46f..1fd3e3c0fc 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -287,6 +287,22 @@ test_expect_success 'refuse to move sparse file to existing destination' '\n> \ttest_cmp expect stderr\n> '\n> \n> +# Need fix.\n> +#\n> +# The *expected* behavior:\n> +#\n> +# Using --sparse to accept a sparse file, --force to overwrite the destination.\n> +# The folder1/file1 should replace the sub/file1 without error.\n> +#\n> +# The *actual* behavior:\n> +#\n> +# It emits a warning:\n> +#\n> +# warning: Path ' sub/file1\n> +# ' already present; will not overwrite with sparse update.\n> +# After fixing the above paths, you may want to run `git sparse-checkout\n> +# reapply`.\n> +\n> test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> \tgit sparse-checkout disable &&\n> \tgit reset --hard &&\n\nThis error is (I think) part of 'update_sparsity()'. If you change the\napproach to only modifying the 'SKIP_WORKTREE' bit, hopefully you'll get the\nbehavior you're looking for.\n"},{"id":"456319","messageId":"xmqq8rqm3fxa.fsf@gitster.g","threadId":"57642","inReplyTo":"077a0579-903e-32ad-029c-48572d471c84@github.com","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-05-27T19:59:29Z","receivedAt":"2022-05-27T19:59:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Victoria Dye writes:\n\n> Note that you'll also probably need to check out the file(s) (if moving into\n> the cone) or remove them from disk (if moving out of cone). If you don't,\n> files moved into cone will appear \"deleted\" on-disk, and files moved\n> out-of-cone that still appear on disk will have 'SKIP_WORKTREE'\n> automatically disabled (see [1]).\n\nDoes it also imply that we should forbid \"git mv\" of a dirty path\nout of the cone? Or is that too draconian and it suffices to tweak\nthe rule slightly to \"remove from the worktree when moving a clean\npath out of cone\", perhaps? When a dirty path is moved out of cone,\nwe would trigger the \"SKIP_WORKTREE automatically disabled\" behaviour\nand that would be a good thing, I imagine?\n\n"},{"id":"456330","messageId":"d0ac1bea-6d98-140c-0e46-d7569e80b29d@github.com","threadId":"57642","inReplyTo":"xmqq8rqm3fxa.fsf@gitster.g","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-05-27T21:24:06Z","receivedAt":"2022-05-27T21:24:12Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Junio C Hamano wrote:\n> Victoria Dye writes:\n> \n>> Note that you'll also probably need to check out the file(s) (if moving into\n>> the cone) or remove them from disk (if moving out of cone). If you don't,\n>> files moved into cone will appear \"deleted\" on-disk, and files moved\n>> out-of-cone that still appear on disk will have 'SKIP_WORKTREE'\n>> automatically disabled (see [1]).\n> \n> Does it also imply that we should forbid \"git mv\" of a dirty path\n> out of the cone? Or is that too draconian and it suffices to tweak\n> the rule slightly to \"remove from the worktree when moving a clean\n> path out of cone\", perhaps? When a dirty path is moved out of cone,\n> we would trigger the \"SKIP_WORKTREE automatically disabled\" behaviour\n> and that would be a good thing, I imagine?\n> \n\nI like the idea of the modified rule as an option since it *does* complete\nthe move in accordance with '--force', but doesn't result in silently lost\ninformation. \n\nAn alternative might be 'mv' refusing to move a modified file out-of-cone\n(despite '--force'), printing something like\n'WARNING_SPARSE_NOT_UPTODATE_FILE' (\"Path 'x' not uptodate; will not remove\nfrom working tree\").\n\nI'm not sure which would provide a more vs. less frustrating experience, but\nboth are at least safe in terms of preserving unstaged changes.\n\n"},{"id":"456333","messageId":"418e50c8-afec-3b51-53c0-bb84e37af6f2@github.com","threadId":"57642","inReplyTo":"20220527100804.209890-4-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v2 3/5] mv: check if exists in index to handle overwriting","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-05-27T22:04:27Z","receivedAt":"2022-05-27T22:04:35Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, moving a sparse file into cone can result in unwarned\n> overwrite of existing entry. The expected behavior is that if the\n> exists in the entry, user should be prompted to supply\n> a [-f|--force] to carry out the operation, or the operation should\n> fail.\n> \n> Add a check mechanism to do that.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 23 +++++++++++------------\n> t/t7002-mv-sparse-checkout.sh | 2 +-\n> 2 files changed, 12 insertions(+), 13 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 32ad4d5682..62284e3f86 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -185,16 +185,6 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \n> \t\tlength = strlen(src);\n> \t\tif (lstat(src, &st) < 0) {\n> -\t\t\t/*\n> -\t\t\t * TODO: for now, when you try to overwrite a \n> -\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n> -\t\t\t * flag, then the action will be done without providing \"--force\"\n> -\t\t\t * and no warning.\n> -\t\t\t *\n> -\t\t\t * This is mainly because the sparse \n> -\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n> -\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n> -\t\t\t */\n> \n\nGiven that this removes the \"TODO\" comment you just added in the previous\npatch, I agree with Stolee's suggestion [1] that you mention this context in\nthe patch 2 commit message rather than a code comment. The commit message of\n*this* patch already explains the behavior you're correcting, so I don't\nthink any other changes would be needed here.\n\n[1] https://lore.kernel.org/git/0884b97b-0745-5cad-3034-a679be5d6c3a@github.com/\n\n> \t\t\tint pos = cache_name_pos(src, length);\n> \t\t\tif (pos >= 0) {\n> @@ -203,8 +193,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\tif (ce_skip_worktree(ce)) {\n> \t\t\t\t\tif (!ignore_sparse)\n> \t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> -\t\t\t\t\telse\n> -\t\t\t\t\t\tmodes[i] = SPARSE;\n> +\t\t\t\t\telse {\n> +\t\t\t\t\t\t/* Check if dst exists in index */\n> +\t\t\t\t\t\tif (cache_name_pos(dst, strlen(dst)) >= 0) {\n> +\t\t\t\t\t\t\tif (force)\n> +\t\t\t\t\t\t\t\tmodes[i] = SPARSE;\n> +\t\t\t\t\t\t\telse\n> +\t\t\t\t\t\t\t\tbad = _(\"destination exists\");\n> +\t\t\t\t\t\t}\n> +\t\t\t\t\t\telse\n> +\t\t\t\t\t\t\tmodes[i] = SPARSE;\n> +\t\t\t\t\t}\n> \t\t\t\t}\n> \t\t\t\telse\n> \t\t\t\t\tbad = _(\"bad source\");\n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 581ef4c0f6..2c9008573a 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -272,7 +272,7 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n> \ttest_path_is_file sub/file1\n> '\n> \n> -test_expect_failure 'refuse to move sparse file to existing destination' '\n> +test_expect_success 'refuse to move sparse file to existing destination' '\n> \tgit sparse-checkout disable &&\n> \tgit reset --hard &&\n> \tmkdir folder1 &&\n\nThe rest of this looks good to me!\n"},{"id":"456334","messageId":"b4c94cd9-4afe-cb0b-f60b-48a359f06ae8@github.com","threadId":"57642","inReplyTo":"0884b97b-0745-5cad-3034-a679be5d6c3a@github.com","subject":"Re: [WIP v2 2/5] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-05-27T22:38:45Z","receivedAt":"2022-05-27T22:38:51Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Derrick Stolee wrote:\n> \n> \n> On 5/27/2022 6:08 AM, Shaoxuan Yuan wrote:\n>> Originally, moving a file which is not on-disk but exists in\n>> index as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\n>> out with \"bad source\".\n>>\n>> Change the checking logic, so that such \n>> file makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n>> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n>> this operation can be carried out successfully.\n>>\n>> Signed-off-by: Shaoxuan Yuan \n>> ---\n>> builtin/mv.c | 26 +++++++++++++++++++++++++-\n>> t/t7002-mv-sparse-checkout.sh | 4 ++--\n>> 2 files changed, 27 insertions(+), 3 deletions(-)\n>>\n>> diff --git a/builtin/mv.c b/builtin/mv.c\n>> index 83a465ba83..32ad4d5682 100644\n>> --- a/builtin/mv.c\n>> +++ b/builtin/mv.c\n>> @@ -185,8 +185,32 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \n>> \t\tlength = strlen(src);\n>> \t\tif (lstat(src, &st) < 0) {\n>> +\t\t\t/*\n>> +\t\t\t * TODO: for now, when you try to overwrite a \n>> +\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n>> +\t\t\t * flag, then the action will be done without providing \"--force\"\n>> +\t\t\t * and no warning.\n>> +\t\t\t *\n>> +\t\t\t * This is mainly because the sparse \n>> +\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n>> +\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n>> +\t\t\t */\n> \n> I wonder if this is worth the comment here, or if we'd rather see\n> the mention in the commit message. You have documented tests that\n> fail in this case, so we already have something that marks this\n> as \"TODO\" in a more discoverable place.\n> \n>> +\t\t\tint pos = cache_name_pos(src, length);\n>> +\t\t\tif (pos >= 0) {\n>> +\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n>> +\n>> +\t\t\t\tif (ce_skip_worktree(ce)) {\n>> +\t\t\t\t\tif (!ignore_sparse)\n>> +\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n>> +\t\t\t\t\telse\n>> +\t\t\t\t\t\tmodes[i] = SPARSE;\n> \n> \n>> +\t\t\t\t}\n>> +\t\t\t\telse\n>> +\t\t\t\t\tbad = _(\"bad source\");\n> \n> style nit:\n> \n> \t} else {\n> \t\tbad = _(\"bad source\");\n> \t}\n> \n\nIn case this advice seems contradictory with past style suggestions, from 'Documentation/CodingGuidelines':\n\n\t- When there are multiple arms to a conditional and some of them\n\t require braces, enclose even a single line block in braces for\n\t consistency. E.g.:\n\n\t\tif (foo) {\n\t\t\tdoit();\n\t\t} else {\n\t\t\tone();\n\t\t\ttwo();\n\t\t\tthree();\n\t\t}\n\n>> +\t\t\t}\n>> \t\t\t/* only error if existence is expected. */\n>> -\t\t\tif (modes[i] != SPARSE)\n>> +\t\t\telse if (modes[i] != SPARSE)\n>> \t\t\t\tbad = _(\"bad source\");\n> \n> For this one, the comment makes it difficult to connect the 'else\n> if' to its corresponding 'if'. Perhaps:\n> \n> \t} else if (modes[i] != SPARSE) {\n> \t\t/* only error if existence is expected. */\n> \t\tbad = _(\"bad source\");\n> \t}\n> \n>> \t\t} else if (!strncmp(src, dst, length) &&\n>> \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n> \n> In general, I found this if/else-if chain hard to grok, and\n> a lot of it is because we have \"simple\" cases at the end\n> and the complicated parts have ever-increasing nesting. This\n> is mostly due to the existing if/else-if chain in this method.\n> \n\nAgreed that the if/else-if chains make 'cmd_mv' complicated. The most\nfrustrating thing about its current state (unrelated to this patch) is how\nunclear it is whether any given conditions are mutually-exclusive vs.\ndependent vs. one taking precedence over another. On that note... \n\n> Here is a diff that replaces that if/else-if chain with a\n> 'goto' trick to jump ahead, allowing some code to decrease in\n> tabbing:\n> \n\n...while I'm usually hesitant to add more 'goto' labels to the code if it\ncan be avoided, I think that model fits this use case well.\n\n> ---- >8 ----\n\n[cutting the proposed refactor for space]\n\n> ---- >8 ---\n> \n> To me, this is a bit easier to parse, since we find the error\n> cases and jump to the action before continuing on the \"happy\n> path\". It does involve that first big refactor first, so I'd\n> like to hear opinions of other contributors before you jump to\n> taking this suggestion.\n> \n\nI like how the refactored version simplifies 'cmd_mv', and how it\ncorrespondingly simplifies the new checks in this (Shaoxuan's) patch. It\ndoes still leave us with one big, monolithic 'cmd_mv', so in an ideal world\nI'd probably lean towards pulling the innards of the main for-loop(s) into a\nfew dedicated functions (like 'validate_move_candidate', 'move_entry').\nHowever, I'm happy with any improvement, and this refactor would certainly\ngive us that!\n\n> Thanks,\n> -Stolee\n\n"},{"id":"456383","messageId":"CAJyCBORP74=nC1cB5r41MqS+WD-JfNX+8AOv7jqevN9FZGWdSw@mail.gmail.com","threadId":"57642","inReplyTo":"0884b97b-0745-5cad-3034-a679be5d6c3a@github.com","subject":"Re: [WIP v2 2/5] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-31T08:06:08Z","receivedAt":"2022-05-31T08:06:25Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Fri, May 27, 2022 at 11:13 PM Derrick Stolee\n wrote:\n> > diff --git a/builtin/mv.c b/builtin/mv.c\n> > index 83a465ba83..32ad4d5682 100644\n> > --- a/builtin/mv.c\n> > +++ b/builtin/mv.c\n> > @@ -185,8 +185,32 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> >\n> > length = strlen(src);\n> > if (lstat(src, &st) < 0) {\n> > + /*\n> > + * TODO: for now, when you try to overwrite a \n> > + * with your as a sparse file, if you supply a \"--sparse\"\n> > + * flag, then the action will be done without providing \"--force\"\n> > + * and no warning.\n> > + *\n> > + * This is mainly because the sparse \n> > + * is not on-disk, and this if-else chain will be cut off early in\n> > + * this check, thus the \"--force\" check is ignored. Need fix.\n> > + */\n>\n> I wonder if this is worth the comment here, or if we'd rather see\n> the mention in the commit message. You have documented tests that\n> fail in this case, so we already have something that marks this\n> as \"TODO\" in a more discoverable place.\n\nThis comment was added during my local development, it should be\nremoved.\n\n> > + int pos = cache_name_pos(src, length);\n> > + if (pos >= 0) {\n> > + const struct cache_entry *ce = active_cache[pos];\n> > +\n> > + if (ce_skip_worktree(ce)) {\n> > + if (!ignore_sparse)\n> > + string_list_append(&only_match_skip_worktree, src);\n> > + else\n> > + modes[i] = SPARSE;\n>\n>\n> > + }\n> > + else\n> > + bad = _(\"bad source\");\n>\n> style nit:\n>\n> } else {\n> bad = _(\"bad source\");\n> }\n>\n> > + }\n> > /* only error if existence is expected. */\n> > - if (modes[i] != SPARSE)\n> > + else if (modes[i] != SPARSE)\n> > bad = _(\"bad source\");\n>\n> For this one, the comment makes it difficult to connect the 'else\n> if' to its corresponding 'if'. Perhaps:\n>\n> } else if (modes[i] != SPARSE) {\n> /* only error if existence is expected. */\n> bad = _(\"bad source\");\n> }\n>\n> > } else if (!strncmp(src, dst, length) &&\n> > (dst[length] == 0 || dst[length] == '/')) {\n>\n> In general, I found this if/else-if chain hard to grok, and\n> a lot of it is because we have \"simple\" cases at the end\n> and the complicated parts have ever-increasing nesting. This\n> is mostly due to the existing if/else-if chain in this method.\n>\n> Here is a diff that replaces that if/else-if chain with a\n> 'goto' trick to jump ahead, allowing some code to decrease in\n> tabbing:\n>\n> ---- >8 ----\n\n[cutting the proposed refactor for space]\n\n> ---- >8 ----\n>\n> But mostly the reason for this refactor is that the following\n> diff should be equivalent to yours:\n>\n> ---- >8 ----\n>\n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index d8b5c24fb5..add48e23b4 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -185,11 +185,28 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>\n> length = strlen(src);\n> if (lstat(src, &st) < 0) {\n> - /* only error if existence is expected. */\n> - if (modes[i] != SPARSE) {\n> + int pos;\n> + const struct cache_entry *ce;\n> +\n> + pos = cache_name_pos(src, length);\n> + if (pos < 0) {\n> + /* only error if existence is expected. */\n> + if (modes[i] != SPARSE)\n> + bad = _(\"bad source\");\n> + goto act_on_entry;\n> + }\n> +\n> + ce = active_cache[pos];\n> + if (!ce_skip_worktree(ce)) {\n> bad = _(\"bad source\");\n> goto act_on_entry;\n> }\n> +\n> + if (!ignore_sparse)\n> + string_list_append(&only_match_skip_worktree, src);\n> + else\n> + modes[i] = SPARSE;\n> + goto act_on_entry;\n> }\n> if (!strncmp(src, dst, length) &&\n> (dst[length] == 0 || dst[length] == '/')) {\n> ---- >8 ---\n>\n> To me, this is a bit easier to parse, since we find the error\n> cases and jump to the action before continuing on the \"happy\n> path\". It does involve that first big refactor first, so I'd\n> like to hear opinions of other contributors before you jump to\n> taking this suggestion.\n\nTrue. I also find it easier to read. Though Victoria mentioned the\ngoto hazard, the gotos here decouples the huge chain and that\nbrings clarity and makes it easier to extend.\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"456385","messageId":"CAJyCBORo-x4jbKhtn+vUE=1TxpM83_3JWj5cvJEJJHHsv2Q0bg@mail.gmail.com","threadId":"57642","inReplyTo":"bc51f198-629f-0b68-a8e4-8135f61c0d03@github.com","subject":"Re: [WIP v2 4/5] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-05-31T09:56:01Z","receivedAt":"2022-05-31T09:56:18Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Fri, May 27, 2022 at 11:27 PM Derrick Stolee\n wrote:\n>\n> On 5/27/2022 6:08 AM, Shaoxuan Yuan wrote:\n> > +/*\n> > + * Check if an out-of-cone directory should be in the index. Imagine this case\n> > + * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n> > + * and thus the directory is sparsified.\n> > + *\n> > + * Return 0 if such directory exist (i.e. with any of its contained files not\n> > + * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n> > + * Return 1 otherwise.\n> > + */\n> > +static int check_dir_in_index(const char *name, int namelen)\n> > +{\n> > + int ret = 1;\n> > + const char *with_slash = add_slash(name);\n> > + int length = namelen + 1;\n> > +\n> > + int pos = cache_name_pos(with_slash, length);\n> > + const struct cache_entry *ce;\n> > +\n> > + if (pos < 0) {\n> > + pos = -pos - 1;\n> > + if (pos >= the_index.cache_nr)\n> > + return ret;\n> > + ce = active_cache[pos];\n> > + if (strncmp(with_slash, ce->name, length))\n> > + return ret;\n> > + if (ce_skip_worktree(ce))\n> > + return ret = 0;\n>\n> This appears to check if the _first_ entry under the directory\n> is sparse, but not if _all_ entries are sparse. These are not\n> the same thing, even in cone-mode sparse-checkout. The t1092\n> test directory has files like \"folder1/0/0/a\" but if\n> \"folder1/1\" is in the sparse-checkout cone, then that first\n> entry has the skip-worktree bit, but \"folder1/1/a\" and \"folder1/a\"\n> do not.\n\nYes, it is checking the first entry and this would not work without the\nlstat in the front. But I think the \"lstat < 0\" makes sure that this directory\ncannot be partially sparsified.\n\nIt is either missing both in the worktree and index, or missing in the worktree\nbut present in index (with all its content sparsified). And because of that,\nI think only the first entry needs to be checked.\n\n> > + }\n> > + return ret;\n>\n> At the moment, it doesn't seem like we need 'ret' since the\n> only place you set it is in \"return ret = 0;\" (which could\n> just be \"return 0;\" while the others are \"return 1;\"). But,\n> perhaps you intended to create a loop over 'pos' while\n> with_slash is a prefix of the cache entry?\n\nI agree that this variable is redundant. But I fail to understand\nthe logical relation between before \"But,\" and after \"But,\". Please\nelaborate on that?\n\n> > + else if (!check_dir_in_index(src, length) &&\n> > + !path_in_sparse_checkout(src_w_slash, &the_index)) {\n>\n> style-nit: You'll want to align the different parts of your\n> logical statement to agree with the end of the \"else if (\",\n>\n> else if (A &&\n> B) {\n>\n\nThis one is interesting because it appears just alright in my VSCode editor.\nLater I found that it is because git-diff is using a tab size of 8 or something,\nbut my VSCode uses tab size of 4. After I configured the git-diff tab rendering\nsize, it looks alright. Same for another style nit down below.\n\n> > + modes[i] = SKIP_WORKTREE_DIR;\n>\n> If we are moving to a flags-based model, should we convert all\n> \"modes[i] =\" to \"modes[i] |=\" as a first step (before adding the\n> SKIP_WORTKREE_DIR flag)?\n>\n> > + goto dir_check;\n>\n> Hm. While I did recommend using 'goto' to jump to a common end\n> place in the loop body, I'm not sure about jumping into another\n> else-if statement. This might be a good time to extract the\n> code from \"else if (src_is_dir)\" below into a helper method that\n> can be used in both places.\n\nRight, this is suspicious. I wasn't familiar at all with C/C++, and being able\nto do this inter-if-else jump also startled me.\nI agree that it should be something more legitimate, like extracting a\nmethod for it.\n\n> > + }\n> > /* only error if existence is expected. */\n> > else if (modes[i] != SPARSE)\n> > bad = _(\"bad source\");\n> > @@ -218,7 +264,9 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > && lstat(dst, &st) == 0)\n> > bad = _(\"cannot move directory over file\");\n> > else if (src_is_dir) {\n> > - int first = cache_name_pos(src, length), last;\n> > + int first, last;\n> > +dir_check:\n> > + first = cache_name_pos(src, length);\n> >\n> > if (first >= 0)\n> > prepare_move_submodule(src, first,\n> > @@ -229,7 +277,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > else { /* last - first >= 1 */\n> > int j, dst_len, n;\n> >\n> > - modes[i] = WORKING_DIRECTORY;\n> > + if (!modes[i])\n> > + modes[i] |= WORKING_DIRECTORY;\n>\n> This appears to only add the WORKING_DIRECTORY flag if modes[i] is\n> already zero. This maybe implies that we wouldn't understand\n> \"WORKING_DIRECTORY | SKIP_WORKTREE_DIR\" as a value.\n\nAt this point, I cannot think of the reason for writing it this way. And yes,\nthis does not make sense...\n\n> > n = argc + last - first;\n> > REALLOC_ARRAY(source, n);\n> > REALLOC_ARRAY(destination, n);\n> > @@ -331,7 +380,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > printf(_(\"Renaming %s to %s\\n\"), src, dst);\n> > if (show_only)\n> > continue;\n> > - if (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n> > + if (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> > + rename(src, dst) < 0) {\n>\n> style-nit: align your logical statements.\n>\n> > if (ignore_errors)\n> > continue;\n> > die_errno(_(\"renaming '%s' failed\"), src);\n> > @@ -345,7 +395,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> > 1);\n> > }\n> >\n> > - if (mode == WORKING_DIRECTORY)\n> > + if (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n> > continue;\n>\n> Ok, here you check if _either_ mode is enabled, which is good. Maybe\n> you don't need the \"if (!mode[i])\" part above.\n>\n> Thanks,\n> -Stolee\n\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"456392","messageId":"4e02f672-b246-d32d-b208-8a51b13858c8@github.com","threadId":"57642","inReplyTo":"CAJyCBORo-x4jbKhtn+vUE=1TxpM83_3JWj5cvJEJJHHsv2Q0bg@mail.gmail.com","subject":"Re: [WIP v2 4/5] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-05-31T15:49:10Z","receivedAt":"2022-05-31T15:49:16Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 5/31/2022 5:56 AM, Shaoxuan Yuan wrote:\n> On Fri, May 27, 2022 at 11:27 PM Derrick Stolee\n> wrote:\n>>\n>> On 5/27/2022 6:08 AM, Shaoxuan Yuan wrote:\n...\n>> This appears to check if the _first_ entry under the directory\n>> is sparse, but not if _all_ entries are sparse. These are not\n>> the same thing, even in cone-mode sparse-checkout. The t1092\n>> test directory has files like \"folder1/0/0/a\" but if\n>> \"folder1/1\" is in the sparse-checkout cone, then that first\n>> entry has the skip-worktree bit, but \"folder1/1/a\" and \"folder1/a\"\n>> do not.\n> \n> Yes, it is checking the first entry and this would not work without the\n> lstat in the front. But I think the \"lstat < 0\" makes sure that this directory\n> cannot be partially sparsified.\n> \n> It is either missing both in the worktree and index, or missing in the worktree\n> but present in index (with all its content sparsified). And because of that,\n> I think only the first entry needs to be checked.\n\nAh! Good thinking. I hadn't considered that extra detail, so\nwe get to save some cycles here.\n\n>>> + }\n>>> + return ret;\n>>\n>> At the moment, it doesn't seem like we need 'ret' since the\n>> only place you set it is in \"return ret = 0;\" (which could\n>> just be \"return 0;\" while the others are \"return 1;\"). But,\n>> perhaps you intended to create a loop over 'pos' while\n>> with_slash is a prefix of the cache entry?\n> \n> I agree that this variable is redundant. But I fail to understand\n> the logical relation between before \"But,\" and after \"But,\". Please\n> elaborate on that?\n\nI was just thinking that if you intended to write a loop as\nI had suggested, then 'ret' could be modified or used in more\nplaces. Feel free to ignore since we resolved that.\n\n>>> + else if (!check_dir_in_index(src, length) &&\n>>> + !path_in_sparse_checkout(src_w_slash, &the_index)) {\n>>\n>> style-nit: You'll want to align the different parts of your\n>> logical statement to agree with the end of the \"else if (\",\n>>\n>> else if (A &&\n>> B) {\n>>\n> \n> This one is interesting because it appears just alright in my VSCode editor.\n> Later I found that it is because git-diff is using a tab size of 8 or something,\n> but my VSCode uses tab size of 4. After I configured the git-diff tab rendering\n> size, it looks alright. Same for another style nit down below.\n\nThat'll do it. You can double-check the alignment in your GGG\nPR, which should use the correct tab width.\n\nThanks,\n-Stolee\n"},{"id":"457377","messageId":"CAJyCBOQGAL9aGW+Gxv8sZH9T_tB6_pdeLNwmNgqPhz7cMdZrbA@mail.gmail.com","threadId":"57642","inReplyTo":"d0ac1bea-6d98-140c-0e46-d7569e80b29d@github.com","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-16T13:51:11Z","receivedAt":"2022-06-16T13:51:29Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Sat, May 28, 2022 at 5:24 AM Victoria Dye wrote:\n>\n> Junio C Hamano wrote:\n> > Victoria Dye writes:\n> >\n> >> Note that you'll also probably need to check out the file(s) (if moving into\n> >> the cone) or remove them from disk (if moving out of cone). If you don't,\n> >> files moved into cone will appear \"deleted\" on-disk, and files moved\n> >> out-of-cone that still appear on disk will have 'SKIP_WORKTREE'\n> >> automatically disabled (see [1]).\n> >\n> > Does it also imply that we should forbid \"git mv\" of a dirty path\n> > out of the cone? Or is that too draconian and it suffices to tweak\n> > the rule slightly to \"remove from the worktree when moving a clean\n> > path out of cone\", perhaps? When a dirty path is moved out of cone,\n> > we would trigger the \"SKIP_WORKTREE automatically disabled\" behaviour\n> > and that would be a good thing, I imagine?\n> >\n>\n> I like the idea of the modified rule as an option since it *does* complete\n> the move in accordance with '--force', but doesn't result in silently lost\n> information.\n>\n> An alternative might be 'mv' refusing to move a modified file out-of-cone\n> (despite '--force'), printing something like\n> 'WARNING_SPARSE_NOT_UPTODATE_FILE' (\"Path 'x' not uptodate; will not remove\n> from working tree\").\n>\n> I'm not sure which would provide a more vs. less frustrating experience, but\n> both are at least safe in terms of preserving unstaged changes.\n\nFor me, the alternative provides a less frustrating experience.\n\nSince it is more explicit (giving a message and directly saying NO).\nAlso, the `sparse-checkout` users should expect the moved file to be\nmissing in the working tree, as opposed to being present.\n\nAnd the tweaked rule suggested by Junio [1] might need an extra\n `git sparse-checkout reapply` to re-sparsify the file that moved out-of-cone\nafter staging its change?\n\n[1] https://lore.kernel.org/git/xmqq8rqm3fxa.fsf@gitster.g/\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"457386","messageId":"6375c172-82cb-dffc-875f-e5e742d5e49e@github.com","threadId":"57642","inReplyTo":"CAJyCBOQGAL9aGW+Gxv8sZH9T_tB6_pdeLNwmNgqPhz7cMdZrbA@mail.gmail.com","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-06-16T16:42:10Z","receivedAt":"2022-06-16T16:42:16Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> On Sat, May 28, 2022 at 5:24 AM Victoria Dye wrote:\n>>\n>> Junio C Hamano wrote:\n>>> Victoria Dye writes:\n>>>\n>>>> Note that you'll also probably need to check out the file(s) (if moving into\n>>>> the cone) or remove them from disk (if moving out of cone). If you don't,\n>>>> files moved into cone will appear \"deleted\" on-disk, and files moved\n>>>> out-of-cone that still appear on disk will have 'SKIP_WORKTREE'\n>>>> automatically disabled (see [1]).\n>>>\n>>> Does it also imply that we should forbid \"git mv\" of a dirty path\n>>> out of the cone? Or is that too draconian and it suffices to tweak\n>>> the rule slightly to \"remove from the worktree when moving a clean\n>>> path out of cone\", perhaps? When a dirty path is moved out of cone,\n>>> we would trigger the \"SKIP_WORKTREE automatically disabled\" behaviour\n>>> and that would be a good thing, I imagine?\n>>>\n>>\n>> I like the idea of the modified rule as an option since it *does* complete\n>> the move in accordance with '--force', but doesn't result in silently lost\n>> information.\n>>\n>> An alternative might be 'mv' refusing to move a modified file out-of-cone\n>> (despite '--force'), printing something like\n>> 'WARNING_SPARSE_NOT_UPTODATE_FILE' (\"Path 'x' not uptodate; will not remove\n>> from working tree\").\n>>\n>> I'm not sure which would provide a more vs. less frustrating experience, but\n>> both are at least safe in terms of preserving unstaged changes.\n> \n> For me, the alternative provides a less frustrating experience.\n> \n> Since it is more explicit (giving a message and directly saying NO).\n>> Also, the `sparse-checkout` users should expect the moved file to be\n> missing in the working tree, as opposed to being present.\n> \n\nGood point, since the sparseness of the destination file would be different\ndepending on whether it had local modifications or not (with no indication\nfrom 'mv' of the different treatment).\n\nIf you're interested, maybe there's a middle-ground option? Suppose you want\nto move a file 'file1' to an out-of-cone location:\n\n1. If 'file1' is clean, regardless of use of '--force', move the file & make\n it sparse.\n2. If 'file1' is *not* clean and '--force' is *not* used, refuse to move the\n file (with a \"Path 'file1' not uptodate; will not move. Use '--force' to\n override.\" type of error).\n3. If 'file1' is *not* clean and '--force' is used, move the file but do not\n make it sparse.\n\nThat way, '--force' really does force the move to happen, but users are\ngenerally warned against it. I'm still not sure what the \"right\" approach\nis, but to your point I think it should err on the side of not surprising\nthe user.\n\n> And the tweaked rule suggested by Junio [1] might need an extra\n> `git sparse-checkout reapply` to re-sparsify the file that moved out-of-cone\n> after staging its change?\n> \n\nJust so I understand correctly, do you mean 'git sparse-checkout reapply'\n*as part of* the 'mv' operation? Or are you thinking that a user might want\nto manually run 'git sparse-checkout reapply' after running 'mv'? \n\nIf it's the former (internally calling 'git sparse-checkout reapply' in\n'mv'), then no, you wouldn't want to do that. In Junio's suggestion, he said\n(emphasis mine):\n\n> When a dirty path is moved out of cone, we would trigger the\n> \"SKIP_WORKTREE automatically disabled\" behaviour\" *and that would be a\n> good thing, I imagine?*\n\nWe don't want the file moved out-of-cone to be sparse again because it has\nlocal (on-disk) modifications that would disappear (since a file needs to be\nremoved from disk to be \"sparse\" in the eyes of 'sparse-checkout'). It's\n*completely valid* behavior to have an out-of-cone file become non-sparse if\na user does something to cause that; it doesn't cause any bugs/corruption\nwith the repo. And, even if you did want to make the file sparse, it should\nbe done by manually setting 'SKIP_WORKTREE' and individually removing the\nfile from disk (for all the reasons I mentioned in my upthread comment [1]).\n\nOn the other hand, if you're talking about a user manually running 'git\nsparse-checkout reapply' after the fact, that wouldn't work either - they'd\nget an error:\n\nwarning: The following paths are not up to date and were left despite sparse patterns:\n \n\n[1] https://lore.kernel.org/git/077a0579-903e-32ad-029c-48572d471c84@github.com/\n\n> [1] https://lore.kernel.org/git/xmqq8rqm3fxa.fsf@gitster.g/\n> \n"},{"id":"457446","messageId":"CAJyCBOS3AKPT43ERsfLetSHRP_tciNvok-MSg5d=buStKPMxYA@mail.gmail.com","threadId":"57642","inReplyTo":"6375c172-82cb-dffc-875f-e5e742d5e49e@github.com","subject":"Re: [WIP v2 5/5] mv: use update_sparsity() after touching sparse contents","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-17T02:15:20Z","receivedAt":"2022-06-17T02:15:36Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On Fri, Jun 17, 2022 at 12:42 AM Victoria Dye wrote:\n*Truncated messages*\n> > For me, the alternative provides a less frustrating experience.\n> >\n> > Since it is more explicit (giving a message and directly saying NO).\n> >> Also, the `sparse-checkout` users should expect the moved file to be\n> > missing in the working tree, as opposed to being present.\n> >\n>\n> Good point, since the sparseness of the destination file would be different\n> depending on whether it had local modifications or not (with no indication\n> from 'mv' of the different treatment).\n>\n> If you're interested, maybe there's a middle-ground option? Suppose you want\n> to move a file 'file1' to an out-of-cone location:\n>\n> 1. If 'file1' is clean, regardless of use of '--force', move the file & make\n> it sparse.\n> 2. If 'file1' is *not* clean and '--force' is *not* used, refuse to move the\n> file (with a \"Path 'file1' not uptodate; will not move. Use '--force' to\n> override.\" type of error).\n> 3. If 'file1' is *not* clean and '--force' is used, move the file but do not\n> make it sparse.\n>\n> That way, '--force' really does force the move to happen, but users are\n> generally warned against it. I'm still not sure what the \"right\" approach\n> is, but to your point I think it should err on the side of not surprising\n> the user.\n\nI generally think this middle-ground option is good. Though I think the sort\nof options that \"messing with sparse contents\" should be handled by\n'--sparse', instead\nof '--force', since the latter is used to \"force move/rename even if\ntarget exists\".\nMixing the usage may cause syntax confusion?\n\n> > And the tweaked rule suggested by Junio [1] might need an extra\n> > `git sparse-checkout reapply` to re-sparsify the file that moved out-of-cone\n> > after staging its change?\n> >\n>\n> Just so I understand correctly, do you mean 'git sparse-checkout reapply'\n> *as part of* the 'mv' operation? Or are you thinking that a user might want\n> to manually run 'git sparse-checkout reapply' after running 'mv'?\n>\n> If it's the former (internally calling 'git sparse-checkout reapply' in\n> 'mv'), then no, you wouldn't want to do that. In Junio's suggestion, he said\n> (emphasis mine):\n>\n> > When a dirty path is moved out of cone, we would trigger the\n> > \"SKIP_WORKTREE automatically disabled\" behaviour\" *and that would be a\n> > good thing, I imagine?*\n>\n> We don't want the file moved out-of-cone to be sparse again because it has\n> local (on-disk) modifications that would disappear (since a file needs to be\n> removed from disk to be \"sparse\" in the eyes of 'sparse-checkout'). It's\n> *completely valid* behavior to have an out-of-cone file become non-sparse if\n> a user does something to cause that; it doesn't cause any bugs/corruption\n> with the repo. And, even if you did want to make the file sparse, it should\n> be done by manually setting 'SKIP_WORKTREE' and individually removing the\n> file from disk (for all the reasons I mentioned in my upthread comment [1]).\n>\n> On the other hand, if you're talking about a user manually running 'git\n> sparse-checkout reapply' after the fact, that wouldn't work either - they'd\n> get an error:\n> warning: The following paths are not up to date and were left despite sparse patterns:\n> \n\nThis is what I meant, a user manually running `git sparse-checkout reapply`.\nThough I did say users should only do this \"after staging its change\".\n\nI propose this solution which sounds good to me:\n\n1. If 'file1' is clean, iff with the use of '--sparse', move the file & make\n it sparse.\n2. If 'file1' is dirty, iff with the use of '--sparse', move the file\n& *do not* make\n it sparse, instead advise something like\n \"file1 is not up to date, keep it non-sparse.\n Stage file1 then run `git sparse-checkout reapply` to re-sparsify it.\"\n\n> [1] https://lore.kernel.org/git/077a0579-903e-32ad-029c-48572d471c84@github.com/\n>\n> > [1] https://lore.kernel.org/git/xmqq8rqm3fxa.fsf@gitster.g/\n> >\n-- \nThanks & Regards,\nShaoxuan\n"},{"id":"457510","messageId":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:42Z","receivedAt":"2022-06-19T03:26:13Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"The range-diff seems a bit messy, because some of the changes are too big\nthen I have to tune the --create-factor big enough to reveal these changes.\n\n## Changes since WIP v2 ##\n\n1. Write helper functions for t7002 to reuse some code.\n\n2. Refactor/decouple the if/else-if checking chain.\n\n3. Separate out the 'update_mode' refactor into a single commit.\n\n4. Stop using update_sparsity() and instead update the SKIP_WORKTREE\n bit for each cache_entry and check it out to the working tree.\n\n## Limitations ##\n\nAt this point, we still don't have in-cone to out-of-cone move, which\nI don't think is too much a problem, since the title says this series is\naround out-of-cone as the .\n\nBut I think it worth discuss if we should implement in-cone to \nout-of-cone move, since it will be nice (naturally) to have it working.\n\nHowever, I noticed this from the mv man page:\n\n\"In the second form, the last argument has to be an existing directory; \nthe given sources will be moved into this directory.\"\n\nI think trying to move out-of-cone, the last argument has to be an non-existent\ndirectory? I'm a bit confused: should we update some of mv basic logic to \naccomplish this?\n\nShaoxuan Yuan (7):\n t7002: add tests for moving out-of-cone file/directory\n mv: decouple if/else-if checks using goto\n mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n mv: check if exists in index to handle overwriting\n mv: use flags mode for update_mode\n mv: add check_dir_in_index() and solve general dir check issue\n mv: update sparsity after moving from out-of-cone to in-cone\n\n builtin/mv.c | 244 +++++++++++++++++++++++++---------\n t/t7002-mv-sparse-checkout.sh | 85 ++++++++++++\n 2 files changed, 264 insertions(+), 65 deletions(-)\n\nRange-diff against v2:\n1: 271445205d ! 1: a08ce96935 t7002: add tests for moving out-of-cone file/directory\n @@ Commit message\n \n Add corresponding tests to test following situations:\n \n - * 'refuse to move out-of-cone directory without --sparse'\n - * 'can move out-of-cone directory with --sparse'\n - * 'refuse to move out-of-cone file without --sparse'\n - * 'can move out-of-cone file with --sparse'\n - * 'refuse to move sparse file to existing destination'\n - * 'move sparse file to existing destination with --force and --sparse'\n + We do not have sufficient coverage of moving files outside\n + of a sparse-checkout cone. Create new tests covering this\n + behavior, keeping in mind that the user can include --sparse\n + (or not), move a file or directory, and the destination can\n + already exist in the index (in this case user can use --force\n + to overwrite existing entry).\n \n + Helped-by: Victoria Dye \n + Helped-by: Derrick Stolee \n Signed-off-by: Shaoxuan Yuan \n \n ## t/t7002-mv-sparse-checkout.sh ##\n +@@ t/t7002-mv-sparse-checkout.sh: test_description='git mv in sparse working trees'\n + \n + . ./test-lib.sh\n + \n ++setup_sparse_checkout () {\n ++\tmkdir folder1 &&\n ++\ttouch folder1/file1 &&\n ++\tgit add folder1 &&\n ++\tgit sparse-checkout set --cone sub\n ++}\n ++\n ++cleanup_sparse_checkout () {\n ++\tgit sparse-checkout disable &&\n ++\tgit reset --hard\n ++}\n ++\n + test_expect_success 'setup' \"\n + \tmkdir -p sub/dir sub/dir2 &&\n + \ttouch a b c sub/d sub/dir/e sub/dir2/e &&\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move files to non-sparse dir' '\n + '\n + \n + test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tgit reset --hard &&\n + \tgit sparse-checkout init --no-cone &&\n + \tgit sparse-checkout set a !/x y/ !x/y/z &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n +test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n -+\tgit sparse-checkout disable &&\n -+\tgit reset --hard &&\n -+\tmkdir folder1 &&\n -+\ttouch folder1/file1 &&\n -+\tgit add folder1 &&\n -+\tgit sparse-checkout init --cone &&\n -+\tgit sparse-checkout set sub &&\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n ++\tsetup_sparse_checkout &&\n +\n +\ttest_must_fail git mv folder1 sub 2>stderr &&\n +\tcat sparse_error_header >expect &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +'\n +\n +test_expect_failure 'can move out-of-cone directory with --sparse' '\n -+\tgit sparse-checkout disable &&\n -+\tgit reset --hard &&\n -+\tmkdir folder1 &&\n -+\ttouch folder1/file1 &&\n -+\tgit add folder1 &&\n -+\tgit sparse-checkout init --cone &&\n -+\tgit sparse-checkout set sub &&\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n ++\tsetup_sparse_checkout &&\n +\n +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n +\ttest_must_be_empty stderr &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +'\n +\n +test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n -+\tgit sparse-checkout disable &&\n -+\tgit reset --hard &&\n -+\tmkdir folder1 &&\n -+\ttouch folder1/file1 &&\n -+\tgit add folder1 &&\n -+\tgit sparse-checkout init --cone &&\n -+\tgit sparse-checkout set sub &&\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n ++\tsetup_sparse_checkout &&\n +\n +\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n +\tcat sparse_error_header >expect &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +'\n +\n +test_expect_failure 'can move out-of-cone file with --sparse' '\n -+\tgit sparse-checkout disable &&\n -+\tgit reset --hard &&\n -+\tmkdir folder1 &&\n -+\ttouch folder1/file1 &&\n -+\tgit add folder1 &&\n -+\tgit sparse-checkout init --cone &&\n -+\tgit sparse-checkout set sub &&\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n ++\tsetup_sparse_checkout &&\n +\n +\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n +\ttest_must_be_empty stderr &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +'\n +\n +test_expect_failure 'refuse to move sparse file to existing destination' '\n -+\tgit sparse-checkout disable &&\n -+\tgit reset --hard &&\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n +\tmkdir folder1 &&\n +\ttouch folder1/file1 &&\n +\ttouch sub/file1 &&\n +\tgit add folder1 sub/file1 &&\n -+\tgit sparse-checkout init --cone &&\n -+\tgit sparse-checkout set sub &&\n ++\tgit sparse-checkout set --cone sub &&\n +\n +\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n +\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +'\n +\n +test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n -+\tgit sparse-checkout disable &&\n -+\tgit reset --hard &&\n ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n +\tmkdir folder1 &&\n +\ttouch folder1/file1 &&\n +\ttouch sub/file1 &&\n +\techo \"overwrite\" >folder1/file1 &&\n +\tgit add folder1 sub/file1 &&\n -+\tgit sparse-checkout init --cone &&\n -+\tgit sparse-checkout set sub &&\n ++\tgit sparse-checkout set --cone sub &&\n +\n +\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n +\ttest_must_be_empty stderr &&\n-: ---------- > 2: 8065fbc232 mv: decouple if/else-if checks using goto\n2: 80f485f146 ! 3: e227fe717b mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n -+\t\t\t/*\n -+\t\t\t * TODO: for now, when you try to overwrite a \n -+\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n -+\t\t\t * flag, then the action will be done without providing \"--force\"\n -+\t\t\t * and no warning.\n -+\t\t\t *\n -+\t\t\t * This is mainly because the sparse \n -+\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n -+\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n -+\t\t\t */\n +-\t\t\t/* only error if existence is expected. */\n +-\t\t\tif (modes[i] != SPARSE) {\n ++\t\t\tint pos;\n ++\t\t\tconst struct cache_entry *ce;\n +\n -+\t\t\tint pos = cache_name_pos(src, length);\n -+\t\t\tif (pos >= 0) {\n -+\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n -+\n -+\t\t\t\tif (ce_skip_worktree(ce)) {\n -+\t\t\t\t\tif (!ignore_sparse)\n -+\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n -+\t\t\t\t\telse\n -+\t\t\t\t\t\tmodes[i] = SPARSE;\n -+\t\t\t\t}\n -+\t\t\t\telse\n ++\t\t\tpos = cache_name_pos(src, length);\n ++\t\t\tif (pos < 0) {\n ++\t\t\t\t/* only error if existence is expected. */\n ++\t\t\t\tif (modes[i] != SPARSE)\n +\t\t\t\t\tbad = _(\"bad source\");\n ++\t\t\t\tgoto act_on_entry;\n +\t\t\t}\n - \t\t\t/* only error if existence is expected. */\n --\t\t\tif (modes[i] != SPARSE)\n -+\t\t\telse if (modes[i] != SPARSE)\n ++\n ++\t\t\tce = active_cache[pos];\n ++\t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n - \t\t} else if (!strncmp(src, dst, length) &&\n - \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n + \t\t\t\tgoto act_on_entry;\n + \t\t\t}\n ++\n ++\t\t\tif (!ignore_sparse)\n ++\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n ++\t\t\telse\n ++\t\t\t\tmodes[i] = SPARSE;\n ++\t\t\tgoto act_on_entry;\n + \t\t}\n + \t\tif (!strncmp(src, dst, length) &&\n + \t\t (dst[length] == 0 || dst[length] == '/')) {\n \n ## t/t7002-mv-sparse-checkout.sh ##\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'can move out-of-cone directory with --sparse' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'can move out-of-cone directo\n \n -test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n +test_expect_success 'refuse to move out-of-cone file without --sparse' '\n - \tgit sparse-checkout disable &&\n - \tgit reset --hard &&\n - \tmkdir folder1 &&\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tsetup_sparse_checkout &&\n + \n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n \ttest_cmp expect stderr\n '\n \n -test_expect_failure 'can move out-of-cone file with --sparse' '\n +test_expect_success 'can move out-of-cone file with --sparse' '\n - \tgit sparse-checkout disable &&\n - \tgit reset --hard &&\n - \tmkdir folder1 &&\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tsetup_sparse_checkout &&\n + \n3: 04572e5e6b ! 4: d0de7678e3 mv: check if exists in index to handle overwriting\n @@ Commit message\n \n ## builtin/mv.c ##\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \n - \t\tlength = strlen(src);\n - \t\tif (lstat(src, &st) < 0) {\n --\t\t\t/*\n --\t\t\t * TODO: for now, when you try to overwrite a \n --\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n --\t\t\t * flag, then the action will be done without providing \"--force\"\n --\t\t\t * and no warning.\n --\t\t\t *\n --\t\t\t * This is mainly because the sparse \n --\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n --\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n --\t\t\t */\n - \n - \t\t\tint pos = cache_name_pos(src, length);\n - \t\t\tif (pos >= 0) {\n -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\t\tif (ce_skip_worktree(ce)) {\n - \t\t\t\t\tif (!ignore_sparse)\n - \t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n --\t\t\t\t\telse\n --\t\t\t\t\t\tmodes[i] = SPARSE;\n -+\t\t\t\t\telse {\n -+\t\t\t\t\t\t/* Check if dst exists in index */\n -+\t\t\t\t\t\tif (cache_name_pos(dst, strlen(dst)) >= 0) {\n -+\t\t\t\t\t\t\tif (force)\n -+\t\t\t\t\t\t\t\tmodes[i] = SPARSE;\n -+\t\t\t\t\t\t\telse\n -+\t\t\t\t\t\t\t\tbad = _(\"destination exists\");\n -+\t\t\t\t\t\t}\n -+\t\t\t\t\t\telse\n -+\t\t\t\t\t\t\tmodes[i] = SPARSE;\n -+\t\t\t\t\t}\n - \t\t\t\t}\n - \t\t\t\telse\n - \t\t\t\t\tbad = _(\"bad source\");\n + \t\t\t\tbad = _(\"bad source\");\n + \t\t\t\tgoto act_on_entry;\n + \t\t\t}\n +-\n +-\t\t\tif (!ignore_sparse)\n ++\t\t\tif (!ignore_sparse) {\n + \t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n +-\t\t\telse\n ++\t\t\t\tgoto act_on_entry;\n ++\t\t\t}\n ++\t\t\t/* Check if dst exists in index */\n ++\t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n + \t\t\t\tmodes[i] = SPARSE;\n ++\t\t\t\tgoto act_on_entry;\n ++\t\t\t}\n ++\t\t\tif (!force) {\n ++\t\t\t\tbad = _(\"destination exists\");\n ++\t\t\t\tgoto act_on_entry;\n ++\t\t\t}\n ++\t\t\tmodes[i] = SPARSE;\n + \t\t\tgoto act_on_entry;\n + \t\t}\n + \t\tif (!strncmp(src, dst, length) &&\n \n ## t/t7002-mv-sparse-checkout.sh ##\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file with --sparse' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file wi\n \n -test_expect_failure 'refuse to move sparse file to existing destination' '\n +test_expect_success 'refuse to move sparse file to existing destination' '\n - \tgit sparse-checkout disable &&\n - \tgit reset --hard &&\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n + \ttouch folder1/file1 &&\n-: ---------- > 5: 70540957b6 mv: use flags mode for update_mode\n4: 4eeae40186 ! 6: f8302f64e0 mv: add check_dir_in_index() and solve general dir check issue\n @@ Commit message\n instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n this operation can be carried out successfully.\n \n - Also, as suggested by Derrick [1],\n - move the in-line definition of \"enum update_mode\" to the top\n - of the file and make it use \"flags\" mode (each state is a different\n - bit in the word).\n -\n - [1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n -\n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n -@@ builtin/mv.c: static const char * const builtin_mv_usage[] = {\n - \tNULL\n - };\n - \n -+enum update_mode {\n -+\tBOTH = 0,\n -+\tWORKING_DIRECTORY = (1 << 1),\n -+\tINDEX = (1 << 2),\n -+\tSPARSE = (1 << 3),\n -+\tSKIP_WORKTREE_DIR = (1 << 4),\n -+};\n -+\n - #define DUP_BASENAME 1\n - #define KEEP_TRAILING_SLASH 2\n - \n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n {\n \tint i, flags, gitmodules_modified = 0;\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\tOPT_END(),\n - \t};\n - \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n --\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n -+\tenum update_mode *modes;\n - \tstruct stat st;\n - \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n - \tstruct lock_file lock_file = LOCK_INIT;\n -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\tif (lstat(src, &st) < 0) {\n - \n - \t\t\tint pos = cache_name_pos(src, length);\n -+\t\t\tconst char *src_w_slash = add_slash(src);\n -+\n - \t\t\tif (pos >= 0) {\n - \t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n + \t/* Checking */\n + \tfor (i = 0; i < argc; i++) {\n + \t\tconst char *src = source[i], *dst = destination[i];\n +-\t\tint length, src_is_dir;\n ++\t\tint length;\n + \t\tconst char *bad = NULL;\n + \t\tint skip_sparse = 0;\n \n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\t\telse\n + \n + \t\t\tpos = cache_name_pos(src, length);\n + \t\t\tif (pos < 0) {\n ++\t\t\t\tconst char *src_w_slash = add_slash(src);\n ++\t\t\t\tif (!check_dir_in_index(src, length) &&\n ++\t\t\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n ++\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n ++\t\t\t\t\tgoto dir_check;\n ++\t\t\t\t}\n + \t\t\t\t/* only error if existence is expected. */\n + \t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n + \t\t\t\tgoto act_on_entry;\n \t\t\t}\n -+\t\t\telse if (!check_dir_in_index(src, length) &&\n -+\t\t\t\t\t !path_in_sparse_checkout(src_w_slash, &the_index)) {\n -+\t\t\t\tmodes[i] = SKIP_WORKTREE_DIR;\n -+\t\t\t\tgoto dir_check;\n -+\t\t\t}\n - \t\t\t/* only error if existence is expected. */\n - \t\t\telse if (modes[i] != SPARSE)\n +-\n + \t\t\tce = active_cache[pos];\n + \t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\t\t&& lstat(dst, &st) == 0)\n + \t\t\tbad = _(\"can not move directory into itself\");\n + \t\t\tgoto act_on_entry;\n + \t\t}\n +-\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n ++\t\tif (S_ISDIR(st.st_mode)\n + \t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n - \t\telse if (src_is_dir) {\n + \t\t\tgoto act_on_entry;\n + \t\t}\n +-\t\tif (src_is_dir) {\n ++\n ++dir_check:\n ++\t\tif (S_ISDIR(st.st_mode)) {\n + \t\t\tint j, dst_len, n;\n -\t\t\tint first = cache_name_pos(src, length), last;\n +\t\t\tint first, last;\n -+dir_check:\n +\t\t\tfirst = cache_name_pos(src, length);\n \n - \t\t\tif (first >= 0)\n + \t\t\tif (first >= 0) {\n \t\t\t\tprepare_move_submodule(src, first,\n -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\telse { /* last - first >= 1 */\n - \t\t\t\tint j, dst_len, n;\n - \n --\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n -+\t\t\t\tif (!modes[i])\n -+\t\t\t\t\tmodes[i] |= WORKING_DIRECTORY;\n - \t\t\t\tn = argc + last - first;\n - \t\t\t\tREALLOC_ARRAY(source, n);\n - \t\t\t\tREALLOC_ARRAY(destination, n);\n -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n - \t\tif (show_only)\n - \t\t\tcontinue;\n --\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n -+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n -+\t\t \trename(src, dst) < 0) {\n - \t\t\tif (ignore_errors)\n - \t\t\t\tcontinue;\n - \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - \t\t\t\t\t\t\t 1);\n - \t\t}\n - \n --\t\tif (mode == WORKING_DIRECTORY)\n -+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n - \t\t\tcontinue;\n - \n - \t\tpos = cache_name_pos(src, strlen(src));\n \n ## t/t7002-mv-sparse-checkout.sh ##\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n \n -test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n +test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n - \tgit sparse-checkout disable &&\n - \tgit reset --hard &&\n - \tmkdir folder1 &&\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tsetup_sparse_checkout &&\n + \n @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n \ttest_cmp expect stderr\n '\n \n -test_expect_failure 'can move out-of-cone directory with --sparse' '\n +test_expect_success 'can move out-of-cone directory with --sparse' '\n - \tgit sparse-checkout disable &&\n - \tgit reset --hard &&\n - \tmkdir folder1 &&\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tsetup_sparse_checkout &&\n + \n5: a3a296c3ef ! 7: bc996931e2 mv: use update_sparsity() after touching sparse contents\n @@ Metadata\n Author: Shaoxuan Yuan \n \n ## Commit message ##\n - mv: use update_sparsity() after touching sparse contents\n + mv: update sparsity after moving from out-of-cone to in-cone\n \n - Originally, \"git mv\" a sparse file/directory from out/in-cone to\n - in/out-cone does not update the sparsity following the sparse-checkout\n - patterns.\n + Originally, \"git mv\" a sparse file from out-of-cone to\n + in-cone does not update the moved file's sparsity (remove its\n + SKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\n + not checked out in the working tree.\n \n - Use update_sparsity() after touching sparse contents, so the sparsity\n - will be updated after the move.\n + Update the behavior so that:\n + 1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n + corresponding cache entry.\n + 2. The moved cache entry is checked out in the working tree to reflect\n + the updated sparsity.\n \n Signed-off-by: Shaoxuan Yuan \n \n @@ builtin/mv.c\n #include \"string-list.h\"\n #include \"parse-options.h\"\n #include \"submodule.h\"\n -+#include \"unpack-trees.h\"\n ++#include \"entry.h\"\n \n static const char * const builtin_mv_usage[] = {\n \tN_(\"git mv [] ... \"),\n -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n - {\n - \tint i, flags, gitmodules_modified = 0;\n - \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n -+\tint sparse_moved = 0;\n - \tstruct option builtin_mv_options[] = {\n - \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n - \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n -+\t\tif (!sparse_moved && mode & (SPARSE | SKIP_WORKTREE_DIR))\n -+\t\t\tsparse_moved = 1;\n ++\t\tstruct checkout state = CHECKOUT_INIT;\n ++\t\tstate.istate = &the_index;\n ++\n ++\t\tif (force)\n ++\t\t\tstate.force = 1;\n \t\tif (show_only || verbose)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n + \t\tpos = cache_name_pos(src, strlen(src));\n + \t\tassert(pos >= 0);\n \t\trename_cache_entry_at(pos, dst);\n ++\n ++\t\tif (mode & SPARSE) {\n ++\t\t\tif (path_in_sparse_checkout(dst, &the_index)) {\n ++\t\t\t\tint dst_pos;\n ++\n ++\t\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n ++\t\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n ++\n ++\t\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n ++\t\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n ++\t\t\t}\n ++\t\t}\n \t}\n \n -+\tif (sparse_moved) {\n -+\t\tstruct unpack_trees_options o;\n -+\t\tmemset(&o, 0, sizeof(o));\n -+\t\to.verbose_update = isatty(2);\n -+\t\to.update = 1;\n -+\t\to.head_idx = -1;\n -+\t\to.src_index = &the_index;\n -+\t\to.dst_index = &the_index;\n -+\t\to.skip_sparse_checkout = 0;\n -+\t\to.pl = the_index.sparse_checkout_patterns;\n -+\t\tsetup_unpack_trees_porcelain(&o, \"mv\");\n -+\t\tupdate_sparsity(&o);\n -+\t\tclear_unpack_trees_porcelain(&o);\n -+\t}\n -+\n \tif (gitmodules_modified)\n - \t\tstage_updated_gitmodules(&the_index);\n - \n \n ## t/t7002-mv-sparse-checkout.sh ##\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone directory with --sparse' '\n + \tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n + \ttest_must_be_empty stderr &&\n + \n +-\tgit sparse-checkout reapply &&\n + \ttest_path_is_dir sub/folder1 &&\n + \ttest_path_is_file sub/folder1/file1\n + '\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file with --sparse' '\n + \tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n + \ttest_must_be_empty stderr &&\n + \n +-\tgit sparse-checkout reapply &&\n + \t! test_path_is_dir sub/folder1 &&\n + \ttest_path_is_file sub/file1\n + '\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move sparse file to existing destination' '\n \ttest_cmp expect stderr\n '\n \n -+# Need fix.\n -+#\n -+# The *expected* behavior:\n -+#\n -+# Using --sparse to accept a sparse file, --force to overwrite the destination.\n -+# The folder1/file1 should replace the sub/file1 without error.\n -+#\n -+# The *actual* behavior:\n -+#\n -+# It emits a warning:\n -+#\n -+# warning: Path ' sub/file1\n -+# ' already present; will not overwrite with sparse update.\n -+# After fixing the above paths, you may want to run `git sparse-checkout\n -+# reapply`.\n -+\n - test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n - \tgit sparse-checkout disable &&\n - \tgit reset --hard &&\n +-test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n ++test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tmkdir folder1 &&\n + \ttouch folder1/file1 &&\n\nbase-commit: 4f6db706e6ad145a9bf6b26a1ca0970bed27bb72\n-- \n2.35.1\n\n"},{"id":"457511","messageId":"20220619032549.156335-2-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 1/7] t7002: add tests for moving out-of-cone file/directory","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:43Z","receivedAt":"2022-06-19T03:26:15Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test following situations:\n\nWe do not have sufficient coverage of moving files outside\nof a sparse-checkout cone. Create new tests covering this\nbehavior, keeping in mind that the user can include --sparse\n(or not), move a file or directory, and the destination can\nalready exist in the index (in this case user can use --force\nto overwrite existing entry).\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 87 +++++++++++++++++++++++++++++++++++\n 1 file changed, 87 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex f0f7cbfcdb..d6e7315a5a 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -4,6 +4,18 @@ test_description='git mv in sparse working trees'\n \n . ./test-lib.sh\n \n+setup_sparse_checkout () {\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout set --cone sub\n+}\n+\n+cleanup_sparse_checkout () {\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard\n+}\n+\n test_expect_success 'setup' \"\n \tmkdir -p sub/dir sub/dir2 &&\n \ttouch a b c sub/d sub/dir/e sub/dir2/e &&\n@@ -196,6 +208,7 @@ test_expect_success 'can move files to non-sparse dir' '\n '\n \n test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tgit reset --hard &&\n \tgit sparse-checkout init --no-cone &&\n \tgit sparse-checkout set a !/x y/ !x/y/z &&\n@@ -206,4 +219,78 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n+test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv folder1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone directory with --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\tgit sparse-checkout reapply &&\n+\ttest_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/folder1/file1\n+'\n+\n+test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone file with --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\tgit sparse-checkout reapply &&\n+\t! test_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/file1\n+'\n+\n+test_expect_failure 'refuse to move sparse file to existing destination' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n+\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\techo \"overwrite\" >folder1/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\techo \"overwrite\" >expect &&\n+\ttest_cmp expect sub/file1\n+'\n+\n test_done\n-- \n2.35.1\n\n"},{"id":"457512","messageId":"20220619032549.156335-3-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 2/7] mv: decouple if/else-if checks using goto","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:44Z","receivedAt":"2022-06-19T03:26:17Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Previous if/else-if chain are highly nested and hard to develop/extend.\n\nRefactor to decouple this if/else-if chain by using goto to jump ahead.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 139 +++++++++++++++++++++++++++++----------------------\n 1 file changed, 80 insertions(+), 59 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba83..1ca2c21da8 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -186,53 +186,68 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\tif (modes[i] != SPARSE) {\n \t\t\t\tbad = _(\"bad source\");\n-\t\t} else if (!strncmp(src, dst, length) &&\n-\t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n+\t\tif (!strncmp(src, dst, length) &&\n+\t\t (dst[length] == 0 || dst[length] == '/')) {\n \t\t\tbad = _(\"can not move directory into itself\");\n-\t\t} else if ((src_is_dir = S_ISDIR(st.st_mode))\n-\t\t\t\t&& lstat(dst, &st) == 0)\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n-\t\telse if (src_is_dir) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (src_is_dir) {\n+\t\t\tint j, dst_len, n;\n \t\t\tint first = cache_name_pos(src, length), last;\n \n-\t\t\tif (first >= 0)\n+\t\t\tif (first >= 0) {\n \t\t\t\tprepare_move_submodule(src, first,\n \t\t\t\t\t\t submodule_gitfile + i);\n-\t\t\telse if (index_range_of_same_dir(src, length,\n-\t\t\t\t\t\t\t &first, &last) < 1)\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t} else if (index_range_of_same_dir(src, length,\n+\t\t\t\t\t\t\t &first, &last) < 1) {\n \t\t\t\tbad = _(\"source directory is empty\");\n-\t\t\telse { /* last - first >= 1 */\n-\t\t\t\tint j, dst_len, n;\n-\n-\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n-\t\t\t\tn = argc + last - first;\n-\t\t\t\tREALLOC_ARRAY(source, n);\n-\t\t\t\tREALLOC_ARRAY(destination, n);\n-\t\t\t\tREALLOC_ARRAY(modes, n);\n-\t\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n-\n-\t\t\t\tdst = add_slash(dst);\n-\t\t\t\tdst_len = strlen(dst);\n-\n-\t\t\t\tfor (j = 0; j < last - first; j++) {\n-\t\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n-\t\t\t\t\tconst char *path = ce->name;\n-\t\t\t\t\tsource[argc + j] = path;\n-\t\t\t\t\tdestination[argc + j] =\n-\t\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n-\t\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n-\t\t\t\t}\n-\t\t\t\targc += last - first;\n+\t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t} else if (!(ce = cache_file_exists(src, length, 0))) {\n+\n+\t\t\t/* last - first >= 1 */\n+\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tn = argc + last - first;\n+\t\t\tREALLOC_ARRAY(source, n);\n+\t\t\tREALLOC_ARRAY(destination, n);\n+\t\t\tREALLOC_ARRAY(modes, n);\n+\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n+\n+\t\t\tdst = add_slash(dst);\n+\t\t\tdst_len = strlen(dst);\n+\n+\t\t\tfor (j = 0; j < last - first; j++) {\n+\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n+\t\t\t\tconst char *path = ce->name;\n+\t\t\t\tsource[argc + j] = path;\n+\t\t\t\tdestination[argc + j] =\n+\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n+\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n+\t\t\t}\n+\t\t\targc += last - first;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (!(ce = cache_file_exists(src, length, 0))) {\n \t\t\tbad = _(\"not under version control\");\n-\t\t} else if (ce_stage(ce)) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (ce_stage(ce)) {\n \t\t\tbad = _(\"conflicted\");\n-\t\t} else if (lstat(dst, &st) == 0 &&\n-\t\t\t (!ignore_case || strcasecmp(src, dst))) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (lstat(dst, &st) == 0 &&\n+\t\t (!ignore_case || strcasecmp(src, dst))) {\n \t\t\tbad = _(\"destination exists\");\n \t\t\tif (force) {\n \t\t\t\t/*\n@@ -246,34 +261,40 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t} else\n \t\t\t\t\tbad = _(\"Cannot overwrite\");\n \t\t\t}\n-\t\t} else if (string_list_has_string(&src_for_dst, dst))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (string_list_has_string(&src_for_dst, dst)) {\n \t\t\tbad = _(\"multiple sources for the same target\");\n-\t\telse if (is_dir_sep(dst[strlen(dst) - 1]))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (is_dir_sep(dst[strlen(dst) - 1])) {\n \t\t\tbad = _(\"destination directory does not exist\");\n-\t\telse {\n-\t\t\t/*\n-\t\t\t * We check if the paths are in the sparse-checkout\n-\t\t\t * definition as a very final check, since that\n-\t\t\t * allows us to point the user to the --sparse\n-\t\t\t * option as a way to have a successful run.\n-\t\t\t */\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(src, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\n-\t\t\tif (skip_sparse)\n-\t\t\t\tgoto remove_entry;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n \n-\t\t\tstring_list_insert(&src_for_dst, dst);\n+\t\t/*\n+\t\t * We check if the paths are in the sparse-checkout\n+\t\t * definition as a very final check, since that\n+\t\t * allows us to point the user to the --sparse\n+\t\t * option as a way to have a successful run.\n+\t\t */\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(src, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\tskip_sparse = 1;\n+\t\t}\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n+\t\t\tskip_sparse = 1;\n \t\t}\n \n+\t\tif (skip_sparse)\n+\t\t\tgoto remove_entry;\n+\n+\t\tstring_list_insert(&src_for_dst, dst);\n+\n+act_on_entry:\n \t\tif (!bad)\n \t\t\tcontinue;\n \t\tif (!ignore_errors)\n-- \n2.35.1\n\n"},{"id":"457513","messageId":"20220619032549.156335-4-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 3/7] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:45Z","receivedAt":"2022-06-19T03:26:26Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a file which is not on-disk but exists in\nindex as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\nout with \"bad source\".\n\nChange the checking logic, so that such \nfile makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 21 +++++++++++++++++++--\n t/t7002-mv-sparse-checkout.sh | 4 ++--\n 2 files changed, 21 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 1ca2c21da8..9d8494a2e4 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -185,11 +185,28 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n-\t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE) {\n+\t\t\tint pos;\n+\t\t\tconst struct cache_entry *ce;\n+\n+\t\t\tpos = cache_name_pos(src, length);\n+\t\t\tif (pos < 0) {\n+\t\t\t\t/* only error if existence is expected. */\n+\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\t\tbad = _(\"bad source\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\n+\t\t\tce = active_cache[pos];\n+\t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n+\n+\t\t\tif (!ignore_sparse)\n+\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\telse\n+\t\t\t\tmodes[i] = SPARSE;\n+\t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n \t\t (dst[length] == 0 || dst[length] == '/')) {\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex d6e7315a5a..1984cf131d 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -242,7 +242,7 @@ test_expect_failure 'can move out-of-cone directory with --sparse' '\n \ttest_path_is_file sub/folder1/file1\n '\n \n-test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+test_expect_success 'refuse to move out-of-cone file without --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -253,7 +253,7 @@ test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone file with --sparse' '\n+test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n-- \n2.35.1\n\n"},{"id":"457514","messageId":"20220619032549.156335-5-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 4/7] mv: check if exists in index to handle overwriting","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:46Z","receivedAt":"2022-06-19T03:26:27Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a sparse file into cone can result in unwarned\noverwrite of existing entry. The expected behavior is that if the\n exists in the entry, user should be prompted to supply\na [-f|--force] to carry out the operation, or the operation should\nfail.\n\nAdd a check mechanism to do that.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 15 ++++++++++++---\n t/t7002-mv-sparse-checkout.sh | 2 +-\n 2 files changed, 13 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 9d8494a2e4..abb90d3266 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -201,11 +201,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\n-\t\t\tif (!ignore_sparse)\n+\t\t\tif (!ignore_sparse) {\n \t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\telse\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\t/* Check if dst exists in index */\n+\t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n \t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\tif (!force) {\n+\t\t\t\tbad = _(\"destination exists\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\tmodes[i] = SPARSE;\n \t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 1984cf131d..5b61fbad5f 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -265,7 +265,7 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_path_is_file sub/file1\n '\n \n-test_expect_failure 'refuse to move sparse file to existing destination' '\n+test_expect_success 'refuse to move sparse file to existing destination' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n-- \n2.35.1\n\n"},{"id":"457515","messageId":"20220619032549.156335-6-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 5/7] mv: use flags mode for update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:47Z","receivedAt":"2022-06-19T03:26:29Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"As suggested by Derrick [1],\nmove the in-line definition of \"enum update_mode\" to the top\nof the file and make it use \"flags\" mode (each state is a different\nbit in the word).\n\n[1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 26 ++++++++++++++++++--------\n 1 file changed, 18 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex abb90d3266..7ce7992d6c 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -19,6 +19,14 @@ static const char * const builtin_mv_usage[] = {\n \tNULL\n };\n \n+enum update_mode {\n+\tBOTH = 0,\n+\tWORKING_DIRECTORY = (1 << 1),\n+\tINDEX = (1 << 2),\n+\tSPARSE = (1 << 3),\n+\tSKIP_WORKTREE_DIR = (1 << 4),\n+};\n+\n #define DUP_BASENAME 1\n #define KEEP_TRAILING_SLASH 2\n \n@@ -129,7 +137,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n-\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n+\tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n@@ -191,7 +199,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n \t\t\t\t/* only error if existence is expected. */\n-\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n@@ -207,14 +215,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \t\t\t/* Check if dst exists in index */\n \t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n-\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\tmodes[i] |= SPARSE;\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n \t\t\tif (!force) {\n \t\t\t\tbad = _(\"destination exists\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t\tmodes[i] = SPARSE;\n+\t\t\tmodes[i] |= SPARSE;\n \t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n@@ -242,7 +250,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \n \t\t\t/* last - first >= 1 */\n-\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tmodes[i] |= WORKING_DIRECTORY;\n \t\t\tn = argc + last - first;\n \t\t\tREALLOC_ARRAY(source, n);\n \t\t\tREALLOC_ARRAY(destination, n);\n@@ -258,7 +266,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tsource[argc + j] = path;\n \t\t\t\tdestination[argc + j] =\n \t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tmemset(modes + argc + j, 0, sizeof(enum update_mode));\n+\t\t\t\tmodes[argc + j] |= ce_skip_worktree(ce) ? SPARSE : INDEX;\n \t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n \t\t\t}\n \t\t\targc += last - first;\n@@ -355,7 +364,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n+\t\t\trename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n@@ -369,7 +379,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode == WORKING_DIRECTORY)\n+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\n-- \n2.35.1\n\n"},{"id":"457516","messageId":"20220619032549.156335-7-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 6/7] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:48Z","receivedAt":"2022-06-19T03:26:35Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a directory which is not on-disk due\nto its existence outside of sparse-checkout cone, \"giv mv\" command\nerrors out with \"bad source\".\n\nAdd a helper check_dir_in_index() function to see if a directory\nname exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\nsuch directories.\n\nChange the checking logic, so that such directory makes\n\"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 49 +++++++++++++++++++++++++++++++----\n t/t7002-mv-sparse-checkout.sh | 4 +--\n 2 files changed, 46 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 7ce7992d6c..cb3441c7cb 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -123,6 +123,37 @@ static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n \n+/*\n+ * Check if an out-of-cone directory should be in the index. Imagine this case\n+ * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n+ * and thus the directory is sparsified.\n+ *\n+ * Return 0 if such directory exist (i.e. with any of its contained files not\n+ * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n+ * Return 1 otherwise.\n+ */\n+static int check_dir_in_index(const char *name, int namelen)\n+{\n+\tint ret = 1;\n+\tconst char *with_slash = add_slash(name);\n+\tint length = namelen + 1;\n+\n+\tint pos = cache_name_pos(with_slash, length);\n+\tconst struct cache_entry *ce;\n+\n+\tif (pos < 0) {\n+\t\tpos = -pos - 1;\n+\t\tif (pos >= the_index.cache_nr)\n+\t\t\treturn ret;\n+\t\tce = active_cache[pos];\n+\t\tif (strncmp(with_slash, ce->name, length))\n+\t\t\treturn ret;\n+\t\tif (ce_skip_worktree(ce))\n+\t\t\treturn ret = 0;\n+\t}\n+\treturn ret;\n+}\n+\n int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n@@ -184,7 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t/* Checking */\n \tfor (i = 0; i < argc; i++) {\n \t\tconst char *src = source[i], *dst = destination[i];\n-\t\tint length, src_is_dir;\n+\t\tint length;\n \t\tconst char *bad = NULL;\n \t\tint skip_sparse = 0;\n \n@@ -198,12 +229,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n+\t\t\t\tconst char *src_w_slash = add_slash(src);\n+\t\t\t\tif (!check_dir_in_index(src, length) &&\n+\t\t\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n+\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n+\t\t\t\t\tgoto dir_check;\n+\t\t\t\t}\n \t\t\t\t/* only error if existence is expected. */\n \t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\n \t\t\tce = active_cache[pos];\n \t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n@@ -230,14 +266,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tbad = _(\"can not move directory into itself\");\n \t\t\tgoto act_on_entry;\n \t\t}\n-\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\tif (S_ISDIR(st.st_mode)\n \t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n \t\t\tgoto act_on_entry;\n \t\t}\n-\t\tif (src_is_dir) {\n+\n+dir_check:\n+\t\tif (S_ISDIR(st.st_mode)) {\n \t\t\tint j, dst_len, n;\n-\t\t\tint first = cache_name_pos(src, length), last;\n+\t\t\tint first, last;\n+\t\t\tfirst = cache_name_pos(src, length);\n \n \t\t\tif (first >= 0) {\n \t\t\t\tprepare_move_submodule(src, first,\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 5b61fbad5f..30e13b9979 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -219,7 +219,7 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -230,7 +230,7 @@ test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone directory with --sparse' '\n+test_expect_success 'can move out-of-cone directory with --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n-- \n2.35.1\n\n"},{"id":"457517","messageId":"20220619032549.156335-8-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"[WIP v3 7/7] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-19T03:25:49Z","receivedAt":"2022-06-19T03:26:36Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, \"git mv\" a sparse file from out-of-cone to\nin-cone does not update the moved file's sparsity (remove its\nSKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\nnot checked out in the working tree.\n\nUpdate the behavior so that:\n1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n corresponding cache entry.\n2. The moved cache entry is checked out in the working tree to reflect\n the updated sparsity.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 18 ++++++++++++++++++\n t/t7002-mv-sparse-checkout.sh | 4 +---\n 2 files changed, 19 insertions(+), 3 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex cb3441c7cb..a8b9f55654 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -13,6 +13,7 @@\n #include \"string-list.h\"\n #include \"parse-options.h\"\n #include \"submodule.h\"\n+#include \"entry.h\"\n \n static const char * const builtin_mv_usage[] = {\n \tN_(\"git mv [] ... \"),\n@@ -399,6 +400,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tstruct checkout state = CHECKOUT_INIT;\n+\t\tstate.istate = &the_index;\n+\n+\t\tif (force)\n+\t\t\tstate.force = 1;\n \t\tif (show_only || verbose)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n@@ -424,6 +430,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n \t\trename_cache_entry_at(pos, dst);\n+\n+\t\tif (mode & SPARSE) {\n+\t\t\tif (path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\t\tint dst_pos;\n+\n+\t\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n+\n+\t\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n+\t\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n+\t\t\t}\n+\t\t}\n \t}\n \n \tif (gitmodules_modified)\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 30e13b9979..7734119197 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -237,7 +237,6 @@ test_expect_success 'can move out-of-cone directory with --sparse' '\n \tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n \ttest_must_be_empty stderr &&\n \n-\tgit sparse-checkout reapply &&\n \ttest_path_is_dir sub/folder1 &&\n \ttest_path_is_file sub/folder1/file1\n '\n@@ -260,7 +259,6 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n \tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n \ttest_must_be_empty stderr &&\n \n-\tgit sparse-checkout reapply &&\n \t! test_path_is_dir sub/folder1 &&\n \ttest_path_is_file sub/file1\n '\n@@ -278,7 +276,7 @@ test_expect_success 'refuse to move sparse file to existing destination' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n-- \n2.35.1\n\n"},{"id":"457642","messageId":"adb795ba-56ce-8441-0c38-a3e6b0a6e861@github.com","threadId":"57642","inReplyTo":"20220619032549.156335-2-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v3 1/7] t7002: add tests for moving out-of-cone file/directory","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-06-21T21:23:29Z","receivedAt":"2022-06-21T21:27:29Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Add corresponding tests to test following situations:\n> \n> We do not have sufficient coverage of moving files outside\n> of a sparse-checkout cone. Create new tests covering this\n> behavior, keeping in mind that the user can include --sparse\n> (or not), move a file or directory, and the destination can\n> already exist in the index (in this case user can use --force\n> to overwrite existing entry).\n> \n> Helped-by: Victoria Dye \n> Helped-by: Derrick Stolee \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> t/t7002-mv-sparse-checkout.sh | 87 +++++++++++++++++++++++++++++++++++\n> 1 file changed, 87 insertions(+)\n> \n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index f0f7cbfcdb..d6e7315a5a 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -4,6 +4,18 @@ test_description='git mv in sparse working trees'\n> \n> . ./test-lib.sh\n> \n> +setup_sparse_checkout () {\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\tgit add folder1 &&\n> +\tgit sparse-checkout set --cone sub\n> +}\n> +\n> +cleanup_sparse_checkout () {\n> +\tgit sparse-checkout disable &&\n> +\tgit reset --hard\n> +}\n> +\n> test_expect_success 'setup' \"\n> \tmkdir -p sub/dir sub/dir2 &&\n> \ttouch a b c sub/d sub/dir/e sub/dir2/e &&\n> @@ -196,6 +208,7 @@ test_expect_success 'can move files to non-sparse dir' '\n> '\n> \n> test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tgit reset --hard &&\n> \tgit sparse-checkout init --no-cone &&\n> \tgit sparse-checkout set a !/x y/ !x/y/z &&\n> @@ -206,4 +219,78 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> \ttest_cmp expect stderr\n> '\n> \n> +test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n\nThe setup and cleanup approach looks good - thanks for updating it!\n\n> +\n> +\ttest_must_fail git mv folder1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone directory with --sparse' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\n> +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\tgit sparse-checkout reapply &&\n\nYou shouldn't need to run 'reapply' here (you remove it in Patch 7, but it\nshould probably be dropped here instead).\n\n> +\ttest_path_is_dir sub/folder1 &&\n> +\ttest_path_is_file sub/folder1/file1\n> +'\n> +\n> +test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\n> +\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> +\techo folder1/file1 >>expect &&\n> +\tcat sparse_hint >>expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone file with --sparse' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tsetup_sparse_checkout &&\n> +\n> +\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\n> +\tgit sparse-checkout reapply &&\n> +\t! test_path_is_dir sub/folder1 &&\n> +\ttest_path_is_file sub/file1\n\nYou can also drop the 'reapply' here (same reason as above), but you'll also\nprobably need to drop the '! test_path_is_dir sub/folder1'. Based on some\nrough testing of the command in its current state, 'git mv' doesn't delete a\ndirectory if 'mv' the last remaining file in that directory. In this test,\nthe directory being deleted is a result of 'sparse-checkout reapply', not\n'mv'.\n\n> +'\n> +\n> +test_expect_failure 'refuse to move sparse file to existing destination' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\ttouch sub/file1 &&\n> +\tgit add folder1 sub/file1 &&\n> +\tgit sparse-checkout set --cone sub &&\n> +\n> +\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n> +\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n> +\ttest_cmp expect stderr\n> +'\n> +\n> +test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\ttouch sub/file1 &&\n> +\techo \"overwrite\" >folder1/file1 &&\n> +\tgit add folder1 sub/file1 &&\n> +\tgit sparse-checkout set --cone sub &&\n> +\n> +\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> +\techo \"overwrite\" >expect &&\n> +\ttest_cmp expect sub/file1\n> +'\n> +\n> test_done\n\n"},{"id":"457650","messageId":"01b39c63-5652-4293-0424-ff99b6f9f7d2@github.com","threadId":"57642","inReplyTo":"20220619032549.156335-6-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v3 5/7] mv: use flags mode for update_mode","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-06-21T22:32:54Z","receivedAt":"2022-06-21T22:33:12Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> As suggested by Derrick [1],\n> move the in-line definition of \"enum update_mode\" to the top\n> of the file and make it use \"flags\" mode (each state is a different\n> bit in the word).\n> \n\nThis message doesn't quite cover all of what's done in the commit. In\naddition to moving the enum definition, you introduce a 'SKIP_WORKTREE_DIR'\nflag and change the flag assignments to '|=' (additive) from '=' (single\nassignment). If those changes belong in this commit (not a later one), they\nshould be explained in the message here.\n\n> [1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 26 ++++++++++++++++++--------\n> 1 file changed, 18 insertions(+), 8 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index abb90d3266..7ce7992d6c 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -19,6 +19,14 @@ static const char * const builtin_mv_usage[] = {\n> \tNULL\n> };\n> \n> +enum update_mode {\n> +\tBOTH = 0,\n\nI know this comes from the original inline enum, but I don't see 'BOTH' used\nanywhere. The name itself is somewhat confusing (I have no idea what \"both\"\nis referring to - possibly \"both\" 'WORKING_DIRECTORY' and 'INDEX'??), so\nwould you mind removing it in the next re-roll? \n\n> +\tWORKING_DIRECTORY = (1 << 1),\n> +\tINDEX = (1 << 2),\n> +\tSPARSE = (1 << 3),\n> +\tSKIP_WORKTREE_DIR = (1 << 4),\n\nYou're not introducing any assignment of 'SKIP_WORKTREE_DIR' in this commit\n(looks like that's done in the next one, patch [6/7]), so you should\nprobably 'SKIP_WORKTREE_DIR' and its corresponding usage in that patch\ninstead of this one.\n\n> +};\n\nWhen the update modes were mutually-exclusive, it made sense for them to be\nrepresented by an enum. Now that they're flags that can be combined, should\nthey instead be pre-processor '#define' values (e.g., like the 'RESET_*'\nmodes in 'reset.h' or 'CE_*' flags in 'cache.h')? I don't actually know what\nthe standard is, since I also see one or two examples of using enums as\nflags (e.g., 'commit_graph_split_flags' in 'commit-graph.h'). Maybe another\ncontributor could clarify? \n\n> +\n> #define DUP_BASENAME 1\n> #define KEEP_TRAILING_SLASH 2\n> \n> @@ -129,7 +137,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tOPT_END(),\n> \t};\n> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n> -\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n> +\tenum update_mode *modes;\n> \tstruct stat st;\n> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n> \tstruct lock_file lock_file = LOCK_INIT;\n> @@ -191,7 +199,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\tpos = cache_name_pos(src, length);\n> \t\t\tif (pos < 0) {\n> \t\t\t\t/* only error if existence is expected. */\n> -\t\t\t\tif (modes[i] != SPARSE)\n> +\t\t\t\tif (!(modes[i] & SPARSE))\n> \t\t\t\t\tbad = _(\"bad source\");\n> \t\t\t\tgoto act_on_entry;\n> \t\t\t}\n> @@ -207,14 +215,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t}\n> \t\t\t/* Check if dst exists in index */\n> \t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n> -\t\t\t\tmodes[i] = SPARSE;\n> +\t\t\t\tmodes[i] |= SPARSE;\n> \t\t\t\tgoto act_on_entry;\n> \t\t\t}\n> \t\t\tif (!force) {\n> \t\t\t\tbad = _(\"destination exists\");\n> \t\t\t\tgoto act_on_entry;\n> \t\t\t}\n> -\t\t\tmodes[i] = SPARSE;\n> +\t\t\tmodes[i] |= SPARSE;\n> \t\t\tgoto act_on_entry;\n> \t\t}\n> \t\tif (!strncmp(src, dst, length) &&\n> @@ -242,7 +250,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t}\n> \n> \t\t\t/* last - first >= 1 */\n> -\t\t\tmodes[i] = WORKING_DIRECTORY;\n> +\t\t\tmodes[i] |= WORKING_DIRECTORY;\n> \t\t\tn = argc + last - first;\n> \t\t\tREALLOC_ARRAY(source, n);\n> \t\t\tREALLOC_ARRAY(destination, n);\n> @@ -258,7 +266,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\tsource[argc + j] = path;\n> \t\t\t\tdestination[argc + j] =\n> \t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n> -\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n> +\t\t\t\tmemset(modes + argc + j, 0, sizeof(enum update_mode));\n\nOne benefit of using '#define' values would be that 'modes' would just be an\narray of unsigned ints, so you could just assign '0' rather than using\nmemset. In terms of the implementation as-is, though, I think what you have\nis correct.\n\n> +\t\t\t\tmodes[argc + j] |= ce_skip_worktree(ce) ? SPARSE : INDEX;\n> \t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n> \t\t\t}\n> \t\t\targc += last - first;\n> @@ -355,7 +364,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> \t\tif (show_only)\n> \t\t\tcontinue;\n> -\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n> +\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> +\t\t\trename(src, dst) < 0) {\n\nNit: could you align 'rename' with the line above it (per the highlighted\nsection in the CodingGuidelines [1])? As far as I can tell, the \"align with\ntabs and spaces\" approach is what's *intended* to be used in 'mv.c'\n(although it's admittedly pretty inconsistent).\n\n[1] https://github.com/git/git/blob/master/Documentation/CodingGuidelines#L371-L383 \n\n> \t\t\tif (ignore_errors)\n> \t\t\t\tcontinue;\n> \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n> @@ -369,7 +379,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\t\t\t\t\t 1);\n> \t\t}\n> \n> -\t\tif (mode == WORKING_DIRECTORY)\n> +\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n> \t\t\tcontinue;\n> \n> \t\tpos = cache_name_pos(src, strlen(src));\n\n"},{"id":"457667","messageId":"11c81f19-a234-7d63-4d77-f1f15bdf70fb@github.com","threadId":"57642","inReplyTo":"20220619032549.156335-7-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v3 6/7] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-06-21T22:55:56Z","receivedAt":"2022-06-21T22:56:04Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, moving a directory which is not on-disk due\n> to its existence outside of sparse-checkout cone, \"giv mv\" command\n> errors out with \"bad source\".\n> \n> Add a helper check_dir_in_index() function to see if a directory\n> name exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\n> such directories.\n> \n> Change the checking logic, so that such directory makes\n> \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n> this operation can be carried out successfully.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 49 +++++++++++++++++++++++++++++++----\n> t/t7002-mv-sparse-checkout.sh | 4 +--\n> 2 files changed, 46 insertions(+), 7 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index 7ce7992d6c..cb3441c7cb 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -123,6 +123,37 @@ static int index_range_of_same_dir(const char *src, int length,\n> \treturn last - first;\n> }\n> \n> +/*\n> + * Check if an out-of-cone directory should be in the index. Imagine this case\n> + * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n> + * and thus the directory is sparsified.> + *\n> + * Return 0 if such directory exist (i.e. with any of its contained files not\n> + * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n> + * Return 1 otherwise.\n> + */\nThis explanation is helpful in clarifying that you don't mean *sparse\ndirectories* (that is, directory entries in a sparse index), you mean\ndirectories whose contents are all sparse. It's a tricky distinction, but\nyou handled it nicely here.\n\n> +static int check_dir_in_index(const char *name, int namelen)\n> +{\n> +\tint ret = 1;\n> +\tconst char *with_slash = add_slash(name);\n> +\tint length = namelen + 1;\n> +\n> +\tint pos = cache_name_pos(with_slash, length);\n> +\tconst struct cache_entry *ce;\n> +\n> +\tif (pos < 0) {\n> +\t\tpos = -pos - 1;\n> +\t\tif (pos >= the_index.cache_nr)\n> +\t\t\treturn ret;\n> +\t\tce = active_cache[pos];\n> +\t\tif (strncmp(with_slash, ce->name, length))\n> +\t\t\treturn ret;\n> +\t\tif (ce_skip_worktree(ce))\n> +\t\t\treturn ret = 0;\n> +\t}\n> +\treturn ret;\n\nThe way 'ret' is handled here is a bit difficult to follow. Would you be\nopposed to returning hardcoded '0' or '1', rather than changing the value of\n'ret' throughout? Something like:\n\nstatic int check_dir_in_index(const char *name, int namelen)\n{\n\tint pos, length = namelen + 1;\n\tconst struct cache_entry *ce;\n\tconst char *with_slash = add_slash(name);\n\n\tpos = cache_name_pos(with_slash, length);\n\tif (pos < 0) {\n\t\tpos = -pos - 1;\n\t\tif (pos >= the_index.cache_nr)\n\t\t\treturn 1;\n\t\tce = active_cache[pos];\n\t\tif (strncmp(with_slash, ce->name, length))\n\t\t\treturn 1;\n\t\tif (ce_skip_worktree(ce))\n\t\t\treturn 0;\n\t}\n\treturn 1;\n}\n\n> +}\n> +\n> int cmd_mv(int argc, const char **argv, const char *prefix)\n> {\n> \tint i, flags, gitmodules_modified = 0;\n> @@ -184,7 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t/* Checking */\n> \tfor (i = 0; i < argc; i++) {\n> \t\tconst char *src = source[i], *dst = destination[i];\n> -\t\tint length, src_is_dir;\n> +\t\tint length;\n> \t\tconst char *bad = NULL;\n> \t\tint skip_sparse = 0;\n> \n> @@ -198,12 +229,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \n> \t\t\tpos = cache_name_pos(src, length);\n> \t\t\tif (pos < 0) {\n> +\t\t\t\tconst char *src_w_slash = add_slash(src);\n> +\t\t\t\tif (!check_dir_in_index(src, length) &&\n> +\t\t\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n\nIn checks like these, the less \"expensive\" one should come first (so that if\nit returns 'false', we completely skip the more expensive one). Since\n'check_dir_in_index()' requires binary searching the index, it's likely to\nbe more expensive than 'path_in_sparse_checkout()', so the condition order\nshould be flipped:\n\n\t\t\t\tif (!path_in_sparse_checkout(src_w_slash, &the_index) &&\n\t\t\t\t !check_dir_in_index(src, length)) {\n\nAlso nit: alignment (more details on why/how in my last message [1]).\n\n[1] https://lore.kernel.org/git/01b39c63-5652-4293-0424-ff99b6f9f7d2@github.com/\n\n> +\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n> +\t\t\t\t\tgoto dir_check;\n> +\t\t\t\t}\n> \t\t\t\t/* only error if existence is expected. */\n> \t\t\t\tif (!(modes[i] & SPARSE))\n> \t\t\t\t\tbad = _(\"bad source\");\n> \t\t\t\tgoto act_on_entry;\n> \t\t\t}\n> -\n> \t\t\tce = active_cache[pos];\n> \t\t\tif (!ce_skip_worktree(ce)) {\n> \t\t\t\tbad = _(\"bad source\");\n> @@ -230,14 +266,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\t\tbad = _(\"can not move directory into itself\");\n> \t\t\tgoto act_on_entry;\n> \t\t}\n> -\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n> +\t\tif (S_ISDIR(st.st_mode)\n> \t\t && lstat(dst, &st) == 0) {\n> \t\t\tbad = _(\"cannot move directory over file\");\n> \t\t\tgoto act_on_entry;\n> \t\t}\n> -\t\tif (src_is_dir) {\n> +\n> +dir_check:\n> +\t\tif (S_ISDIR(st.st_mode)) {\n> \t\t\tint j, dst_len, n;\n> -\t\t\tint first = cache_name_pos(src, length), last;\n> +\t\t\tint first, last;\n> +\t\t\tfirst = cache_name_pos(src, length);\n\nSuper-nit: why did this line change? It looks like it just rearranges the\nlines for no functional purpose.\n\n> \n> \t\t\tif (first >= 0) {\n> \t\t\t\tprepare_move_submodule(src, first,\n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 5b61fbad5f..30e13b9979 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -219,7 +219,7 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> \ttest_cmp expect stderr\n> '\n> \n> -test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> +test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \n> @@ -230,7 +230,7 @@ test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> \ttest_cmp expect stderr\n> '\n> \n> -test_expect_failure 'can move out-of-cone directory with --sparse' '\n> +test_expect_success 'can move out-of-cone directory with --sparse' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tsetup_sparse_checkout &&\n> \n\n"},{"id":"457668","messageId":"cabeda2a-9681-a026-fbd7-87aa6139b881@github.com","threadId":"57642","inReplyTo":"20220619032549.156335-8-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v3 7/7] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-06-21T23:11:16Z","receivedAt":"2022-06-21T23:11:22Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> Originally, \"git mv\" a sparse file from out-of-cone to\n> in-cone does not update the moved file's sparsity (remove its\n> SKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\n> not checked out in the working tree.\n> \n> Update the behavior so that:\n> 1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n> corresponding cache entry.\n> 2. The moved cache entry is checked out in the working tree to reflect\n> the updated sparsity.\n> \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 18 ++++++++++++++++++\n> t/t7002-mv-sparse-checkout.sh | 4 +---\n> 2 files changed, 19 insertions(+), 3 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index cb3441c7cb..a8b9f55654 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -13,6 +13,7 @@\n> #include \"string-list.h\"\n> #include \"parse-options.h\"\n> #include \"submodule.h\"\n> +#include \"entry.h\"\n> \n> static const char * const builtin_mv_usage[] = {\n> \tN_(\"git mv [] ... \"),\n> @@ -399,6 +400,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tconst char *src = source[i], *dst = destination[i];\n> \t\tenum update_mode mode = modes[i];\n> \t\tint pos;\n> +\t\tstruct checkout state = CHECKOUT_INIT;\n> +\t\tstate.istate = &the_index;\n> +\n> +\t\tif (force)\n> +\t\t\tstate.force = 1;\n> \t\tif (show_only || verbose)\n> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> \t\tif (show_only)\n> @@ -424,6 +430,18 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tpos = cache_name_pos(src, strlen(src));\n> \t\tassert(pos >= 0);\n> \t\trename_cache_entry_at(pos, dst);\n\nAt first I wasn't sure how this would handle moving whole \"sparse\"\ndirectories (i.e., directories containing all 'SKIP_WORKTREE' entries),\nsince this loop only iterates over 'argc'. The good news is: it does work,\nsuccessfully moving each file in the directory individually! Unfortunately,\nthe *reason* it works is because 'mv' changes the value of 'argc' to include\nthe new directories.\n\nAll this to say - your implementation is good (and IMO doesn't require any\nchanges), it just happens to sit alongside somewhat questionable code. :)\n\n> +\n> +\t\tif (mode & SPARSE) {\n> +\t\t\tif (path_in_sparse_checkout(dst, &the_index)) {\n\nNit: this can be consolidated into a single condition:\n\n\t\tif ((mode & SPARSE) && \n\t\t path_in_sparse_checkout(dst, &the_index)) {\n\n> +\t\t\t\tint dst_pos;\n> +\n> +\t\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n> +\t\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n> +\n> +\t\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n> +\t\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n> +\t\t\t}\n> +\t\t}\n> \t}\n> \n> \tif (gitmodules_modified)\n> diff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\n> index 30e13b9979..7734119197 100755\n> --- a/t/t7002-mv-sparse-checkout.sh\n> +++ b/t/t7002-mv-sparse-checkout.sh\n> @@ -237,7 +237,6 @@ test_expect_success 'can move out-of-cone directory with --sparse' '\n> \tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n> \ttest_must_be_empty stderr &&\n> \n> -\tgit sparse-checkout reapply &&\n> \ttest_path_is_dir sub/folder1 &&\n> \ttest_path_is_file sub/folder1/file1\n> '\n> @@ -260,7 +259,6 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n> \tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n> \ttest_must_be_empty stderr &&\n> \n> -\tgit sparse-checkout reapply &&\n> \t! test_path_is_dir sub/folder1 &&\n> \ttest_path_is_file sub/file1\n> '\n> @@ -278,7 +276,7 @@ test_expect_success 'refuse to move sparse file to existing destination' '\n> \ttest_cmp expect stderr\n> '\n> \n> -test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> +test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n> \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tmkdir folder1 &&\n> \ttouch folder1/file1 &&\n\n"},{"id":"457669","messageId":"d3aa35e4-5bad-2bbb-2a25-d82064d6ec81@github.com","threadId":"57642","inReplyTo":"20220619032549.156335-1-shaoxuan.yuan02@gmail.com","subject":"Re: [WIP v3 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Victoria Dye","fromEmail":"vdye@github.com","sentAt":"2022-06-21T23:30:38Z","receivedAt":"2022-06-21T23:30:47Z","isPatch":false,"sender":{"key":"vdye@github.com","avatar":"https://avatars.githubusercontent.com/u/3619353?v=4"},"body":"Shaoxuan Yuan wrote:\n> The range-diff seems a bit messy, because some of the changes are too big\n> then I have to tune the --create-factor big enough to reveal these changes.\n> \n> ## Changes since WIP v2 ##\n> \n> 1. Write helper functions for t7002 to reuse some code.\n> \n> 2. Refactor/decouple the if/else-if checking chain.\n> \n> 3. Separate out the 'update_mode' refactor into a single commit.\n> \n> 4. Stop using update_sparsity() and instead update the SKIP_WORKTREE\n> bit for each cache_entry and check it out to the working tree.\n\nAll of this looks great! My comments are all minor syntax nits - the\nfunctionality is sound. Thanks again for working through this tangled mess\nof a problem!\n\n> \n> ## Limitations ##\n> \n> At this point, we still don't have in-cone to out-of-cone move, which\n> I don't think is too much a problem, since the title says this series is\n> around out-of-cone as the .\n> \n\nWhile it would be nice to include in-cone -> out-of-cone here, I'm also\ncontent with you moving it to a later series (the first couple patches in a\nsparse index integration or as a standalone series; up to you). \n\n> But I think it worth discuss if we should implement in-cone to \n> out-of-cone move, since it will be nice (naturally) to have it working.\n> \n> However, I noticed this from the mv man page:\n> \n> \"In the second form, the last argument has to be an existing directory; \n> the given sources will be moved into this directory.\"\n> \n> I think trying to move out-of-cone, the last argument has to be an non-existent\n> directory? I'm a bit confused: should we update some of mv basic logic to \n> accomplish this?\n> \n\nI suspect this requirement is related to the POSIX 'mv' [1] (and\ncorresponding 'rename()', used in 'git mv'), which also requires that the\ndestination directory exists. I personally don't think this requirement\nneeds to apply to 'git mv' at all, but note that changing the behavior would\nrequire first creating the necessary directories before calling 'rename()'. \n\nAs a more conservative solution, you could do the parent directory creation\n*only* in the case of moving to a sparse contents-only directory (using\nsomething like the 'check_dir_in_index()' function you introduced to\nidentify).\n\nI'm also interested in hearing what others have to say, especially regarding\nhistorical context/use cases of 'git mv'.\n\n[1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/mv.html\n\n> Shaoxuan Yuan (7):\n> t7002: add tests for moving out-of-cone file/directory\n> mv: decouple if/else-if checks using goto\n> mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n> mv: check if exists in index to handle overwriting\n> mv: use flags mode for update_mode\n> mv: add check_dir_in_index() and solve general dir check issue\n> mv: update sparsity after moving from out-of-cone to in-cone\n> \n> builtin/mv.c | 244 +++++++++++++++++++++++++---------\n> t/t7002-mv-sparse-checkout.sh | 85 ++++++++++++\n> 2 files changed, 264 insertions(+), 65 deletions(-)\n> \n> Range-diff against v2:\n> 1: 271445205d ! 1: a08ce96935 t7002: add tests for moving out-of-cone file/directory\n> @@ Commit message\n> \n> Add corresponding tests to test following situations:\n> \n> - * 'refuse to move out-of-cone directory without --sparse'\n> - * 'can move out-of-cone directory with --sparse'\n> - * 'refuse to move out-of-cone file without --sparse'\n> - * 'can move out-of-cone file with --sparse'\n> - * 'refuse to move sparse file to existing destination'\n> - * 'move sparse file to existing destination with --force and --sparse'\n> + We do not have sufficient coverage of moving files outside\n> + of a sparse-checkout cone. Create new tests covering this\n> + behavior, keeping in mind that the user can include --sparse\n> + (or not), move a file or directory, and the destination can\n> + already exist in the index (in this case user can use --force\n> + to overwrite existing entry).\n> \n> + Helped-by: Victoria Dye \n> + Helped-by: Derrick Stolee \n> Signed-off-by: Shaoxuan Yuan \n> \n> ## t/t7002-mv-sparse-checkout.sh ##\n> +@@ t/t7002-mv-sparse-checkout.sh: test_description='git mv in sparse working trees'\n> + \n> + . ./test-lib.sh\n> + \n> ++setup_sparse_checkout () {\n> ++\tmkdir folder1 &&\n> ++\ttouch folder1/file1 &&\n> ++\tgit add folder1 &&\n> ++\tgit sparse-checkout set --cone sub\n> ++}\n> ++\n> ++cleanup_sparse_checkout () {\n> ++\tgit sparse-checkout disable &&\n> ++\tgit reset --hard\n> ++}\n> ++\n> + test_expect_success 'setup' \"\n> + \tmkdir -p sub/dir sub/dir2 &&\n> + \ttouch a b c sub/d sub/dir/e sub/dir2/e &&\n> +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move files to non-sparse dir' '\n> + '\n> + \n> + test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tgit reset --hard &&\n> + \tgit sparse-checkout init --no-cone &&\n> + \tgit sparse-checkout set a !/x y/ !x/y/z &&\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> \ttest_cmp expect stderr\n> '\n> \n> +test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> -+\tgit sparse-checkout disable &&\n> -+\tgit reset --hard &&\n> -+\tmkdir folder1 &&\n> -+\ttouch folder1/file1 &&\n> -+\tgit add folder1 &&\n> -+\tgit sparse-checkout init --cone &&\n> -+\tgit sparse-checkout set sub &&\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> ++\tsetup_sparse_checkout &&\n> +\n> +\ttest_must_fail git mv folder1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone directory with --sparse' '\n> -+\tgit sparse-checkout disable &&\n> -+\tgit reset --hard &&\n> -+\tmkdir folder1 &&\n> -+\ttouch folder1/file1 &&\n> -+\tgit add folder1 &&\n> -+\tgit sparse-checkout init --cone &&\n> -+\tgit sparse-checkout set sub &&\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> ++\tsetup_sparse_checkout &&\n> +\n> +\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n> +'\n> +\n> +test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n> -+\tgit sparse-checkout disable &&\n> -+\tgit reset --hard &&\n> -+\tmkdir folder1 &&\n> -+\ttouch folder1/file1 &&\n> -+\tgit add folder1 &&\n> -+\tgit sparse-checkout init --cone &&\n> -+\tgit sparse-checkout set sub &&\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> ++\tsetup_sparse_checkout &&\n> +\n> +\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n> +\tcat sparse_error_header >expect &&\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n> +'\n> +\n> +test_expect_failure 'can move out-of-cone file with --sparse' '\n> -+\tgit sparse-checkout disable &&\n> -+\tgit reset --hard &&\n> -+\tmkdir folder1 &&\n> -+\ttouch folder1/file1 &&\n> -+\tgit add folder1 &&\n> -+\tgit sparse-checkout init --cone &&\n> -+\tgit sparse-checkout set sub &&\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> ++\tsetup_sparse_checkout &&\n> +\n> +\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n> +'\n> +\n> +test_expect_failure 'refuse to move sparse file to existing destination' '\n> -+\tgit sparse-checkout disable &&\n> -+\tgit reset --hard &&\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\ttouch sub/file1 &&\n> +\tgit add folder1 sub/file1 &&\n> -+\tgit sparse-checkout init --cone &&\n> -+\tgit sparse-checkout set sub &&\n> ++\tgit sparse-checkout set --cone sub &&\n> +\n> +\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n> +\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n> +'\n> +\n> +test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> -+\tgit sparse-checkout disable &&\n> -+\tgit reset --hard &&\n> ++\ttest_when_finished \"cleanup_sparse_checkout\" &&\n> +\tmkdir folder1 &&\n> +\ttouch folder1/file1 &&\n> +\ttouch sub/file1 &&\n> +\techo \"overwrite\" >folder1/file1 &&\n> +\tgit add folder1 sub/file1 &&\n> -+\tgit sparse-checkout init --cone &&\n> -+\tgit sparse-checkout set sub &&\n> ++\tgit sparse-checkout set --cone sub &&\n> +\n> +\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n> +\ttest_must_be_empty stderr &&\n> -: ---------- > 2: 8065fbc232 mv: decouple if/else-if checks using goto\n> 2: 80f485f146 ! 3: e227fe717b mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> \n> \t\tlength = strlen(src);\n> \t\tif (lstat(src, &st) < 0) {\n> -+\t\t\t/*\n> -+\t\t\t * TODO: for now, when you try to overwrite a \n> -+\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n> -+\t\t\t * flag, then the action will be done without providing \"--force\"\n> -+\t\t\t * and no warning.\n> -+\t\t\t *\n> -+\t\t\t * This is mainly because the sparse \n> -+\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n> -+\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n> -+\t\t\t */\n> +-\t\t\t/* only error if existence is expected. */\n> +-\t\t\tif (modes[i] != SPARSE) {\n> ++\t\t\tint pos;\n> ++\t\t\tconst struct cache_entry *ce;\n> +\n> -+\t\t\tint pos = cache_name_pos(src, length);\n> -+\t\t\tif (pos >= 0) {\n> -+\t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n> -+\n> -+\t\t\t\tif (ce_skip_worktree(ce)) {\n> -+\t\t\t\t\tif (!ignore_sparse)\n> -+\t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> -+\t\t\t\t\telse\n> -+\t\t\t\t\t\tmodes[i] = SPARSE;\n> -+\t\t\t\t}\n> -+\t\t\t\telse\n> ++\t\t\tpos = cache_name_pos(src, length);\n> ++\t\t\tif (pos < 0) {\n> ++\t\t\t\t/* only error if existence is expected. */\n> ++\t\t\t\tif (modes[i] != SPARSE)\n> +\t\t\t\t\tbad = _(\"bad source\");\n> ++\t\t\t\tgoto act_on_entry;\n> +\t\t\t}\n> - \t\t\t/* only error if existence is expected. */\n> --\t\t\tif (modes[i] != SPARSE)\n> -+\t\t\telse if (modes[i] != SPARSE)\n> ++\n> ++\t\t\tce = active_cache[pos];\n> ++\t\t\tif (!ce_skip_worktree(ce)) {\n> \t\t\t\tbad = _(\"bad source\");\n> - \t\t} else if (!strncmp(src, dst, length) &&\n> - \t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n> + \t\t\t\tgoto act_on_entry;\n> + \t\t\t}\n> ++\n> ++\t\t\tif (!ignore_sparse)\n> ++\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> ++\t\t\telse\n> ++\t\t\t\tmodes[i] = SPARSE;\n> ++\t\t\tgoto act_on_entry;\n> + \t\t}\n> + \t\tif (!strncmp(src, dst, length) &&\n> + \t\t (dst[length] == 0 || dst[length] == '/')) {\n> \n> ## t/t7002-mv-sparse-checkout.sh ##\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'can move out-of-cone directory with --sparse' '\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'can move out-of-cone directo\n> \n> -test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n> +test_expect_success 'refuse to move out-of-cone file without --sparse' '\n> - \tgit sparse-checkout disable &&\n> - \tgit reset --hard &&\n> - \tmkdir folder1 &&\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tsetup_sparse_checkout &&\n> + \n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n> \ttest_cmp expect stderr\n> '\n> \n> -test_expect_failure 'can move out-of-cone file with --sparse' '\n> +test_expect_success 'can move out-of-cone file with --sparse' '\n> - \tgit sparse-checkout disable &&\n> - \tgit reset --hard &&\n> - \tmkdir folder1 &&\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tsetup_sparse_checkout &&\n> + \n> 3: 04572e5e6b ! 4: d0de7678e3 mv: check if exists in index to handle overwriting\n> @@ Commit message\n> \n> ## builtin/mv.c ##\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \n> - \t\tlength = strlen(src);\n> - \t\tif (lstat(src, &st) < 0) {\n> --\t\t\t/*\n> --\t\t\t * TODO: for now, when you try to overwrite a \n> --\t\t\t * with your as a sparse file, if you supply a \"--sparse\"\n> --\t\t\t * flag, then the action will be done without providing \"--force\"\n> --\t\t\t * and no warning.\n> --\t\t\t *\n> --\t\t\t * This is mainly because the sparse \n> --\t\t\t * is not on-disk, and this if-else chain will be cut off early in\n> --\t\t\t * this check, thus the \"--force\" check is ignored. Need fix.\n> --\t\t\t */\n> - \n> - \t\t\tint pos = cache_name_pos(src, length);\n> - \t\t\tif (pos >= 0) {\n> -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\t\t\tif (ce_skip_worktree(ce)) {\n> - \t\t\t\t\tif (!ignore_sparse)\n> - \t\t\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> --\t\t\t\t\telse\n> --\t\t\t\t\t\tmodes[i] = SPARSE;\n> -+\t\t\t\t\telse {\n> -+\t\t\t\t\t\t/* Check if dst exists in index */\n> -+\t\t\t\t\t\tif (cache_name_pos(dst, strlen(dst)) >= 0) {\n> -+\t\t\t\t\t\t\tif (force)\n> -+\t\t\t\t\t\t\t\tmodes[i] = SPARSE;\n> -+\t\t\t\t\t\t\telse\n> -+\t\t\t\t\t\t\t\tbad = _(\"destination exists\");\n> -+\t\t\t\t\t\t}\n> -+\t\t\t\t\t\telse\n> -+\t\t\t\t\t\t\tmodes[i] = SPARSE;\n> -+\t\t\t\t\t}\n> - \t\t\t\t}\n> - \t\t\t\telse\n> - \t\t\t\t\tbad = _(\"bad source\");\n> + \t\t\t\tbad = _(\"bad source\");\n> + \t\t\t\tgoto act_on_entry;\n> + \t\t\t}\n> +-\n> +-\t\t\tif (!ignore_sparse)\n> ++\t\t\tif (!ignore_sparse) {\n> + \t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n> +-\t\t\telse\n> ++\t\t\t\tgoto act_on_entry;\n> ++\t\t\t}\n> ++\t\t\t/* Check if dst exists in index */\n> ++\t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n> + \t\t\t\tmodes[i] = SPARSE;\n> ++\t\t\t\tgoto act_on_entry;\n> ++\t\t\t}\n> ++\t\t\tif (!force) {\n> ++\t\t\t\tbad = _(\"destination exists\");\n> ++\t\t\t\tgoto act_on_entry;\n> ++\t\t\t}\n> ++\t\t\tmodes[i] = SPARSE;\n> + \t\t\tgoto act_on_entry;\n> + \t\t}\n> + \t\tif (!strncmp(src, dst, length) &&\n> \n> ## t/t7002-mv-sparse-checkout.sh ##\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file with --sparse' '\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file wi\n> \n> -test_expect_failure 'refuse to move sparse file to existing destination' '\n> +test_expect_success 'refuse to move sparse file to existing destination' '\n> - \tgit sparse-checkout disable &&\n> - \tgit reset --hard &&\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> \tmkdir folder1 &&\n> + \ttouch folder1/file1 &&\n> -: ---------- > 5: 70540957b6 mv: use flags mode for update_mode\n> 4: 4eeae40186 ! 6: f8302f64e0 mv: add check_dir_in_index() and solve general dir check issue\n> @@ Commit message\n> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n> this operation can be carried out successfully.\n> \n> - Also, as suggested by Derrick [1],\n> - move the in-line definition of \"enum update_mode\" to the top\n> - of the file and make it use \"flags\" mode (each state is a different\n> - bit in the word).\n> -\n> - [1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n> -\n> Signed-off-by: Shaoxuan Yuan \n> \n> ## builtin/mv.c ##\n> -@@ builtin/mv.c: static const char * const builtin_mv_usage[] = {\n> - \tNULL\n> - };\n> - \n> -+enum update_mode {\n> -+\tBOTH = 0,\n> -+\tWORKING_DIRECTORY = (1 << 1),\n> -+\tINDEX = (1 << 2),\n> -+\tSPARSE = (1 << 3),\n> -+\tSKIP_WORKTREE_DIR = (1 << 4),\n> -+};\n> -+\n> - #define DUP_BASENAME 1\n> - #define KEEP_TRAILING_SLASH 2\n> - \n> @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n> \treturn last - first;\n> }\n> @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n> {\n> \tint i, flags, gitmodules_modified = 0;\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\tOPT_END(),\n> - \t};\n> - \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n> --\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n> -+\tenum update_mode *modes;\n> - \tstruct stat st;\n> - \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n> - \tstruct lock_file lock_file = LOCK_INIT;\n> -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\tif (lstat(src, &st) < 0) {\n> - \n> - \t\t\tint pos = cache_name_pos(src, length);\n> -+\t\t\tconst char *src_w_slash = add_slash(src);\n> -+\n> - \t\t\tif (pos >= 0) {\n> - \t\t\t\tconst struct cache_entry *ce = active_cache[pos];\n> + \t/* Checking */\n> + \tfor (i = 0; i < argc; i++) {\n> + \t\tconst char *src = source[i], *dst = destination[i];\n> +-\t\tint length, src_is_dir;\n> ++\t\tint length;\n> + \t\tconst char *bad = NULL;\n> + \t\tint skip_sparse = 0;\n> \n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\t\t\telse\n> + \n> + \t\t\tpos = cache_name_pos(src, length);\n> + \t\t\tif (pos < 0) {\n> ++\t\t\t\tconst char *src_w_slash = add_slash(src);\n> ++\t\t\t\tif (!check_dir_in_index(src, length) &&\n> ++\t\t\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n> ++\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n> ++\t\t\t\t\tgoto dir_check;\n> ++\t\t\t\t}\n> + \t\t\t\t/* only error if existence is expected. */\n> + \t\t\t\tif (!(modes[i] & SPARSE))\n> \t\t\t\t\tbad = _(\"bad source\");\n> + \t\t\t\tgoto act_on_entry;\n> \t\t\t}\n> -+\t\t\telse if (!check_dir_in_index(src, length) &&\n> -+\t\t\t\t\t !path_in_sparse_checkout(src_w_slash, &the_index)) {\n> -+\t\t\t\tmodes[i] = SKIP_WORKTREE_DIR;\n> -+\t\t\t\tgoto dir_check;\n> -+\t\t\t}\n> - \t\t\t/* only error if existence is expected. */\n> - \t\t\telse if (modes[i] != SPARSE)\n> +-\n> + \t\t\tce = active_cache[pos];\n> + \t\t\tif (!ce_skip_worktree(ce)) {\n> \t\t\t\tbad = _(\"bad source\");\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\t\t\t&& lstat(dst, &st) == 0)\n> + \t\t\tbad = _(\"can not move directory into itself\");\n> + \t\t\tgoto act_on_entry;\n> + \t\t}\n> +-\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n> ++\t\tif (S_ISDIR(st.st_mode)\n> + \t\t && lstat(dst, &st) == 0) {\n> \t\t\tbad = _(\"cannot move directory over file\");\n> - \t\telse if (src_is_dir) {\n> + \t\t\tgoto act_on_entry;\n> + \t\t}\n> +-\t\tif (src_is_dir) {\n> ++\n> ++dir_check:\n> ++\t\tif (S_ISDIR(st.st_mode)) {\n> + \t\t\tint j, dst_len, n;\n> -\t\t\tint first = cache_name_pos(src, length), last;\n> +\t\t\tint first, last;\n> -+dir_check:\n> +\t\t\tfirst = cache_name_pos(src, length);\n> \n> - \t\t\tif (first >= 0)\n> + \t\t\tif (first >= 0) {\n> \t\t\t\tprepare_move_submodule(src, first,\n> -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\t\telse { /* last - first >= 1 */\n> - \t\t\t\tint j, dst_len, n;\n> - \n> --\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n> -+\t\t\t\tif (!modes[i])\n> -+\t\t\t\t\tmodes[i] |= WORKING_DIRECTORY;\n> - \t\t\t\tn = argc + last - first;\n> - \t\t\t\tREALLOC_ARRAY(source, n);\n> - \t\t\t\tREALLOC_ARRAY(destination, n);\n> -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> - \t\tif (show_only)\n> - \t\t\tcontinue;\n> --\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n> -+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n> -+\t\t \trename(src, dst) < 0) {\n> - \t\t\tif (ignore_errors)\n> - \t\t\t\tcontinue;\n> - \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n> -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - \t\t\t\t\t\t\t 1);\n> - \t\t}\n> - \n> --\t\tif (mode == WORKING_DIRECTORY)\n> -+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n> - \t\t\tcontinue;\n> - \n> - \t\tpos = cache_name_pos(src, strlen(src));\n> \n> ## t/t7002-mv-sparse-checkout.sh ##\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n> \n> -test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> +test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n> - \tgit sparse-checkout disable &&\n> - \tgit reset --hard &&\n> - \tmkdir folder1 &&\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tsetup_sparse_checkout &&\n> + \n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n> \ttest_cmp expect stderr\n> '\n> \n> -test_expect_failure 'can move out-of-cone directory with --sparse' '\n> +test_expect_success 'can move out-of-cone directory with --sparse' '\n> - \tgit sparse-checkout disable &&\n> - \tgit reset --hard &&\n> - \tmkdir folder1 &&\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tsetup_sparse_checkout &&\n> + \n> 5: a3a296c3ef ! 7: bc996931e2 mv: use update_sparsity() after touching sparse contents\n> @@ Metadata\n> Author: Shaoxuan Yuan \n> \n> ## Commit message ##\n> - mv: use update_sparsity() after touching sparse contents\n> + mv: update sparsity after moving from out-of-cone to in-cone\n> \n> - Originally, \"git mv\" a sparse file/directory from out/in-cone to\n> - in/out-cone does not update the sparsity following the sparse-checkout\n> - patterns.\n> + Originally, \"git mv\" a sparse file from out-of-cone to\n> + in-cone does not update the moved file's sparsity (remove its\n> + SKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\n> + not checked out in the working tree.\n> \n> - Use update_sparsity() after touching sparse contents, so the sparsity\n> - will be updated after the move.\n> + Update the behavior so that:\n> + 1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n> + corresponding cache entry.\n> + 2. The moved cache entry is checked out in the working tree to reflect\n> + the updated sparsity.\n> \n> Signed-off-by: Shaoxuan Yuan \n> \n> @@ builtin/mv.c\n> #include \"string-list.h\"\n> #include \"parse-options.h\"\n> #include \"submodule.h\"\n> -+#include \"unpack-trees.h\"\n> ++#include \"entry.h\"\n> \n> static const char * const builtin_mv_usage[] = {\n> \tN_(\"git mv [] ... \"),\n> -@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> - {\n> - \tint i, flags, gitmodules_modified = 0;\n> - \tint verbose = 0, show_only = 0, force = 0, ignore_errors = 0, ignore_sparse = 0;\n> -+\tint sparse_moved = 0;\n> - \tstruct option builtin_mv_options[] = {\n> - \t\tOPT__VERBOSE(&verbose, N_(\"be verbose\")),\n> - \t\tOPT__DRY_RUN(&show_only, N_(\"dry run\")),\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> \t\tconst char *src = source[i], *dst = destination[i];\n> \t\tenum update_mode mode = modes[i];\n> \t\tint pos;\n> -+\t\tif (!sparse_moved && mode & (SPARSE | SKIP_WORKTREE_DIR))\n> -+\t\t\tsparse_moved = 1;\n> ++\t\tstruct checkout state = CHECKOUT_INIT;\n> ++\t\tstate.istate = &the_index;\n> ++\n> ++\t\tif (force)\n> ++\t\t\tstate.force = 1;\n> \t\tif (show_only || verbose)\n> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n> \t\tif (show_only)\n> @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n> + \t\tpos = cache_name_pos(src, strlen(src));\n> + \t\tassert(pos >= 0);\n> \t\trename_cache_entry_at(pos, dst);\n> ++\n> ++\t\tif (mode & SPARSE) {\n> ++\t\t\tif (path_in_sparse_checkout(dst, &the_index)) {\n> ++\t\t\t\tint dst_pos;\n> ++\n> ++\t\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n> ++\t\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n> ++\n> ++\t\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n> ++\t\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n> ++\t\t\t}\n> ++\t\t}\n> \t}\n> \n> -+\tif (sparse_moved) {\n> -+\t\tstruct unpack_trees_options o;\n> -+\t\tmemset(&o, 0, sizeof(o));\n> -+\t\to.verbose_update = isatty(2);\n> -+\t\to.update = 1;\n> -+\t\to.head_idx = -1;\n> -+\t\to.src_index = &the_index;\n> -+\t\to.dst_index = &the_index;\n> -+\t\to.skip_sparse_checkout = 0;\n> -+\t\to.pl = the_index.sparse_checkout_patterns;\n> -+\t\tsetup_unpack_trees_porcelain(&o, \"mv\");\n> -+\t\tupdate_sparsity(&o);\n> -+\t\tclear_unpack_trees_porcelain(&o);\n> -+\t}\n> -+\n> \tif (gitmodules_modified)\n> - \t\tstage_updated_gitmodules(&the_index);\n> - \n> \n> ## t/t7002-mv-sparse-checkout.sh ##\n> +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone directory with --sparse' '\n> + \tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n> + \ttest_must_be_empty stderr &&\n> + \n> +-\tgit sparse-checkout reapply &&\n> + \ttest_path_is_dir sub/folder1 &&\n> + \ttest_path_is_file sub/folder1/file1\n> + '\n> +@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file with --sparse' '\n> + \tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n> + \ttest_must_be_empty stderr &&\n> + \n> +-\tgit sparse-checkout reapply &&\n> + \t! test_path_is_dir sub/folder1 &&\n> + \ttest_path_is_file sub/file1\n> + '\n> @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move sparse file to existing destination' '\n> \ttest_cmp expect stderr\n> '\n> \n> -+# Need fix.\n> -+#\n> -+# The *expected* behavior:\n> -+#\n> -+# Using --sparse to accept a sparse file, --force to overwrite the destination.\n> -+# The folder1/file1 should replace the sub/file1 without error.\n> -+#\n> -+# The *actual* behavior:\n> -+#\n> -+# It emits a warning:\n> -+#\n> -+# warning: Path ' sub/file1\n> -+# ' already present; will not overwrite with sparse update.\n> -+# After fixing the above paths, you may want to run `git sparse-checkout\n> -+# reapply`.\n> -+\n> - test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> - \tgit sparse-checkout disable &&\n> - \tgit reset --hard &&\n> +-test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n> ++test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n> + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n> + \tmkdir folder1 &&\n> + \ttouch folder1/file1 &&\n> \n> base-commit: 4f6db706e6ad145a9bf6b26a1ca0970bed27bb72\n\n"},{"id":"457688","messageId":"94540a13-2f6d-8a92-6426-29df8284f2a7@gmail.com","threadId":"57642","inReplyTo":"01b39c63-5652-4293-0424-ff99b6f9f7d2@github.com","subject":"Re: [WIP v3 5/7] mv: use flags mode for update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-22T09:37:34Z","receivedAt":"2022-06-22T09:37:48Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\nOn 6/22/2022 6:32 AM, Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> As suggested by Derrick [1],\n>> move the in-line definition of \"enum update_mode\" to the top\n>> of the file and make it use \"flags\" mode (each state is a different\n>> bit in the word).\n>>\n> This message doesn't quite cover all of what's done in the commit. In\n> addition to moving the enum definition, you introduce a 'SKIP_WORKTREE_DIR'\n> flag and change the flag assignments to '|=' (additive) from '=' (single\n> assignment). If those changes belong in this commit (not a later one), they\n> should be explained in the message here.\n\n\nSure! It makes the commit sound clearer.\n\n\n>> [1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n>>\n>> Signed-off-by: Shaoxuan Yuan \n>> ---\n>> builtin/mv.c | 26 ++++++++++++++++++--------\n>> 1 file changed, 18 insertions(+), 8 deletions(-)\n>>\n>> diff --git a/builtin/mv.c b/builtin/mv.c\n>> index abb90d3266..7ce7992d6c 100644\n>> --- a/builtin/mv.c\n>> +++ b/builtin/mv.c\n>> @@ -19,6 +19,14 @@ static const char * const builtin_mv_usage[] = {\n>> \tNULL\n>> };\n>> \n>> +enum update_mode {\n>> +\tBOTH = 0,\n> I know this comes from the original inline enum, but I don't see 'BOTH' used\n> anywhere. The name itself is somewhat confusing (I have no idea what \"both\"\n> is referring to - possibly \"both\" 'WORKING_DIRECTORY' and 'INDEX'??), so\n> would you mind removing it in the next re-roll?\n\n\nYep, this BOTH is confusing.\n\nI think it could be changed to something like UNDECIDED?\n\nIf taking it as UNDECIDED, it could potentially help determine if an \nargument\n\nis untouched throughout the checking, then we can give the argument a \nREGULAR\n\nflag (this may serve a purpose in my planned in-cone to out-of-cone move).\n\n\n>> +\tWORKING_DIRECTORY = (1 << 1),\n>> +\tINDEX = (1 << 2),\n>> +\tSPARSE = (1 << 3),\n>> +\tSKIP_WORKTREE_DIR = (1 << 4),\n> You're not introducing any assignment of 'SKIP_WORKTREE_DIR' in this commit\n> (looks like that's done in the next one, patch [6/7]), so you should\n> probably 'SKIP_WORKTREE_DIR' and its corresponding usage in that patch\n> instead of this one.\n\n\nRight. SKIP_WORKTREE_DIR should go to a different patch.\n\n\n>> +};\n> When the update modes were mutually-exclusive, it made sense for them to be\n> represented by an enum. Now that they're flags that can be combined, should\n> they instead be pre-processor '#define' values (e.g., like the 'RESET_*'\n> modes in 'reset.h' or 'CE_*' flags in 'cache.h')? I don't actually know what\n> the standard is, since I also see one or two examples of using enums as\n> flags (e.g., 'commit_graph_split_flags' in 'commit-graph.h'). Maybe another\n> contributor could clarify?\n\n\nI'm not sure either. Though a enum groups the flags together, which \nseems nice to me\n\nas the flags are naturally distinguished from other '#define's.\n\n\n>> +\n>> #define DUP_BASENAME 1\n>> #define KEEP_TRAILING_SLASH 2\n>> \n>> @@ -129,7 +137,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\tOPT_END(),\n>> \t};\n>> \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n>> -\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n>> +\tenum update_mode *modes;\n>> \tstruct stat st;\n>> \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n>> \tstruct lock_file lock_file = LOCK_INIT;\n>> @@ -191,7 +199,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\t\tpos = cache_name_pos(src, length);\n>> \t\t\tif (pos < 0) {\n>> \t\t\t\t/* only error if existence is expected. */\n>> -\t\t\t\tif (modes[i] != SPARSE)\n>> +\t\t\t\tif (!(modes[i] & SPARSE))\n>> \t\t\t\t\tbad = _(\"bad source\");\n>> \t\t\t\tgoto act_on_entry;\n>> \t\t\t}\n>> @@ -207,14 +215,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\t\t}\n>> \t\t\t/* Check if dst exists in index */\n>> \t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n>> -\t\t\t\tmodes[i] = SPARSE;\n>> +\t\t\t\tmodes[i] |= SPARSE;\n>> \t\t\t\tgoto act_on_entry;\n>> \t\t\t}\n>> \t\t\tif (!force) {\n>> \t\t\t\tbad = _(\"destination exists\");\n>> \t\t\t\tgoto act_on_entry;\n>> \t\t\t}\n>> -\t\t\tmodes[i] = SPARSE;\n>> +\t\t\tmodes[i] |= SPARSE;\n>> \t\t\tgoto act_on_entry;\n>> \t\t}\n>> \t\tif (!strncmp(src, dst, length) &&\n>> @@ -242,7 +250,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\t\t}\n>> \n>> \t\t\t/* last - first >= 1 */\n>> -\t\t\tmodes[i] = WORKING_DIRECTORY;\n>> +\t\t\tmodes[i] |= WORKING_DIRECTORY;\n>> \t\t\tn = argc + last - first;\n>> \t\t\tREALLOC_ARRAY(source, n);\n>> \t\t\tREALLOC_ARRAY(destination, n);\n>> @@ -258,7 +266,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\t\t\tsource[argc + j] = path;\n>> \t\t\t\tdestination[argc + j] =\n>> \t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n>> -\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n>> +\t\t\t\tmemset(modes + argc + j, 0, sizeof(enum update_mode));\n> One benefit of using '#define' values would be that 'modes' would just be an\n> array of unsigned ints, so you could just assign '0' rather than using\n> memset. In terms of the implementation as-is, though, I think what you have\n> is correct.\n>\n>> +\t\t\t\tmodes[argc + j] |= ce_skip_worktree(ce) ? SPARSE : INDEX;\n>> \t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n>> \t\t\t}\n>> \t\t\targc += last - first;\n>> @@ -355,7 +364,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n>> \t\tif (show_only)\n>> \t\t\tcontinue;\n>> -\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n>> +\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n>> +\t\t\trename(src, dst) < 0) {\n> Nit: could you align 'rename' with the line above it (per the highlighted\n> section in the CodingGuidelines [1])? As far as I can tell, the \"align with\n> tabs and spaces\" approach is what's *intended* to be used in 'mv.c'\n> (although it's admittedly pretty inconsistent).\n>\n> [1] https://github.com/git/git/blob/master/Documentation/CodingGuidelines#L371-L383\n\n\nSure. My 'format-patch' always gives me this issue, will check.\n\n\n>> \t\t\tif (ignore_errors)\n>> \t\t\t\tcontinue;\n>> \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n>> @@ -369,7 +379,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n>> \t\t\t\t\t\t\t 1);\n>> \t\t}\n>> \n>> -\t\tif (mode == WORKING_DIRECTORY)\n>> +\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n>> \t\t\tcontinue;\n>> \n>> \t\tpos = cache_name_pos(src, strlen(src));\n"},{"id":"457774","messageId":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:13Z","receivedAt":"2022-06-23T11:41:47Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"## Changes since WIP v3 ##\n\n1. Fix style-nits.\n\n2. Move \"mv: update sparsity after moving from out-of-cone to in-cone\" to the \n (2/7) position (was (7/7) position in WIP v3). Make this move so that we\n can drop \"git sparse-checkout reapply\" from the tests, as suggested by\n Victoria [1]. The reason is we need to check out the moved cache_entry\n for all out-of-cone to in-cone moves, so this commit works better if being the\n first one.\n\n3. Fix the commit message of \"mv: use flags mode for update_mode\", as suggested\n here [2].\n\n4. Add \"Helped-by\" and \"Suggested-by\" trailers.\n\n5. In \"mv: add check_dir_in_index() and solve general dir check issue\", change\n the 'check_dir_in_index()' to no more accept 'namelen' as argument. The\n original \"namelen + 1\" logic can be erroneous, for example when 'name' already\n has a trailing slash. So just use strlen() to save the trouble.\n\n## Notes ##\n\nAs discussed in WIP v3, we can postpone the \"in-cone to out-of-cone\" move\nto a later series. Hence, this series' functionality part has been done,\nremove the WIP prefix.\n\n[1] https://lore.kernel.org/git/adb795ba-56ce-8441-0c38-a3e6b0a6e861@github.com/\n[2] https://lore.kernel.org/git/01b39c63-5652-4293-0424-ff99b6f9f7d2@github.com/\n\nShaoxuan Yuan (7):\n t7002: add tests for moving out-of-cone file/directory\n mv: update sparsity after moving from out-of-cone to in-cone\n mv: decouple if/else-if checks using goto\n mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n mv: check if exists in index to handle overwriting\n mv: use flags mode for update_mode\n mv: add check_dir_in_index() and solve general dir check issue\n\n builtin/mv.c | 239 +++++++++++++++++++++++++---------\n t/t7002-mv-sparse-checkout.sh | 84 ++++++++++++\n 2 files changed, 259 insertions(+), 64 deletions(-)\n\nRange-diff against v3:\n1: 574cbdfdb4 ! 1: 90c38f479b t7002: add tests for moving out-of-cone file/directory\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n +\tsetup_sparse_checkout &&\n +\n -+\tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n ++\tgit mv --sparse folder1 sub 2>stderr &&\n +\ttest_must_be_empty stderr &&\n +\n -+\tgit sparse-checkout reapply &&\n +\ttest_path_is_dir sub/folder1 &&\n +\ttest_path_is_file sub/folder1/file1\n +'\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-s\n +\ttest_when_finished \"cleanup_sparse_checkout\" &&\n +\tsetup_sparse_checkout &&\n +\n -+\tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n ++\tgit mv --sparse folder1/file1 sub 2>stderr &&\n +\ttest_must_be_empty stderr &&\n +\n -+\tgit sparse-checkout reapply &&\n -+\t! test_path_is_dir sub/folder1 &&\n +\ttest_path_is_file sub/file1\n +'\n +\n7: 6fa630203b ! 2: c6fcaf8313 mv: update sparsity after moving from out-of-cone to in-cone\n @@ Commit message\n 2. The moved cache entry is checked out in the working tree to reflect\n the updated sparsity.\n \n + Helped-by: Victoria Dye \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tassert(pos >= 0);\n \t\trename_cache_entry_at(pos, dst);\n +\n -+\t\tif (mode & SPARSE) {\n -+\t\t\tif (path_in_sparse_checkout(dst, &the_index)) {\n -+\t\t\t\tint dst_pos;\n ++\t\tif ((mode & SPARSE) &&\n ++\t\t (path_in_sparse_checkout(dst, &the_index))) {\n ++\t\t\tint dst_pos;\n +\n -+\t\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n -+\t\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n ++\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n ++\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n +\n -+\t\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n -+\t\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n -+\t\t\t}\n ++\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n ++\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n +\t\t}\n \t}\n \n \tif (gitmodules_modified)\n -\n - ## t/t7002-mv-sparse-checkout.sh ##\n -@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone directory with --sparse' '\n - \tgit mv --sparse folder1 sub 1>actual 2>stderr &&\n - \ttest_must_be_empty stderr &&\n - \n --\tgit sparse-checkout reapply &&\n - \ttest_path_is_dir sub/folder1 &&\n - \ttest_path_is_file sub/folder1/file1\n - '\n -@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file with --sparse' '\n - \tgit mv --sparse folder1/file1 sub 1>actual 2>stderr &&\n - \ttest_must_be_empty stderr &&\n - \n --\tgit sparse-checkout reapply &&\n - \t! test_path_is_dir sub/folder1 &&\n - \ttest_path_is_file sub/file1\n - '\n -@@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move sparse file to existing destination' '\n - \ttest_cmp expect stderr\n - '\n - \n --test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n -+test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n - \ttest_when_finished \"cleanup_sparse_checkout\" &&\n - \tmkdir folder1 &&\n - \ttouch folder1/file1 &&\n2: aac267091b ! 3: aee896a3f2 mv: decouple if/else-if checks using goto\n @@ Commit message\n Refactor to decouple this if/else-if chain by using goto to jump ahead.\n \n Suggested-by: Derrick Stolee \n + Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n3: b0e90cfa01 = 4: 82c10486ec mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n4: c0c9dc9575 ! 5: 59dcf1a55f mv: check if exists in index to handle overwriting\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'can move out-of-cone file wi\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n +@@ t/t7002-mv-sparse-checkout.sh: test_expect_failure 'refuse to move sparse file to existing destination' '\n + \ttest_cmp expect stderr\n + '\n + \n +-test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n ++test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n + \ttest_when_finished \"cleanup_sparse_checkout\" &&\n + \tmkdir folder1 &&\n + \ttouch folder1/file1 &&\n5: 3088724e72 ! 6: b8e3094178 mv: use flags mode for update_mode\n @@ Commit message\n of the file and make it use \"flags\" mode (each state is a different\n bit in the word).\n \n + Change the flag assignments from '=' (single assignment) to '|='\n + (additive). Also change flag evaluation from '==' to '&', etc.\n +\n [1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n \n + Helped-by: Victoria Dye \n + Helped-by: Derrick Stolee \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n @@ builtin/mv.c: static const char * const builtin_mv_usage[] = {\n +\tWORKING_DIRECTORY = (1 << 1),\n +\tINDEX = (1 << 2),\n +\tSPARSE = (1 << 3),\n -+\tSKIP_WORKTREE_DIR = (1 << 4),\n +};\n +\n #define DUP_BASENAME 1\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tif (show_only)\n \t\t\tcontinue;\n -\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n -+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n -+\t\t\trename(src, dst) < 0) {\n ++\t\tif (!(mode & (INDEX | SPARSE)) &&\n ++\t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t}\n \n -\t\tif (mode == WORKING_DIRECTORY)\n -+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n ++\t\tif (mode & (WORKING_DIRECTORY))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\n6: 0c7ba28ddc ! 7: 080f34ee01 mv: add check_dir_in_index() and solve general dir check issue\n @@ Commit message\n instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n this operation can be carried out successfully.\n \n + Helped-by: Victoria Dye \n + Helped-by: Derrick Stolee \n Signed-off-by: Shaoxuan Yuan \n \n ## builtin/mv.c ##\n +@@ builtin/mv.c: enum update_mode {\n + \tWORKING_DIRECTORY = (1 << 1),\n + \tINDEX = (1 << 2),\n + \tSPARSE = (1 << 3),\n ++\tSKIP_WORKTREE_DIR = (1 << 4),\n + };\n + \n + #define DUP_BASENAME 1\n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n + * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n + * Return 1 otherwise.\n + */\n -+static int check_dir_in_index(const char *name, int namelen)\n ++static int check_dir_in_index(const char *name)\n +{\n -+\tint ret = 1;\n +\tconst char *with_slash = add_slash(name);\n -+\tint length = namelen + 1;\n ++\tint length = strlen(with_slash);\n +\n +\tint pos = cache_name_pos(with_slash, length);\n +\tconst struct cache_entry *ce;\n @@ builtin/mv.c: static int index_range_of_same_dir(const char *src, int length,\n +\tif (pos < 0) {\n +\t\tpos = -pos - 1;\n +\t\tif (pos >= the_index.cache_nr)\n -+\t\t\treturn ret;\n ++\t\t\treturn 1;\n +\t\tce = active_cache[pos];\n +\t\tif (strncmp(with_slash, ce->name, length))\n -+\t\t\treturn ret;\n ++\t\t\treturn 1;\n +\t\tif (ce_skip_worktree(ce))\n -+\t\t\treturn ret = 0;\n ++\t\t\treturn 0;\n +\t}\n -+\treturn ret;\n ++\treturn 1;\n +}\n +\n int cmd_mv(int argc, const char **argv, const char *prefix)\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n +\t\t\t\tconst char *src_w_slash = add_slash(src);\n -+\t\t\t\tif (!check_dir_in_index(src, length) &&\n -+\t\t\t\t\t!path_in_sparse_checkout(src_w_slash, &the_index)) {\n ++\t\t\t\tif (!path_in_sparse_checkout(src_w_slash, &the_index) &&\n ++\t\t\t\t !check_dir_in_index(src)) {\n +\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n +\t\t\t\t\tgoto dir_check;\n +\t\t\t\t}\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n +dir_check:\n +\t\tif (S_ISDIR(st.st_mode)) {\n \t\t\tint j, dst_len, n;\n --\t\t\tint first = cache_name_pos(src, length), last;\n -+\t\t\tint first, last;\n -+\t\t\tfirst = cache_name_pos(src, length);\n + \t\t\tint first = cache_name_pos(src, length), last;\n + \n +@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n + \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n + \t\tif (show_only)\n + \t\t\tcontinue;\n +-\t\tif (!(mode & (INDEX | SPARSE)) &&\n ++\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n + \t\t rename(src, dst) < 0) {\n + \t\t\tif (ignore_errors)\n + \t\t\t\tcontinue;\n +@@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n + \t\t\t\t\t\t\t 1);\n + \t\t}\n + \n +-\t\tif (mode & (WORKING_DIRECTORY))\n ++\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n + \t\t\tcontinue;\n \n - \t\t\tif (first >= 0) {\n - \t\t\t\tprepare_move_submodule(src, first,\n + \t\tpos = cache_name_pos(src, strlen(src));\n \n ## t/t7002-mv-sparse-checkout.sh ##\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n\nbase-commit: ddbc07872e86265dc30aefa5f4d881f762120044\n-- \n2.35.1\n\n"},{"id":"457775","messageId":"20220623114120.12768-2-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 1/7] t7002: add tests for moving out-of-cone file/directory","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:14Z","receivedAt":"2022-06-23T11:41:56Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test following situations:\n\nWe do not have sufficient coverage of moving files outside\nof a sparse-checkout cone. Create new tests covering this\nbehavior, keeping in mind that the user can include --sparse\n(or not), move a file or directory, and the destination can\nalready exist in the index (in this case user can use --force\nto overwrite existing entry).\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 84 +++++++++++++++++++++++++++++++++++\n 1 file changed, 84 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex f0f7cbfcdb..023e657c9e 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -4,6 +4,18 @@ test_description='git mv in sparse working trees'\n \n . ./test-lib.sh\n \n+setup_sparse_checkout () {\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout set --cone sub\n+}\n+\n+cleanup_sparse_checkout () {\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard\n+}\n+\n test_expect_success 'setup' \"\n \tmkdir -p sub/dir sub/dir2 &&\n \ttouch a b c sub/d sub/dir/e sub/dir2/e &&\n@@ -196,6 +208,7 @@ test_expect_success 'can move files to non-sparse dir' '\n '\n \n test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tgit reset --hard &&\n \tgit sparse-checkout init --no-cone &&\n \tgit sparse-checkout set a !/x y/ !x/y/z &&\n@@ -206,4 +219,75 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n+test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv folder1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone directory with --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\tgit mv --sparse folder1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/folder1/file1\n+'\n+\n+test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone file with --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\tgit mv --sparse folder1/file1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_file sub/file1\n+'\n+\n+test_expect_failure 'refuse to move sparse file to existing destination' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n+\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\techo \"overwrite\" >folder1/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\techo \"overwrite\" >expect &&\n+\ttest_cmp expect sub/file1\n+'\n+\n test_done\n-- \n2.35.1\n\n"},{"id":"457776","messageId":"20220623114120.12768-3-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 2/7] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:15Z","receivedAt":"2022-06-23T11:41:57Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, \"git mv\" a sparse file from out-of-cone to\nin-cone does not update the moved file's sparsity (remove its\nSKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\nnot checked out in the working tree.\n\nUpdate the behavior so that:\n1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n corresponding cache entry.\n2. The moved cache entry is checked out in the working tree to reflect\n the updated sparsity.\n\nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba83..0c0c2b4914 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -13,6 +13,7 @@\n #include \"string-list.h\"\n #include \"parse-options.h\"\n #include \"submodule.h\"\n+#include \"entry.h\"\n \n static const char * const builtin_mv_usage[] = {\n \tN_(\"git mv [] ... \"),\n@@ -304,6 +305,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tstruct checkout state = CHECKOUT_INIT;\n+\t\tstate.istate = &the_index;\n+\n+\t\tif (force)\n+\t\t\tstate.force = 1;\n \t\tif (show_only || verbose)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n@@ -328,6 +334,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n \t\trename_cache_entry_at(pos, dst);\n+\n+\t\tif ((mode & SPARSE) &&\n+\t\t (path_in_sparse_checkout(dst, &the_index))) {\n+\t\t\tint dst_pos;\n+\n+\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n+\n+\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n+\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n+\t\t}\n \t}\n \n \tif (gitmodules_modified)\n-- \n2.35.1\n\n"},{"id":"457777","messageId":"20220623114120.12768-4-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 3/7] mv: decouple if/else-if checks using goto","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:16Z","receivedAt":"2022-06-23T11:41:59Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Previous if/else-if chain are highly nested and hard to develop/extend.\n\nRefactor to decouple this if/else-if chain by using goto to jump ahead.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 139 +++++++++++++++++++++++++++++----------------------\n 1 file changed, 80 insertions(+), 59 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 0c0c2b4914..7f8d658028 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -187,53 +187,68 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\tif (modes[i] != SPARSE) {\n \t\t\t\tbad = _(\"bad source\");\n-\t\t} else if (!strncmp(src, dst, length) &&\n-\t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n+\t\tif (!strncmp(src, dst, length) &&\n+\t\t (dst[length] == 0 || dst[length] == '/')) {\n \t\t\tbad = _(\"can not move directory into itself\");\n-\t\t} else if ((src_is_dir = S_ISDIR(st.st_mode))\n-\t\t\t\t&& lstat(dst, &st) == 0)\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n-\t\telse if (src_is_dir) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (src_is_dir) {\n+\t\t\tint j, dst_len, n;\n \t\t\tint first = cache_name_pos(src, length), last;\n \n-\t\t\tif (first >= 0)\n+\t\t\tif (first >= 0) {\n \t\t\t\tprepare_move_submodule(src, first,\n \t\t\t\t\t\t submodule_gitfile + i);\n-\t\t\telse if (index_range_of_same_dir(src, length,\n-\t\t\t\t\t\t\t &first, &last) < 1)\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t} else if (index_range_of_same_dir(src, length,\n+\t\t\t\t\t\t\t &first, &last) < 1) {\n \t\t\t\tbad = _(\"source directory is empty\");\n-\t\t\telse { /* last - first >= 1 */\n-\t\t\t\tint j, dst_len, n;\n-\n-\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n-\t\t\t\tn = argc + last - first;\n-\t\t\t\tREALLOC_ARRAY(source, n);\n-\t\t\t\tREALLOC_ARRAY(destination, n);\n-\t\t\t\tREALLOC_ARRAY(modes, n);\n-\t\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n-\n-\t\t\t\tdst = add_slash(dst);\n-\t\t\t\tdst_len = strlen(dst);\n-\n-\t\t\t\tfor (j = 0; j < last - first; j++) {\n-\t\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n-\t\t\t\t\tconst char *path = ce->name;\n-\t\t\t\t\tsource[argc + j] = path;\n-\t\t\t\t\tdestination[argc + j] =\n-\t\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n-\t\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n-\t\t\t\t}\n-\t\t\t\targc += last - first;\n+\t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t} else if (!(ce = cache_file_exists(src, length, 0))) {\n+\n+\t\t\t/* last - first >= 1 */\n+\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tn = argc + last - first;\n+\t\t\tREALLOC_ARRAY(source, n);\n+\t\t\tREALLOC_ARRAY(destination, n);\n+\t\t\tREALLOC_ARRAY(modes, n);\n+\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n+\n+\t\t\tdst = add_slash(dst);\n+\t\t\tdst_len = strlen(dst);\n+\n+\t\t\tfor (j = 0; j < last - first; j++) {\n+\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n+\t\t\t\tconst char *path = ce->name;\n+\t\t\t\tsource[argc + j] = path;\n+\t\t\t\tdestination[argc + j] =\n+\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n+\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n+\t\t\t}\n+\t\t\targc += last - first;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (!(ce = cache_file_exists(src, length, 0))) {\n \t\t\tbad = _(\"not under version control\");\n-\t\t} else if (ce_stage(ce)) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (ce_stage(ce)) {\n \t\t\tbad = _(\"conflicted\");\n-\t\t} else if (lstat(dst, &st) == 0 &&\n-\t\t\t (!ignore_case || strcasecmp(src, dst))) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (lstat(dst, &st) == 0 &&\n+\t\t (!ignore_case || strcasecmp(src, dst))) {\n \t\t\tbad = _(\"destination exists\");\n \t\t\tif (force) {\n \t\t\t\t/*\n@@ -247,34 +262,40 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t} else\n \t\t\t\t\tbad = _(\"Cannot overwrite\");\n \t\t\t}\n-\t\t} else if (string_list_has_string(&src_for_dst, dst))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (string_list_has_string(&src_for_dst, dst)) {\n \t\t\tbad = _(\"multiple sources for the same target\");\n-\t\telse if (is_dir_sep(dst[strlen(dst) - 1]))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (is_dir_sep(dst[strlen(dst) - 1])) {\n \t\t\tbad = _(\"destination directory does not exist\");\n-\t\telse {\n-\t\t\t/*\n-\t\t\t * We check if the paths are in the sparse-checkout\n-\t\t\t * definition as a very final check, since that\n-\t\t\t * allows us to point the user to the --sparse\n-\t\t\t * option as a way to have a successful run.\n-\t\t\t */\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(src, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\n-\t\t\tif (skip_sparse)\n-\t\t\t\tgoto remove_entry;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n \n-\t\t\tstring_list_insert(&src_for_dst, dst);\n+\t\t/*\n+\t\t * We check if the paths are in the sparse-checkout\n+\t\t * definition as a very final check, since that\n+\t\t * allows us to point the user to the --sparse\n+\t\t * option as a way to have a successful run.\n+\t\t */\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(src, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\tskip_sparse = 1;\n+\t\t}\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n+\t\t\tskip_sparse = 1;\n \t\t}\n \n+\t\tif (skip_sparse)\n+\t\t\tgoto remove_entry;\n+\n+\t\tstring_list_insert(&src_for_dst, dst);\n+\n+act_on_entry:\n \t\tif (!bad)\n \t\t\tcontinue;\n \t\tif (!ignore_errors)\n-- \n2.35.1\n\n"},{"id":"457778","messageId":"20220623114120.12768-5-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 4/7] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:17Z","receivedAt":"2022-06-23T11:42:01Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a file which is not on-disk but exists in\nindex as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\nout with \"bad source\".\n\nChange the checking logic, so that such \nfile makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 21 +++++++++++++++++++--\n t/t7002-mv-sparse-checkout.sh | 4 ++--\n 2 files changed, 21 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 7f8d658028..d1b3229be6 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -186,11 +186,28 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n-\t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE) {\n+\t\t\tint pos;\n+\t\t\tconst struct cache_entry *ce;\n+\n+\t\t\tpos = cache_name_pos(src, length);\n+\t\t\tif (pos < 0) {\n+\t\t\t\t/* only error if existence is expected. */\n+\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\t\tbad = _(\"bad source\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\n+\t\t\tce = active_cache[pos];\n+\t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n+\n+\t\t\tif (!ignore_sparse)\n+\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\telse\n+\t\t\t\tmodes[i] = SPARSE;\n+\t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n \t\t (dst[length] == 0 || dst[length] == '/')) {\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 023e657c9e..1510b5ed6a 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -241,7 +241,7 @@ test_expect_failure 'can move out-of-cone directory with --sparse' '\n \ttest_path_is_file sub/folder1/file1\n '\n \n-test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+test_expect_success 'refuse to move out-of-cone file without --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -252,7 +252,7 @@ test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone file with --sparse' '\n+test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n-- \n2.35.1\n\n"},{"id":"457779","messageId":"20220623114120.12768-6-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 5/7] mv: check if exists in index to handle overwriting","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:18Z","receivedAt":"2022-06-23T11:42:07Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a sparse file into cone can result in unwarned\noverwrite of existing entry. The expected behavior is that if the\n exists in the entry, user should be prompted to supply\na [-f|--force] to carry out the operation, or the operation should\nfail.\n\nAdd a check mechanism to do that.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 15 ++++++++++++---\n t/t7002-mv-sparse-checkout.sh | 4 ++--\n 2 files changed, 14 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex d1b3229be6..40a3a5c5ff 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -202,11 +202,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\n-\t\t\tif (!ignore_sparse)\n+\t\t\tif (!ignore_sparse) {\n \t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\telse\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\t/* Check if dst exists in index */\n+\t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n \t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\tif (!force) {\n+\t\t\t\tbad = _(\"destination exists\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\tmodes[i] = SPARSE;\n \t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 1510b5ed6a..6d2fb4f8d2 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -262,7 +262,7 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_path_is_file sub/file1\n '\n \n-test_expect_failure 'refuse to move sparse file to existing destination' '\n+test_expect_success 'refuse to move sparse file to existing destination' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n@@ -275,7 +275,7 @@ test_expect_failure 'refuse to move sparse file to existing destination' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n-- \n2.35.1\n\n"},{"id":"457780","messageId":"20220623114120.12768-7-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 6/7] mv: use flags mode for update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:19Z","receivedAt":"2022-06-23T11:42:09Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"As suggested by Derrick [1],\nmove the in-line definition of \"enum update_mode\" to the top\nof the file and make it use \"flags\" mode (each state is a different\nbit in the word).\n\nChange the flag assignments from '=' (single assignment) to '|='\n(additive). Also change flag evaluation from '==' to '&', etc.\n\n[1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 25 +++++++++++++++++--------\n 1 file changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 40a3a5c5ff..aa29da4337 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -20,6 +20,13 @@ static const char * const builtin_mv_usage[] = {\n \tNULL\n };\n \n+enum update_mode {\n+\tBOTH = 0,\n+\tWORKING_DIRECTORY = (1 << 1),\n+\tINDEX = (1 << 2),\n+\tSPARSE = (1 << 3),\n+};\n+\n #define DUP_BASENAME 1\n #define KEEP_TRAILING_SLASH 2\n \n@@ -130,7 +137,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n-\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n+\tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n@@ -192,7 +199,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n \t\t\t\t/* only error if existence is expected. */\n-\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n@@ -208,14 +215,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \t\t\t/* Check if dst exists in index */\n \t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n-\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\tmodes[i] |= SPARSE;\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n \t\t\tif (!force) {\n \t\t\t\tbad = _(\"destination exists\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t\tmodes[i] = SPARSE;\n+\t\t\tmodes[i] |= SPARSE;\n \t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n@@ -243,7 +250,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \n \t\t\t/* last - first >= 1 */\n-\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tmodes[i] |= WORKING_DIRECTORY;\n \t\t\tn = argc + last - first;\n \t\t\tREALLOC_ARRAY(source, n);\n \t\t\tREALLOC_ARRAY(destination, n);\n@@ -259,7 +266,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tsource[argc + j] = path;\n \t\t\t\tdestination[argc + j] =\n \t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tmemset(modes + argc + j, 0, sizeof(enum update_mode));\n+\t\t\t\tmodes[argc + j] |= ce_skip_worktree(ce) ? SPARSE : INDEX;\n \t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n \t\t\t}\n \t\t\targc += last - first;\n@@ -361,7 +369,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n+\t\tif (!(mode & (INDEX | SPARSE)) &&\n+\t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n@@ -375,7 +384,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode == WORKING_DIRECTORY)\n+\t\tif (mode & (WORKING_DIRECTORY))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\n-- \n2.35.1\n\n"},{"id":"457781","messageId":"20220623114120.12768-8-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v4 7/7] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-23T11:41:20Z","receivedAt":"2022-06-23T11:42:16Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a directory which is not on-disk due\nto its existence outside of sparse-checkout cone, \"giv mv\" command\nerrors out with \"bad source\".\n\nAdd a helper check_dir_in_index() function to see if a directory\nname exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\nsuch directories.\n\nChange the checking logic, so that such directory makes\n\"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 50 ++++++++++++++++++++++++++++++-----\n t/t7002-mv-sparse-checkout.sh | 4 +--\n 2 files changed, 46 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex aa29da4337..b5d0d8ef4f 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -25,6 +25,7 @@ enum update_mode {\n \tWORKING_DIRECTORY = (1 << 1),\n \tINDEX = (1 << 2),\n \tSPARSE = (1 << 3),\n+\tSKIP_WORKTREE_DIR = (1 << 4),\n };\n \n #define DUP_BASENAME 1\n@@ -123,6 +124,36 @@ static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n \n+/*\n+ * Check if an out-of-cone directory should be in the index. Imagine this case\n+ * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n+ * and thus the directory is sparsified.\n+ *\n+ * Return 0 if such directory exist (i.e. with any of its contained files not\n+ * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n+ * Return 1 otherwise.\n+ */\n+static int check_dir_in_index(const char *name)\n+{\n+\tconst char *with_slash = add_slash(name);\n+\tint length = strlen(with_slash);\n+\n+\tint pos = cache_name_pos(with_slash, length);\n+\tconst struct cache_entry *ce;\n+\n+\tif (pos < 0) {\n+\t\tpos = -pos - 1;\n+\t\tif (pos >= the_index.cache_nr)\n+\t\t\treturn 1;\n+\t\tce = active_cache[pos];\n+\t\tif (strncmp(with_slash, ce->name, length))\n+\t\t\treturn 1;\n+\t\tif (ce_skip_worktree(ce))\n+\t\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\n+\n int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n@@ -184,7 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t/* Checking */\n \tfor (i = 0; i < argc; i++) {\n \t\tconst char *src = source[i], *dst = destination[i];\n-\t\tint length, src_is_dir;\n+\t\tint length;\n \t\tconst char *bad = NULL;\n \t\tint skip_sparse = 0;\n \n@@ -198,12 +229,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n+\t\t\t\tconst char *src_w_slash = add_slash(src);\n+\t\t\t\tif (!path_in_sparse_checkout(src_w_slash, &the_index) &&\n+\t\t\t\t !check_dir_in_index(src)) {\n+\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n+\t\t\t\t\tgoto dir_check;\n+\t\t\t\t}\n \t\t\t\t/* only error if existence is expected. */\n \t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\n \t\t\tce = active_cache[pos];\n \t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n@@ -230,12 +266,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tbad = _(\"can not move directory into itself\");\n \t\t\tgoto act_on_entry;\n \t\t}\n-\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\tif (S_ISDIR(st.st_mode)\n \t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n \t\t\tgoto act_on_entry;\n \t\t}\n-\t\tif (src_is_dir) {\n+\n+dir_check:\n+\t\tif (S_ISDIR(st.st_mode)) {\n \t\t\tint j, dst_len, n;\n \t\t\tint first = cache_name_pos(src, length), last;\n \n@@ -369,7 +407,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (!(mode & (INDEX | SPARSE)) &&\n+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n \t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n@@ -384,7 +422,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode & (WORKING_DIRECTORY))\n+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 6d2fb4f8d2..71fe29690f 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -219,7 +219,7 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -230,7 +230,7 @@ test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone directory with --sparse' '\n+test_expect_success 'can move out-of-cone directory with --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n-- \n2.35.1\n\n"},{"id":"457787","messageId":"553ea547-9641-eb95-a474-db046fdc05dc@github.com","threadId":"57642","inReplyTo":"d3aa35e4-5bad-2bbb-2a25-d82064d6ec81@github.com","subject":"Re: [WIP v3 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-23T15:06:47Z","receivedAt":"2022-06-23T15:06:52Z","isPatch":false,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/21/2022 7:30 PM, Victoria Dye wrote:\n> Shaoxuan Yuan wrote:\n>> But I think it worth discuss if we should implement in-cone to \n>> out-of-cone move, since it will be nice (naturally) to have it working.\n>>\n>> However, I noticed this from the mv man page:\n>>\n>> \"In the second form, the last argument has to be an existing directory; \n>> the given sources will be moved into this directory.\"\n>>\n>> I think trying to move out-of-cone, the last argument has to be an non-existent\n>> directory? I'm a bit confused: should we update some of mv basic logic to \n>> accomplish this?\n>>\n> \n> I suspect this requirement is related to the POSIX 'mv' [1] (and\n> corresponding 'rename()', used in 'git mv'), which also requires that the\n> destination directory exists. I personally don't think this requirement\n> needs to apply to 'git mv' at all, but note that changing the behavior would\n> require first creating the necessary directories before calling 'rename()'. \n> \n> As a more conservative solution, you could do the parent directory creation\n> *only* in the case of moving to a sparse contents-only directory (using\n> something like the 'check_dir_in_index()' function you introduced to\n> identify).\n> \n> I'm also interested in hearing what others have to say, especially regarding\n> historical context/use cases of 'git mv'.\n> \n> [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/mv.html\n\nI wanted to reply here to maybe get more attention on this point.\n\nMy personal opinion is that `git mv` should move to the location requested,\neven if it requires adding parent directories. Changing that behavior might\nneed to come as its own topic, before doing the in-cone-to-out-of-cone work.\nKnowing if this behavior can change (or must stay the same) informs how that\nsparse case will work.\n\nThanks,\n-Stolee\n"},{"id":"457788","messageId":"3e84bdc5-31e7-a7aa-8404-228424810f35@github.com","threadId":"57642","inReplyTo":"20220623114120.12768-3-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v4 2/7] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-23T15:08:50Z","receivedAt":"2022-06-23T15:08:55Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n> Originally, \"git mv\" a sparse file from out-of-cone to\n> in-cone does not update the moved file's sparsity (remove its\n> SKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\n> not checked out in the working tree.\n> \n> Update the behavior so that:\n> 1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n> corresponding cache entry.\n> 2. The moved cache entry is checked out in the working tree to reflect\n> the updated sparsity.\n\nSince this is a behavior change, can we test it? It would be good\nto verify that the new path exists in the worktree after 'git mv'\nsucceeds.\n \nThanks,\n-Stolee\n"},{"id":"457789","messageId":"c232ccf7-1f73-030c-b7f8-02bdef342ba4@github.com","threadId":"57642","inReplyTo":"20220623114120.12768-7-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v4 6/7] mv: use flags mode for update_mode","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-23T15:10:58Z","receivedAt":"2022-06-23T15:11:03Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n> As suggested by Derrick [1],\n> move the in-line definition of \"enum update_mode\" to the top\n> of the file and make it use \"flags\" mode (each state is a different\n> bit in the word).\n\nnit: strange line-wrapping here.\n\n> Change the flag assignments from '=' (single assignment) to '|='\n> (additive). Also change flag evaluation from '==' to '&', etc.\n\nCode looks good.\n\nThanks,\n-Stolee\n"},{"id":"457790","messageId":"17a34570-f823-7908-1ea5-8e7bf8680b23@github.com","threadId":"57642","inReplyTo":"20220623114120.12768-8-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v4 7/7] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-23T15:14:18Z","receivedAt":"2022-06-23T15:14:24Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n> Originally, moving a directory which is not on-disk due\n> to its existence outside of sparse-checkout cone, \"giv mv\" command\n> errors out with \"bad source\".\n> \n> Add a helper check_dir_in_index() function to see if a directory\n> name exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\n> such directories.\n> \n> Change the checking logic, so that such directory makes\n> \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n> this operation can be carried out successfully.\n> \n> Helped-by: Victoria Dye \n> Helped-by: Derrick Stolee \n> Signed-off-by: Shaoxuan Yuan \n> ---\n> builtin/mv.c | 50 ++++++++++++++++++++++++++++++-----\n> t/t7002-mv-sparse-checkout.sh | 4 +--\n> 2 files changed, 46 insertions(+), 8 deletions(-)\n> \n> diff --git a/builtin/mv.c b/builtin/mv.c\n> index aa29da4337..b5d0d8ef4f 100644\n> --- a/builtin/mv.c\n> +++ b/builtin/mv.c\n> @@ -25,6 +25,7 @@ enum update_mode {\n> \tWORKING_DIRECTORY = (1 << 1),\n> \tINDEX = (1 << 2),\n> \tSPARSE = (1 << 3),\n> +\tSKIP_WORKTREE_DIR = (1 << 4),\n> };\n> \n> #define DUP_BASENAME 1\n> @@ -123,6 +124,36 @@ static int index_range_of_same_dir(const char *src, int length,\n> \treturn last - first;\n> }\n> \n> +/*\n> + * Check if an out-of-cone directory should be in the index. Imagine this case\n> + * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n> + * and thus the directory is sparsified.\n> + *\n> + * Return 0 if such directory exist (i.e. with any of its contained files not\n> + * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n> + * Return 1 otherwise.\n> + */\n\nThis description and the implementation seems like it will work\neven if the path exists as a sparse directory in a sparse index.\n\nIt would be good to consider testing this kind of move for a\ndirectory on the sparse boundary (where it would be a sparse\ndirectory in a sparse index) _and_ if it is deeper than the\nboundary (so the sparse index would expand in the cache_name_pos()\nmethod). These tests can be written now for correctness, but later\nthe first case can be updated to use the 'ensure_not_expanded'\nhelper in t1092.\n\n> +static int check_dir_in_index(const char *name)\n\nThanks,\n-Stolee\n"},{"id":"457792","messageId":"4d181d4f-b86c-42f9-eeca-000ec4a06015@github.com","threadId":"57642","inReplyTo":"20220623114120.12768-1-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v4 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-23T15:16:15Z","receivedAt":"2022-06-23T15:16:20Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n> ## Changes since WIP v3 ##\n\nIt's good to keep the main cover letter body around, even as the version\nupdates are included. Sometimes reviewers come to a topic late and want\nto see the latest version be completely self-contained.\n \nAfter reading all the patches with fresh eyes, I think the code looks\nvery good. I have a nitpick in a commit message and some recommendations\nfor additional tests, but otherwise I'm pretty happy with this version.\n\nThanks,\n-Stolee\n"},{"id":"457801","messageId":"xmqqr13f1het.fsf@gitster.g","threadId":"57642","inReplyTo":"553ea547-9641-eb95-a474-db046fdc05dc@github.com","subject":"Re: [WIP v3 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-23T16:19:38Z","receivedAt":"2022-06-23T16:19:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 6/21/2022 7:30 PM, Victoria Dye wrote:\n>> Shaoxuan Yuan wrote:\n>>> But I think it worth discuss if we should implement in-cone to \n>>> out-of-cone move, since it will be nice (naturally) to have it working.\n>>>\n>>> However, I noticed this from the mv man page:\n>>>\n>>> \"In the second form, the last argument has to be an existing directory; \n>>> the given sources will be moved into this directory.\"\n>>>\n>>> I think trying to move out-of-cone, the last argument has to be an non-existent\n>>> directory? I'm a bit confused: should we update some of mv basic logic to \n>>> accomplish this?\n>>>\n>> \n>> I suspect this requirement is related to the POSIX 'mv' [1] (and\n>> corresponding 'rename()', used in 'git mv'), which also requires that the\n>> destination directory exists. I personally don't think this requirement\n>> needs to apply to 'git mv' at all, but note that changing the behavior would\n>> require first creating the necessary directories before calling 'rename()'. \n>> \n>> As a more conservative solution, you could do the parent directory creation\n>> *only* in the case of moving to a sparse contents-only directory (using\n>> something like the 'check_dir_in_index()' function you introduced to\n>> identify).\n>> \n>> I'm also interested in hearing what others have to say, especially regarding\n>> historical context/use cases of 'git mv'.\n>> \n>> [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/mv.html\n>\n> I wanted to reply here to maybe get more attention on this point.\n>\n> My personal opinion is that `git mv` should move to the location requested,\n> even if it requires adding parent directories. Changing that behavior might\n> need to come as its own topic, before doing the in-cone-to-out-of-cone work.\n> Knowing if this behavior can change (or must stay the same) informs how that\n> sparse case will work.\n\nWhen a particular checkout excludes directory, say, Documentation/,\nfrom its cone of interest, we may not have that directory in the\nworking tree. In such a scenario, if you did \n\n $ git mv new.txt Documentation/technical/\n\nthe index may not even know if \"Documentation/\" (which most likely\nis represented as a sparse \"tree entry in the index\") has the\n\"technical\" subdirectory in it, so it may have to expand it\non-demand. I do not have an objection against making it easier for\nusers to do this.\n\nAs part of its implementation, you may have to do an equivalent of\n\"mkdir -p Documentation/technical/\" before you can even materialize\nthe \"new.txt\" file in the directory. I do not think that breaks the\nparallel to POSIX \"mv\" in that case, as Documentation/technical/ is\n*NOT* really a destination directory that does not exist.\nConceptually, the directory (and all the directories in the full\ntree) exists---it is just the sparse-checkout is hiding it from the\nview.\n\nA corollary to the above is what should happen when you did\n\n $ git mv new.txt Documentation/no-such-directory/\n\ni.e. you try to do a move that would fail even if you weren't using\nthe sparse-checkout feature. I think that *SHOULD* fail, if we\nwanted to be parallel to what POSIX \"mv\" does.\n\nThanks.\n\n\n\n\n\n\n\n\n"},{"id":"457807","messageId":"xmqqzgi3z25g.fsf@gitster.g","threadId":"57642","inReplyTo":"4d181d4f-b86c-42f9-eeca-000ec4a06015@github.com","subject":"Re: [PATCH v4 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-06-23T18:05:15Z","receivedAt":"2022-06-23T19:00:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n>> ## Changes since WIP v3 ##\n>\n> It's good to keep the main cover letter body around, even as the version\n> updates are included. Sometimes reviewers come to a topic late and want\n> to see the latest version be completely self-contained.\n\nYup. Thanks for stressing on it.\n\n> After reading all the patches with fresh eyes, I think the code looks\n> very good. I have a nitpick in a commit message and some recommendations\n> for additional tests, but otherwise I'm pretty happy with this version.\n\nThanks for ushering the topic forward.\n\n\n"},{"id":"457831","messageId":"a97eea15-00fe-dd33-2e4e-3ea91cb053f5@gmail.com","threadId":"57642","inReplyTo":"17a34570-f823-7908-1ea5-8e7bf8680b23@github.com","subject":"Re: [PATCH v4 7/7] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-24T07:57:23Z","receivedAt":"2022-06-24T07:57:30Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"On 6/23/2022 11:14 PM, Derrick Stolee wrote:\n > On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n >> Originally, moving a directory which is not on-disk due\n >> to its existence outside of sparse-checkout cone, \"giv mv\" command\n >> errors out with \"bad source\".\n >>\n >> Add a helper check_dir_in_index() function to see if a directory\n >> name exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\n >> such directories.\n >>\n >> Change the checking logic, so that such directory makes\n >> \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\n >> instead of \"bad source\"; also user now can supply a \"--sparse\" flag so\n >> this operation can be carried out successfully.\n >>\n >> Helped-by: Victoria Dye \n >> Helped-by: Derrick Stolee \n >> Signed-off-by: Shaoxuan Yuan \n >> ---\n >>  builtin/mv.c                  | 50 ++++++++++++++++++++++++++++++-----\n >>  t/t7002-mv-sparse-checkout.sh |  4 +--\n >>  2 files changed, 46 insertions(+), 8 deletions(-)\n >>\n >> diff --git a/builtin/mv.c b/builtin/mv.c\n >> index aa29da4337..b5d0d8ef4f 100644\n >> --- a/builtin/mv.c\n >> +++ b/builtin/mv.c\n >> @@ -25,6 +25,7 @@ enum update_mode {\n >>      WORKING_DIRECTORY = (1 << 1),\n >>      INDEX = (1 << 2),\n >>      SPARSE = (1 << 3),\n >> +    SKIP_WORKTREE_DIR = (1 << 4),\n >>  };\n >>\n >>  #define DUP_BASENAME 1\n >> @@ -123,6 +124,36 @@ static int index_range_of_same_dir(const char \n*src, int length,\n >>      return last - first;\n >>  }\n >>\n >> +/*\n >> + * Check if an out-of-cone directory should be in the index. \nImagine this case\n >> + * that all the files under a directory are marked with \n'CE_SKIP_WORKTREE' bit\n >> + * and thus the directory is sparsified.\n >> + *\n >> + * Return 0 if such directory exist (i.e. with any of its contained \nfiles not\n >> + * marked with CE_SKIP_WORKTREE, the directory would be present in \nworking tree).\n >> + * Return 1 otherwise.\n >> + */\n >\n > This description and the implementation seems like it will work\n > even if the path exists as a sparse directory in a sparse index.\n >\n > It would be good to consider testing this kind of move for a\n > directory on the sparse boundary (where it would be a sparse\n > directory in a sparse index) _and_ if it is deeper than the\n > boundary (so the sparse index would expand in the cache_name_pos()\n > method). These tests can be written now for correctness, but later\n > the first case can be updated to use the 'ensure_not_expanded'\n > helper in t1092.\n\nI'm a bit confused here. Shouldn't we turn on the sparse-index\nfeature for 'mv' before adding sparse-index related tests? Since this\nseries does not go into sparse-index, I'm not sure how the tests can\npass. Perhaps we can test about this in the future sparse-index\nintegration series, no?\n\nThanks & Regards,\nShaoxuan\n\n"},{"id":"457832","messageId":"f9e32b31-8b4a-2d07-4074-be10347764fa@gmail.com","threadId":"57642","inReplyTo":"3e84bdc5-31e7-a7aa-8404-228424810f35@github.com","subject":"Re: [PATCH v4 2/7] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-24T08:04:23Z","receivedAt":"2022-06-24T08:04:37Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\nOn 6/23/2022 11:08 PM, Derrick Stolee wrote:\n > On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n >> Originally, \"git mv\" a sparse file from out-of-cone to\n >> in-cone does not update the moved file's sparsity (remove its\n >> SKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\n >> not checked out in the working tree.\n >>\n >> Update the behavior so that:\n >> 1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n >>    corresponding cache entry.\n >> 2. The moved cache entry is checked out in the working tree to reflect\n >>    the updated sparsity.\n >\n > Since this is a behavior change, can we test it? It would be good\n > to verify that the new path exists in the worktree after 'git mv'\n > succeeds.\n\nI don't think we can effectively test this based on the change per se.\nThis change is preparing a correct behavior for the next few\ncommits, so I'll say it's tested along with the next few commits\n(i.e. move \"sparse\" file/directory ones)?\n\nThanks & Regards,\nShaoxuan\n\n"},{"id":"457834","messageId":"668d7a5f-7c1c-92ca-3db8-62f9bdca3006@gmail.com","threadId":"57642","inReplyTo":"xmqqr13f1het.fsf@gitster.g","subject":"Re: [WIP v3 0/7] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-24T08:26:51Z","receivedAt":"2022-06-24T08:27:00Z","isPatch":false,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"\nOn 6/24/2022 12:19 AM, Junio C Hamano wrote:\n > Derrick Stolee writes:\n >\n >> On 6/21/2022 7:30 PM, Victoria Dye wrote:\n >>> Shaoxuan Yuan wrote:\n >>>> But I think it worth discuss if we should implement in-cone to\n >>>> out-of-cone move, since it will be nice (naturally) to have it \nworking.\n >>>>\n >>>> However, I noticed this from the mv man page:\n >>>>\n >>>> \"In the second form, the last argument has to be an existing \ndirectory;\n >>>> the given sources will be moved into this directory.\"\n >>>>\n >>>> I think trying to move out-of-cone, the last argument has to be an \nnon-existent\n >>>> directory? I'm a bit confused: should we update some of mv basic \nlogic to\n >>>> accomplish this?\n >>>>\n >>>\n >>> I suspect this requirement is related to the POSIX 'mv' [1] (and\n >>> corresponding 'rename()', used in 'git mv'), which also requires \nthat the\n >>> destination directory exists. I personally don't think this requirement\n >>> needs to apply to 'git mv' at all, but note that changing the \nbehavior would\n >>> require first creating the necessary directories before calling \n'rename()'.\n >>>\n >>> As a more conservative solution, you could do the parent directory \ncreation\n >>> *only* in the case of moving to a sparse contents-only directory (using\n >>> something like the 'check_dir_in_index()' function you introduced to\n >>> identify).\n >>>\n >>> I'm also interested in hearing what others have to say, especially \nregarding\n >>> historical context/use cases of 'git mv'.\n >>>\n >>> [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/mv.html\n >>\n >> I wanted to reply here to maybe get more attention on this point.\n >>\n >> My personal opinion is that `git mv` should move to the location \nrequested,\n >> even if it requires adding parent directories. Changing that \nbehavior might\n >> need to come as its own topic, before doing the \nin-cone-to-out-of-cone work.\n >> Knowing if this behavior can change (or must stay the same) informs \nhow that\n >> sparse case will work.\n >\n > When a particular checkout excludes directory, say, Documentation/,\n > from its cone of interest, we may not have that directory in the\n > working tree.  In such a scenario, if you did\n >\n >     $ git mv new.txt Documentation/technical/\n >\n > the index may not even know if \"Documentation/\" (which most likely\n > is represented as a sparse \"tree entry in the index\") has the\n > \"technical\" subdirectory in it, so it may have to expand it\n > on-demand.  I do not have an objection against making it easier for\n > users to do this.\n >\n > As part of its implementation, you may have to do an equivalent of\n > \"mkdir -p Documentation/technical/\" before you can even materialize\n > the \"new.txt\" file in the directory.  I do not think that breaks the\n > parallel to POSIX \"mv\" in that case, as Documentation/technical/ is\n > *NOT* really a destination directory that does not exist.\n > Conceptually, the directory (and all the directories in the full\n > tree) exists---it is just the sparse-checkout is hiding it from the\n > view.\n >\n > A corollary to the above is what should happen when you did\n >\n >     $ git mv new.txt Documentation/no-such-directory/\n >\n > i.e. you try to do a move that would fail even if you weren't using\n > the sparse-checkout feature.  I think that *SHOULD* fail, if we\n > wanted to be parallel to what POSIX \"mv\" does.\n >\n > Thanks.\n\nAgree.\n\nThanks & Regards,\nShaoxuan\n\n"},{"id":"457904","messageId":"5585614d-0bb2-a889-4be3-1d1710f9ddb5@github.com","threadId":"57642","inReplyTo":"f9e32b31-8b4a-2d07-4074-be10347764fa@gmail.com","subject":"Re: [PATCH v4 2/7] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-27T13:55:55Z","receivedAt":"2022-06-27T13:56:00Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/24/2022 4:04 AM, Shaoxuan Yuan wrote:\n> \n> On 6/23/2022 11:08 PM, Derrick Stolee wrote:\n>> On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n>>> Originally, \"git mv\" a sparse file from out-of-cone to\n>>> in-cone does not update the moved file's sparsity (remove its\n>>> SKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\n>>> not checked out in the working tree.\n>>>\n>>> Update the behavior so that:\n>>> 1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n>>>    corresponding cache entry.\n>>> 2. The moved cache entry is checked out in the working tree to reflect\n>>>    the updated sparsity.\n>>\n>> Since this is a behavior change, can we test it? It would be good\n>> to verify that the new path exists in the worktree after 'git mv'\n>> succeeds.\n> \n> I don't think we can effectively test this based on the change per se.\n> This change is preparing a correct behavior for the next few\n> commits, so I'll say it's tested along with the next few commits\n> (i.e. move \"sparse\" file/directory ones)?\n\nAh, right. There are other reasons why moving from out-of-cone to\nin-cone is blocked at this point in time.\n\nThanks,\n-Stolee\n"},{"id":"457905","messageId":"baa7fed8-2309-8f08-92fd-d47831fad702@github.com","threadId":"57642","inReplyTo":"a97eea15-00fe-dd33-2e4e-3ea91cb053f5@gmail.com","subject":"Re: [PATCH v4 7/7] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-06-27T13:59:04Z","receivedAt":"2022-06-27T13:59:18Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/24/2022 3:57 AM, Shaoxuan Yuan wrote:\n> On 6/23/2022 11:14 PM, Derrick Stolee wrote:\n>> On 6/23/2022 7:41 AM, Shaoxuan Yuan wrote:\n>>> +/*\n>>> + * Check if an out-of-cone directory should be in the index. Imagine this case\n>>> + * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n>>> + * and thus the directory is sparsified.\n>>> + *\n>>> + * Return 0 if such directory exist (i.e. with any of its contained files not\n>>> + * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n>>> + * Return 1 otherwise.\n>>> + */\n>>\n>> This description and the implementation seems like it will work\n>> even if the path exists as a sparse directory in a sparse index.\n>>\n>> It would be good to consider testing this kind of move for a\n>> directory on the sparse boundary (where it would be a sparse\n>> directory in a sparse index) _and_ if it is deeper than the\n>> boundary (so the sparse index would expand in the cache_name_pos()\n>> method). These tests can be written now for correctness, but later\n>> the first case can be updated to use the 'ensure_not_expanded'\n>> helper in t1092.\n> \n> I'm a bit confused here. Shouldn't we turn on the sparse-index\n> feature for 'mv' before adding sparse-index related tests? Since this\n> series does not go into sparse-index, I'm not sure how the tests can\n> pass. Perhaps we can test about this in the future sparse-index\n> integration series, no?\n\nI mean that you are making a change right now that might lead to\ndifferent behavior _when you enable the sparse index later_. Since\nwe are looking at this behavior now, it might be interesting to add\nthe extra test coverage now while we are thinking about this data\nshape.\n\nWe can add tests to t1092-sparse-checkout-compatibility.sh even\nthough the sparse index is expanded on every 'git mv' instance, but\nit can help when you eventually update 'git mv' to not expand on a\nsparse index.\n\nThe bit about ensure_not_expanded can't be added until you integrate\nwith the sparse index, but it is easier to add that when you already\nhave tests that check these boundary cases.\n\nThanks,\n-Stolee\n"},{"id":"458143","messageId":"20220630023737.473690-2-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 1/8] t7002: add tests for moving out-of-cone file/directory","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:30Z","receivedAt":"2022-06-30T02:38:01Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add corresponding tests to test following situations:\n\nWe do not have sufficient coverage of moving files outside\nof a sparse-checkout cone. Create new tests covering this\nbehavior, keeping in mind that the user can include --sparse\n(or not), move a file or directory, and the destination can\nalready exist in the index (in this case user can use --force\nto overwrite existing entry).\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n t/t7002-mv-sparse-checkout.sh | 84 +++++++++++++++++++++++++++++++++++\n 1 file changed, 84 insertions(+)\n\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex f0f7cbfcdb..023e657c9e 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -4,6 +4,18 @@ test_description='git mv in sparse working trees'\n \n . ./test-lib.sh\n \n+setup_sparse_checkout () {\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\tgit add folder1 &&\n+\tgit sparse-checkout set --cone sub\n+}\n+\n+cleanup_sparse_checkout () {\n+\tgit sparse-checkout disable &&\n+\tgit reset --hard\n+}\n+\n test_expect_success 'setup' \"\n \tmkdir -p sub/dir sub/dir2 &&\n \ttouch a b c sub/d sub/dir/e sub/dir2/e &&\n@@ -196,6 +208,7 @@ test_expect_success 'can move files to non-sparse dir' '\n '\n \n test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tgit reset --hard &&\n \tgit sparse-checkout init --no-cone &&\n \tgit sparse-checkout set a !/x y/ !x/y/z &&\n@@ -206,4 +219,75 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n+test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv folder1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone directory with --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\tgit mv --sparse folder1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_dir sub/folder1 &&\n+\ttest_path_is_file sub/folder1/file1\n+'\n+\n+test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\ttest_must_fail git mv folder1/file1 sub 2>stderr &&\n+\tcat sparse_error_header >expect &&\n+\techo folder1/file1 >>expect &&\n+\tcat sparse_hint >>expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'can move out-of-cone file with --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tsetup_sparse_checkout &&\n+\n+\tgit mv --sparse folder1/file1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\n+\ttest_path_is_file sub/file1\n+'\n+\n+test_expect_failure 'refuse to move sparse file to existing destination' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\ttest_must_fail git mv --sparse folder1/file1 sub 2>stderr &&\n+\techo \"fatal: destination exists, source=folder1/file1, destination=sub/file1\" >expect &&\n+\ttest_cmp expect stderr\n+'\n+\n+test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+\ttest_when_finished \"cleanup_sparse_checkout\" &&\n+\tmkdir folder1 &&\n+\ttouch folder1/file1 &&\n+\ttouch sub/file1 &&\n+\techo \"overwrite\" >folder1/file1 &&\n+\tgit add folder1 sub/file1 &&\n+\tgit sparse-checkout set --cone sub &&\n+\n+\tgit mv --sparse --force folder1/file1 sub 2>stderr &&\n+\ttest_must_be_empty stderr &&\n+\techo \"overwrite\" >expect &&\n+\ttest_cmp expect sub/file1\n+'\n+\n test_done\n-- \n2.35.1\n\n"},{"id":"458144","messageId":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220331091755.385961-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 0/8] mv: fix out-of-cone file/directory move logic","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:29Z","receivedAt":"2022-06-30T02:38:04Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"## Changes since PATCH v4 ##\n\n1. Fix style-nits.\n\n2. Add t1092 tests (2/8) for \"mv: add check_dir_in_index() and solve \n general dir check issue\" (8/8).\n\n## Changes since WIP v3 ##\n\n1. Fix style-nits.\n\n2. Move \"mv: update sparsity after moving from out-of-cone to in-cone\" to the \n (2/7) position (was (7/7) position in WIP v3). Make this move so that we\n can drop \"git sparse-checkout reapply\" from the tests, as suggested by\n Victoria [1]. The reason is we need to check out the moved cache_entry\n for all out-of-cone to in-cone moves, so this commit works better if being the\n first one.\n\n3. Fix the commit message of \"mv: use flags mode for update_mode\", as suggested\n here [2].\n\n4. Add \"Helped-by\" and \"Suggested-by\" trailers.\n\n5. In \"mv: add check_dir_in_index() and solve general dir check issue\", change\n the 'check_dir_in_index()' to no more accept 'namelen' as argument. The\n original \"namelen + 1\" logic can be erroneous, for example when 'name' already\n has a trailing slash. So just use strlen() to save the trouble.\n\n[1] https://lore.kernel.org/git/adb795ba-56ce-8441-0c38-a3e6b0a6e861@github.com/\n[2] https://lore.kernel.org/git/01b39c63-5652-4293-0424-ff99b6f9f7d2@github.com/\n\n## Changes since WIP v2 ##\n\n1. Write helper functions for t7002 to reuse some code.\n\n2. Refactor/decouple the if/else-if checking chain.\n\n3. Separate out the 'update_mode' refactor into a single commit.\n\n4. Stop using update_sparsity() and instead update the SKIP_WORKTREE\n bit for each cache_entry and check it out to the working tree.\n\n## Changes since WIP v1 ##\n\n1. Move t7002 tests to the front and turn corresponding tests to \n test_expect_success along with corresponding commits.\n\n2. Add two tests to t7002.\n\n3. Update check_dir_in_index() and added corresponding documentation.\n\n4. Turn update_mode into enum flags.\n\n5. Use update_sparsity() to replace advise*() function after touching\n sparse contents (this change is INCOMPLETE, NEED FIX).\n\n6. Fix some format issues.\n\n## Starting WIP v1 ##\n\nBefore integrating 'mv' with sparse-index, I still find some possibly buggy\nUX when 'mv' is interacting with 'sparse-checkout'. \n\nSo I kept sparse-index off in order to sort things out without a sparse index.\nWe can proceed to integrate with sparse-index once these changes are solid.\n\nNote that this patch is tentative, and still have known glitches, but it \nillustrates a general approach that I intended to harmonize 'mv' \nwith 'sparse-checkout'.\n\n## Pre WIP v1 discussion ##\n\nRFC patch about integarting sparse index with 'mv' [3]\n\n[3] https://lore.kernel.org/git/20220315100145.214054-1-shaoxuan.yuan02@gmail.com/\n\nShaoxuan Yuan (8):\n t7002: add tests for moving out-of-cone file/directory\n t1092: mv directory from out-of-cone to in-cone\n mv: update sparsity after moving from out-of-cone to in-cone\n mv: decouple if/else-if checks using goto\n mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n mv: check if exists in index to handle overwriting\n mv: use flags mode for update_mode\n mv: add check_dir_in_index() and solve general dir check issue\n\n builtin/mv.c | 239 +++++++++++++++++------\n t/t1092-sparse-checkout-compatibility.sh | 25 +++\n t/t7002-mv-sparse-checkout.sh | 84 ++++++++\n 3 files changed, 284 insertions(+), 64 deletions(-)\n\nRange-diff against v4:\n1: 6ec0fee96c = 1: c2715ffd84 t7002: add tests for moving out-of-cone file/directory\n-: ---------- > 2: 7069755abd t1092: mv directory from out-of-cone to in-cone\n2: 0887fbbc82 ! 3: 33188b9f1c mv: update sparsity after moving from out-of-cone to in-cone\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n +\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n +\n +\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n -+\t\t\t\tdie(_(\"cannot checkout %s\"), ce->name);\n ++\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n +\t\t}\n \t}\n \n3: e79785487e = 4: 6100d6ca40 mv: decouple if/else-if checks using goto\n4: bdbfd90843 = 5: c34e2871d1 mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit\n5: 807de09f19 = 6: e4a0ba40df mv: check if exists in index to handle overwriting\n6: af84df7dd0 ! 7: 7bc5cddcf4 mv: use flags mode for update_mode\n @@ Metadata\n ## Commit message ##\n mv: use flags mode for update_mode\n \n - As suggested by Derrick [1],\n - move the in-line definition of \"enum update_mode\" to the top\n - of the file and make it use \"flags\" mode (each state is a different\n - bit in the word).\n + As suggested by Derrick [1], move the in-line definition of\n + \"enum update_mode\" to the top of the file and make it use \"flags\"\n + mode (each state is a different bit in the word).\n \n Change the flag assignments from '=' (single assignment) to '|='\n (additive). Also change flag evaluation from '==' to '&', etc.\n7: d9fd1c452c ! 8: 9f89faa557 mv: add check_dir_in_index() and solve general dir check issue\n @@ builtin/mv.c: int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tpos = cache_name_pos(src, strlen(src));\n \n + ## t/t1092-sparse-checkout-compatibility.sh ##\n +@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'checkout behaves oddly with df-conflict-2' '\n + \ttest_cmp full-checkout-err sparse-index-err\n + '\n + \n +-test_expect_failure 'mv directory from out-of-cone to in-cone' '\n ++test_expect_success 'mv directory from out-of-cone to in-cone' '\n + \tinit_repos &&\n + \n + \t# as a sparse directory (or SKIP_WORKTREE_DIR without enabling\n +\n ## t/t7002-mv-sparse-checkout.sh ##\n @@ t/t7002-mv-sparse-checkout.sh: test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n\nbase-commit: e4a4b31577c7419497ac30cebe30d755b97752c5\n-- \n2.35.1\n\n"},{"id":"458145","messageId":"20220630023737.473690-3-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 2/8] t1092: mv directory from out-of-cone to in-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:31Z","receivedAt":"2022-06-30T02:38:04Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Add test for \"mv: add check_dir_in_index() and solve general dir check\nissue\" in this series.\n\nThis change tests the following:\n\n1. mv as a directory on the sparse index boundary (where it\n would be a sparse directory in a sparse index).\n2. mv as a directory which is deeper than the boundary (so\n the sparse index would expand in the cache_name_pos() method).\n\nThese tests can be written now for correctness, but later the first case\ncan be updated to use the 'ensure_not_expanded' helper in t1092.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n t/t1092-sparse-checkout-compatibility.sh | 25 ++++++++++++++++++++++++\n 1 file changed, 25 insertions(+)\n\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex f9f8c988bb..5eef799e25 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -1828,4 +1828,29 @@ test_expect_success 'checkout behaves oddly with df-conflict-2' '\n \ttest_cmp full-checkout-err sparse-index-err\n '\n \n+test_expect_failure 'mv directory from out-of-cone to in-cone' '\n+\tinit_repos &&\n+\n+\t# as a sparse directory (or SKIP_WORKTREE_DIR without enabling\n+\t# sparse index).\n+\ttest_all_match git mv --sparse folder1 deep &&\n+\ttest_all_match git status --porcelain=v2 &&\n+\ttest_sparse_match git ls-files -t &&\n+\tgit -C sparse-checkout ls-files -t >actual &&\n+\tgrep -e \"H deep/folder1/0/0/0\" actual &&\n+\tgrep -e \"H deep/folder1/0/1\" actual &&\n+\tgrep -e \"H deep/folder1/a\" actual &&\n+\n+\ttest_all_match git reset --hard &&\n+\n+\t# as a directory deeper than sparse index boundary (where\n+\t# sparse index will expand).\n+\ttest_sparse_match git mv --sparse folder1/0 deep &&\n+\ttest_sparse_match git status --porcelain=v2 &&\n+\ttest_sparse_match git ls-files -t &&\n+\tgit -C sparse-checkout ls-files -t >actual &&\n+\tgrep -e \"H deep/0/0/0\" actual &&\n+\tgrep -e \"H deep/0/1\" actual\n+'\n+\n test_done\n-- \n2.35.1\n\n"},{"id":"458146","messageId":"20220630023737.473690-4-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 3/8] mv: update sparsity after moving from out-of-cone to in-cone","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:32Z","receivedAt":"2022-06-30T02:38:07Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, \"git mv\" a sparse file from out-of-cone to\nin-cone does not update the moved file's sparsity (remove its\nSKIP_WORKTREE bit). And the corresponding cache entry is, unexpectedly,\nnot checked out in the working tree.\n\nUpdate the behavior so that:\n1. Moving from out-of-cone to in-cone removes the SKIP_WORKTREE bit from\n corresponding cache entry.\n2. The moved cache entry is checked out in the working tree to reflect\n the updated sparsity.\n\nHelped-by: Victoria Dye \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 83a465ba83..5f4eeadd5d 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -13,6 +13,7 @@\n #include \"string-list.h\"\n #include \"parse-options.h\"\n #include \"submodule.h\"\n+#include \"entry.h\"\n \n static const char * const builtin_mv_usage[] = {\n \tN_(\"git mv [] ... \"),\n@@ -304,6 +305,11 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tconst char *src = source[i], *dst = destination[i];\n \t\tenum update_mode mode = modes[i];\n \t\tint pos;\n+\t\tstruct checkout state = CHECKOUT_INIT;\n+\t\tstate.istate = &the_index;\n+\n+\t\tif (force)\n+\t\t\tstate.force = 1;\n \t\tif (show_only || verbose)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n@@ -328,6 +334,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tpos = cache_name_pos(src, strlen(src));\n \t\tassert(pos >= 0);\n \t\trename_cache_entry_at(pos, dst);\n+\n+\t\tif ((mode & SPARSE) &&\n+\t\t (path_in_sparse_checkout(dst, &the_index))) {\n+\t\t\tint dst_pos;\n+\n+\t\t\tdst_pos = cache_name_pos(dst, strlen(dst));\n+\t\t\tactive_cache[dst_pos]->ce_flags &= ~CE_SKIP_WORKTREE;\n+\n+\t\t\tif (checkout_entry(active_cache[dst_pos], &state, NULL, NULL))\n+\t\t\t\tdie(_(\"cannot checkout %s\"), active_cache[dst_pos]->name);\n+\t\t}\n \t}\n \n \tif (gitmodules_modified)\n-- \n2.35.1\n\n"},{"id":"458147","messageId":"20220630023737.473690-5-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 4/8] mv: decouple if/else-if checks using goto","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:33Z","receivedAt":"2022-06-30T02:38:10Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Previous if/else-if chain are highly nested and hard to develop/extend.\n\nRefactor to decouple this if/else-if chain by using goto to jump ahead.\n\nSuggested-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 139 +++++++++++++++++++++++++++++----------------------\n 1 file changed, 80 insertions(+), 59 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 5f4eeadd5d..e800da3ab8 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -187,53 +187,68 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n \t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE)\n+\t\t\tif (modes[i] != SPARSE) {\n \t\t\t\tbad = _(\"bad source\");\n-\t\t} else if (!strncmp(src, dst, length) &&\n-\t\t\t\t(dst[length] == 0 || dst[length] == '/')) {\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t}\n+\t\tif (!strncmp(src, dst, length) &&\n+\t\t (dst[length] == 0 || dst[length] == '/')) {\n \t\t\tbad = _(\"can not move directory into itself\");\n-\t\t} else if ((src_is_dir = S_ISDIR(st.st_mode))\n-\t\t\t\t&& lstat(dst, &st) == 0)\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n-\t\telse if (src_is_dir) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (src_is_dir) {\n+\t\t\tint j, dst_len, n;\n \t\t\tint first = cache_name_pos(src, length), last;\n \n-\t\t\tif (first >= 0)\n+\t\t\tif (first >= 0) {\n \t\t\t\tprepare_move_submodule(src, first,\n \t\t\t\t\t\t submodule_gitfile + i);\n-\t\t\telse if (index_range_of_same_dir(src, length,\n-\t\t\t\t\t\t\t &first, &last) < 1)\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t} else if (index_range_of_same_dir(src, length,\n+\t\t\t\t\t\t\t &first, &last) < 1) {\n \t\t\t\tbad = _(\"source directory is empty\");\n-\t\t\telse { /* last - first >= 1 */\n-\t\t\t\tint j, dst_len, n;\n-\n-\t\t\t\tmodes[i] = WORKING_DIRECTORY;\n-\t\t\t\tn = argc + last - first;\n-\t\t\t\tREALLOC_ARRAY(source, n);\n-\t\t\t\tREALLOC_ARRAY(destination, n);\n-\t\t\t\tREALLOC_ARRAY(modes, n);\n-\t\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n-\n-\t\t\t\tdst = add_slash(dst);\n-\t\t\t\tdst_len = strlen(dst);\n-\n-\t\t\t\tfor (j = 0; j < last - first; j++) {\n-\t\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n-\t\t\t\t\tconst char *path = ce->name;\n-\t\t\t\t\tsource[argc + j] = path;\n-\t\t\t\t\tdestination[argc + j] =\n-\t\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n-\t\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n-\t\t\t\t}\n-\t\t\t\targc += last - first;\n+\t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t} else if (!(ce = cache_file_exists(src, length, 0))) {\n+\n+\t\t\t/* last - first >= 1 */\n+\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tn = argc + last - first;\n+\t\t\tREALLOC_ARRAY(source, n);\n+\t\t\tREALLOC_ARRAY(destination, n);\n+\t\t\tREALLOC_ARRAY(modes, n);\n+\t\t\tREALLOC_ARRAY(submodule_gitfile, n);\n+\n+\t\t\tdst = add_slash(dst);\n+\t\t\tdst_len = strlen(dst);\n+\n+\t\t\tfor (j = 0; j < last - first; j++) {\n+\t\t\t\tconst struct cache_entry *ce = active_cache[first + j];\n+\t\t\t\tconst char *path = ce->name;\n+\t\t\t\tsource[argc + j] = path;\n+\t\t\t\tdestination[argc + j] =\n+\t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n+\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n+\t\t\t}\n+\t\t\targc += last - first;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (!(ce = cache_file_exists(src, length, 0))) {\n \t\t\tbad = _(\"not under version control\");\n-\t\t} else if (ce_stage(ce)) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (ce_stage(ce)) {\n \t\t\tbad = _(\"conflicted\");\n-\t\t} else if (lstat(dst, &st) == 0 &&\n-\t\t\t (!ignore_case || strcasecmp(src, dst))) {\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (lstat(dst, &st) == 0 &&\n+\t\t (!ignore_case || strcasecmp(src, dst))) {\n \t\t\tbad = _(\"destination exists\");\n \t\t\tif (force) {\n \t\t\t\t/*\n@@ -247,34 +262,40 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t} else\n \t\t\t\t\tbad = _(\"Cannot overwrite\");\n \t\t\t}\n-\t\t} else if (string_list_has_string(&src_for_dst, dst))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (string_list_has_string(&src_for_dst, dst)) {\n \t\t\tbad = _(\"multiple sources for the same target\");\n-\t\telse if (is_dir_sep(dst[strlen(dst) - 1]))\n+\t\t\tgoto act_on_entry;\n+\t\t}\n+\t\tif (is_dir_sep(dst[strlen(dst) - 1])) {\n \t\t\tbad = _(\"destination directory does not exist\");\n-\t\telse {\n-\t\t\t/*\n-\t\t\t * We check if the paths are in the sparse-checkout\n-\t\t\t * definition as a very final check, since that\n-\t\t\t * allows us to point the user to the --sparse\n-\t\t\t * option as a way to have a successful run.\n-\t\t\t */\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(src, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\t\t\tif (!ignore_sparse &&\n-\t\t\t !path_in_sparse_checkout(dst, &the_index)) {\n-\t\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n-\t\t\t\tskip_sparse = 1;\n-\t\t\t}\n-\n-\t\t\tif (skip_sparse)\n-\t\t\t\tgoto remove_entry;\n+\t\t\tgoto act_on_entry;\n+\t\t}\n \n-\t\t\tstring_list_insert(&src_for_dst, dst);\n+\t\t/*\n+\t\t * We check if the paths are in the sparse-checkout\n+\t\t * definition as a very final check, since that\n+\t\t * allows us to point the user to the --sparse\n+\t\t * option as a way to have a successful run.\n+\t\t */\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(src, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\tskip_sparse = 1;\n+\t\t}\n+\t\tif (!ignore_sparse &&\n+\t\t !path_in_sparse_checkout(dst, &the_index)) {\n+\t\t\tstring_list_append(&only_match_skip_worktree, dst);\n+\t\t\tskip_sparse = 1;\n \t\t}\n \n+\t\tif (skip_sparse)\n+\t\t\tgoto remove_entry;\n+\n+\t\tstring_list_insert(&src_for_dst, dst);\n+\n+act_on_entry:\n \t\tif (!bad)\n \t\t\tcontinue;\n \t\tif (!ignore_errors)\n-- \n2.35.1\n\n"},{"id":"458148","messageId":"20220630023737.473690-6-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 5/8] mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:34Z","receivedAt":"2022-06-30T02:38:19Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a file which is not on-disk but exists in\nindex as a SKIP_WORKTREE enabled cache entry, \"giv mv\" command errors\nout with \"bad source\".\n\nChange the checking logic, so that such \nfile makes \"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 21 +++++++++++++++++++--\n t/t7002-mv-sparse-checkout.sh | 4 ++--\n 2 files changed, 21 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex e800da3ab8..520be85774 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -186,11 +186,28 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\tlength = strlen(src);\n \t\tif (lstat(src, &st) < 0) {\n-\t\t\t/* only error if existence is expected. */\n-\t\t\tif (modes[i] != SPARSE) {\n+\t\t\tint pos;\n+\t\t\tconst struct cache_entry *ce;\n+\n+\t\t\tpos = cache_name_pos(src, length);\n+\t\t\tif (pos < 0) {\n+\t\t\t\t/* only error if existence is expected. */\n+\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\t\tbad = _(\"bad source\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\n+\t\t\tce = active_cache[pos];\n+\t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n+\n+\t\t\tif (!ignore_sparse)\n+\t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n+\t\t\telse\n+\t\t\t\tmodes[i] = SPARSE;\n+\t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n \t\t (dst[length] == 0 || dst[length] == '/')) {\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 023e657c9e..1510b5ed6a 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -241,7 +241,7 @@ test_expect_failure 'can move out-of-cone directory with --sparse' '\n \ttest_path_is_file sub/folder1/file1\n '\n \n-test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n+test_expect_success 'refuse to move out-of-cone file without --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -252,7 +252,7 @@ test_expect_failure 'refuse to move out-of-cone file without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone file with --sparse' '\n+test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n-- \n2.35.1\n\n"},{"id":"458149","messageId":"20220630023737.473690-7-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 6/8] mv: check if exists in index to handle overwriting","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:35Z","receivedAt":"2022-06-30T02:38:37Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a sparse file into cone can result in unwarned\noverwrite of existing entry. The expected behavior is that if the\n exists in the entry, user should be prompted to supply\na [-f|--force] to carry out the operation, or the operation should\nfail.\n\nAdd a check mechanism to do that.\n\nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 15 ++++++++++++---\n t/t7002-mv-sparse-checkout.sh | 4 ++--\n 2 files changed, 14 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 520be85774..7d9627938a 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -202,11 +202,20 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\n-\t\t\tif (!ignore_sparse)\n+\t\t\tif (!ignore_sparse) {\n \t\t\t\tstring_list_append(&only_match_skip_worktree, src);\n-\t\t\telse\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\t/* Check if dst exists in index */\n+\t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n \t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\tif (!force) {\n+\t\t\t\tbad = _(\"destination exists\");\n+\t\t\t\tgoto act_on_entry;\n+\t\t\t}\n+\t\t\tmodes[i] = SPARSE;\n \t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 1510b5ed6a..6d2fb4f8d2 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -262,7 +262,7 @@ test_expect_success 'can move out-of-cone file with --sparse' '\n \ttest_path_is_file sub/file1\n '\n \n-test_expect_failure 'refuse to move sparse file to existing destination' '\n+test_expect_success 'refuse to move sparse file to existing destination' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n@@ -275,7 +275,7 @@ test_expect_failure 'refuse to move sparse file to existing destination' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'move sparse file to existing destination with --force and --sparse' '\n+test_expect_success 'move sparse file to existing destination with --force and --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tmkdir folder1 &&\n \ttouch folder1/file1 &&\n-- \n2.35.1\n\n"},{"id":"458150","messageId":"20220630023737.473690-8-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 7/8] mv: use flags mode for update_mode","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:36Z","receivedAt":"2022-06-30T02:38:39Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"As suggested by Derrick [1], move the in-line definition of\n\"enum update_mode\" to the top of the file and make it use \"flags\"\nmode (each state is a different bit in the word).\n\nChange the flag assignments from '=' (single assignment) to '|='\n(additive). Also change flag evaluation from '==' to '&', etc.\n\n[1] https://lore.kernel.org/git/22aadea2-9330-aa9e-7b6a-834585189144@github.com/\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 25 +++++++++++++++++--------\n 1 file changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex 7d9627938a..b805a0d0f6 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -20,6 +20,13 @@ static const char * const builtin_mv_usage[] = {\n \tNULL\n };\n \n+enum update_mode {\n+\tBOTH = 0,\n+\tWORKING_DIRECTORY = (1 << 1),\n+\tINDEX = (1 << 2),\n+\tSPARSE = (1 << 3),\n+};\n+\n #define DUP_BASENAME 1\n #define KEEP_TRAILING_SLASH 2\n \n@@ -130,7 +137,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\tOPT_END(),\n \t};\n \tconst char **source, **destination, **dest_path, **submodule_gitfile;\n-\tenum update_mode { BOTH = 0, WORKING_DIRECTORY, INDEX, SPARSE } *modes;\n+\tenum update_mode *modes;\n \tstruct stat st;\n \tstruct string_list src_for_dst = STRING_LIST_INIT_NODUP;\n \tstruct lock_file lock_file = LOCK_INIT;\n@@ -192,7 +199,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n \t\t\t\t/* only error if existence is expected. */\n-\t\t\t\tif (modes[i] != SPARSE)\n+\t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n@@ -208,14 +215,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \t\t\t/* Check if dst exists in index */\n \t\t\tif (cache_name_pos(dst, strlen(dst)) < 0) {\n-\t\t\t\tmodes[i] = SPARSE;\n+\t\t\t\tmodes[i] |= SPARSE;\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n \t\t\tif (!force) {\n \t\t\t\tbad = _(\"destination exists\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\t\t\tmodes[i] = SPARSE;\n+\t\t\tmodes[i] |= SPARSE;\n \t\t\tgoto act_on_entry;\n \t\t}\n \t\tif (!strncmp(src, dst, length) &&\n@@ -243,7 +250,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t}\n \n \t\t\t/* last - first >= 1 */\n-\t\t\tmodes[i] = WORKING_DIRECTORY;\n+\t\t\tmodes[i] |= WORKING_DIRECTORY;\n \t\t\tn = argc + last - first;\n \t\t\tREALLOC_ARRAY(source, n);\n \t\t\tREALLOC_ARRAY(destination, n);\n@@ -259,7 +266,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\tsource[argc + j] = path;\n \t\t\t\tdestination[argc + j] =\n \t\t\t\t\tprefix_path(dst, dst_len, path + length + 1);\n-\t\t\t\tmodes[argc + j] = ce_skip_worktree(ce) ? SPARSE : INDEX;\n+\t\t\t\tmemset(modes + argc + j, 0, sizeof(enum update_mode));\n+\t\t\t\tmodes[argc + j] |= ce_skip_worktree(ce) ? SPARSE : INDEX;\n \t\t\t\tsubmodule_gitfile[argc + j] = NULL;\n \t\t\t}\n \t\t\targc += last - first;\n@@ -361,7 +369,8 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (mode != INDEX && mode != SPARSE && rename(src, dst) < 0) {\n+\t\tif (!(mode & (INDEX | SPARSE)) &&\n+\t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n \t\t\tdie_errno(_(\"renaming '%s' failed\"), src);\n@@ -375,7 +384,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode == WORKING_DIRECTORY)\n+\t\tif (mode & (WORKING_DIRECTORY))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\n-- \n2.35.1\n\n"},{"id":"458151","messageId":"20220630023737.473690-9-shaoxuan.yuan02@gmail.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"[PATCH v5 8/8] mv: add check_dir_in_index() and solve general dir check issue","fromName":"Shaoxuan Yuan","fromEmail":"shaoxuan.yuan02@gmail.com","sentAt":"2022-06-30T02:37:37Z","receivedAt":"2022-06-30T02:38:48Z","isPatch":true,"sender":{"key":"shaoxuan.yuan02@gmail.com","avatar":"https://avatars.githubusercontent.com/u/46557895?v=4"},"body":"Originally, moving a directory which is not on-disk due\nto its existence outside of sparse-checkout cone, \"giv mv\" command\nerrors out with \"bad source\".\n\nAdd a helper check_dir_in_index() function to see if a directory\nname exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark\nsuch directories.\n\nChange the checking logic, so that such directory makes\n\"giv mv\" command warns with \"advise_on_updating_sparse_paths()\"\ninstead of \"bad source\"; also user now can supply a \"--sparse\" flag so\nthis operation can be carried out successfully.\n\nHelped-by: Victoria Dye \nHelped-by: Derrick Stolee \nSigned-off-by: Shaoxuan Yuan \n---\n builtin/mv.c | 50 +++++++++++++++++++++---\n t/t1092-sparse-checkout-compatibility.sh | 2 +-\n t/t7002-mv-sparse-checkout.sh | 4 +-\n 3 files changed, 47 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/mv.c b/builtin/mv.c\nindex b805a0d0f6..2a38e2af46 100644\n--- a/builtin/mv.c\n+++ b/builtin/mv.c\n@@ -25,6 +25,7 @@ enum update_mode {\n \tWORKING_DIRECTORY = (1 << 1),\n \tINDEX = (1 << 2),\n \tSPARSE = (1 << 3),\n+\tSKIP_WORKTREE_DIR = (1 << 4),\n };\n \n #define DUP_BASENAME 1\n@@ -123,6 +124,36 @@ static int index_range_of_same_dir(const char *src, int length,\n \treturn last - first;\n }\n \n+/*\n+ * Check if an out-of-cone directory should be in the index. Imagine this case\n+ * that all the files under a directory are marked with 'CE_SKIP_WORKTREE' bit\n+ * and thus the directory is sparsified.\n+ *\n+ * Return 0 if such directory exist (i.e. with any of its contained files not\n+ * marked with CE_SKIP_WORKTREE, the directory would be present in working tree).\n+ * Return 1 otherwise.\n+ */\n+static int check_dir_in_index(const char *name)\n+{\n+\tconst char *with_slash = add_slash(name);\n+\tint length = strlen(with_slash);\n+\n+\tint pos = cache_name_pos(with_slash, length);\n+\tconst struct cache_entry *ce;\n+\n+\tif (pos < 0) {\n+\t\tpos = -pos - 1;\n+\t\tif (pos >= the_index.cache_nr)\n+\t\t\treturn 1;\n+\t\tce = active_cache[pos];\n+\t\tif (strncmp(with_slash, ce->name, length))\n+\t\t\treturn 1;\n+\t\tif (ce_skip_worktree(ce))\n+\t\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\n+\n int cmd_mv(int argc, const char **argv, const char *prefix)\n {\n \tint i, flags, gitmodules_modified = 0;\n@@ -184,7 +215,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t/* Checking */\n \tfor (i = 0; i < argc; i++) {\n \t\tconst char *src = source[i], *dst = destination[i];\n-\t\tint length, src_is_dir;\n+\t\tint length;\n \t\tconst char *bad = NULL;\n \t\tint skip_sparse = 0;\n \n@@ -198,12 +229,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \n \t\t\tpos = cache_name_pos(src, length);\n \t\t\tif (pos < 0) {\n+\t\t\t\tconst char *src_w_slash = add_slash(src);\n+\t\t\t\tif (!path_in_sparse_checkout(src_w_slash, &the_index) &&\n+\t\t\t\t !check_dir_in_index(src)) {\n+\t\t\t\t\tmodes[i] |= SKIP_WORKTREE_DIR;\n+\t\t\t\t\tgoto dir_check;\n+\t\t\t\t}\n \t\t\t\t/* only error if existence is expected. */\n \t\t\t\tif (!(modes[i] & SPARSE))\n \t\t\t\t\tbad = _(\"bad source\");\n \t\t\t\tgoto act_on_entry;\n \t\t\t}\n-\n \t\t\tce = active_cache[pos];\n \t\t\tif (!ce_skip_worktree(ce)) {\n \t\t\t\tbad = _(\"bad source\");\n@@ -230,12 +266,14 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tbad = _(\"can not move directory into itself\");\n \t\t\tgoto act_on_entry;\n \t\t}\n-\t\tif ((src_is_dir = S_ISDIR(st.st_mode))\n+\t\tif (S_ISDIR(st.st_mode)\n \t\t && lstat(dst, &st) == 0) {\n \t\t\tbad = _(\"cannot move directory over file\");\n \t\t\tgoto act_on_entry;\n \t\t}\n-\t\tif (src_is_dir) {\n+\n+dir_check:\n+\t\tif (S_ISDIR(st.st_mode)) {\n \t\t\tint j, dst_len, n;\n \t\t\tint first = cache_name_pos(src, length), last;\n \n@@ -369,7 +407,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\tprintf(_(\"Renaming %s to %s\\n\"), src, dst);\n \t\tif (show_only)\n \t\t\tcontinue;\n-\t\tif (!(mode & (INDEX | SPARSE)) &&\n+\t\tif (!(mode & (INDEX | SPARSE | SKIP_WORKTREE_DIR)) &&\n \t\t rename(src, dst) < 0) {\n \t\t\tif (ignore_errors)\n \t\t\t\tcontinue;\n@@ -384,7 +422,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)\n \t\t\t\t\t\t\t 1);\n \t\t}\n \n-\t\tif (mode & (WORKING_DIRECTORY))\n+\t\tif (mode & (WORKING_DIRECTORY | SKIP_WORKTREE_DIR))\n \t\t\tcontinue;\n \n \t\tpos = cache_name_pos(src, strlen(src));\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 5eef799e25..763c6cc684 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -1828,7 +1828,7 @@ test_expect_success 'checkout behaves oddly with df-conflict-2' '\n \ttest_cmp full-checkout-err sparse-index-err\n '\n \n-test_expect_failure 'mv directory from out-of-cone to in-cone' '\n+test_expect_success 'mv directory from out-of-cone to in-cone' '\n \tinit_repos &&\n \n \t# as a sparse directory (or SKIP_WORKTREE_DIR without enabling\ndiff --git a/t/t7002-mv-sparse-checkout.sh b/t/t7002-mv-sparse-checkout.sh\nindex 6d2fb4f8d2..71fe29690f 100755\n--- a/t/t7002-mv-sparse-checkout.sh\n+++ b/t/t7002-mv-sparse-checkout.sh\n@@ -219,7 +219,7 @@ test_expect_success 'refuse to move file to non-skip-worktree sparse path' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n+test_expect_success 'refuse to move out-of-cone directory without --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n@@ -230,7 +230,7 @@ test_expect_failure 'refuse to move out-of-cone directory without --sparse' '\n \ttest_cmp expect stderr\n '\n \n-test_expect_failure 'can move out-of-cone directory with --sparse' '\n+test_expect_success 'can move out-of-cone directory with --sparse' '\n \ttest_when_finished \"cleanup_sparse_checkout\" &&\n \tsetup_sparse_checkout &&\n \n-- \n2.35.1\n\n"},{"id":"458411","messageId":"b06e2765-940e-0157-4eab-4faf8aefa2f4@github.com","threadId":"57642","inReplyTo":"20220630023737.473690-1-shaoxuan.yuan02@gmail.com","subject":"Re: [PATCH v5 0/8] mv: fix out-of-cone file/directory move logic","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-07-01T19:43:26Z","receivedAt":"2022-07-01T19:43:33Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 6/29/2022 10:37 PM, Shaoxuan Yuan wrote:\n> ## Changes since PATCH v4 ##\n> \n> 1. Fix style-nits.\n> \n> 2. Add t1092 tests (2/8) for \"mv: add check_dir_in_index() and solve \n> general dir check issue\" (8/8).\n\nThank you for these updates. I just took a quick\nre-read and I'm happy with this version.\n\nThanks,\n-Stolee\n"},{"id":"458424","messageId":"xmqq35fk7ba4.fsf@gitster.g","threadId":"57642","inReplyTo":"b06e2765-940e-0157-4eab-4faf8aefa2f4@github.com","subject":"Re: [PATCH v5 0/8] mv: fix out-of-cone file/directory move logic","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-01T21:50:27Z","receivedAt":"2022-07-01T21:50:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 6/29/2022 10:37 PM, Shaoxuan Yuan wrote:\n>> ## Changes since PATCH v4 ##\n>> \n>> 1. Fix style-nits.\n>> \n>> 2. Add t1092 tests (2/8) for \"mv: add check_dir_in_index() and solve \n>> general dir check issue\" (8/8).\n>\n> Thank you for these updates. I just took a quick\n> re-read and I'm happy with this version.\n>\n> Thanks,\n> -Stolee\n\nThanks, all. Will queue.\n"}]}