{"thread":{"id":"63760","subject":"[PATCH 0/3] sparse-checkout: add 'clean' command","startedAt":"2025-07-08T11:20:00Z","lastAt":"2025-10-24T02:22:48Z","messageCount":69,"participants":["Derrick Stolee via GitGitGadget","Patrick Steinhardt","Junio C Hamano","Elijah Newren","Derrick Stolee"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"521530","messageId":"pull.1941.git.1751973594.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":null,"subject":"[PATCH 0/3] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-08T11:19:50Z","receivedAt":"2025-07-08T11:20:00Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"When using cone-mode sparse-checkout, users specify which tracked\ndirectories they want (recursively) and any directory not part of the parent\npaths for those directories are considered \"out of scope\". When changing\nsparse-checkouts, there are a variety of reasons why these \"out of scope\"\ndirectories could remain, including:\n\n * The user has .gitignore or .git/info/exclude files that tell Git to not\n remove files of a certain type.\n * Some filesystem blocker prevented the removal of a tracked file. This is\n usually more of an issue on Windows where a read handle will block file\n deletion.\n\nTypically, this would not mean too much for the user experience. A few extra\nfilesystem checks might be required to satisfy git status commands, but the\nscope of the performance hit is relative to how many cruft files are left\nover in this situation.\n\nHowever, when using the sparse index, these tracked sparse directories cause\nsignificant performance issues. When noticing that the index contains a\nsparse directory but that directory exists on disk, Git needs to expand that\nsparse directory to determine which files are tracked or untracked. The\ncurrent mechanism expands the entire index to a full one, an expensive\noperation that scales with the total number of paths at HEAD and not just\nthe number of cruft files left over.\n\nAdvice was added in 9479a31d603 (advice: warn when sparse index expands,\n2024-07-08) to help users determine that they were in this state. However,\nthe advice doesn't actually recommend helpful ways to get out of this state.\nRecommending \"git clean\" on its own is incomplete, as typically users\nactually need 'git clean -dfx' to clear out the ignored or excluded files.\nEven then, they may need 'git sparse-checkout reapply' afterwards to clear\nthe sparse directories.\n\nThe advice was successful in helping to alert users to the problem, which is\nhow I got wind of many of these cases for how users get into this state.\nIt's now time to give them a tool that helps them out of this state.\n\nThis series adds a new 'git sparse-checkout clean' command that currently\nonly works for cone-mode sparse-checkouts. The only thing it does is\ncollapse the index to a sparse index (as much as possible) and make sure\nthat any sparse directories are removed. These directories are listed to\nstdout.\n\nA --dry-run option is available to list the directories that would be\nremoved without actually deleting the directories.\n\nThis option would be preferred to something like 'git clean -dfx' since it\ndoes not clear the excluded files that are still within the sparse-checkout.\nInstead, it performs the exact filesystem operations required to refresh the\nsparse index performance back to what is expected.\n\nI spent a few weeks debating with myself about whether or not this was the\nright interface, so please suggest alternatives if you have better ideas.\nAmong my rejected ideas include:\n\n * 'git sparse-checkout reapply -f -x' or similar augmentations of\n 'reapply'.\n * 'git clean --sparse' to focus the clean operation on things outside of\n the sparse-checkout.\n\nThe implementation is rather simple with the current CLI. Future\naugmentations could include a --quiet option to silence the output and a\n--verbose option to list the files that exist within each directory and\nwould/will be removed.\n\nThanks, -Stolee\n\nDerrick Stolee (3):\n sparse-checkout: remove use of the_repository\n sparse-checkout: add 'clean' command\n sparse-index: point users to new 'clean' action\n\n Documentation/git-sparse-checkout.adoc | 13 +-\n builtin/sparse-checkout.c | 192 +++++++++++++++++--------\n sparse-index.c | 3 +-\n t/t1091-sparse-checkout-builtin.sh | 48 +++++++\n 4 files changed, 197 insertions(+), 59 deletions(-)\n\n\nbase-commit: 8b6f19ccfc3aefbd0f22f6b7d56ad6a3fc5e4f37\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1941%2Fderrickstolee%2Fgit-sparse-checkout-clean-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1941/derrickstolee/git-sparse-checkout-clean-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1941\n-- \ngitgitgadget\n"},{"id":"521531","messageId":"3cdc44a9e8c8e6718b24bc25def291348a0b2613.1751973594.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"[PATCH 1/3] sparse-checkout: remove use of the_repository","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-08T11:19:51Z","receivedAt":"2025-07-08T11:20:05Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe logic for the 'git sparse-checkout' builtin uses the_repository all\nover the place, despite some use of a repository struct in different\nmethod parameters. Complete this removal of the_repository by using\n'repo' when possible.\n\nIn one place, there was already a local variable 'r' that was set to\nthe_repository, so move that to a method parameter.\n\nWe cannot remove the USE_THE_REPOSITORY_VARIABLE declaration as we are\nstill using global constants for the state of the sparse-checkout.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/sparse-checkout.c | 119 ++++++++++++++++++++------------------\n 1 file changed, 63 insertions(+), 56 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 1bf01591b275..8b70d0c6a441 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -204,12 +204,12 @@ static void clean_tracked_sparse_directories(struct repository *r)\n \t\tensure_full_index(r->index);\n }\n \n-static int update_working_directory(struct pattern_list *pl)\n+static int update_working_directory(struct repository *r,\n+\t\t\t\t struct pattern_list *pl)\n {\n \tenum update_sparsity_result result;\n \tstruct unpack_trees_options o;\n \tstruct lock_file lock_file = LOCK_INIT;\n-\tstruct repository *r = the_repository;\n \tstruct pattern_list *old_pl;\n \n \t/* If no branch has been checked out, there are no updates to make. */\n@@ -327,7 +327,8 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)\n \tstring_list_clear(&sl, 0);\n }\n \n-static int write_patterns_and_update(struct pattern_list *pl)\n+static int write_patterns_and_update(struct repository *repo,\n+\t\t\t\t struct pattern_list *pl)\n {\n \tchar *sparse_filename;\n \tFILE *fp;\n@@ -336,15 +337,15 @@ static int write_patterns_and_update(struct pattern_list *pl)\n \n \tsparse_filename = get_sparse_checkout_filename();\n \n-\tif (safe_create_leading_directories(the_repository, sparse_filename))\n+\tif (safe_create_leading_directories(repo, sparse_filename))\n \t\tdie(_(\"failed to create directory for sparse-checkout file\"));\n \n \thold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR);\n \n-\tresult = update_working_directory(pl);\n+\tresult = update_working_directory(repo, pl);\n \tif (result) {\n \t\trollback_lock_file(&lk);\n-\t\tupdate_working_directory(NULL);\n+\t\tupdate_working_directory(repo, NULL);\n \t\tgoto out;\n \t}\n \n@@ -372,25 +373,26 @@ enum sparse_checkout_mode {\n \tMODE_CONE_PATTERNS = 2,\n };\n \n-static int set_config(enum sparse_checkout_mode mode)\n+static int set_config(struct repository *repo,\n+\t\t enum sparse_checkout_mode mode)\n {\n \t/* Update to use worktree config, if not already. */\n-\tif (init_worktree_config(the_repository)) {\n+\tif (init_worktree_config(repo)) {\n \t\terror(_(\"failed to initialize worktree config\"));\n \t\treturn 1;\n \t}\n \n-\tif (repo_config_set_worktree_gently(the_repository,\n+\tif (repo_config_set_worktree_gently(repo,\n \t\t\t\t\t \"core.sparseCheckout\",\n \t\t\t\t\t mode ? \"true\" : \"false\") ||\n-\t repo_config_set_worktree_gently(the_repository,\n+\t repo_config_set_worktree_gently(repo,\n \t\t\t\t\t \"core.sparseCheckoutCone\",\n \t\t\t\t\t mode == MODE_CONE_PATTERNS ?\n \t\t\t\t\t\t\"true\" : \"false\"))\n \t\treturn 1;\n \n \tif (mode == MODE_NO_PATTERNS)\n-\t\treturn set_sparse_index_config(the_repository, 0);\n+\t\treturn set_sparse_index_config(repo, 0);\n \n \treturn 0;\n }\n@@ -410,7 +412,7 @@ static enum sparse_checkout_mode update_cone_mode(int *cone_mode) {\n \treturn MODE_ALL_PATTERNS;\n }\n \n-static int update_modes(int *cone_mode, int *sparse_index)\n+static int update_modes(struct repository *repo, int *cone_mode, int *sparse_index)\n {\n \tint mode, record_mode;\n \n@@ -418,20 +420,20 @@ static int update_modes(int *cone_mode, int *sparse_index)\n \trecord_mode = (*cone_mode != -1) || !core_apply_sparse_checkout;\n \n \tmode = update_cone_mode(cone_mode);\n-\tif (record_mode && set_config(mode))\n+\tif (record_mode && set_config(repo, mode))\n \t\treturn 1;\n \n \t/* Set sparse-index/non-sparse-index mode if specified */\n \tif (*sparse_index >= 0) {\n-\t\tif (set_sparse_index_config(the_repository, *sparse_index) < 0)\n+\t\tif (set_sparse_index_config(repo, *sparse_index) < 0)\n \t\t\tdie(_(\"failed to modify sparse-index config\"));\n \n \t\t/* force an index rewrite */\n-\t\trepo_read_index(the_repository);\n-\t\tthe_repository->index->updated_workdir = 1;\n+\t\trepo_read_index(repo);\n+\t\trepo->index->updated_workdir = 1;\n \n \t\tif (!*sparse_index)\n-\t\t\tensure_full_index(the_repository->index);\n+\t\t\tensure_full_index(repo->index);\n \t}\n \n \treturn 0;\n@@ -448,7 +450,7 @@ static struct sparse_checkout_init_opts {\n } init_opts;\n \n static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n-\t\t\t\tstruct repository *repo UNUSED)\n+\t\t\t\tstruct repository *repo)\n {\n \tstruct pattern_list pl;\n \tchar *sparse_filename;\n@@ -464,7 +466,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \t};\n \n \tsetup_work_tree();\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tinit_opts.cone_mode = -1;\n \tinit_opts.sparse_index = -1;\n@@ -473,7 +475,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \t\t\t builtin_sparse_checkout_init_options,\n \t\t\t builtin_sparse_checkout_init_usage, 0);\n \n-\tif (update_modes(&init_opts.cone_mode, &init_opts.sparse_index))\n+\tif (update_modes(repo, &init_opts.cone_mode, &init_opts.sparse_index))\n \t\treturn 1;\n \n \tmemset(&pl, 0, sizeof(pl));\n@@ -485,14 +487,14 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \tif (res >= 0) {\n \t\tfree(sparse_filename);\n \t\tclear_pattern_list(&pl);\n-\t\treturn update_working_directory(NULL);\n+\t\treturn update_working_directory(repo, NULL);\n \t}\n \n-\tif (repo_get_oid(the_repository, \"HEAD\", &oid)) {\n+\tif (repo_get_oid(repo, \"HEAD\", &oid)) {\n \t\tFILE *fp;\n \n \t\t/* assume we are in a fresh repo, but update the sparse-checkout file */\n-\t\tif (safe_create_leading_directories(the_repository, sparse_filename))\n+\t\tif (safe_create_leading_directories(repo, sparse_filename))\n \t\t\tdie(_(\"unable to create leading directories of %s\"),\n \t\t\t sparse_filename);\n \t\tfp = xfopen(sparse_filename, \"w\");\n@@ -511,7 +513,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \tadd_pattern(\"!/*/\", empty_base, 0, &pl, 0);\n \tpl.use_cone_patterns = init_opts.cone_mode;\n \n-\treturn write_patterns_and_update(&pl);\n+\treturn write_patterns_and_update(repo, &pl);\n }\n \n static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *path)\n@@ -674,7 +676,8 @@ static void add_patterns_literal(int argc, const char **argv,\n \tadd_patterns_from_input(pl, argc, argv, use_stdin ? stdin : NULL);\n }\n \n-static int modify_pattern_list(struct strvec *args, int use_stdin,\n+static int modify_pattern_list(struct repository *repo,\n+\t\t\t struct strvec *args, int use_stdin,\n \t\t\t enum modify_type m)\n {\n \tint result;\n@@ -696,22 +699,23 @@ static int modify_pattern_list(struct strvec *args, int use_stdin,\n \t}\n \n \tif (!core_apply_sparse_checkout) {\n-\t\tset_config(MODE_ALL_PATTERNS);\n+\t\tset_config(repo, MODE_ALL_PATTERNS);\n \t\tcore_apply_sparse_checkout = 1;\n \t\tchanged_config = 1;\n \t}\n \n-\tresult = write_patterns_and_update(pl);\n+\tresult = write_patterns_and_update(repo, pl);\n \n \tif (result && changed_config)\n-\t\tset_config(MODE_NO_PATTERNS);\n+\t\tset_config(repo, MODE_NO_PATTERNS);\n \n \tclear_pattern_list(pl);\n \tfree(pl);\n \treturn result;\n }\n \n-static void sanitize_paths(struct strvec *args,\n+static void sanitize_paths(struct repository *repo,\n+\t\t\t struct strvec *args,\n \t\t\t const char *prefix, int skip_checks)\n {\n \tint i;\n@@ -752,7 +756,7 @@ static void sanitize_paths(struct strvec *args,\n \n \tfor (i = 0; i < args->nr; i++) {\n \t\tstruct cache_entry *ce;\n-\t\tstruct index_state *index = the_repository->index;\n+\t\tstruct index_state *index = repo->index;\n \t\tint pos = index_name_pos(index, args->v[i], strlen(args->v[i]));\n \n \t\tif (pos < 0)\n@@ -779,7 +783,7 @@ static struct sparse_checkout_add_opts {\n } add_opts;\n \n static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n-\t\t\t struct repository *repo UNUSED)\n+\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_add_options[] = {\n \t\tOPT_BOOL_F(0, \"skip-checks\", &add_opts.skip_checks,\n@@ -796,7 +800,7 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n \tif (!core_apply_sparse_checkout)\n \t\tdie(_(\"no sparse-checkout to add to\"));\n \n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \targc = parse_options(argc, argv, prefix,\n \t\t\t builtin_sparse_checkout_add_options,\n@@ -804,9 +808,9 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n \n \tfor (int i = 0; i < argc; i++)\n \t\tstrvec_push(&patterns, argv[i]);\n-\tsanitize_paths(&patterns, prefix, add_opts.skip_checks);\n+\tsanitize_paths(repo, &patterns, prefix, add_opts.skip_checks);\n \n-\tret = modify_pattern_list(&patterns, add_opts.use_stdin, ADD);\n+\tret = modify_pattern_list(repo, &patterns, add_opts.use_stdin, ADD);\n \n \tstrvec_clear(&patterns);\n \treturn ret;\n@@ -825,7 +829,7 @@ static struct sparse_checkout_set_opts {\n } set_opts;\n \n static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n-\t\t\t struct repository *repo UNUSED)\n+\t\t\t struct repository *repo)\n {\n \tint default_patterns_nr = 2;\n \tconst char *default_patterns[] = {\"/*\", \"!/*/\", NULL};\n@@ -847,7 +851,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \tint ret;\n \n \tsetup_work_tree();\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tset_opts.cone_mode = -1;\n \tset_opts.sparse_index = -1;\n@@ -856,7 +860,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \t\t\t builtin_sparse_checkout_set_options,\n \t\t\t builtin_sparse_checkout_set_usage, 0);\n \n-\tif (update_modes(&set_opts.cone_mode, &set_opts.sparse_index))\n+\tif (update_modes(repo, &set_opts.cone_mode, &set_opts.sparse_index))\n \t\treturn 1;\n \n \t/*\n@@ -870,10 +874,10 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \t} else {\n \t\tfor (int i = 0; i < argc; i++)\n \t\t\tstrvec_push(&patterns, argv[i]);\n-\t\tsanitize_paths(&patterns, prefix, set_opts.skip_checks);\n+\t\tsanitize_paths(repo, &patterns, prefix, set_opts.skip_checks);\n \t}\n \n-\tret = modify_pattern_list(&patterns, set_opts.use_stdin, REPLACE);\n+\tret = modify_pattern_list(repo, &patterns, set_opts.use_stdin, REPLACE);\n \n \tstrvec_clear(&patterns);\n \treturn ret;\n@@ -891,7 +895,7 @@ static struct sparse_checkout_reapply_opts {\n \n static int sparse_checkout_reapply(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_reapply_options[] = {\n \t\tOPT_BOOL(0, \"cone\", &reapply_opts.cone_mode,\n@@ -912,12 +916,12 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n \t\t\t builtin_sparse_checkout_reapply_options,\n \t\t\t builtin_sparse_checkout_reapply_usage, 0);\n \n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n-\tif (update_modes(&reapply_opts.cone_mode, &reapply_opts.sparse_index))\n+\tif (update_modes(repo, &reapply_opts.cone_mode, &reapply_opts.sparse_index))\n \t\treturn 1;\n \n-\treturn update_working_directory(NULL);\n+\treturn update_working_directory(repo, NULL);\n }\n \n static char const * const builtin_sparse_checkout_disable_usage[] = {\n@@ -927,7 +931,7 @@ static char const * const builtin_sparse_checkout_disable_usage[] = {\n \n static int sparse_checkout_disable(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_disable_options[] = {\n \t\tOPT_END(),\n@@ -955,7 +959,7 @@ static int sparse_checkout_disable(int argc, const char **argv,\n \t * are expecting to do that when disabling sparse-checkout.\n \t */\n \tgive_advice_on_expansion = 0;\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tmemset(&pl, 0, sizeof(pl));\n \thashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);\n@@ -965,14 +969,14 @@ static int sparse_checkout_disable(int argc, const char **argv,\n \n \tadd_pattern(\"/*\", empty_base, 0, &pl, 0);\n \n-\tprepare_repo_settings(the_repository);\n-\tthe_repository->settings.sparse_index = 0;\n+\tprepare_repo_settings(repo);\n+\trepo->settings.sparse_index = 0;\n \n-\tif (update_working_directory(&pl))\n+\tif (update_working_directory(repo, &pl))\n \t\tdie(_(\"error while refreshing working directory\"));\n \n \tclear_pattern_list(&pl);\n-\treturn set_config(MODE_NO_PATTERNS);\n+\treturn set_config(repo, MODE_NO_PATTERNS);\n }\n \n static char const * const builtin_sparse_checkout_check_rules_usage[] = {\n@@ -987,14 +991,17 @@ static struct sparse_checkout_check_rules_opts {\n \tchar *rules_file;\n } check_rules_opts;\n \n-static int check_rules(struct pattern_list *pl, int null_terminated) {\n+static int check_rules(struct repository *repo,\n+\t\t struct pattern_list *pl,\n+\t\t int null_terminated)\n+{\n \tstruct strbuf line = STRBUF_INIT;\n \tstruct strbuf unquoted = STRBUF_INIT;\n \tchar *path;\n \tint line_terminator = null_terminated ? 0 : '\\n';\n \tstrbuf_getline_fn getline_fn = null_terminated ? strbuf_getline_nul\n \t\t: strbuf_getline;\n-\tthe_repository->index->sparse_checkout_patterns = pl;\n+\trepo->index->sparse_checkout_patterns = pl;\n \twhile (!getline_fn(&line, stdin)) {\n \t\tpath = line.buf;\n \t\tif (!null_terminated && line.buf[0] == '\"') {\n@@ -1006,7 +1013,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n \t\t\tpath = unquoted.buf;\n \t\t}\n \n-\t\tif (path_in_sparse_checkout(path, the_repository->index))\n+\t\tif (path_in_sparse_checkout(path, repo->index))\n \t\t\twrite_name_quoted(path, stdout, line_terminator);\n \t}\n \tstrbuf_release(&line);\n@@ -1016,7 +1023,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n }\n \n static int sparse_checkout_check_rules(int argc, const char **argv, const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_check_rules_options[] = {\n \t\tOPT_BOOL('z', NULL, &check_rules_opts.null_termination,\n@@ -1055,7 +1062,7 @@ static int sparse_checkout_check_rules(int argc, const char **argv, const char *\n \t\tfree(sparse_filename);\n \t}\n \n-\tret = check_rules(&pl, check_rules_opts.null_termination);\n+\tret = check_rules(repo, &pl, check_rules_opts.null_termination);\n \tclear_pattern_list(&pl);\n \tfree(check_rules_opts.rules_file);\n \treturn ret;\n@@ -1084,8 +1091,8 @@ int cmd_sparse_checkout(int argc,\n \n \tgit_config(git_default_config, NULL);\n \n-\tprepare_repo_settings(the_repository);\n-\tthe_repository->settings.command_requires_full_index = 0;\n+\tprepare_repo_settings(repo);\n+\trepo->settings.command_requires_full_index = 0;\n \n \treturn fn(argc, argv, prefix, repo);\n }\n-- \ngitgitgadget\n\n"},{"id":"521532","messageId":"49418e8ec8a4c3e0ce9c65aa700042b6f3f3f4d7.1751973594.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"[PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-08T11:19:52Z","receivedAt":"2025-07-08T11:20:08Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nWhen users change their sparse-checkout definitions to add new\ndirectories and remove old ones, there may be a few reasons why\ndirectories no longer in scope remain (ignored or excluded files still\nexist, Windows handles are still open, etc.). When these files still\nexist, the sparse index feature notices that a tracked, but sparse,\ndirectory still exists on disk and thus the index expands. This causes a\nperformance hit _and_ the advice printed isn't very helpful. Using 'git\nclean' isn't enough (generally '-dfx' may be needed) but also this may\nnot be sufficient.\n\nAdd a new subcommand to 'git sparse-checkout' that removes these\ntracked-but-sparse directories, including any excluded or ignored files\nunderneath. This is the most extreme method for doing this, but it works\nwhen the sparse-checkout is in cone mode and is expected to rescope\nbased on directories, not files.\n\nBe sure to add a --dry-run option so users can predict what will be\ndeleted. In general, output the directories that are being removed so\nusers can know what was removed.\n\nNote that untracked directories remain. Further, directories that\ncontain staged changes are not deleted. This is a detail that is partly\nhidden by the implementation which relies on collapsing the index to a\nsparse index in-memory and only deleting directories that are listed as\nsparse in the index. If a staged change exists, then that entry is not\nstored as a sparse tree entry and thus remains on-disk until committed\nor reset.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 13 ++++-\n builtin/sparse-checkout.c | 73 +++++++++++++++++++++++++-\n t/t1091-sparse-checkout-builtin.sh | 48 +++++++++++++++++\n 3 files changed, 132 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex 529a8edd9c1e..21ba6f759905 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -9,7 +9,7 @@ git-sparse-checkout - Reduce your working tree to a subset of tracked files\n SYNOPSIS\n --------\n [verse]\n-'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) []\n+'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules | clean) []\n \n \n DESCRIPTION\n@@ -111,6 +111,17 @@ flags, with the same meaning as the flags from the `set` command, in order\n to change which sparsity mode you are using without needing to also respecify\n all sparsity paths.\n \n+'clean'::\n+\tRemove all files in tracked directories that are outside of the\n+\tsparse-checkout definition. This subcommand requires cone-mode\n+\tsparse-checkout to be sure that we know which directories are\n+\tboth tracked and all contained paths are not in the sparse-checkout.\n+\tThis command can be used to be sure the sparse index works\n+\tefficiently.\n++\n+The `clean` command can also take the `--dry-run` (`-n`) option to list\n+the directories it would remove without performing any filesystem changes.\n+\n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\n \tworking directory to include all files.\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 8b70d0c6a441..6d2843827367 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -23,7 +23,7 @@\n static const char *empty_base = \"\";\n \n static char const * const builtin_sparse_checkout_usage[] = {\n-\tN_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules) []\"),\n+\tN_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules | clean) []\"),\n \tNULL\n };\n \n@@ -924,6 +924,76 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n \treturn update_working_directory(repo, NULL);\n }\n \n+static char const * const builtin_sparse_checkout_clean_usage[] = {\n+\t\"git sparse-checkout clean [-n|--dry-run]\",\n+\tNULL\n+};\n+\n+static struct sparse_checkout_clean_opts {\n+\tint dry_run;\n+} clean_opts;\n+\n+static int sparse_checkout_clean(int argc, const char **argv,\n+\t\t\t\t const char *prefix,\n+\t\t\t\t struct repository *repo)\n+{\n+\tstruct strbuf full_path = STRBUF_INIT;\n+\tsize_t worktree_len;\n+\tstatic struct option builtin_sparse_checkout_clean_options[] = {\n+\t\tOPT_BOOL('n', \"dry-run\", &clean_opts.dry_run,\n+\t\t\t N_(\"list the directories that would be removed without making filesystem changes\")),\n+\t\tOPT_END(),\n+\t};\n+\n+\tsetup_work_tree();\n+\tif (!core_apply_sparse_checkout)\n+\t\tdie(_(\"must be in a sparse-checkout to clean directories\"));\n+\tif (!core_sparse_checkout_cone)\n+\t\tdie(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n+\n+\targc = parse_options(argc, argv, prefix,\n+\t\t\t builtin_sparse_checkout_clean_options,\n+\t\t\t builtin_sparse_checkout_clean_usage, 0);\n+\n+\tif (repo_read_index(repo) < 0)\n+\t\tdie(_(\"failed to read index\"));\n+\n+\tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY))\n+\t\tdie(_(\"failed to convert index to a sparse index\"));\n+\n+\tstrbuf_addstr(&full_path, repo->worktree);\n+\tstrbuf_addch(&full_path, '/');\n+\tworktree_len = full_path.len;\n+\n+\tfor (size_t i = 0; i < repo->index->cache_nr; i++) {\n+\t\tDIR* dir;\n+\t\tstruct cache_entry *ce = repo->index->cache[i];\n+\t\tif (!S_ISSPARSEDIR(ce->ce_mode))\n+\t\t\tcontinue;\n+\t\tstrbuf_setlen(&full_path, worktree_len);\n+\t\tstrbuf_add(&full_path, ce->name, ce->ce_namelen);\n+\n+\t\tdir = opendir(full_path.buf);\n+\t\tif (!dir)\n+\t\t\tcontinue;\n+\t\telse if (ENOENT != errno) {\n+\t\t\twarning_errno(_(\"failed to check for existence of '%s'\"), ce->name);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tclosedir(dir);\n+\n+\t\tprintf(\"%s\\n\", ce->name);\n+\t\tif (!clean_opts.dry_run) {\n+\t\t\tif (remove_dir_recursively(&full_path, 0))\n+\t\t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n+\t\t}\n+\t}\n+\n+\tstrbuf_release(&full_path);\n+\treturn 0;\n+}\n+\n static char const * const builtin_sparse_checkout_disable_usage[] = {\n \t\"git sparse-checkout disable\",\n \tNULL\n@@ -1080,6 +1150,7 @@ int cmd_sparse_checkout(int argc,\n \t\tOPT_SUBCOMMAND(\"set\", &fn, sparse_checkout_set),\n \t\tOPT_SUBCOMMAND(\"add\", &fn, sparse_checkout_add),\n \t\tOPT_SUBCOMMAND(\"reapply\", &fn, sparse_checkout_reapply),\n+\t\tOPT_SUBCOMMAND(\"clean\", &fn, sparse_checkout_clean),\n \t\tOPT_SUBCOMMAND(\"disable\", &fn, sparse_checkout_disable),\n \t\tOPT_SUBCOMMAND(\"check-rules\", &fn, sparse_checkout_check_rules),\n \t\tOPT_END(),\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex ab3a105ffff2..7f8a444541f7 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1050,5 +1050,53 @@ test_expect_success 'check-rules null termination' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'clean' '\n+\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n+\tmkdir repo/deep/deeper2 repo/folder1 &&\n+\ttouch repo/deep/deeper2/file &&\n+\ttouch repo/folder1/file &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tdeep/deeper2/\n+\tfolder1/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run >out &&\n+\ttest_cmp expect out &&\n+\n+\ttest_path_exists repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1 &&\n+\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_cmp expect out &&\n+\n+\t! test_path_exists repo/deep/deeper2 &&\n+\t! test_path_exists repo/folder1\n+'\n+\n+test_expect_success 'clean with staged sparse change' '\n+\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n+\tmkdir repo/deep/deeper2 repo/folder1 &&\n+\ttouch repo/deep/deeper2/file &&\n+\ttouch repo/folder1/file &&\n+\n+\tgit -C repo add --sparse folder1/file &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tdeep/deeper2/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run >out &&\n+\ttest_cmp expect out &&\n+\n+\ttest_path_exists repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1 &&\n+\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_cmp expect out &&\n+\n+\t! test_path_exists repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1\n+'\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"521533","messageId":"80d7a7641daa7a6f0e1db73e0a433701e9f37209.1751973594.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"[PATCH 3/3] sparse-index: point users to new 'clean' action","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-08T11:19:53Z","receivedAt":"2025-07-08T11:20:10Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nIn my experience, the most-common reason that the sparse index must\nexpand to a full one is because there is some leftover file in a tracked\ndirectory that is now outside of the sparse-checkout. The new 'git\nsparse-checkout clean' command will find and delete these directories,\nso point users to it when they hit the sparse index expansion advice.\n\nSigned-off-by: Derrick Stolee \n---\n sparse-index.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/sparse-index.c b/sparse-index.c\nindex 5634abafaa07..5d14795063b5 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -32,7 +32,8 @@ int give_advice_on_expansion = 1;\n \t\"Your working directory likely has contents that are outside of\\n\" \\\n \t\"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n \t\"see your sparse-checkout definition and compare it to your working\\n\" \\\n-\t\"directory contents. Running 'git clean' may assist in this cleanup.\"\n+\t\"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n+\t\"in this cleanup.\"\n \n struct modify_index_context {\n \tstruct index_state *write;\n-- \ngitgitgadget\n"},{"id":"521538","messageId":"aG0LyDAUSM7F7OmH@pks.im","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] sparse-checkout: add 'clean' command","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-08T12:15:04Z","receivedAt":"2025-07-08T12:15:11Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Jul 08, 2025 at 11:19:50AM +0000, Derrick Stolee via GitGitGadget wrote:\n> When using cone-mode sparse-checkout, users specify which tracked\n> directories they want (recursively) and any directory not part of the parent\n> paths for those directories are considered \"out of scope\". When changing\n> sparse-checkouts, there are a variety of reasons why these \"out of scope\"\n> directories could remain, including:\n> \n> * The user has .gitignore or .git/info/exclude files that tell Git to not\n> remove files of a certain type.\n> * Some filesystem blocker prevented the removal of a tracked file. This is\n> usually more of an issue on Windows where a read handle will block file\n> deletion.\n> \n> Typically, this would not mean too much for the user experience. A few extra\n> filesystem checks might be required to satisfy git status commands, but the\n> scope of the performance hit is relative to how many cruft files are left\n> over in this situation.\n> \n> However, when using the sparse index, these tracked sparse directories cause\n> significant performance issues. When noticing that the index contains a\n> sparse directory but that directory exists on disk, Git needs to expand that\n> sparse directory to determine which files are tracked or untracked. The\n> current mechanism expands the entire index to a full one, an expensive\n> operation that scales with the total number of paths at HEAD and not just\n> the number of cruft files left over.\n> \n> Advice was added in 9479a31d603 (advice: warn when sparse index expands,\n> 2024-07-08) to help users determine that they were in this state. However,\n> the advice doesn't actually recommend helpful ways to get out of this state.\n> Recommending \"git clean\" on its own is incomplete, as typically users\n> actually need 'git clean -dfx' to clear out the ignored or excluded files.\n> Even then, they may need 'git sparse-checkout reapply' afterwards to clear\n> the sparse directories.\n> \n> The advice was successful in helping to alert users to the problem, which is\n> how I got wind of many of these cases for how users get into this state.\n> It's now time to give them a tool that helps them out of this state.\n\nAs usual for you, this is a nicely-written summary of how we got here\nand why the current mechanisms are insufficient for mere mortals.\n\n> This series adds a new 'git sparse-checkout clean' command that currently\n> only works for cone-mode sparse-checkouts. The only thing it does is\n> collapse the index to a sparse index (as much as possible) and make sure\n> that any sparse directories are removed. These directories are listed to\n> stdout.\n> \n> A --dry-run option is available to list the directories that would be\n> removed without actually deleting the directories.\n> \n> This option would be preferred to something like 'git clean -dfx' since it\n> does not clear the excluded files that are still within the sparse-checkout.\n> Instead, it performs the exact filesystem operations required to refresh the\n> sparse index performance back to what is expected.\n> \n> I spent a few weeks debating with myself about whether or not this was the\n> right interface, so please suggest alternatives if you have better ideas.\n> Among my rejected ideas include:\n> \n> * 'git sparse-checkout reapply -f -x' or similar augmentations of\n> 'reapply'.\n> * 'git clean --sparse' to focus the clean operation on things outside of\n> the sparse-checkout.\n> \n> The implementation is rather simple with the current CLI. Future\n> augmentations could include a --quiet option to silence the output and a\n> --verbose option to list the files that exist within each directory and\n> would/will be removed.\n\nOne of the benefits of your new command is that we can extend it in the\nfuture as necessary if we ever notice that there are other things that\nwe need to do to bring the sparse checkout up to date again. So without\nyet having had a look at the implementation I think this direction is\nquite sensible.\n\nIdeally it would of course be great if we could automatically fix the\nissue for our users. But as we have to prune potentially-ignored data it\nis very much a no-go to do that in the automatically.\n\nPatrick\n"},{"id":"521539","messageId":"aG0LzlnfoQVDdetk@pks.im","threadId":"63760","inReplyTo":"49418e8ec8a4c3e0ce9c65aa700042b6f3f3f4d7.1751973594.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-08T12:15:10Z","receivedAt":"2025-07-08T12:15:15Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Jul 08, 2025 at 11:19:52AM +0000, Derrick Stolee via GitGitGadget wrote:\n> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n> index 529a8edd9c1e..21ba6f759905 100644\n> --- a/Documentation/git-sparse-checkout.adoc\n> +++ b/Documentation/git-sparse-checkout.adoc\n> @@ -111,6 +111,17 @@ flags, with the same meaning as the flags from the `set` command, in order\n> to change which sparsity mode you are using without needing to also respecify\n> all sparsity paths.\n> \n> +'clean'::\n> +\tRemove all files in tracked directories that are outside of the\n> +\tsparse-checkout definition. This subcommand requires cone-mode\n> +\tsparse-checkout to be sure that we know which directories are\n> +\tboth tracked and all contained paths are not in the sparse-checkout.\n> +\tThis command can be used to be sure the sparse index works\n> +\tefficiently.\n> ++\n> +The `clean` command can also take the `--dry-run` (`-n`) option to list\n> +the directories it would remove without performing any filesystem changes.\n> +\n\nHm. This is somewhat different from `git clean`, where you have to pass\n`-f` to make it delete any data. I'm not particularly a fan of that\nmode, but should we maybe retain it regardless to ensure that things are\nat least a tiny bit more consistent?\n\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index 8b70d0c6a441..6d2843827367 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -924,6 +924,76 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n> \treturn update_working_directory(repo, NULL);\n> }\n> \n> +static char const * const builtin_sparse_checkout_clean_usage[] = {\n> +\t\"git sparse-checkout clean [-n|--dry-run]\",\n> +\tNULL\n> +};\n> +\n> +static struct sparse_checkout_clean_opts {\n> +\tint dry_run;\n> +} clean_opts;\n> +\n> +static int sparse_checkout_clean(int argc, const char **argv,\n> +\t\t\t\t const char *prefix,\n> +\t\t\t\t struct repository *repo)\n> +{\n> +\tstruct strbuf full_path = STRBUF_INIT;\n> +\tsize_t worktree_len;\n> +\tstatic struct option builtin_sparse_checkout_clean_options[] = {\n> +\t\tOPT_BOOL('n', \"dry-run\", &clean_opts.dry_run,\n> +\t\t\t N_(\"list the directories that would be removed without making filesystem changes\")),\n> +\t\tOPT_END(),\n> +\t};\n> +\n> +\tsetup_work_tree();\n> +\tif (!core_apply_sparse_checkout)\n> +\t\tdie(_(\"must be in a sparse-checkout to clean directories\"));\n> +\tif (!core_sparse_checkout_cone)\n> +\t\tdie(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n> +\n> +\targc = parse_options(argc, argv, prefix,\n> +\t\t\t builtin_sparse_checkout_clean_options,\n> +\t\t\t builtin_sparse_checkout_clean_usage, 0);\n> +\n> +\tif (repo_read_index(repo) < 0)\n> +\t\tdie(_(\"failed to read index\"));\n> +\n> +\tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY))\n> +\t\tdie(_(\"failed to convert index to a sparse index\"));\n\nI noticed that there are several cases in `convert_to_sparse()` where we\nsimply do nothing. Should we check whether `repo->index->sparse_index`\nmatches `INDEX_COLLAPSED` after the operation?\n\n> +\tstrbuf_addstr(&full_path, repo->worktree);\n> +\tstrbuf_addch(&full_path, '/');\n> +\tworktree_len = full_path.len;\n> +\n> +\tfor (size_t i = 0; i < repo->index->cache_nr; i++) {\n> +\t\tDIR* dir;\n\nNit: the `*` goes with the variable, not the type.\n\n> +\t\tstruct cache_entry *ce = repo->index->cache[i];\n> +\t\tif (!S_ISSPARSEDIR(ce->ce_mode))\n> +\t\t\tcontinue;\n\nOkay, we only need to handle sparse directories.\n\n> +\t\tstrbuf_setlen(&full_path, worktree_len);\n> +\t\tstrbuf_add(&full_path, ce->name, ce->ce_namelen);\n> +\n> +\t\tdir = opendir(full_path.buf);\n\nShouldn't it be sufficient to use `is_directory()`?\n\n> +\t\tif (!dir)\n> +\t\t\tcontinue;\n\nThis is the good and expected case, right? The entry is sparse, so\nideally it doesn't exist. If it does we have to recurse into to end up\nwith the full index.\n\n> +\t\telse if (ENOENT != errno) {\n\nNit: style. If one branches requires curly braces, all branches should\nuse them.\n\n> +\t\t\twarning_errno(_(\"failed to check for existence of '%s'\"), ce->name);\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tclosedir(dir);\n> +\n> +\t\tprintf(\"%s\\n\", ce->name);\n\ngit-clean(1) says \"Removing %s\\n\". Should we do the same here?\n\n> +\t\tif (!clean_opts.dry_run) {\n> +\t\t\tif (remove_dir_recursively(&full_path, 0))\n> +\t\t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n> +\t\t}\n> +\t}\n> +\n> +\tstrbuf_release(&full_path);\n> +\treturn 0;\n> +}\n> +\n> static char const * const builtin_sparse_checkout_disable_usage[] = {\n> \t\"git sparse-checkout disable\",\n> \tNULL\n\nPatrick\n"},{"id":"521568","messageId":"xmqqv7o2togi.fsf@gitster.g","threadId":"63760","inReplyTo":"aG0LzlnfoQVDdetk@pks.im","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-08T20:30:37Z","receivedAt":"2025-07-08T20:30:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Tue, Jul 08, 2025 at 11:19:52AM +0000, Derrick Stolee via GitGitGadget wrote:\n>> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n>> index 529a8edd9c1e..21ba6f759905 100644\n>> --- a/Documentation/git-sparse-checkout.adoc\n>> +++ b/Documentation/git-sparse-checkout.adoc\n>> @@ -111,6 +111,17 @@ flags, with the same meaning as the flags from the `set` command, in order\n>> to change which sparsity mode you are using without needing to also respecify\n>> all sparsity paths.\n>> \n>> +'clean'::\n>> +\tRemove all files in tracked directories that are outside of the\n>> +\tsparse-checkout definition. This subcommand requires cone-mode\n>> +\tsparse-checkout to be sure that we know which directories are\n>> +\tboth tracked and all contained paths are not in the sparse-checkout.\n>> +\tThis command can be used to be sure the sparse index works\n>> +\tefficiently.\n>> ++\n>> +The `clean` command can also take the `--dry-run` (`-n`) option to list\n>> +the directories it would remove without performing any filesystem changes.\n>> +\n>\n> Hm. This is somewhat different from `git clean`, where you have to pass\n> `-f` to make it delete any data. I'm not particularly a fan of that\n> mode, but should we maybe retain it regardless to ensure that things are\n> at least a tiny bit more consistent?\n\nAh, it reminds me of my favorite \"regret\". We may want to consider\nmaking the --force/--dry-run used in \"git clean\" saner at a major\nversion boundary. I am not particulary a fan of that mode, and\nwould oppose a patch made as a part of regular \"let's change this,\nas I do not like it\" exercise, but as a known-breaking change, I do\nnot mind it at all. Essentially the change to propose would be to\ndeprecate clean.requireForce and internally make it a constant\nfalse.\n\nBut that is a tangent ;-)\n"},{"id":"521569","messageId":"CABPp-BGSEsQ5Ljm4j81VUuM+_U6CJYDG64wKxj_-oEK=WVD4iw@mail.gmail.com","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-08T20:36:51Z","receivedAt":"2025-07-08T20:37:03Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 8, 2025 at 4:19 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> When using cone-mode sparse-checkout, users specify which tracked\n> directories they want (recursively) and any directory not part of the parent\n> paths for those directories are considered \"out of scope\". When changing\n> sparse-checkouts, there are a variety of reasons why these \"out of scope\"\n> directories could remain, including:\n>\n> * The user has .gitignore or .git/info/exclude files that tell Git to not\n> remove files of a certain type.\n> * Some filesystem blocker prevented the removal of a tracked file. This is\n> usually more of an issue on Windows where a read handle will block file\n> deletion.\n>\n> Typically, this would not mean too much for the user experience. A few extra\n> filesystem checks might be required to satisfy git status commands, but the\n> scope of the performance hit is relative to how many cruft files are left\n> over in this situation.\n>\n> However, when using the sparse index, these tracked sparse directories cause\n> significant performance issues. When noticing that the index contains a\n> sparse directory but that directory exists on disk, Git needs to expand that\n> sparse directory to determine which files are tracked or untracked. The\n> current mechanism expands the entire index to a full one, an expensive\n> operation that scales with the total number of paths at HEAD and not just\n> the number of cruft files left over.\n>\n> Advice was added in 9479a31d603 (advice: warn when sparse index expands,\n> 2024-07-08) to help users determine that they were in this state. However,\n> the advice doesn't actually recommend helpful ways to get out of this state.\n> Recommending \"git clean\" on its own is incomplete, as typically users\n> actually need 'git clean -dfx' to clear out the ignored or excluded files.\n> Even then, they may need 'git sparse-checkout reapply' afterwards to clear\n> the sparse directories.\n>\n> The advice was successful in helping to alert users to the problem, which is\n> how I got wind of many of these cases for how users get into this state.\n> It's now time to give them a tool that helps them out of this state.\n>\n> This series adds a new 'git sparse-checkout clean' command that currently\n> only works for cone-mode sparse-checkouts. The only thing it does is\n> collapse the index to a sparse index (as much as possible) and make sure\n> that any sparse directories are removed. These directories are listed to\n> stdout.\n\nBut what does it clean up?\n - untracked files?\n - ignored files?\n - tracked-but-unmodified files?\n - tracked-and-modified files?\n - tracked-and-conflicted files? (which is probably a subset of\ntracked-and-modified, but thought I'd call it out)\n\nNote: \"tracked\" probably has a slightly ambiguous connotation here\nsince we sometimes mean \"is it in the index\", and there's a difference\nbetween \"would it be in the sparse index\" and \"would it be in the\nfully expanded index\". Here, by \"tracked\" I mean the latter -- \"is it\nin the fully expanded index\".\n\n> A --dry-run option is available to list the directories that would be\n> removed without actually deleting the directories.\n>\n> This option would be preferred to something like 'git clean -dfx' since it\n> does not clear the excluded files that are still within the sparse-checkout.\n\nThis seems to suggest you are only interested in untracked and ignored\nfiles. I'm sure that's by far the most common case, but I'm curious\nabout the others. Are you expecting users to sometimes need to run\nboth 'git sparse-checkout clean' and 'git sparse-checkout reapply'?\n\n> Instead, it performs the exact filesystem operations required to refresh the\n> sparse index performance back to what is expected.\n\nBut what operations are those and what is expected?\n\nAs you mentioned above, for untracked or ignored files, the\nexpectation is that those would be removed.\n\nI think if there are tracked-but-unmodified files, I'd expect those to\nbe removed as well.\n\nIf only the above filetypes exist, then we'd expect the directory to\nbe nuked and sparse index performance to be improved back to \"normal\".\n\nHowever, if there are tracked-and-modified files, I'd expect an error\nand for the sparse index performance to continue to suffer until those\npaths are resolved. (Or, pie-in-sky spitballing:maybe we could\nattempt to do something smarter like make sibling directories to the\ntracked-and-modified path be treated as sparse directories, so that\nperformance only suffers a little).\n\n> I spent a few weeks debating with myself about whether or not this was the\n> right interface, so please suggest alternatives if you have better ideas.\n> Among my rejected ideas include:\n>\n> * 'git sparse-checkout reapply -f -x' or similar augmentations of\n> 'reapply'.\n\nThe connection to sparse-checkout reapply at least would make it\nclearer what you are doing with tracked files, since its explanation\nexplicitly mentions those. However, reapply doesn't say anything\nabout untracked or ignored files, which we'd need to start explaining\nand perhaps isn't as clean a fit, especially since your new usecase is\npredominantly about untracked and ignored files. I don't have a\nstrong opinion here, but I think I also like your choice of a separate\n'clean' subcommand better.\n\n> * 'git clean --sparse' to focus the clean operation on things outside of\n> the sparse-checkout.\n\nYeah, this choice would have likely prevented you from cleaning up\ntracked files, and required users to run both 'clean --sparse' and\n'sparse-checkout reapply'. And this command feels more tightly\nconnected to sparse-checkouts to me, so I wouldn't have liked this\nchoice either.\n\n> The implementation is rather simple with the current CLI. Future\n> augmentations could include a --quiet option to silence the output and a\n> --verbose option to list the files that exist within each directory and\n> would/will be removed.\n\nI'm also curious what happens when (1) you are in cone mode and there\nis no sparse index, or (2) when you are not in cone mode. I suspect\nthose and the questions above will be answered as I read the\nindividual patches, so I'll keep going...\n\n> Thanks, -Stolee\n>\n> Derrick Stolee (3):\n> sparse-checkout: remove use of the_repository\n> sparse-checkout: add 'clean' command\n> sparse-index: point users to new 'clean' action\n>\n> Documentation/git-sparse-checkout.adoc | 13 +-\n> builtin/sparse-checkout.c | 192 +++++++++++++++++--------\n> sparse-index.c | 3 +-\n> t/t1091-sparse-checkout-builtin.sh | 48 +++++++\n> 4 files changed, 197 insertions(+), 59 deletions(-)\n>\n>\n> base-commit: 8b6f19ccfc3aefbd0f22f6b7d56ad6a3fc5e4f37\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1941%2Fderrickstolee%2Fgit-sparse-checkout-clean-v1\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1941/derrickstolee/git-sparse-checkout-clean-v1\n> Pull-Request: https://github.com/gitgitgadget/git/pull/1941\n> --\n> gitgitgadget\n"},{"id":"521571","messageId":"CABPp-BFu+0jME-56y+yyFYAUomfBHYrwDhr8hDLNCrp5oiOaXQ@mail.gmail.com","threadId":"63760","inReplyTo":"3cdc44a9e8c8e6718b24bc25def291348a0b2613.1751973594.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/3] sparse-checkout: remove use of the_repository","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-08T20:49:01Z","receivedAt":"2025-07-08T20:49:13Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 8, 2025 at 4:20 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The logic for the 'git sparse-checkout' builtin uses the_repository all\n> over the place, despite some use of a repository struct in different\n> method parameters. Complete this removal of the_repository by using\n> 'repo' when possible.\n>\n> In one place, there was already a local variable 'r' that was set to\n> the_repository, so move that to a method parameter.\n\nAlways nice to see these cleanups.\n\n> We cannot remove the USE_THE_REPOSITORY_VARIABLE declaration as we are\n> still using global constants for the state of the sparse-checkout.\n\nThanks for calling this out and explaining it.\n\n> Signed-off-by: Derrick Stolee \n> ---\n> builtin/sparse-checkout.c | 119 ++++++++++++++++++++------------------\n> 1 file changed, 63 insertions(+), 56 deletions(-)\n>\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index 1bf01591b275..8b70d0c6a441 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -204,12 +204,12 @@ static void clean_tracked_sparse_directories(struct repository *r)\n> ensure_full_index(r->index);\n> }\n>\n> -static int update_working_directory(struct pattern_list *pl)\n> +static int update_working_directory(struct repository *r,\n> + struct pattern_list *pl)\n> {\n> enum update_sparsity_result result;\n> struct unpack_trees_options o;\n> struct lock_file lock_file = LOCK_INIT;\n> - struct repository *r = the_repository;\n> struct pattern_list *old_pl;\n>\n> /* If no branch has been checked out, there are no updates to make. */\n> @@ -327,7 +327,8 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)\n> string_list_clear(&sl, 0);\n> }\n>\n> -static int write_patterns_and_update(struct pattern_list *pl)\n> +static int write_patterns_and_update(struct repository *repo,\n> + struct pattern_list *pl)\n> {\n> char *sparse_filename;\n> FILE *fp;\n> @@ -336,15 +337,15 @@ static int write_patterns_and_update(struct pattern_list *pl)\n>\n> sparse_filename = get_sparse_checkout_filename();\n>\n> - if (safe_create_leading_directories(the_repository, sparse_filename))\n> + if (safe_create_leading_directories(repo, sparse_filename))\n> die(_(\"failed to create directory for sparse-checkout file\"));\n>\n> hold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR);\n>\n> - result = update_working_directory(pl);\n> + result = update_working_directory(repo, pl);\n> if (result) {\n> rollback_lock_file(&lk);\n> - update_working_directory(NULL);\n> + update_working_directory(repo, NULL);\n> goto out;\n> }\n>\n> @@ -372,25 +373,26 @@ enum sparse_checkout_mode {\n> MODE_CONE_PATTERNS = 2,\n> };\n>\n> -static int set_config(enum sparse_checkout_mode mode)\n> +static int set_config(struct repository *repo,\n> + enum sparse_checkout_mode mode)\n> {\n> /* Update to use worktree config, if not already. */\n> - if (init_worktree_config(the_repository)) {\n> + if (init_worktree_config(repo)) {\n> error(_(\"failed to initialize worktree config\"));\n> return 1;\n> }\n>\n> - if (repo_config_set_worktree_gently(the_repository,\n> + if (repo_config_set_worktree_gently(repo,\n> \"core.sparseCheckout\",\n> mode ? \"true\" : \"false\") ||\n> - repo_config_set_worktree_gently(the_repository,\n> + repo_config_set_worktree_gently(repo,\n> \"core.sparseCheckoutCone\",\n> mode == MODE_CONE_PATTERNS ?\n> \"true\" : \"false\"))\n> return 1;\n>\n> if (mode == MODE_NO_PATTERNS)\n> - return set_sparse_index_config(the_repository, 0);\n> + return set_sparse_index_config(repo, 0);\n>\n> return 0;\n> }\n> @@ -410,7 +412,7 @@ static enum sparse_checkout_mode update_cone_mode(int *cone_mode) {\n> return MODE_ALL_PATTERNS;\n> }\n>\n> -static int update_modes(int *cone_mode, int *sparse_index)\n> +static int update_modes(struct repository *repo, int *cone_mode, int *sparse_index)\n> {\n> int mode, record_mode;\n>\n> @@ -418,20 +420,20 @@ static int update_modes(int *cone_mode, int *sparse_index)\n> record_mode = (*cone_mode != -1) || !core_apply_sparse_checkout;\n>\n> mode = update_cone_mode(cone_mode);\n> - if (record_mode && set_config(mode))\n> + if (record_mode && set_config(repo, mode))\n> return 1;\n>\n> /* Set sparse-index/non-sparse-index mode if specified */\n> if (*sparse_index >= 0) {\n> - if (set_sparse_index_config(the_repository, *sparse_index) < 0)\n> + if (set_sparse_index_config(repo, *sparse_index) < 0)\n> die(_(\"failed to modify sparse-index config\"));\n>\n> /* force an index rewrite */\n> - repo_read_index(the_repository);\n> - the_repository->index->updated_workdir = 1;\n> + repo_read_index(repo);\n> + repo->index->updated_workdir = 1;\n>\n> if (!*sparse_index)\n> - ensure_full_index(the_repository->index);\n> + ensure_full_index(repo->index);\n> }\n>\n> return 0;\n> @@ -448,7 +450,7 @@ static struct sparse_checkout_init_opts {\n> } init_opts;\n>\n> static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n> - struct repository *repo UNUSED)\n> + struct repository *repo)\n> {\n> struct pattern_list pl;\n> char *sparse_filename;\n> @@ -464,7 +466,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n> };\n>\n> setup_work_tree();\n> - repo_read_index(the_repository);\n> + repo_read_index(repo);\n>\n> init_opts.cone_mode = -1;\n> init_opts.sparse_index = -1;\n> @@ -473,7 +475,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n> builtin_sparse_checkout_init_options,\n> builtin_sparse_checkout_init_usage, 0);\n>\n> - if (update_modes(&init_opts.cone_mode, &init_opts.sparse_index))\n> + if (update_modes(repo, &init_opts.cone_mode, &init_opts.sparse_index))\n> return 1;\n>\n> memset(&pl, 0, sizeof(pl));\n> @@ -485,14 +487,14 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n> if (res >= 0) {\n> free(sparse_filename);\n> clear_pattern_list(&pl);\n> - return update_working_directory(NULL);\n> + return update_working_directory(repo, NULL);\n> }\n>\n> - if (repo_get_oid(the_repository, \"HEAD\", &oid)) {\n> + if (repo_get_oid(repo, \"HEAD\", &oid)) {\n> FILE *fp;\n>\n> /* assume we are in a fresh repo, but update the sparse-checkout file */\n> - if (safe_create_leading_directories(the_repository, sparse_filename))\n> + if (safe_create_leading_directories(repo, sparse_filename))\n> die(_(\"unable to create leading directories of %s\"),\n> sparse_filename);\n> fp = xfopen(sparse_filename, \"w\");\n> @@ -511,7 +513,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n> add_pattern(\"!/*/\", empty_base, 0, &pl, 0);\n> pl.use_cone_patterns = init_opts.cone_mode;\n>\n> - return write_patterns_and_update(&pl);\n> + return write_patterns_and_update(repo, &pl);\n> }\n>\n> static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *path)\n> @@ -674,7 +676,8 @@ static void add_patterns_literal(int argc, const char **argv,\n> add_patterns_from_input(pl, argc, argv, use_stdin ? stdin : NULL);\n> }\n>\n> -static int modify_pattern_list(struct strvec *args, int use_stdin,\n> +static int modify_pattern_list(struct repository *repo,\n> + struct strvec *args, int use_stdin,\n> enum modify_type m)\n> {\n> int result;\n> @@ -696,22 +699,23 @@ static int modify_pattern_list(struct strvec *args, int use_stdin,\n> }\n>\n> if (!core_apply_sparse_checkout) {\n> - set_config(MODE_ALL_PATTERNS);\n> + set_config(repo, MODE_ALL_PATTERNS);\n> core_apply_sparse_checkout = 1;\n> changed_config = 1;\n> }\n>\n> - result = write_patterns_and_update(pl);\n> + result = write_patterns_and_update(repo, pl);\n>\n> if (result && changed_config)\n> - set_config(MODE_NO_PATTERNS);\n> + set_config(repo, MODE_NO_PATTERNS);\n>\n> clear_pattern_list(pl);\n> free(pl);\n> return result;\n> }\n>\n> -static void sanitize_paths(struct strvec *args,\n> +static void sanitize_paths(struct repository *repo,\n> + struct strvec *args,\n> const char *prefix, int skip_checks)\n> {\n> int i;\n> @@ -752,7 +756,7 @@ static void sanitize_paths(struct strvec *args,\n>\n> for (i = 0; i < args->nr; i++) {\n> struct cache_entry *ce;\n> - struct index_state *index = the_repository->index;\n> + struct index_state *index = repo->index;\n> int pos = index_name_pos(index, args->v[i], strlen(args->v[i]));\n>\n> if (pos < 0)\n> @@ -779,7 +783,7 @@ static struct sparse_checkout_add_opts {\n> } add_opts;\n>\n> static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n> - struct repository *repo UNUSED)\n> + struct repository *repo)\n> {\n> static struct option builtin_sparse_checkout_add_options[] = {\n> OPT_BOOL_F(0, \"skip-checks\", &add_opts.skip_checks,\n> @@ -796,7 +800,7 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n> if (!core_apply_sparse_checkout)\n> die(_(\"no sparse-checkout to add to\"));\n>\n> - repo_read_index(the_repository);\n> + repo_read_index(repo);\n>\n> argc = parse_options(argc, argv, prefix,\n> builtin_sparse_checkout_add_options,\n> @@ -804,9 +808,9 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n>\n> for (int i = 0; i < argc; i++)\n> strvec_push(&patterns, argv[i]);\n> - sanitize_paths(&patterns, prefix, add_opts.skip_checks);\n> + sanitize_paths(repo, &patterns, prefix, add_opts.skip_checks);\n>\n> - ret = modify_pattern_list(&patterns, add_opts.use_stdin, ADD);\n> + ret = modify_pattern_list(repo, &patterns, add_opts.use_stdin, ADD);\n>\n> strvec_clear(&patterns);\n> return ret;\n> @@ -825,7 +829,7 @@ static struct sparse_checkout_set_opts {\n> } set_opts;\n>\n> static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n> - struct repository *repo UNUSED)\n> + struct repository *repo)\n> {\n> int default_patterns_nr = 2;\n> const char *default_patterns[] = {\"/*\", \"!/*/\", NULL};\n> @@ -847,7 +851,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n> int ret;\n>\n> setup_work_tree();\n> - repo_read_index(the_repository);\n> + repo_read_index(repo);\n>\n> set_opts.cone_mode = -1;\n> set_opts.sparse_index = -1;\n> @@ -856,7 +860,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n> builtin_sparse_checkout_set_options,\n> builtin_sparse_checkout_set_usage, 0);\n>\n> - if (update_modes(&set_opts.cone_mode, &set_opts.sparse_index))\n> + if (update_modes(repo, &set_opts.cone_mode, &set_opts.sparse_index))\n> return 1;\n>\n> /*\n> @@ -870,10 +874,10 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n> } else {\n> for (int i = 0; i < argc; i++)\n> strvec_push(&patterns, argv[i]);\n> - sanitize_paths(&patterns, prefix, set_opts.skip_checks);\n> + sanitize_paths(repo, &patterns, prefix, set_opts.skip_checks);\n> }\n>\n> - ret = modify_pattern_list(&patterns, set_opts.use_stdin, REPLACE);\n> + ret = modify_pattern_list(repo, &patterns, set_opts.use_stdin, REPLACE);\n>\n> strvec_clear(&patterns);\n> return ret;\n> @@ -891,7 +895,7 @@ static struct sparse_checkout_reapply_opts {\n>\n> static int sparse_checkout_reapply(int argc, const char **argv,\n> const char *prefix,\n> - struct repository *repo UNUSED)\n> + struct repository *repo)\n> {\n> static struct option builtin_sparse_checkout_reapply_options[] = {\n> OPT_BOOL(0, \"cone\", &reapply_opts.cone_mode,\n> @@ -912,12 +916,12 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n> builtin_sparse_checkout_reapply_options,\n> builtin_sparse_checkout_reapply_usage, 0);\n>\n> - repo_read_index(the_repository);\n> + repo_read_index(repo);\n>\n> - if (update_modes(&reapply_opts.cone_mode, &reapply_opts.sparse_index))\n> + if (update_modes(repo, &reapply_opts.cone_mode, &reapply_opts.sparse_index))\n> return 1;\n>\n> - return update_working_directory(NULL);\n> + return update_working_directory(repo, NULL);\n> }\n>\n> static char const * const builtin_sparse_checkout_disable_usage[] = {\n> @@ -927,7 +931,7 @@ static char const * const builtin_sparse_checkout_disable_usage[] = {\n>\n> static int sparse_checkout_disable(int argc, const char **argv,\n> const char *prefix,\n> - struct repository *repo UNUSED)\n> + struct repository *repo)\n> {\n> static struct option builtin_sparse_checkout_disable_options[] = {\n> OPT_END(),\n> @@ -955,7 +959,7 @@ static int sparse_checkout_disable(int argc, const char **argv,\n> * are expecting to do that when disabling sparse-checkout.\n> */\n> give_advice_on_expansion = 0;\n> - repo_read_index(the_repository);\n> + repo_read_index(repo);\n>\n> memset(&pl, 0, sizeof(pl));\n> hashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);\n> @@ -965,14 +969,14 @@ static int sparse_checkout_disable(int argc, const char **argv,\n>\n> add_pattern(\"/*\", empty_base, 0, &pl, 0);\n>\n> - prepare_repo_settings(the_repository);\n> - the_repository->settings.sparse_index = 0;\n> + prepare_repo_settings(repo);\n> + repo->settings.sparse_index = 0;\n>\n> - if (update_working_directory(&pl))\n> + if (update_working_directory(repo, &pl))\n> die(_(\"error while refreshing working directory\"));\n>\n> clear_pattern_list(&pl);\n> - return set_config(MODE_NO_PATTERNS);\n> + return set_config(repo, MODE_NO_PATTERNS);\n> }\n>\n> static char const * const builtin_sparse_checkout_check_rules_usage[] = {\n> @@ -987,14 +991,17 @@ static struct sparse_checkout_check_rules_opts {\n> char *rules_file;\n> } check_rules_opts;\n>\n> -static int check_rules(struct pattern_list *pl, int null_terminated) {\n> +static int check_rules(struct repository *repo,\n> + struct pattern_list *pl,\n> + int null_terminated)\n> +{\n> struct strbuf line = STRBUF_INIT;\n> struct strbuf unquoted = STRBUF_INIT;\n> char *path;\n> int line_terminator = null_terminated ? 0 : '\\n';\n> strbuf_getline_fn getline_fn = null_terminated ? strbuf_getline_nul\n> : strbuf_getline;\n> - the_repository->index->sparse_checkout_patterns = pl;\n> + repo->index->sparse_checkout_patterns = pl;\n> while (!getline_fn(&line, stdin)) {\n> path = line.buf;\n> if (!null_terminated && line.buf[0] == '\"') {\n> @@ -1006,7 +1013,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n> path = unquoted.buf;\n> }\n>\n> - if (path_in_sparse_checkout(path, the_repository->index))\n> + if (path_in_sparse_checkout(path, repo->index))\n> write_name_quoted(path, stdout, line_terminator);\n> }\n> strbuf_release(&line);\n> @@ -1016,7 +1023,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n> }\n>\n> static int sparse_checkout_check_rules(int argc, const char **argv, const char *prefix,\n> - struct repository *repo UNUSED)\n> + struct repository *repo)\n> {\n> static struct option builtin_sparse_checkout_check_rules_options[] = {\n> OPT_BOOL('z', NULL, &check_rules_opts.null_termination,\n> @@ -1055,7 +1062,7 @@ static int sparse_checkout_check_rules(int argc, const char **argv, const char *\n> free(sparse_filename);\n> }\n>\n> - ret = check_rules(&pl, check_rules_opts.null_termination);\n> + ret = check_rules(repo, &pl, check_rules_opts.null_termination);\n> clear_pattern_list(&pl);\n> free(check_rules_opts.rules_file);\n> return ret;\n> @@ -1084,8 +1091,8 @@ int cmd_sparse_checkout(int argc,\n>\n> git_config(git_default_config, NULL);\n>\n> - prepare_repo_settings(the_repository);\n> - the_repository->settings.command_requires_full_index = 0;\n> + prepare_repo_settings(repo);\n> + repo->settings.command_requires_full_index = 0;\n>\n> return fn(argc, argv, prefix, repo);\n> }\n> --\n> gitgitgadget\n\nPatch looks good to me.\n"},{"id":"521573","messageId":"xmqqikk2tn53.fsf@gitster.g","threadId":"63760","inReplyTo":"3cdc44a9e8c8e6718b24bc25def291348a0b2613.1751973594.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/3] sparse-checkout: remove use of the_repository","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-08T20:59:04Z","receivedAt":"2025-07-08T20:59:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> From: Derrick Stolee \n>\n> The logic for the 'git sparse-checkout' builtin uses the_repository all\n> over the place, despite some use of a repository struct in different\n> method parameters. Complete this removal of the_repository by using\n> 'repo' when possible.\n>\n> In one place, there was already a local variable 'r' that was set to\n> the_repository, so move that to a method parameter.\n>\n> We cannot remove the USE_THE_REPOSITORY_VARIABLE declaration as we are\n> still using global constants for the state of the sparse-checkout.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> builtin/sparse-checkout.c | 119 ++++++++++++++++++++------------------\n> 1 file changed, 63 insertions(+), 56 deletions(-)\n\nOK. The damage is not too bad for a partial update ;-).\n\nAs the file-scope static functions in builtin/sparse-checkout.c are\nnot going to be called by anybody else, it does not really matter if\nthey internally pass an extra parameter around or use the_repository\nsince the end result is the same. But doing this may hopefully help\nthose that may want to move some of these functions to a more\nlibrary-ish part of the system outside builtin/ hierarchy.\n\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index 1bf01591b275..8b70d0c6a441 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -204,12 +204,12 @@ static void clean_tracked_sparse_directories(struct repository *r)\n> \t\tensure_full_index(r->index);\n> }\n> \n> -static int update_working_directory(struct pattern_list *pl)\n> +static int update_working_directory(struct repository *r,\n> +\t\t\t\t struct pattern_list *pl)\n> {\n> \tenum update_sparsity_result result;\n> \tstruct unpack_trees_options o;\n> \tstruct lock_file lock_file = LOCK_INIT;\n> -\tstruct repository *r = the_repository;\n> \tstruct pattern_list *old_pl;\n\nAs this already used short-and-sweet 'r', we just follow suit to\nminimize the damage, which is fine.\n\n> @@ -327,7 +327,8 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)\n> \tstring_list_clear(&sl, 0);\n> }\n> \n> -static int write_patterns_and_update(struct pattern_list *pl)\n> +static int write_patterns_and_update(struct repository *repo,\n> +\t\t\t\t struct pattern_list *pl)\n> {\n> \tchar *sparse_filename;\n> \tFILE *fp;\n> @@ -336,15 +337,15 @@ static int write_patterns_and_update(struct pattern_list *pl)\n> \n> \tsparse_filename = get_sparse_checkout_filename();\n> \n> -\tif (safe_create_leading_directories(the_repository, sparse_filename))\n> +\tif (safe_create_leading_directories(repo, sparse_filename))\n> \t\tdie(_(\"failed to create directory for sparse-checkout file\"));\n> \n> \thold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR);\n> \n> -\tresult = update_working_directory(pl);\n> +\tresult = update_working_directory(repo, pl);\n> \tif (result) {\n> \t\trollback_lock_file(&lk);\n> -\t\tupdate_working_directory(NULL);\n> +\t\tupdate_working_directory(repo, NULL);\n> \t\tgoto out;\n> \t}\n\nBut this introduces a new parameter. Both of two instances of\nrepository struct used in the existing code in this function, other\nthan references to struct repository *UNUSED, use \"r\", and with this\npatch, the name \"repo\" becomes more prevanent.\n\nWe would probably want to rename \"r\" to \"repo\" for consistency in\nclean_tracked_sparse_repositories() and update_working_directory(),\nbut that is better done later after the dust settles and the code\naround here becomes quiescent again, not as part of this topic as an\nextra churn.\n\nLooking good. Thanks. Will queue.\n"},{"id":"521575","messageId":"xmqqa55etm5g.fsf@gitster.g","threadId":"63760","inReplyTo":"49418e8ec8a4c3e0ce9c65aa700042b6f3f3f4d7.1751973594.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-08T21:20:27Z","receivedAt":"2025-07-08T21:20:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> From: Derrick Stolee \n>\n> When users change their sparse-checkout definitions to add new\n> directories and remove old ones, there may be a few reasons why\n> directories no longer in scope remain (ignored or excluded files still\n> exist, Windows handles are still open, etc.). When these files still\n> exist, the sparse index feature notices that a tracked, but sparse,\n> directory still exists on disk and thus the index expands. This causes a\n> performance hit _and_ the advice printed isn't very helpful. Using 'git\n> clean' isn't enough (generally '-dfx' may be needed) but also this may\n> not be sufficient.\n>\n> Add a new subcommand to 'git sparse-checkout' that removes these\n> tracked-but-sparse directories, including any excluded or ignored files\n\nAre excluded files and ignored files form two separate sets, or are\nthey one and the same? Do files that users forgot to add (e.g. new\nsource file that would not match any patterns listed in .gitignore)\nand object files left over from the previous compilation (most\nlikely match *.o in .gitignore) treated the same way for the purpose\nof determining if the directory that is no longer in the cone can be\nremoved?\n\n> underneath. This is the most extreme method for doing this, but it works\n> when the sparse-checkout is in cone mode and is expected to rescope\n> based on directories, not files.\n>\n> Be sure to add a --dry-run option so users can predict what will be\n> deleted. In general, output the directories that are being removed so\n> users can know what was removed.\n\nHmph. It would be safer to show not just the directories but which\nexcluded files are about to be lost, wouldn't it, especially when\nthe user is trying to play safe and see what potential damage they\nare looking at?\n\nAlso even though ignored files are \"ignored and expendable\", nobody\nmarks their temporary file as \"ignored but precious\" (yet), so \"it\nis listed in .gitignore so we can safely remove it\" may not be a\nsafe assumption for us to be making (yet). Shouldn't we at least be\nlisting these ignored files in --dry-run output, next to those files\nthat the user may have forgotten to add?\n\n> Note that untracked directories remain. Further, directories that\n> contain staged changes are not deleted. This is a detail that is partly\n> hidden by the implementation which relies on collapsing the index to a\n> sparse index in-memory and only deleting directories that are listed as\n> sparse in the index. If a staged change exists, then that entry is not\n> stored as a sparse tree entry and thus remains on-disk until committed\n> or reset.\n\nRemoving untracked directories is a job for \"clean -d\", so it makes\nsense for this new command not to touch them. Not losing changes\nthat have already been added is just a bad as losing new files that\nthe user forgot to add, so it does make sense not to remove them.\n\nI wonder if we need \"-x\" and/or \"-X\" options \"clean\" has (and\nperhaps \"-d\" that is a no-op, as the whole point of this subcommand\nis about removing directories from the working tree) to control its\noperation a bit finer-grained way.\n\n> +\tfor (size_t i = 0; i < repo->index->cache_nr; i++) {\n> +\t\tDIR* dir;\n\nThe asterisk sticks to the variable, not the type, i.e.\n\n\t\tDIR *dir;\n\nThanks.\n"},{"id":"521577","messageId":"CABPp-BH_JGbtDsne_tKBoqDr9LzL==gs0E5xyPdbqygyXs8+vA@mail.gmail.com","threadId":"63760","inReplyTo":"49418e8ec8a4c3e0ce9c65aa700042b6f3f3f4d7.1751973594.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-08T21:43:14Z","receivedAt":"2025-07-08T21:43:27Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 8, 2025 at 4:20 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> When users change their sparse-checkout definitions to add new\n> directories and remove old ones, there may be a few reasons why\n> directories no longer in scope remain (ignored or excluded files still\n> exist, Windows handles are still open, etc.).\n\nGood background; I am still particularly interested in the \"etc.\" part...\n\n> When these files still\n> exist, the sparse index feature notices that a tracked, but sparse,\n> directory still exists on disk and thus the index expands. This causes a\n> performance hit _and_ the advice printed isn't very helpful. Using 'git\n> clean' isn't enough (generally '-dfx' may be needed) but also this may\n> not be sufficient.\n\nVery well motivated.\n\n> Add a new subcommand to 'git sparse-checkout' that removes these\n> tracked-but-sparse directories, including any excluded or ignored files\n> underneath.\n\n\"including\"?\n\n> This is the most extreme method for doing this, but it works\n> when the sparse-checkout is in cone mode and is expected to rescope\n> based on directories, not files.\n\nSo is this also meant for cone mode without sparse index turned on?\nWhat about non-cone mode?\n\n> Be sure to add a --dry-run option so users can predict what will be\n> deleted. In general, output the directories that are being removed so\n> users can know what was removed.\n\nIs greater fidelity of interest when there are multiple different\ntypes of files contained? For example, \"git status\" lists individual\nfiles within a directory, unless it find an ignored directory and then\nit simply lists the directory. That means we get more fidelity when\nit's warranted, and less when it's not. I'm not sure if that's a\nperfect analogy, though; it may well be that we don't need the same\nkind of fidelity that `git status` provides. (And I'm kind of\nguessing it isn't needed, except in error cases, but I'm just asking.)\n\n> Note that untracked directories remain.\n\nWhat does this mean? If the sparse directory had an untracked\ndirectory within it then it'll be left on disk, you will only clean up\nuntracked files at a depth of 1 within the sparse directory?\n\nOr that untracked directories not contained within a sparse directory\nwill be left alone?\n\n> Further, directories that\n> contain staged changes are not deleted.\n\nShouldn't those be safe to delete? When a sparse directory has files\nunderneath it with staged changes, those roll-up into a staged\nsparse-directory tree value, and so we should be able to delete the\nfile.\n\nIn contrast, the files under the sparse directory with unstaged\nchanges would be problematic to simply remove.\n\n> This is a detail that is partly\n> hidden by the implementation which relies on collapsing the index to a\n> sparse index in-memory and only deleting directories that are listed as\n> sparse in the index. If a staged change exists, then that entry is not\n> stored as a sparse tree entry and thus remains on-disk until committed\n> or reset.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> Documentation/git-sparse-checkout.adoc | 13 ++++-\n> builtin/sparse-checkout.c | 73 +++++++++++++++++++++++++-\n> t/t1091-sparse-checkout-builtin.sh | 48 +++++++++++++++++\n> 3 files changed, 132 insertions(+), 2 deletions(-)\n>\n> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n> index 529a8edd9c1e..21ba6f759905 100644\n> --- a/Documentation/git-sparse-checkout.adoc\n> +++ b/Documentation/git-sparse-checkout.adoc\n> @@ -9,7 +9,7 @@ git-sparse-checkout - Reduce your working tree to a subset of tracked files\n> SYNOPSIS\n> --------\n> [verse]\n> -'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) []\n> +'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules | clean) []\n>\n>\n> DESCRIPTION\n> @@ -111,6 +111,17 @@ flags, with the same meaning as the flags from the `set` command, in order\n> to change which sparsity mode you are using without needing to also respecify\n> all sparsity paths.\n>\n> +'clean'::\n> + Remove all files in tracked directories that are outside of the\n> + sparse-checkout definition.\n\nIf literal, this sounds unsafe, particularly if run while resolving\nmerge or rebase conflicts (since those conflicts may occur in paths\noutside the sparse checkout definition).\n\n> + This subcommand requires cone-mode\n> + sparse-checkout to be sure that we know which directories are\n> + both tracked and all contained paths are not in the sparse-checkout.\n> + This command can be used to be sure the sparse index works\n> + efficiently.\n\nSo...what does it do when in cone mode and the sparse index is not enabled?\n\n> ++\n> +The `clean` command can also take the `--dry-run` (`-n`) option to list\n> +the directories it would remove without performing any filesystem changes.\n> +\n> 'disable'::\n> Disable the `core.sparseCheckout` config setting, and restore the\n> working directory to include all files.\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index 8b70d0c6a441..6d2843827367 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -23,7 +23,7 @@\n> static const char *empty_base = \"\";\n>\n> static char const * const builtin_sparse_checkout_usage[] = {\n> - N_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules) []\"),\n> + N_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules | clean) []\"),\n> NULL\n> };\n>\n> @@ -924,6 +924,76 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n> return update_working_directory(repo, NULL);\n> }\n>\n> +static char const * const builtin_sparse_checkout_clean_usage[] = {\n> + \"git sparse-checkout clean [-n|--dry-run]\",\n> + NULL\n> +};\n> +\n> +static struct sparse_checkout_clean_opts {\n> + int dry_run;\n> +} clean_opts;\n> +\n> +static int sparse_checkout_clean(int argc, const char **argv,\n> + const char *prefix,\n> + struct repository *repo)\n> +{\n> + struct strbuf full_path = STRBUF_INIT;\n> + size_t worktree_len;\n> + static struct option builtin_sparse_checkout_clean_options[] = {\n> + OPT_BOOL('n', \"dry-run\", &clean_opts.dry_run,\n> + N_(\"list the directories that would be removed without making filesystem changes\")),\n> + OPT_END(),\n> + };\n> +\n> + setup_work_tree();\n> + if (!core_apply_sparse_checkout)\n> + die(_(\"must be in a sparse-checkout to clean directories\"));\n> + if (!core_sparse_checkout_cone)\n> + die(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n> +\n> + argc = parse_options(argc, argv, prefix,\n> + builtin_sparse_checkout_clean_options,\n> + builtin_sparse_checkout_clean_usage, 0);\n> +\n> + if (repo_read_index(repo) < 0)\n> + die(_(\"failed to read index\"));\n> +\n> + if (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY))\n> + die(_(\"failed to convert index to a sparse index\"));\n\nSo, you make the in-memory index sparse; I don't remember the details\non this function so it might invalidate some things I say below...but\nafter this point you then...\n\n> +\n> + strbuf_addstr(&full_path, repo->worktree);\n> + strbuf_addch(&full_path, '/');\n> + worktree_len = full_path.len;\n> +\n> + for (size_t i = 0; i < repo->index->cache_nr; i++) {\n> + DIR* dir;\n> + struct cache_entry *ce = repo->index->cache[i];\n> + if (!S_ISSPARSEDIR(ce->ce_mode))\n> + continue;\n\n...skip the entries that aren't sparse directories.\n\n> + strbuf_setlen(&full_path, worktree_len);\n> + strbuf_add(&full_path, ce->name, ce->ce_namelen);\n> +\n> + dir = opendir(full_path.buf);\n> + if (!dir)\n> + continue;\n\n...skip the sparse directories that, as expected, don't exist on disk.\n\n> + else if (ENOENT != errno) {\n> + warning_errno(_(\"failed to check for existence of '%s'\"), ce->name);\n> + continue;\n> + }\n> +\n> + closedir(dir);\n> +\n> + printf(\"%s\\n\", ce->name);\n> + if (!clean_opts.dry_run) {\n> + if (remove_dir_recursively(&full_path, 0))\n> + warning_errno(_(\"failed to remove '%s'\"), ce->name);\n> + }\n\n...and then unconditionally remove the directory, as you stated in the\ndocumentation for this clean option.\n\nI'm worried whether this is safe; if someone does a merge or rebase,\nthere could be tracked-and-modified/conflicted files outside the\nsparse specification in the working tree.\n\nEven after resolving such a merge and committing, the paths may remain\naround until the user does a 'git sparse-checkout reapply' (I don't\nremember details here, but our documentation for reapply certainly\nsays so), and since the file might stick around, the user may make\nfurther modifications to such a file.\n\n...or will the convert_to_sparse() call above fail in all these cases?\n If it does, should it give a better and more useful error message\nthan \"failed to convert index to a sparse index\" and rather e.g. \"path\n%s has modifications; please stage or revert first\"?\n\n> + }\n> +\n> + strbuf_release(&full_path);\n> + return 0;\n> +}\n> +\n> static char const * const builtin_sparse_checkout_disable_usage[] = {\n> \"git sparse-checkout disable\",\n> NULL\n> @@ -1080,6 +1150,7 @@ int cmd_sparse_checkout(int argc,\n> OPT_SUBCOMMAND(\"set\", &fn, sparse_checkout_set),\n> OPT_SUBCOMMAND(\"add\", &fn, sparse_checkout_add),\n> OPT_SUBCOMMAND(\"reapply\", &fn, sparse_checkout_reapply),\n> + OPT_SUBCOMMAND(\"clean\", &fn, sparse_checkout_clean),\n> OPT_SUBCOMMAND(\"disable\", &fn, sparse_checkout_disable),\n> OPT_SUBCOMMAND(\"check-rules\", &fn, sparse_checkout_check_rules),\n> OPT_END(),\n> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\n> index ab3a105ffff2..7f8a444541f7 100755\n> --- a/t/t1091-sparse-checkout-builtin.sh\n> +++ b/t/t1091-sparse-checkout-builtin.sh\n> @@ -1050,5 +1050,53 @@ test_expect_success 'check-rules null termination' '\n> test_cmp expect actual\n> '\n>\n> +test_expect_success 'clean' '\n> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n> + mkdir repo/deep/deeper2 repo/folder1 &&\n> + touch repo/deep/deeper2/file &&\n> + touch repo/folder1/file &&\n> +\n> + cat >expect <<-\\EOF &&\n> + deep/deeper2/\n> + folder1/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean --dry-run >out &&\n> + test_cmp expect out &&\n> +\n> + test_path_exists repo/deep/deeper2 &&\n> + test_path_exists repo/folder1 &&\n> +\n> + git -C repo sparse-checkout clean >out &&\n> + test_cmp expect out &&\n> +\n> + ! test_path_exists repo/deep/deeper2 &&\n> + ! test_path_exists repo/folder1\n> +'\n> +\n> +test_expect_success 'clean with staged sparse change' '\n> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n> + mkdir repo/deep/deeper2 repo/folder1 &&\n> + touch repo/deep/deeper2/file &&\n> + touch repo/folder1/file &&\n> +\n> + git -C repo add --sparse folder1/file &&\n> +\n> + cat >expect <<-\\EOF &&\n> + deep/deeper2/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean --dry-run >out &&\n> + test_cmp expect out &&\n> +\n> + test_path_exists repo/deep/deeper2 &&\n> + test_path_exists repo/folder1 &&\n> +\n> + git -C repo sparse-checkout clean >out &&\n> + test_cmp expect out &&\n> +\n> + ! test_path_exists repo/deep/deeper2 &&\n> + test_path_exists repo/folder1\n> +'\n>\n> test_done\n> --\n> gitgitgadget\n"},{"id":"521578","messageId":"CABPp-BHTZgJTpxFSszEUvfrk1AP=OnTC8mKJ5ddZfH2mZu3WaQ@mail.gmail.com","threadId":"63760","inReplyTo":"80d7a7641daa7a6f0e1db73e0a433701e9f37209.1751973594.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 3/3] sparse-index: point users to new 'clean' action","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-08T21:45:33Z","receivedAt":"2025-07-08T21:45:45Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 8, 2025 at 4:20 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> In my experience, the most-common reason that the sparse index must\n> expand to a full one is because there is some leftover file in a tracked\n> directory that is now outside of the sparse-checkout. The new 'git\n> sparse-checkout clean' command will find and delete these directories,\n> so point users to it when they hit the sparse index expansion advice.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> sparse-index.c | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n>\n> diff --git a/sparse-index.c b/sparse-index.c\n> index 5634abafaa07..5d14795063b5 100644\n> --- a/sparse-index.c\n> +++ b/sparse-index.c\n> @@ -32,7 +32,8 @@ int give_advice_on_expansion = 1;\n> \"Your working directory likely has contents that are outside of\\n\" \\\n> \"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n> \"see your sparse-checkout definition and compare it to your working\\n\" \\\n> - \"directory contents. Running 'git clean' may assist in this cleanup.\"\n> + \"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n> + \"in this cleanup.\"\n>\n> struct modify_index_context {\n> struct index_state *write;\n> --\n> gitgitgadget\n\nMakes sense, once we work out any wrinkles with `git sparse-checkout clean`.\n"},{"id":"521581","messageId":"CABPp-BEh5FdPSr84-YG=Bw6fBaE1P2s8Zcy5D68bSZ+_nsQrvw@mail.gmail.com","threadId":"63760","inReplyTo":"CABPp-BGSEsQ5Ljm4j81VUuM+_U6CJYDG64wKxj_-oEK=WVD4iw@mail.gmail.com","subject":"Re: [PATCH 0/3] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-08T22:01:21Z","receivedAt":"2025-07-08T22:01:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 8, 2025 at 1:36 PM Elijah Newren wrote:\n>\n> On Tue, Jul 8, 2025 at 4:19 AM Derrick Stolee via GitGitGadget\n> wrote:\n> >\n[...]\n> I'm also curious what happens when (1) you are in cone mode and there\n> is no sparse index, or (2) when you are not in cone mode. I suspect\n> those and the questions above will be answered as I read the\n> individual patches, so I'll keep going...\n\nAfter reading the series, I know the answer to (2). I think the\nanswer to (1) is that it effectively turns into a silent (but not\ninstantaneous) no-op, which may be confusing for users. We might want\nto provide them with an alternative implementation, or at least a\nwarning or error that the mode doesn't (currently?) do anything when\nsparse index isn't in use.\n\nAnyway, I think the series is a good direction and you've explained\nthe motivation very well, but I'm a bit worried the current\nimplementation might be using too coarse of a hammer.\n"},{"id":"521591","messageId":"xmqq1pqqs127.fsf@gitster.g","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-08T23:41:20Z","receivedAt":"2025-07-08T23:41:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> The implementation is rather simple with the current CLI. Future\n> augmentations could include a --quiet option to silence the output and a\n> --verbose option to list the files that exist within each directory and\n> would/will be removed.\n\nI liked the overall idea but this has some interactions with a topic\nin flight. 2c5b5565 (environment: remove the global variable\n'sparse_expect_files_outside_of_patterns', 2025-07-01). I may have\nbotched (semantic) conflict resolution but with both merged to\n'seen', a few steps in the sparse test seem to fail.\n\nFor tonight's integration, I'll leave the topic out of 'seen' so\nthat we can pass other new topics that we acquired through the CI.\n\nI may re-attempt merging this topic later, or I may eject the other\ntopic from 'seen' and queue this one first, asking the other topic\nto be redone on top. We'll see.\n\nThanks.\n"},{"id":"521679","messageId":"2503c79c-68f3-4ed5-bbfd-3a7af07a89cc@gmail.com","threadId":"63760","inReplyTo":"xmqqa55etm5g.fsf@gitster.g","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-07-09T14:39:29Z","receivedAt":"2025-07-09T14:39:31Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/8/2025 5:20 PM, Junio C Hamano wrote:\n> \"Derrick Stolee via GitGitGadget\" writes:\n> \n>> From: Derrick Stolee \n>>\n>> When users change their sparse-checkout definitions to add new\n>> directories and remove old ones, there may be a few reasons why\n>> directories no longer in scope remain (ignored or excluded files still\n>> exist, Windows handles are still open, etc.). When these files still\n>> exist, the sparse index feature notices that a tracked, but sparse,\n>> directory still exists on disk and thus the index expands. This causes a\n>> performance hit _and_ the advice printed isn't very helpful. Using 'git\n>> clean' isn't enough (generally '-dfx' may be needed) but also this may\n>> not be sufficient.\n>>\n>> Add a new subcommand to 'git sparse-checkout' that removes these\n>> tracked-but-sparse directories, including any excluded or ignored files\n> \n> Are excluded files and ignored files form two separate sets, or are\n> they one and the same? Do files that users forgot to add (e.g. new\n> source file that would not match any patterns listed in .gitignore)\n> and object files left over from the previous compilation (most\n> likely match *.o in .gitignore) treated the same way for the purpose\n> of determining if the directory that is no longer in the cone can be\n> removed?\n\nI think of them as separate in my head because:\n\n* .gitignore is committed to the repo, and is common to all users of\n the repo.\n\n* .git/info/exclude is custom to each user, so users are choosing to\n ignore extra files that are atypical from most users.\n\nIn the monorepo I'm thinking about, .gitignore files are rather small\nbecause all build output has already been redirected out of the\nworktree for performance reasons. Thus, _most_ users don't have this\nproblem. However, some users add extra excludes for things like vim\nfiles and those get leftover, causing invisible (to 'git status') pain.\n\n>> underneath. This is the most extreme method for doing this, but it works\n>> when the sparse-checkout is in cone mode and is expected to rescope\n>> based on directories, not files.\n>>\n>> Be sure to add a --dry-run option so users can predict what will be\n>> deleted. In general, output the directories that are being removed so\n>> users can know what was removed.\n> \n> Hmph. It would be safer to show not just the directories but which\n> excluded files are about to be lost, wouldn't it, especially when\n> the user is trying to play safe and see what potential damage they\n> are looking at?\n> > Also even though ignored files are \"ignored and expendable\", nobody\n> marks their temporary file as \"ignored but precious\" (yet), so \"it\n> is listed in .gitignore so we can safely remove it\" may not be a\n> safe assumption for us to be making (yet). Shouldn't we at least be\n> listing these ignored files in --dry-run output, next to those files\n> that the user may have forgotten to add?\n\nI considered this, but mostly behind a potential --verbose option to\nlist the files that are leftover. Much of the design here is that\nthese _directories_ are out of scope, skipping over any details about\nthe contained files, so I thought this directory-based output would\ncommunicate enough information.\n\nA curious user may want to know \"why are these directories still\naround?\" and the more verbose output would assist.\n\n>> Note that untracked directories remain. Further, directories that\n>> contain staged changes are not deleted. This is a detail that is partly\n>> hidden by the implementation which relies on collapsing the index to a\n>> sparse index in-memory and only deleting directories that are listed as\n>> sparse in the index. If a staged change exists, then that entry is not\n>> stored as a sparse tree entry and thus remains on-disk until committed\n>> or reset.\n> \n> Removing untracked directories is a job for \"clean -d\", so it makes\n> sense for this new command not to touch them. Not losing changes\n> that have already been added is just a bad as losing new files that\n> the user forgot to add, so it does make sense not to remove them.\n> \n> I wonder if we need \"-x\" and/or \"-X\" options \"clean\" has (and\n> perhaps \"-d\" that is a no-op, as the whole point of this subcommand\n> is about removing directories from the working tree) to control its\n> operation a bit finer-grained way.\nI'm of two minds here.\n\nMy first inclination is \"we already have 'git clean' for fine-grained\ncontrol of removing ignored/excluded files\".\n\nMy second inclination is \"'git clean' would remove these ignored files\neven when they are within the sparse-checkout, so that's too big of a\nhammer\".\n\nThere are a lot of ways to filter the files that would be removed,\nbut I think that in this case most users are wanting a one-command way\nto get their sparse-checkout into a better state.\n\nI'm not making any final statements here. I appreciate all of the\nthoughts around which options should be default and which should be\nhidden behind options.\n\nThanks,\n-Stolee\n\n"},{"id":"521694","messageId":"06c1d2a9-bc74-43c7-8198-cc508b56f34a@gmail.com","threadId":"63760","inReplyTo":"xmqq1pqqs127.fsf@gitster.g","subject":"Re: [PATCH 0/3] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-07-09T15:41:26Z","receivedAt":"2025-07-09T15:41:30Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/8/2025 7:41 PM, Junio C Hamano wrote:\n> \"Derrick Stolee via GitGitGadget\" writes:\n> \n>> The implementation is rather simple with the current CLI. Future\n>> augmentations could include a --quiet option to silence the output and a\n>> --verbose option to list the files that exist within each directory and\n>> would/will be removed.\n> \n> I liked the overall idea but this has some interactions with a topic\n> in flight. 2c5b5565 (environment: remove the global variable\n> 'sparse_expect_files_outside_of_patterns', 2025-07-01). I may have\n> botched (semantic) conflict resolution but with both merged to\n> 'seen', a few steps in the sparse test seem to fail.\n> \n> For tonight's integration, I'll leave the topic out of 'seen' so\n> that we can pass other new topics that we acquired through the CI.\n> \n> I may re-attempt merging this topic later, or I may eject the other\n> topic from 'seen' and queue this one first, asking the other topic\n> to be redone on top. We'll see.\n\nI'll update my next version to be built on top of that one so\nwe don't need to worry about semantic merge resolutions.\n\nThanks,\n-Stolee\n\n"},{"id":"521701","messageId":"492244b9-2402-4e8e-8599-8bcd5d27e2f8@gmail.com","threadId":"63760","inReplyTo":"CABPp-BH_JGbtDsne_tKBoqDr9LzL==gs0E5xyPdbqygyXs8+vA@mail.gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-07-09T16:13:00Z","receivedAt":"2025-07-09T16:13:02Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/8/2025 5:43 PM, Elijah Newren wrote:\n> On Tue, Jul 8, 2025 at 4:20 AM Derrick Stolee via GitGitGadget\n> wrote:\n>>\n>> From: Derrick Stolee \n>>\n>> When users change their sparse-checkout definitions to add new\n>> directories and remove old ones, there may be a few reasons why\n>> directories no longer in scope remain (ignored or excluded files still\n>> exist, Windows handles are still open, etc.).\n> \n> Good background; I am still particularly interested in the \"etc.\" part...\n\nI listed the cases that I've confirmed to be problems. There are\nperhaps some that I'm missing or overlap (such as \"I had my terminal\nwindow open on that directory\" which is really a handle problem). \n>> When these files still\n>> exist, the sparse index feature notices that a tracked, but sparse,\n>> directory still exists on disk and thus the index expands. This causes a\n>> performance hit _and_ the advice printed isn't very helpful. Using 'git\n>> clean' isn't enough (generally '-dfx' may be needed) but also this may\n>> not be sufficient.\n> \n> Very well motivated.\n> \n>> Add a new subcommand to 'git sparse-checkout' that removes these\n>> tracked-but-sparse directories, including any excluded or ignored files\n>> underneath.\n> \n> \"including\"?\n\nYes. If we leave the ignored files then we have not accomplished our\ngoal in deleting the sparse directories.\n\n>> This is the most extreme method for doing this, but it works\n>> when the sparse-checkout is in cone mode and is expected to rescope\n>> based on directories, not files.\n> \n> So is this also meant for cone mode without sparse index turned on?\n> What about non-cone mode?\n\nThis command die()s if not in cone mode. We can consider future\nchanges that perform similar actions in non-cone mode, but I'm\nnot sure if there is a valuable need in that case.\n\n>> Be sure to add a --dry-run option so users can predict what will be\n>> deleted. In general, output the directories that are being removed so\n>> users can know what was removed.\n> \n> Is greater fidelity of interest when there are multiple different\n> types of files contained? For example, \"git status\" lists individual\n> files within a directory, unless it find an ignored directory and then\n> it simply lists the directory. That means we get more fidelity when\n> it's warranted, and less when it's not. I'm not sure if that's a\n> perfect analogy, though; it may well be that we don't need the same\n> kind of fidelity that `git status` provides. (And I'm kind of\n> guessing it isn't needed, except in error cases, but I'm just asking.)\n> \n>> Note that untracked directories remain.\n> \n> What does this mean? If the sparse directory had an untracked\n> directory within it then it'll be left on disk, you will only clean up\n> untracked files at a depth of 1 within the sparse directory?\n> \n> Or that untracked directories not contained within a sparse directory\n> will be left alone?\n\nThis second part: \"untracked directories not contained within\na sparse directory will remain\". This is mostly to point out\nthat we are not saying \"the only directories that remain are\ntracked directories within the sparse-checkout\" as that could\nremove valuable temporary directories that are covered by\n.gitignore or exclude files.\n\n>> Further, directories that\n>> contain staged changes are not deleted.\n> \n> Shouldn't those be safe to delete? When a sparse directory has files\n> underneath it with staged changes, those roll-up into a staged\n> sparse-directory tree value, and so we should be able to delete the\n> file.\n\nThis is _mostly_ an implementation detail. The sparse index will\nnot represent this directory as a sparse directory, so it's not\ndeleted. (see the next paragraph:)\n\n>> This is a detail that is partly\n>> hidden by the implementation which relies on collapsing the index to a\n>> sparse index in-memory and only deleting directories that are listed as\n>> sparse in the index. If a staged change exists, then that entry is not\n>> stored as a sparse tree entry and thus remains on-disk until committed\n>> or reset. \n> In contrast, the files under the sparse directory with unstaged\n> changes would be problematic to simply remove.\nExcept that a user is only using this command when they want\nfiles outside of the sparse-checkout to be deleted.\n\nI'd like to find the right way to make it clear to users who\ndiscover this command that they are asking for the following:\n\n \"I changed my sparse-checkout and some directories that I\n expected to be deleted are still around. Delete them as I\n don't care about them or the files inside anymore.\"\n\nSome of the discussion around having a --verbose option (in\nconjunction with --dry-run) would allow for the following\nuser scenario:\n\n \"I changed my sparse-checkout and some directories that I\n expected to be deleted are still around. Which files are\n preventing that deletion? I'd like to know what's in the\n way so I can evaluate if those files are important to me.\"\n\n>> +'clean'::\n>> + Remove all files in tracked directories that are outside of the\n>> + sparse-checkout definition.\n> \n> If literal, this sounds unsafe, particularly if run while resolving\n> merge or rebase conflicts (since those conflicts may occur in paths\n> outside the sparse checkout definition).\n\nIf we are in a merge-conflict state, the directory is not\ncollapsed in the sparse index.. \n>> + This subcommand requires cone-mode\n>> + sparse-checkout to be sure that we know which directories are\n>> + both tracked and all contained paths are not in the sparse-checkout.\n>> + This command can be used to be sure the sparse index works\n>> + efficiently.\n> \n> So...what does it do when in cone mode and the sparse index is not enabled?\n\nIt doesn't effect the behavior, since we don't care about the on-disk\nformat and instead use an in-memory sparse index to determine which\ndirectories to delete.\n\nThere could be a benefit for users wanting to clean up extra files in\ntheir worktree even if they are not using a sparse index. It is less\nlikely that they will discover that they are in that state if they\nare not pestered by the index expansion advice message.\n\n>> + if (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY))\n>> + die(_(\"failed to convert index to a sparse index\"));\n> \n> So, you make the in-memory index sparse; I don't remember the details\n> on this function so it might invalidate some things I say below...but\n> after this point you then...\n> \n>> +\n>> + strbuf_addstr(&full_path, repo->worktree);\n>> + strbuf_addch(&full_path, '/');\n>> + worktree_len = full_path.len;\n>> +\n>> + for (size_t i = 0; i < repo->index->cache_nr; i++) {\n>> + DIR* dir;\n>> + struct cache_entry *ce = repo->index->cache[i];\n>> + if (!S_ISSPARSEDIR(ce->ce_mode))\n>> + continue;\n> \n> ...skip the entries that aren't sparse directories.\n> \n>> + strbuf_setlen(&full_path, worktree_len);\n>> + strbuf_add(&full_path, ce->name, ce->ce_namelen);\n>> +\n>> + dir = opendir(full_path.buf);\n>> + if (!dir)\n>> + continue;\n> \n> ...skip the sparse directories that, as expected, don't exist on disk.\n> \n>> + else if (ENOENT != errno) {\n>> + warning_errno(_(\"failed to check for existence of '%s'\"), ce->name);\n>> + continue;\n>> + }\n>> +\n>> + closedir(dir);\n>> +\n>> + printf(\"%s\\n\", ce->name);\n>> + if (!clean_opts.dry_run) {\n>> + if (remove_dir_recursively(&full_path, 0))\n>> + warning_errno(_(\"failed to remove '%s'\"), ce->name);\n>> + }\n> \n> ...and then unconditionally remove the directory, as you stated in the\n> documentation for this clean option.\n> \n> I'm worried whether this is safe; if someone does a merge or rebase,\n> there could be tracked-and-modified/conflicted files outside the\n> sparse specification in the working tree.\n\nThe conflicted files will not collapse to sparse directory entries.\n\nDoes that ease your concern on that front?\n\n> Even after resolving such a merge and committing, the paths may remain\n> around until the user does a 'git sparse-checkout reapply' (I don't\n> remember details here, but our documentation for reapply certainly\n> says so), and since the file might stick around, the user may make\n> further modifications to such a file.\n> \n> ...or will the convert_to_sparse() call above fail in all these cases?\n> If it does, should it give a better and more useful error message\n> than \"failed to convert index to a sparse index\" and rather e.g. \"path\n> %s has modifications; please stage or revert first\"?\n\nIt won't fail. It just won't collapse as far.\n\nYou do make a good point that there could be extra help messages to say\nthat there are uncollapsed directories (detectable by seeing a blob path\nwith the skip-worktree bit on, maybe). I will think on this.\n\nThanks,\n-Stolee\n\n"},{"id":"521706","messageId":"xmqq5xg1cnwd.fsf@gitster.g","threadId":"63760","inReplyTo":"2503c79c-68f3-4ed5-bbfd-3a7af07a89cc@gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-09T16:46:58Z","receivedAt":"2025-07-09T16:47:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> A curious user may want to know \"why are these directories still\n> around?\" and the more verbose output would assist.\n\nUnderstood. That one is what I was primarily after, as opposed to\n\"These directories will be gone, as there is nothing interesting or\nworth saving\", which I find much less interesting (and perhaps\nshould only be shown with --verbose, as opposed to \"this will be\nkept even though it is out of cone, as it contains these things that\nmay worth saving\", which I think is something the user would care\nmore).\n\n>> I wonder if we need \"-x\" and/or \"-X\" options \"clean\" has (and\n>> perhaps \"-d\" that is a no-op, as the whole point of this subcommand\n>> is about removing directories from the working tree) to control its\n>> operation a bit finer-grained way.\n> I'm of two minds here.\n\nSame here, and that is why I said \"I wonder\" ;-)\n"},{"id":"521710","messageId":"CABPp-BFO3Mx9LHPxjbdm2ayvfs2gx9nC9vJnW=zuARgCYrQzNQ@mail.gmail.com","threadId":"63760","inReplyTo":"492244b9-2402-4e8e-8599-8bcd5d27e2f8@gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-09T17:35:02Z","receivedAt":"2025-07-09T17:35:14Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jul 9, 2025 at 9:13 AM Derrick Stolee wrote:\n>\n> On 7/8/2025 5:43 PM, Elijah Newren wrote:\n> > On Tue, Jul 8, 2025 at 4:20 AM Derrick Stolee via GitGitGadget\n> > wrote:\n> >>\n\n[...]\n> >> Add a new subcommand to 'git sparse-checkout' that removes these\n> >> tracked-but-sparse directories, including any excluded or ignored files\n> >> underneath.\n> >\n> > \"including\"?\n>\n> Yes. If we leave the ignored files then we have not accomplished our\n> goal in deleting the sparse directories.\n\nI understood that; my interest was more in what is being glossed over\nwith this word, i.e. what else would be removed.\n\n> >> This is the most extreme method for doing this, but it works\n> >> when the sparse-checkout is in cone mode and is expected to rescope\n> >> based on directories, not files.\n> >\n> > So is this also meant for cone mode without sparse index turned on?\n> > What about non-cone mode?\n>\n> This command die()s if not in cone mode. We can consider future\n> changes that perform similar actions in non-cone mode, but I'm\n> not sure if there is a valuable need in that case.\n\nThat answers the second question; what about the first -- cone mode\nwithout a sparse index? (Edit: you discuss that below, so I'll add\nmore there.)\n\n> >> Be sure to add a --dry-run option so users can predict what will be\n> >> deleted. In general, output the directories that are being removed so\n> >> users can know what was removed.\n> >\n> > Is greater fidelity of interest when there are multiple different\n> > types of files contained? For example, \"git status\" lists individual\n> > files within a directory, unless it find an ignored directory and then\n> > it simply lists the directory. That means we get more fidelity when\n> > it's warranted, and less when it's not. I'm not sure if that's a\n> > perfect analogy, though; it may well be that we don't need the same\n> > kind of fidelity that `git status` provides. (And I'm kind of\n> > guessing it isn't needed, except in error cases, but I'm just asking.)\n> >\n> >> Note that untracked directories remain.\n> >\n> > What does this mean? If the sparse directory had an untracked\n> > directory within it then it'll be left on disk, you will only clean up\n> > untracked files at a depth of 1 within the sparse directory?\n> >\n> > Or that untracked directories not contained within a sparse directory\n> > will be left alone?\n>\n> This second part: \"untracked directories not contained within\n> a sparse directory will remain\". This is mostly to point out\n> that we are not saying \"the only directories that remain are\n> tracked directories within the sparse-checkout\" as that could\n> remove valuable temporary directories that are covered by\n> .gitignore or exclude files.\n\nThanks; could this be clarified in the commit message?\n\n> >> Further, directories that\n> >> contain staged changes are not deleted.\n> >\n> > Shouldn't those be safe to delete? When a sparse directory has files\n> > underneath it with staged changes, those roll-up into a staged\n> > sparse-directory tree value, and so we should be able to delete the\n> > file.\n>\n> This is _mostly_ an implementation detail. The sparse index will\n> not represent this directory as a sparse directory, so it's not\n> deleted. (see the next paragraph:)\n\nIf there are files with staged changes underneath a directory, I don't\nsee why the sparse index would not be able to represent that directory\nas a sparse directory.\n\nFrom the implementation, I'm wondering if you meant something entirely\ndifferent by this sentence and that \"contain\" was perhaps a misleading\nword choice; in particular, did you mean that when the directory\nitself has been replaced in the index with some staged file, then you\nleave that staged file/directory alone? If this was the intended\nmeaning, perhaps we just need some rewording to clarify the commit\nmessage since \"contain\" makes me think about paths below the\ndirectory, not another path to replace the directory.\n\nAnd if neither of my guesses are what you meant by this sentence,\nplease do clue me in.\n\n> >> This is a detail that is partly\n> >> hidden by the implementation which relies on collapsing the index to a\n> >> sparse index in-memory and only deleting directories that are listed as\n> >> sparse in the index. If a staged change exists, then that entry is not\n> >> stored as a sparse tree entry and thus remains on-disk until committed\n> >> or reset.\n> > In contrast, the files under the sparse directory with unstaged\n> > changes would be problematic to simply remove.\n> Except that a user is only using this command when they want\n> files outside of the sparse-checkout to be deleted.\n>\n> I'd like to find the right way to make it clear to users who\n> discover this command that they are asking for the following:\n>\n> \"I changed my sparse-checkout and some directories that I\n> expected to be deleted are still around. Delete them as I\n> don't care about them or the files inside anymore.\"\n>\n> Some of the discussion around having a --verbose option (in\n> conjunction with --dry-run) would allow for the following\n> user scenario:\n>\n> \"I changed my sparse-checkout and some directories that I\n> expected to be deleted are still around. Which files are\n> preventing that deletion? I'd like to know what's in the\n> way so I can evaluate if those files are important to me.\"\n\nYes, and this might even be a status-like output, showing whether the\nfiles are untracked, ignored, tracked-and-unmodified, or\ntracked-and-modified.\n\n> >> +'clean'::\n> >> + Remove all files in tracked directories that are outside of the\n> >> + sparse-checkout definition.\n> >\n> > If literal, this sounds unsafe, particularly if run while resolving\n> > merge or rebase conflicts (since those conflicts may occur in paths\n> > outside the sparse checkout definition).\n>\n> If we are in a merge-conflict state, the directory is not\n> collapsed in the sparse index..\n\nAh, good to know.\n\n> >> + This subcommand requires cone-mode\n> >> + sparse-checkout to be sure that we know which directories are\n> >> + both tracked and all contained paths are not in the sparse-checkout.\n> >> + This command can be used to be sure the sparse index works\n> >> + efficiently.\n> >\n> > So...what does it do when in cone mode and the sparse index is not enabled?\n>\n> It doesn't effect the behavior, since we don't care about the on-disk\n> format and instead use an in-memory sparse index to determine which\n> directories to delete.\n>\n> There could be a benefit for users wanting to clean up extra files in\n> their worktree even if they are not using a sparse index. It is less\n> likely that they will discover that they are in that state if they\n> are not pestered by the index expansion advice message.\n\nRight, for cone mode without the sparse index turned on, this new\nsubcommand seems to be a silent no-op (other than burning some\ncomputation time), despite the fact that the wording in the manual\nmight lead users to believe it will do some nice tidying for them.\nWhen the command spends some time working but doesn't do anything and\ndoesn't report anything, the user might then think the command is just\nbuggy. I think the command should probably either (a) do some tidying\nfor them, or (b) give them a warning that the command has no effect\nwhen sparse index is not turned on. Thoughts?\n\n[...]\n> >> + printf(\"%s\\n\", ce->name);\n> >> + if (!clean_opts.dry_run) {\n> >> + if (remove_dir_recursively(&full_path, 0))\n> >> + warning_errno(_(\"failed to remove '%s'\"), ce->name);\n> >> + }\n> >\n> > ...and then unconditionally remove the directory, as you stated in the\n> > documentation for this clean option.\n> >\n> > I'm worried whether this is safe; if someone does a merge or rebase,\n> > there could be tracked-and-modified/conflicted files outside the\n> > sparse specification in the working tree.\n>\n> The conflicted files will not collapse to sparse directory entries.\n>\n> Does that ease your concern on that front?\n\nYes, that does ease my concerns...but it doesn't erase them.\n\nIf someone resolves the conflicted merge or rebase and commits (long\nbefore running this `git sparse-checkout clean` command), what happens\nto those paths? Do these materialized paths persist in the worktree\nafter the commit? I know they did at some point in our\nimplementation, and the current wording of `git sparse-checkout\nreapply` in the manual certainly suggests such paths may stick around\nuntil the user takes manual action:\n\n Reapply the sparsity pattern rules to paths in the working tree.\n Commands like merge or rebase can materialize paths to do their\n work (e.g. in order to show you a conflict), and other\n sparse-checkout commands might fail to sparsify an individual file\n (e.g. because it has unstaged changes or conflicts). In such cases,\n it can make sense to run git sparse-checkout reapply later after\n cleaning up affected paths (e.g. resolving conflicts, undoing or\n committing changes, etc.).\n\nNow, if these paths stick around despite being outside the sparsity\nspecification, users may decide to modify them. And if they have\nmodified them and run your command, won't you succeed in collapsing to\na sparse directory tree? And then wouldn't this cause unstaged\nchanges to be discarded? (I know this is a rare case, because users\nwould probably only merge or rebase changes they made while in their\nsparse-checkout, and thus conflicts would generally be limited to the\nsparse-checkout, but there are at least three ways I can think of that\nconflicts could be triggered outside their sparse checkout, so I think\nit's a realistic scenario that we should think through.)\n\nThrowing away unstaged changes is something that is usually gated\nbehind a forcing flag (e.g. `git reset --hard` or `git checkout\n--force`). I know, we currently also gate removal of untracked files\nin `git clean` behind a `--force` flag as well and I'm not concerned\nwith throwing away untracked files with your command without a forcing\nflag, but I'm just wondering if we should be more careful with\nunstaged changes than with untracked files.\n\nPerhaps everyone is fine with also throwing away unstaged changes as\npart of this command. I could be convinced of that. But if so, that\nshould at least be called out rather explicitly in the commit message,\nthe documentation, and the tests, whereas currently your patch is\nsilent on anything other than untracked and ignored files.\n\n(Or, if my memory is out-of-date about materialized paths persisting\nafter their conflicts are resolved and a commit happens, then it'd\nprobably be worth calling that out in the commit message and perhaps\nalso updating some wording on the reapply subcommand.)\n\n> > Even after resolving such a merge and committing, the paths may remain\n> > around until the user does a 'git sparse-checkout reapply' (I don't\n> > remember details here, but our documentation for reapply certainly\n> > says so), and since the file might stick around, the user may make\n> > further modifications to such a file.\n> >\n> > ...or will the convert_to_sparse() call above fail in all these cases?\n> > If it does, should it give a better and more useful error message\n> > than \"failed to convert index to a sparse index\" and rather e.g. \"path\n> > %s has modifications; please stage or revert first\"?\n>\n> It won't fail. It just won't collapse as far.\n\nOh! Based on this hint, I went and looked up the code for this; it's\nfrom convert_to_sparse_rec(), right? I see something interesting\nthere; does the present-despite-skipped checks (from 82386b44963f\n(Merge branch 'en/present-despite-skipped', 2022-03-09)) cause this\ncollapsing to also fail for unstaged entries? I.e. this part of\nconvert_to_sparse_rec():\n\n if (ce_stage(ce) ||\n S_ISGITLINK(ce->ce_mode) ||\n !(ce->ce_flags & CE_SKIP_WORKTREE))\n can_convert = 0;\n\nThe `ce_stage(ce)` part of it is what prevent it from collapsing when\nthere are conflicts, and I think the `!(ce->ce_flags &\nCE_SKIP_WORKTREE))` would prevent it from collapsing any tracked files\nwhatsoever, whether modified or not, due to the\npresent-despite-skipped checks. Does that sound right?\n\nIn other words, perhaps your clean command as implemented really does\nonly handle untracked and ignored files, and if the user also has\ntracked-but-unmodified or tracked-with-unstaged-changes or\ntracked-with-staged-changes then this command won't actually restore\nperformance for them until they _also_ run `git sparse-checkout\nreapply` ?\n\n> You do make a good point that there could be extra help messages to say\n> that there are uncollapsed directories (detectable by seeing a blob path\n> with the skip-worktree bit on, maybe). I will think on this.\n\nIn order to detect this by the skip-worktree bit, you'd probably need\nto run it as part of the present-despite-skipped checks mentioned\nabove (or run it before those checks clear the skip-worktree bit for\npresent files).\n"},{"id":"521992","messageId":"1af54798-12d0-4ff4-978c-254d9d0312b6@gmail.com","threadId":"63760","inReplyTo":"CABPp-BFO3Mx9LHPxjbdm2ayvfs2gx9nC9vJnW=zuARgCYrQzNQ@mail.gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-07-15T13:38:09Z","receivedAt":"2025-07-15T13:38:11Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 7/9/2025 1:35 PM, Elijah Newren wrote:\n> On Wed, Jul 9, 2025 at 9:13 AM Derrick Stolee wrote:\n>>\n>> On 7/8/2025 5:43 PM, Elijah Newren wrote:\n\n>>>> + This subcommand requires cone-mode\n>>>> + sparse-checkout to be sure that we know which directories are\n>>>> + both tracked and all contained paths are not in the sparse-checkout.\n>>>> + This command can be used to be sure the sparse index works\n>>>> + efficiently.\n>>>\n>>> So...what does it do when in cone mode and the sparse index is not enabled?\n>>\n>> It doesn't effect the behavior, since we don't care about the on-disk\n>> format and instead use an in-memory sparse index to determine which\n>> directories to delete.\n>>\n>> There could be a benefit for users wanting to clean up extra files in\n>> their worktree even if they are not using a sparse index. It is less\n>> likely that they will discover that they are in that state if they\n>> are not pestered by the index expansion advice message.\n> \n> Right, for cone mode without the sparse index turned on, this new\n> subcommand seems to be a silent no-op (other than burning some\n> computation time),\n\nNo, it works without the sparse index off. The sparse index config\nis about whether or not Git _writes_ the index in the sparse format.\n\nThis command works even if the sparse index is not enabled for the\nwritten format, since we can manipulate the in-memory index for the\npurpose of discovering which tracked directories should be sparse\nand thus not in the worktree.\n\n> despite the fact that the wording in the manual> might lead users to believe it will do some nice tidying for them.\n> When the command spends some time working but doesn't do anything and\n> doesn't report anything, the user might then think the command is just\n> buggy. I think the command should probably either (a) do some tidying\n> for them, or (b) give them a warning that the command has no effect\n> when sparse index is not turned on. Thoughts?\nSo your concerns here should not be a problem, since the command\n_does_ do the expected action even if index.sparse=false.\n\n>>> I'm worried whether this is safe; if someone does a merge or rebase,\n>>> there could be tracked-and-modified/conflicted files outside the\n>>> sparse specification in the working tree.\n>>\n>> The conflicted files will not collapse to sparse directory entries.\n>>\n>> Does that ease your concern on that front?\n> \n> Yes, that does ease my concerns...but it doesn't erase them.\n> \n> If someone resolves the conflicted merge or rebase and commits (long\n> before running this `git sparse-checkout clean` command), what happens\n> to those paths? Do these materialized paths persist in the worktree\n> after the commit? I know they did at some point in our\n> implementation, and the current wording of `git sparse-checkout\n> reapply` in the manual certainly suggests such paths may stick around\n> until the user takes manual action:\n> \n> Reapply the sparsity pattern rules to paths in the working tree.\n> Commands like merge or rebase can materialize paths to do their\n> work (e.g. in order to show you a conflict), and other\n> sparse-checkout commands might fail to sparsify an individual file\n> (e.g. because it has unstaged changes or conflicts). In such cases,\n> it can make sense to run git sparse-checkout reapply later after\n> cleaning up affected paths (e.g. resolving conflicts, undoing or\n> committing changes, etc.).\n\nI have a TODO to add test cases around this behavior, so we can\nhave concrete expectations. I'll incorporate them into the existing\ntest cases around merge conflicts.\n\nThis may be another example of Git leaving files around that should\nbe deleted in order to efficiently work with a sparse index.\n\n> Now, if these paths stick around despite being outside the sparsity\n> specification, users may decide to modify them. And if they have\n> modified them and run your command, won't you succeed in collapsing to\n> a sparse directory tree? And then wouldn't this cause unstaged\n> changes to be discarded? (I know this is a rare case, because users\n> would probably only merge or rebase changes they made while in their\n> sparse-checkout, and thus conflicts would generally be limited to the\n> sparse-checkout, but there are at least three ways I can think of that\n> conflicts could be triggered outside their sparse checkout, so I think\n> it's a realistic scenario that we should think through.)\n> \n> Throwing away unstaged changes is something that is usually gated\n> behind a forcing flag (e.g. `git reset --hard` or `git checkout\n> --force`). I know, we currently also gate removal of untracked files\n> in `git clean` behind a `--force` flag as well and I'm not concerned\n> with throwing away untracked files with your command without a forcing\n> flag, but I'm just wondering if we should be more careful with\n> unstaged changes than with untracked files.\n> \n> Perhaps everyone is fine with also throwing away unstaged changes as\n> part of this command. I could be convinced of that. But if so, that\n> should at least be called out rather explicitly in the commit message,\n> the documentation, and the tests, whereas currently your patch is\n> silent on anything other than untracked and ignored files.\n> \n> (Or, if my memory is out-of-date about materialized paths persisting\n> after their conflicts are resolved and a commit happens, then it'd\n> probably be worth calling that out in the commit message and perhaps\n> also updating some wording on the reapply subcommand.)\n\nThese are good considerations that I'll explore with tests (or point\nto existing tests as I investigate). One thing to keep in mind is\nthat the SKIP_WORKTREE bit does some amount of ignoring the worktree\nby not modifying what is there, and that may include some issues\naround reporting the changes in 'status' or staging the changes in\n'add'.\n\nA lot of your concerns seem like they would be satisfied by providing\na verbose file-by-file output of what would be deleted and potentially\nhaving --dry-run be the default mode.\n\n>>> Even after resolving such a merge and committing, the paths may remain\n>>> around until the user does a 'git sparse-checkout reapply' (I don't\n>>> remember details here, but our documentation for reapply certainly\n>>> says so), and since the file might stick around, the user may make\n>>> further modifications to such a file.\n>>>\n>>> ...or will the convert_to_sparse() call above fail in all these cases?\n>>> If it does, should it give a better and more useful error message\n>>> than \"failed to convert index to a sparse index\" and rather e.g. \"path\n>>> %s has modifications; please stage or revert first\"?\n>>\n>> It won't fail. It just won't collapse as far.\n> \n> Oh! Based on this hint, I went and looked up the code for this; it's\n> from convert_to_sparse_rec(), right? I see something interesting\n> there; does the present-despite-skipped checks (from 82386b44963f\n> (Merge branch 'en/present-despite-skipped', 2022-03-09)) cause this\n> collapsing to also fail for unstaged entries? I.e. this part of\n> convert_to_sparse_rec():\n> \n> if (ce_stage(ce) ||\n> S_ISGITLINK(ce->ce_mode) ||\n> !(ce->ce_flags & CE_SKIP_WORKTREE))\n> can_convert = 0;\n> \n> The `ce_stage(ce)` part of it is what prevent it from collapsing when\n> there are conflicts, and I think the `!(ce->ce_flags &\n> CE_SKIP_WORKTREE))` would prevent it from collapsing any tracked files\n> whatsoever, whether modified or not, due to the\n> present-despite-skipped checks. Does that sound right?\n\nThis matches my expectations.\n\n> In other words, perhaps your clean command as implemented really does\n> only handle untracked and ignored files, and if the user also has\n> tracked-but-unmodified or tracked-with-unstaged-changes or\n> tracked-with-staged-changes then this command won't actually restore\n> performance for them until they _also_ run `git sparse-checkout\n> reapply` ?\n\nWorth adding testing to be sure, though I believe 'git sparse-checkout\nclean' would help if they stage any changes (resolving conflicts, if\nany) and commit those results. The files won't get cleaned by 'git\ncommit' but the clean operation should work after that.\n\n>> You do make a good point that there could be extra help messages to say\n>> that there are uncollapsed directories (detectable by seeing a blob path\n>> with the skip-worktree bit on, maybe). I will think on this.\n> \n> In order to detect this by the skip-worktree bit, you'd probably need\n> to run it as part of the present-despite-skipped checks mentioned\n> above (or run it before those checks clear the skip-worktree bit for\n> present files).\n\nI'll check in on this as I explore more deeply in the code.\n\nThanks for the careful review of these fine details!\n-Stolee\n\n"},{"id":"522014","messageId":"CABPp-BEROPjoi8DcfCoPX+2GoS2eysdZK1eiMO3=knYz5VAnqw@mail.gmail.com","threadId":"63760","inReplyTo":"1af54798-12d0-4ff4-978c-254d9d0312b6@gmail.com","subject":"Re: [PATCH 2/3] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-15T17:17:24Z","receivedAt":"2025-07-15T17:17:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 15, 2025 at 6:38 AM Derrick Stolee wrote:\n>\n> On 7/9/2025 1:35 PM, Elijah Newren wrote:\n> > On Wed, Jul 9, 2025 at 9:13 AM Derrick Stolee wrote:\n> >>\n> >> On 7/8/2025 5:43 PM, Elijah Newren wrote:\n>\n[...]\n> > Right, for cone mode without the sparse index turned on, this new\n> > subcommand seems to be a silent no-op (other than burning some\n> > computation time),\n>\n> No, it works without the sparse index off. The sparse index config\n> is about whether or not Git _writes_ the index in the sparse format.\n>\n> This command works even if the sparse index is not enabled for the\n> written format, since we can manipulate the in-memory index for the\n> purpose of discovering which tracked directories should be sparse\n> and thus not in the worktree.\n\nAh...manipulating the in-memory index despite the sparse-index not\nbeing enabled was the detail I was missing. Thanks for explaining.\n\n[...]\n> So your concerns here should not be a problem, since the command\n> _does_ do the expected action even if index.sparse=false.\n\nYep, thanks for straightening me out on this point.\n\n[...]\n> > If someone resolves the conflicted merge or rebase and commits (long\n> > before running this `git sparse-checkout clean` command), what happens\n> > to those paths? Do these materialized paths persist in the worktree\n> > after the commit?\n[...]\n> I have a TODO to add test cases around this behavior, so we can\n> have concrete expectations. I'll incorporate them into the existing\n> test cases around merge conflicts.\n\nThanks.\n\n> This may be another example of Git leaving files around that should\n> be deleted in order to efficiently work with a sparse index.\n\nDeleted...whether or not they have unstaged changes?\n\n[...]\n> One thing to keep in mind is\n> that the SKIP_WORKTREE bit does some amount of ignoring the worktree\n> by not modifying what is there, and that may include some issues\n> around reporting the changes in 'status' or staging the changes in\n> 'add'.\n\nNot sure I follow. SKIP_WORKTREE bit will be cleared for files that\nare present in the working tree before 'git status' or 'git add' ever\nperform their core logic (due to 82386b44963f (Merge branch\n'en/present-despite-skipped', 2022-03-09)), so isn't this point you\nraise moot, or am I misunderstanding something here?\n\n> A lot of your concerns seem like they would be satisfied by providing\n> a verbose file-by-file output of what would be deleted and potentially\n> having --dry-run be the default mode.\n\nI think that'd be helpful, but primarily I wanted either the commit\nmessage to explain why tracked-but-unmodified files and\ntracked-with-unstaged-changes files under the intended-to-be-sparse\ndirectory aren't expected to ever happen in practice, or for the\nmanual to explain what the clean command does with such files.\n\n[...]\n> > Oh! Based on this hint, I went and looked up the code for this; it's\n> > from convert_to_sparse_rec(), right? I see something interesting\n> > there; does the present-despite-skipped checks (from 82386b44963f\n> > (Merge branch 'en/present-despite-skipped', 2022-03-09)) cause this\n> > collapsing to also fail for unstaged entries? I.e. this part of\n> > convert_to_sparse_rec():\n> >\n> > if (ce_stage(ce) ||\n> > S_ISGITLINK(ce->ce_mode) ||\n> > !(ce->ce_flags & CE_SKIP_WORKTREE))\n> > can_convert = 0;\n> >\n> > The `ce_stage(ce)` part of it is what prevent it from collapsing when\n> > there are conflicts, and I think the `!(ce->ce_flags &\n> > CE_SKIP_WORKTREE))` would prevent it from collapsing any tracked files\n> > whatsoever, whether modified or not, due to the\n> > present-despite-skipped checks. Does that sound right?\n>\n> This matches my expectations.\n\nCool.\n\n> > In other words, perhaps your clean command as implemented really does\n> > only handle untracked and ignored files, and if the user also has\n> > tracked-but-unmodified or tracked-with-unstaged-changes or\n> > tracked-with-staged-changes then this command won't actually restore\n> > performance for them until they _also_ run `git sparse-checkout\n> > reapply` ?\n>\n> Worth adding testing to be sure, though I believe 'git sparse-checkout\n> clean' would help if they stage any changes (resolving conflicts, if\n> any) and commit those results. The files won't get cleaned by 'git\n> commit' but the clean operation should work after that.\n\nWait, what? Doesn't this contradict what you just said above about my\nexplanation matching your expectations? ...or by \"work\" are you just\ncomparing to when we previously talked about how the clean command\nwould abort early when there are conflicts, so by \"work\" you just mean\nthat when conflicts are resolved, the clean command will then \"run\nwithout aborting even though it doesn't actually clean up these\ntracked files either\"?\n\n[...]\n> Thanks for the careful review of these fine details!\n\nThanks for working on this series, being willing to dive into these\ndetails, and patiently explain stuff when there were details I\nmisunderstood!\n"},{"id":"522137","messageId":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.git.1751973594.gitgitgadget@gmail.com","subject":"[PATCH v2 0/8] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:06Z","receivedAt":"2025-07-17T01:34:18Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"NEW: This series is based on 2c5b5565981 (environment: remove the global\nvariable 'sparse_expect_files_outside_of_patterns', 2025-07-01) to build\nupon those cleanups in builtin/sparse-checkout.c.\n\nWhen using cone-mode sparse-checkout, users specify which tracked\ndirectories they want (recursively) and any directory not part of the parent\npaths for those directories are considered \"out of scope\". When changing\nsparse-checkouts, there are a variety of reasons why these \"out of scope\"\ndirectories could remain, including:\n\n * The user has .gitignore or .git/info/exclude files that tell Git to not\n remove files of a certain type.\n * Some filesystem blocker prevented the removal of a tracked file. This is\n usually more of an issue on Windows where a read handle will block file\n deletion.\n\nTypically, this would not mean too much for the user experience. A few extra\nfilesystem checks might be required to satisfy git status commands, but the\nscope of the performance hit is relative to how many cruft files are left\nover in this situation.\n\nHowever, when using the sparse index, these tracked sparse directories cause\nsignificant performance issues. When noticing that the index contains a\nsparse directory but that directory exists on disk, Git needs to expand that\nsparse directory to determine which files are tracked or untracked. The\ncurrent mechanism expands the entire index to a full one, an expensive\noperation that scales with the total number of paths at HEAD and not just\nthe number of cruft files left over.\n\nAdvice was added in 9479a31d603 (advice: warn when sparse index expands,\n2024-07-08) to help users determine that they were in this state. However,\nthe advice doesn't actually recommend helpful ways to get out of this state.\nRecommending \"git clean\" on its own is incomplete, as typically users\nactually need 'git clean -dfx' to clear out the ignored or excluded files.\nEven then, they may need 'git sparse-checkout reapply' afterwards to clear\nthe sparse directories.\n\nThe advice was successful in helping to alert users to the problem, which is\nhow I got wind of many of these cases for how users get into this state.\nIt's now time to give them a tool that helps them out of this state.\n\nThis series adds a new 'git sparse-checkout clean' command that currently\nonly works for cone-mode sparse-checkouts. The only thing it does is\ncollapse the index to a sparse index (as much as possible) and make sure\nthat any sparse directories are removed. These directories are listed to\nstdout.\n\nThis command uses the same '--force' and '--dry-run' options as 'git clean',\nwith integrations with the 'clean.requireForce' config option. There are\nsome concerns that this isn't an obvious way to work with the 'git clean'\ncommand, but I thought we should be consistent here. I did change the error\nmessage to point users to the necessary options.\n\nThis option would be preferred to something like 'git clean -dfx' since it\ndoes not clear the excluded files that are still within the sparse-checkout.\nInstead, it performs the exact filesystem operations required to refresh the\nsparse index performance back to what is expected.\n\nI spent a few weeks debating with myself about whether or not this was the\nright interface, so please suggest alternatives if you have better ideas.\nAmong my rejected ideas include:\n\n * 'git sparse-checkout reapply -f -x' or similar augmentations of\n 'reapply'.\n * 'git clean --sparse' to focus the clean operation on things outside of\n the sparse-checkout.\n\n\nUpdates in V2\n=============\n\n * This series is based on 2c5b5565981 (environment: remove the global\n variable 'sparse_expect_files_outside_of_patterns', 2025-07-01) to build\n upon those cleanups in builtin/sparse-checkout.c.\n * The --force and --dry-run options match 'git clean'.\n * A --verbose option is added. It does not link to the index for\n tracked/untracked/ignored/excluded or clean/modified/staged/conflicted\n status, but instead gives the full list for information.\n * To support the --verbose option, a new for_each_file_in_dir() method is\n added to dir.h.\n * Tests are added to demonstrate the behavior when a sparse directory has a\n merge conflict (fails with an explanation). When adding the test based on\n the previous version's functionality, I realized that the behavior is\n sometimes less effective than git sparse-checkout reapply even after a\n sparse file is committed. To demonstrate this change, the full test is\n created on its own and then a code change is added with the impact on the\n test.\n\nThanks, -Stolee\n\nDerrick Stolee (8):\n sparse-checkout: remove use of the_repository\n sparse-checkout: add basics of 'clean' command\n sparse-checkout: match some 'clean' behavior\n dir: add generic \"walk all files\" helper\n sparse-checkout: add --verbose option to 'clean'\n sparse-index: point users to new 'clean' action\n t: expand tests around sparse merges and clean\n sparse-checkout: make 'clean' clear more files\n\n Documentation/git-sparse-checkout.adoc | 25 ++-\n builtin/sparse-checkout.c | 230 ++++++++++++++++++-------\n dir.c | 28 +++\n dir.h | 14 ++\n sparse-index.c | 3 +-\n t/t1091-sparse-checkout-builtin.sh | 130 ++++++++++++++\n unpack-trees.c | 2 +-\n 7 files changed, 371 insertions(+), 61 deletions(-)\n\n\nbase-commit: 2c5b556598191ae64159dc998dc8f0917d412808\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1941%2Fderrickstolee%2Fgit-sparse-checkout-clean-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1941/derrickstolee/git-sparse-checkout-clean-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1941\n\nRange-diff vs v1:\n\n 1: 3cdc44a9e8c ! 1: 92d0cd41a41 sparse-checkout: remove use of the_repository\n @@ builtin/sparse-checkout.c: static enum sparse_checkout_mode update_cone_mode(int\n \tint mode, record_mode;\n \n @@ builtin/sparse-checkout.c: static int update_modes(int *cone_mode, int *sparse_index)\n - \trecord_mode = (*cone_mode != -1) || !core_apply_sparse_checkout;\n + \trecord_mode = (*cone_mode != -1) || !the_repository->settings.sparse_checkout;\n \n \tmode = update_cone_mode(cone_mode);\n -\tif (record_mode && set_config(mode))\n @@ builtin/sparse-checkout.c: static void add_patterns_literal(int argc, const char\n {\n \tint result;\n @@ builtin/sparse-checkout.c: static int modify_pattern_list(struct strvec *args, int use_stdin,\n + \t\tbreak;\n \t}\n \n - \tif (!core_apply_sparse_checkout) {\n +-\tif (!the_repository->settings.sparse_checkout) {\n -\t\tset_config(MODE_ALL_PATTERNS);\n +-\t\tthe_repository->settings.sparse_checkout = 1;\n ++\tif (!repo->settings.sparse_checkout) {\n +\t\tset_config(repo, MODE_ALL_PATTERNS);\n - \t\tcore_apply_sparse_checkout = 1;\n ++\t\trepo->settings.sparse_checkout = 1;\n \t\tchanged_config = 1;\n \t}\n \n @@ builtin/sparse-checkout.c: static struct sparse_checkout_add_opts {\n \tstatic struct option builtin_sparse_checkout_add_options[] = {\n \t\tOPT_BOOL_F(0, \"skip-checks\", &add_opts.skip_checks,\n @@ builtin/sparse-checkout.c: static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n - \tif (!core_apply_sparse_checkout)\n + \tif (!the_repository->settings.sparse_checkout)\n \t\tdie(_(\"no sparse-checkout to add to\"));\n \n -\trepo_read_index(the_repository);\n @@ builtin/sparse-checkout.c: static int sparse_checkout_disable(int argc, const ch\n \n \tadd_pattern(\"/*\", empty_base, 0, &pl, 0);\n \n --\tprepare_repo_settings(the_repository);\n -\tthe_repository->settings.sparse_index = 0;\n -+\tprepare_repo_settings(repo);\n +\trepo->settings.sparse_index = 0;\n \n -\tif (update_working_directory(&pl))\n 2: 49418e8ec8a ! 2: 7e8f7c2d6c8 sparse-checkout: add 'clean' command\n @@ Metadata\n Author: Derrick Stolee \n \n ## Commit message ##\n - sparse-checkout: add 'clean' command\n + sparse-checkout: add basics of 'clean' command\n \n When users change their sparse-checkout definitions to add new\n directories and remove old ones, there may be a few reasons why\n @@ Commit message\n not be sufficient.\n \n Add a new subcommand to 'git sparse-checkout' that removes these\n - tracked-but-sparse directories, including any excluded or ignored files\n - underneath. This is the most extreme method for doing this, but it works\n + tracked-but-sparse directories. This necessarily removes all files\n + contained within, including tracked and untracked files. Of particular\n + importance are ignored and excluded files which would normally be\n + ignored even by 'git clean -f' unless the '-x' or '-X' option is\n + provided. This is the most extreme method for doing this, but it works\n when the sparse-checkout is in cone mode and is expected to rescope\n based on directories, not files.\n \n - Be sure to add a --dry-run option so users can predict what will be\n - deleted. In general, output the directories that are being removed so\n - users can know what was removed.\n + The current implementation always deletes these sparse directories\n + without warning. This is unacceptable for a released version, but those\n + features will be added in changes coming immediately after this one.\n \n - Note that untracked directories remain. Further, directories that\n - contain staged changes are not deleted. This is a detail that is partly\n - hidden by the implementation which relies on collapsing the index to a\n - sparse index in-memory and only deleting directories that are listed as\n - sparse in the index. If a staged change exists, then that entry is not\n - stored as a sparse tree entry and thus remains on-disk until committed\n - or reset.\n + Note that untracked directories within the sparse-checkout remain.\n + Further, directories that contain staged changes or files in merge\n + conflict states are not deleted. This is a detail that is partly hidden\n + by the implementation which relies on collapsing the index to a sparse\n + index in-memory and only deleting directories that are listed as sparse\n + in the index.\n +\n + If a staged change exists, then that entry is not stored as a sparse\n + tree entry and thus remains on-disk until committed or reset.\n +\n + There are some interesting cases around merge conflict resolution, but\n + that will be carefully analyzed in the future.\n \n Signed-off-by: Derrick Stolee \n \n @@ Documentation/git-sparse-checkout.adoc: flags, with the same meaning as the flag\n +\tsparse-checkout to be sure that we know which directories are\n +\tboth tracked and all contained paths are not in the sparse-checkout.\n +\tThis command can be used to be sure the sparse index works\n -+\tefficiently.\n -++\n -+The `clean` command can also take the `--dry-run` (`-n`) option to list\n -+the directories it would remove without performing any filesystem changes.\n ++\tefficiently, though it does not require enabling the sparse index\n ++ feature via the `index.sparse=true` configuration.\n +\n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\n \tworking directory to include all files.\n \n ## builtin/sparse-checkout.c ##\n +@@\n + #define DISABLE_SIGN_COMPARE_WARNINGS\n + \n + #include \"builtin.h\"\n ++#include \"abspath.h\"\n + #include \"config.h\"\n + #include \"dir.h\"\n + #include \"environment.h\"\n @@\n static const char *empty_base = \"\";\n \n @@ builtin/sparse-checkout.c: static int sparse_checkout_reapply(int argc, const ch\n +\tNULL\n +};\n +\n -+static struct sparse_checkout_clean_opts {\n -+\tint dry_run;\n -+} clean_opts;\n ++static const char *msg_remove = N_(\"Removing %s\\n\");\n +\n +static int sparse_checkout_clean(int argc, const char **argv,\n +\t\t\t\t const char *prefix,\n +\t\t\t\t struct repository *repo)\n +{\n +\tstruct strbuf full_path = STRBUF_INIT;\n ++\tconst char *msg = msg_remove;\n +\tsize_t worktree_len;\n -+\tstatic struct option builtin_sparse_checkout_clean_options[] = {\n -+\t\tOPT_BOOL('n', \"dry-run\", &clean_opts.dry_run,\n -+\t\t\t N_(\"list the directories that would be removed without making filesystem changes\")),\n ++\n ++\tstruct option builtin_sparse_checkout_clean_options[] = {\n +\t\tOPT_END(),\n +\t};\n +\n +\tsetup_work_tree();\n -+\tif (!core_apply_sparse_checkout)\n ++\tif (!repo->settings.sparse_checkout)\n +\t\tdie(_(\"must be in a sparse-checkout to clean directories\"));\n -+\tif (!core_sparse_checkout_cone)\n ++\tif (!repo->settings.sparse_checkout_cone)\n +\t\tdie(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n +\n +\targc = parse_options(argc, argv, prefix,\n @@ builtin/sparse-checkout.c: static int sparse_checkout_reapply(int argc, const ch\n +\tif (repo_read_index(repo) < 0)\n +\t\tdie(_(\"failed to read index\"));\n +\n -+\tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY))\n -+\t\tdie(_(\"failed to convert index to a sparse index\"));\n ++\tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n ++\t repo->index->sparse_index == INDEX_EXPANDED)\n ++\t\tdie(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n +\n +\tstrbuf_addstr(&full_path, repo->worktree);\n +\tstrbuf_addch(&full_path, '/');\n +\tworktree_len = full_path.len;\n +\n +\tfor (size_t i = 0; i < repo->index->cache_nr; i++) {\n -+\t\tDIR* dir;\n +\t\tstruct cache_entry *ce = repo->index->cache[i];\n +\t\tif (!S_ISSPARSEDIR(ce->ce_mode))\n +\t\t\tcontinue;\n +\t\tstrbuf_setlen(&full_path, worktree_len);\n +\t\tstrbuf_add(&full_path, ce->name, ce->ce_namelen);\n +\n -+\t\tdir = opendir(full_path.buf);\n -+\t\tif (!dir)\n -+\t\t\tcontinue;\n -+\t\telse if (ENOENT != errno) {\n -+\t\t\twarning_errno(_(\"failed to check for existence of '%s'\"), ce->name);\n ++\t\tif (!is_directory(full_path.buf))\n +\t\t\tcontinue;\n -+\t\t}\n +\n -+\t\tclosedir(dir);\n ++\t\tprintf(msg, ce->name);\n +\n -+\t\tprintf(\"%s\\n\", ce->name);\n -+\t\tif (!clean_opts.dry_run) {\n -+\t\t\tif (remove_dir_recursively(&full_path, 0))\n -+\t\t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n -+\t\t}\n ++\t\tif (remove_dir_recursively(&full_path, 0))\n ++\t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n +\t}\n +\n +\tstrbuf_release(&full_path);\n @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'check-rules null termin\n +\ttouch repo/folder1/file &&\n +\n +\tcat >expect <<-\\EOF &&\n -+\tdeep/deeper2/\n -+\tfolder1/\n ++\tRemoving deep/deeper2/\n ++\tRemoving folder1/\n +\tEOF\n +\n -+\tgit -C repo sparse-checkout clean --dry-run >out &&\n -+\ttest_cmp expect out &&\n -+\n -+\ttest_path_exists repo/deep/deeper2 &&\n -+\ttest_path_exists repo/folder1 &&\n -+\n +\tgit -C repo sparse-checkout clean >out &&\n +\ttest_cmp expect out &&\n +\n -+\t! test_path_exists repo/deep/deeper2 &&\n -+\t! test_path_exists repo/folder1\n ++\ttest_path_is_missing repo/deep/deeper2 &&\n ++\ttest_path_is_missing repo/folder1\n +'\n +\n +test_expect_success 'clean with staged sparse change' '\n +\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n -+\tmkdir repo/deep/deeper2 repo/folder1 &&\n ++\tmkdir repo/deep/deeper2 repo/folder1 repo/folder2 &&\n +\ttouch repo/deep/deeper2/file &&\n +\ttouch repo/folder1/file &&\n ++\techo dirty >repo/folder2/a &&\n +\n +\tgit -C repo add --sparse folder1/file &&\n +\n ++\t# deletes deep/deeper2/ but leaves folder1/ and folder2/\n +\tcat >expect <<-\\EOF &&\n -+\tdeep/deeper2/\n ++\tRemoving deep/deeper2/\n +\tEOF\n +\n -+\tgit -C repo sparse-checkout clean --dry-run >out &&\n -+\ttest_cmp expect out &&\n -+\n -+\ttest_path_exists repo/deep/deeper2 &&\n -+\ttest_path_exists repo/folder1 &&\n -+\n +\tgit -C repo sparse-checkout clean >out &&\n +\ttest_cmp expect out &&\n +\n -+\t! test_path_exists repo/deep/deeper2 &&\n ++\ttest_path_is_missing repo/deep/deeper2 &&\n +\ttest_path_exists repo/folder1\n +'\n \n -: ----------- > 3: 221f3e5fb0c sparse-checkout: match some 'clean' behavior\n -: ----------- > 4: fd9a20a3922 dir: add generic \"walk all files\" helper\n -: ----------- > 5: f464bb5ed6b sparse-checkout: add --verbose option to 'clean'\n 3: 80d7a7641da = 6: d6dbc0b5ca9 sparse-index: point users to new 'clean' action\n -: ----------- > 7: 0b1a2895b90 t: expand tests around sparse merges and clean\n -: ----------- > 8: 82c24ce5198 sparse-checkout: make 'clean' clear more files\n\n-- \ngitgitgadget\n"},{"id":"522135","messageId":"92d0cd41a412307cf319337ac6ea7e677975e506.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 1/8] sparse-checkout: remove use of the_repository","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:07Z","receivedAt":"2025-07-17T01:34:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe logic for the 'git sparse-checkout' builtin uses the_repository all\nover the place, despite some use of a repository struct in different\nmethod parameters. Complete this removal of the_repository by using\n'repo' when possible.\n\nIn one place, there was already a local variable 'r' that was set to\nthe_repository, so move that to a method parameter.\n\nWe cannot remove the USE_THE_REPOSITORY_VARIABLE declaration as we are\nstill using global constants for the state of the sparse-checkout.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/sparse-checkout.c | 121 ++++++++++++++++++++------------------\n 1 file changed, 64 insertions(+), 57 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 8a0ffba9d4b3..61714bf80be0 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -204,12 +204,12 @@ static void clean_tracked_sparse_directories(struct repository *r)\n \t\tensure_full_index(r->index);\n }\n \n-static int update_working_directory(struct pattern_list *pl)\n+static int update_working_directory(struct repository *r,\n+\t\t\t\t struct pattern_list *pl)\n {\n \tenum update_sparsity_result result;\n \tstruct unpack_trees_options o;\n \tstruct lock_file lock_file = LOCK_INIT;\n-\tstruct repository *r = the_repository;\n \tstruct pattern_list *old_pl;\n \n \t/* If no branch has been checked out, there are no updates to make. */\n@@ -327,7 +327,8 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)\n \tstring_list_clear(&sl, 0);\n }\n \n-static int write_patterns_and_update(struct pattern_list *pl)\n+static int write_patterns_and_update(struct repository *repo,\n+\t\t\t\t struct pattern_list *pl)\n {\n \tchar *sparse_filename;\n \tFILE *fp;\n@@ -336,15 +337,15 @@ static int write_patterns_and_update(struct pattern_list *pl)\n \n \tsparse_filename = get_sparse_checkout_filename();\n \n-\tif (safe_create_leading_directories(the_repository, sparse_filename))\n+\tif (safe_create_leading_directories(repo, sparse_filename))\n \t\tdie(_(\"failed to create directory for sparse-checkout file\"));\n \n \thold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR);\n \n-\tresult = update_working_directory(pl);\n+\tresult = update_working_directory(repo, pl);\n \tif (result) {\n \t\trollback_lock_file(&lk);\n-\t\tupdate_working_directory(NULL);\n+\t\tupdate_working_directory(repo, NULL);\n \t\tgoto out;\n \t}\n \n@@ -372,25 +373,26 @@ enum sparse_checkout_mode {\n \tMODE_CONE_PATTERNS = 2,\n };\n \n-static int set_config(enum sparse_checkout_mode mode)\n+static int set_config(struct repository *repo,\n+\t\t enum sparse_checkout_mode mode)\n {\n \t/* Update to use worktree config, if not already. */\n-\tif (init_worktree_config(the_repository)) {\n+\tif (init_worktree_config(repo)) {\n \t\terror(_(\"failed to initialize worktree config\"));\n \t\treturn 1;\n \t}\n \n-\tif (repo_config_set_worktree_gently(the_repository,\n+\tif (repo_config_set_worktree_gently(repo,\n \t\t\t\t\t \"core.sparseCheckout\",\n \t\t\t\t\t mode ? \"true\" : \"false\") ||\n-\t repo_config_set_worktree_gently(the_repository,\n+\t repo_config_set_worktree_gently(repo,\n \t\t\t\t\t \"core.sparseCheckoutCone\",\n \t\t\t\t\t mode == MODE_CONE_PATTERNS ?\n \t\t\t\t\t\t\"true\" : \"false\"))\n \t\treturn 1;\n \n \tif (mode == MODE_NO_PATTERNS)\n-\t\treturn set_sparse_index_config(the_repository, 0);\n+\t\treturn set_sparse_index_config(repo, 0);\n \n \treturn 0;\n }\n@@ -410,7 +412,7 @@ static enum sparse_checkout_mode update_cone_mode(int *cone_mode) {\n \treturn MODE_ALL_PATTERNS;\n }\n \n-static int update_modes(int *cone_mode, int *sparse_index)\n+static int update_modes(struct repository *repo, int *cone_mode, int *sparse_index)\n {\n \tint mode, record_mode;\n \n@@ -418,20 +420,20 @@ static int update_modes(int *cone_mode, int *sparse_index)\n \trecord_mode = (*cone_mode != -1) || !the_repository->settings.sparse_checkout;\n \n \tmode = update_cone_mode(cone_mode);\n-\tif (record_mode && set_config(mode))\n+\tif (record_mode && set_config(repo, mode))\n \t\treturn 1;\n \n \t/* Set sparse-index/non-sparse-index mode if specified */\n \tif (*sparse_index >= 0) {\n-\t\tif (set_sparse_index_config(the_repository, *sparse_index) < 0)\n+\t\tif (set_sparse_index_config(repo, *sparse_index) < 0)\n \t\t\tdie(_(\"failed to modify sparse-index config\"));\n \n \t\t/* force an index rewrite */\n-\t\trepo_read_index(the_repository);\n-\t\tthe_repository->index->updated_workdir = 1;\n+\t\trepo_read_index(repo);\n+\t\trepo->index->updated_workdir = 1;\n \n \t\tif (!*sparse_index)\n-\t\t\tensure_full_index(the_repository->index);\n+\t\t\tensure_full_index(repo->index);\n \t}\n \n \treturn 0;\n@@ -448,7 +450,7 @@ static struct sparse_checkout_init_opts {\n } init_opts;\n \n static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n-\t\t\t\tstruct repository *repo UNUSED)\n+\t\t\t\tstruct repository *repo)\n {\n \tstruct pattern_list pl;\n \tchar *sparse_filename;\n@@ -464,7 +466,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \t};\n \n \tsetup_work_tree();\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tinit_opts.cone_mode = -1;\n \tinit_opts.sparse_index = -1;\n@@ -473,7 +475,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \t\t\t builtin_sparse_checkout_init_options,\n \t\t\t builtin_sparse_checkout_init_usage, 0);\n \n-\tif (update_modes(&init_opts.cone_mode, &init_opts.sparse_index))\n+\tif (update_modes(repo, &init_opts.cone_mode, &init_opts.sparse_index))\n \t\treturn 1;\n \n \tmemset(&pl, 0, sizeof(pl));\n@@ -485,14 +487,14 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \tif (res >= 0) {\n \t\tfree(sparse_filename);\n \t\tclear_pattern_list(&pl);\n-\t\treturn update_working_directory(NULL);\n+\t\treturn update_working_directory(repo, NULL);\n \t}\n \n-\tif (repo_get_oid(the_repository, \"HEAD\", &oid)) {\n+\tif (repo_get_oid(repo, \"HEAD\", &oid)) {\n \t\tFILE *fp;\n \n \t\t/* assume we are in a fresh repo, but update the sparse-checkout file */\n-\t\tif (safe_create_leading_directories(the_repository, sparse_filename))\n+\t\tif (safe_create_leading_directories(repo, sparse_filename))\n \t\t\tdie(_(\"unable to create leading directories of %s\"),\n \t\t\t sparse_filename);\n \t\tfp = xfopen(sparse_filename, \"w\");\n@@ -511,7 +513,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \tadd_pattern(\"!/*/\", empty_base, 0, &pl, 0);\n \tpl.use_cone_patterns = init_opts.cone_mode;\n \n-\treturn write_patterns_and_update(&pl);\n+\treturn write_patterns_and_update(repo, &pl);\n }\n \n static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *path)\n@@ -674,7 +676,8 @@ static void add_patterns_literal(int argc, const char **argv,\n \tadd_patterns_from_input(pl, argc, argv, use_stdin ? stdin : NULL);\n }\n \n-static int modify_pattern_list(struct strvec *args, int use_stdin,\n+static int modify_pattern_list(struct repository *repo,\n+\t\t\t struct strvec *args, int use_stdin,\n \t\t\t enum modify_type m)\n {\n \tint result;\n@@ -695,23 +698,24 @@ static int modify_pattern_list(struct strvec *args, int use_stdin,\n \t\tbreak;\n \t}\n \n-\tif (!the_repository->settings.sparse_checkout) {\n-\t\tset_config(MODE_ALL_PATTERNS);\n-\t\tthe_repository->settings.sparse_checkout = 1;\n+\tif (!repo->settings.sparse_checkout) {\n+\t\tset_config(repo, MODE_ALL_PATTERNS);\n+\t\trepo->settings.sparse_checkout = 1;\n \t\tchanged_config = 1;\n \t}\n \n-\tresult = write_patterns_and_update(pl);\n+\tresult = write_patterns_and_update(repo, pl);\n \n \tif (result && changed_config)\n-\t\tset_config(MODE_NO_PATTERNS);\n+\t\tset_config(repo, MODE_NO_PATTERNS);\n \n \tclear_pattern_list(pl);\n \tfree(pl);\n \treturn result;\n }\n \n-static void sanitize_paths(struct strvec *args,\n+static void sanitize_paths(struct repository *repo,\n+\t\t\t struct strvec *args,\n \t\t\t const char *prefix, int skip_checks)\n {\n \tint i;\n@@ -752,7 +756,7 @@ static void sanitize_paths(struct strvec *args,\n \n \tfor (i = 0; i < args->nr; i++) {\n \t\tstruct cache_entry *ce;\n-\t\tstruct index_state *index = the_repository->index;\n+\t\tstruct index_state *index = repo->index;\n \t\tint pos = index_name_pos(index, args->v[i], strlen(args->v[i]));\n \n \t\tif (pos < 0)\n@@ -779,7 +783,7 @@ static struct sparse_checkout_add_opts {\n } add_opts;\n \n static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n-\t\t\t struct repository *repo UNUSED)\n+\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_add_options[] = {\n \t\tOPT_BOOL_F(0, \"skip-checks\", &add_opts.skip_checks,\n@@ -796,7 +800,7 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n \tif (!the_repository->settings.sparse_checkout)\n \t\tdie(_(\"no sparse-checkout to add to\"));\n \n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \targc = parse_options(argc, argv, prefix,\n \t\t\t builtin_sparse_checkout_add_options,\n@@ -804,9 +808,9 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n \n \tfor (int i = 0; i < argc; i++)\n \t\tstrvec_push(&patterns, argv[i]);\n-\tsanitize_paths(&patterns, prefix, add_opts.skip_checks);\n+\tsanitize_paths(repo, &patterns, prefix, add_opts.skip_checks);\n \n-\tret = modify_pattern_list(&patterns, add_opts.use_stdin, ADD);\n+\tret = modify_pattern_list(repo, &patterns, add_opts.use_stdin, ADD);\n \n \tstrvec_clear(&patterns);\n \treturn ret;\n@@ -825,7 +829,7 @@ static struct sparse_checkout_set_opts {\n } set_opts;\n \n static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n-\t\t\t struct repository *repo UNUSED)\n+\t\t\t struct repository *repo)\n {\n \tint default_patterns_nr = 2;\n \tconst char *default_patterns[] = {\"/*\", \"!/*/\", NULL};\n@@ -847,7 +851,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \tint ret;\n \n \tsetup_work_tree();\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tset_opts.cone_mode = -1;\n \tset_opts.sparse_index = -1;\n@@ -856,7 +860,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \t\t\t builtin_sparse_checkout_set_options,\n \t\t\t builtin_sparse_checkout_set_usage, 0);\n \n-\tif (update_modes(&set_opts.cone_mode, &set_opts.sparse_index))\n+\tif (update_modes(repo, &set_opts.cone_mode, &set_opts.sparse_index))\n \t\treturn 1;\n \n \t/*\n@@ -870,10 +874,10 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \t} else {\n \t\tfor (int i = 0; i < argc; i++)\n \t\t\tstrvec_push(&patterns, argv[i]);\n-\t\tsanitize_paths(&patterns, prefix, set_opts.skip_checks);\n+\t\tsanitize_paths(repo, &patterns, prefix, set_opts.skip_checks);\n \t}\n \n-\tret = modify_pattern_list(&patterns, set_opts.use_stdin, REPLACE);\n+\tret = modify_pattern_list(repo, &patterns, set_opts.use_stdin, REPLACE);\n \n \tstrvec_clear(&patterns);\n \treturn ret;\n@@ -891,7 +895,7 @@ static struct sparse_checkout_reapply_opts {\n \n static int sparse_checkout_reapply(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_reapply_options[] = {\n \t\tOPT_BOOL(0, \"cone\", &reapply_opts.cone_mode,\n@@ -912,12 +916,12 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n \t\t\t builtin_sparse_checkout_reapply_options,\n \t\t\t builtin_sparse_checkout_reapply_usage, 0);\n \n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n-\tif (update_modes(&reapply_opts.cone_mode, &reapply_opts.sparse_index))\n+\tif (update_modes(repo, &reapply_opts.cone_mode, &reapply_opts.sparse_index))\n \t\treturn 1;\n \n-\treturn update_working_directory(NULL);\n+\treturn update_working_directory(repo, NULL);\n }\n \n static char const * const builtin_sparse_checkout_disable_usage[] = {\n@@ -927,7 +931,7 @@ static char const * const builtin_sparse_checkout_disable_usage[] = {\n \n static int sparse_checkout_disable(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_disable_options[] = {\n \t\tOPT_END(),\n@@ -955,7 +959,7 @@ static int sparse_checkout_disable(int argc, const char **argv,\n \t * are expecting to do that when disabling sparse-checkout.\n \t */\n \tgive_advice_on_expansion = 0;\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tmemset(&pl, 0, sizeof(pl));\n \thashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);\n@@ -965,13 +969,13 @@ static int sparse_checkout_disable(int argc, const char **argv,\n \n \tadd_pattern(\"/*\", empty_base, 0, &pl, 0);\n \n-\tthe_repository->settings.sparse_index = 0;\n+\trepo->settings.sparse_index = 0;\n \n-\tif (update_working_directory(&pl))\n+\tif (update_working_directory(repo, &pl))\n \t\tdie(_(\"error while refreshing working directory\"));\n \n \tclear_pattern_list(&pl);\n-\treturn set_config(MODE_NO_PATTERNS);\n+\treturn set_config(repo, MODE_NO_PATTERNS);\n }\n \n static char const * const builtin_sparse_checkout_check_rules_usage[] = {\n@@ -986,14 +990,17 @@ static struct sparse_checkout_check_rules_opts {\n \tchar *rules_file;\n } check_rules_opts;\n \n-static int check_rules(struct pattern_list *pl, int null_terminated) {\n+static int check_rules(struct repository *repo,\n+\t\t struct pattern_list *pl,\n+\t\t int null_terminated)\n+{\n \tstruct strbuf line = STRBUF_INIT;\n \tstruct strbuf unquoted = STRBUF_INIT;\n \tchar *path;\n \tint line_terminator = null_terminated ? 0 : '\\n';\n \tstrbuf_getline_fn getline_fn = null_terminated ? strbuf_getline_nul\n \t\t: strbuf_getline;\n-\tthe_repository->index->sparse_checkout_patterns = pl;\n+\trepo->index->sparse_checkout_patterns = pl;\n \twhile (!getline_fn(&line, stdin)) {\n \t\tpath = line.buf;\n \t\tif (!null_terminated && line.buf[0] == '\"') {\n@@ -1005,7 +1012,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n \t\t\tpath = unquoted.buf;\n \t\t}\n \n-\t\tif (path_in_sparse_checkout(path, the_repository->index))\n+\t\tif (path_in_sparse_checkout(path, repo->index))\n \t\t\twrite_name_quoted(path, stdout, line_terminator);\n \t}\n \tstrbuf_release(&line);\n@@ -1015,7 +1022,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n }\n \n static int sparse_checkout_check_rules(int argc, const char **argv, const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_check_rules_options[] = {\n \t\tOPT_BOOL('z', NULL, &check_rules_opts.null_termination,\n@@ -1054,7 +1061,7 @@ static int sparse_checkout_check_rules(int argc, const char **argv, const char *\n \t\tfree(sparse_filename);\n \t}\n \n-\tret = check_rules(&pl, check_rules_opts.null_termination);\n+\tret = check_rules(repo, &pl, check_rules_opts.null_termination);\n \tclear_pattern_list(&pl);\n \tfree(check_rules_opts.rules_file);\n \treturn ret;\n@@ -1083,8 +1090,8 @@ int cmd_sparse_checkout(int argc,\n \n \tgit_config(git_default_config, NULL);\n \n-\tprepare_repo_settings(the_repository);\n-\tthe_repository->settings.command_requires_full_index = 0;\n+\tprepare_repo_settings(repo);\n+\trepo->settings.command_requires_full_index = 0;\n \n \treturn fn(argc, argv, prefix, repo);\n }\n-- \ngitgitgadget\n\n"},{"id":"522136","messageId":"7e8f7c2d6c8c740d42bc6d157fa491b558b9ff6a.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 2/8] sparse-checkout: add basics of 'clean' command","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:08Z","receivedAt":"2025-07-17T01:34:20Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nWhen users change their sparse-checkout definitions to add new\ndirectories and remove old ones, there may be a few reasons why\ndirectories no longer in scope remain (ignored or excluded files still\nexist, Windows handles are still open, etc.). When these files still\nexist, the sparse index feature notices that a tracked, but sparse,\ndirectory still exists on disk and thus the index expands. This causes a\nperformance hit _and_ the advice printed isn't very helpful. Using 'git\nclean' isn't enough (generally '-dfx' may be needed) but also this may\nnot be sufficient.\n\nAdd a new subcommand to 'git sparse-checkout' that removes these\ntracked-but-sparse directories. This necessarily removes all files\ncontained within, including tracked and untracked files. Of particular\nimportance are ignored and excluded files which would normally be\nignored even by 'git clean -f' unless the '-x' or '-X' option is\nprovided. This is the most extreme method for doing this, but it works\nwhen the sparse-checkout is in cone mode and is expected to rescope\nbased on directories, not files.\n\nThe current implementation always deletes these sparse directories\nwithout warning. This is unacceptable for a released version, but those\nfeatures will be added in changes coming immediately after this one.\n\nNote that untracked directories within the sparse-checkout remain.\nFurther, directories that contain staged changes or files in merge\nconflict states are not deleted. This is a detail that is partly hidden\nby the implementation which relies on collapsing the index to a sparse\nindex in-memory and only deleting directories that are listed as sparse\nin the index.\n\nIf a staged change exists, then that entry is not stored as a sparse\ntree entry and thus remains on-disk until committed or reset.\n\nThere are some interesting cases around merge conflict resolution, but\nthat will be carefully analyzed in the future.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 11 ++++-\n builtin/sparse-checkout.c | 64 +++++++++++++++++++++++++-\n t/t1091-sparse-checkout-builtin.sh | 38 +++++++++++++++\n 3 files changed, 111 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex 529a8edd9c1e..6db88f00781d 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -9,7 +9,7 @@ git-sparse-checkout - Reduce your working tree to a subset of tracked files\n SYNOPSIS\n --------\n [verse]\n-'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) []\n+'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules | clean) []\n \n \n DESCRIPTION\n@@ -111,6 +111,15 @@ flags, with the same meaning as the flags from the `set` command, in order\n to change which sparsity mode you are using without needing to also respecify\n all sparsity paths.\n \n+'clean'::\n+\tRemove all files in tracked directories that are outside of the\n+\tsparse-checkout definition. This subcommand requires cone-mode\n+\tsparse-checkout to be sure that we know which directories are\n+\tboth tracked and all contained paths are not in the sparse-checkout.\n+\tThis command can be used to be sure the sparse index works\n+\tefficiently, though it does not require enabling the sparse index\n+ feature via the `index.sparse=true` configuration.\n+\n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\n \tworking directory to include all files.\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 61714bf80be0..6fe6ec718fe3 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -2,6 +2,7 @@\n #define DISABLE_SIGN_COMPARE_WARNINGS\n \n #include \"builtin.h\"\n+#include \"abspath.h\"\n #include \"config.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n@@ -23,7 +24,7 @@\n static const char *empty_base = \"\";\n \n static char const * const builtin_sparse_checkout_usage[] = {\n-\tN_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules) []\"),\n+\tN_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules | clean) []\"),\n \tNULL\n };\n \n@@ -924,6 +925,66 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n \treturn update_working_directory(repo, NULL);\n }\n \n+static char const * const builtin_sparse_checkout_clean_usage[] = {\n+\t\"git sparse-checkout clean [-n|--dry-run]\",\n+\tNULL\n+};\n+\n+static const char *msg_remove = N_(\"Removing %s\\n\");\n+\n+static int sparse_checkout_clean(int argc, const char **argv,\n+\t\t\t\t const char *prefix,\n+\t\t\t\t struct repository *repo)\n+{\n+\tstruct strbuf full_path = STRBUF_INIT;\n+\tconst char *msg = msg_remove;\n+\tsize_t worktree_len;\n+\n+\tstruct option builtin_sparse_checkout_clean_options[] = {\n+\t\tOPT_END(),\n+\t};\n+\n+\tsetup_work_tree();\n+\tif (!repo->settings.sparse_checkout)\n+\t\tdie(_(\"must be in a sparse-checkout to clean directories\"));\n+\tif (!repo->settings.sparse_checkout_cone)\n+\t\tdie(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n+\n+\targc = parse_options(argc, argv, prefix,\n+\t\t\t builtin_sparse_checkout_clean_options,\n+\t\t\t builtin_sparse_checkout_clean_usage, 0);\n+\n+\tif (repo_read_index(repo) < 0)\n+\t\tdie(_(\"failed to read index\"));\n+\n+\tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n+\t repo->index->sparse_index == INDEX_EXPANDED)\n+\t\tdie(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n+\n+\tstrbuf_addstr(&full_path, repo->worktree);\n+\tstrbuf_addch(&full_path, '/');\n+\tworktree_len = full_path.len;\n+\n+\tfor (size_t i = 0; i < repo->index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = repo->index->cache[i];\n+\t\tif (!S_ISSPARSEDIR(ce->ce_mode))\n+\t\t\tcontinue;\n+\t\tstrbuf_setlen(&full_path, worktree_len);\n+\t\tstrbuf_add(&full_path, ce->name, ce->ce_namelen);\n+\n+\t\tif (!is_directory(full_path.buf))\n+\t\t\tcontinue;\n+\n+\t\tprintf(msg, ce->name);\n+\n+\t\tif (remove_dir_recursively(&full_path, 0))\n+\t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n+\t}\n+\n+\tstrbuf_release(&full_path);\n+\treturn 0;\n+}\n+\n static char const * const builtin_sparse_checkout_disable_usage[] = {\n \t\"git sparse-checkout disable\",\n \tNULL\n@@ -1079,6 +1140,7 @@ int cmd_sparse_checkout(int argc,\n \t\tOPT_SUBCOMMAND(\"set\", &fn, sparse_checkout_set),\n \t\tOPT_SUBCOMMAND(\"add\", &fn, sparse_checkout_add),\n \t\tOPT_SUBCOMMAND(\"reapply\", &fn, sparse_checkout_reapply),\n+\t\tOPT_SUBCOMMAND(\"clean\", &fn, sparse_checkout_clean),\n \t\tOPT_SUBCOMMAND(\"disable\", &fn, sparse_checkout_disable),\n \t\tOPT_SUBCOMMAND(\"check-rules\", &fn, sparse_checkout_check_rules),\n \t\tOPT_END(),\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex ab3a105ffff2..a48eedf766d2 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1050,5 +1050,43 @@ test_expect_success 'check-rules null termination' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'clean' '\n+\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n+\tmkdir repo/deep/deeper2 repo/folder1 &&\n+\ttouch repo/deep/deeper2/file &&\n+\ttouch repo/folder1/file &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tRemoving deep/deeper2/\n+\tRemoving folder1/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_cmp expect out &&\n+\n+\ttest_path_is_missing repo/deep/deeper2 &&\n+\ttest_path_is_missing repo/folder1\n+'\n+\n+test_expect_success 'clean with staged sparse change' '\n+\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n+\tmkdir repo/deep/deeper2 repo/folder1 repo/folder2 &&\n+\ttouch repo/deep/deeper2/file &&\n+\ttouch repo/folder1/file &&\n+\techo dirty >repo/folder2/a &&\n+\n+\tgit -C repo add --sparse folder1/file &&\n+\n+\t# deletes deep/deeper2/ but leaves folder1/ and folder2/\n+\tcat >expect <<-\\EOF &&\n+\tRemoving deep/deeper2/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_cmp expect out &&\n+\n+\ttest_path_is_missing repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1\n+'\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"522138","messageId":"221f3e5fb0c56b75f8fbfa9f4aa34ae93fad0cdb.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 3/8] sparse-checkout: match some 'clean' behavior","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:09Z","receivedAt":"2025-07-17T01:34:20Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe 'git sparse-checkout clean' subcommand is somewhat similar to 'git\nclean' in that it will delete files that should not be in the worktree.\nThe big difference is that it focuses on the directories that should not\nbe in the worktree due to cone-mode sparse-checkout. It also does not\ndiscriminate in the kinds of files and focuses on deleting entire\ndirectories.\n\nHowever, there are some restrictions that would be good to bring over\nfrom 'git clean', specifically how it refuses to do anything without the\n'-f'/'--force' or '-n'/'--dry-run' arguments. The 'clean.requireForce'\nconfig can be set to 'false' to imply '--force'.\n\nAdd this behavior to avoid accidental deletion of files that cannot be\nrecovered from Git.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 9 ++++\n builtin/sparse-checkout.c | 15 +++++-\n t/t1091-sparse-checkout-builtin.sh | 66 +++++++++++++++++++++++++-\n 3 files changed, 87 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex 6db88f00781d..823a66c40bc5 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -119,6 +119,15 @@ all sparsity paths.\n \tThis command can be used to be sure the sparse index works\n \tefficiently, though it does not require enabling the sparse index\n feature via the `index.sparse=true` configuration.\n++\n+To prevent accidental deletion of worktree files, the `clean` subcommand\n+will not delete any files without the `-f` or `--force` option, unless\n+the `clean.requireForce` config option is set to `false`.\n++\n+The `--dry-run` option will list the directories that would be removed\n+without deleting them. Running in this mode can be helpful to predict the\n+behavior of the clean comand or to determine which kinds of files are left\n+in the sparse directories.\n \n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 6fe6ec718fe3..fe332ff5f941 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -931,6 +931,7 @@ static char const * const builtin_sparse_checkout_clean_usage[] = {\n };\n \n static const char *msg_remove = N_(\"Removing %s\\n\");\n+static const char *msg_would_remove = N_(\"Would remove %s\\n\");\n \n static int sparse_checkout_clean(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n@@ -939,8 +940,12 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \tstruct strbuf full_path = STRBUF_INIT;\n \tconst char *msg = msg_remove;\n \tsize_t worktree_len;\n+\tint force = 0, dry_run = 0;\n+\tint require_force = 1;\n \n \tstruct option builtin_sparse_checkout_clean_options[] = {\n+\t\tOPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n+\t\tOPT__FORCE(&force, N_(\"force\"), PARSE_OPT_NOCOMPLETE),\n \t\tOPT_END(),\n \t};\n \n@@ -954,6 +959,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \t\t\t builtin_sparse_checkout_clean_options,\n \t\t\t builtin_sparse_checkout_clean_usage, 0);\n \n+\trepo_config_get_bool(repo, \"clean.requireforce\", &require_force);\n+\tif (require_force && !force && !dry_run)\n+\t\tdie(_(\"for safety, refusing to clean without one of --force or --dry-run\"));\n+\n+\tif (dry_run)\n+\t\tmsg = msg_would_remove;\n+\n \tif (repo_read_index(repo) < 0)\n \t\tdie(_(\"failed to read index\"));\n \n@@ -977,7 +989,8 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \n \t\tprintf(msg, ce->name);\n \n-\t\tif (remove_dir_recursively(&full_path, 0))\n+\t\tif (dry_run <= 0 &&\n+\t\t remove_dir_recursively(&full_path, 0))\n \t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n \t}\n \ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex a48eedf766d2..69f5a6dcc689 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1056,12 +1056,29 @@ test_expect_success 'clean' '\n \ttouch repo/deep/deeper2/file &&\n \ttouch repo/folder1/file &&\n \n+\ttest_must_fail git -C repo sparse-checkout clean 2>err &&\n+\tgrep \"refusing to clean\" err &&\n+\n+\tgit -C repo config clean.requireForce true &&\n+\ttest_must_fail git -C repo sparse-checkout clean 2>err &&\n+\tgrep \"refusing to clean\" err &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tWould remove deep/deeper2/\n+\tWould remove folder1/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run >out &&\n+\ttest_cmp expect out &&\n+\ttest_path_exists repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1 &&\n+\n \tcat >expect <<-\\EOF &&\n \tRemoving deep/deeper2/\n \tRemoving folder1/\n \tEOF\n \n-\tgit -C repo sparse-checkout clean >out &&\n+\tgit -C repo sparse-checkout clean -f >out &&\n \ttest_cmp expect out &&\n \n \ttest_path_is_missing repo/deep/deeper2 &&\n@@ -1077,16 +1094,61 @@ test_expect_success 'clean with staged sparse change' '\n \n \tgit -C repo add --sparse folder1/file &&\n \n+\tcat >expect <<-\\EOF &&\n+\tWould remove deep/deeper2/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run >out &&\n+\ttest_cmp expect out &&\n+\ttest_path_exists repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1 &&\n+\ttest_path_exists repo/folder2 &&\n+\n \t# deletes deep/deeper2/ but leaves folder1/ and folder2/\n \tcat >expect <<-\\EOF &&\n \tRemoving deep/deeper2/\n \tEOF\n \n+\t# The previous test case checked the -f option, so\n+\t# test the config option in this one.\n+\tgit -C repo config clean.requireForce false &&\n \tgit -C repo sparse-checkout clean >out &&\n \ttest_cmp expect out &&\n \n \ttest_path_is_missing repo/deep/deeper2 &&\n-\ttest_path_exists repo/folder1\n+\ttest_path_exists repo/folder1 &&\n+\ttest_path_exists repo/folder2\n+'\n+\n+test_expect_success 'clean with merge conflict status' '\n+\tgit clone repo clean-merge &&\n+\n+\techo dirty >clean-merge/deep/deeper2/a &&\n+\ttouch clean-merge/folder2/extra &&\n+\n+\tcat >input <<-EOF &&\n+\t0 $ZERO_OID\tfolder1/a\n+\t100644 $(git -C clean-merge rev-parse HEAD:folder1/a) 1\tfolder1/a\n+\tEOF\n+\tgit -C clean-merge update-index --index-info err &&\n+\tgrep \"failed to convert index to a sparse index\" err &&\n+\n+\tmkdir -p clean-merge/folder1/ &&\n+\techo merged >clean-merge/folder1/a &&\n+\tgit -C clean-merge add --sparse folder1/a &&\n+\n+\t# deletes folder2/ but leaves staged change in folder1\n+\t# and dirty change in deep/deeper2/\n+\tcat >expect <<-\\EOF &&\n+\tRemoving folder2/\n+\tEOF\n+\n+\tgit -C clean-merge sparse-checkout clean -f >out &&\n+\ttest_cmp expect out\n '\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"522139","messageId":"fd9a20a392265a5aaed27301a3b324164bb0af86.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 4/8] dir: add generic \"walk all files\" helper","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:10Z","receivedAt":"2025-07-17T01:34:21Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThere is sometimes a need to visit every file within a directory,\nrecursively. The main example is remove_dir_recursively(), though it has\nsome extra flags that make it want to iterate over paths in a custom\nway. There is also the fill_directory() approach but that involves an\nindex and a pathspec.\n\nThis change adds a new for_each_file_in_dir() method that will be\nhelpful in the next change.\n\nSigned-off-by: Derrick Stolee \n---\n dir.c | 28 ++++++++++++++++++++++++++++\n dir.h | 14 ++++++++++++++\n 2 files changed, 42 insertions(+)\n\ndiff --git a/dir.c b/dir.c\nindex d2b0a5aef670..2e567ff92746 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -30,6 +30,7 @@\n #include \"read-cache-ll.h\"\n #include \"setup.h\"\n #include \"sparse-index.h\"\n+#include \"strbuf.h\"\n #include \"submodule-config.h\"\n #include \"symlinks.h\"\n #include \"trace2.h\"\n@@ -87,6 +88,33 @@ struct dirent *readdir_skip_dot_and_dotdot(DIR *dirp)\n \treturn e;\n }\n \n+int for_each_file_in_dir(struct strbuf *path, file_iterator fn, const void *data)\n+{\n+\tstruct dirent *e;\n+\tint res = 0;\n+\tsize_t baselen = path->len;\n+\tDIR *dir = opendir(path->buf);\n+\n+\tif (!dir)\n+\t\treturn 0;\n+\n+\twhile (!res && (e = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n+\t\tunsigned char dtype = get_dtype(e, path, 0);\n+\t\tstrbuf_setlen(path, baselen);\n+\t\tstrbuf_addstr(path, e->d_name);\n+\n+\t\tif (dtype == DT_REG) {\n+\t\t\tres = fn(path->buf, data);\n+\t\t} else if (dtype == DT_DIR) {\n+\t\t\tstrbuf_addch(path, '/');\n+\t\t\tres = for_each_file_in_dir(path, fn, data);\n+\t\t}\n+\t}\n+\n+\tclosedir(dir);\n+\treturn res;\n+}\n+\n int count_slashes(const char *s)\n {\n \tint cnt = 0;\ndiff --git a/dir.h b/dir.h\nindex d7e71aa8daa7..f4235cc12a2f 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -536,6 +536,20 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);\n */\n int remove_dir_recursively(struct strbuf *path, int flag);\n \n+/*\n+ * This function pointer type is called on each file discovered in\n+ * for_each_file_in_dir. The iteration stops if this method returns\n+ * non-zero.\n+ */\n+typedef int (*file_iterator)(const char *path, const void *data);\n+\n+struct strbuf;\n+/*\n+ * Given a directory path, recursively visit each file within, including\n+ * within subdirectories.\n+ */\n+int for_each_file_in_dir(struct strbuf *path, file_iterator fn, const void *data);\n+\n /*\n * Tries to remove the path, along with leading empty directories so long as\n * those empty directories are not startup_info->original_cwd. Ignores\n-- \ngitgitgadget\n\n"},{"id":"522140","messageId":"f464bb5ed6be91940c3abb54b77cb7b9d893bd67.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 5/8] sparse-checkout: add --verbose option to 'clean'","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:11Z","receivedAt":"2025-07-17T01:34:22Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe 'git sparse-checkout clean' subcommand is focused on directories,\ndeleting any tracked sparse directories to clean up the worktree and\nmake the sparse index feature work optimally.\n\nHowever, this directory-focused approach can leave users wondering why\nthose directories exist at all. In my experience, these files are left\nover due to ignore or exclude patterns, Windows file handles, or\npossibly merge conflict resolutions.\n\nAdd a new '--verbose' option for users to see all the files that are\nbeing deleted (with '--force') or would be deleted (with '--dry-run').\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 5 +++++\n builtin/sparse-checkout.c | 28 ++++++++++++++++++++++++--\n t/t1091-sparse-checkout-builtin.sh | 14 ++++++++++---\n 3 files changed, 42 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex 823a66c40bc5..604f53f77caf 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -128,6 +128,11 @@ The `--dry-run` option will list the directories that would be removed\n without deleting them. Running in this mode can be helpful to predict the\n behavior of the clean comand or to determine which kinds of files are left\n in the sparse directories.\n++\n+The `--verbose` option will list every file within the directories that\n+are considered for removal. This option is helpful to determine if those\n+files are actually important or perhaps to explain why the directory is\n+still present despite the current sparse-checkout.\n \n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex fe332ff5f941..f38a0809c098 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -930,6 +930,26 @@ static char const * const builtin_sparse_checkout_clean_usage[] = {\n \tNULL\n };\n \n+static int list_file_iterator(const char *path, const void *data)\n+{\n+\tconst char *msg = data;\n+\n+\tprintf(msg, path);\n+\treturn 0;\n+}\n+\n+static void list_every_file_in_dir(const char *msg,\n+\t\t\t\t const char *directory)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&path, directory);\n+\tfprintf(stderr, \"list every file in %s\\n\", directory);\n+\n+\tfor_each_file_in_dir(&path, list_file_iterator, msg);\n+\tstrbuf_release(&path);\n+}\n+\n static const char *msg_remove = N_(\"Removing %s\\n\");\n static const char *msg_would_remove = N_(\"Would remove %s\\n\");\n \n@@ -940,12 +960,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \tstruct strbuf full_path = STRBUF_INIT;\n \tconst char *msg = msg_remove;\n \tsize_t worktree_len;\n-\tint force = 0, dry_run = 0;\n+\tint force = 0, dry_run = 0, verbose = 0;\n \tint require_force = 1;\n \n \tstruct option builtin_sparse_checkout_clean_options[] = {\n \t\tOPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n \t\tOPT__FORCE(&force, N_(\"force\"), PARSE_OPT_NOCOMPLETE),\n+\t\tOPT__VERBOSE(&verbose, N_(\"report each affected file, not just directories\")),\n \t\tOPT_END(),\n \t};\n \n@@ -987,7 +1008,10 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \t\tif (!is_directory(full_path.buf))\n \t\t\tcontinue;\n \n-\t\tprintf(msg, ce->name);\n+\t\tif (verbose)\n+\t\t\tlist_every_file_in_dir(msg, ce->name);\n+\t\telse\n+\t\t\tprintf(msg, ce->name);\n \n \t\tif (dry_run <= 0 &&\n \t\t remove_dir_recursively(&full_path, 0))\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex 69f5a6dcc689..9a89b902c3f5 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1052,9 +1052,9 @@ test_expect_success 'check-rules null termination' '\n \n test_expect_success 'clean' '\n \tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n-\tmkdir repo/deep/deeper2 repo/folder1 &&\n+\tmkdir -p repo/deep/deeper2 repo/folder1/extra/inside &&\n \ttouch repo/deep/deeper2/file &&\n-\ttouch repo/folder1/file &&\n+\ttouch repo/folder1/extra/inside/file &&\n \n \ttest_must_fail git -C repo sparse-checkout clean 2>err &&\n \tgrep \"refusing to clean\" err &&\n@@ -1071,7 +1071,15 @@ test_expect_success 'clean' '\n \tgit -C repo sparse-checkout clean --dry-run >out &&\n \ttest_cmp expect out &&\n \ttest_path_exists repo/deep/deeper2 &&\n-\ttest_path_exists repo/folder1 &&\n+\ttest_path_exists repo/folder1/extra/inside/file &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tWould remove deep/deeper2/file\n+\tWould remove folder1/extra/inside/file\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run --verbose >out &&\n+\ttest_cmp expect out &&\n \n \tcat >expect <<-\\EOF &&\n \tRemoving deep/deeper2/\n-- \ngitgitgadget\n\n"},{"id":"522141","messageId":"d6dbc0b5ca9cdcd5956e02a0fe603b170dbae1ce.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 6/8] sparse-index: point users to new 'clean' action","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:12Z","receivedAt":"2025-07-17T01:34:23Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nIn my experience, the most-common reason that the sparse index must\nexpand to a full one is because there is some leftover file in a tracked\ndirectory that is now outside of the sparse-checkout. The new 'git\nsparse-checkout clean' command will find and delete these directories,\nso point users to it when they hit the sparse index expansion advice.\n\nSigned-off-by: Derrick Stolee \n---\n sparse-index.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/sparse-index.c b/sparse-index.c\nindex ff33b8516b9f..cbf2bf618c37 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -31,7 +31,8 @@ int give_advice_on_expansion = 1;\n \t\"Your working directory likely has contents that are outside of\\n\" \\\n \t\"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n \t\"see your sparse-checkout definition and compare it to your working\\n\" \\\n-\t\"directory contents. Running 'git clean' may assist in this cleanup.\"\n+\t\"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n+\t\"in this cleanup.\"\n \n struct modify_index_context {\n \tstruct index_state *write;\n-- \ngitgitgadget\n\n"},{"id":"522142","messageId":"0b1a2895b9018fe15a931a097ca18fed8d9c7676.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 7/8] t: expand tests around sparse merges and clean","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:13Z","receivedAt":"2025-07-17T01:34:24Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nWith the current implementation of 'git sparse-checkout clean', we\nnotice that a file that was in a conflicted state does not get cleaned\nup because of some internal details around the SKIP_WORKTREE bit.\n\nThis test is documenting the current behavior before we update it in the\nfollowing change.\n\nSigned-off-by: Derrick Stolee \n---\n t/t1091-sparse-checkout-builtin.sh | 56 ++++++++++++++++++------------\n 1 file changed, 34 insertions(+), 22 deletions(-)\n\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex 9a89b902c3f5..116ad7c9a20e 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1128,35 +1128,47 @@ test_expect_success 'clean with staged sparse change' '\n \ttest_path_exists repo/folder2\n '\n \n-test_expect_success 'clean with merge conflict status' '\n-\tgit clone repo clean-merge &&\n+test_expect_success 'sparse-checkout operations with merge conflicts' '\n+\tgit clone repo merge &&\n \n-\techo dirty >clean-merge/deep/deeper2/a &&\n-\ttouch clean-merge/folder2/extra &&\n+\t(\n+\t\tcd merge &&\n+\t\tmkdir -p folder1/even/more/dirs &&\n+\t\techo base >folder1/even/more/dirs/file &&\n+\t\tgit add folder1 &&\n+\t\tgit commit -m \"base\" &&\n \n-\tcat >input <<-EOF &&\n-\t0 $ZERO_OID\tfolder1/a\n-\t100644 $(git -C clean-merge rev-parse HEAD:folder1/a) 1\tfolder1/a\n-\tEOF\n-\tgit -C clean-merge update-index --index-info folder1/even/more/dirs/file &&\n+\t\tgit commit -a -m \"right\" &&\n \n-\tgit -C clean-merge sparse-checkout set deep/deeper1 &&\n+\t\tgit checkout -b left HEAD~1 &&\n+\t\techo left >folder1/even/more/dirs/file &&\n+\t\tgit commit -a -m \"left\" &&\n \n-\ttest_must_fail git -C clean-merge sparse-checkout clean -f 2>err &&\n-\tgrep \"failed to convert index to a sparse index\" err &&\n+\t\tgit checkout -b merge &&\n+\t\tgit sparse-checkout set deep/deeper1 &&\n \n-\tmkdir -p clean-merge/folder1/ &&\n-\techo merged >clean-merge/folder1/a &&\n-\tgit -C clean-merge add --sparse folder1/a &&\n+\t\ttest_must_fail git merge -m \"will-conflict\" right &&\n \n-\t# deletes folder2/ but leaves staged change in folder1\n-\t# and dirty change in deep/deeper2/\n-\tcat >expect <<-\\EOF &&\n-\tRemoving folder2/\n-\tEOF\n+\t\ttest_must_fail git sparse-checkout clean -f 2>err &&\n+\t\tgrep \"failed to convert index to a sparse index\" err &&\n \n-\tgit -C clean-merge sparse-checkout clean -f >out &&\n-\ttest_cmp expect out\n+\t\techo merged >folder1/even/more/dirs/file &&\n+\t\tgit add --sparse folder1 &&\n+\t\tgit merge --continue &&\n+\n+\t\ttest_path_exists folder1/even/more/dirs/file &&\n+\n+\t\t# clean does not remove the file, because the\n+\t\t# SKIP_WORKTREE bit was not cleared by the merge command.\n+\t\tgit sparse-checkout clean -f >out &&\n+\t\ttest_line_count = 0 out &&\n+\t\ttest_path_exists folder1/even/more/dirs/file &&\n+\n+\t\tgit sparse-checkout reapply &&\n+\t\ttest_path_is_missing folder1\n+\t)\n '\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"522143","messageId":"82c24ce51980d85e1a53e746b462397e6e6c908a.1752716054.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v2 8/8] sparse-checkout: make 'clean' clear more files","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T01:34:14Z","receivedAt":"2025-07-17T01:34:25Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe 'git sparse-checkout clean' command is designed to be a one-command\nway to get the worktree in a state such that a sparse index would\noperate efficiently. The previous change demonstrated that files outside\nthe sparse-checkout that were committed due to a merge conflict would\npersist despite attempts to run 'git sparse-checkout clean' and instead\na 'git sparse-checkout reapply' would be required.\n\nInstead of requiring users to run both commands, update 'clean' to be\nmore ruthless about tracked sparse directories. The key here is to make\nsure that the SKIP_WORKTREE bit is removed from more paths in the index\nusing update_sparsity() before compressing the index to a sparse one\nin-memory.\n\nThe tricky part here is that update_sparsity() was previously assuming\nthat it would be in 'update' mode and would change the worktree as it\nmade changes. However, we do not want to make these worktree changes at\nthis point, instead relying on our later logic (that integrates with\n--dry-run and --verbose options) to perform those steps.\n\nOne side-effect here is that we also clear out staged files that exist\nin the worktree, but they would also appear in the verbose output as\npart of the dry run.\n\nThe final test in t1091 demonstrates that we no longer need the\n'reapply' subcommand for merge resolutions. It also fixes an earlier\ncase where 'git add --sparse' clears the SKIP_WORKTREE bit and avoids a\ndirectory deletion.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/sparse-checkout.c | 8 ++++++++\n t/t1091-sparse-checkout-builtin.sh | 24 +++++++++++++++++-------\n unpack-trees.c | 2 +-\n 3 files changed, 26 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex f38a0809c098..1d1d5208a3ba 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -962,6 +962,7 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \tsize_t worktree_len;\n \tint force = 0, dry_run = 0, verbose = 0;\n \tint require_force = 1;\n+\tstruct unpack_trees_options o = { 0 };\n \n \tstruct option builtin_sparse_checkout_clean_options[] = {\n \t\tOPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n@@ -990,6 +991,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \tif (repo_read_index(repo) < 0)\n \t\tdie(_(\"failed to read index\"));\n \n+\to.verbose_update = verbose;\n+\to.update = 0; /* skip modifying the worktree here. */\n+\to.head_idx = -1;\n+\to.src_index = o.dst_index = repo->index;\n+\tif (update_sparsity(&o, NULL))\n+\t\twarning(_(\"failed to reapply sparse-checkout patterns\"));\n+\n \tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n \t repo->index->sparse_index == INDEX_EXPANDED)\n \t\tdie(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex 116ad7c9a20e..4b9078d90a61 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1104,6 +1104,7 @@ test_expect_success 'clean with staged sparse change' '\n \n \tcat >expect <<-\\EOF &&\n \tWould remove deep/deeper2/\n+\tWould remove folder1/\n \tEOF\n \n \tgit -C repo sparse-checkout clean --dry-run >out &&\n@@ -1115,6 +1116,7 @@ test_expect_success 'clean with staged sparse change' '\n \t# deletes deep/deeper2/ but leaves folder1/ and folder2/\n \tcat >expect <<-\\EOF &&\n \tRemoving deep/deeper2/\n+\tRemoving folder1/\n \tEOF\n \n \t# The previous test case checked the -f option, so\n@@ -1124,7 +1126,7 @@ test_expect_success 'clean with staged sparse change' '\n \ttest_cmp expect out &&\n \n \ttest_path_is_missing repo/deep/deeper2 &&\n-\ttest_path_exists repo/folder1 &&\n+\ttest_path_is_missing repo/folder1 &&\n \ttest_path_exists repo/folder2\n '\n \n@@ -1147,7 +1149,11 @@ test_expect_success 'sparse-checkout operations with merge conflicts' '\n \t\tgit commit -a -m \"left\" &&\n \n \t\tgit checkout -b merge &&\n-\t\tgit sparse-checkout set deep/deeper1 &&\n+\n+\t\ttouch deep/deeper2/extra &&\n+\t\tgit sparse-checkout set deep/deeper1 2>err &&\n+\t\tgrep \"contains untracked files\" err &&\n+\t\ttest_path_exists deep/deeper2/extra &&\n \n \t\ttest_must_fail git merge -m \"will-conflict\" right &&\n \n@@ -1159,15 +1165,19 @@ test_expect_success 'sparse-checkout operations with merge conflicts' '\n \t\tgit merge --continue &&\n \n \t\ttest_path_exists folder1/even/more/dirs/file &&\n+\t\ttest_path_exists deep/deeper2/extra &&\n+\n+\t\tcat >expect <<-\\EOF &&\n+\t\tRemoving deep/deeper2/\n+\t\tRemoving folder1/\n+\t\tEOF\n \n \t\t# clean does not remove the file, because the\n \t\t# SKIP_WORKTREE bit was not cleared by the merge command.\n \t\tgit sparse-checkout clean -f >out &&\n-\t\ttest_line_count = 0 out &&\n-\t\ttest_path_exists folder1/even/more/dirs/file &&\n-\n-\t\tgit sparse-checkout reapply &&\n-\t\ttest_path_is_missing folder1\n+\t\ttest_cmp expect out &&\n+\t\ttest_path_is_missing folder1 &&\n+\t\ttest_path_is_missing deep/deeper2\n \t)\n '\n \ndiff --git a/unpack-trees.c b/unpack-trees.c\nindex 0e9813bddf04..b8814af1b07c 100644\n--- a/unpack-trees.c\n+++ b/unpack-trees.c\n@@ -2138,7 +2138,7 @@ enum update_sparsity_result update_sparsity(struct unpack_trees_options *o,\n \tindex_state_init(&o->internal.result, o->src_index->repo);\n \n \t/* Sanity checks */\n-\tif (!o->update || o->index_only || o->skip_sparse_checkout)\n+\tif (o->index_only || o->skip_sparse_checkout)\n \t\tBUG(\"update_sparsity() is for reflecting sparsity patterns in working directory\");\n \tif (o->src_index != o->dst_index || o->fn)\n \t\tBUG(\"update_sparsity() called wrong\");\n-- \ngitgitgadget\n"},{"id":"523608","messageId":"CABPp-BFzMLGJwz4QqYtvw3zRYgmC=Mb8T8GCOsrLZqT2z+8H7A@mail.gmail.com","threadId":"63760","inReplyTo":"7e8f7c2d6c8c740d42bc6d157fa491b558b9ff6a.1752716054.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/8] sparse-checkout: add basics of 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-05T21:32:22Z","receivedAt":"2025-08-05T21:32:34Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n wrote:\n\nSorry for the long delay in responding...\n\n[...]\n> Add a new subcommand to 'git sparse-checkout' that removes these\n> tracked-but-sparse directories. This necessarily removes all files\n> contained within, including tracked and untracked files. Of particular\n\nNice to see tracked files also being addressed in v2.\n\n> importance are ignored and excluded files which would normally be\n> ignored even by 'git clean -f' unless the '-x' or '-X' option is\n> provided. This is the most extreme method for doing this, but it works\n> when the sparse-checkout is in cone mode and is expected to rescope\n> based on directories, not files.\n>\n> The current implementation always deletes these sparse directories\n> without warning. This is unacceptable for a released version, but those\n> features will be added in changes coming immediately after this one.\n>\n> Note that untracked directories within the sparse-checkout remain.\n\nYou've changed the wording here relative to v1, but you haven't\naddressed the part that was ambiguous/misleading in v1. In fact, you\nmay have made a different part ambiguous as well, and made readers\nthink that this sentence contradicts your above claims that this\ncommand is meant to clean out untracked directories underneath sparse\ndirectories. Perhaps something like:\n\n\"Note that untracked directories in the sparse-checkout that are not\nwithin sparse directories will not be removed by this command; it only\ncleans up paths under directories that are supposed to be sparse.\"\n\n> Further, directories that contain staged changes or files in merge\n> conflict states are not deleted.\n\nDoesn't this sentence conflict with your above statement that \"This\nnecessarily removes all files contained within, including tracked and\nuntracked files.\"?\n\n> This is a detail that is partly hidden\n> by the implementation which relies on collapsing the index to a sparse\n> index in-memory and only deleting directories that are listed as sparse\n> in the index.\n>\n> If a staged change exists, then that entry is not stored as a sparse\n> tree entry and thus remains on-disk until committed or reset.\n\nThis seems a bit surprising -- if a file's modifications are staged,\nthen you can use it and other files in the index to write new trees\nall the way up to the toplevel, so you should be able to get a sparse\ntree entry without problem. I'd only expect problems if you had\nunstaged changes, or higher order stages; perhaps you could clarify\nhere?\n\n> There are some interesting cases around merge conflict resolution, but\n> that will be carefully analyzed in the future.\n\n...okay, so you did clarify for the higher order stages, but I'm still\nconfused about staged vs unstaged without conflicts.\n\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> Documentation/git-sparse-checkout.adoc | 11 ++++-\n> builtin/sparse-checkout.c | 64 +++++++++++++++++++++++++-\n> t/t1091-sparse-checkout-builtin.sh | 38 +++++++++++++++\n> 3 files changed, 111 insertions(+), 2 deletions(-)\n>\n> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n> index 529a8edd9c1e..6db88f00781d 100644\n> --- a/Documentation/git-sparse-checkout.adoc\n> +++ b/Documentation/git-sparse-checkout.adoc\n> @@ -9,7 +9,7 @@ git-sparse-checkout - Reduce your working tree to a subset of tracked files\n> SYNOPSIS\n> --------\n> [verse]\n> -'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) []\n> +'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules | clean) []\n>\n>\n> DESCRIPTION\n> @@ -111,6 +111,15 @@ flags, with the same meaning as the flags from the `set` command, in order\n> to change which sparsity mode you are using without needing to also respecify\n> all sparsity paths.\n>\n> +'clean'::\n> + Remove all files in tracked directories that are outside of the\n> + sparse-checkout definition. This subcommand requires cone-mode\n\nSo, this sentence implies that it'll wipe out all untracked or ignored\nfiles or tracked files with either unstaged, staged, or conflicted\nentries. Your commit message says it'll discuss conflicted entries\nlater, but conflicts about whether staged or unstaged changes will be\nwiped.\n\n> + sparse-checkout to be sure that we know which directories are\n> + both tracked and all contained paths are not in the sparse-checkout.\n> + This command can be used to be sure the sparse index works\n> + efficiently, though it does not require enabling the sparse index\n\n[...]\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n[...]\n> + if (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n> + repo->index->sparse_index == INDEX_EXPANDED)\n> + die(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n\nIn the commit message, though, you said that it'd also fail to convert\na tree with a staged change to a sparse directory. And I thought in\nour discussion on v1 we found out that unstaged changes would prevent\nconverting to sparse. Shouldn't the error message be more general,\nthen?\n\n[...]\n> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\n> index ab3a105ffff2..a48eedf766d2 100755\n> --- a/t/t1091-sparse-checkout-builtin.sh\n> +++ b/t/t1091-sparse-checkout-builtin.sh\n> @@ -1050,5 +1050,43 @@ test_expect_success 'check-rules null termination' '\n> test_cmp expect actual\n> '\n>\n> +test_expect_success 'clean' '\n> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n> + mkdir repo/deep/deeper2 repo/folder1 &&\n> + touch repo/deep/deeper2/file &&\n> + touch repo/folder1/file &&\n> +\n> + cat >expect <<-\\EOF &&\n> + Removing deep/deeper2/\n> + Removing folder1/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean >out &&\n> + test_cmp expect out &&\n> +\n> + test_path_is_missing repo/deep/deeper2 &&\n> + test_path_is_missing repo/folder1\n> +'\n> +\n> +test_expect_success 'clean with staged sparse change' '\n> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n> + mkdir repo/deep/deeper2 repo/folder1 repo/folder2 &&\n> + touch repo/deep/deeper2/file &&\n> + touch repo/folder1/file &&\n> + echo dirty >repo/folder2/a &&\n> +\n> + git -C repo add --sparse folder1/file &&\n> +\n> + # deletes deep/deeper2/ but leaves folder1/ and folder2/\n> + cat >expect <<-\\EOF &&\n> + Removing deep/deeper2/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean >out &&\n> + test_cmp expect out &&\n> +\n> + test_path_is_missing repo/deep/deeper2 &&\n> + test_path_exists repo/folder1\n\nWhat about repo/folder2/ ?\n\nAnyway, this test shows that neither staged nor unstaged changes are\ncleaned up (which at least resolves the conflicting documentation you\nprovided on the matter) -- or would if you also checked repo/folder2.\n\nWhat it doesn't show is that tracked files with neither staged nor\nunstaged changes are not cleaned up either:\n\n$ mkdir repo/folder2\n$ echo dirty >repo/folder2/a\n$ touch repo/folder2/untracked\n$ cd repo\n$ git status --porcelain\n M folder2/a\n?? folder2/untracked\n\n# So, we have both a unstaged change and an untracked file; let's undo\nthe unstaged change\n\n$ git checkout HEAD folder2/a\nUpdated 1 path from 8cc814f\n$ git status --porcelain\n?? folder2/untracked\n$ ls folder2/\na untracked\n\n# Both files are still present -- the untracked file, and the\nuntracked file with no changes either staged or unstaged -- what does\n`git sparse-checkout clean` do?\n\n$ git sparse-checkout clean\n$ ls folder2/\na untracked\n$ git status --porcelain\n?? folder2/untracked\n\n# Absolutely nothing. Not only does it not clean anything up, it\ngives no warnings about not cleaning up what should be cleaned up.\nLet's try sparse-checkout reapply:\n\n$ git sparse-checkout reapply\nwarning: directory 'folder2/' contains untracked files, but is not in\nthe sparse-checkout cone\n$ git status --porcelain\n?? folder2/untracked\n$ ls folder2/\nuntracked\n\n# So `sparse-checkout reapply` does correctly remove folder2/a for us,\nwhile warning about the untracked file. (If folder2/a would have\nstill had changes, it would have warned about it instead of\nremoving.). Let's try `sparse-checkout clean` now...\n\n$ git sparse-checkout clean\nRemoving folder2/\n$ git status --porcelain\n$ ls folder2/\nls: cannot access 'folder2/': No such file or directory\n$\n\nI think these cases either need to be a new testcase or part of this\nlast testcase, and the commit message and documentation should be\nclearer about tracked-and-staged, tracked-with-unstaged-changes, and\ntracked-with-no-changes files...or at least comment that they'll be\ndiscussed later in the patch series. (I have a feeling I just did a\nlot of work to discover as I read your next patches that you cover\nthese later...)\n"},{"id":"523611","messageId":"CABPp-BGQmoPQ9TOLjZPpPPEJ__0rsHgRMJ-nkiW4GcQjPCvaBA@mail.gmail.com","threadId":"63760","inReplyTo":"221f3e5fb0c56b75f8fbfa9f4aa34ae93fad0cdb.1752716054.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 3/8] sparse-checkout: match some 'clean' behavior","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-05T22:06:52Z","receivedAt":"2025-08-05T22:07:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The 'git sparse-checkout clean' subcommand is somewhat similar to 'git\n> clean' in that it will delete files that should not be in the worktree.\n> The big difference is that it focuses on the directories that should not\n> be in the worktree due to cone-mode sparse-checkout. It also does not\n> discriminate in the kinds of files and focuses on deleting entire\n> directories.\n>\n> However, there are some restrictions that would be good to bring over\n> from 'git clean', specifically how it refuses to do anything without the\n> '-f'/'--force' or '-n'/'--dry-run' arguments. The 'clean.requireForce'\n> config can be set to 'false' to imply '--force'.\n>\n> Add this behavior to avoid accidental deletion of files that cannot be\n> recovered from Git.\n\nI'm a bit surprised by this. Given that the only kinds of files that\nthis command cleans out are untracked and ignored files, and Junio's\ncomments about clean.requireForce over in\nhttps://lore.kernel.org/git/xmqqv7o2togi.fsf@gitster.g/, I thought his\ncomments could be interpreted as not wanting clean.requireForce to\napply in more places. Did I misunderstand?\n\nAlternatively, maybe you thought that there were files other than\nuntracked and ignored which `sparse-checkout clean` would clean up,\nand it was because of those files that we wanted the extra protection?\n (In that case, it'd make sense, but it seems to go against what was\ndemonstrated in the final testcase of the previous patch.)\n\n> Signed-off-by: Derrick Stolee \n> ---\n> Documentation/git-sparse-checkout.adoc | 9 ++++\n> builtin/sparse-checkout.c | 15 +++++-\n> t/t1091-sparse-checkout-builtin.sh | 66 +++++++++++++++++++++++++-\n> 3 files changed, 87 insertions(+), 3 deletions(-)\n>\n> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n> index 6db88f00781d..823a66c40bc5 100644\n> --- a/Documentation/git-sparse-checkout.adoc\n> +++ b/Documentation/git-sparse-checkout.adoc\n> @@ -119,6 +119,15 @@ all sparsity paths.\n> This command can be used to be sure the sparse index works\n> efficiently, though it does not require enabling the sparse index\n> feature via the `index.sparse=true` configuration.\n> ++\n> +To prevent accidental deletion of worktree files, the `clean` subcommand\n> +will not delete any files without the `-f` or `--force` option, unless\n> +the `clean.requireForce` config option is set to `false`.\n> ++\n> +The `--dry-run` option will list the directories that would be removed\n> +without deleting them. Running in this mode can be helpful to predict the\n> +behavior of the clean comand or to determine which kinds of files are left\n> +in the sparse directories.\n>\n> 'disable'::\n> Disable the `core.sparseCheckout` config setting, and restore the\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index 6fe6ec718fe3..fe332ff5f941 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -931,6 +931,7 @@ static char const * const builtin_sparse_checkout_clean_usage[] = {\n> };\n>\n> static const char *msg_remove = N_(\"Removing %s\\n\");\n> +static const char *msg_would_remove = N_(\"Would remove %s\\n\");\n>\n> static int sparse_checkout_clean(int argc, const char **argv,\n> const char *prefix,\n> @@ -939,8 +940,12 @@ static int sparse_checkout_clean(int argc, const char **argv,\n> struct strbuf full_path = STRBUF_INIT;\n> const char *msg = msg_remove;\n> size_t worktree_len;\n> + int force = 0, dry_run = 0;\n> + int require_force = 1;\n>\n> struct option builtin_sparse_checkout_clean_options[] = {\n> + OPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n> + OPT__FORCE(&force, N_(\"force\"), PARSE_OPT_NOCOMPLETE),\n> OPT_END(),\n> };\n>\n> @@ -954,6 +959,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n> builtin_sparse_checkout_clean_options,\n> builtin_sparse_checkout_clean_usage, 0);\n>\n> + repo_config_get_bool(repo, \"clean.requireforce\", &require_force);\n> + if (require_force && !force && !dry_run)\n> + die(_(\"for safety, refusing to clean without one of --force or --dry-run\"));\n> +\n> + if (dry_run)\n> + msg = msg_would_remove;\n> +\n> if (repo_read_index(repo) < 0)\n> die(_(\"failed to read index\"));\n>\n> @@ -977,7 +989,8 @@ static int sparse_checkout_clean(int argc, const char **argv,\n>\n> printf(msg, ce->name);\n>\n> - if (remove_dir_recursively(&full_path, 0))\n> + if (dry_run <= 0 &&\n> + remove_dir_recursively(&full_path, 0))\n> warning_errno(_(\"failed to remove '%s'\"), ce->name);\n> }\n>\n> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\n> index a48eedf766d2..69f5a6dcc689 100755\n> --- a/t/t1091-sparse-checkout-builtin.sh\n> +++ b/t/t1091-sparse-checkout-builtin.sh\n> @@ -1056,12 +1056,29 @@ test_expect_success 'clean' '\n> touch repo/deep/deeper2/file &&\n> touch repo/folder1/file &&\n>\n> + test_must_fail git -C repo sparse-checkout clean 2>err &&\n> + grep \"refusing to clean\" err &&\n> +\n> + git -C repo config clean.requireForce true &&\n> + test_must_fail git -C repo sparse-checkout clean 2>err &&\n> + grep \"refusing to clean\" err &&\n> +\n> + cat >expect <<-\\EOF &&\n> + Would remove deep/deeper2/\n> + Would remove folder1/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean --dry-run >out &&\n> + test_cmp expect out &&\n> + test_path_exists repo/deep/deeper2 &&\n> + test_path_exists repo/folder1 &&\n> +\n> cat >expect <<-\\EOF &&\n> Removing deep/deeper2/\n> Removing folder1/\n> EOF\n>\n> - git -C repo sparse-checkout clean >out &&\n> + git -C repo sparse-checkout clean -f >out &&\n> test_cmp expect out &&\n>\n> test_path_is_missing repo/deep/deeper2 &&\n> @@ -1077,16 +1094,61 @@ test_expect_success 'clean with staged sparse change' '\n>\n> git -C repo add --sparse folder1/file &&\n>\n> + cat >expect <<-\\EOF &&\n> + Would remove deep/deeper2/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean --dry-run >out &&\n> + test_cmp expect out &&\n> + test_path_exists repo/deep/deeper2 &&\n> + test_path_exists repo/folder1 &&\n> + test_path_exists repo/folder2 &&\n> +\n> # deletes deep/deeper2/ but leaves folder1/ and folder2/\n> cat >expect <<-\\EOF &&\n> Removing deep/deeper2/\n> EOF\n>\n> + # The previous test case checked the -f option, so\n> + # test the config option in this one.\n> + git -C repo config clean.requireForce false &&\n> git -C repo sparse-checkout clean >out &&\n> test_cmp expect out &&\n>\n> test_path_is_missing repo/deep/deeper2 &&\n> - test_path_exists repo/folder1\n> + test_path_exists repo/folder1 &&\n> + test_path_exists repo/folder2\n> +'\n> +\n> +test_expect_success 'clean with merge conflict status' '\n> + git clone repo clean-merge &&\n> +\n> + echo dirty >clean-merge/deep/deeper2/a &&\n> + touch clean-merge/folder2/extra &&\n> +\n> + cat >input <<-EOF &&\n> + 0 $ZERO_OID folder1/a\n> + 100644 $(git -C clean-merge rev-parse HEAD:folder1/a) 1 folder1/a\n> + EOF\n> + git -C clean-merge update-index --index-info +\n> + git -C clean-merge sparse-checkout set deep/deeper1 &&\n> +\n> + test_must_fail git -C clean-merge sparse-checkout clean -f 2>err &&\n> + grep \"failed to convert index to a sparse index\" err &&\n\nOh, interesting...with merge conflicts you at least get an error that\nit can't convert, whereas when there are tracked files (whether with\nstaged changes or unstaged changes or no changes), you don't? That\nseems to at least be good for the merge conflicts case, but it seems\nlike there's something to fix with the non-conflicted tracked files.\nBut that's kind of tangential to this patch.\n\n> + mkdir -p clean-merge/folder1/ &&\n> + echo merged >clean-merge/folder1/a &&\n> + git -C clean-merge add --sparse folder1/a &&\n> +\n> + # deletes folder2/ but leaves staged change in folder1\n> + # and dirty change in deep/deeper2/\n> + cat >expect <<-\\EOF &&\n> + Removing folder2/\n> + EOF\n> +\n> + git -C clean-merge sparse-checkout clean -f >out &&\n> + test_cmp expect out\n> '\n>\n> test_done\n> --\n> gitgitgadget\n\nPatch appears to correctly implement what was stated in the commit message.\n"},{"id":"523613","messageId":"CABPp-BFaPG1YpGOcTm=aX6n1XZ3upQ4iYB=nNND4bvsNZ=CqqQ@mail.gmail.com","threadId":"63760","inReplyTo":"f464bb5ed6be91940c3abb54b77cb7b9d893bd67.1752716054.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 5/8] sparse-checkout: add --verbose option to 'clean'","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-05T22:22:32Z","receivedAt":"2025-08-05T22:22:44Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The 'git sparse-checkout clean' subcommand is focused on directories,\n> deleting any tracked sparse directories to clean up the worktree and\n> make the sparse index feature work optimally.\n>\n> However, this directory-focused approach can leave users wondering why\n> those directories exist at all. In my experience, these files are left\n> over due to ignore or exclude patterns, Windows file handles, or\n> possibly merge conflict resolutions.\n\nSeems reasonable. And based on your previous testcases, it might not\neven be merge conflict resolutions, but just someone placing a\n(possibly-modified) copy of a tracked file back into the directory.\n\n(I've seen folks do that, so it's not \"just\" your testcase doing\nsomething unusual.)\n\n> Add a new '--verbose' option for users to see all the files that are\n> being deleted (with '--force') or would be deleted (with '--dry-run').\n\nDoes that answer the users' question? You said above in your\nexperience it came from a few different reasons; will users want to\nknow which reason(s) for which files, or will they only want to know\nthe files that are present?\n\n\n> Signed-off-by: Derrick Stolee \n> ---\n> Documentation/git-sparse-checkout.adoc | 5 +++++\n> builtin/sparse-checkout.c | 28 ++++++++++++++++++++++++--\n> t/t1091-sparse-checkout-builtin.sh | 14 ++++++++++---\n> 3 files changed, 42 insertions(+), 5 deletions(-)\n>\n> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n> index 823a66c40bc5..604f53f77caf 100644\n> --- a/Documentation/git-sparse-checkout.adoc\n> +++ b/Documentation/git-sparse-checkout.adoc\n> @@ -128,6 +128,11 @@ The `--dry-run` option will list the directories that would be removed\n> without deleting them. Running in this mode can be helpful to predict the\n> behavior of the clean comand or to determine which kinds of files are left\n> in the sparse directories.\n> ++\n> +The `--verbose` option will list every file within the directories that\n> +are considered for removal. This option is helpful to determine if those\n> +files are actually important or perhaps to explain why the directory is\n> +still present despite the current sparse-checkout.\n>\n> 'disable'::\n> Disable the `core.sparseCheckout` config setting, and restore the\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index fe332ff5f941..f38a0809c098 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -930,6 +930,26 @@ static char const * const builtin_sparse_checkout_clean_usage[] = {\n> NULL\n> };\n>\n> +static int list_file_iterator(const char *path, const void *data)\n> +{\n> + const char *msg = data;\n> +\n> + printf(msg, path);\n> + return 0;\n> +}\n> +\n> +static void list_every_file_in_dir(const char *msg,\n> + const char *directory)\n> +{\n> + struct strbuf path = STRBUF_INIT;\n> +\n> + strbuf_addstr(&path, directory);\n> + fprintf(stderr, \"list every file in %s\\n\", directory);\n> +\n> + for_each_file_in_dir(&path, list_file_iterator, msg);\n> + strbuf_release(&path);\n> +}\n> +\n> static const char *msg_remove = N_(\"Removing %s\\n\");\n> static const char *msg_would_remove = N_(\"Would remove %s\\n\");\n>\n> @@ -940,12 +960,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n> struct strbuf full_path = STRBUF_INIT;\n> const char *msg = msg_remove;\n> size_t worktree_len;\n> - int force = 0, dry_run = 0;\n> + int force = 0, dry_run = 0, verbose = 0;\n> int require_force = 1;\n>\n> struct option builtin_sparse_checkout_clean_options[] = {\n> OPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n> OPT__FORCE(&force, N_(\"force\"), PARSE_OPT_NOCOMPLETE),\n> + OPT__VERBOSE(&verbose, N_(\"report each affected file, not just directories\")),\n> OPT_END(),\n> };\n>\n> @@ -987,7 +1008,10 @@ static int sparse_checkout_clean(int argc, const char **argv,\n> if (!is_directory(full_path.buf))\n> continue;\n>\n> - printf(msg, ce->name);\n> + if (verbose)\n> + list_every_file_in_dir(msg, ce->name);\n> + else\n> + printf(msg, ce->name);\n>\n> if (dry_run <= 0 &&\n> remove_dir_recursively(&full_path, 0))\n> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\n> index 69f5a6dcc689..9a89b902c3f5 100755\n> --- a/t/t1091-sparse-checkout-builtin.sh\n> +++ b/t/t1091-sparse-checkout-builtin.sh\n> @@ -1052,9 +1052,9 @@ test_expect_success 'check-rules null termination' '\n>\n> test_expect_success 'clean' '\n> git -C repo sparse-checkout set --cone deep/deeper1 &&\n> - mkdir repo/deep/deeper2 repo/folder1 &&\n> + mkdir -p repo/deep/deeper2 repo/folder1/extra/inside &&\n> touch repo/deep/deeper2/file &&\n> - touch repo/folder1/file &&\n> + touch repo/folder1/extra/inside/file &&\n>\n> test_must_fail git -C repo sparse-checkout clean 2>err &&\n> grep \"refusing to clean\" err &&\n> @@ -1071,7 +1071,15 @@ test_expect_success 'clean' '\n> git -C repo sparse-checkout clean --dry-run >out &&\n> test_cmp expect out &&\n> test_path_exists repo/deep/deeper2 &&\n> - test_path_exists repo/folder1 &&\n> + test_path_exists repo/folder1/extra/inside/file &&\n> +\n> + cat >expect <<-\\EOF &&\n> + Would remove deep/deeper2/file\n> + Would remove folder1/extra/inside/file\n> + EOF\n> +\n> + git -C repo sparse-checkout clean --dry-run --verbose >out &&\n> + test_cmp expect out &&\n>\n> cat >expect <<-\\EOF &&\n> Removing deep/deeper2/\n> --\n> gitgitgadget\n\nYou stated in the commit message that \"users wonder...why those\ndirectories exist at all.\" Presuming that listing files is sufficient\nto answer those users questions, this patch looks good to me. I'm\nunsure if that answers the question, or if some kind of classification\nof the files would also be wanted (ignored, untracked, conflicted,\ntracked-with-unstaged-changes, tracked-wtih-no-changes,\ntracked-with-staged-changes). Maybe the answer is we start with this\nand wait for user feedback and only add more if there's demand, but if\nso it might be nice to state as much in the commit message.\n"},{"id":"523614","messageId":"CABPp-BEEr=pcF4ySreK4MeVUZ-L+HeCCKbZ=a0R+AkAVCqgFJw@mail.gmail.com","threadId":"63760","inReplyTo":"fd9a20a392265a5aaed27301a3b324164bb0af86.1752716054.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 4/8] dir: add generic \"walk all files\" helper","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-05T22:22:39Z","receivedAt":"2025-08-05T22:22:51Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> There is sometimes a need to visit every file within a directory,\n> recursively. The main example is remove_dir_recursively(), though it has\n> some extra flags that make it want to iterate over paths in a custom\n> way. There is also the fill_directory() approach but that involves an\n> index and a pathspec.\n>\n> This change adds a new for_each_file_in_dir() method that will be\n> helpful in the next change.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> dir.c | 28 ++++++++++++++++++++++++++++\n> dir.h | 14 ++++++++++++++\n> 2 files changed, 42 insertions(+)\n>\n> diff --git a/dir.c b/dir.c\n> index d2b0a5aef670..2e567ff92746 100644\n> --- a/dir.c\n> +++ b/dir.c\n> @@ -30,6 +30,7 @@\n> #include \"read-cache-ll.h\"\n> #include \"setup.h\"\n> #include \"sparse-index.h\"\n> +#include \"strbuf.h\"\n> #include \"submodule-config.h\"\n> #include \"symlinks.h\"\n> #include \"trace2.h\"\n> @@ -87,6 +88,33 @@ struct dirent *readdir_skip_dot_and_dotdot(DIR *dirp)\n> return e;\n> }\n>\n> +int for_each_file_in_dir(struct strbuf *path, file_iterator fn, const void *data)\n> +{\n> + struct dirent *e;\n> + int res = 0;\n> + size_t baselen = path->len;\n> + DIR *dir = opendir(path->buf);\n> +\n> + if (!dir)\n> + return 0;\n> +\n> + while (!res && (e = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n> + unsigned char dtype = get_dtype(e, path, 0);\n> + strbuf_setlen(path, baselen);\n> + strbuf_addstr(path, e->d_name);\n> +\n> + if (dtype == DT_REG) {\n> + res = fn(path->buf, data);\n> + } else if (dtype == DT_DIR) {\n> + strbuf_addch(path, '/');\n> + res = for_each_file_in_dir(path, fn, data);\n> + }\n> + }\n> +\n> + closedir(dir);\n> + return res;\n> +}\n> +\n> int count_slashes(const char *s)\n> {\n> int cnt = 0;\n> diff --git a/dir.h b/dir.h\n> index d7e71aa8daa7..f4235cc12a2f 100644\n> --- a/dir.h\n> +++ b/dir.h\n> @@ -536,6 +536,20 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);\n> */\n> int remove_dir_recursively(struct strbuf *path, int flag);\n>\n> +/*\n> + * This function pointer type is called on each file discovered in\n> + * for_each_file_in_dir. The iteration stops if this method returns\n> + * non-zero.\n> + */\n> +typedef int (*file_iterator)(const char *path, const void *data);\n> +\n> +struct strbuf;\n> +/*\n> + * Given a directory path, recursively visit each file within, including\n> + * within subdirectories.\n> + */\n> +int for_each_file_in_dir(struct strbuf *path, file_iterator fn, const void *data);\n> +\n> /*\n> * Tries to remove the path, along with leading empty directories so long as\n> * those empty directories are not startup_info->original_cwd. Ignores\n> --\n> gitgitgadget\n\nLooks reasonable.\n"},{"id":"523615","messageId":"CABPp-BHLcy-A4yLR8gP1Sjt_EKQ4K08kPyb7G6yifdZj+0MJNg@mail.gmail.com","threadId":"63760","inReplyTo":"82c24ce51980d85e1a53e746b462397e6e6c908a.1752716054.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 8/8] sparse-checkout: make 'clean' clear more files","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-06T00:21:50Z","receivedAt":"2025-08-06T00:22:04Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> The 'git sparse-checkout clean' command is designed to be a one-command\n> way to get the worktree in a state such that a sparse index would\n> operate efficiently. The previous change demonstrated that files outside\n> the sparse-checkout that were committed due to a merge conflict would\n> persist despite attempts to run 'git sparse-checkout clean' and instead\n> a 'git sparse-checkout reapply' would be required.\n>\n> Instead of requiring users to run both commands, update 'clean' to be\n> more ruthless about tracked sparse directories. The key here is to make\n> sure that the SKIP_WORKTREE bit is removed from more paths in the index\n> using update_sparsity() before compressing the index to a sparse one\n> in-memory.\n>\n> The tricky part here is that update_sparsity() was previously assuming\n> that it would be in 'update' mode and would change the worktree as it\n> made changes. However, we do not want to make these worktree changes at\n> this point, instead relying on our later logic (that integrates with\n> --dry-run and --verbose options) to perform those steps.\n>\n> One side-effect here is that we also clear out staged files that exist\n> in the worktree, but they would also appear in the verbose output as\n> part of the dry run.\n>\n> The final test in t1091 demonstrates that we no longer need the\n> 'reapply' subcommand for merge resolutions. It also fixes an earlier\n> case where 'git add --sparse' clears the SKIP_WORKTREE bit and avoids a\n> directory deletion.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> builtin/sparse-checkout.c | 8 ++++++++\n> t/t1091-sparse-checkout-builtin.sh | 24 +++++++++++++++++-------\n> unpack-trees.c | 2 +-\n> 3 files changed, 26 insertions(+), 8 deletions(-)\n>\n> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\n> index f38a0809c098..1d1d5208a3ba 100644\n> --- a/builtin/sparse-checkout.c\n> +++ b/builtin/sparse-checkout.c\n> @@ -962,6 +962,7 @@ static int sparse_checkout_clean(int argc, const char **argv,\n> size_t worktree_len;\n> int force = 0, dry_run = 0, verbose = 0;\n> int require_force = 1;\n> + struct unpack_trees_options o = { 0 };\n>\n> struct option builtin_sparse_checkout_clean_options[] = {\n> OPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n> @@ -990,6 +991,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n> if (repo_read_index(repo) < 0)\n> die(_(\"failed to read index\"));\n>\n> + o.verbose_update = verbose;\n> + o.update = 0; /* skip modifying the worktree here. */\n> + o.head_idx = -1;\n> + o.src_index = o.dst_index = repo->index;\n> + if (update_sparsity(&o, NULL))\n> + warning(_(\"failed to reapply sparse-checkout patterns\"));\n> +\n> if (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n> repo->index->sparse_index == INDEX_EXPANDED)\n> die(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\n> index 116ad7c9a20e..4b9078d90a61 100755\n> --- a/t/t1091-sparse-checkout-builtin.sh\n> +++ b/t/t1091-sparse-checkout-builtin.sh\n> @@ -1104,6 +1104,7 @@ test_expect_success 'clean with staged sparse change' '\n>\n> cat >expect <<-\\EOF &&\n> Would remove deep/deeper2/\n> + Would remove folder1/\n> EOF\n>\n> git -C repo sparse-checkout clean --dry-run >out &&\n> @@ -1115,6 +1116,7 @@ test_expect_success 'clean with staged sparse change' '\n> # deletes deep/deeper2/ but leaves folder1/ and folder2/\n> cat >expect <<-\\EOF &&\n> Removing deep/deeper2/\n> + Removing folder1/\n> EOF\n>\n> # The previous test case checked the -f option, so\n> @@ -1124,7 +1126,7 @@ test_expect_success 'clean with staged sparse change' '\n> test_cmp expect out &&\n>\n> test_path_is_missing repo/deep/deeper2 &&\n> - test_path_exists repo/folder1 &&\n> + test_path_is_missing repo/folder1 &&\n> test_path_exists repo/folder2\n\nWhat this doesn't show is that afterwards:\n\n$ git -C repo status\n\nOn branch main\nYou are in a sparse checkout with 78% of tracked files present.\n\nChanges to be committed:\n (use \"git restore --staged ...\" to unstage)\n new file: folder1/file\n\nChanges not staged for commit:\n (use \"git add/rm ...\" to update what will be committed)\n (use \"git restore ...\" to discard changes in working directory)\n deleted: folder1/file\n modified: folder2/a\n\n\n==> In other words, folder1/file and folder1/ were successfully\nremoved by this clean command, but when the user runs status, they see\nthat folder1/file has been deleted (and has a staged change). This is\nprobably correct behavior and may be expected, but might be surprising\nto users at first; it probably deserves to be documented, or at least\ncovered in the commit message. Especially since it's somewhat\ndifficult for users to work with:\n\n$ cd repo\n$ git checkout HEAD -- folder1/file\nerror: pathspec 'folder1/file' did not match any file(s) known to git\n\n$ git add -- folder1/file\nThe following paths and/or pathspecs matched paths that exist\noutside of your sparse-checkout definition, so will not be\nupdated in the index:\nfolder1/file\nhint: If you intend to update such entries, try one of the following:\nhint: * Use the --sparse option.\nhint: * Disable or modify the sparsity rules.\nhint: Disable this message with \"git config set advice.updateSparsePath false\"\n\n$ git add --sparse -- folder1/file\nfatal: pathspec 'folder1/file' did not match any files\n\n$ git reset -- folder1/file\nUnstaged changes after reset:\nM folder2/a\n\n$\n\n==> So, both checkout and add fail to work with this path while reset\nworks. `git commit` also would have worked; let's back up to the\npoint of status above, then get rid of the folder2/a modification, and\nthen use git commit to commit our folder1/file change:\n\n$ git checkout folder2/a\nUpdated 1 path from the index\n$ git commit -m \"A change\"\n[main aad2d89] A change\n 1 file changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 folder1/file\n$ git status\nOn branch main\nYou are in a sparse checkout with 78% of tracked files present.\n\nChanges not staged for commit:\n (use \"git add/rm ...\" to update what will be committed)\n (use \"git restore ...\" to discard changes in working directory)\n deleted: folder1/file\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n==> Great, so we got rid of the folder2/a modification, and committed\nthe staged folder1/file modification. The folder1/file showing as\ndeleted is annoying but a `sparse-checkout clean` should get rid of it\nas well as the pesky folder2/ directory, right?\n\n$ git sparse-checkout clean -f\nRemoving folder2/\n\n$ git status\nOn branch main\nYou are in a sparse checkout with 78% of tracked files present.\n\nChanges not staged for commit:\n (use \"git add/rm ...\" to update what will be committed)\n (use \"git restore ...\" to discard changes in working directory)\n deleted: folder1/file\n deleted: folder2/a\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n==> Huh, `git sparse-checkout clean -f` got rid of folder2, but now\nfolder2/a shows up as deleted locally...and folder1/file is still\nshowing as deleted locally. And it doesn't matter how many times we\nrun `git sparse-checkout clean -f`; this is the end state for it. But\nif we run `git sparse-checkout reapply`:\n\n$ git sparse-checkout reapply\n$ git status\nOn branch main\nYou are in a sparse checkout with 56% of tracked files present.\n\nnothing to commit, working tree clean\n\n==> So, you still need to use `git sparse-checkout reapply` together\nwith `git sparse-checkout clean -f`; you haven't fixed that yet.\n\n\n> @@ -1147,7 +1149,11 @@ test_expect_success 'sparse-checkout operations with merge conflicts' '\n> git commit -a -m \"left\" &&\n>\n> git checkout -b merge &&\n> - git sparse-checkout set deep/deeper1 &&\n> +\n> + touch deep/deeper2/extra &&\n> + git sparse-checkout set deep/deeper1 2>err &&\n> + grep \"contains untracked files\" err &&\n> + test_path_exists deep/deeper2/extra &&\n>\n> test_must_fail git merge -m \"will-conflict\" right &&\n>\n> @@ -1159,15 +1165,19 @@ test_expect_success 'sparse-checkout operations with merge conflicts' '\n> git merge --continue &&\n>\n> test_path_exists folder1/even/more/dirs/file &&\n> + test_path_exists deep/deeper2/extra &&\n> +\n> + cat >expect <<-\\EOF &&\n> + Removing deep/deeper2/\n> + Removing folder1/\n> + EOF\n>\n> # clean does not remove the file, because the\n> # SKIP_WORKTREE bit was not cleared by the merge command.\n\nShouldn't the comment be updated, given the testcase updates?\n\n> git sparse-checkout clean -f >out &&\n> - test_line_count = 0 out &&\n> - test_path_exists folder1/even/more/dirs/file &&\n> -\n> - git sparse-checkout reapply &&\n> - test_path_is_missing folder1\n> + test_cmp expect out &&\n> + test_path_is_missing folder1 &&\n> + test_path_is_missing deep/deeper2\n\nYes, but why does `git status` show folder1/even/more/dirs/file as\nbeing locally deleted? Does the code forget to update the\nSKIP_WORKTREE status after clearing out the files?\n"},{"id":"525167","messageId":"xmqqzfbjqbfo.fsf@gitster.g","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/8] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-28T23:22:03Z","receivedAt":"2025-08-28T23:22:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> This command uses the same '--force' and '--dry-run' options as 'git clean',\n> with integrations with the 'clean.requireForce' config option. There are\n> some concerns that this isn't an obvious way to work with the 'git clean'\n> command, but I thought we should be consistent here. I did change the error\n> message to point users to the necessary options.\n>\n> This option would be preferred to something like 'git clean -dfx' since it\n> does not clear the excluded files that are still within the sparse-checkout.\n> Instead, it performs the exact filesystem operations required to refresh the\n> sparse index performance back to what is expected.\n>\n> I spent a few weeks debating with myself about whether or not this was the\n> right interface, so please suggest alternatives if you have better ideas.\n> Among my rejected ideas include:\n>\n> * 'git sparse-checkout reapply -f -x' or similar augmentations of\n> 'reapply'.\n> * 'git clean --sparse' to focus the clean operation on things outside of\n> the sparse-checkout.\n>\n>\n> Updates in V2\n> =============\n>\n> * This series is based on 2c5b5565981 (environment: remove the global\n> variable 'sparse_expect_files_outside_of_patterns', 2025-07-01) to build\n> upon those cleanups in builtin/sparse-checkout.c.\n> * The --force and --dry-run options match 'git clean'.\n> * A --verbose option is added. It does not link to the index for\n> tracked/untracked/ignored/excluded or clean/modified/staged/conflicted\n> status, but instead gives the full list for information.\n> * To support the --verbose option, a new for_each_file_in_dir() method is\n> added to dir.h.\n> * Tests are added to demonstrate the behavior when a sparse directory has a\n> merge conflict (fails with an explanation). When adding the test based on\n> the previous version's functionality, I realized that the behavior is\n> sometimes less effective than git sparse-checkout reapply even after a\n> sparse file is committed. To demonstrate this change, the full test is\n> created on its own and then a code change is added with the impact on the\n> test.\n\nThis seems to have a few comments that haven't been responded to\n(plus a \"This step looks good to me\" or two). Can we get it unstuck\nsoonish? The topic is from mid July and I do not like to hold topics\nin 'seen' for longer than a month without any activity.\n\nThanks.\n"},{"id":"525173","messageId":"CABPp-BH=tk3eenHJkbRcD8uLGuakNMT5GkjVt6WfmOO8P+xq7A@mail.gmail.com","threadId":"63760","inReplyTo":"xmqqzfbjqbfo.fsf@gitster.g","subject":"Re: [PATCH v2 0/8] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-29T00:15:14Z","receivedAt":"2025-08-29T00:15:26Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Aug 28, 2025 at 4:22 PM Junio C Hamano wrote:\n>\n> This seems to have a few comments that haven't been responded to\n> (plus a \"This step looks good to me\" or two). Can we get it unstuck\n> soonish? The topic is from mid July and I do not like to hold topics\n> in 'seen' for longer than a month without any activity.\n\nStolee built this series on top of Ayush's topic to avoid conflicts\nfor you, and he said\n(https://lore.kernel.org/git/c3c0fbef-f395-4972-8352-dd89af6799d5@gmail.com/)\nthat since you marked this as blocking on Ayush's topic, he didn't\nwant to update until that topic moved.\n\nDo you want to instead kick Ayush's topic out and have Stolee rebase\nto no longer be on top of Ayush's, and have Ayush rebase anything he\nmight do on top of Stolee's work? (See also Ayush's recent update at\nhttps://lore.kernel.org/git/CAE7as+ZpEwiNsDAozoZXqHRLOF3+hT++uo=mzZqEvTPovQN9uw@mail.gmail.com/)\n"},{"id":"525174","messageId":"xmqq5xe7q8f5.fsf@gitster.g","threadId":"63760","inReplyTo":"CABPp-BH=tk3eenHJkbRcD8uLGuakNMT5GkjVt6WfmOO8P+xq7A@mail.gmail.com","subject":"Re: [PATCH v2 0/8] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-29T00:27:10Z","receivedAt":"2025-08-29T00:27:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> On Thu, Aug 28, 2025 at 4:22 PM Junio C Hamano wrote:\n>>\n>> This seems to have a few comments that haven't been responded to\n>> (plus a \"This step looks good to me\" or two). Can we get it unstuck\n>> soonish? The topic is from mid July and I do not like to hold topics\n>> in 'seen' for longer than a month without any activity.\n>\n> Stolee built this series on top of Ayush's topic to avoid conflicts\n> for you, and he said\n> (https://lore.kernel.org/git/c3c0fbef-f395-4972-8352-dd89af6799d5@gmail.com/)\n> that since you marked this as blocking on Ayush's topic, he didn't\n> want to update until that topic moved.\n>\n> Do you want to instead kick Ayush's topic out and have Stolee rebase\n> to no longer be on top of Ayush's, and have Ayush rebase anything he\n> might do on top of Stolee's work? (See also Ayush's recent update at\n> https://lore.kernel.org/git/CAE7as+ZpEwiNsDAozoZXqHRLOF3+hT++uo=mzZqEvTPovQN9uw@mail.gmail.com/)\n\nIt really depends on how unstable the base topic would be, but I\nknow Stolee is better than building his stuff on unusably unstable\ncrap, and that was the reason why I thought that updating this topic\non top of the same base would allow us to move forward faster, as it\nwould mean that everything would hopefully be ready _UNLESS_ the\nchange that needs to be made to the base topic is so extensive that\nthe topic on top would also need heavy updates _again_ once an\nupdate to the base topic comes.\n\n"},{"id":"525244","messageId":"xmqqcy8dn8mk.fsf@gitster.g","threadId":"63760","inReplyTo":"xmqq5xe7q8f5.fsf@gitster.g","subject":"Re: [PATCH v2 0/8] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-29T21:03:15Z","receivedAt":"2025-08-29T21:03:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Elijah Newren writes:\n>\n>> On Thu, Aug 28, 2025 at 4:22 PM Junio C Hamano wrote:\n>>>\n>>> This seems to have a few comments that haven't been responded to\n>>> (plus a \"This step looks good to me\" or two). Can we get it unstuck\n>>> soonish? The topic is from mid July and I do not like to hold topics\n>>> in 'seen' for longer than a month without any activity.\n>>\n>> Stolee built this series on top of Ayush's topic to avoid conflicts\n>> for you, and he said\n>> (https://lore.kernel.org/git/c3c0fbef-f395-4972-8352-dd89af6799d5@gmail.com/)\n>> that since you marked this as blocking on Ayush's topic, he didn't\n>> want to update until that topic moved.\n>>\n>> Do you want to instead kick Ayush's topic out and have Stolee rebase\n>> to no longer be on top of Ayush's, and have Ayush rebase anything he\n>> might do on top of Stolee's work? (See also Ayush's recent update at\n>> https://lore.kernel.org/git/CAE7as+ZpEwiNsDAozoZXqHRLOF3+hT++uo=mzZqEvTPovQN9uw@mail.gmail.com/)\n>\n> It really depends on how unstable the base topic would be, but I\n> know Stolee is better than building his stuff on unusably unstable\n> crap, and that was the reason why I thought that updating this topic\n> on top of the same base would allow us to move forward faster, as it\n> would mean that everything would hopefully be ready _UNLESS_ the\n> change that needs to be made to the base topic is so extensive that\n> the topic on top would also need heavy updates _again_ once an\n> update to the base topic comes.\n\n(Sorry, but sent before finishing).\n\nYes, it may be simpler to kick out a stalled topic and give it a\nfresh restart when it becomes ready. If Derrick wants to go that\nroute, I am totally fine with that.\n\nThanks.\n\n"},{"id":"525252","messageId":"7bc3e3ba-872d-4c03-a032-7b6a6251daf8@gmail.com","threadId":"63760","inReplyTo":"xmqqcy8dn8mk.fsf@gitster.g","subject":"Re: [PATCH v2 0/8] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-08-30T13:41:55Z","receivedAt":"2025-08-30T13:41:57Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/29/25 5:03 PM, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n>> Elijah Newren writes:\n>>\n>>> On Thu, Aug 28, 2025 at 4:22 PM Junio C Hamano wrote:\n>>>>\n>>>> This seems to have a few comments that haven't been responded to\n>>>> (plus a \"This step looks good to me\" or two). Can we get it unstuck\n>>>> soonish? The topic is from mid July and I do not like to hold topics\n>>>> in 'seen' for longer than a month without any activity.\n>>>\n>>> Stolee built this series on top of Ayush's topic to avoid conflicts\n>>> for you, and he said\n>>> (https://lore.kernel.org/git/c3c0fbef-f395-4972-8352-dd89af6799d5@gmail.com/)\n>>> that since you marked this as blocking on Ayush's topic, he didn't\n>>> want to update until that topic moved.\n>>>\n>>> Do you want to instead kick Ayush's topic out and have Stolee rebase\n>>> to no longer be on top of Ayush's, and have Ayush rebase anything he\n>>> might do on top of Stolee's work? (See also Ayush's recent update at\n>>> https://lore.kernel.org/git/CAE7as+ZpEwiNsDAozoZXqHRLOF3+hT++uo=mzZqEvTPovQN9uw@mail.gmail.com/)\n>>\n>> It really depends on how unstable the base topic would be, but I\n>> know Stolee is better than building his stuff on unusably unstable\n>> crap, and that was the reason why I thought that updating this topic\n>> on top of the same base would allow us to move forward faster, as it\n>> would mean that everything would hopefully be ready _UNLESS_ the\n>> change that needs to be made to the base topic is so extensive that\n>> the topic on top would also need heavy updates _again_ once an\n>> update to the base topic comes.\n> \n> (Sorry, but sent before finishing).\n> \n> Yes, it may be simpler to kick out a stalled topic and give it a\n> fresh restart when it becomes ready. If Derrick wants to go that\n> route, I am totally fine with that.\n\nI'll see how things go this coming week as I page this series\nback into my active work. If Ayush resubmits before I get to it,\nthen I won't be upset.\n\nThanks,\n-Stolee\n\n\n"},{"id":"526127","messageId":"4ce92ef9-61ef-491e-80a3-370e92fd10fd@gmail.com","threadId":"63760","inReplyTo":"CABPp-BFzMLGJwz4QqYtvw3zRYgmC=Mb8T8GCOsrLZqT2z+8H7A@mail.gmail.com","subject":"Re: [PATCH v2 2/8] sparse-checkout: add basics of 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-11T13:37:46Z","receivedAt":"2025-09-11T13:37:49Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/5/25 5:32 PM, Elijah Newren wrote:\n> On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n> wrote:\n> \n> Sorry for the long delay in responding...\n> \n> [...]\n>> Add a new subcommand to 'git sparse-checkout' that removes these\n>> tracked-but-sparse directories. This necessarily removes all files\n>> contained within, including tracked and untracked files. Of particular\n> \n> Nice to see tracked files also being addressed in v2.\n> \n>> importance are ignored and excluded files which would normally be\n>> ignored even by 'git clean -f' unless the '-x' or '-X' option is\n>> provided. This is the most extreme method for doing this, but it works\n>> when the sparse-checkout is in cone mode and is expected to rescope\n>> based on directories, not files.\n>>\n>> The current implementation always deletes these sparse directories\n>> without warning. This is unacceptable for a released version, but those\n>> features will be added in changes coming immediately after this one.\n>>\n>> Note that untracked directories within the sparse-checkout remain.\n> \n> You've changed the wording here relative to v1, but you haven't\n> addressed the part that was ambiguous/misleading in v1. In fact, you\n> may have made a different part ambiguous as well, and made readers\n> think that this sentence contradicts your above claims that this\n> command is meant to clean out untracked directories underneath sparse\n> directories. Perhaps something like:\n> \n> \"Note that untracked directories in the sparse-checkout that are not\n> within sparse directories will not be removed by this command; it only\n> cleans up paths under directories that are supposed to be sparse.\"\n\nBoth here and in the documentation, things can get a bit confusing.\nIn the v3 I'm preparing, I'm taking the following approach:\n\n * In the commit message, focus on the implementation details and how\n that impacts the behavior of the tool.\n\n * In the documentation, focus on the list of files that will be\n \"considered for removal\". Use the most broad definition there:\n in a tracked directory that is outside of the sparse-checkout.\n Add pointers that could explain exceptions and how to remove\n these exceptions, but don't attempt to explain all special\n cases.\n\n>> +test_expect_success 'clean with staged sparse change' '\n>> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n>> + mkdir repo/deep/deeper2 repo/folder1 repo/folder2 &&\n>> + touch repo/deep/deeper2/file &&\n>> + touch repo/folder1/file &&\n>> + echo dirty >repo/folder2/a &&\n>> +\n>> + git -C repo add --sparse folder1/file &&\n>> +\n>> + # deletes deep/deeper2/ but leaves folder1/ and folder2/\n>> + cat >expect <<-\\EOF &&\n>> + Removing deep/deeper2/\n>> + EOF\n>> +\n>> + git -C repo sparse-checkout clean >out &&\n>> + test_cmp expect out &&\n>> +\n>> + test_path_is_missing repo/deep/deeper2 &&\n>> + test_path_exists repo/folder1\n> \n> What about repo/folder2/ ?\n> \n> Anyway, this test shows that neither staged nor unstaged changes are\n> cleaned up (which at least resolves the conflicting documentation you\n> provided on the matter) -- or would if you also checked repo/folder2.\n> \n> What it doesn't show is that tracked files with neither staged nor\n> unstaged changes are not cleaned up either:\n> \n> $ mkdir repo/folder2\n> $ echo dirty >repo/folder2/a\n> $ touch repo/folder2/untracked\n> $ cd repo\n> $ git status --porcelain\n> M folder2/a\n> ?? folder2/untracked\n> \n> # So, we have both a unstaged change and an untracked file; let's undo\n> the unstaged change\n> \n> $ git checkout HEAD folder2/a\n> Updated 1 path from 8cc814f\n> $ git status --porcelain\n> ?? folder2/untracked\n> $ ls folder2/\n> a untracked\n> \n> # Both files are still present -- the untracked file, and the\n> untracked file with no changes either staged or unstaged -- what does\n> `git sparse-checkout clean` do?\n\nIt seems that the unstaged modification to a tracked, sparse file\nis enough to prevent the sparse directory collapse. This is\nsimilar to how 'git sparse-checkout reapply' will refuse to remove\nthose modified changes. I'll be sure to update my advice around\nspecial cases to include this (and lock it in with a test case).\n\n> $ git sparse-checkout clean\n> $ ls folder2/\n> a untracked\n> $ git status --porcelain\n> ?? folder2/untracked\n> \n> # Absolutely nothing. Not only does it not clean anything up, it\n> gives no warnings about not cleaning up what should be cleaned up.\n\nAt this point, the SKIP_WORKTREE bit is still removed because\nwe've staged the change.\n\n> Let's try sparse-checkout reapply:\n> \n> $ git sparse-checkout reapply\n> warning: directory 'folder2/' contains untracked files, but is not in\n> the sparse-checkout cone\n> $ git status --porcelain\n> ?? folder2/untracked\n> $ ls folder2/\n> untracked\n>\n> # So `sparse-checkout reapply` does correctly remove folder2/a for us,\n> while warning about the untracked file. (If folder2/a would have\n> still had changes, it would have warned about it instead of\n> removing.). Let's try `sparse-checkout clean` now...\n> \n> $ git sparse-checkout clean\n> Removing folder2/\n> $ git status --porcelain\n> $ ls folder2/\n> ls: cannot access 'folder2/': No such file or directory\n> $\n> \n> I think these cases either need to be a new testcase or part of this\n> last testcase, and the commit message and documentation should be\n> clearer about tracked-and-staged, tracked-with-unstaged-changes, and\n> tracked-with-no-changes files...or at least comment that they'll be\n> discussed later in the patch series. (I have a feeling I just did a\n> lot of work to discover as I read your next patches that you cover\n> these later...)\n\nNo! you found interesting ways to test special cases. Thanks!\n\nDescribing the lifecycle of a sparse file (with sibling untracked\nchange) going from modified to staged to sparse to unlock the\ncleaning would be helpful documentation.\n\nI do think there is an interesting extra functionality that we\nshould consider for the future: \"What files are in my worktree\nthat _should_ be sparse? Why is 'clean' not removing them?\"\n\nThanks,\n-Stolee\n\n"},{"id":"526128","messageId":"4ed3dd55-6eba-4cce-ba00-626f4d9cddab@gmail.com","threadId":"63760","inReplyTo":"CABPp-BGQmoPQ9TOLjZPpPPEJ__0rsHgRMJ-nkiW4GcQjPCvaBA@mail.gmail.com","subject":"Re: [PATCH v2 3/8] sparse-checkout: match some 'clean' behavior","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-11T13:52:30Z","receivedAt":"2025-09-11T13:52:34Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/5/25 6:06 PM, Elijah Newren wrote:\n> On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n> wrote:\n>>\n>> From: Derrick Stolee \n>>\n>> The 'git sparse-checkout clean' subcommand is somewhat similar to 'git\n>> clean' in that it will delete files that should not be in the worktree.\n>> The big difference is that it focuses on the directories that should not\n>> be in the worktree due to cone-mode sparse-checkout. It also does not\n>> discriminate in the kinds of files and focuses on deleting entire\n>> directories.\n>>\n>> However, there are some restrictions that would be good to bring over\n>> from 'git clean', specifically how it refuses to do anything without the\n>> '-f'/'--force' or '-n'/'--dry-run' arguments. The 'clean.requireForce'\n>> config can be set to 'false' to imply '--force'.\n>>\n>> Add this behavior to avoid accidental deletion of files that cannot be\n>> recovered from Git.\n> \n> I'm a bit surprised by this. Given that the only kinds of files that\n> this command cleans out are untracked and ignored files, and Junio's\n> comments about clean.requireForce over in\n> https://lore.kernel.org/git/xmqqv7o2togi.fsf@gitster.g/, I thought his\n> comments could be interpreted as not wanting clean.requireForce to\n> apply in more places. Did I misunderstand?\n> \n> Alternatively, maybe you thought that there were files other than\n> untracked and ignored which `sparse-checkout clean` would clean up,\n> and it was because of those files that we wanted the extra protection?\n> (In that case, it'd make sense, but it seems to go against what was\n> demonstrated in the final testcase of the previous patch.)\n\nMy thought process here was that users expect 'git clean' to be extra\ncareful to prevent removing files. While Junio mentioned regret in\nthe decision to require the '-f', I didn't want to have such a major\ndifference between the two commands.\n\nI also interpreted Junio's comments to be that he wished that\nclean.requireForce was 'false' by default instead of the current\nassumption of 'true' if unset.\n\nI'm open to reviewers providing a firm stance towards \"the new\ncommand should deviate closer to how we wish 'git clean' worked\"\nand overriding my choice to match behavior.\n\nThanks,\n-Stolee\n\n\n"},{"id":"526129","messageId":"879b6673-1af9-4bb3-add3-a61c87d98777@gmail.com","threadId":"63760","inReplyTo":"CABPp-BFaPG1YpGOcTm=aX6n1XZ3upQ4iYB=nNND4bvsNZ=CqqQ@mail.gmail.com","subject":"Re: [PATCH v2 5/8] sparse-checkout: add --verbose option to 'clean'","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-11T14:06:59Z","receivedAt":"2025-09-11T14:07:01Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/5/25 6:22 PM, Elijah Newren wrote:\n> On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n> wrote:\n>>\n>> From: Derrick Stolee \n>>\n>> The 'git sparse-checkout clean' subcommand is focused on directories,\n>> deleting any tracked sparse directories to clean up the worktree and\n>> make the sparse index feature work optimally.\n>>\n>> However, this directory-focused approach can leave users wondering why\n>> those directories exist at all. In my experience, these files are left\n>> over due to ignore or exclude patterns, Windows file handles, or\n>> possibly merge conflict resolutions.\n> \n> Seems reasonable. And based on your previous testcases, it might not\n> even be merge conflict resolutions, but just someone placing a\n> (possibly-modified) copy of a tracked file back into the directory.\n> \n> (I've seen folks do that, so it's not \"just\" your testcase doing\n> something unusual.)\n> \n>> Add a new '--verbose' option for users to see all the files that are\n>> being deleted (with '--force') or would be deleted (with '--dry-run').\n> \n> Does that answer the users' question? You said above in your\n> experience it came from a few different reasons; will users want to\n> know which reason(s) for which files, or will they only want to know\n> the files that are present?\n\nI've had users asking for answers to both of these questions:\n\n 1. What files will this remove? (I want to make sure it won't remove\n something I care about.)\n\n 2. What files were causing my sparse index to expand? (I want to\n understand how my workflow impacts this behavior.)\n\n> You stated in the commit message that \"users wonder...why those\n> directories exist at all.\" Presuming that listing files is sufficient\n> to answer those users questions, this patch looks good to me. I'm\n> unsure if that answers the question, or if some kind of classification\n> of the files would also be wanted (ignored, untracked, conflicted,\n> tracked-with-unstaged-changes, tracked-wtih-no-changes,\n> tracked-with-staged-changes). Maybe the answer is we start with this\n> and wait for user feedback and only add more if there's demand, but if\n> so it might be nice to state as much in the commit message.\n\nSounds like a plan. This verbose output is intended to be human-\nreadable and can easily be expanded in the future.\n\nThanks,\n-Stolee\n\n"},{"id":"526131","messageId":"9b8d6c20-e91d-4ef6-9260-5016870f27c7@gmail.com","threadId":"63760","inReplyTo":"CABPp-BHLcy-A4yLR8gP1Sjt_EKQ4K08kPyb7G6yifdZj+0MJNg@mail.gmail.com","subject":"Re: [PATCH v2 8/8] sparse-checkout: make 'clean' clear more files","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-11T15:26:56Z","receivedAt":"2025-09-11T15:26:59Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/5/25 8:21 PM, Elijah Newren wrote:\n> On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n\n>> test_path_is_missing repo/deep/deeper2 &&\n>> - test_path_exists repo/folder1 &&\n>> + test_path_is_missing repo/folder1 &&\n>> test_path_exists repo/folder2\n> \n> What this doesn't show is that afterwards:\n> \n> $ git -C repo status\n> \n> On branch main\n> You are in a sparse checkout with 78% of tracked files present.\n> \n> Changes to be committed:\n> (use \"git restore --staged ...\" to unstage)\n> new file: folder1/file\n> \n> Changes not staged for commit:\n> (use \"git add/rm ...\" to update what will be committed)\n> (use \"git restore ...\" to discard changes in working directory)\n> deleted: folder1/file\n> modified: folder2/a\n\nYou make an excellent point about SKIP_WORKTREE bit states across\nchanges like this. I'll expand on this flow in the test script.\n\n>> + test_path_exists deep/deeper2/extra &&\n>> +\n>> + cat >expect <<-\\EOF &&\n>> + Removing deep/deeper2/\n>> + Removing folder1/\n>> + EOF\n>>\n>> # clean does not remove the file, because the\n>> # SKIP_WORKTREE bit was not cleared by the merge command.\n> \n> Shouldn't the comment be updated, given the testcase updates?\n\nYes. Good catch.\n\n>> git sparse-checkout clean -f >out &&\n>> - test_line_count = 0 out &&\n>> - test_path_exists folder1/even/more/dirs/file &&\n>> -\n>> - git sparse-checkout reapply &&\n>> - test_path_is_missing folder1\n>> + test_cmp expect out &&\n>> + test_path_is_missing folder1 &&\n>> + test_path_is_missing deep/deeper2\n> \n> Yes, but why does `git status` show folder1/even/more/dirs/file as\n> being locally deleted? Does the code forget to update the\n> SKIP_WORKTREE status after clearing out the files?\n\nThis is an interesting quirk about how \"git add --sparse \"\nworks, which is to remove the SKIP_WORKTREE bit. It's further\ninteresting that the file is still being \"tracked as a deletion\"\nwhile also being \"collapsed to a sparse directory\".\n\nBy expanding the earlier test cases to include these post-clean\nstatuses, we can see that this change is causing this new view\nof a deleted file. I will see what I can do to determine exactly\nwhat's different this time and how we can minimize that\nstrangeness (or: consider dropping this patch).\n\nThanks,\n-Stolee\n\n"},{"id":"526135","messageId":"ad8a0156-bfd8-4a6e-b196-6e591e536b8c@gmail.com","threadId":"63760","inReplyTo":"9b8d6c20-e91d-4ef6-9260-5016870f27c7@gmail.com","subject":"Re: [PATCH v2 8/8] sparse-checkout: make 'clean' clear more files","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-11T16:21:49Z","receivedAt":"2025-09-11T16:21:52Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/11/25 11:26 AM, Derrick Stolee wrote:\n> On 8/5/25 8:21 PM, Elijah Newren wrote:\n>> On Wed, Jul 16, 2025 at 6:34 PM Derrick Stolee via GitGitGadget\n\n>>>                  git sparse-checkout clean -f >out &&\n>>> -               test_line_count = 0 out &&\n>>> -               test_path_exists folder1/even/more/dirs/file &&\n>>> -\n>>> -               git sparse-checkout reapply &&\n>>> -               test_path_is_missing folder1\n>>> +               test_cmp expect out &&\n>>> +               test_path_is_missing folder1 &&\n>>> +               test_path_is_missing deep/deeper2\n>>\n>> Yes, but why does `git status` show folder1/even/more/dirs/file as\n>> being locally deleted?  Does the code forget to update the\n>> SKIP_WORKTREE status after clearing out the files?\n> \n> This is an interesting quirk about how \"git add --sparse \"\n> works, which is to remove the SKIP_WORKTREE bit. It's further\n> interesting that the file is still being \"tracked as a deletion\"\n> while also being \"collapsed to a sparse directory\".\n> \n> By expanding the earlier test cases to include these post-clean\n> statuses, we can see that this change is causing this new view\n> of a deleted file. I will see what I can do to determine exactly\n> what's different this time and how we can minimize that\n> strangeness (or: consider dropping this patch).\n\nI'm coming to the conclusion that I should drop this patch\nfor now and consider this more aggressive cleaning mechanism\nfor a later update.\n\nThe strangest part of what's going on here is that the in-\nmemory sparse index collapses the folder2/ entry but an on-\ndisk sparse index does not in this state. That's what's\nleading to this deleted entry.\n\nI think the unpack_trees_options values are involved, but a\nfew experiments led to no change in my tests.\n\n(With that, I have a v3 nearly ready, but I'll wait a day to\nsee if feedback on top of my comments causes me to rethink\nanything.)\n\nThanks,\n-Stolee\n\n"},{"id":"526168","messageId":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v2.git.1752716054.gitgitgadget@gmail.com","subject":"[PATCH v3 0/7] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:04Z","receivedAt":"2025-09-12T10:30:15Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"NEW: This series is rebased on a recent master to remove dependence on the\nupdates to the global variables used by the sparse-checkout system.\n\nWhen using cone-mode sparse-checkout, users specify which tracked\ndirectories they want (recursively) and any directory not part of the parent\npaths for those directories are considered \"out of scope\". When changing\nsparse-checkouts, there are a variety of reasons why these \"out of scope\"\ndirectories could remain, including:\n\n * The user has .gitignore or .git/info/exclude files that tell Git to not\n remove files of a certain type.\n * Some filesystem blocker prevented the removal of a tracked file. This is\n usually more of an issue on Windows where a read handle will block file\n deletion.\n\nTypically, this would not mean too much for the user experience. A few extra\nfilesystem checks might be required to satisfy git status commands, but the\nscope of the performance hit is relative to how many cruft files are left\nover in this situation.\n\nHowever, when using the sparse index, these tracked sparse directories cause\nsignificant performance issues. When noticing that the index contains a\nsparse directory but that directory exists on disk, Git needs to expand that\nsparse directory to determine which files are tracked or untracked. The\ncurrent mechanism expands the entire index to a full one, an expensive\noperation that scales with the total number of paths at HEAD and not just\nthe number of cruft files left over.\n\nAdvice was added in 9479a31d603 (advice: warn when sparse index expands,\n2024-07-08) to help users determine that they were in this state. However,\nthe advice doesn't actually recommend helpful ways to get out of this state.\nRecommending \"git clean\" on its own is incomplete, as typically users\nactually need 'git clean -dfx' to clear out the ignored or excluded files.\nEven then, they may need 'git sparse-checkout reapply' afterwards to clear\nthe sparse directories.\n\nThe advice was successful in helping to alert users to the problem, which is\nhow I got wind of many of these cases for how users get into this state.\nIt's now time to give them a tool that helps them out of this state.\n\nThis series adds a new 'git sparse-checkout clean' command that currently\nonly works for cone-mode sparse-checkouts. The only thing it does is\ncollapse the index to a sparse index (as much as possible) and make sure\nthat any sparse directories are removed. These directories are listed to\nstdout.\n\nThis command uses the same '--force' and '--dry-run' options as 'git clean',\nwith integrations with the 'clean.requireForce' config option. There are\nsome concerns that this isn't an obvious way to work with the 'git clean'\ncommand, but I thought we should be consistent here. I did change the error\nmessage to point users to the necessary options.\n\nThis option would be preferred to something like 'git clean -dfx' since it\ndoes not clear the excluded files that are still within the sparse-checkout.\nInstead, it performs the exact filesystem operations required to refresh the\nsparse index performance back to what is expected.\n\n\nUpdates in V3\n=============\n\nHuge thanks to Elijah for such a detailed review. Apologies for the delay in\nresponding.\n\n * Removed dependency on stalled series around updating the sparse-checkout\n globals.\n * Commit message and documentation is updated to better describe the\n conditions that qualify a file or directory for removal.\n * Tests are expanded significantly to include special cases and\n aftereffects.\n * A note is added around possible future expansion of the --verbose option\n to include more detailed status information on the files that would be\n deleted.\n * Due to a situation where a file appears as \"modified and deleted\" after\n the more aggressive updating of the tree, the previous patch 8 is removed\n (for now). I may reconsider and send a version in the future that avoids\n this issue. Tests from the earlier patches are more expanded in such a\n way that the aggressive implementation requires test changes that reveal\n this problem. See [1] for a copy of this change and how it impacts the\n latest tests.\n\n[1]\nhttps://github.com/derrickstolee/git/commit/f9eb8e03361d61023fec33386fc9898f60b65a31\n\n\nUpdates in V2\n=============\n\n * This series is based on 2c5b5565981 (environment: remove the global\n variable 'sparse_expect_files_outside_of_patterns', 2025-07-01) to build\n upon those cleanups in builtin/sparse-checkout.c.\n * The --force and --dry-run options match 'git clean'.\n * A --verbose option is added. It does not link to the index for\n tracked/untracked/ignored/excluded or clean/modified/staged/conflicted\n status, but instead gives the full list for information.\n * To support the --verbose option, a new for_each_file_in_dir() method is\n added to dir.h.\n * Tests are added to demonstrate the behavior when a sparse directory has a\n merge conflict (fails with an explanation). When adding the test based on\n the previous version's functionality, I realized that the behavior is\n sometimes less effective than git sparse-checkout reapply even after a\n sparse file is committed. To demonstrate this change, the full test is\n created on its own and then a code change is added with the impact on the\n test.\n\nThanks, -Stolee\n\nDerrick Stolee (7):\n sparse-checkout: remove use of the_repository\n sparse-checkout: add basics of 'clean' command\n sparse-checkout: match some 'clean' behavior\n dir: add generic \"walk all files\" helper\n sparse-checkout: add --verbose option to 'clean'\n sparse-index: point users to new 'clean' action\n t: expand tests around sparse merges and clean\n\n Documentation/git-sparse-checkout.adoc | 33 +++-\n builtin/sparse-checkout.c | 218 ++++++++++++++++++-------\n dir.c | 28 ++++\n dir.h | 14 ++\n sparse-index.c | 3 +-\n t/t1091-sparse-checkout-builtin.sh | 175 ++++++++++++++++++++\n 6 files changed, 413 insertions(+), 58 deletions(-)\n\n\nbase-commit: ab427cd991100e94792fce124b0934135abdea4b\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1941%2Fderrickstolee%2Fgit-sparse-checkout-clean-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1941/derrickstolee/git-sparse-checkout-clean-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/1941\n\nRange-diff vs v2:\n\n 1: 92d0cd41a4 ! 1: 437a57b665 sparse-checkout: remove use of the_repository\n @@ builtin/sparse-checkout.c: static enum sparse_checkout_mode update_cone_mode(int\n \tint mode, record_mode;\n \n @@ builtin/sparse-checkout.c: static int update_modes(int *cone_mode, int *sparse_index)\n - \trecord_mode = (*cone_mode != -1) || !the_repository->settings.sparse_checkout;\n + \trecord_mode = (*cone_mode != -1) || !core_apply_sparse_checkout;\n \n \tmode = update_cone_mode(cone_mode);\n -\tif (record_mode && set_config(mode))\n @@ builtin/sparse-checkout.c: static void add_patterns_literal(int argc, const char\n {\n \tint result;\n @@ builtin/sparse-checkout.c: static int modify_pattern_list(struct strvec *args, int use_stdin,\n - \t\tbreak;\n \t}\n \n --\tif (!the_repository->settings.sparse_checkout) {\n + \tif (!core_apply_sparse_checkout) {\n -\t\tset_config(MODE_ALL_PATTERNS);\n --\t\tthe_repository->settings.sparse_checkout = 1;\n -+\tif (!repo->settings.sparse_checkout) {\n +\t\tset_config(repo, MODE_ALL_PATTERNS);\n -+\t\trepo->settings.sparse_checkout = 1;\n + \t\tcore_apply_sparse_checkout = 1;\n \t\tchanged_config = 1;\n \t}\n \n @@ builtin/sparse-checkout.c: static struct sparse_checkout_add_opts {\n \tstatic struct option builtin_sparse_checkout_add_options[] = {\n \t\tOPT_BOOL_F(0, \"skip-checks\", &add_opts.skip_checks,\n @@ builtin/sparse-checkout.c: static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n - \tif (!the_repository->settings.sparse_checkout)\n + \tif (!core_apply_sparse_checkout)\n \t\tdie(_(\"no sparse-checkout to add to\"));\n \n -\trepo_read_index(the_repository);\n @@ builtin/sparse-checkout.c: static int sparse_checkout_disable(int argc, const ch\n \tmemset(&pl, 0, sizeof(pl));\n \thashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);\n @@ builtin/sparse-checkout.c: static int sparse_checkout_disable(int argc, const char **argv,\n - \n \tadd_pattern(\"/*\", empty_base, 0, &pl, 0);\n \n + \tprepare_repo_settings(the_repository);\n -\tthe_repository->settings.sparse_index = 0;\n +\trepo->settings.sparse_index = 0;\n \n @@ builtin/sparse-checkout.c: static int sparse_checkout_check_rules(int argc, cons\n \treturn ret;\n @@ builtin/sparse-checkout.c: int cmd_sparse_checkout(int argc,\n \n - \tgit_config(git_default_config, NULL);\n + \trepo_config(the_repository, git_default_config, NULL);\n \n -\tprepare_repo_settings(the_repository);\n -\tthe_repository->settings.command_requires_full_index = 0;\n 2: 7e8f7c2d6c ! 2: a1564f74cf sparse-checkout: add basics of 'clean' command\n @@ Commit message\n not be sufficient.\n \n Add a new subcommand to 'git sparse-checkout' that removes these\n - tracked-but-sparse directories. This necessarily removes all files\n - contained within, including tracked and untracked files. Of particular\n - importance are ignored and excluded files which would normally be\n - ignored even by 'git clean -f' unless the '-x' or '-X' option is\n - provided. This is the most extreme method for doing this, but it works\n - when the sparse-checkout is in cone mode and is expected to rescope\n - based on directories, not files.\n + tracked-but-sparse directories.\n +\n + The implementation details provide a clear definition of what is happening,\n + but it is difficult to describe this without including the internal\n + implementation details. The core operation converts the index to a sparse\n + index (in memory if not already on disk) and then deletes any directories in\n + the worktree that correspond with a sparse directory entry in that sparse\n + index.\n +\n + In the most common case, this means that a file will be removed if it is\n + contained within a directory that is both tracked and outside of the\n + sparse-checkout definition. However, there can be exceptions depending on\n + the current state of the index:\n +\n + * If the worktree has a modification to a tracked, sparse file, then that\n + file's parent directories will be expanded instead of represented as\n + sparse directories. Siblings of those parent directories may be\n + considered sparse.\n +\n + * If the user staged a sparse file with \"git add --sparse\", then that file\n + loses the SKIP_WORKTREE bit until the sparse-checkout is reapplied. Until\n + then, that file's parent directories are not represented as sparse\n + directory entries and thus will not be removed. Siblings of those parent\n + directories may be considered sparse. (There may be other reasons why\n + the SKIP_WORKTREE bit was removed for a file and this impact on the\n + sparse directories will apply to those as well.)\n +\n + * If the user has a merge conflict outside of the sparse-checkout\n + definition, then those conflict entries prevent the parent directories\n + from being represented as sparse directory entries and thus are not\n + removed.\n +\n + * The cases above present reasons why certain _file conditions_ will impact\n + which _directories_ are considered sparse. The list of tracked\n + directories that are outside of the sparse-checkout definition but not\n + represented as a sparse directory further reduces the list of files that\n + will be removed.\n +\n + For these complicated reasons, the documentation details a potential list of\n + files that will be \"considered for removal\" instead of defining the list\n + concretely. The special cases can be handled by resolving conflicts,\n + committing staged changes, and running 'git sparse-checkout reapply' to\n + update the SKIP_WORKTREE bits as expected by the sparse-checkout definition.\n +\n + It is important to make clear that this operation will remove ignored and\n + excluded files which would normally be ignored even by 'git clean -f' unless\n + the '-x' or '-X' option is provided. This is the most extreme method for\n + doing this, but it works when the sparse-checkout is in cone mode and is\n + expected to rescope based on directories, not files.\n \n The current implementation always deletes these sparse directories\n without warning. This is unacceptable for a released version, but those\n features will be added in changes coming immediately after this one.\n \n - Note that untracked directories within the sparse-checkout remain.\n - Further, directories that contain staged changes or files in merge\n - conflict states are not deleted. This is a detail that is partly hidden\n - by the implementation which relies on collapsing the index to a sparse\n - index in-memory and only deleting directories that are listed as sparse\n - in the index.\n + Note that this will not remove an untracked directory (or any of its\n + contents) if its parent is a tracked directory within the sparse-checkout\n + definition. This is required to prevent removing data created by tools that\n + perform caching operations for editors or build tools.\n +\n + Thus, 'git sparse-checkout clean' is both more aggressive and more careful\n + than 'git clean -fx':\n \n - If a staged change exists, then that entry is not stored as a sparse\n - tree entry and thus remains on-disk until committed or reset.\n + * It is more aggressive because it will remove _tracked_ files within the\n + sparse directories.\n \n - There are some interesting cases around merge conflict resolution, but\n - that will be carefully analyzed in the future.\n + * It is less aggressive because it will leave _untracked_ files that are\n + not contained in sparse directories.\n +\n + These special cases will be handled more explicitly in a future change that\n + expands tests for the 'git sparse-checkout clean' command. We handle some of\n + the modified, staged, and committed states including some impact on 'git\n + status' after cleaning.\n \n Signed-off-by: Derrick Stolee \n \n @@ Documentation/git-sparse-checkout.adoc: flags, with the same meaning as the flag\n all sparsity paths.\n \n +'clean'::\n -+\tRemove all files in tracked directories that are outside of the\n -+\tsparse-checkout definition. This subcommand requires cone-mode\n -+\tsparse-checkout to be sure that we know which directories are\n -+\tboth tracked and all contained paths are not in the sparse-checkout.\n -+\tThis command can be used to be sure the sparse index works\n -+\tefficiently, though it does not require enabling the sparse index\n -+ feature via the `index.sparse=true` configuration.\n ++\tOpportunistically remove files outside of the sparse-checkout\n ++\tdefinition. This command requires cone mode to use recursive\n ++\tdirectory matches to determine which files should be removed. A\n ++\tfile is considered for removal if it is contained within a tracked\n ++\tdirectory that is outside of the sparse-checkout definition.\n +++\n ++Some special cases, such as merge conflicts or modified files outside of\n ++the sparse-checkout definition could lead to keeping files that would\n ++otherwise be removed. Resolve conflicts, stage modifications, and use\n ++`git sparse-checkout reapply` in conjunction with `git sparse-checkout\n ++clean` to resolve these cases.\n +++\n ++This command can be used to be sure the sparse index works efficiently,\n ++though it does not require enabling the sparse index feature via the\n ++`index.sparse=true` configuration.\n +\n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\n @@ builtin/sparse-checkout.c: static int sparse_checkout_reapply(int argc, const ch\n +\t};\n +\n +\tsetup_work_tree();\n -+\tif (!repo->settings.sparse_checkout)\n ++\tif (!core_apply_sparse_checkout)\n +\t\tdie(_(\"must be in a sparse-checkout to clean directories\"));\n -+\tif (!repo->settings.sparse_checkout_cone)\n ++\tif (!core_sparse_checkout_cone)\n +\t\tdie(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n +\n +\targc = parse_options(argc, argv, prefix,\n @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'check-rules null termin\n \n +test_expect_success 'clean' '\n +\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n ++\tgit -C repo sparse-checkout reapply &&\n +\tmkdir repo/deep/deeper2 repo/folder1 &&\n ++\n ++\t# Add untracked files\n +\ttouch repo/deep/deeper2/file &&\n +\ttouch repo/folder1/file &&\n +\n @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'check-rules null termin\n +\ttest_path_is_missing repo/folder1\n +'\n +\n -+test_expect_success 'clean with staged sparse change' '\n ++test_expect_success 'clean with sparse file states' '\n ++\ttest_when_finished git reset --hard &&\n +\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n -+\tmkdir repo/deep/deeper2 repo/folder1 repo/folder2 &&\n -+\ttouch repo/deep/deeper2/file &&\n -+\ttouch repo/folder1/file &&\n ++\tmkdir repo/folder2 &&\n ++\n ++\t# create an untracked file and a modified file\n ++\ttouch repo/folder2/file &&\n +\techo dirty >repo/folder2/a &&\n +\n -+\tgit -C repo add --sparse folder1/file &&\n ++\t# First clean/reapply pass will do nothing.\n ++\tgit -C repo sparse-checkout clean >out &&\n ++\ttest_must_be_empty out &&\n ++\ttest_path_exists repo/folder2/a &&\n ++\ttest_path_exists repo/folder2/file &&\n ++\n ++\tgit -C repo sparse-checkout reapply 2>err &&\n ++\ttest_grep folder2 err &&\n ++\ttest_path_exists repo/folder2/a &&\n ++\ttest_path_exists repo/folder2/file &&\n ++\n ++\t# Now, stage the change to the tracked file.\n ++\tgit -C repo add --sparse folder2/a &&\n ++\n ++\t# Clean will continue not doing anything.\n ++\tgit -C repo sparse-checkout clean >out &&\n ++\ttest_line_count = 0 out &&\n ++\ttest_path_exists repo/folder2/a &&\n ++\ttest_path_exists repo/folder2/file &&\n ++\n ++\t# But we can reapply to remove the staged change.\n ++\tgit -C repo sparse-checkout reapply 2>err &&\n ++\ttest_grep folder2 err &&\n ++\ttest_path_is_missing repo/folder2/a &&\n ++\ttest_path_exists repo/folder2/file &&\n +\n -+\t# deletes deep/deeper2/ but leaves folder1/ and folder2/\n ++\t# We can clean now.\n +\tcat >expect <<-\\EOF &&\n -+\tRemoving deep/deeper2/\n ++\tRemoving folder2/\n +\tEOF\n -+\n +\tgit -C repo sparse-checkout clean >out &&\n +\ttest_cmp expect out &&\n ++\ttest_path_is_missing repo/folder2 &&\n +\n -+\ttest_path_is_missing repo/deep/deeper2 &&\n -+\ttest_path_exists repo/folder1\n ++\t# At the moment, the file is staged.\n ++\tcat >expect <<-\\EOF &&\n ++\tM folder2/a\n ++\tEOF\n ++\n ++\tgit -C repo status -s >out &&\n ++\ttest_cmp expect out &&\n ++\n ++\t# Reapply persists the modified state.\n ++\tgit -C repo sparse-checkout reapply &&\n ++\tcat >expect <<-\\EOF &&\n ++\tM folder2/a\n ++\tEOF\n ++\tgit -C repo status -s >out &&\n ++\ttest_cmp expect out &&\n ++\n ++\t# Committing the change leads to resolved status.\n ++\tgit -C repo commit -m \"modified\" &&\n ++\tgit -C repo status -s >out &&\n ++\ttest_must_be_empty out &&\n ++\n ++\t# Repeat, but this time commit before reapplying.\n ++\tmkdir repo/folder2/ &&\n ++\techo dirtier >repo/folder2/a &&\n ++\tgit -C repo add --sparse folder2/a &&\n ++\tgit -C repo sparse-checkout clean >out &&\n ++\ttest_must_be_empty out &&\n ++\ttest_path_exists repo/folder2/a &&\n ++\n ++\t# Committing without reapplying makes it look like a deletion\n ++\t# due to no skip-worktree bit.\n ++\tgit -C repo commit -m \"dirtier\" &&\n ++\tgit -C repo status -s >out &&\n ++\ttest_must_be_empty out &&\n ++\n ++\tgit -C repo sparse-checkout reapply &&\n ++\tgit -C repo status -s >out &&\n ++\ttest_must_be_empty out\n +'\n \n test_done\n 3: 221f3e5fb0 ! 3: 71a498db65 sparse-checkout: match some 'clean' behavior\n @@ Commit message\n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-sparse-checkout.adoc ##\n -@@ Documentation/git-sparse-checkout.adoc: all sparsity paths.\n - \tThis command can be used to be sure the sparse index works\n - \tefficiently, though it does not require enabling the sparse index\n - feature via the `index.sparse=true` configuration.\n +@@ Documentation/git-sparse-checkout.adoc: clean` to resolve these cases.\n + This command can be used to be sure the sparse index works efficiently,\n + though it does not require enabling the sparse index feature via the\n + `index.sparse=true` configuration.\n ++\n +To prevent accidental deletion of worktree files, the `clean` subcommand\n +will not delete any files without the `-f` or `--force` option, unless\n @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean' '\n \ttest_cmp expect out &&\n \n \ttest_path_is_missing repo/deep/deeper2 &&\n -@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean with staged sparse change' '\n - \n - \tgit -C repo add --sparse folder1/file &&\n - \n -+\tcat >expect <<-\\EOF &&\n -+\tWould remove deep/deeper2/\n -+\tEOF\n -+\n -+\tgit -C repo sparse-checkout clean --dry-run >out &&\n -+\ttest_cmp expect out &&\n -+\ttest_path_exists repo/deep/deeper2 &&\n -+\ttest_path_exists repo/folder1 &&\n -+\ttest_path_exists repo/folder2 &&\n -+\n - \t# deletes deep/deeper2/ but leaves folder1/ and folder2/\n - \tcat >expect <<-\\EOF &&\n - \tRemoving deep/deeper2/\n - \tEOF\n +@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean with sparse file states' '\n + \tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n + \tmkdir repo/folder2 &&\n \n +\t# The previous test case checked the -f option, so\n +\t# test the config option in this one.\n +\tgit -C repo config clean.requireForce false &&\n - \tgit -C repo sparse-checkout clean >out &&\n - \ttest_cmp expect out &&\n - \n - \ttest_path_is_missing repo/deep/deeper2 &&\n --\ttest_path_exists repo/folder1\n -+\ttest_path_exists repo/folder1 &&\n -+\ttest_path_exists repo/folder2\n -+'\n +\n + \t# create an untracked file and a modified file\n + \ttouch repo/folder2/file &&\n + \techo dirty >repo/folder2/a &&\n +@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean with sparse file states' '\n + \ttest_must_be_empty out\n + '\n + \n +test_expect_success 'clean with merge conflict status' '\n +\tgit clone repo clean-merge &&\n +\n @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean with staged spars\n +\n +\tgit -C clean-merge sparse-checkout clean -f >out &&\n +\ttest_cmp expect out\n - '\n - \n ++'\n ++\n test_done\n 4: fd9a20a392 = 4: 6b29dda4b8 dir: add generic \"walk all files\" helper\n 5: f464bb5ed6 ! 5: 2cde464fd4 sparse-checkout: add --verbose option to 'clean'\n @@ Commit message\n Add a new '--verbose' option for users to see all the files that are\n being deleted (with '--force') or would be deleted (with '--dry-run').\n \n + Based on usage, users may request further context on this list of files for\n + states such as tracked/untracked, unstaged/staged/conflicted, etc.\n +\n Signed-off-by: Derrick Stolee \n \n ## Documentation/git-sparse-checkout.adoc ##\n @@ builtin/sparse-checkout.c: static int sparse_checkout_clean(int argc, const char\n \n ## t/t1091-sparse-checkout-builtin.sh ##\n @@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'check-rules null termination' '\n - \n test_expect_success 'clean' '\n \tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n + \tgit -C repo sparse-checkout reapply &&\n -\tmkdir repo/deep/deeper2 repo/folder1 &&\n +\tmkdir -p repo/deep/deeper2 repo/folder1/extra/inside &&\n + \n + \t# Add untracked files\n \ttouch repo/deep/deeper2/file &&\n -\ttouch repo/folder1/file &&\n +\ttouch repo/folder1/extra/inside/file &&\n 6: d6dbc0b5ca = 6: 460e5e8157 sparse-index: point users to new 'clean' action\n 7: 0b1a2895b9 ! 7: 7f6f62bce6 t: expand tests around sparse merges and clean\n @@ Commit message\n Signed-off-by: Derrick Stolee \n \n ## t/t1091-sparse-checkout-builtin.sh ##\n -@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean with staged sparse change' '\n - \ttest_path_exists repo/folder2\n +@@ t/t1091-sparse-checkout-builtin.sh: test_expect_success 'clean with sparse file states' '\n + \ttest_must_be_empty out\n '\n \n -test_expect_success 'clean with merge conflict status' '\n 8: 82c24ce519 < -: ---------- sparse-checkout: make 'clean' clear more files\n\n-- \ngitgitgadget\n"},{"id":"526169","messageId":"437a57b665786a3d3b5142ce5f79d839decd486f.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 1/7] sparse-checkout: remove use of the_repository","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:05Z","receivedAt":"2025-09-12T10:30:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe logic for the 'git sparse-checkout' builtin uses the_repository all\nover the place, despite some use of a repository struct in different\nmethod parameters. Complete this removal of the_repository by using\n'repo' when possible.\n\nIn one place, there was already a local variable 'r' that was set to\nthe_repository, so move that to a method parameter.\n\nWe cannot remove the USE_THE_REPOSITORY_VARIABLE declaration as we are\nstill using global constants for the state of the sparse-checkout.\n\nSigned-off-by: Derrick Stolee \n---\n builtin/sparse-checkout.c | 117 ++++++++++++++++++++------------------\n 1 file changed, 62 insertions(+), 55 deletions(-)\n\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 8c333b3e2e..06de61bd9d 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -204,12 +204,12 @@ static void clean_tracked_sparse_directories(struct repository *r)\n \t\tensure_full_index(r->index);\n }\n \n-static int update_working_directory(struct pattern_list *pl)\n+static int update_working_directory(struct repository *r,\n+\t\t\t\t struct pattern_list *pl)\n {\n \tenum update_sparsity_result result;\n \tstruct unpack_trees_options o;\n \tstruct lock_file lock_file = LOCK_INIT;\n-\tstruct repository *r = the_repository;\n \tstruct pattern_list *old_pl;\n \n \t/* If no branch has been checked out, there are no updates to make. */\n@@ -327,7 +327,8 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)\n \tstring_list_clear(&sl, 0);\n }\n \n-static int write_patterns_and_update(struct pattern_list *pl)\n+static int write_patterns_and_update(struct repository *repo,\n+\t\t\t\t struct pattern_list *pl)\n {\n \tchar *sparse_filename;\n \tFILE *fp;\n@@ -336,15 +337,15 @@ static int write_patterns_and_update(struct pattern_list *pl)\n \n \tsparse_filename = get_sparse_checkout_filename();\n \n-\tif (safe_create_leading_directories(the_repository, sparse_filename))\n+\tif (safe_create_leading_directories(repo, sparse_filename))\n \t\tdie(_(\"failed to create directory for sparse-checkout file\"));\n \n \thold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR);\n \n-\tresult = update_working_directory(pl);\n+\tresult = update_working_directory(repo, pl);\n \tif (result) {\n \t\trollback_lock_file(&lk);\n-\t\tupdate_working_directory(NULL);\n+\t\tupdate_working_directory(repo, NULL);\n \t\tgoto out;\n \t}\n \n@@ -372,25 +373,26 @@ enum sparse_checkout_mode {\n \tMODE_CONE_PATTERNS = 2,\n };\n \n-static int set_config(enum sparse_checkout_mode mode)\n+static int set_config(struct repository *repo,\n+\t\t enum sparse_checkout_mode mode)\n {\n \t/* Update to use worktree config, if not already. */\n-\tif (init_worktree_config(the_repository)) {\n+\tif (init_worktree_config(repo)) {\n \t\terror(_(\"failed to initialize worktree config\"));\n \t\treturn 1;\n \t}\n \n-\tif (repo_config_set_worktree_gently(the_repository,\n+\tif (repo_config_set_worktree_gently(repo,\n \t\t\t\t\t \"core.sparseCheckout\",\n \t\t\t\t\t mode ? \"true\" : \"false\") ||\n-\t repo_config_set_worktree_gently(the_repository,\n+\t repo_config_set_worktree_gently(repo,\n \t\t\t\t\t \"core.sparseCheckoutCone\",\n \t\t\t\t\t mode == MODE_CONE_PATTERNS ?\n \t\t\t\t\t\t\"true\" : \"false\"))\n \t\treturn 1;\n \n \tif (mode == MODE_NO_PATTERNS)\n-\t\treturn set_sparse_index_config(the_repository, 0);\n+\t\treturn set_sparse_index_config(repo, 0);\n \n \treturn 0;\n }\n@@ -410,7 +412,7 @@ static enum sparse_checkout_mode update_cone_mode(int *cone_mode) {\n \treturn MODE_ALL_PATTERNS;\n }\n \n-static int update_modes(int *cone_mode, int *sparse_index)\n+static int update_modes(struct repository *repo, int *cone_mode, int *sparse_index)\n {\n \tint mode, record_mode;\n \n@@ -418,20 +420,20 @@ static int update_modes(int *cone_mode, int *sparse_index)\n \trecord_mode = (*cone_mode != -1) || !core_apply_sparse_checkout;\n \n \tmode = update_cone_mode(cone_mode);\n-\tif (record_mode && set_config(mode))\n+\tif (record_mode && set_config(repo, mode))\n \t\treturn 1;\n \n \t/* Set sparse-index/non-sparse-index mode if specified */\n \tif (*sparse_index >= 0) {\n-\t\tif (set_sparse_index_config(the_repository, *sparse_index) < 0)\n+\t\tif (set_sparse_index_config(repo, *sparse_index) < 0)\n \t\t\tdie(_(\"failed to modify sparse-index config\"));\n \n \t\t/* force an index rewrite */\n-\t\trepo_read_index(the_repository);\n-\t\tthe_repository->index->updated_workdir = 1;\n+\t\trepo_read_index(repo);\n+\t\trepo->index->updated_workdir = 1;\n \n \t\tif (!*sparse_index)\n-\t\t\tensure_full_index(the_repository->index);\n+\t\t\tensure_full_index(repo->index);\n \t}\n \n \treturn 0;\n@@ -448,7 +450,7 @@ static struct sparse_checkout_init_opts {\n } init_opts;\n \n static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n-\t\t\t\tstruct repository *repo UNUSED)\n+\t\t\t\tstruct repository *repo)\n {\n \tstruct pattern_list pl;\n \tchar *sparse_filename;\n@@ -464,7 +466,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \t};\n \n \tsetup_work_tree();\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tinit_opts.cone_mode = -1;\n \tinit_opts.sparse_index = -1;\n@@ -473,7 +475,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \t\t\t builtin_sparse_checkout_init_options,\n \t\t\t builtin_sparse_checkout_init_usage, 0);\n \n-\tif (update_modes(&init_opts.cone_mode, &init_opts.sparse_index))\n+\tif (update_modes(repo, &init_opts.cone_mode, &init_opts.sparse_index))\n \t\treturn 1;\n \n \tmemset(&pl, 0, sizeof(pl));\n@@ -485,14 +487,14 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \tif (res >= 0) {\n \t\tfree(sparse_filename);\n \t\tclear_pattern_list(&pl);\n-\t\treturn update_working_directory(NULL);\n+\t\treturn update_working_directory(repo, NULL);\n \t}\n \n-\tif (repo_get_oid(the_repository, \"HEAD\", &oid)) {\n+\tif (repo_get_oid(repo, \"HEAD\", &oid)) {\n \t\tFILE *fp;\n \n \t\t/* assume we are in a fresh repo, but update the sparse-checkout file */\n-\t\tif (safe_create_leading_directories(the_repository, sparse_filename))\n+\t\tif (safe_create_leading_directories(repo, sparse_filename))\n \t\t\tdie(_(\"unable to create leading directories of %s\"),\n \t\t\t sparse_filename);\n \t\tfp = xfopen(sparse_filename, \"w\");\n@@ -511,7 +513,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,\n \tadd_pattern(\"!/*/\", empty_base, 0, &pl, 0);\n \tpl.use_cone_patterns = init_opts.cone_mode;\n \n-\treturn write_patterns_and_update(&pl);\n+\treturn write_patterns_and_update(repo, &pl);\n }\n \n static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *path)\n@@ -674,7 +676,8 @@ static void add_patterns_literal(int argc, const char **argv,\n \tadd_patterns_from_input(pl, argc, argv, use_stdin ? stdin : NULL);\n }\n \n-static int modify_pattern_list(struct strvec *args, int use_stdin,\n+static int modify_pattern_list(struct repository *repo,\n+\t\t\t struct strvec *args, int use_stdin,\n \t\t\t enum modify_type m)\n {\n \tint result;\n@@ -696,22 +699,23 @@ static int modify_pattern_list(struct strvec *args, int use_stdin,\n \t}\n \n \tif (!core_apply_sparse_checkout) {\n-\t\tset_config(MODE_ALL_PATTERNS);\n+\t\tset_config(repo, MODE_ALL_PATTERNS);\n \t\tcore_apply_sparse_checkout = 1;\n \t\tchanged_config = 1;\n \t}\n \n-\tresult = write_patterns_and_update(pl);\n+\tresult = write_patterns_and_update(repo, pl);\n \n \tif (result && changed_config)\n-\t\tset_config(MODE_NO_PATTERNS);\n+\t\tset_config(repo, MODE_NO_PATTERNS);\n \n \tclear_pattern_list(pl);\n \tfree(pl);\n \treturn result;\n }\n \n-static void sanitize_paths(struct strvec *args,\n+static void sanitize_paths(struct repository *repo,\n+\t\t\t struct strvec *args,\n \t\t\t const char *prefix, int skip_checks)\n {\n \tint i;\n@@ -752,7 +756,7 @@ static void sanitize_paths(struct strvec *args,\n \n \tfor (i = 0; i < args->nr; i++) {\n \t\tstruct cache_entry *ce;\n-\t\tstruct index_state *index = the_repository->index;\n+\t\tstruct index_state *index = repo->index;\n \t\tint pos = index_name_pos(index, args->v[i], strlen(args->v[i]));\n \n \t\tif (pos < 0)\n@@ -779,7 +783,7 @@ static struct sparse_checkout_add_opts {\n } add_opts;\n \n static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n-\t\t\t struct repository *repo UNUSED)\n+\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_add_options[] = {\n \t\tOPT_BOOL_F(0, \"skip-checks\", &add_opts.skip_checks,\n@@ -796,7 +800,7 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n \tif (!core_apply_sparse_checkout)\n \t\tdie(_(\"no sparse-checkout to add to\"));\n \n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \targc = parse_options(argc, argv, prefix,\n \t\t\t builtin_sparse_checkout_add_options,\n@@ -804,9 +808,9 @@ static int sparse_checkout_add(int argc, const char **argv, const char *prefix,\n \n \tfor (int i = 0; i < argc; i++)\n \t\tstrvec_push(&patterns, argv[i]);\n-\tsanitize_paths(&patterns, prefix, add_opts.skip_checks);\n+\tsanitize_paths(repo, &patterns, prefix, add_opts.skip_checks);\n \n-\tret = modify_pattern_list(&patterns, add_opts.use_stdin, ADD);\n+\tret = modify_pattern_list(repo, &patterns, add_opts.use_stdin, ADD);\n \n \tstrvec_clear(&patterns);\n \treturn ret;\n@@ -825,7 +829,7 @@ static struct sparse_checkout_set_opts {\n } set_opts;\n \n static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n-\t\t\t struct repository *repo UNUSED)\n+\t\t\t struct repository *repo)\n {\n \tint default_patterns_nr = 2;\n \tconst char *default_patterns[] = {\"/*\", \"!/*/\", NULL};\n@@ -847,7 +851,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \tint ret;\n \n \tsetup_work_tree();\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tset_opts.cone_mode = -1;\n \tset_opts.sparse_index = -1;\n@@ -856,7 +860,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \t\t\t builtin_sparse_checkout_set_options,\n \t\t\t builtin_sparse_checkout_set_usage, 0);\n \n-\tif (update_modes(&set_opts.cone_mode, &set_opts.sparse_index))\n+\tif (update_modes(repo, &set_opts.cone_mode, &set_opts.sparse_index))\n \t\treturn 1;\n \n \t/*\n@@ -870,10 +874,10 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix,\n \t} else {\n \t\tfor (int i = 0; i < argc; i++)\n \t\t\tstrvec_push(&patterns, argv[i]);\n-\t\tsanitize_paths(&patterns, prefix, set_opts.skip_checks);\n+\t\tsanitize_paths(repo, &patterns, prefix, set_opts.skip_checks);\n \t}\n \n-\tret = modify_pattern_list(&patterns, set_opts.use_stdin, REPLACE);\n+\tret = modify_pattern_list(repo, &patterns, set_opts.use_stdin, REPLACE);\n \n \tstrvec_clear(&patterns);\n \treturn ret;\n@@ -891,7 +895,7 @@ static struct sparse_checkout_reapply_opts {\n \n static int sparse_checkout_reapply(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_reapply_options[] = {\n \t\tOPT_BOOL(0, \"cone\", &reapply_opts.cone_mode,\n@@ -912,12 +916,12 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n \t\t\t builtin_sparse_checkout_reapply_options,\n \t\t\t builtin_sparse_checkout_reapply_usage, 0);\n \n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n-\tif (update_modes(&reapply_opts.cone_mode, &reapply_opts.sparse_index))\n+\tif (update_modes(repo, &reapply_opts.cone_mode, &reapply_opts.sparse_index))\n \t\treturn 1;\n \n-\treturn update_working_directory(NULL);\n+\treturn update_working_directory(repo, NULL);\n }\n \n static char const * const builtin_sparse_checkout_disable_usage[] = {\n@@ -927,7 +931,7 @@ static char const * const builtin_sparse_checkout_disable_usage[] = {\n \n static int sparse_checkout_disable(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_disable_options[] = {\n \t\tOPT_END(),\n@@ -955,7 +959,7 @@ static int sparse_checkout_disable(int argc, const char **argv,\n \t * are expecting to do that when disabling sparse-checkout.\n \t */\n \tgive_advice_on_expansion = 0;\n-\trepo_read_index(the_repository);\n+\trepo_read_index(repo);\n \n \tmemset(&pl, 0, sizeof(pl));\n \thashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);\n@@ -966,13 +970,13 @@ static int sparse_checkout_disable(int argc, const char **argv,\n \tadd_pattern(\"/*\", empty_base, 0, &pl, 0);\n \n \tprepare_repo_settings(the_repository);\n-\tthe_repository->settings.sparse_index = 0;\n+\trepo->settings.sparse_index = 0;\n \n-\tif (update_working_directory(&pl))\n+\tif (update_working_directory(repo, &pl))\n \t\tdie(_(\"error while refreshing working directory\"));\n \n \tclear_pattern_list(&pl);\n-\treturn set_config(MODE_NO_PATTERNS);\n+\treturn set_config(repo, MODE_NO_PATTERNS);\n }\n \n static char const * const builtin_sparse_checkout_check_rules_usage[] = {\n@@ -987,14 +991,17 @@ static struct sparse_checkout_check_rules_opts {\n \tchar *rules_file;\n } check_rules_opts;\n \n-static int check_rules(struct pattern_list *pl, int null_terminated) {\n+static int check_rules(struct repository *repo,\n+\t\t struct pattern_list *pl,\n+\t\t int null_terminated)\n+{\n \tstruct strbuf line = STRBUF_INIT;\n \tstruct strbuf unquoted = STRBUF_INIT;\n \tchar *path;\n \tint line_terminator = null_terminated ? 0 : '\\n';\n \tstrbuf_getline_fn getline_fn = null_terminated ? strbuf_getline_nul\n \t\t: strbuf_getline;\n-\tthe_repository->index->sparse_checkout_patterns = pl;\n+\trepo->index->sparse_checkout_patterns = pl;\n \twhile (!getline_fn(&line, stdin)) {\n \t\tpath = line.buf;\n \t\tif (!null_terminated && line.buf[0] == '\"') {\n@@ -1006,7 +1013,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n \t\t\tpath = unquoted.buf;\n \t\t}\n \n-\t\tif (path_in_sparse_checkout(path, the_repository->index))\n+\t\tif (path_in_sparse_checkout(path, repo->index))\n \t\t\twrite_name_quoted(path, stdout, line_terminator);\n \t}\n \tstrbuf_release(&line);\n@@ -1016,7 +1023,7 @@ static int check_rules(struct pattern_list *pl, int null_terminated) {\n }\n \n static int sparse_checkout_check_rules(int argc, const char **argv, const char *prefix,\n-\t\t\t\t struct repository *repo UNUSED)\n+\t\t\t\t struct repository *repo)\n {\n \tstatic struct option builtin_sparse_checkout_check_rules_options[] = {\n \t\tOPT_BOOL('z', NULL, &check_rules_opts.null_termination,\n@@ -1055,7 +1062,7 @@ static int sparse_checkout_check_rules(int argc, const char **argv, const char *\n \t\tfree(sparse_filename);\n \t}\n \n-\tret = check_rules(&pl, check_rules_opts.null_termination);\n+\tret = check_rules(repo, &pl, check_rules_opts.null_termination);\n \tclear_pattern_list(&pl);\n \tfree(check_rules_opts.rules_file);\n \treturn ret;\n@@ -1084,8 +1091,8 @@ int cmd_sparse_checkout(int argc,\n \n \trepo_config(the_repository, git_default_config, NULL);\n \n-\tprepare_repo_settings(the_repository);\n-\tthe_repository->settings.command_requires_full_index = 0;\n+\tprepare_repo_settings(repo);\n+\trepo->settings.command_requires_full_index = 0;\n \n \treturn fn(argc, argv, prefix, repo);\n }\n-- \ngitgitgadget\n\n"},{"id":"526170","messageId":"a1564f74cfa19eee528a1a0c54b87ac52d1e8975.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 2/7] sparse-checkout: add basics of 'clean' command","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:06Z","receivedAt":"2025-09-12T10:30:18Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nWhen users change their sparse-checkout definitions to add new\ndirectories and remove old ones, there may be a few reasons why\ndirectories no longer in scope remain (ignored or excluded files still\nexist, Windows handles are still open, etc.). When these files still\nexist, the sparse index feature notices that a tracked, but sparse,\ndirectory still exists on disk and thus the index expands. This causes a\nperformance hit _and_ the advice printed isn't very helpful. Using 'git\nclean' isn't enough (generally '-dfx' may be needed) but also this may\nnot be sufficient.\n\nAdd a new subcommand to 'git sparse-checkout' that removes these\ntracked-but-sparse directories.\n\nThe implementation details provide a clear definition of what is happening,\nbut it is difficult to describe this without including the internal\nimplementation details. The core operation converts the index to a sparse\nindex (in memory if not already on disk) and then deletes any directories in\nthe worktree that correspond with a sparse directory entry in that sparse\nindex.\n\nIn the most common case, this means that a file will be removed if it is\ncontained within a directory that is both tracked and outside of the\nsparse-checkout definition. However, there can be exceptions depending on\nthe current state of the index:\n\n * If the worktree has a modification to a tracked, sparse file, then that\n file's parent directories will be expanded instead of represented as\n sparse directories. Siblings of those parent directories may be\n considered sparse.\n\n * If the user staged a sparse file with \"git add --sparse\", then that file\n loses the SKIP_WORKTREE bit until the sparse-checkout is reapplied. Until\n then, that file's parent directories are not represented as sparse\n directory entries and thus will not be removed. Siblings of those parent\n directories may be considered sparse. (There may be other reasons why\n the SKIP_WORKTREE bit was removed for a file and this impact on the\n sparse directories will apply to those as well.)\n\n * If the user has a merge conflict outside of the sparse-checkout\n definition, then those conflict entries prevent the parent directories\n from being represented as sparse directory entries and thus are not\n removed.\n\n * The cases above present reasons why certain _file conditions_ will impact\n which _directories_ are considered sparse. The list of tracked\n directories that are outside of the sparse-checkout definition but not\n represented as a sparse directory further reduces the list of files that\n will be removed.\n\nFor these complicated reasons, the documentation details a potential list of\nfiles that will be \"considered for removal\" instead of defining the list\nconcretely. The special cases can be handled by resolving conflicts,\ncommitting staged changes, and running 'git sparse-checkout reapply' to\nupdate the SKIP_WORKTREE bits as expected by the sparse-checkout definition.\n\nIt is important to make clear that this operation will remove ignored and\nexcluded files which would normally be ignored even by 'git clean -f' unless\nthe '-x' or '-X' option is provided. This is the most extreme method for\ndoing this, but it works when the sparse-checkout is in cone mode and is\nexpected to rescope based on directories, not files.\n\nThe current implementation always deletes these sparse directories\nwithout warning. This is unacceptable for a released version, but those\nfeatures will be added in changes coming immediately after this one.\n\nNote that this will not remove an untracked directory (or any of its\ncontents) if its parent is a tracked directory within the sparse-checkout\ndefinition. This is required to prevent removing data created by tools that\nperform caching operations for editors or build tools.\n\nThus, 'git sparse-checkout clean' is both more aggressive and more careful\nthan 'git clean -fx':\n\n * It is more aggressive because it will remove _tracked_ files within the\n sparse directories.\n\n * It is less aggressive because it will leave _untracked_ files that are\n not contained in sparse directories.\n\nThese special cases will be handled more explicitly in a future change that\nexpands tests for the 'git sparse-checkout clean' command. We handle some of\nthe modified, staged, and committed states including some impact on 'git\nstatus' after cleaning.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 19 ++++-\n builtin/sparse-checkout.c | 64 ++++++++++++++-\n t/t1091-sparse-checkout-builtin.sh | 103 +++++++++++++++++++++++++\n 3 files changed, 184 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex 529a8edd9c..baaebce746 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -9,7 +9,7 @@ git-sparse-checkout - Reduce your working tree to a subset of tracked files\n SYNOPSIS\n --------\n [verse]\n-'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) []\n+'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules | clean) []\n \n \n DESCRIPTION\n@@ -111,6 +111,23 @@ flags, with the same meaning as the flags from the `set` command, in order\n to change which sparsity mode you are using without needing to also respecify\n all sparsity paths.\n \n+'clean'::\n+\tOpportunistically remove files outside of the sparse-checkout\n+\tdefinition. This command requires cone mode to use recursive\n+\tdirectory matches to determine which files should be removed. A\n+\tfile is considered for removal if it is contained within a tracked\n+\tdirectory that is outside of the sparse-checkout definition.\n++\n+Some special cases, such as merge conflicts or modified files outside of\n+the sparse-checkout definition could lead to keeping files that would\n+otherwise be removed. Resolve conflicts, stage modifications, and use\n+`git sparse-checkout reapply` in conjunction with `git sparse-checkout\n+clean` to resolve these cases.\n++\n+This command can be used to be sure the sparse index works efficiently,\n+though it does not require enabling the sparse index feature via the\n+`index.sparse=true` configuration.\n+\n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\n \tworking directory to include all files.\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex 06de61bd9d..f7caa28f3f 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -2,6 +2,7 @@\n #define DISABLE_SIGN_COMPARE_WARNINGS\n \n #include \"builtin.h\"\n+#include \"abspath.h\"\n #include \"config.h\"\n #include \"dir.h\"\n #include \"environment.h\"\n@@ -23,7 +24,7 @@\n static const char *empty_base = \"\";\n \n static char const * const builtin_sparse_checkout_usage[] = {\n-\tN_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules) []\"),\n+\tN_(\"git sparse-checkout (init | list | set | add | reapply | disable | check-rules | clean) []\"),\n \tNULL\n };\n \n@@ -924,6 +925,66 @@ static int sparse_checkout_reapply(int argc, const char **argv,\n \treturn update_working_directory(repo, NULL);\n }\n \n+static char const * const builtin_sparse_checkout_clean_usage[] = {\n+\t\"git sparse-checkout clean [-n|--dry-run]\",\n+\tNULL\n+};\n+\n+static const char *msg_remove = N_(\"Removing %s\\n\");\n+\n+static int sparse_checkout_clean(int argc, const char **argv,\n+\t\t\t\t const char *prefix,\n+\t\t\t\t struct repository *repo)\n+{\n+\tstruct strbuf full_path = STRBUF_INIT;\n+\tconst char *msg = msg_remove;\n+\tsize_t worktree_len;\n+\n+\tstruct option builtin_sparse_checkout_clean_options[] = {\n+\t\tOPT_END(),\n+\t};\n+\n+\tsetup_work_tree();\n+\tif (!core_apply_sparse_checkout)\n+\t\tdie(_(\"must be in a sparse-checkout to clean directories\"));\n+\tif (!core_sparse_checkout_cone)\n+\t\tdie(_(\"must be in a cone-mode sparse-checkout to clean directories\"));\n+\n+\targc = parse_options(argc, argv, prefix,\n+\t\t\t builtin_sparse_checkout_clean_options,\n+\t\t\t builtin_sparse_checkout_clean_usage, 0);\n+\n+\tif (repo_read_index(repo) < 0)\n+\t\tdie(_(\"failed to read index\"));\n+\n+\tif (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n+\t repo->index->sparse_index == INDEX_EXPANDED)\n+\t\tdie(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n+\n+\tstrbuf_addstr(&full_path, repo->worktree);\n+\tstrbuf_addch(&full_path, '/');\n+\tworktree_len = full_path.len;\n+\n+\tfor (size_t i = 0; i < repo->index->cache_nr; i++) {\n+\t\tstruct cache_entry *ce = repo->index->cache[i];\n+\t\tif (!S_ISSPARSEDIR(ce->ce_mode))\n+\t\t\tcontinue;\n+\t\tstrbuf_setlen(&full_path, worktree_len);\n+\t\tstrbuf_add(&full_path, ce->name, ce->ce_namelen);\n+\n+\t\tif (!is_directory(full_path.buf))\n+\t\t\tcontinue;\n+\n+\t\tprintf(msg, ce->name);\n+\n+\t\tif (remove_dir_recursively(&full_path, 0))\n+\t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n+\t}\n+\n+\tstrbuf_release(&full_path);\n+\treturn 0;\n+}\n+\n static char const * const builtin_sparse_checkout_disable_usage[] = {\n \t\"git sparse-checkout disable\",\n \tNULL\n@@ -1080,6 +1141,7 @@ int cmd_sparse_checkout(int argc,\n \t\tOPT_SUBCOMMAND(\"set\", &fn, sparse_checkout_set),\n \t\tOPT_SUBCOMMAND(\"add\", &fn, sparse_checkout_add),\n \t\tOPT_SUBCOMMAND(\"reapply\", &fn, sparse_checkout_reapply),\n+\t\tOPT_SUBCOMMAND(\"clean\", &fn, sparse_checkout_clean),\n \t\tOPT_SUBCOMMAND(\"disable\", &fn, sparse_checkout_disable),\n \t\tOPT_SUBCOMMAND(\"check-rules\", &fn, sparse_checkout_check_rules),\n \t\tOPT_END(),\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex ab3a105fff..bdb7b21e32 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1050,5 +1050,108 @@ test_expect_success 'check-rules null termination' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'clean' '\n+\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n+\tgit -C repo sparse-checkout reapply &&\n+\tmkdir repo/deep/deeper2 repo/folder1 &&\n+\n+\t# Add untracked files\n+\ttouch repo/deep/deeper2/file &&\n+\ttouch repo/folder1/file &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tRemoving deep/deeper2/\n+\tRemoving folder1/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_cmp expect out &&\n+\n+\ttest_path_is_missing repo/deep/deeper2 &&\n+\ttest_path_is_missing repo/folder1\n+'\n+\n+test_expect_success 'clean with sparse file states' '\n+\ttest_when_finished git reset --hard &&\n+\tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n+\tmkdir repo/folder2 &&\n+\n+\t# create an untracked file and a modified file\n+\ttouch repo/folder2/file &&\n+\techo dirty >repo/folder2/a &&\n+\n+\t# First clean/reapply pass will do nothing.\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_must_be_empty out &&\n+\ttest_path_exists repo/folder2/a &&\n+\ttest_path_exists repo/folder2/file &&\n+\n+\tgit -C repo sparse-checkout reapply 2>err &&\n+\ttest_grep folder2 err &&\n+\ttest_path_exists repo/folder2/a &&\n+\ttest_path_exists repo/folder2/file &&\n+\n+\t# Now, stage the change to the tracked file.\n+\tgit -C repo add --sparse folder2/a &&\n+\n+\t# Clean will continue not doing anything.\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_line_count = 0 out &&\n+\ttest_path_exists repo/folder2/a &&\n+\ttest_path_exists repo/folder2/file &&\n+\n+\t# But we can reapply to remove the staged change.\n+\tgit -C repo sparse-checkout reapply 2>err &&\n+\ttest_grep folder2 err &&\n+\ttest_path_is_missing repo/folder2/a &&\n+\ttest_path_exists repo/folder2/file &&\n+\n+\t# We can clean now.\n+\tcat >expect <<-\\EOF &&\n+\tRemoving folder2/\n+\tEOF\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_cmp expect out &&\n+\ttest_path_is_missing repo/folder2 &&\n+\n+\t# At the moment, the file is staged.\n+\tcat >expect <<-\\EOF &&\n+\tM folder2/a\n+\tEOF\n+\n+\tgit -C repo status -s >out &&\n+\ttest_cmp expect out &&\n+\n+\t# Reapply persists the modified state.\n+\tgit -C repo sparse-checkout reapply &&\n+\tcat >expect <<-\\EOF &&\n+\tM folder2/a\n+\tEOF\n+\tgit -C repo status -s >out &&\n+\ttest_cmp expect out &&\n+\n+\t# Committing the change leads to resolved status.\n+\tgit -C repo commit -m \"modified\" &&\n+\tgit -C repo status -s >out &&\n+\ttest_must_be_empty out &&\n+\n+\t# Repeat, but this time commit before reapplying.\n+\tmkdir repo/folder2/ &&\n+\techo dirtier >repo/folder2/a &&\n+\tgit -C repo add --sparse folder2/a &&\n+\tgit -C repo sparse-checkout clean >out &&\n+\ttest_must_be_empty out &&\n+\ttest_path_exists repo/folder2/a &&\n+\n+\t# Committing without reapplying makes it look like a deletion\n+\t# due to no skip-worktree bit.\n+\tgit -C repo commit -m \"dirtier\" &&\n+\tgit -C repo status -s >out &&\n+\ttest_must_be_empty out &&\n+\n+\tgit -C repo sparse-checkout reapply &&\n+\tgit -C repo status -s >out &&\n+\ttest_must_be_empty out\n+'\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"526171","messageId":"71a498db65440ad40c6cbde92b74ea1105075737.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 3/7] sparse-checkout: match some 'clean' behavior","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:07Z","receivedAt":"2025-09-12T10:30:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe 'git sparse-checkout clean' subcommand is somewhat similar to 'git\nclean' in that it will delete files that should not be in the worktree.\nThe big difference is that it focuses on the directories that should not\nbe in the worktree due to cone-mode sparse-checkout. It also does not\ndiscriminate in the kinds of files and focuses on deleting entire\ndirectories.\n\nHowever, there are some restrictions that would be good to bring over\nfrom 'git clean', specifically how it refuses to do anything without the\n'-f'/'--force' or '-n'/'--dry-run' arguments. The 'clean.requireForce'\nconfig can be set to 'false' to imply '--force'.\n\nAdd this behavior to avoid accidental deletion of files that cannot be\nrecovered from Git.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 9 +++++\n builtin/sparse-checkout.c | 15 ++++++-\n t/t1091-sparse-checkout-builtin.sh | 54 +++++++++++++++++++++++++-\n 3 files changed, 76 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex baaebce746..42050ff5b5 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -127,6 +127,15 @@ clean` to resolve these cases.\n This command can be used to be sure the sparse index works efficiently,\n though it does not require enabling the sparse index feature via the\n `index.sparse=true` configuration.\n++\n+To prevent accidental deletion of worktree files, the `clean` subcommand\n+will not delete any files without the `-f` or `--force` option, unless\n+the `clean.requireForce` config option is set to `false`.\n++\n+The `--dry-run` option will list the directories that would be removed\n+without deleting them. Running in this mode can be helpful to predict the\n+behavior of the clean comand or to determine which kinds of files are left\n+in the sparse directories.\n \n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex f7caa28f3f..d777b64960 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -931,6 +931,7 @@ static char const * const builtin_sparse_checkout_clean_usage[] = {\n };\n \n static const char *msg_remove = N_(\"Removing %s\\n\");\n+static const char *msg_would_remove = N_(\"Would remove %s\\n\");\n \n static int sparse_checkout_clean(int argc, const char **argv,\n \t\t\t\t const char *prefix,\n@@ -939,8 +940,12 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \tstruct strbuf full_path = STRBUF_INIT;\n \tconst char *msg = msg_remove;\n \tsize_t worktree_len;\n+\tint force = 0, dry_run = 0;\n+\tint require_force = 1;\n \n \tstruct option builtin_sparse_checkout_clean_options[] = {\n+\t\tOPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n+\t\tOPT__FORCE(&force, N_(\"force\"), PARSE_OPT_NOCOMPLETE),\n \t\tOPT_END(),\n \t};\n \n@@ -954,6 +959,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \t\t\t builtin_sparse_checkout_clean_options,\n \t\t\t builtin_sparse_checkout_clean_usage, 0);\n \n+\trepo_config_get_bool(repo, \"clean.requireforce\", &require_force);\n+\tif (require_force && !force && !dry_run)\n+\t\tdie(_(\"for safety, refusing to clean without one of --force or --dry-run\"));\n+\n+\tif (dry_run)\n+\t\tmsg = msg_would_remove;\n+\n \tif (repo_read_index(repo) < 0)\n \t\tdie(_(\"failed to read index\"));\n \n@@ -977,7 +989,8 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \n \t\tprintf(msg, ce->name);\n \n-\t\tif (remove_dir_recursively(&full_path, 0))\n+\t\tif (dry_run <= 0 &&\n+\t\t remove_dir_recursively(&full_path, 0))\n \t\t\twarning_errno(_(\"failed to remove '%s'\"), ce->name);\n \t}\n \ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex bdb7b21e32..e6b768a8da 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1059,12 +1059,29 @@ test_expect_success 'clean' '\n \ttouch repo/deep/deeper2/file &&\n \ttouch repo/folder1/file &&\n \n+\ttest_must_fail git -C repo sparse-checkout clean 2>err &&\n+\tgrep \"refusing to clean\" err &&\n+\n+\tgit -C repo config clean.requireForce true &&\n+\ttest_must_fail git -C repo sparse-checkout clean 2>err &&\n+\tgrep \"refusing to clean\" err &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tWould remove deep/deeper2/\n+\tWould remove folder1/\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run >out &&\n+\ttest_cmp expect out &&\n+\ttest_path_exists repo/deep/deeper2 &&\n+\ttest_path_exists repo/folder1 &&\n+\n \tcat >expect <<-\\EOF &&\n \tRemoving deep/deeper2/\n \tRemoving folder1/\n \tEOF\n \n-\tgit -C repo sparse-checkout clean >out &&\n+\tgit -C repo sparse-checkout clean -f >out &&\n \ttest_cmp expect out &&\n \n \ttest_path_is_missing repo/deep/deeper2 &&\n@@ -1076,6 +1093,10 @@ test_expect_success 'clean with sparse file states' '\n \tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n \tmkdir repo/folder2 &&\n \n+\t# The previous test case checked the -f option, so\n+\t# test the config option in this one.\n+\tgit -C repo config clean.requireForce false &&\n+\n \t# create an untracked file and a modified file\n \ttouch repo/folder2/file &&\n \techo dirty >repo/folder2/a &&\n@@ -1154,4 +1175,35 @@ test_expect_success 'clean with sparse file states' '\n \ttest_must_be_empty out\n '\n \n+test_expect_success 'clean with merge conflict status' '\n+\tgit clone repo clean-merge &&\n+\n+\techo dirty >clean-merge/deep/deeper2/a &&\n+\ttouch clean-merge/folder2/extra &&\n+\n+\tcat >input <<-EOF &&\n+\t0 $ZERO_OID\tfolder1/a\n+\t100644 $(git -C clean-merge rev-parse HEAD:folder1/a) 1\tfolder1/a\n+\tEOF\n+\tgit -C clean-merge update-index --index-info err &&\n+\tgrep \"failed to convert index to a sparse index\" err &&\n+\n+\tmkdir -p clean-merge/folder1/ &&\n+\techo merged >clean-merge/folder1/a &&\n+\tgit -C clean-merge add --sparse folder1/a &&\n+\n+\t# deletes folder2/ but leaves staged change in folder1\n+\t# and dirty change in deep/deeper2/\n+\tcat >expect <<-\\EOF &&\n+\tRemoving folder2/\n+\tEOF\n+\n+\tgit -C clean-merge sparse-checkout clean -f >out &&\n+\ttest_cmp expect out\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"526172","messageId":"6b29dda4b8961a999855ada42996de120513f218.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 4/7] dir: add generic \"walk all files\" helper","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:08Z","receivedAt":"2025-09-12T10:30:22Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThere is sometimes a need to visit every file within a directory,\nrecursively. The main example is remove_dir_recursively(), though it has\nsome extra flags that make it want to iterate over paths in a custom\nway. There is also the fill_directory() approach but that involves an\nindex and a pathspec.\n\nThis change adds a new for_each_file_in_dir() method that will be\nhelpful in the next change.\n\nSigned-off-by: Derrick Stolee \n---\n dir.c | 28 ++++++++++++++++++++++++++++\n dir.h | 14 ++++++++++++++\n 2 files changed, 42 insertions(+)\n\ndiff --git a/dir.c b/dir.c\nindex 71108ac79b..194b36a8c4 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -30,6 +30,7 @@\n #include \"read-cache-ll.h\"\n #include \"setup.h\"\n #include \"sparse-index.h\"\n+#include \"strbuf.h\"\n #include \"submodule-config.h\"\n #include \"symlinks.h\"\n #include \"trace2.h\"\n@@ -87,6 +88,33 @@ struct dirent *readdir_skip_dot_and_dotdot(DIR *dirp)\n \treturn e;\n }\n \n+int for_each_file_in_dir(struct strbuf *path, file_iterator fn, const void *data)\n+{\n+\tstruct dirent *e;\n+\tint res = 0;\n+\tsize_t baselen = path->len;\n+\tDIR *dir = opendir(path->buf);\n+\n+\tif (!dir)\n+\t\treturn 0;\n+\n+\twhile (!res && (e = readdir_skip_dot_and_dotdot(dir)) != NULL) {\n+\t\tunsigned char dtype = get_dtype(e, path, 0);\n+\t\tstrbuf_setlen(path, baselen);\n+\t\tstrbuf_addstr(path, e->d_name);\n+\n+\t\tif (dtype == DT_REG) {\n+\t\t\tres = fn(path->buf, data);\n+\t\t} else if (dtype == DT_DIR) {\n+\t\t\tstrbuf_addch(path, '/');\n+\t\t\tres = for_each_file_in_dir(path, fn, data);\n+\t\t}\n+\t}\n+\n+\tclosedir(dir);\n+\treturn res;\n+}\n+\n int count_slashes(const char *s)\n {\n \tint cnt = 0;\ndiff --git a/dir.h b/dir.h\nindex fc9be7b427..20d4a078d6 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -536,6 +536,20 @@ int get_sparse_checkout_patterns(struct pattern_list *pl);\n */\n int remove_dir_recursively(struct strbuf *path, int flag);\n \n+/*\n+ * This function pointer type is called on each file discovered in\n+ * for_each_file_in_dir. The iteration stops if this method returns\n+ * non-zero.\n+ */\n+typedef int (*file_iterator)(const char *path, const void *data);\n+\n+struct strbuf;\n+/*\n+ * Given a directory path, recursively visit each file within, including\n+ * within subdirectories.\n+ */\n+int for_each_file_in_dir(struct strbuf *path, file_iterator fn, const void *data);\n+\n /*\n * Tries to remove the path, along with leading empty directories so long as\n * those empty directories are not startup_info->original_cwd. Ignores\n-- \ngitgitgadget\n\n"},{"id":"526173","messageId":"2cde464fd4c225144489c222537e5d7549f81849.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 5/7] sparse-checkout: add --verbose option to 'clean'","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:09Z","receivedAt":"2025-09-12T10:30:23Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nThe 'git sparse-checkout clean' subcommand is focused on directories,\ndeleting any tracked sparse directories to clean up the worktree and\nmake the sparse index feature work optimally.\n\nHowever, this directory-focused approach can leave users wondering why\nthose directories exist at all. In my experience, these files are left\nover due to ignore or exclude patterns, Windows file handles, or\npossibly merge conflict resolutions.\n\nAdd a new '--verbose' option for users to see all the files that are\nbeing deleted (with '--force') or would be deleted (with '--dry-run').\n\nBased on usage, users may request further context on this list of files for\nstates such as tracked/untracked, unstaged/staged/conflicted, etc.\n\nSigned-off-by: Derrick Stolee \n---\n Documentation/git-sparse-checkout.adoc | 5 +++++\n builtin/sparse-checkout.c | 28 ++++++++++++++++++++++++--\n t/t1091-sparse-checkout-builtin.sh | 14 ++++++++++---\n 3 files changed, 42 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\nindex 42050ff5b5..113728a0e7 100644\n--- a/Documentation/git-sparse-checkout.adoc\n+++ b/Documentation/git-sparse-checkout.adoc\n@@ -136,6 +136,11 @@ The `--dry-run` option will list the directories that would be removed\n without deleting them. Running in this mode can be helpful to predict the\n behavior of the clean comand or to determine which kinds of files are left\n in the sparse directories.\n++\n+The `--verbose` option will list every file within the directories that\n+are considered for removal. This option is helpful to determine if those\n+files are actually important or perhaps to explain why the directory is\n+still present despite the current sparse-checkout.\n \n 'disable'::\n \tDisable the `core.sparseCheckout` config setting, and restore the\ndiff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c\nindex d777b64960..8d3c3485f5 100644\n--- a/builtin/sparse-checkout.c\n+++ b/builtin/sparse-checkout.c\n@@ -930,6 +930,26 @@ static char const * const builtin_sparse_checkout_clean_usage[] = {\n \tNULL\n };\n \n+static int list_file_iterator(const char *path, const void *data)\n+{\n+\tconst char *msg = data;\n+\n+\tprintf(msg, path);\n+\treturn 0;\n+}\n+\n+static void list_every_file_in_dir(const char *msg,\n+\t\t\t\t const char *directory)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tstrbuf_addstr(&path, directory);\n+\tfprintf(stderr, \"list every file in %s\\n\", directory);\n+\n+\tfor_each_file_in_dir(&path, list_file_iterator, msg);\n+\tstrbuf_release(&path);\n+}\n+\n static const char *msg_remove = N_(\"Removing %s\\n\");\n static const char *msg_would_remove = N_(\"Would remove %s\\n\");\n \n@@ -940,12 +960,13 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \tstruct strbuf full_path = STRBUF_INIT;\n \tconst char *msg = msg_remove;\n \tsize_t worktree_len;\n-\tint force = 0, dry_run = 0;\n+\tint force = 0, dry_run = 0, verbose = 0;\n \tint require_force = 1;\n \n \tstruct option builtin_sparse_checkout_clean_options[] = {\n \t\tOPT__DRY_RUN(&dry_run, N_(\"dry run\")),\n \t\tOPT__FORCE(&force, N_(\"force\"), PARSE_OPT_NOCOMPLETE),\n+\t\tOPT__VERBOSE(&verbose, N_(\"report each affected file, not just directories\")),\n \t\tOPT_END(),\n \t};\n \n@@ -987,7 +1008,10 @@ static int sparse_checkout_clean(int argc, const char **argv,\n \t\tif (!is_directory(full_path.buf))\n \t\t\tcontinue;\n \n-\t\tprintf(msg, ce->name);\n+\t\tif (verbose)\n+\t\t\tlist_every_file_in_dir(msg, ce->name);\n+\t\telse\n+\t\t\tprintf(msg, ce->name);\n \n \t\tif (dry_run <= 0 &&\n \t\t remove_dir_recursively(&full_path, 0))\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex e6b768a8da..7b15fa669c 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1053,11 +1053,11 @@ test_expect_success 'check-rules null termination' '\n test_expect_success 'clean' '\n \tgit -C repo sparse-checkout set --cone deep/deeper1 &&\n \tgit -C repo sparse-checkout reapply &&\n-\tmkdir repo/deep/deeper2 repo/folder1 &&\n+\tmkdir -p repo/deep/deeper2 repo/folder1/extra/inside &&\n \n \t# Add untracked files\n \ttouch repo/deep/deeper2/file &&\n-\ttouch repo/folder1/file &&\n+\ttouch repo/folder1/extra/inside/file &&\n \n \ttest_must_fail git -C repo sparse-checkout clean 2>err &&\n \tgrep \"refusing to clean\" err &&\n@@ -1074,7 +1074,15 @@ test_expect_success 'clean' '\n \tgit -C repo sparse-checkout clean --dry-run >out &&\n \ttest_cmp expect out &&\n \ttest_path_exists repo/deep/deeper2 &&\n-\ttest_path_exists repo/folder1 &&\n+\ttest_path_exists repo/folder1/extra/inside/file &&\n+\n+\tcat >expect <<-\\EOF &&\n+\tWould remove deep/deeper2/file\n+\tWould remove folder1/extra/inside/file\n+\tEOF\n+\n+\tgit -C repo sparse-checkout clean --dry-run --verbose >out &&\n+\ttest_cmp expect out &&\n \n \tcat >expect <<-\\EOF &&\n \tRemoving deep/deeper2/\n-- \ngitgitgadget\n\n"},{"id":"526174","messageId":"460e5e8157fc87a4246c49f53e34495bc33d4432.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 6/7] sparse-index: point users to new 'clean' action","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:10Z","receivedAt":"2025-09-12T10:30:25Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nIn my experience, the most-common reason that the sparse index must\nexpand to a full one is because there is some leftover file in a tracked\ndirectory that is now outside of the sparse-checkout. The new 'git\nsparse-checkout clean' command will find and delete these directories,\nso point users to it when they hit the sparse index expansion advice.\n\nSigned-off-by: Derrick Stolee \n---\n sparse-index.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/sparse-index.c b/sparse-index.c\nindex 5634abafaa..5d14795063 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -32,7 +32,8 @@ int give_advice_on_expansion = 1;\n \t\"Your working directory likely has contents that are outside of\\n\" \\\n \t\"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n \t\"see your sparse-checkout definition and compare it to your working\\n\" \\\n-\t\"directory contents. Running 'git clean' may assist in this cleanup.\"\n+\t\"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n+\t\"in this cleanup.\"\n \n struct modify_index_context {\n \tstruct index_state *write;\n-- \ngitgitgadget\n\n"},{"id":"526175","messageId":"7f6f62bce607fd4d0438f05f9ee76f8547b96caf.1757673011.git.gitgitgadget@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH v3 7/7] t: expand tests around sparse merges and clean","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-09-12T10:30:11Z","receivedAt":"2025-09-12T10:30:26Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee \n\nWith the current implementation of 'git sparse-checkout clean', we\nnotice that a file that was in a conflicted state does not get cleaned\nup because of some internal details around the SKIP_WORKTREE bit.\n\nThis test is documenting the current behavior before we update it in the\nfollowing change.\n\nSigned-off-by: Derrick Stolee \n---\n t/t1091-sparse-checkout-builtin.sh | 56 ++++++++++++++++++------------\n 1 file changed, 34 insertions(+), 22 deletions(-)\n\ndiff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\nindex 7b15fa669c..b2da4feaef 100755\n--- a/t/t1091-sparse-checkout-builtin.sh\n+++ b/t/t1091-sparse-checkout-builtin.sh\n@@ -1183,35 +1183,47 @@ test_expect_success 'clean with sparse file states' '\n \ttest_must_be_empty out\n '\n \n-test_expect_success 'clean with merge conflict status' '\n-\tgit clone repo clean-merge &&\n+test_expect_success 'sparse-checkout operations with merge conflicts' '\n+\tgit clone repo merge &&\n \n-\techo dirty >clean-merge/deep/deeper2/a &&\n-\ttouch clean-merge/folder2/extra &&\n+\t(\n+\t\tcd merge &&\n+\t\tmkdir -p folder1/even/more/dirs &&\n+\t\techo base >folder1/even/more/dirs/file &&\n+\t\tgit add folder1 &&\n+\t\tgit commit -m \"base\" &&\n \n-\tcat >input <<-EOF &&\n-\t0 $ZERO_OID\tfolder1/a\n-\t100644 $(git -C clean-merge rev-parse HEAD:folder1/a) 1\tfolder1/a\n-\tEOF\n-\tgit -C clean-merge update-index --index-info folder1/even/more/dirs/file &&\n+\t\tgit commit -a -m \"right\" &&\n \n-\tgit -C clean-merge sparse-checkout set deep/deeper1 &&\n+\t\tgit checkout -b left HEAD~1 &&\n+\t\techo left >folder1/even/more/dirs/file &&\n+\t\tgit commit -a -m \"left\" &&\n \n-\ttest_must_fail git -C clean-merge sparse-checkout clean -f 2>err &&\n-\tgrep \"failed to convert index to a sparse index\" err &&\n+\t\tgit checkout -b merge &&\n+\t\tgit sparse-checkout set deep/deeper1 &&\n \n-\tmkdir -p clean-merge/folder1/ &&\n-\techo merged >clean-merge/folder1/a &&\n-\tgit -C clean-merge add --sparse folder1/a &&\n+\t\ttest_must_fail git merge -m \"will-conflict\" right &&\n \n-\t# deletes folder2/ but leaves staged change in folder1\n-\t# and dirty change in deep/deeper2/\n-\tcat >expect <<-\\EOF &&\n-\tRemoving folder2/\n-\tEOF\n+\t\ttest_must_fail git sparse-checkout clean -f 2>err &&\n+\t\tgrep \"failed to convert index to a sparse index\" err &&\n \n-\tgit -C clean-merge sparse-checkout clean -f >out &&\n-\ttest_cmp expect out\n+\t\techo merged >folder1/even/more/dirs/file &&\n+\t\tgit add --sparse folder1 &&\n+\t\tgit merge --continue &&\n+\n+\t\ttest_path_exists folder1/even/more/dirs/file &&\n+\n+\t\t# clean does not remove the file, because the\n+\t\t# SKIP_WORKTREE bit was not cleared by the merge command.\n+\t\tgit sparse-checkout clean -f >out &&\n+\t\ttest_line_count = 0 out &&\n+\t\ttest_path_exists folder1/even/more/dirs/file &&\n+\n+\t\tgit sparse-checkout reapply &&\n+\t\ttest_path_is_missing folder1\n+\t)\n '\n \n test_done\n-- \ngitgitgadget\n"},{"id":"526199","messageId":"xmqq348rodiw.fsf@gitster.g","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/7] sparse-checkout: add 'clean' command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-12T16:12:55Z","receivedAt":"2025-09-12T16:12:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" writes:\n\n> NEW: This series is rebased on a recent master to remove dependence on the\n> updates to the global variables used by the sparse-checkout system.\n>\n> When using cone-mode sparse-checkout, users specify which tracked\n> directories they want (recursively) and any directory not part of the parent\n> paths for those directories are considered \"out of scope\". When changing\n> sparse-checkouts, there are a variety of reasons why these \"out of scope\"\n> directories could remain, including:\n>\n> * The user has .gitignore or .git/info/exclude files that tell Git to not\n> remove files of a certain type.\n> * Some filesystem blocker prevented the removal of a tracked file. This is\n> usually more of an issue on Windows where a read handle will block file\n> deletion.\n\nThe updated documentation was easeier to follow (even though I had a\n\"Huh?\" moment with \"Opportunistically\" a bit). Comparing with the\nprevious version (with my rebase to get rid of the dependence on the\nother topic) and this one, I see a few more code paths have learned\nto pass \"struct repository *\" pointers throughout the callchain,\nwhich is very nice.\n\nWill replace. Thanks.\n"},{"id":"526381","messageId":"373d3441-6489-4b46-bd80-bc1ad68f0e9a@gmail.com","threadId":"63760","inReplyTo":"2cde464fd4c225144489c222537e5d7549f81849.1757673011.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 5/7] sparse-checkout: add --verbose option to 'clean'","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-15T18:09:14Z","receivedAt":"2025-09-15T18:09:38Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/12/2025 6:30 AM, Derrick Stolee via GitGitGadget wrote:\n> From: Derrick Stolee \n\n> +static void list_every_file_in_dir(const char *msg,\n> +\t\t\t\t const char *directory)\n> +{\n> +\tstruct strbuf path = STRBUF_INIT;\n> +\n> +\tstrbuf_addstr(&path, directory);\n> +\tfprintf(stderr, \"list every file in %s\\n\", directory);\n\nI don't know how I missed that this debugging output line snuck\nin and stayed through my testing. This line should be removed.\n\n> +\tfor_each_file_in_dir(&path, list_file_iterator, msg);\n> +\tstrbuf_release(&path);\n> +}\n> +\n\nThanks,\n-Stolee\n\n"},{"id":"526383","messageId":"xmqq348nh6mj.fsf@gitster.g","threadId":"63760","inReplyTo":"373d3441-6489-4b46-bd80-bc1ad68f0e9a@gmail.com","subject":"Re: [PATCH v3 5/7] sparse-checkout: add --verbose option to 'clean'","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-15T19:12:52Z","receivedAt":"2025-09-15T19:12:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\n> On 9/12/2025 6:30 AM, Derrick Stolee via GitGitGadget wrote:\n>> From: Derrick Stolee \n>\n>> +static void list_every_file_in_dir(const char *msg,\n>> +\t\t\t\t const char *directory)\n>> +{\n>> +\tstruct strbuf path = STRBUF_INIT;\n>> +\n>> +\tstrbuf_addstr(&path, directory);\n>> +\tfprintf(stderr, \"list every file in %s\\n\", directory);\n>\n> I don't know how I missed that this debugging output line snuck\n> in and stayed through my testing. This line should be removed.\n>\n>> +\tfor_each_file_in_dir(&path, list_file_iterator, msg);\n>> +\tstrbuf_release(&path);\n>> +}\n\n;-) Don't feel bad. Nobody among other people caught it either.\n\nLocally amended so no need to resubmit only to fix this.\n\nThanks.\n"},{"id":"526413","messageId":"9b001093-0d9d-47b2-be10-333534a748e2@gmail.com","threadId":"63760","inReplyTo":"xmqq348nh6mj.fsf@gitster.g","subject":"Re: [PATCH v3 5/7] sparse-checkout: add --verbose option to 'clean'","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-16T02:00:45Z","receivedAt":"2025-09-16T02:00:47Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/15/2025 3:12 PM, Junio C Hamano wrote:\n> Derrick Stolee writes:\n> \n>> On 9/12/2025 6:30 AM, Derrick Stolee via GitGitGadget wrote:\n>>> From: Derrick Stolee \n>>\n>>> +static void list_every_file_in_dir(const char *msg,\n>>> +\t\t\t\t const char *directory)\n>>> +{\n>>> +\tstruct strbuf path = STRBUF_INIT;\n>>> +\n>>> +\tstrbuf_addstr(&path, directory);\n>>> +\tfprintf(stderr, \"list every file in %s\\n\", directory);\n>>\n>> I don't know how I missed that this debugging output line snuck\n>> in and stayed through my testing. This line should be removed.\n>>\n>>> +\tfor_each_file_in_dir(&path, list_file_iterator, msg);\n>>> +\tstrbuf_release(&path);\n>>> +}\n> \n> ;-) Don't feel bad. Nobody among other people caught it either.\n> \n> Locally amended so no need to resubmit only to fix this.\n\nThanks!\n-Stolee\n"},{"id":"527421","messageId":"71c5d266-18ff-4ad3-81a3-30b9a1fefde2@gmail.com","threadId":"63760","inReplyTo":"xmqq348rodiw.fsf@gitster.g","subject":"Re: [PATCH v3 0/7] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-09-26T13:40:10Z","receivedAt":"2025-09-26T13:40:13Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 9/12/2025 12:12 PM, Junio C Hamano wrote:\n> \"Derrick Stolee via GitGitGadget\" writes:\n> \n>> NEW: This series is rebased on a recent master to remove dependence on the\n>> updates to the global variables used by the sparse-checkout system.\n>>\n>> When using cone-mode sparse-checkout, users specify which tracked\n>> directories they want (recursively) and any directory not part of the parent\n>> paths for those directories are considered \"out of scope\". When changing\n>> sparse-checkouts, there are a variety of reasons why these \"out of scope\"\n>> directories could remain, including:\n>>\n>> * The user has .gitignore or .git/info/exclude files that tell Git to not\n>> remove files of a certain type.\n>> * Some filesystem blocker prevented the removal of a tracked file. This is\n>> usually more of an issue on Windows where a read handle will block file\n>> deletion.\n> \n> The updated documentation was easeier to follow (even though I had a\n> \"Huh?\" moment with \"Opportunistically\" a bit). Comparing with the\n> previous version (with my rebase to get rid of the dependence on the\n> other topic) and this one, I see a few more code paths have learned\n> to pass \"struct repository *\" pointers throughout the callchain,\n> which is very nice.\n\nI'm hoping to see some feedback from Elijah whose feedback on v2 was\nvery helpful. Here is a ping to see if he's available.\n\nThanks,\n-Stolee\n\n"},{"id":"527441","messageId":"CABPp-BFMQSuKd03mv8oJ3+Bm7j4TDs2UrhhRnoyttB3mtdRFiQ@mail.gmail.com","threadId":"63760","inReplyTo":"71c5d266-18ff-4ad3-81a3-30b9a1fefde2@gmail.com","subject":"Re: [PATCH v3 0/7] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-09-26T18:58:32Z","receivedAt":"2025-09-26T18:58:44Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Sep 26, 2025 at 6:40 AM Derrick Stolee wrote:\n>\n> > The updated documentation was easeier to follow (even though I had a\n> > \"Huh?\" moment with \"Opportunistically\" a bit). Comparing with the\n> > previous version (with my rebase to get rid of the dependence on the\n> > other topic) and this one, I see a few more code paths have learned\n> > to pass \"struct repository *\" pointers throughout the callchain,\n> > which is very nice.\n>\n> I'm hoping to see some feedback from Elijah whose feedback on v2 was\n> very helpful. Here is a ping to see if he's available.\n\nSorry for the delay; I saw the series and I've been meaning to review,\nbut my daughter's hospitalization a few weeks ago (she recovered and\nis doing fine now) put me behind on several things. I should have\nsome time to take a look next week after the Git Merge conference.\n"},{"id":"528179","messageId":"CABPp-BGJ66e=znwOxvU0fLiw=KXwMg8FZxa283Jr5PXut_ECWg@mail.gmail.com","threadId":"63760","inReplyTo":"a1564f74cfa19eee528a1a0c54b87ac52d1e8975.1757673011.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 2/7] sparse-checkout: add basics of 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-10-07T22:49:20Z","receivedAt":"2025-10-07T22:49:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Sep 12, 2025 at 3:30 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> When users change their sparse-checkout definitions to add new\n> directories and remove old ones, there may be a few reasons why\n> directories no longer in scope remain (ignored or excluded files still\n> exist, Windows handles are still open, etc.). When these files still\n> exist, the sparse index feature notices that a tracked, but sparse,\n> directory still exists on disk and thus the index expands. This causes a\n> performance hit _and_ the advice printed isn't very helpful. Using 'git\n> clean' isn't enough (generally '-dfx' may be needed) but also this may\n> not be sufficient.\n>\n> Add a new subcommand to 'git sparse-checkout' that removes these\n> tracked-but-sparse directories.\n>\n> The implementation details provide a clear definition of what is happening,\n> but it is difficult to describe this without including the internal\n> implementation details. The core operation converts the index to a sparse\n> index (in memory if not already on disk) and then deletes any directories in\n> the worktree that correspond with a sparse directory entry in that sparse\n> index.\n>\n> In the most common case, this means that a file will be removed if it is\n> contained within a directory that is both tracked and outside of the\n> sparse-checkout definition. However, there can be exceptions depending on\n> the current state of the index:\n>\n> * If the worktree has a modification to a tracked, sparse file, then that\n> file's parent directories will be expanded instead of represented as\n> sparse directories. Siblings of those parent directories may be\n> considered sparse.\n>\n> * If the user staged a sparse file with \"git add --sparse\", then that file\n> loses the SKIP_WORKTREE bit until the sparse-checkout is reapplied. Until\n> then, that file's parent directories are not represented as sparse\n> directory entries and thus will not be removed. Siblings of those parent\n> directories may be considered sparse. (There may be other reasons why\n> the SKIP_WORKTREE bit was removed for a file and this impact on the\n> sparse directories will apply to those as well.)\n>\n> * If the user has a merge conflict outside of the sparse-checkout\n> definition, then those conflict entries prevent the parent directories\n> from being represented as sparse directory entries and thus are not\n> removed.\n>\n> * The cases above present reasons why certain _file conditions_ will impact\n> which _directories_ are considered sparse. The list of tracked\n> directories that are outside of the sparse-checkout definition but not\n> represented as a sparse directory further reduces the list of files that\n> will be removed.\n>\n> For these complicated reasons, the documentation details a potential list of\n> files that will be \"considered for removal\" instead of defining the list\n> concretely. The special cases can be handled by resolving conflicts,\n> committing staged changes, and running 'git sparse-checkout reapply' to\n> update the SKIP_WORKTREE bits as expected by the sparse-checkout definition.\n>\n> It is important to make clear that this operation will remove ignored and\n> excluded files which would normally be ignored even by 'git clean -f' unless\n> the '-x' or '-X' option is provided. This is the most extreme method for\n> doing this, but it works when the sparse-checkout is in cone mode and is\n> expected to rescope based on directories, not files.\n>\n> The current implementation always deletes these sparse directories\n> without warning. This is unacceptable for a released version, but those\n> features will be added in changes coming immediately after this one.\n>\n> Note that this will not remove an untracked directory (or any of its\n> contents) if its parent is a tracked directory within the sparse-checkout\n> definition. This is required to prevent removing data created by tools that\n> perform caching operations for editors or build tools.\n>\n> Thus, 'git sparse-checkout clean' is both more aggressive and more careful\n> than 'git clean -fx':\n>\n> * It is more aggressive because it will remove _tracked_ files within the\n> sparse directories.\n>\n> * It is less aggressive because it will leave _untracked_ files that are\n> not contained in sparse directories.\n>\n> These special cases will be handled more explicitly in a future change that\n> expands tests for the 'git sparse-checkout clean' command. We handle some of\n> the modified, staged, and committed states including some impact on 'git\n> status' after cleaning.\n\nI appreciate the more detailed explanation.\n\n> Signed-off-by: Derrick Stolee \n> ---\n> Documentation/git-sparse-checkout.adoc | 19 ++++-\n> builtin/sparse-checkout.c | 64 ++++++++++++++-\n> t/t1091-sparse-checkout-builtin.sh | 103 +++++++++++++++++++++++++\n> 3 files changed, 184 insertions(+), 2 deletions(-)\n>\n> diff --git a/Documentation/git-sparse-checkout.adoc b/Documentation/git-sparse-checkout.adoc\n> index 529a8edd9c..baaebce746 100644\n> --- a/Documentation/git-sparse-checkout.adoc\n> +++ b/Documentation/git-sparse-checkout.adoc\n> @@ -9,7 +9,7 @@ git-sparse-checkout - Reduce your working tree to a subset of tracked files\n> SYNOPSIS\n> --------\n> [verse]\n> -'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules) []\n> +'git sparse-checkout' (init | list | set | add | reapply | disable | check-rules | clean) []\n>\n>\n> DESCRIPTION\n> @@ -111,6 +111,23 @@ flags, with the same meaning as the flags from the `set` command, in order\n> to change which sparsity mode you are using without needing to also respecify\n> all sparsity paths.\n>\n> +'clean'::\n> + Opportunistically remove files outside of the sparse-checkout\n> + definition. This command requires cone mode to use recursive\n> + directory matches to determine which files should be removed. A\n> + file is considered for removal if it is contained within a tracked\n> + directory that is outside of the sparse-checkout definition.\n> ++\n> +Some special cases, such as merge conflicts or modified files outside of\n> +the sparse-checkout definition could lead to keeping files that would\n> +otherwise be removed. Resolve conflicts, stage modifications, and use\n> +`git sparse-checkout reapply` in conjunction with `git sparse-checkout\n> +clean` to resolve these cases.\n> ++\n> +This command can be used to be sure the sparse index works efficiently,\n> +though it does not require enabling the sparse index feature via the\n> +`index.sparse=true` configuration.\n\nThis expanded explanation for users is nice too. I particularly like\nthat you called out three things users need to use in conjunction with\nthis command -- resolving conflicts, staging modifications, and using\n`git sparse-checkout reapply`...\n\n[...]\n> + if (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n> + repo->index->sparse_index == INDEX_EXPANDED)\n> + die(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n\n...yet the error message you give to users only lists one of those\nthree things even though the other two may be the problem. Could we\nfix up the error message?\n\n> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh\n> index ab3a105fff..bdb7b21e32 100755\n> --- a/t/t1091-sparse-checkout-builtin.sh\n> +++ b/t/t1091-sparse-checkout-builtin.sh\n> @@ -1050,5 +1050,108 @@ test_expect_success 'check-rules null termination' '\n> test_cmp expect actual\n> '\n>\n> +test_expect_success 'clean' '\n> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n> + git -C repo sparse-checkout reapply &&\n> + mkdir repo/deep/deeper2 repo/folder1 &&\n> +\n> + # Add untracked files\n> + touch repo/deep/deeper2/file &&\n> + touch repo/folder1/file &&\n> +\n> + cat >expect <<-\\EOF &&\n> + Removing deep/deeper2/\n> + Removing folder1/\n> + EOF\n> +\n> + git -C repo sparse-checkout clean >out &&\n> + test_cmp expect out &&\n> +\n> + test_path_is_missing repo/deep/deeper2 &&\n> + test_path_is_missing repo/folder1\n> +'\n> +\n> +test_expect_success 'clean with sparse file states' '\n> + test_when_finished git reset --hard &&\n> + git -C repo sparse-checkout set --cone deep/deeper1 &&\n> + mkdir repo/folder2 &&\n> +\n> + # create an untracked file and a modified file\n> + touch repo/folder2/file &&\n> + echo dirty >repo/folder2/a &&\n> +\n> + # First clean/reapply pass will do nothing.\n> + git -C repo sparse-checkout clean >out &&\n> + test_must_be_empty out &&\n> + test_path_exists repo/folder2/a &&\n> + test_path_exists repo/folder2/file &&\n> +\n> + git -C repo sparse-checkout reapply 2>err &&\n> + test_grep folder2 err &&\n> + test_path_exists repo/folder2/a &&\n> + test_path_exists repo/folder2/file &&\n> +\n> + # Now, stage the change to the tracked file.\n> + git -C repo add --sparse folder2/a &&\n> +\n> + # Clean will continue not doing anything.\n> + git -C repo sparse-checkout clean >out &&\n> + test_line_count = 0 out &&\n> + test_path_exists repo/folder2/a &&\n> + test_path_exists repo/folder2/file &&\n> +\n> + # But we can reapply to remove the staged change.\n> + git -C repo sparse-checkout reapply 2>err &&\n> + test_grep folder2 err &&\n> + test_path_is_missing repo/folder2/a &&\n> + test_path_exists repo/folder2/file &&\n> +\n> + # We can clean now.\n> + cat >expect <<-\\EOF &&\n> + Removing folder2/\n> + EOF\n> + git -C repo sparse-checkout clean >out &&\n> + test_cmp expect out &&\n> + test_path_is_missing repo/folder2 &&\n> +\n> + # At the moment, the file is staged.\n> + cat >expect <<-\\EOF &&\n> + M folder2/a\n> + EOF\n> +\n> + git -C repo status -s >out &&\n> + test_cmp expect out &&\n> +\n> + # Reapply persists the modified state.\n> + git -C repo sparse-checkout reapply &&\n> + cat >expect <<-\\EOF &&\n> + M folder2/a\n> + EOF\n> + git -C repo status -s >out &&\n> + test_cmp expect out &&\n> +\n> + # Committing the change leads to resolved status.\n> + git -C repo commit -m \"modified\" &&\n> + git -C repo status -s >out &&\n> + test_must_be_empty out &&\n> +\n> + # Repeat, but this time commit before reapplying.\n> + mkdir repo/folder2/ &&\n> + echo dirtier >repo/folder2/a &&\n> + git -C repo add --sparse folder2/a &&\n> + git -C repo sparse-checkout clean >out &&\n> + test_must_be_empty out &&\n> + test_path_exists repo/folder2/a &&\n> +\n> + # Committing without reapplying makes it look like a deletion\n> + # due to no skip-worktree bit.\n> + git -C repo commit -m \"dirtier\" &&\n> + git -C repo status -s >out &&\n> + test_must_be_empty out &&\n> +\n> + git -C repo sparse-checkout reapply &&\n> + git -C repo status -s >out &&\n> + test_must_be_empty out\n> +'\n>\n> test_done\n> --\n> gitgitgadget\n\nI very much appreciate the extended test.\n"},{"id":"528180","messageId":"CABPp-BG_TwQRcCp4iCpcDngb3W7rprnX4FHLs4A884BXQVk_eQ@mail.gmail.com","threadId":"63760","inReplyTo":"460e5e8157fc87a4246c49f53e34495bc33d4432.1757673011.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 6/7] sparse-index: point users to new 'clean' action","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-10-07T22:53:22Z","receivedAt":"2025-10-07T22:53:34Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Sep 12, 2025 at 3:30 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> From: Derrick Stolee \n>\n> In my experience, the most-common reason that the sparse index must\n> expand to a full one is because there is some leftover file in a tracked\n> directory that is now outside of the sparse-checkout. The new 'git\n> sparse-checkout clean' command will find and delete these directories,\n> so point users to it when they hit the sparse index expansion advice.\n>\n> Signed-off-by: Derrick Stolee \n> ---\n> sparse-index.c | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n>\n> diff --git a/sparse-index.c b/sparse-index.c\n> index 5634abafaa..5d14795063 100644\n> --- a/sparse-index.c\n> +++ b/sparse-index.c\n> @@ -32,7 +32,8 @@ int give_advice_on_expansion = 1;\n> \"Your working directory likely has contents that are outside of\\n\" \\\n> \"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n> \"see your sparse-checkout definition and compare it to your working\\n\" \\\n> - \"directory contents. Running 'git clean' may assist in this cleanup.\"\n> + \"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n> + \"in this cleanup.\"\n\nGiven that you dropped patch 8 and explicitly call out in the\ndocumentation of `git sparse-checkout clean` that it alone is not\nsufficient to do the cleanup, should this advice be calling out a\ncombination of `git sparse-checkout clean` and `git sparse-checkout\nreapply` ? (Should it also suggest an order for running those two; I\nseem to recall that the order mattered, but can't recall which one\nneeds to run first or if it is situation dependent.)\n"},{"id":"528183","messageId":"CABPp-BEEHsFwE-bDjcUoDtAYm9pvVN0tGUaoh0KPEJu23LywOQ@mail.gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/7] sparse-checkout: add 'clean' command","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-10-07T23:07:09Z","receivedAt":"2025-10-07T23:07:21Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Sep 12, 2025 at 3:30 AM Derrick Stolee via GitGitGadget\n wrote:\n>\n> NEW: This series is rebased on a recent master to remove dependence on the\n> updates to the global variables used by the sparse-checkout system.\n>\n> When using cone-mode sparse-checkout, users specify which tracked\n> directories they want (recursively) and any directory not part of the parent\n> paths for those directories are considered \"out of scope\". When changing\n> sparse-checkouts, there are a variety of reasons why these \"out of scope\"\n> directories could remain, including:\n>\n> * The user has .gitignore or .git/info/exclude files that tell Git to not\n> remove files of a certain type.\n> * Some filesystem blocker prevented the removal of a tracked file. This is\n> usually more of an issue on Windows where a read handle will block file\n> deletion.\n>\n> Typically, this would not mean too much for the user experience. A few extra\n> filesystem checks might be required to satisfy git status commands, but the\n> scope of the performance hit is relative to how many cruft files are left\n> over in this situation.\n>\n> However, when using the sparse index, these tracked sparse directories cause\n> significant performance issues. When noticing that the index contains a\n> sparse directory but that directory exists on disk, Git needs to expand that\n> sparse directory to determine which files are tracked or untracked. The\n> current mechanism expands the entire index to a full one, an expensive\n> operation that scales with the total number of paths at HEAD and not just\n> the number of cruft files left over.\n>\n> Advice was added in 9479a31d603 (advice: warn when sparse index expands,\n> 2024-07-08) to help users determine that they were in this state. However,\n> the advice doesn't actually recommend helpful ways to get out of this state.\n> Recommending \"git clean\" on its own is incomplete, as typically users\n> actually need 'git clean -dfx' to clear out the ignored or excluded files.\n> Even then, they may need 'git sparse-checkout reapply' afterwards to clear\n> the sparse directories.\n>\n> The advice was successful in helping to alert users to the problem, which is\n> how I got wind of many of these cases for how users get into this state.\n> It's now time to give them a tool that helps them out of this state.\n\n...in v2, I found some cases where the tool doesn't help them get out\nof this state. In v3, you documented those cases, and didn't attempt\nto provide a combined tool. I'm a little disappointed at the\nend-state, because it means we tell users to use a combination of\ncommands, and they may have to figure out the order to run those\ncommands in. However, I think with the documentation you've got,\nwe've at least improved on the status quo, so we could always make\nfurther improvements later.\n\nThere was an error message and an advice message that I think could be\ntouched up to improve on this (commented on both in v3), otherwise I\nthink this series is good enough to merge down.\n\n[...]\n> This option would be preferred to something like 'git clean -dfx' since it\n> does not clear the excluded files that are still within the sparse-checkout.\n> Instead, it performs the exact filesystem operations required to refresh the\n> sparse index performance back to what is expected.\n\nThis paragraph is the same from v2 of the cover letter, but we know\nthis paragraph to be false -- the new command only works in a subset\nof applicable cases, otherwise an additional command (sparse-checkout\nreapply) is also needed. So, it feels like this paragraph should be\nupdated.\n\n> Updates in V3\n> =============\n>\n> Huge thanks to Elijah for such a detailed review. Apologies for the delay in\n> responding.\n\nLikewise...it's nearly been a month since you sent this. :-(\n\n> * Removed dependency on stalled series around updating the sparse-checkout\n> globals.\n> * Commit message and documentation is updated to better describe the\n> conditions that qualify a file or directory for removal.\n> * Tests are expanded significantly to include special cases and\n> aftereffects.\n> * A note is added around possible future expansion of the --verbose option\n> to include more detailed status information on the files that would be\n> deleted.\n\nAll much appreciated.\n\n> * Due to a situation where a file appears as \"modified and deleted\" after\n> the more aggressive updating of the tree, the previous patch 8 is removed\n> (for now). I may reconsider and send a version in the future that avoids\n> this issue. Tests from the earlier patches are more expanded in such a\n> way that the aggressive implementation requires test changes that reveal\n> this problem. See [1] for a copy of this change and how it impacts the\n> latest tests.\n\nYeah, I think I was hoping that patch 8 would instead be modified to\nhandle the additional cases (or more patches added to make it all work\nout), but punting that for future work seems viable too.\n\nIn summary, I think this series is close to ready to merge, but I\nthink a couple wording improvements to an error message and advice\nmessage that I called out in separate emails on this series makes\nsense to fix up first.\n\nThanks for working on this!\n"},{"id":"529166","messageId":"0f517519-74bb-4f46-b7df-5e25a780a536@gmail.com","threadId":"63760","inReplyTo":"CABPp-BGJ66e=znwOxvU0fLiw=KXwMg8FZxa283Jr5PXut_ECWg@mail.gmail.com","subject":"Re: [PATCH v3 2/7] sparse-checkout: add basics of 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-10-20T14:16:42Z","receivedAt":"2025-10-20T14:16:45Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/7/25 6:49 PM, Elijah Newren wrote:\n> On Fri, Sep 12, 2025 at 3:30 AM Derrick Stolee via GitGitGadget\n\n>> +Some special cases, such as merge conflicts or modified files outside of\n>> +the sparse-checkout definition could lead to keeping files that would\n>> +otherwise be removed. Resolve conflicts, stage modifications, and use\n>> +`git sparse-checkout reapply` in conjunction with `git sparse-checkout\n>> +clean` to resolve these cases.\n>> ++\n>> +This command can be used to be sure the sparse index works efficiently,\n>> +though it does not require enabling the sparse index feature via the\n>> +`index.sparse=true` configuration.\n> \n> This expanded explanation for users is nice too. I particularly like\n> that you called out three things users need to use in conjunction with\n> this command -- resolving conflicts, staging modifications, and using\n> `git sparse-checkout reapply`...\n\nThese cases are about 'git sparse-checkout clean' itself...\n> [...]\n>> + if (convert_to_sparse(repo->index, SPARSE_INDEX_MEMORY_ONLY) ||\n>> + repo->index->sparse_index == INDEX_EXPANDED)\n>> + die(_(\"failed to convert index to a sparse index; resolve merge conflicts and try again\"));\n> \n> ...yet the error message you give to users only lists one of those\n> three things even though the other two may be the problem. Could we\n> fix up the error message?\n\nBut this is specifically about the convert_to_sparse() action, which\nis very limited in what could cause it to fail.\n\nI _will_ update the advice later (added in Patch 6) to include the\nmore detailed set of actions.\n\nThanks,\n-Stolee\n\n"},{"id":"529167","messageId":"3848ce2c-c2d3-47db-81cb-14e79d3d7f0e@gmail.com","threadId":"63760","inReplyTo":"CABPp-BG_TwQRcCp4iCpcDngb3W7rprnX4FHLs4A884BXQVk_eQ@mail.gmail.com","subject":"Re: [PATCH v3 6/7] sparse-index: point users to new 'clean' action","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-10-20T14:17:13Z","receivedAt":"2025-10-20T14:17:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/7/25 6:53 PM, Elijah Newren wrote:\n> On Fri, Sep 12, 2025 at 3:30 AM Derrick Stolee via GitGitGadget\n> wrote:\n>>\n>> From: Derrick Stolee \n>>\n>> In my experience, the most-common reason that the sparse index must\n>> expand to a full one is because there is some leftover file in a tracked\n>> directory that is now outside of the sparse-checkout. The new 'git\n>> sparse-checkout clean' command will find and delete these directories,\n>> so point users to it when they hit the sparse index expansion advice.\n>>\n>> Signed-off-by: Derrick Stolee \n>> ---\n>> sparse-index.c | 3 ++-\n>> 1 file changed, 2 insertions(+), 1 deletion(-)\n>>\n>> diff --git a/sparse-index.c b/sparse-index.c\n>> index 5634abafaa..5d14795063 100644\n>> --- a/sparse-index.c\n>> +++ b/sparse-index.c\n>> @@ -32,7 +32,8 @@ int give_advice_on_expansion = 1;\n>> \"Your working directory likely has contents that are outside of\\n\" \\\n>> \"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n>> \"see your sparse-checkout definition and compare it to your working\\n\" \\\n>> - \"directory contents. Running 'git clean' may assist in this cleanup.\"\n>> + \"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n>> + \"in this cleanup.\"\n> \n> Given that you dropped patch 8 and explicitly call out in the\n> documentation of `git sparse-checkout clean` that it alone is not\n> sufficient to do the cleanup, should this advice be calling out a\n> combination of `git sparse-checkout clean` and `git sparse-checkout\n> reapply` ? (Should it also suggest an order for running those two; I\n> seem to recall that the order mattered, but can't recall which one\n> needs to run first or if it is situation dependent.)\n\nI'll expand this advice in an upcoming patch 8.\n\nThanks,\n-Stolee\n\n"},{"id":"529168","messageId":"a34cc559-5823-4e68-8f3f-07c182f7299b@gmail.com","threadId":"63760","inReplyTo":"pull.1941.v3.git.1757673011.gitgitgadget@gmail.com","subject":"[PATCH 8/8] sparse-index: improve advice message instructions","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-10-20T14:24:04Z","receivedAt":"2025-10-20T14:24:06Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":" From 0ee829fea73d495dd32deda4553ea00f9299c701 Mon Sep 17 00:00:00 2001\nFrom: Derrick Stolee \nDate: Mon, 20 Oct 2025 10:19:22 -0400\nSubject: [PATCH 8/8] sparse-index: improve advice message instructions\n\nWhen an on-disk sparse index is expanded to a full one, this could be due to\nsome worktree state that requires looking at file entries hidden within\nsparse tree entries. These can be avoided if the worktree is cleaned up and\nsome other issues related to the index state. Expand the advice message to\ninclude all of these cases, since 'git sparse-checkout clean' is not\ncurrently capable of handling all cases.\n\nIn the future, we may improve the behavior of 'git sparse-checkout clean' to\nhandle all of the cases.\n\nHelped-by: Elijah Newren \nSigned-off-by: Derrick Stolee \n---\n\nHere is an add-on patch to add to this series to hopefully satisfy\nElijah's feedback. Sorry it took so long to be able to get back to\nthis!\n\n-Stolee\n\n\n sparse-index.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/sparse-index.c b/sparse-index.c\nindex 5d14795063b..76f90da5f5f 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -32,8 +32,9 @@ int give_advice_on_expansion = 1;\n \t\"Your working directory likely has contents that are outside of\\n\" \\\n \t\"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n \t\"see your sparse-checkout definition and compare it to your working\\n\" \\\n-\t\"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n-\t\"in this cleanup.\"\n+\t\"directory contents. Cleaning up any merge conflicts or staged\\n\" \\\n+\t\"changes before running 'git sparse-checkout clean' or 'git\\n\" \\\n+\t\"sparse-checkout reapply' may assist in this cleanup.\"\n\n struct modify_index_context {\n \tstruct index_state *write;\n-- \n2.47.0.vfs.0.3\n\n\n"},{"id":"529169","messageId":"528686b1-e0e6-4c4b-8339-ef894652f774@gmail.com","threadId":"63760","inReplyTo":"CABPp-BEEHsFwE-bDjcUoDtAYm9pvVN0tGUaoh0KPEJu23LywOQ@mail.gmail.com","subject":"Re: [PATCH v3 0/7] sparse-checkout: add 'clean' command","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-10-20T14:25:59Z","receivedAt":"2025-10-20T14:26:01Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/7/25 7:07 PM, Elijah Newren wrote:\n> On Fri, Sep 12, 2025 at 3:30 AM Derrick Stolee via GitGitGadget\n\n>> Updates in V3\n>> =============\n>>\n>> Huge thanks to Elijah for such a detailed review. Apologies for the delay in\n>> responding.\n> \n> Likewise...it's nearly been a month since you sent this. :-(\n\nIt's my turn to be late in responding. :(\n\n> In summary, I think this series is close to ready to merge, but I\n> think a couple wording improvements to an error message and advice\n> message that I called out in separate emails on this series makes\n> sense to fix up first.\n\nI sent a patch 8 to the v3 thread [1] that includes an update to\nthe error message. This could be squashed, but I sent an extra patch\nto avoid a reroll of 'next'.\n\n[1] https://lore.kernel.org/git/a34cc559-5823-4e68-8f3f-07c182f7299b@gmail.com/\n\nThanks,\n-Stolee\n\n"},{"id":"529176","messageId":"xmqq7bwp1qqn.fsf@gitster.g","threadId":"63760","inReplyTo":"a34cc559-5823-4e68-8f3f-07c182f7299b@gmail.com","subject":"Re: [PATCH 8/8] sparse-index: improve advice message instructions","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-10-20T16:29:20Z","receivedAt":"2025-10-20T16:29:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Derrick Stolee writes:\n\nI am getting\n\nwarning: Patch sent with format=flowed; space at the end of lines might be lost.\nApplying: sparse-index: improve advice message instructions\n\nbut hopefully the result is correct.\n\n> From 0ee829fea73d495dd32deda4553ea00f9299c701 Mon Sep 17 00:00:00 2001\n> From: Derrick Stolee \n> Date: Mon, 20 Oct 2025 10:19:22 -0400\n> Subject: [PATCH 8/8] sparse-index: improve advice message instructions\n\nI will strip the above out with \"commit --amend\"\n\n> When an on-disk sparse index is expanded to a full one, this could be due to\n> some worktree state that requires looking at file entries hidden within\n> sparse tree entries.\n\nI would find it smoother to read with \"this could be\" -> \"it could\nbe\", but that would be just me.\n\n> These can be avoided if the worktree is cleaned up and\n> some other issues related to the index state.\n\nNow \"These\" confused me. Does it refer to the same thing as the\nprevious sentence refers to with \"this\"? Also, I can understand up\nto \"if the worktree is cleaned up\", but the rest of the sentence\ndoes not quite parse for me. It may be that we are missing \" are\nresolved\" between \"state\" and the full stop? \n\nEven though it would leave readers in suspense to know what \"some\nother issues\" are, it is answered by reading the message updated by\nthe patch, so it is OK ;-)\n\n> Expand the advice message to\n> include all of these cases, since 'git sparse-checkout clean' is not\n> currently capable of handling all cases.\n>\n> In the future, we may improve the behavior of 'git sparse-checkout clean' to\n> handle all of the cases.\n>\n> Helped-by: Elijah Newren \n> Signed-off-by: Derrick Stolee \n> ---\n> Here is an add-on patch to add to this series to hopefully satisfy\n> Elijah's feedback. Sorry it took so long to be able to get back to\n> this!\n\nGreat. Thanks, both.\n\nIf the title were numbered [8/7], that would have been even nicer,\nbut I was following the discussion this time, so it was not a\nsurprise to me to see only [8/8] in my mailbox.\n\n> diff --git a/sparse-index.c b/sparse-index.c\n> index 5d14795063b..76f90da5f5f 100644\n> --- a/sparse-index.c\n> +++ b/sparse-index.c\n> @@ -32,8 +32,9 @@ int give_advice_on_expansion = 1;\n> \t\"Your working directory likely has contents that are outside of\\n\" \\\n> \t\"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n> \t\"see your sparse-checkout definition and compare it to your working\\n\" \\\n> -\t\"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n> -\t\"in this cleanup.\"\n> +\t\"directory contents. Cleaning up any merge conflicts or staged\\n\" \\\n> +\t\"changes before running 'git sparse-checkout clean' or 'git\\n\" \\\n> +\t\"sparse-checkout reapply' may assist in this cleanup.\"\n>\n> struct modify_index_context {\n> \tstruct index_state *write;\n"},{"id":"529551","messageId":"CABPp-BEvCiwxQWSX-JUCA=CGqu6TmuG+vJSUstQ+AediyPKZxg@mail.gmail.com","threadId":"63760","inReplyTo":"a34cc559-5823-4e68-8f3f-07c182f7299b@gmail.com","subject":"Re: [PATCH 8/8] sparse-index: improve advice message instructions","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-10-24T02:22:36Z","receivedAt":"2025-10-24T02:22:48Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Oct 20, 2025 at 10:24 AM Derrick Stolee wrote:\n>\n> From 0ee829fea73d495dd32deda4553ea00f9299c701 Mon Sep 17 00:00:00 2001\n> From: Derrick Stolee \n> Date: Mon, 20 Oct 2025 10:19:22 -0400\n> Subject: [PATCH 8/8] sparse-index: improve advice message instructions\n>\n> When an on-disk sparse index is expanded to a full one, this could be due to\n> some worktree state that requires looking at file entries hidden within\n> sparse tree entries. These can be avoided if the worktree is cleaned up and\n> some other issues related to the index state. Expand the advice message to\n> include all of these cases, since 'git sparse-checkout clean' is not\n> currently capable of handling all cases.\n\nThis paragraph feels slightly clumsy or awkward to parse, but...\n\n>\n> In the future, we may improve the behavior of 'git sparse-checkout clean' to\n> handle all of the cases.\n>\n> Helped-by: Elijah Newren \n> Signed-off-by: Derrick Stolee \n> ---\n>\n> Here is an add-on patch to add to this series to hopefully satisfy\n> Elijah's feedback. Sorry it took so long to be able to get back to\n> this!\n>\n> -Stolee\n>\n>\n> sparse-index.c | 5 +++--\n> 1 file changed, 3 insertions(+), 2 deletions(-)\n>\n> diff --git a/sparse-index.c b/sparse-index.c\n> index 5d14795063b..76f90da5f5f 100644\n> --- a/sparse-index.c\n> +++ b/sparse-index.c\n> @@ -32,8 +32,9 @@ int give_advice_on_expansion = 1;\n> \"Your working directory likely has contents that are outside of\\n\" \\\n> \"your sparse-checkout patterns. Use 'git sparse-checkout list' to\\n\" \\\n> \"see your sparse-checkout definition and compare it to your working\\n\" \\\n> - \"directory contents. Running 'git sparse-checkout clean' may assist\\n\" \\\n> - \"in this cleanup.\"\n> + \"directory contents. Cleaning up any merge conflicts or staged\\n\" \\\n> + \"changes before running 'git sparse-checkout clean' or 'git\\n\" \\\n> + \"sparse-checkout reapply' may assist in this cleanup.\"\n\nI like the message change; thanks for sending this in!\n\n>\n> struct modify_index_context {\n> struct index_state *write;\n> --\n> 2.47.0.vfs.0.3\n"}]}