{"thread":{"id":"58202","subject":"[PATCH] repository:new config to only check files matched sparse-checkout patterns.","startedAt":"2022-07-21T09:14:22Z","lastAt":"2022-07-21T09:14:22Z","messageCount":1,"participants":["v via GitGitGadget"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"459605","messageId":"pull.1295.git.1658394854464.gitgitgadget@gmail.com","threadId":"58202","inReplyTo":null,"subject":"[PATCH] repository:new config to only check files matched sparse-checkout patterns.","fromName":"v via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-07-21T09:14:14Z","receivedAt":"2022-07-21T09:14:22Z","isPatch":true,"sender":{"key":"name:v","avatar":null},"body":"From: Canwei He <fvoidcn@gmail.com>\n\nBy Default with sparse checkouts, all files with SKIP_WORKTREE bit\nin the index must check whether exists in the worktree. And Git will\nexpand all of that files in the index and must traverse recursively\nin the worktree. This option can be used to tell Git that just check\nfiles which can match the sparse-checkout patterns.\n\nIt is useful when you are using a visual file system in order to\non-demand loading working tree lazily. It can control which directory\nshould load by using sparse-checkout patterns. But it doesn't want to\nvisit other directory which haven't been load but just have an empty\ndirectory for the entry of the file loading switch. In default model,\nGit would traverse root direcotry recursively however all files would\nbe loaded, this behaviour would very expensive. With\n`sparse.onlyCheckFilesMatchPatterns` model, Git would just check the\npath witch only matched the sparse-checkout patterns to prevent loading\nall the directory.\n\nSigned-off-by: Canwei He <fvoidcn@gmail.com>\n---\n    repository:new config to only check files matched sparse-checkout\n    patterns.\n    \n    repository:new config to only check files matched sparse-checkout\n    patterns.\n    \n    By Default with sparse checkouts, all files with SKIP_WORKTREE bit in\n    the index must check whether exists in the worktree. And Git will expand\n    all of that files in the index and must traverse recursively in the\n    worktree. This option can be used to tell Git that just check files\n    which can match the sparse-checkout patterns.\n    \n    It is useful when you are using a visual file system in order to\n    on-demand loading working tree lazily. It can control which directory\n    should load by using sparse-checkout patterns. But it doesn't want to\n    visit other directory which haven't been load but just have an empty\n    directory for the entry of the file loading switch. In default model,\n    Git would traverse root direcotry recursively however all files would be\n    loaded, this behaviour would very expensive. With\n    sparse.onlyCheckFilesMatchPatterns model, Git would just check the path\n    witch only matched the sparse-checkout patterns to prevent loading all\n    the directory.\n    \n    Signed-off-by: Canwei He fvoidcn@gmail.com\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1295%2FfvoidCN%2Fvoidhe-upstream-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1295/fvoidCN/voidhe-upstream-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1295\n\n Documentation/config/sparse.txt  | 25 ++++++++++++++++\n cache.h                          |  1 +\n config.c                         |  5 ++++\n environment.c                    |  1 +\n sparse-index.c                   | 19 ++++++++----\n t/t1090-sparse-checkout-scope.sh | 50 ++++++++++++++++++++++++++++++++\n 6 files changed, 95 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/config/sparse.txt b/Documentation/config/sparse.txt\nindex aff49a8d3a3..f71716f446a 100644\n--- a/Documentation/config/sparse.txt\n+++ b/Documentation/config/sparse.txt\n@@ -25,3 +25,28 @@ Regardless of this setting, Git does not check for\n present-despite-skipped files unless sparse checkout is enabled, so\n this config option has no effect unless `core.sparseCheckout` is\n `true`.\n+\n+sparse.onlyCheckFilesMatchPatterns::\n+    By Default with sparse checkouts, all files with SKIP_WORKTREE bit\n+    in the index must check whether exists in the worktree. And Git will\n+    expand all of that files in the index and must traverse recursively\n+    in the worktree. This option can be used to tell Git that just check\n+    files which can match the sparse-checkout patterns.\n++\n+The default is `false`, which allows Git to automatically recover\n+from the list of files in the index and working tree falling out of\n+sync.\n++\n+Set this to `true` if you just hope Git maintain the consistency\n+between the presence of working tree files and sparsity patterns\n+confining to which match the sparse-checkout patterns.For example,\n+if you are using a visual file system in order to on-demand loading\n+working tree lazily. It can control which directory should load by\n+using sparse-checkout patterns. But it doesn't want to visit other\n+directory which haven't been load but just have an empty directory\n+for the entry of the file loading switch.\n++\n+Regardless of this setting, Git does not check for\n+present-despite-skipped files unless sparse checkout is enabled, so\n+this config option has no effect unless `core.sparseCheckout` is\n+`true`.\n\\ No newline at end of file\ndiff --git a/cache.h b/cache.h\nindex ac5ab4ef9d3..d1ed717403c 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1074,6 +1074,7 @@ extern int protect_ntfs;\n extern int core_apply_sparse_checkout;\n extern int core_sparse_checkout_cone;\n extern int sparse_expect_files_outside_of_patterns;\n+extern int sparse_only_check_files_match_patterns;\n \n /*\n  * Returns the boolean value of $GIT_OPTIONAL_LOCKS (or the default value).\ndiff --git a/config.c b/config.c\nindex 9b0e9c93285..5ab9719479a 100644\n--- a/config.c\n+++ b/config.c\n@@ -1758,6 +1758,11 @@ static int git_default_sparse_config(const char *var, const char *value)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(var, \"sparse.onlycheckfilesmatchpatterns\")) {\n+\t\tsparse_only_check_files_match_patterns = git_config_bool(var, value);\n+\t\treturn 0;\n+\t}\n+\n \t/* Add other config variables here and to Documentation/config/sparse.txt. */\n \treturn 0;\n }\ndiff --git a/environment.c b/environment.c\nindex b3296ce7d15..0ee4190c7dc 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -73,6 +73,7 @@ int grafts_replace_parents = 1;\n int core_apply_sparse_checkout;\n int core_sparse_checkout_cone;\n int sparse_expect_files_outside_of_patterns;\n+int sparse_only_check_files_match_patterns;\n int merge_log_config = -1;\n int precomposed_unicode = -1; /* see probe_utf8_pathname_composition() */\n unsigned long pack_size_limit_cfg;\ndiff --git a/sparse-index.c b/sparse-index.c\nindex e4a54ce1943..264f4965e24 100644\n--- a/sparse-index.c\n+++ b/sparse-index.c\n@@ -502,14 +502,21 @@ restart:\n \tfor (i = 0; i < istate->cache_nr; i++) {\n \t\tstruct cache_entry *ce = istate->cache[i];\n \n-\t\tif (ce_skip_worktree(ce) &&\n-\t\t    path_found(ce->name, &last_dirname, &dir_len, &dir_found)) {\n-\t\t\tif (S_ISSPARSEDIR(ce->ce_mode)) {\n-\t\t\t\tensure_full_index(istate);\n-\t\t\t\tgoto restart;\n+\t\tif (ce_skip_worktree(ce)) {\n+\t\t\tif (sparse_only_check_files_match_patterns &&\n+\t\t\t    !path_in_sparse_checkout(ce->name, istate)){\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tif (path_found(ce->name, &last_dirname, &dir_len, &dir_found)) {\n+\t\t\t\tif (S_ISSPARSEDIR(ce->ce_mode)) {\n+\t\t\t\t\tensure_full_index(istate);\n+\t\t\t\t\tgoto restart;\n+\t\t\t\t}\n+\t\t\t\tce->ce_flags &= ~CE_SKIP_WORKTREE;\n \t\t\t}\n-\t\t\tce->ce_flags &= ~CE_SKIP_WORKTREE;\n \t\t}\n+\n \t}\n }\n \ndiff --git a/t/t1090-sparse-checkout-scope.sh b/t/t1090-sparse-checkout-scope.sh\nindex d1833c0f31b..da8173ebe5d 100755\n--- a/t/t1090-sparse-checkout-scope.sh\n+++ b/t/t1090-sparse-checkout-scope.sh\n@@ -71,6 +71,56 @@ test_expect_success 'skip-worktree on files outside sparse patterns' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'skip-worktree on files only matched patterns. default' '\n+\tgit clean -f . &&\n+\tgit update-index  --refresh &&\n+\tgit sparse-checkout disable &&\n+\tgit sparse-checkout set --no-cone \"a*\" &&\n+\ttest_config sparse.expectFilesOutsideOfPatterns false &&\n+\tgit checkout-index --all --ignore-skip-worktree-bits &&\n+\n+\tgit ls-files -t >output &&\n+\t! grep ^S output >actual &&\n+\ttest_must_be_empty actual\n+'\n+\n+test_expect_success 'skip-worktree on files only matched patterns. b,c not in patterns' '\n+\tgit clean -f . &&\n+\tgit update-index  --refresh &&\n+\tgit sparse-checkout disable &&\n+\tgit sparse-checkout set --no-cone \"a*\" &&\n+\ttest_config sparse.onlyCheckFilesMatchPatterns true &&\n+\tgit checkout-index --all --ignore-skip-worktree-bits &&\n+\n+\tgit ls-files -t >output &&\n+\tcat <<-\\EOF >expect &&\n+\tH a\n+\tS b\n+\tS c\n+\tEOF\n+\ttest_cmp expect output >>actual &&\n+\ttest_must_be_empty actual\n+'\n+\n+test_expect_success 'skip-worktree on files only matched patterns. file b in patterns c is not' '\n+\tgit clean -f . &&\n+\tgit update-index  --refresh &&\n+\tgit sparse-checkout disable &&\n+\tgit sparse-checkout set --no-cone \"a*\" &&\n+\ttest_config sparse.onlyCheckFilesMatchPatterns true &&\n+\techo \"b*\" >>.git/info/sparse-checkout &&\n+\tgit checkout-index --all --ignore-skip-worktree-bits &&\n+\n+\tgit ls-files -t >output &&\n+\tcat <<-\\EOF >expect &&\n+\tH a\n+\tH b\n+\tS c\n+\tEOF\n+\ttest_cmp expect output >>actual &&\n+\ttest_must_be_empty actual\n+'\n+\n test_expect_success 'in partial clone, sparse checkout only fetches needed blobs' '\n \ttest_create_repo server &&\n \tgit clone \"file://$(pwd)/server\" client &&\n\nbase-commit: 9dd64cb4d310986dd7b8ca7fff92f9b61e0bd21a\n-- \ngitgitgadget\n"}]}