git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Provide config option to expect files outside sparse patterns

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 20, 2022, 05:05 UTC
Message-ID
<pull.1153.git.1645333542011.gitgitgadget@gmail.com>
From: Elijah Newren <newren@gmail.com>

Typically with sparse checkouts, we expect files outside the sparsity patterns to be marked as SKIP_WORKTREE and be missing from the working tree. VFS for Git can be used to turn this expectation on its head: all files are considered present in the working copy, though they are not vivified until first access access. With VFS for Git, most of the files do not match the sparsity patterns at first, and the VFS layer automatically updates the sparsity patterns to add more files whenever files are written.

With this background, this special usecase does not play well with the safety check we added in commit 11d46a399d ("repo_read_index: clear SKIP_WORKTREE bit from files present in worktree", 2022-01-06). Checking SKIP_WORKTREE files to see if they are present in the working tree causes them all to be immediately vivified. Further, the special VFS layer, by virtue of automatically updating the sparsity patterns and catching all accesses, isn't in need of that safety check either. Provide a configuration option, core.expectFilesOutsideSparsePatterns so that those with this special usecase can turn off the safety check.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
    Provide config option to expect files outside sparse patterns
    
    Builds on en/present-despite-skipped, and addresses issue reported at
    https://lore.kernel.org/git/YhBCsg2DCEd9FXjE@google.com/
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1153%2Fnewren%2Fpresent-despite-skip-worktree-toggle-for-vfs-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1153/newren/present-despite-skip-worktree-toggle-for-vfs-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1153
 cache.h                          |  1 +
 config.c                         |  5 +++++
 environment.c                    |  1 +
 sparse-index.c                   |  3 ++-
 t/t1090-sparse-checkout-scope.sh | 19 +++++++++++++++++++
 5 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/cache.h b/cache.h
index 281f00ab1b1..1f35d8a59ea 100644
--- a/cache.h
+++ b/cache.h
@@ -1003,6 +1003,7 @@ extern const char *core_fsmonitor;
 
 extern int core_apply_sparse_checkout;
 extern int core_sparse_checkout_cone;
+extern int core_expect_files_outside_sparse_patterns;
 
 /*
  * Returns the boolean value of $GIT_OPTIONAL_LOCKS (or the default value).
diff --git a/config.c b/config.c
index 2bffa8d4a01..68e877a1d80 100644
--- a/config.c
+++ b/config.c
@@ -1520,6 +1520,11 @@ static int git_default_core_config(const char *var, const char *value, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(var, "core.expectfilesoutsidesparsepatterns")) {
+		core_expect_files_outside_sparse_patterns = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "core.precomposeunicode")) {
 		precomposed_unicode = git_config_bool(var, value);
 		return 0;
diff --git a/environment.c b/environment.c
index fd0501e77a5..c86ebe9ce74 100644
--- a/environment.c
+++ b/environment.c
@@ -70,6 +70,7 @@ char *notes_ref_name;
 int grafts_replace_parents = 1;
 int core_apply_sparse_checkout;
 int core_sparse_checkout_cone;
+int core_expect_files_outside_sparse_patterns;
 int merge_log_config = -1;
 int precomposed_unicode = -1; /* see probe_utf8_pathname_composition() */
 unsigned long pack_size_limit_cfg;
diff --git a/sparse-index.c b/sparse-index.c
index eed170cd8f7..8e2a3973856 100644
--- a/sparse-index.c
+++ b/sparse-index.c
@@ -396,7 +396,8 @@ void clear_skip_worktree_from_present_files(struct index_state *istate)
 
 	int i;
 
-	if (!core_apply_sparse_checkout)
+	if (!core_apply_sparse_checkout ||
+	    core_expect_files_outside_sparse_patterns)
 		return;
 
 restart:
diff --git a/t/t1090-sparse-checkout-scope.sh b/t/t1090-sparse-checkout-scope.sh
index 3deb4901874..feef883523b 100755
--- a/t/t1090-sparse-checkout-scope.sh
+++ b/t/t1090-sparse-checkout-scope.sh
@@ -84,4 +84,23 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs
 	test_cmp expect actual
 '
 
+test_expect_success 'skip-worktree on files outside sparse patterns' '
+	git sparse-checkout disable &&
+	git sparse-checkout set --no-cone "a*" &&
+	git checkout-index --all --ignore-skip-worktree-bits &&
+
+	git ls-files -t >output &&
+	! grep ^S output >actual &&
+	test_must_be_empty actual &&
+
+	test_config core.expectFilesOutsideSparsePatterns true &&
+	cat <<-\EOF >expect &&
+	S b
+	S c
+	EOF
+	git ls-files -t >output &&
+	grep ^S output >actual &&
+	test_cmp expect actual
+'
+
 test_done

base-commit: d79d29935262f8422a6e620bd33cc2fce28e669d

-- 
gitgitgadget
Next: Derrick Stolee
Message 1 of 29 in “Provide config option to expect files outside sparse patterns”
  1. Provide config option to expect files outside sparse patternsElijah Newren via GitGitGadget, Feb 20, 2022
  2. Derrick StoleeFeb 20, 2022
  3. Junio C HamanoFeb 20, 2022
  4. Elijah NewrenFeb 22, 2022
  5. Johannes SchindelinFeb 22, 2022
  6. Derrick StoleeFeb 22, 2022
  7. Johannes SchindelinFeb 21, 2022
  8. Ævar Arnfjörð BjarmasonFeb 21, 2022
  9. Elijah NewrenFeb 22, 2022
  10. Johannes SchindelinFeb 22, 2022
  11. Ævar Arnfjörð BjarmasonFeb 22, 2022
  12. Jonathan NiederFeb 22, 2022
  13. Ævar Arnfjörð BjarmasonFeb 25, 2022
  14. Elijah NewrenFeb 22, 2022
  15. Ævar Arnfjörð BjarmasonFeb 22, 2022
  16. Johannes SchindelinFeb 22, 2022
  17. Derrick StoleeFeb 22, 2022
  18. repo_read_index: add config to expect files outside sparse patternsJonathan Nieder, Feb 23, 2022
  19. Elijah NewrenFeb 23, 2022
  20. repo_read_index: add config to expect files outside sparse patternsElijah Newren, Feb 24, 2022
  21. Junio C HamanoFeb 24, 2022
  22. Elijah NewrenFeb 26, 2022
  23. Jonathan NiederFeb 25, 2022
  24. Elijah NewrenFeb 26, 2022
  25. repo_read_index: add config to expect files outside sparse patternsElijah Newren, Feb 26, 2022
  26. repo_read_index: add config to expect files outside sparse patternsElijah Newren, Mar 2, 2022
  27. Junio C HamanoMar 2, 2022
  28. Elijah NewrenMar 2, 2022
  29. Derrick StoleeMar 2, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.