git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 4/6] read-cache: set sparsity when index is new

From
Victoria Dye via GitGitGadget <gitgitgadget@gmail.com>
Date
May 10, 2022, 23:32 UTC
Message-ID
<bb092c075f499acd5aa86debee0a23fa66ac523d.1652225552.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1171.v3.git.1652225552.gitgitgadget@gmail.com>
From: Victoria Dye <vdye@github.com>

When the index read in 'do_read_index()' does not exist on-disk, mark the index "sparse" if the executing command does not require a full index and sparse index is otherwise enabled.

Some commands (such as 'git stash -u') implicitly create a new index (when the 'GIT_INDEX_FILE' variable points to a non-existent file) and perform some operation on it. However, when this index is created, it isn't created with the same sparsity settings as the repo index. As a result, while these indexes may be sparse during the operation, they are always expanded before being written to disk. We can avoid that expansion by defaulting the index to "sparse", in which case it will only be expanded if the full index is needed.

Note that the function 'set_new_index_sparsity()' is created despite having only a single caller because additional callers will be added in a subsequent patch.

Signed-off-by: Victoria Dye <vdye@github.com>
---
 read-cache.c                             | 18 +++++++++++++++++-
 t/t1092-sparse-checkout-compatibility.sh |  2 +-
 2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 4df97e185e9..60355f5ad6a 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2260,6 +2260,20 @@ static unsigned long load_cache_entries_threaded(struct index_state *istate, con
 	return consumed;
 }
 
+static void set_new_index_sparsity(struct index_state *istate)
+{
+	/*
+	 * If the index's repo exists, mark it sparse according to
+	 * repo settings.
+	 */
+	if (istate->repo) {
+		prepare_repo_settings(istate->repo);
+		if (!istate->repo->settings.command_requires_full_index &&
+		    is_sparse_index_allowed(istate, 0))
+			istate->sparse_index = 1;
+	}
+}
+
 /* remember to discard_cache() before reading a different cache! */
 int do_read_index(struct index_state *istate, const char *path, int must_exist)
 {
@@ -2281,8 +2295,10 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 	istate->timestamp.nsec = 0;
 	fd = open(path, O_RDONLY);
 	if (fd < 0) {
-		if (!must_exist && errno == ENOENT)
+		if (!must_exist && errno == ENOENT) {
+			set_new_index_sparsity(istate);
 			return 0;
+		}
 		die_errno(_("%s: index file open failed"), path);
 	}
 
diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
index 75d844cd71d..85c6a56f1b7 100755
--- a/t/t1092-sparse-checkout-compatibility.sh
+++ b/t/t1092-sparse-checkout-compatibility.sh
@@ -1389,7 +1389,7 @@ test_expect_success 'sparse-index is not expanded: stash' '
 	ensure_not_expanded stash drop stash@{0} &&
 
 	echo >>sparse-index/deep/new &&
-	! ensure_not_expanded stash -u &&
+	ensure_not_expanded stash -u &&
 	(
 		WITHOUT_UNTRACKED_TXT=1 &&
 		! ensure_not_expanded stash pop
-- 
gitgitgadget
Previous: Victoria Dye via GitGitGadgetNext: Victoria Dye via GitGitGadget
Message 35 of 42 in “Sparse index: integrate with 'git stash'”
  1. 0/7 Sparse index: integrate with 'git stash'Victoria Dye via GitGitGadget, Apr 25, 2022
  2. 1/7 stash: expand sparse-checkout compatibility testingVictoria Dye via GitGitGadget, Apr 25, 2022
  3. 3/7 sparse-index: expose 'is_sparse_index_allowed()'Victoria Dye via GitGitGadget, Apr 25, 2022
  4. 7/7 unpack-trees: preserve index sparsityVictoria Dye via GitGitGadget, Apr 25, 2022
  5. 6/7 stash: merge applied stash with merge-ortVictoria Dye via GitGitGadget, Apr 25, 2022
  6. Derrick StoleeApr 26, 2022
  7. 4/7 read-cache: set sparsity when index is newVictoria Dye via GitGitGadget, Apr 25, 2022
  8. Junio C HamanoApr 25, 2022
  9. 2/7 stash: integrate with sparse indexVictoria Dye via GitGitGadget, Apr 25, 2022
  10. Junio C HamanoApr 25, 2022
  11. Derrick StoleeApr 26, 2022
  12. Victoria DyeApr 26, 2022
  13. Junio C HamanoApr 26, 2022
  14. 5/7 merge-recursive: add merge function arg to 'merge_recursive_generic'Victoria Dye via GitGitGadget, Apr 25, 2022
  15. Junio C HamanoApr 25, 2022
  16. Derrick StoleeApr 26, 2022
  17. Derrick StoleeApr 26, 2022
  18. Derrick StoleeApr 26, 2022
  19. 0/7 Sparse index: integrate with 'git stash'Victoria Dye via GitGitGadget, Apr 27, 2022
  20. 1/7 stash: expand sparse-checkout compatibility testingVictoria Dye via GitGitGadget, Apr 27, 2022
  21. 2/7 stash: integrate with sparse indexVictoria Dye via GitGitGadget, Apr 27, 2022
  22. 3/7 sparse-index: expose 'is_sparse_index_allowed()'Victoria Dye via GitGitGadget, Apr 27, 2022
  23. 4/7 read-cache: set sparsity when index is newVictoria Dye via GitGitGadget, Apr 27, 2022
  24. 6/7 stash: merge applied stash with merge-ortVictoria Dye via GitGitGadget, Apr 27, 2022
  25. 7/7 unpack-trees: preserve index sparsityVictoria Dye via GitGitGadget, Apr 27, 2022
  26. 5/7 merge-recursive: add merge function arg to 'merge_recursive_generic'Victoria Dye via GitGitGadget, Apr 27, 2022
  27. Elijah NewrenMay 6, 2022
  28. Victoria DyeMay 9, 2022
  29. Elijah NewrenMay 10, 2022
  30. Elijah NewrenMay 6, 2022
  31. 0/6 Sparse index: integrate with 'git stash'Victoria Dye via GitGitGadget, May 10, 2022
  32. 1/6 stash: expand sparse-checkout compatibility testingVictoria Dye via GitGitGadget, May 10, 2022
  33. 2/6 stash: integrate with sparse indexVictoria Dye via GitGitGadget, May 10, 2022
  34. 3/6 sparse-index: expose 'is_sparse_index_allowed()'Victoria Dye via GitGitGadget, May 10, 2022
  35. 4/6 read-cache: set sparsity when index is newVictoria Dye via GitGitGadget, May 10, 2022
  36. 5/6 stash: apply stash using 'merge_ort_nonrecursive()'Victoria Dye via GitGitGadget, May 10, 2022
  37. Junio C HamanoMay 11, 2022
  38. Jonathan TanMay 12, 2022
  39. Elijah NewrenMay 12, 2022
  40. Jonathan TanMay 12, 2022
  41. Elijah NewrenMay 12, 2022
  42. 6/6 unpack-trees: preserve index sparsityVictoria Dye via GitGitGadget, May 10, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.