git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 18/20] cache-tree: integrate with sparse directory entries

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 16, 2021, 16:43 UTC
Message-ID
<c2d0c17db31a283ff9f9a137c4d497d878f13458.1615912983.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.883.v3.git.1615912983.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

The cache-tree extension was previously disabled with sparse indexes. However, the cache-tree is an important performance feature for commands like 'git status' and 'git add'. Integrate it with sparse directory entries.

When writing a sparse index, completely clear and recalculate the cache tree. By starting from scratch, the only integration necessary is to check if we hit a sparse directory entry and create a leaf of the cache-tree that has an entry_count of one and no subtrees.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 cache-tree.c   | 18 ++++++++++++++++++
 sparse-index.c | 10 +++++++++-
 2 files changed, 27 insertions(+), 1 deletion(-)
diff --git a/cache-tree.c b/cache-tree.c
index 5f07a39e501e..950a9615db8f 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -256,6 +256,24 @@ static int update_one(struct cache_tree *it,
 
 	*skip_count = 0;
 
+	/*
+	 * If the first entry of this region is a sparse directory
+	 * entry corresponding exactly to 'base', then this cache_tree
+	 * struct is a "leaf" in the data structure, pointing to the
+	 * tree OID specified in the entry.
+	 */
+	if (entries > 0) {
+		const struct cache_entry *ce = cache[0];
+
+		if (S_ISSPARSEDIR(ce->ce_mode) &&
+		    ce->ce_namelen == baselen &&
+		    !strncmp(ce->name, base, baselen)) {
+			it->entry_count = 1;
+			oidcpy(&it->oid, &ce->oid);
+			return 1;
+		}
+	}
+
 	if (0 <= it->entry_count && has_object_file(&it->oid))
 		return it->entry_count;
 
diff --git a/sparse-index.c b/sparse-index.c
index 30c1a11fd62d..56313e805d9d 100644
--- a/sparse-index.c
+++ b/sparse-index.c
@@ -180,7 +180,11 @@ int convert_to_sparse(struct index_state *istate)
 	istate->cache_nr = convert_to_sparse_rec(istate,
 						 0, 0, istate->cache_nr,
 						 "", 0, istate->cache_tree);
-	istate->drop_cache_tree = 1;
+
+	/* Clear and recompute the cache-tree */
+	cache_tree_free(&istate->cache_tree);
+	cache_tree_update(istate, 0);
+
 	istate->sparse_index = 1;
 	trace2_region_leave("index", "convert_to_sparse", istate->repo);
 	return 0;
@@ -281,5 +285,9 @@ void ensure_full_index(struct index_state *istate)
 	strbuf_release(&base);
 	free(full);
 
+	/* Clear and recompute the cache-tree */
+	cache_tree_free(&istate->cache_tree);
+	cache_tree_update(istate, 0);
+
 	trace2_region_leave("index", "ensure_full_index", istate->repo);
 }
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 134 of 203 in “Sparse Index: Design, Format, Tests”
  1. 00/20 Sparse Index: Design, Format, TestsDerrick Stolee via GitGitGadget, Feb 23, 2021
  2. 01/20 sparse-index: design doc and format updateDerrick Stolee via GitGitGadget, Feb 23, 2021
  3. Elijah NewrenFeb 24, 2021
  4. Derrick StoleeFeb 25, 2021
  5. Elijah NewrenFeb 25, 2021
  6. 03/20 t1092: clean up script quotingDerrick Stolee via GitGitGadget, Feb 23, 2021
  7. 02/20 t/perf: add performance test for sparse operationsDerrick Stolee via GitGitGadget, Feb 23, 2021
  8. Elijah NewrenFeb 24, 2021
  9. Derrick StoleeMar 9, 2021
  10. 04/20 sparse-index: add guard to ensure full indexDerrick Stolee via GitGitGadget, Feb 23, 2021
  11. Elijah NewrenFeb 24, 2021
  12. 05/20 sparse-index: implement ensure_full_index()Derrick Stolee via GitGitGadget, Feb 23, 2021
  13. Elijah NewrenFeb 24, 2021
  14. 14/20 sparse-index: check index conversion happensDerrick Stolee via GitGitGadget, Feb 23, 2021
  15. 16/20 sparse-checkout: toggle sparse index from builtinDerrick Stolee via GitGitGadget, Feb 23, 2021
  16. Martin ÅgrenFeb 24, 2021
  17. Derrick StoleeMar 9, 2021
  18. Elijah NewrenMar 9, 2021
  19. Derrick StoleeMar 9, 2021
  20. Elijah NewrenMar 9, 2021
  21. Martin ÅgrenMar 14, 2021
  22. Derrick StoleeMar 15, 2021
  23. 07/20 test-read-cache: print cache entries with --tableDerrick Stolee via GitGitGadget, Feb 23, 2021
  24. Elijah NewrenFeb 25, 2021
  25. Derrick StoleeMar 9, 2021
  26. 06/20 t1092: compare sparse-checkout to sparse-indexDerrick Stolee via GitGitGadget, Feb 23, 2021
  27. Elijah NewrenFeb 25, 2021
  28. 11/20 sparse-index: convert from full to sparseDerrick Stolee via GitGitGadget, Feb 23, 2021
  29. Elijah NewrenFeb 25, 2021
  30. Derrick StoleeMar 9, 2021
  31. 10/20 sparse-checkout: hold pattern list in indexDerrick Stolee via GitGitGadget, Feb 23, 2021
  32. Elijah NewrenFeb 25, 2021
  33. 09/20 unpack-trees: ensure full indexDerrick Stolee via GitGitGadget, Feb 23, 2021
  34. 08/20 test-tool: don't force full indexDerrick Stolee via GitGitGadget, Feb 23, 2021
  35. 13/20 unpack-trees: allow sparse directoriesDerrick Stolee via GitGitGadget, Feb 23, 2021
  36. Elijah NewrenFeb 25, 2021
  37. Derrick StoleeMar 9, 2021
  38. Elijah NewrenMar 9, 2021
  39. 12/20 submodule: sparse-index should not collapse linksDerrick Stolee via GitGitGadget, Feb 23, 2021
  40. 15/20 sparse-index: create extension for compatibilityDerrick Stolee via GitGitGadget, Feb 23, 2021
  41. Elijah NewrenFeb 25, 2021
  42. Derrick StoleeMar 9, 2021
  43. 17/20 sparse-checkout: disable sparse-indexDerrick Stolee via GitGitGadget, Feb 23, 2021
  44. SZEDER GáborFeb 27, 2021
  45. Derrick StoleeMar 9, 2021
  46. Derrick StoleeMar 10, 2021
  47. 18/20 cache-tree: integrate with sparse directory entriesDerrick Stolee via GitGitGadget, Feb 23, 2021
  48. 20/20 p2000: add sparse-index reposDerrick Stolee via GitGitGadget, Feb 23, 2021
  49. 19/20 sparse-index: loose integration with cache_tree_verify()Derrick Stolee via GitGitGadget, Feb 23, 2021
  50. Elijah NewrenFeb 23, 2021
  51. Elijah NewrenFeb 26, 2021
  52. 00/20 Sparse Index: Design, Format, TestsDerrick Stolee via GitGitGadget, Mar 10, 2021
  53. 01/20 sparse-index: design doc and format updateDerrick Stolee via GitGitGadget, Mar 10, 2021
  54. Elijah NewrenMar 10, 2021
  55. 02/20 t/perf: add performance test for sparse operationsDerrick Stolee via GitGitGadget, Mar 10, 2021
  56. 05/20 sparse-index: implement ensure_full_index()Derrick Stolee via GitGitGadget, Mar 10, 2021
  57. Junio C HamanoMar 12, 2021
  58. Derrick StoleeMar 12, 2021
  59. Junio C HamanoMar 12, 2021
  60. Derrick StoleeMar 12, 2021
  61. Ævar Arnfjörð BjarmasonMar 15, 2021
  62. 04/20 sparse-index: add guard to ensure full indexDerrick Stolee via GitGitGadget, Mar 10, 2021
  63. 03/20 t1092: clean up script quotingDerrick Stolee via GitGitGadget, Mar 10, 2021
  64. 06/20 t1092: compare sparse-checkout to sparse-indexDerrick Stolee via GitGitGadget, Mar 10, 2021
  65. Elijah NewrenMar 10, 2021
  66. Derrick StoleeMar 11, 2021
  67. 07/20 test-read-cache: print cache entries with --tableDerrick Stolee via GitGitGadget, Mar 10, 2021
  68. 14/20 sparse-index: check index conversion happensDerrick Stolee via GitGitGadget, Mar 10, 2021
  69. 08/20 test-tool: don't force full indexDerrick Stolee via GitGitGadget, Mar 10, 2021
  70. 12/20 submodule: sparse-index should not collapse linksDerrick Stolee via GitGitGadget, Mar 10, 2021
  71. 13/20 unpack-trees: allow sparse directoriesDerrick Stolee via GitGitGadget, Mar 10, 2021
  72. 11/20 sparse-index: convert from full to sparseDerrick Stolee via GitGitGadget, Mar 10, 2021
  73. Elijah NewrenMar 10, 2021
  74. Derrick StoleeMar 11, 2021
  75. 09/20 unpack-trees: ensure full indexDerrick Stolee via GitGitGadget, Mar 10, 2021
  76. 15/20 sparse-index: create extension for compatibilityDerrick Stolee via GitGitGadget, Mar 10, 2021
  77. 19/20 sparse-index: loose integration with cache_tree_verify()Derrick Stolee via GitGitGadget, Mar 10, 2021
  78. 10/20 sparse-checkout: hold pattern list in indexDerrick Stolee via GitGitGadget, Mar 10, 2021
  79. 17/20 sparse-checkout: disable sparse-indexDerrick Stolee via GitGitGadget, Mar 10, 2021
  80. 18/20 cache-tree: integrate with sparse directory entriesDerrick Stolee via GitGitGadget, Mar 10, 2021
  81. 20/20 p2000: add sparse-index reposDerrick Stolee via GitGitGadget, Mar 10, 2021
  82. 16/20 sparse-checkout: toggle sparse index from builtinDerrick Stolee via GitGitGadget, Mar 10, 2021
  83. Elijah NewrenMar 11, 2021
  84. 00/20 Sparse Index: Design, Format, TestsDerrick Stolee via GitGitGadget, Mar 16, 2021
  85. 01/20 sparse-index: design doc and format updateDerrick Stolee via GitGitGadget, Mar 16, 2021
  86. Junio C HamanoMar 19, 2021
  87. Derrick StoleeMar 23, 2021
  88. Junio C HamanoMar 23, 2021
  89. Derrick StoleeMar 23, 2021
  90. 03/20 t1092: clean up script quotingDerrick Stolee via GitGitGadget, Mar 16, 2021
  91. Ævar Arnfjörð BjarmasonMar 17, 2021
  92. 02/20 t/perf: add performance test for sparse operationsDerrick Stolee via GitGitGadget, Mar 16, 2021
  93. Ævar Arnfjörð BjarmasonMar 17, 2021
  94. Derrick StoleeMar 17, 2021
  95. Ævar Arnfjörð BjarmasonMar 17, 2021
  96. Derrick StoleeMar 17, 2021
  97. 08/20 test-tool: don't force full indexDerrick Stolee via GitGitGadget, Mar 16, 2021
  98. 06/20 t1092: compare sparse-checkout to sparse-indexDerrick Stolee via GitGitGadget, Mar 16, 2021
  99. 04/20 sparse-index: add guard to ensure full indexDerrick Stolee via GitGitGadget, Mar 16, 2021
  100. 09/20 unpack-trees: ensure full indexDerrick Stolee via GitGitGadget, Mar 16, 2021
  101. 07/20 test-read-cache: print cache entries with --tableDerrick Stolee via GitGitGadget, Mar 16, 2021
  102. 1/5 ls-files: defer read_index() after parse_options() etc.Ævar Arnfjörð Bjarmason, Mar 17, 2021
  103. 2/5 ls-files: make "mode" in show_ce() loop a variableÆvar Arnfjörð Bjarmason, Mar 17, 2021
  104. Elijah NewrenMar 17, 2021
  105. Ævar Arnfjörð BjarmasonMar 24, 2021
  106. 3/5 ls-files: add and use a new --sparse optionÆvar Arnfjörð Bjarmason, Mar 17, 2021
  107. Elijah NewrenMar 17, 2021
  108. Ævar Arnfjörð BjarmasonMar 17, 2021
  109. Elijah NewrenMar 17, 2021
  110. Derrick StoleeMar 17, 2021
  111. Ævar Arnfjörð BjarmasonMar 24, 2021
  112. 4/5 test-tool read-cache: --table is redundant to ls-filesÆvar Arnfjörð Bjarmason, Mar 17, 2021
  113. 0/5 Re: [PATCH v3 07/20] test-read-cache: print cache entries with --tableÆvar Arnfjörð Bjarmason, Mar 17, 2021
  114. Elijah NewrenMar 17, 2021
  115. Derrick StoleeMar 17, 2021
  116. Elijah NewrenMar 17, 2021
  117. Derrick StoleeMar 17, 2021
  118. 5/5 test-tool: split up test-tool read-cacheÆvar Arnfjörð Bjarmason, Mar 17, 2021
  119. Ævar Arnfjörð BjarmasonMar 17, 2021
  120. 05/20 sparse-index: implement ensure_full_index()Derrick Stolee via GitGitGadget, Mar 16, 2021
  121. Ævar Arnfjörð BjarmasonMar 17, 2021
  122. 13/20 unpack-trees: allow sparse directoriesDerrick Stolee via GitGitGadget, Mar 16, 2021
  123. Ævar Arnfjörð BjarmasonMar 17, 2021
  124. 12/20 submodule: sparse-index should not collapse linksDerrick Stolee via GitGitGadget, Mar 16, 2021
  125. 10/20 sparse-checkout: hold pattern list in indexDerrick Stolee via GitGitGadget, Mar 16, 2021
  126. 11/20 sparse-index: convert from full to sparseDerrick Stolee via GitGitGadget, Mar 16, 2021
  127. Ævar Arnfjörð BjarmasonMar 17, 2021
  128. Derrick StoleeMar 17, 2021
  129. Derrick StoleeMar 18, 2021
  130. 14/20 sparse-index: check index conversion happensDerrick Stolee via GitGitGadget, Mar 16, 2021
  131. 15/20 sparse-index: create extension for compatibilityDerrick Stolee via GitGitGadget, Mar 16, 2021
  132. 17/20 sparse-checkout: disable sparse-indexDerrick Stolee via GitGitGadget, Mar 16, 2021
  133. 16/20 sparse-checkout: toggle sparse index from builtinDerrick Stolee via GitGitGadget, Mar 16, 2021
  134. 18/20 cache-tree: integrate with sparse directory entriesDerrick Stolee via GitGitGadget, Mar 16, 2021
  135. 19/20 sparse-index: loose integration with cache_tree_verify()Derrick Stolee via GitGitGadget, Mar 16, 2021
  136. 20/20 p2000: add sparse-index reposDerrick Stolee via GitGitGadget, Mar 16, 2021
  137. Derrick StoleeMar 16, 2021
  138. Elijah NewrenMar 16, 2021
  139. Junio C HamanoMar 18, 2021
  140. Derrick StoleeMar 19, 2021
  141. 00/20 Sparse Index: Design, Format, TestsDerrick Stolee via GitGitGadget, Mar 23, 2021
  142. 01/20 sparse-index: design doc and format updateDerrick Stolee via GitGitGadget, Mar 23, 2021
  143. SZEDER GáborMar 26, 2021
  144. Junio C HamanoMar 28, 2021
  145. Derrick StoleeMar 29, 2021
  146. 05/20 sparse-index: implement ensure_full_index()Derrick Stolee via GitGitGadget, Mar 23, 2021
  147. 04/20 sparse-index: add guard to ensure full indexDerrick Stolee via GitGitGadget, Mar 23, 2021
  148. 03/20 t1092: clean up script quotingDerrick Stolee via GitGitGadget, Mar 23, 2021
  149. 02/20 t/perf: add performance test for sparse operationsDerrick Stolee via GitGitGadget, Mar 23, 2021
  150. 10/20 sparse-checkout: hold pattern list in indexDerrick Stolee via GitGitGadget, Mar 23, 2021
  151. 07/20 test-read-cache: print cache entries with --tableDerrick Stolee via GitGitGadget, Mar 23, 2021
  152. Ævar Arnfjörð BjarmasonMar 24, 2021
  153. Derrick StoleeMar 24, 2021
  154. Ævar Arnfjörð BjarmasonMar 25, 2021
  155. Elijah NewrenMar 26, 2021
  156. Ævar Arnfjörð BjarmasonMar 28, 2021
  157. Derrick StoleeMar 29, 2021
  158. Junio C HamanoMar 29, 2021
  159. Derrick StoleeMar 30, 2021
  160. Ævar Arnfjörð BjarmasonMar 29, 2021
  161. Derrick StoleeMar 30, 2021
  162. Elijah NewrenMar 29, 2021
  163. 08/20 test-tool: don't force full indexDerrick Stolee via GitGitGadget, Mar 23, 2021
  164. 09/20 unpack-trees: ensure full indexDerrick Stolee via GitGitGadget, Mar 23, 2021
  165. 12/20 submodule: sparse-index should not collapse linksDerrick Stolee via GitGitGadget, Mar 23, 2021
  166. 06/20 t1092: compare sparse-checkout to sparse-indexDerrick Stolee via GitGitGadget, Mar 23, 2021
  167. 13/20 unpack-trees: allow sparse directoriesDerrick Stolee via GitGitGadget, Mar 23, 2021
  168. 11/20 sparse-index: convert from full to sparseDerrick Stolee via GitGitGadget, Mar 23, 2021
  169. 18/20 cache-tree: integrate with sparse directory entriesDerrick Stolee via GitGitGadget, Mar 23, 2021
  170. 15/20 sparse-index: create extension for compatibilityDerrick Stolee via GitGitGadget, Mar 23, 2021
  171. 16/20 sparse-checkout: toggle sparse index from builtinDerrick Stolee via GitGitGadget, Mar 23, 2021
  172. 14/20 sparse-index: check index conversion happensDerrick Stolee via GitGitGadget, Mar 23, 2021
  173. 17/20 sparse-checkout: disable sparse-indexDerrick Stolee via GitGitGadget, Mar 23, 2021
  174. 19/20 sparse-index: loose integration with cache_tree_verify()Derrick Stolee via GitGitGadget, Mar 23, 2021
  175. 20/20 p2000: add sparse-index reposDerrick Stolee via GitGitGadget, Mar 23, 2021
  176. Elijah NewrenMar 23, 2021
  177. 00/21 Sparse Index: Design, Format, TestsDerrick Stolee via GitGitGadget, Mar 30, 2021
  178. 09/21 unpack-trees: ensure full indexDerrick Stolee via GitGitGadget, Mar 30, 2021
  179. 01/21 sparse-index: design doc and format updateDerrick Stolee via GitGitGadget, Mar 30, 2021
  180. 05/21 sparse-index: implement ensure_full_index()Derrick Stolee via GitGitGadget, Mar 30, 2021
  181. 08/21 test-tool: don't force full indexDerrick Stolee via GitGitGadget, Mar 30, 2021
  182. 06/21 t1092: compare sparse-checkout to sparse-indexDerrick Stolee via GitGitGadget, Mar 30, 2021
  183. 04/21 sparse-index: add guard to ensure full indexDerrick Stolee via GitGitGadget, Mar 30, 2021
  184. 10/21 sparse-checkout: hold pattern list in indexDerrick Stolee via GitGitGadget, Mar 30, 2021
  185. 02/21 t/perf: add performance test for sparse operationsDerrick Stolee via GitGitGadget, Mar 30, 2021
  186. 13/21 submodule: sparse-index should not collapse linksDerrick Stolee via GitGitGadget, Mar 30, 2021
  187. 15/21 sparse-index: check index conversion happensDerrick Stolee via GitGitGadget, Mar 30, 2021
  188. 16/21 sparse-index: add index.sparse config optionDerrick Stolee via GitGitGadget, Mar 30, 2021
  189. 17/21 sparse-checkout: toggle sparse index from builtinDerrick Stolee via GitGitGadget, Mar 30, 2021
  190. 03/21 t1092: clean up script quotingDerrick Stolee via GitGitGadget, Mar 30, 2021
  191. 07/21 test-read-cache: print cache entries with --tableDerrick Stolee via GitGitGadget, Mar 30, 2021
  192. 11/21 sparse-index: add 'sdir' index extensionDerrick Stolee via GitGitGadget, Mar 30, 2021
  193. 20/21 sparse-index: loose integration with cache_tree_verify()Derrick Stolee via GitGitGadget, Mar 30, 2021
  194. 14/21 unpack-trees: allow sparse directoriesDerrick Stolee via GitGitGadget, Mar 30, 2021
  195. 12/21 sparse-index: convert from full to sparseDerrick Stolee via GitGitGadget, Mar 30, 2021
  196. 21/21 p2000: add sparse-index reposDerrick Stolee via GitGitGadget, Mar 30, 2021
  197. 18/21 sparse-checkout: disable sparse-indexDerrick Stolee via GitGitGadget, Mar 30, 2021
  198. 19/21 cache-tree: integrate with sparse directory entriesDerrick Stolee via GitGitGadget, Mar 30, 2021
  199. Junio C HamanoMar 30, 2021
  200. Derrick StoleeMar 30, 2021
  201. Junio C HamanoMar 30, 2021
  202. Elijah NewrenApr 1, 2021
  203. Elijah NewrenApr 1, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.