git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 7/8] sparse-index: add SPARSE_INDEX_IGNORE_CONFIG flag

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Aug 17, 2021, 13:23 UTC
Message-ID
<b0ece4b7dccf0cbe0477b3a6d238ea1362aed24d.1629206603.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1009.v3.git.1629206602.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

The convert_to_sparse() method checks for the GIT_TEST_SPARSE_INDEX environment variable or the "index.sparse" config setting before converting the index to a sparse one. This is for ease of use since all current consumers are preparing to compress the index before writing it to disk. If these settings are not enabled, then convert_to_sparse() silently returns without doing anything.

We will add a consumer in the next change that wants to use the sparse index as an in-memory data structure, regardless of whether the on-disk format should be sparse.

To that end, create the SPARSE_INDEX_IGNORE_CONFIG flag that will skip these config checks when enabled. All current consumers are modified to pass '0' in the new 'flags' parameter.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 read-cache.c   |  4 ++--
 sparse-index.c | 30 ++++++++++++++++--------------
 sparse-index.h |  3 ++-
 3 files changed, 20 insertions(+), 17 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 9048ef9e905..f5d4385c408 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3069,7 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	int ret;
 	int was_full = !istate->sparse_index;
 
-	ret = convert_to_sparse(istate);
+	ret = convert_to_sparse(istate, 0);
 
 	if (ret) {
 		warning(_("failed to convert to a sparse-index"));
@@ -3182,7 +3182,7 @@ static int write_shared_index(struct index_state *istate,
 	int ret, was_full = !istate->sparse_index;
 
 	move_cache_to_base_index(istate);
-	convert_to_sparse(istate);
+	convert_to_sparse(istate, 0);
 
 	trace2_region_enter_printf("index", "shared/do_write_index",
 				   the_repository, "%s", get_tempfile_path(*temp));
diff --git a/sparse-index.c b/sparse-index.c
index 2efc9fd4910..532fd11787e 100644
--- a/sparse-index.c
+++ b/sparse-index.c
@@ -124,7 +124,7 @@ static int index_has_unmerged_entries(struct index_state *istate)
 	return 0;
 }
 
-int convert_to_sparse(struct index_state *istate)
+int convert_to_sparse(struct index_state *istate, int flags)
 {
 	int test_env;
 
@@ -135,20 +135,22 @@ int convert_to_sparse(struct index_state *istate)
 	if (!istate->repo)
 		istate->repo = the_repository;
 
-	/*
-	 * The GIT_TEST_SPARSE_INDEX environment variable triggers the
-	 * index.sparse config variable to be on.
-	 */
-	test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1);
-	if (test_env >= 0)
-		set_sparse_index_config(istate->repo, test_env);
+	if (!(flags & SPARSE_INDEX_IGNORE_CONFIG)) {
+		/*
+		 * The GIT_TEST_SPARSE_INDEX environment variable triggers the
+		 * index.sparse config variable to be on.
+		 */
+		test_env = git_env_bool("GIT_TEST_SPARSE_INDEX", -1);
+		if (test_env >= 0)
+			set_sparse_index_config(istate->repo, test_env);
 
-	/*
-	 * Only convert to sparse if index.sparse is set.
-	 */
-	prepare_repo_settings(istate->repo);
-	if (!istate->repo->settings.sparse_index)
-		return 0;
+		/*
+		 * Only convert to sparse if index.sparse is set.
+		 */
+		prepare_repo_settings(istate->repo);
+		if (!istate->repo->settings.sparse_index)
+			return 0;
+	}
 
 	if (init_sparse_checkout_patterns(istate) < 0)
 		return 0;
diff --git a/sparse-index.h b/sparse-index.h
index 1115a0d7dd9..475f4f0f8da 100644
--- a/sparse-index.h
+++ b/sparse-index.h
@@ -2,7 +2,8 @@
 #define SPARSE_INDEX_H__
 
 struct index_state;
-int convert_to_sparse(struct index_state *istate);
+#define SPARSE_INDEX_IGNORE_CONFIG (1 << 0)
+int convert_to_sparse(struct index_state *istate, int flags);
 
 /*
  * Some places in the codebase expect to search for a specific path.
-- 
gitgitgadget
Previous: Johannes SchindelinNext: Derrick Stolee
Message 44 of 92 in “Sparse index: delete ignored files outside sparse cone”
  1. 0/2 Sparse index: delete ignored files outside sparse coneDerrick Stolee via GitGitGadget, Jul 29, 2021
  2. 1/2 t7519: rewrite sparse index testDerrick Stolee via GitGitGadget, Jul 29, 2021
  3. 2/2 sparse-checkout: clear tracked sparse dirsDerrick Stolee via GitGitGadget, Jul 29, 2021
  4. Elijah NewrenJul 30, 2021
  5. Derrick StoleeAug 2, 2021
  6. Elijah NewrenAug 2, 2021
  7. Derrick StoleeAug 5, 2021
  8. Elijah NewrenAug 5, 2021
  9. Elijah NewrenJul 30, 2021
  10. 0/8 Sparse index: delete ignored files outside sparse coneDerrick Stolee via GitGitGadget, Aug 10, 2021
  11. 1/8 t7519: rewrite sparse index testDerrick Stolee via GitGitGadget, Aug 10, 2021
  12. 2/8 sparse-index: silently return when not using cone-mode patternsDerrick Stolee via GitGitGadget, Aug 10, 2021
  13. 3/8 sparse-index: silently return when cache tree failsDerrick Stolee via GitGitGadget, Aug 10, 2021
  14. Elijah NewrenAug 19, 2021
  15. Derrick StoleeAug 20, 2021
  16. 4/8 unpack-trees: fix nested sparse-dir searchDerrick Stolee via GitGitGadget, Aug 10, 2021
  17. 5/8 sparse-checkout: create helper methodsDerrick Stolee via GitGitGadget, Aug 10, 2021
  18. Derrick StoleeAug 12, 2021
  19. 6/8 attr: be careful about sparse directoriesDerrick Stolee via GitGitGadget, Aug 10, 2021
  20. 7/8 sparse-index: add SPARSE_INDEX_IGNORE_CONFIG flagDerrick Stolee via GitGitGadget, Aug 10, 2021
  21. 8/8 sparse-checkout: clear tracked sparse dirsDerrick Stolee via GitGitGadget, Aug 10, 2021
  22. 0/8 Sparse index: delete ignored files outside sparse coneDerrick Stolee via GitGitGadget, Aug 17, 2021
  23. 2/8 sparse-index: silently return when not using cone-mode patternsDerrick Stolee via GitGitGadget, Aug 17, 2021
  24. 1/8 t7519: rewrite sparse index testDerrick Stolee via GitGitGadget, Aug 17, 2021
  25. Johannes SchindelinAug 19, 2021
  26. Derrick StoleeAug 20, 2021
  27. Eric SunshineAug 20, 2021
  28. 3/8 sparse-index: silently return when cache tree failsDerrick Stolee via GitGitGadget, Aug 17, 2021
  29. 5/8 sparse-checkout: create helper methodsDerrick Stolee via GitGitGadget, Aug 17, 2021
  30. Johannes SchindelinAug 19, 2021
  31. Derrick StoleeAug 20, 2021
  32. 4/8 unpack-trees: fix nested sparse-dir searchDerrick Stolee via GitGitGadget, Aug 17, 2021
  33. Johannes SchindelinAug 19, 2021
  34. Derrick StoleeAug 20, 2021
  35. René ScharfeAug 20, 2021
  36. René ScharfeAug 20, 2021
  37. Elijah NewrenAug 19, 2021
  38. 8/8 sparse-checkout: clear tracked sparse dirsDerrick Stolee via GitGitGadget, Aug 17, 2021
  39. Johannes SchindelinAug 19, 2021
  40. Derrick StoleeAug 20, 2021
  41. Elijah NewrenAug 20, 2021
  42. Elijah NewrenAug 20, 2021
  43. Johannes SchindelinAug 23, 2021
  44. 7/8 sparse-index: add SPARSE_INDEX_IGNORE_CONFIG flagDerrick Stolee via GitGitGadget, Aug 17, 2021
  45. Derrick StoleeAug 18, 2021
  46. 6/8 attr: be careful about sparse directoriesDerrick Stolee via GitGitGadget, Aug 17, 2021
  47. Johannes SchindelinAug 19, 2021
  48. Derrick StoleeAug 20, 2021
  49. Elijah NewrenAug 19, 2021
  50. Derrick StoleeAug 20, 2021
  51. Elijah NewrenAug 20, 2021
  52. Elijah NewrenAug 17, 2021
  53. 00/10 Sparse index: delete ignored files outside sparse coneDerrick Stolee via GitGitGadget, Aug 24, 2021
  54. 01/10 t7519: rewrite sparse index testDerrick Stolee via GitGitGadget, Aug 24, 2021
  55. 02/10 sparse-index: silently return when not using cone-mode patternsDerrick Stolee via GitGitGadget, Aug 24, 2021
  56. 03/10 sparse-index: silently return when cache tree failsDerrick Stolee via GitGitGadget, Aug 24, 2021
  57. 04/10 sparse-index: use WRITE_TREE_MISSING_OKDerrick Stolee via GitGitGadget, Aug 24, 2021
  58. Elijah NewrenAug 27, 2021
  59. Derrick StoleeAug 30, 2021
  60. Elijah NewrenAug 30, 2021
  61. 05/10 unpack-trees: fix nested sparse-dir searchDerrick Stolee via GitGitGadget, Aug 24, 2021
  62. René ScharfeAug 24, 2021
  63. Derrick StoleeAug 25, 2021
  64. 06/10 sparse-checkout: create helper methodsDerrick Stolee via GitGitGadget, Aug 24, 2021
  65. 07/10 attr: be careful about sparse directoriesDerrick Stolee via GitGitGadget, Aug 24, 2021
  66. 08/10 sparse-index: add SPARSE_INDEX_MEMORY_ONLY flagDerrick Stolee via GitGitGadget, Aug 24, 2021
  67. 09/10 sparse-checkout: clear tracked sparse dirsDerrick Stolee via GitGitGadget, Aug 24, 2021
  68. 10/10 sparse-checkout: add config to disable deleting dirsDerrick Stolee via GitGitGadget, Aug 24, 2021
  69. Elijah NewrenAug 27, 2021
  70. Derrick StoleeAug 30, 2021
  71. Elijah NewrenAug 30, 2021
  72. Elijah NewrenAug 27, 2021
  73. Elijah NewrenAug 27, 2021
  74. Derrick StoleeAug 30, 2021
  75. Elijah NewrenAug 30, 2021
  76. Derrick StoleeAug 30, 2021
  77. Elijah NewrenAug 30, 2021
  78. 0/9 Sparse index: delete ignored files outside sparse coneDerrick Stolee via GitGitGadget, Sep 8, 2021
  79. 1/9 t7519: rewrite sparse index testDerrick Stolee via GitGitGadget, Sep 8, 2021
  80. 2/9 sparse-index: silently return when not using cone-mode patternsDerrick Stolee via GitGitGadget, Sep 8, 2021
  81. 3/9 unpack-trees: fix nested sparse-dir searchDerrick Stolee via GitGitGadget, Sep 8, 2021
  82. 5/9 sparse-index: use WRITE_TREE_MISSING_OKDerrick Stolee via GitGitGadget, Sep 8, 2021
  83. 4/9 sparse-index: silently return when cache tree failsDerrick Stolee via GitGitGadget, Sep 8, 2021
  84. 6/9 sparse-checkout: create helper methodsDerrick Stolee via GitGitGadget, Sep 8, 2021
  85. 7/9 attr: be careful about sparse directoriesDerrick Stolee via GitGitGadget, Sep 8, 2021
  86. 8/9 sparse-index: add SPARSE_INDEX_MEMORY_ONLY flagDerrick Stolee via GitGitGadget, Sep 8, 2021
  87. 9/9 sparse-checkout: clear tracked sparse dirsDerrick Stolee via GitGitGadget, Sep 8, 2021
  88. Junio C HamanoSep 8, 2021
  89. Junio C HamanoSep 8, 2021
  90. Derrick StoleeSep 8, 2021
  91. Junio C HamanoSep 8, 2021
  92. Elijah NewrenSep 8, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.