git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 11/22] read-cache: regenerate shared index if necessary

From
Christian Couder <christian.couder@gmail.com>
Date
Mar 6, 2017, 09:41 UTC
Message-ID
<20170306094203.28250-12-chriscool@tuxfamily.org>
In-Reply-To
<20170306094203.28250-1-chriscool@tuxfamily.org>

When writing a new split-index and there is a big number of cache entries in the split-index compared to the shared index, it is a good idea to regenerate the shared index.

By default when the ratio reaches 20%, we will push back all the entries from the split-index into a new shared index file instead of just creating a new split-index file.

The threshold can be configured using the "splitIndex.maxPercentChange" config variable.

We need to adjust the existing tests in t1700 by setting "splitIndex.maxPercentChange" to 100 at the beginning of t1700, as the existing tests are assuming that the shared index is regenerated only when `git update-index --split-index` is used.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 read-cache.c           | 32 ++++++++++++++++++++++++++++++++
 t/t1700-split-index.sh |  1 +
 2 files changed, 33 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 99bc274b8d..aeb413a508 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2212,6 +2212,36 @@ static int write_shared_index(struct index_state *istate,
 	return ret;
 }
 
+static const int default_max_percent_split_change = 20;
+
+static int too_many_not_shared_entries(struct index_state *istate)
+{
+	int i, not_shared = 0;
+	int max_split = git_config_get_max_percent_split_change();
+
+	switch (max_split) {
+	case -1:
+		/* not or badly configured: use the default value */
+		max_split = default_max_percent_split_change;
+		break;
+	case 0:
+		return 1; /* 0% means always write a new shared index */
+	case 100:
+		return 0; /* 100% means never write a new shared index */
+	default:
+		break; /* just use the configured value */
+	}
+
+	/* Count not shared entries */
+	for (i = 0; i < istate->cache_nr; i++) {
+		struct cache_entry *ce = istate->cache[i];
+		if (!ce->index)
+			not_shared++;
+	}
+
+	return (int64_t)istate->cache_nr * max_split < (int64_t)not_shared * 100;
+}
+
 int write_locked_index(struct index_state *istate, struct lock_file *lock,
 		       unsigned flags)
 {
@@ -2229,6 +2259,8 @@ int write_locked_index(struct index_state *istate, struct lock_file *lock,
 		if ((v & 15) < 6)
 			istate->cache_changed |= SPLIT_INDEX_ORDERED;
 	}
+	if (too_many_not_shared_entries(istate))
+		istate->cache_changed |= SPLIT_INDEX_ORDERED;
 	if (istate->cache_changed & SPLIT_INDEX_ORDERED) {
 		int ret = write_shared_index(istate, lock, flags);
 		if (ret)
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index aa2aff1778..9d7c01c3e1 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -8,6 +8,7 @@ test_description='split index mode tests'
 sane_unset GIT_TEST_SPLIT_INDEX
 
 test_expect_success 'enable split index' '
+	git config splitIndex.maxPercentChange 100 &&
 	git update-index --split-index &&
 	test-dump-split-index .git/index >actual &&
 	indexversion=$(test-index-version <.git/index) &&
-- 
2.12.0.206.g74921e51d6.dirty
Previous: Christian CouderNext: Christian Couder
Message 2 of 23 in “Add configuration options for split-index”
  1. 00/22 Add configuration options for split-indexChristian Couder, Mar 6, 2017
  2. 11/22 read-cache: regenerate shared index if necessaryChristian Couder, Mar 6, 2017
  3. 17/22 read-cache: unlink old sharedindex filesChristian Couder, Mar 6, 2017
  4. 09/22 Documentation/git-update-index: talk about core.splitIndex config varChristian Couder, Mar 6, 2017
  5. 22/22 Documentation/git-update-index: explain splitIndex.*Christian Couder, Mar 6, 2017
  6. 14/22 sha1_file: make check_and_freshen_file() non staticChristian Couder, Mar 6, 2017
  7. 20/22 read-cache: use freshen_shared_index() in read_index_from()Christian Couder, Mar 6, 2017
  8. 19/22 read-cache: refactor read_index_from()Christian Couder, Mar 6, 2017
  9. 13/22 Documentation/config: add splitIndex.maxPercentChangeChristian Couder, Mar 6, 2017
  10. 10/22 config: add git_config_get_max_percent_split_change()Christian Couder, Mar 6, 2017
  11. 06/22 update-index: warn in case of split-index incoherencyChristian Couder, Mar 6, 2017
  12. 18/22 t1700: test shared index file expirationChristian Couder, Mar 6, 2017
  13. 15/22 read-cache: touch shared index files when usedChristian Couder, Mar 6, 2017
  14. 07/22 t1700: add tests for core.splitIndexChristian Couder, Mar 6, 2017
  15. 16/22 config: add git_config_get_expiry() from gc.cChristian Couder, Mar 6, 2017
  16. 12/22 t1700: add tests for splitIndex.maxPercentChangeChristian Couder, Mar 6, 2017
  17. 04/22 split-index: add {add,remove}_split_index() functionsChristian Couder, Mar 6, 2017
  18. 21/22 Documentation/config: add splitIndex.sharedIndexExpireChristian Couder, Mar 6, 2017
  19. 02/22 t1700: change here document styleChristian Couder, Mar 6, 2017
  20. 08/22 Documentation/config: add information for core.splitIndexChristian Couder, Mar 6, 2017
  21. 05/22 read-cache: add and then use tweak_split_index()Christian Couder, Mar 6, 2017
  22. 01/22 config: mark an error message up for translationChristian Couder, Mar 6, 2017
  23. 03/22 config: add git_config_get_split_index()Christian Couder, Mar 6, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.