git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 17/22] read-cache: unlink old sharedindex files

From
Christian Couder <christian.couder@gmail.com>
Date
Mar 6, 2017, 09:41 UTC
Message-ID
<20170306094203.28250-18-chriscool@tuxfamily.org>
In-Reply-To
<20170306094203.28250-1-chriscool@tuxfamily.org>

Everytime split index is turned on, it creates a "sharedindex.XXXX" file in the git directory. This change makes sure that shared index files that haven't been used for a long time are removed when a new shared index file is created.

The new "splitIndex.sharedIndexExpire" config variable is created to tell the delay after which an unused shared index file can be deleted. It defaults to "2.weeks.ago".

A previous commit made sure that each time a split index file is created the mtime of the shared index file it references is updated. This makes sure that recently used shared index file will not be deleted.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 read-cache.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 63 insertions(+), 1 deletion(-)
diff --git a/read-cache.c b/read-cache.c
index 13375fa0ff..16c05f359b 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2199,6 +2199,65 @@ static int write_split_index(struct index_state *istate,
 	return ret;
 }
 
+static const char *shared_index_expire = "2.weeks.ago";
+
+static unsigned long get_shared_index_expire_date(void)
+{
+	static unsigned long shared_index_expire_date;
+	static int shared_index_expire_date_prepared;
+
+	if (!shared_index_expire_date_prepared) {
+		git_config_get_expiry("splitindex.sharedindexexpire",
+				      &shared_index_expire);
+		shared_index_expire_date = approxidate(shared_index_expire);
+		shared_index_expire_date_prepared = 1;
+	}
+
+	return shared_index_expire_date;
+}
+
+static int should_delete_shared_index(const char *shared_index_path)
+{
+	struct stat st;
+	unsigned long expiration;
+
+	/* Check timestamp */
+	expiration = get_shared_index_expire_date();
+	if (!expiration)
+		return 0;
+	if (stat(shared_index_path, &st))
+		return error_errno(_("could not stat '%s"), shared_index_path);
+	if (st.st_mtime > expiration)
+		return 0;
+
+	return 1;
+}
+
+static int clean_shared_index_files(const char *current_hex)
+{
+	struct dirent *de;
+	DIR *dir = opendir(get_git_dir());
+
+	if (!dir)
+		return error_errno(_("unable to open git dir: %s"), get_git_dir());
+
+	while ((de = readdir(dir)) != NULL) {
+		const char *sha1_hex;
+		const char *shared_index_path;
+		if (!skip_prefix(de->d_name, "sharedindex.", &sha1_hex))
+			continue;
+		if (!strcmp(sha1_hex, current_hex))
+			continue;
+		shared_index_path = git_path("%s", de->d_name);
+		if (should_delete_shared_index(shared_index_path) > 0 &&
+		    unlink(shared_index_path))
+			warning_errno(_("unable to unlink: %s"), shared_index_path);
+	}
+	closedir(dir);
+
+	return 0;
+}
+
 static struct tempfile temporary_sharedindex;
 
 static int write_shared_index(struct index_state *istate,
@@ -2220,8 +2279,11 @@ static int write_shared_index(struct index_state *istate,
 	}
 	ret = rename_tempfile(&temporary_sharedindex,
 			      git_path("sharedindex.%s", sha1_to_hex(si->base->sha1)));
-	if (!ret)
+	if (!ret) {
 		hashcpy(si->base_sha1, si->base->sha1);
+		clean_shared_index_files(sha1_to_hex(si->base->sha1));
+	}
+
 	return ret;
 }
 
-- 
2.12.0.206.g74921e51d6.dirty
Previous: Christian CouderNext: Christian Couder
Message 3 of 23 in “Add configuration options for split-index”
  1. 00/22 Add configuration options for split-indexChristian Couder, Mar 6, 2017
  2. 11/22 read-cache: regenerate shared index if necessaryChristian Couder, Mar 6, 2017
  3. 17/22 read-cache: unlink old sharedindex filesChristian Couder, Mar 6, 2017
  4. 09/22 Documentation/git-update-index: talk about core.splitIndex config varChristian Couder, Mar 6, 2017
  5. 22/22 Documentation/git-update-index: explain splitIndex.*Christian Couder, Mar 6, 2017
  6. 14/22 sha1_file: make check_and_freshen_file() non staticChristian Couder, Mar 6, 2017
  7. 20/22 read-cache: use freshen_shared_index() in read_index_from()Christian Couder, Mar 6, 2017
  8. 19/22 read-cache: refactor read_index_from()Christian Couder, Mar 6, 2017
  9. 13/22 Documentation/config: add splitIndex.maxPercentChangeChristian Couder, Mar 6, 2017
  10. 10/22 config: add git_config_get_max_percent_split_change()Christian Couder, Mar 6, 2017
  11. 06/22 update-index: warn in case of split-index incoherencyChristian Couder, Mar 6, 2017
  12. 18/22 t1700: test shared index file expirationChristian Couder, Mar 6, 2017
  13. 15/22 read-cache: touch shared index files when usedChristian Couder, Mar 6, 2017
  14. 07/22 t1700: add tests for core.splitIndexChristian Couder, Mar 6, 2017
  15. 16/22 config: add git_config_get_expiry() from gc.cChristian Couder, Mar 6, 2017
  16. 12/22 t1700: add tests for splitIndex.maxPercentChangeChristian Couder, Mar 6, 2017
  17. 04/22 split-index: add {add,remove}_split_index() functionsChristian Couder, Mar 6, 2017
  18. 21/22 Documentation/config: add splitIndex.sharedIndexExpireChristian Couder, Mar 6, 2017
  19. 02/22 t1700: change here document styleChristian Couder, Mar 6, 2017
  20. 08/22 Documentation/config: add information for core.splitIndexChristian Couder, Mar 6, 2017
  21. 05/22 read-cache: add and then use tweak_split_index()Christian Couder, Mar 6, 2017
  22. 01/22 config: mark an error message up for translationChristian Couder, Mar 6, 2017
  23. 03/22 config: add git_config_get_split_index()Christian Couder, Mar 6, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.