git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 1/3] read-cache: fix reading the shared index for other repos

From
Thomas Gummerer <t.gummerer@gmail.com>
Date
Jan 7, 2018, 22:30 UTC
Message-ID
<20180107223015.17720-2-t.gummerer@gmail.com>
In-Reply-To
<20180107223015.17720-1-t.gummerer@gmail.com>

read_index_from() takes a path argument for the location of the index file. For reading the shared index in split index mode however it just ignores that path argument, and reads it from the gitdir of the current repository.

This works as long as an index in the_repository is read. Once that changes, such as when we read the index of a submodule, or of a different working tree than the current one, the gitdir of the_repository will no longer contain the appropriate shared index, and git will fail to read it.

For example t3007-ls-files-recurse-submodules.sh was broken with GIT_TEST_SPLIT_INDEX set in 188dce131f ("ls-files: use repository object", 2017-06-22), and t7814-grep-recurse-submodules.sh was also broken in a similar manner, probably by introducing struct repository there, although I didn't track down the exact commit for that.

be489d02d2 ("revision.c: --indexed-objects add objects from all worktrees", 2017-08-23) breaks with split index mode in a similar manner, not erroring out when it can't read the index, but instead carrying on with pruning, without taking the index of the worktree into account.

Fix this by passing an additional gitdir parameter to read_index_from, to indicate where it should look for and read the shared index from.

read_cache_from() defaults to using the gitdir of the_repository. As it is mostly a convenience macro, having to pass get_git_dir() for every call seems overkill, and if necessary users can have more control by using read_index_from().

Helped-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
---
 cache-tree.c |  2 +-
 cache.h      |  5 +++--
 read-cache.c | 23 +++++++++++++----------
 repository.c |  2 +-
 revision.c   |  3 ++-
 5 files changed, 20 insertions(+), 15 deletions(-)
diff --git a/cache-tree.c b/cache-tree.c
index e03e72c34a..0dd6292a94 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -608,7 +608,7 @@ int write_index_as_tree(unsigned char *sha1, struct index_state *index_state, co
 
 	hold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);
 
-	entries = read_index_from(index_state, index_path);
+	entries = read_index_from(index_state, index_path, get_git_dir());
 	if (entries < 0) {
 		ret = WRITE_TREE_UNREADABLE_INDEX;
 		goto out;
diff --git a/cache.h b/cache.h
index d8b975a571..d5a7d1d7f1 100644
--- a/cache.h
+++ b/cache.h
@@ -371,7 +371,7 @@ extern void free_name_hash(struct index_state *istate);
 #define active_cache_tree (the_index.cache_tree)
 
 #define read_cache() read_index(&the_index)
-#define read_cache_from(path) read_index_from(&the_index, (path))
+#define read_cache_from(path) read_index_from(&the_index, (path), (get_git_dir()))
 #define read_cache_preload(pathspec) read_index_preload(&the_index, (pathspec))
 #define is_cache_unborn() is_index_unborn(&the_index)
 #define read_cache_unmerged() read_index_unmerged(&the_index)
@@ -616,7 +616,8 @@ extern int read_index(struct index_state *);
 extern int read_index_preload(struct index_state *, const struct pathspec *pathspec);
 extern int do_read_index(struct index_state *istate, const char *path,
 			 int must_exist); /* for testting only! */
-extern int read_index_from(struct index_state *, const char *path);
+extern int read_index_from(struct index_state *, const char *path,
+			   const char *gitdir);
 extern int is_index_unborn(struct index_state *);
 extern int read_index_unmerged(struct index_state *);
 
diff --git a/read-cache.c b/read-cache.c
index 2eb81a66b9..3f5b4afa36 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1603,7 +1603,7 @@ int hold_locked_index(struct lock_file *lk, int lock_flags)
 
 int read_index(struct index_state *istate)
 {
-	return read_index_from(istate, get_index_file());
+	return read_index_from(istate, get_index_file(), get_git_dir());
 }
 
 static struct cache_entry *cache_entry_from_ondisk(struct ondisk_cache_entry *ondisk,
@@ -1863,20 +1863,19 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
  * This way, shared index can be removed if they have not been used
  * for some time.
  */
-static void freshen_shared_index(char *base_sha1_hex, int warn)
+static void freshen_shared_index(const char *shared_index, int warn)
 {
-	char *shared_index = git_pathdup("sharedindex.%s", base_sha1_hex);
 	if (!check_and_freshen_file(shared_index, 1) && warn)
 		warning("could not freshen shared index '%s'", shared_index);
-	free(shared_index);
 }
 
-int read_index_from(struct index_state *istate, const char *path)
+int read_index_from(struct index_state *istate, const char *path,
+		    const char *gitdir)
 {
 	struct split_index *split_index;
 	int ret;
 	char *base_sha1_hex;
-	const char *base_path;
+	char *base_path;
 
 	/* istate->initialized covers both .git/index and .git/sharedindex.xxx */
 	if (istate->initialized)
@@ -1896,16 +1895,17 @@ int read_index_from(struct index_state *istate, const char *path)
 		split_index->base = xcalloc(1, sizeof(*split_index->base));
 
 	base_sha1_hex = sha1_to_hex(split_index->base_sha1);
-	base_path = git_path("sharedindex.%s", base_sha1_hex);
+	base_path = xstrfmt("%s/sharedindex.%s", gitdir, base_sha1_hex);
 	ret = do_read_index(split_index->base, base_path, 1);
 	if (hashcmp(split_index->base_sha1, split_index->base->sha1))
 		die("broken index, expect %s in %s, got %s",
 		    base_sha1_hex, base_path,
 		    sha1_to_hex(split_index->base->sha1));
 
-	freshen_shared_index(base_sha1_hex, 0);
+	freshen_shared_index(base_path, 0);
 	merge_base_index(istate);
 	post_read_index_from(istate);
+	free(base_path);
 	return ret;
 }
 
@@ -2573,8 +2573,11 @@ int write_locked_index(struct index_state *istate, struct lock_file *lock,
 	ret = write_split_index(istate, lock, flags);
 
 	/* Freshen the shared index only if the split-index was written */
-	if (!ret && !new_shared_index)
-		freshen_shared_index(sha1_to_hex(si->base_sha1), 1);
+	if (!ret && !new_shared_index) {
+		const char *shared_index = git_path("sharedindex.%s",
+						    sha1_to_hex(si->base_sha1));
+		freshen_shared_index(shared_index, 1);
+	}
 
 out:
 	if (flags & COMMIT_LOCK)
diff --git a/repository.c b/repository.c
index 998413b8bb..0d715f4fdb 100644
--- a/repository.c
+++ b/repository.c
@@ -236,5 +236,5 @@ int repo_read_index(struct repository *repo)
 	if (!repo->index)
 		repo->index = xcalloc(1, sizeof(*repo->index));
 
-	return read_index_from(repo->index, repo->index_file);
+	return read_index_from(repo->index, repo->index_file, repo->gitdir);
 }
diff --git a/revision.c b/revision.c
index 72f2b4572e..76d6d125b3 100644
--- a/revision.c
+++ b/revision.c
@@ -1352,7 +1352,8 @@ void add_index_objects_to_pending(struct rev_info *revs, unsigned int flags)
 			continue; /* current index already taken care of */
 
 		if (read_index_from(&istate,
-				    worktree_git_path(wt, "index")) > 0)
+				    worktree_git_path(wt, "index"),
+				    get_worktree_git_dir(wt)) > 0)
 			do_add_index_objects_to_pending(revs, &istate);
 		discard_index(&istate);
 	}

-- 
2.16.0.rc1.238.g530d649a79
Previous: Thomas GummererNext: Duy Nguyen
Message 36 of 95 in “fixes for split index mode”
  1. 0/3 fixes for split index modeThomas Gummerer, Dec 10, 2017
  2. 1/3 repository: fix repo_read_index with submodulesThomas Gummerer, Dec 10, 2017
  3. Brandon WilliamsDec 11, 2017
  4. Thomas GummererDec 11, 2017
  5. 2/3 prune: fix pruning with multiple worktrees and split indexThomas Gummerer, Dec 10, 2017
  6. Brandon WilliamsDec 11, 2017
  7. Thomas GummererDec 11, 2017
  8. 3/3 travis: run tests with GIT_TEST_SPLIT_INDEXThomas Gummerer, Dec 10, 2017
  9. Eric SunshineDec 10, 2017
  10. SZEDER GáborDec 11, 2017
  11. Thomas GummererDec 11, 2017
  12. Lars SchneiderDec 12, 2017
  13. Junio C HamanoDec 12, 2017
  14. Thomas GummererDec 12, 2017
  15. Junio C HamanoDec 12, 2017
  16. Thomas GummererDec 13, 2017
  17. Lars SchneiderDec 13, 2017
  18. Junio C HamanoDec 13, 2017
  19. Lars SchneiderDec 13, 2017
  20. Thomas GummererDec 13, 2017
  21. 0/3 fixes for split index modeThomas Gummerer, Dec 17, 2017
  22. 2/3 prune: fix pruning with multiple worktrees and split indexThomas Gummerer, Dec 17, 2017
  23. Brandon WilliamsDec 18, 2017
  24. Thomas GummererJan 3, 2018
  25. Junio C HamanoJan 4, 2018
  26. 1/3 repository: fix repo_read_index with submodulesThomas Gummerer, Dec 17, 2017
  27. Brandon WilliamsDec 18, 2017
  28. Thomas GummererDec 18, 2017
  29. Brandon WilliamsDec 18, 2017
  30. 3/3 travis: run tests with GIT_TEST_SPLIT_INDEXThomas Gummerer, Dec 17, 2017
  31. Lars SchneiderDec 18, 2017
  32. Thomas GummererJan 4, 2018
  33. Lars SchneiderJan 5, 2018
  34. Thomas GummererJan 7, 2018
  35. 0/3 fixes for split index modeThomas Gummerer, Jan 7, 2018
  36. 1/3 read-cache: fix reading the shared index for other reposThomas Gummerer, Jan 7, 2018
  37. Duy NguyenJan 8, 2018
  38. Thomas GummererJan 8, 2018
  39. Thomas GummererJan 13, 2018
  40. Brandon WilliamsJan 8, 2018
  41. Duy NguyenJan 9, 2018
  42. Brandon WilliamsJan 16, 2018
  43. Duy NguyenJan 17, 2018
  44. Brandon WilliamsJan 17, 2018
  45. Jonathan NiederJan 17, 2018
  46. Duy NguyenJan 18, 2018
  47. Junio C HamanoJan 19, 2018
  48. Thomas GummererJan 20, 2018
  49. Junio C HamanoJan 22, 2018
  50. Thomas GummererJan 27, 2018
  51. Junio C HamanoFeb 7, 2018
  52. 2/3 split-index: don't write cache tree with null oid entriesThomas Gummerer, Jan 7, 2018
  53. 3/3 travis: run tests with GIT_TEST_SPLIT_INDEXThomas Gummerer, Jan 7, 2018
  54. 4/3 read-cache: don't try to write index if we can't write shared indexThomas Gummerer, Jan 13, 2018
  55. Duy NguyenJan 14, 2018
  56. 1/3 read-cache.c: change type of "temp" in write_shared_index()Nguyễn Thái Ngọc Duy, Jan 14, 2018
  57. 2/3 read-cache.c: move tempfile creation/cleanup out of write_shared_indexNguyễn Thái Ngọc Duy, Jan 14, 2018
  58. 3/3 read-cache: don't write index twice if we can't write shared indexNguyễn Thái Ngọc Duy, Jan 14, 2018
  59. SZEDER GáborJan 18, 2018
  60. Duy NguyenJan 18, 2018
  61. Jeff KingJan 18, 2018
  62. Duy NguyenJan 18, 2018
  63. Duy NguyenJan 18, 2018
  64. Jeff KingJan 18, 2018
  65. SZEDER GáborJan 18, 2018
  66. Duy NguyenJan 19, 2018
  67. 0/5 Travis CI: don't run the test suite as root in the 32 bit Linux buildSZEDER Gábor, Jan 22, 2018
  68. 2/5 travis-ci: use 'set -e' in the 32 bit Linux build jobSZEDER Gábor, Jan 22, 2018
  69. Jeff KingJan 23, 2018
  70. Jeff KingJan 23, 2018
  71. SZEDER GáborJan 24, 2018
  72. Jeff KingJan 24, 2018
  73. 5/5 travis-ci: don't fail if user already exists on 32 bit Linux build jobSZEDER Gábor, Jan 22, 2018
  74. Jeff KingJan 23, 2018
  75. Duy NguyenJan 24, 2018
  76. SZEDER GáborJan 24, 2018
  77. 1/5 travis-ci: use 'set -x' for the commands under 'su' in the 32 bit Linux buildSZEDER Gábor, Jan 22, 2018
  78. 3/5 travis-ci: don't repeat the path of the cache directorySZEDER Gábor, Jan 22, 2018
  79. Jeff KingJan 23, 2018
  80. SZEDER GáborJan 24, 2018
  81. 4/5 travis-ci: don't run the test suite as root in the 32 bit Linux buildSZEDER Gábor, Jan 22, 2018
  82. Jeff KingJan 23, 2018
  83. SZEDER GáborJan 24, 2018
  84. Jeff KingJan 24, 2018
  85. Jeff KingJan 24, 2018
  86. Jeff KingJan 24, 2018
  87. SZEDER GáborJan 22, 2018
  88. Eric SunshineJan 22, 2018
  89. SZEDER GáborJan 22, 2018
  90. Duy NguyenJan 24, 2018
  91. Lars SchneiderJan 26, 2018
  92. Thomas GummererJan 14, 2018
  93. Thomas GummererJan 18, 2018
  94. Junio C HamanoJan 19, 2018
  95. Thomas GummererJan 19, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.