git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 06/19] index-helper: add --strict

From
David Turner <dturner@twopensource.com>
Date
Mar 9, 2016, 18:36 UTC
Message-ID
<1457548582-28302-7-git-send-email-dturner@twopensource.com>
In-Reply-To
<1457548582-28302-1-git-send-email-dturner@twopensource.com>
From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>

There's are "holes" in the index-helper approach because the shared memory is not verified again by git. If $USER is compromised, shared memory could be modified. But then they can already modify $GIT_DIR/index. A more realistic risk is some bugs in index-helper produce corrupt shared memory. --strict is added to avoid that

Strictly speaking there's still a very small gap where corrupt shared memory could still be read by git: after we write the trailing SHA-1 in the shared memory (thus signaling "this shm is ready") and before verify_shm() detects an error.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-index-helper.txt |  9 +++++++
 cache.h                            |  1 +
 index-helper.c                     | 48 ++++++++++++++++++++++++++++++++++++++
 read-cache.c                       |  9 ++++---
 4 files changed, 64 insertions(+), 3 deletions(-)
diff --git a/Documentation/git-index-helper.txt b/Documentation/git-index-helper.txt
index 9db28cf..ad40366 100644
--- a/Documentation/git-index-helper.txt
+++ b/Documentation/git-index-helper.txt
@@ -22,6 +22,15 @@ OPTIONS
 	Exit if the cached index is not accessed for `<n>`
 	minutes. Specify 0 to wait forever. Default is 10.
 
+--strict::
+--no-strict::
+	Strict mode makes index-helper verify the shared memory after
+	it's created. If the result does not match what's read from
+	$GIT_DIR/index, the shared memory is destroyed. This makes
+	index-helper take more than double the amount of time required
+	for reading an index, but because it will happen in the
+	background, it's not noticable. `--strict` is enabled by default.
+
 NOTES
 -----
 On UNIX-like systems, $GIT_DIR/index-helper.pid contains the process
diff --git a/cache.h b/cache.h
index e22296c..1d7e561 100644
--- a/cache.h
+++ b/cache.h
@@ -336,6 +336,7 @@ struct index_state {
 		 keep_mmap : 1,
 		 from_shm : 1,
 		 to_shm : 1,
+		 always_verify_trailing_sha1 : 1,
 		 initialized : 1;
 	struct hashmap name_hash;
 	struct hashmap dir_hash;
diff --git a/index-helper.c b/index-helper.c
index cf2971d..1140bc0 100644
--- a/index-helper.c
+++ b/index-helper.c
@@ -14,6 +14,7 @@ struct shm {
 
 static struct shm shm_index;
 static struct shm shm_base_index;
+static int to_verify = 1;
 
 static void release_index_shm(struct shm *is)
 {
@@ -69,11 +70,56 @@ static void share_index(struct index_state *istate, struct shm *is)
 	hashcpy((unsigned char *)new_mmap + istate->mmap_size - 20, is->sha1);
 }
 
+static int verify_shm(void)
+{
+	int i;
+	struct index_state istate;
+	memset(&istate, 0, sizeof(istate));
+	istate.always_verify_trailing_sha1 = 1;
+	istate.to_shm = 1;
+	i = read_index(&istate);
+	if (i != the_index.cache_nr)
+		goto done;
+	for (; i < the_index.cache_nr; i++) {
+		struct cache_entry *base, *ce;
+		/* namelen is checked separately */
+		const unsigned int ondisk_flags =
+			CE_STAGEMASK | CE_VALID | CE_EXTENDED_FLAGS;
+		unsigned int ce_flags, base_flags, ret;
+		base = the_index.cache[i];
+		ce = istate.cache[i];
+		if (ce->ce_namelen != base->ce_namelen ||
+		    strcmp(ce->name, base->name)) {
+			warning("mismatch at entry %d", i);
+			break;
+		}
+		ce_flags = ce->ce_flags;
+		base_flags = base->ce_flags;
+		/* only on-disk flags matter */
+		ce->ce_flags   &= ondisk_flags;
+		base->ce_flags &= ondisk_flags;
+		ret = memcmp(&ce->ce_stat_data, &base->ce_stat_data,
+			     offsetof(struct cache_entry, name) -
+			     offsetof(struct cache_entry, ce_stat_data));
+		ce->ce_flags = ce_flags;
+		base->ce_flags = base_flags;
+		if (ret) {
+			warning("mismatch at entry %d", i);
+			break;
+		}
+	}
+done:
+	discard_index(&istate);
+	return i == the_index.cache_nr;
+}
+
 static void share_the_index(void)
 {
 	if (the_index.split_index && the_index.split_index->base)
 		share_index(the_index.split_index->base, &shm_base_index);
 	share_index(&the_index, &shm_index);
+	if (to_verify && !verify_shm())
+		cleanup_shm();
 	discard_index(&the_index);
 }
 
@@ -130,6 +176,8 @@ int main(int argc, char **argv)
 	struct option options[] = {
 		OPT_INTEGER(0, "exit-after", &idle_in_minutes,
 			    N_("exit if not used after some minutes")),
+		OPT_BOOL(0, "strict", &to_verify,
+			 "verify shared memory after creating"),
 		OPT_END()
 	};
 
diff --git a/read-cache.c b/read-cache.c
index 7bd3ce4..1a0ab0c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1660,9 +1660,12 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 
 	istate->mmap = mmap;
 	istate->mmap_size = mmap_size;
-	if (try_shm(istate) &&
-	    verify_hdr(istate->mmap, istate->mmap_size) < 0)
-		goto unmap;
+	if (try_shm(istate)) {
+		if (verify_hdr(istate->mmap, istate->mmap_size) < 0)
+			goto unmap;
+	} else if (istate->always_verify_trailing_sha1 &&
+		   verify_hdr(istate->mmap, istate->mmap_size) < 0)
+			goto unmap;
 	hdr = mmap = istate->mmap;
 	mmap_size = istate->mmap_size;
 	if (!istate->keep_mmap)

-- 
2.4.2.767.g62658d5-twtrsrc
Previous: David TurnerNext: David Turner
Message 22 of 60 in “index-helper, watchman”
  1. 00/19 index-helper, watchmanDavid Turner, Mar 9, 2016
  2. 01/19 trace.c: add GIT_TRACE_PACK_STATS for pack usage statisticsDavid Turner, Mar 9, 2016
  3. Junio C HamanoMar 9, 2016
  4. David TurnerMar 10, 2016
  5. Duy NguyenMar 10, 2016
  6. 02/19 read-cache.c: fix constness of verify_hdr()David Turner, Mar 9, 2016
  7. 03/19 read-cache: allow to keep mmap'd memory after readingDavid Turner, Mar 9, 2016
  8. Junio C HamanoMar 9, 2016
  9. David TurnerMar 10, 2016
  10. 04/19 index-helper: new daemon for caching index and related stuffDavid Turner, Mar 9, 2016
  11. Junio C HamanoMar 9, 2016
  12. Junio C HamanoMar 9, 2016
  13. David TurnerMar 10, 2016
  14. Duy NguyenMar 10, 2016
  15. David TurnerMar 10, 2016
  16. Duy NguyenMar 11, 2016
  17. David TurnerMar 10, 2016
  18. Duy NguyenMar 15, 2016
  19. Junio C HamanoMar 15, 2016
  20. Duy NguyenMar 15, 2016
  21. 05/19 trace.c: add GIT_TRACE_INDEX_STATS for index statisticsDavid Turner, Mar 9, 2016
  22. 06/19 index-helper: add --strictDavid Turner, Mar 9, 2016
  23. 07/19 daemonize(): set a flag before exiting the main processDavid Turner, Mar 9, 2016
  24. 08/19 index-helper: add --detachDavid Turner, Mar 9, 2016
  25. 09/19 index-helper: add Windows supportDavid Turner, Mar 9, 2016
  26. Duy NguyenMar 16, 2016
  27. Johannes SchindelinMar 17, 2016
  28. Duy NguyenMar 17, 2016
  29. 10/19 read-cache: add watchman 'WAMA' extensionDavid Turner, Mar 9, 2016
  30. 11/19 Add watchman support to reduce index refresh costDavid Turner, Mar 9, 2016
  31. 12/19 read-cache: allow index-helper to prepare shm before git reads itDavid Turner, Mar 9, 2016
  32. 13/19 index-helper: use watchman to avoid refreshing index with lstat()David Turner, Mar 9, 2016
  33. 14/19 update-index: enable/disable watchman supportDavid Turner, Mar 9, 2016
  34. 15/19 unpack-trees: preserve index extensionsDavid Turner, Mar 9, 2016
  35. 16/19 index-helper: rewrite pidfile after daemonizingDavid Turner, Mar 9, 2016
  36. 17/19 index-helper: process managementDavid Turner, Mar 9, 2016
  37. 18/19 index-helper: autorunDavid Turner, Mar 9, 2016
  38. Duy NguyenMar 15, 2016
  39. Johannes SchindelinMar 15, 2016
  40. Duy NguyenMar 16, 2016
  41. David TurnerMar 16, 2016
  42. Johannes SchindelinMar 16, 2016
  43. Duy NguyenMar 17, 2016
  44. Johannes SchindelinMar 17, 2016
  45. David TurnerMar 17, 2016
  46. Duy NguyenMar 18, 2016
  47. Johannes SchindelinMar 18, 2016
  48. Duy NguyenMar 18, 2016
  49. David TurnerMar 18, 2016
  50. Duy NguyenMar 18, 2016
  51. Johannes SchindelinMar 18, 2016
  52. Duy NguyenMar 18, 2016
  53. Johannes SchindelinMar 18, 2016
  54. 19/19 hack: watchman/untracked cache mashupDavid Turner, Mar 9, 2016
  55. Duy NguyenMar 15, 2016
  56. David TurnerMar 17, 2016
  57. Duy NguyenMar 17, 2016
  58. David TurnerMar 17, 2016
  59. Torsten BögershausenMar 29, 2016
  60. David TurnerMar 29, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.