git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v14 01/21] read-cache: allow to keep mmap'd memory after reading

From
David Turner <novalis@novalis.org>
Date
Jul 3, 2016, 07:57 UTC
Message-ID
<1467532693-20017-2-git-send-email-novalis@novalis.org>
In-Reply-To
<1467532693-20017-1-git-send-email-novalis@novalis.org>
From: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>

Later, we will introduce git index-helper to share this memory with other git processes.

We only unmap it when we discard the index (although the kernel may of course choose to page it out).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 cache.h      |  3 +++
 read-cache.c | 13 ++++++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/cache.h b/cache.h
index b829410..4180e2b 100644
--- a/cache.h
+++ b/cache.h
@@ -333,11 +333,14 @@ struct index_state {
 	struct split_index *split_index;
 	struct cache_time timestamp;
 	unsigned name_hash_initialized : 1,
+		 keep_mmap : 1,
 		 initialized : 1;
 	struct hashmap name_hash;
 	struct hashmap dir_hash;
 	unsigned char sha1[20];
 	struct untracked_cache *untracked;
+	void *mmap;
+	size_t mmap_size;
 };
 
 extern struct index_state the_index;
diff --git a/read-cache.c b/read-cache.c
index 16cc487..3cb0ec3 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1574,6 +1574,10 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 	mmap = xmmap(NULL, mmap_size, PROT_READ, MAP_PRIVATE, fd, 0);
 	if (mmap == MAP_FAILED)
 		die_errno("unable to map index file");
+	if (istate->keep_mmap) {
+		istate->mmap = mmap;
+		istate->mmap_size = mmap_size;
+	}
 	close(fd);
 
 	hdr = mmap;
@@ -1626,11 +1630,13 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 		src_offset += 8;
 		src_offset += extsize;
 	}
-	munmap(mmap, mmap_size);
+	if (!istate->keep_mmap)
+		munmap(mmap, mmap_size);
 	return istate->cache_nr;
 
 unmap:
 	munmap(mmap, mmap_size);
+	istate->mmap = NULL;
 	die("index file corrupt");
 }
 
@@ -1655,6 +1661,7 @@ int read_index_from(struct index_state *istate, const char *path)
 		discard_index(split_index->base);
 	else
 		split_index->base = xcalloc(1, sizeof(*split_index->base));
+	split_index->base->keep_mmap = istate->keep_mmap;
 	ret = do_read_index(split_index->base,
 			    git_path("sharedindex.%s",
 				     sha1_to_hex(split_index->base_sha1)), 1);
@@ -1698,6 +1705,10 @@ int discard_index(struct index_state *istate)
 	free(istate->cache);
 	istate->cache = NULL;
 	istate->cache_alloc = 0;
+	if (istate->keep_mmap && istate->mmap) {
+		munmap(istate->mmap, istate->mmap_size);
+		istate->mmap = NULL;
+	}
 	discard_split_index(istate);
 	free_untracked_cache(istate->untracked);
 	istate->untracked = NULL;
-- 
1.9.1
Previous: David TurnerNext: David Turner
Message 6 of 30 in “index-helper/watchman”
  1. 00/21 index-helper/watchmanDavid Turner, Jul 3, 2016
  2. 03/21 unix-socket.c: add stub implementation when unix sockets are not supportedDavid Turner, Jul 3, 2016
  3. 05/21 index-helper: add --strictDavid Turner, Jul 3, 2016
  4. 06/21 daemonize(): set a flag before exiting the main processDavid Turner, Jul 3, 2016
  5. 02/21 pkt-line: add gentle version of packet_writeDavid Turner, Jul 3, 2016
  6. 01/21 read-cache: allow to keep mmap'd memory after readingDavid Turner, Jul 3, 2016
  7. 04/21 index-helper: new daemon for caching index and related stuffDavid Turner, Jul 3, 2016
  8. 16/21 index-helper: don't run if already runningDavid Turner, Jul 3, 2016
  9. 17/21 index-helper: autorun modeDavid Turner, Jul 3, 2016
  10. 18/21 index-helper: optionally automatically runDavid Turner, Jul 3, 2016
  11. 19/21 trace: measure where the time is spent in the index-heavy operationsDavid Turner, Jul 3, 2016
  12. 07/21 index-helper: add --detachDavid Turner, Jul 3, 2016
  13. 13/21 unpack-trees: preserve index extensionsDavid Turner, Jul 3, 2016
  14. 15/21 index-helper: kill modeDavid Turner, Jul 3, 2016
  15. Johannes SchindelinJul 6, 2016
  16. Duy NguyenJul 6, 2016
  17. 12/21 update-index: enable/disable watchman supportDavid Turner, Jul 3, 2016
  18. 14/21 watchman: add a config option to enable the extensionDavid Turner, Jul 3, 2016
  19. 09/21 read-cache: add watchman 'WAMA' extensionDavid Turner, Jul 3, 2016
  20. 11/21 index-helper: use watchman to avoid refreshing index with lstat()David Turner, Jul 3, 2016
  21. 10/21 watchman: support watchman to reduce index refresh costDavid Turner, Jul 3, 2016
  22. 08/21 index-helper: log warningsDavid Turner, Jul 3, 2016
  23. Johannes SchindelinJul 3, 2016
  24. Johannes SchindelinJul 4, 2016
  25. Junio C HamanoJul 6, 2016
  26. Duy NguyenJul 12, 2016
  27. David TurnerJul 13, 2016
  28. Duy NguyenJul 14, 2016
  29. Duy NguyenJul 14, 2016
  30. Ben PeartJul 15, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.