git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v6 4/6] Introduce GIT_MMAP_LIMIT to allow testing expected mmap size

From
Steffen Prohaska <prohaska@zib.de>
Date
Aug 26, 2014, 15:23 UTC
Message-ID
<1409066605-4851-5-git-send-email-prohaska@zib.de>
In-Reply-To
<1409066605-4851-1-git-send-email-prohaska@zib.de>

Similar to testing expectations about malloc with GIT_ALLOC_LIMIT (see commit d41489 and previous commit), it can be useful to test expectations about mmap.

This introduces a new environment variable GIT_MMAP_LIMIT to limit the largest allowed mmap length. xmmap() is modified to check the limit. Together with GIT_ALLOC_LIMIT tests can now easily confirm expectations about memory consumption.

GIT_MMAP_LIMIT will be used in the next commit to test that data will be streamed to an external filter without mmaping the entire file.

[commit d41489]: d41489a6424308dc9a0409bc2f6845aa08bd4f7d Add more large
    blob test cases
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
---
 sha1_file.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/sha1_file.c b/sha1_file.c
index 00c07f2..d9b5157 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -663,10 +663,26 @@ void release_pack_memory(size_t need)
 		; /* nothing */
 }
 
+static void mmap_limit_check(size_t length)
+{
+	static size_t limit = 0;
+	if (!limit) {
+		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
+		if (!limit)
+			limit = SIZE_MAX;
+	}
+	if (length > limit)
+		die("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX,
+		    (uintmax_t)length, (uintmax_t)limit);
+}
+
 void *xmmap(void *start, size_t length,
 	int prot, int flags, int fd, off_t offset)
 {
-	void *ret = mmap(start, length, prot, flags, fd, offset);
+	void *ret;
+
+	mmap_limit_check(length);
+	ret = mmap(start, length, prot, flags, fd, offset);
 	if (ret == MAP_FAILED) {
 		if (!length)
 			return NULL;
-- 
2.1.0.8.gf3a29c8
Previous: Steffen ProhaskaNext: Steffen Prohaska
Message 13 of 19 in “Stream fd to clean filter; GIT_MMAP_LIMIT, GIT_ALLOC_LIMIT with git_env_ulong()”
  1. 0/6 Stream fd to clean filter; GIT_MMAP_LIMIT, GIT_ALLOC_LIMIT with git_env_ulong()Steffen Prohaska, Aug 26, 2014
  2. 1/6 convert: drop arguments other than 'path' from would_convert_to_git()Steffen Prohaska, Aug 26, 2014
  3. 2/6 Add git_env_ulong() to parse environment variableSteffen Prohaska, Aug 26, 2014
  4. Jeff KingAug 26, 2014
  5. Junio C HamanoAug 26, 2014
  6. Jeff KingAug 26, 2014
  7. Junio C HamanoAug 26, 2014
  8. Jeff KingAug 27, 2014
  9. Junio C HamanoAug 27, 2014
  10. Steffen ProhaskaAug 28, 2014
  11. Junio C HamanoAug 28, 2014
  12. 3/6 Change GIT_ALLOC_LIMIT check to use git_env_ulong()Steffen Prohaska, Aug 26, 2014
  13. 4/6 Introduce GIT_MMAP_LIMIT to allow testing expected mmap sizeSteffen Prohaska, Aug 26, 2014
  14. 5/6 Change copy_fd() to not close input fdSteffen Prohaska, Aug 26, 2014
  15. Junio C HamanoAug 26, 2014
  16. Jeff KingAug 26, 2014
  17. Steffen ProhaskaAug 28, 2014
  18. Junio C HamanoAug 28, 2014
  19. 6/6 convert: stream from fd to required clean filter to reduce used address spaceSteffen Prohaska, Aug 26, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.