[PATCH 1/4] use_pack: attempt to handle ENOMEM from mmap

From: Eric Wong <e@80x24.org>
Date: Jun 29, 2021, 08:11 UTC
Message-ID: <20210629081108.28657-2-e@80x24.org>
In-Reply-To: <20210629081108.28657-1-e@80x24.org>

Since use_pack() can already safely munmap packs to respect core.packedGitLimit, attempt to gracefully handle ENOMEM errors the same way by unmapping a window and retrying.

This benefits unprivileged users who lack permissions to raise the `sys.vm.max_map_count' sysctl and/or RLIMIT_DATA resource limit.

I've also verified it is safe to release a pack here by unconditionally calling unuse_one_window() before xmmap_gently():

	--- a/packfile.c
	+++ b/packfile.c
	@@ -649,6 +649,7 @@ unsigned char *use_pack(struct packed_git *p,
					&& unuse_one_window(p))
					; /* nothing */
				do {
	+				unuse_one_window(p);
					win->base = xmmap_gently(NULL, win->len,
						PROT_READ, MAP_PRIVATE,
						p->pack_fd, win->offset);

Signed-off-by: Eric Wong <e@80x24.org>
---
 packfile.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/packfile.c b/packfile.c
index 755aa7aec5..a0da790fb4 100644
--- a/packfile.c
+++ b/packfile.c
@@ -648,9 +648,12 @@ unsigned char *use_pack(struct packed_git *p,
 			while (packed_git_limit < pack_mapped
 				&& unuse_one_window(p))
 				; /* nothing */
-			win->base = xmmap_gently(NULL, win->len,
-				PROT_READ, MAP_PRIVATE,
-				p->pack_fd, win->offset);
+			do {
+				win->base = xmmap_gently(NULL, win->len,
+					PROT_READ, MAP_PRIVATE,
+					p->pack_fd, win->offset);
+			} while (win->base == MAP_FAILED && errno == ENOMEM
+				&& unuse_one_window(p));
 			if (win->base == MAP_FAILED)
 				die_errno("packfile %s cannot be mapped",
 					  p->pack_name);