git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 5/6] reftable/system: add abstraction to mmap files

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 31, 2026, 11:26 UTC
Message-ID
<20260331-pks-reftable-portability-fixes-v1-5-46bfae55c68c@pks.im>
In-Reply-To
<20260331-pks-reftable-portability-fixes-v1-0-46bfae55c68c@pks.im>

In our codebase we have a couple of wrappers around mmap(3p) that allow us to reimplement the syscall on platforms that don't have it natively, like for example Windows. Other projects that embed the reftable library may have a different infra though to hook up mmap wrappers, but these are currently hard to integrate.

Provide the infrastructure to let projects easily define the mmap interface with a custom struct and custom functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/blocksource.c | 19 +++++++------------
 reftable/system.c      | 20 ++++++++++++++++++++
 reftable/system.h      | 18 ++++++++++++++++++
 3 files changed, 45 insertions(+), 12 deletions(-)
diff --git a/reftable/blocksource.c b/reftable/blocksource.c
index 573c81287f..7f7441f751 100644
--- a/reftable/blocksource.c
+++ b/reftable/blocksource.c
@@ -93,13 +93,12 @@ void block_source_from_buf(struct reftable_block_source *bs,
 }
 
 struct file_block_source {
-	uint64_t size;
-	unsigned char *data;
+	struct reftable_mmap mmap;
 };
 
 static uint64_t file_size(void *b)
 {
-	return ((struct file_block_source *)b)->size;
+	return ((struct file_block_source *)b)->mmap.size;
 }
 
 static void file_release_data(void *b REFTABLE_UNUSED, struct reftable_block_data *dest REFTABLE_UNUSED)
@@ -109,7 +108,7 @@ static void file_release_data(void *b REFTABLE_UNUSED, struct reftable_block_dat
 static void file_close(void *v)
 {
 	struct file_block_source *b = v;
-	munmap(b->data, b->size);
+	reftable_munmap(&b->mmap);
 	reftable_free(b);
 }
 
@@ -117,8 +116,8 @@ static ssize_t file_read_data(void *v, struct reftable_block_data *dest, uint64_
 			      uint32_t size)
 {
 	struct file_block_source *b = v;
-	assert(off + size <= b->size);
-	dest->data = b->data + off;
+	assert(off + size <= b->mmap.size);
+	dest->data = (unsigned char *) b->mmap.data + off;
 	dest->len = size;
 	return size;
 }
@@ -156,13 +155,9 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,
 		goto out;
 	}
 
-	p->size = st.st_size;
-	p->data = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
-	if (p->data == MAP_FAILED) {
-		err = REFTABLE_IO_ERROR;
-		p->data = NULL;
+	err = reftable_mmap(&p->mmap, fd, st.st_size);
+	if (err < 0)
 		goto out;
-	}
 
 	assert(!bs->ops);
 	bs->ops = &file_vtable;
diff --git a/reftable/system.c b/reftable/system.c
index 7aecd3859d..a5603f5f61 100644
--- a/reftable/system.c
+++ b/reftable/system.c
@@ -137,3 +137,23 @@ uint64_t reftable_time_ms(void)
 {
 	return getnanotime() / 1000000;
 }
+
+int reftable_mmap(struct reftable_mmap *out, int fd, size_t len)
+{
+	void *data = xmmap_gently(NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
+	if (data == MAP_FAILED)
+		return REFTABLE_IO_ERROR;
+
+	out->data = data;
+	out->size = len;
+
+	return 0;
+}
+
+int reftable_munmap(struct reftable_mmap *mmap)
+{
+	if (munmap(mmap->data, mmap->size) < 0)
+		return REFTABLE_IO_ERROR;
+	memset(mmap, 0, sizeof(*mmap));
+	return 0;
+}
diff --git a/reftable/system.h b/reftable/system.h
index 6e00cd32a3..dffc717bd4 100644
--- a/reftable/system.h
+++ b/reftable/system.h
@@ -113,4 +113,22 @@ int flock_commit(struct reftable_flock *l);
 /* Report the time in milliseconds. */
 uint64_t reftable_time_ms(void);
 
+struct reftable_mmap {
+       void *data;
+       size_t size;
+       void *priv;
+};
+
+/*
+ * Map the file into memory. Returns 0 on success, a reftable error code on
+ * error.
+ */
+int reftable_mmap(struct reftable_mmap *out, int fd, size_t len);
+
+/*
+ * Unmap the file from memory. Returns 0 on success, a reftable error code on
+ * error.
+ */
+int reftable_munmap(struct reftable_mmap *mmap);
+
 #endif
-- 
2.53.0.1185.g05d4b7b318.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 15 of 28 in “reftable: some more portability improvements”
  1. 0/6 reftable: some more portability improvementsPatrick Steinhardt, Mar 31, 2026
  2. 1/6 reftable/system: provide `REFTABLE_INLINE()` macroPatrick Steinhardt, Mar 31, 2026
  3. Junio C HamanoMar 31, 2026
  4. René ScharfeMar 31, 2026
  5. Junio C HamanoMar 31, 2026
  6. René ScharfeMar 31, 2026
  7. brian m. carlsonMar 31, 2026
  8. Patrick SteinhardtApr 1, 2026
  9. 2/6 reftable/stack: don't call fsync(3p) unless providedPatrick Steinhardt, Mar 31, 2026
  10. Junio C HamanoMar 31, 2026
  11. Patrick SteinhardtApr 1, 2026
  12. 3/6 reftable/fsck: use REFTABLE_UNUSED instead of UNUSEDPatrick Steinhardt, Mar 31, 2026
  13. Junio C HamanoMar 31, 2026
  14. 4/6 reftable/system: add abstraction to retrieve time in millisecondsPatrick Steinhardt, Mar 31, 2026
  15. 5/6 reftable/system: add abstraction to mmap filesPatrick Steinhardt, Mar 31, 2026
  16. 6/6 reftable: introduce "reftable-system.h" headerPatrick Steinhardt, Mar 31, 2026
  17. Junio C HamanoMar 31, 2026
  18. Patrick SteinhardtApr 1, 2026
  19. Junio C HamanoApr 1, 2026
  20. Patrick SteinhardtApr 2, 2026
  21. 0/5 reftable: some more portability improvementsPatrick Steinhardt, Apr 2, 2026
  22. 1/5 reftable: introduce "reftable-system.h" headerPatrick Steinhardt, Apr 2, 2026
  23. 2/5 reftable/stack: provide fsync(3p) via system headerPatrick Steinhardt, Apr 2, 2026
  24. Junio C HamanoApr 2, 2026
  25. 3/5 reftable/fsck: use REFTABLE_UNUSED instead of UNUSEDPatrick Steinhardt, Apr 2, 2026
  26. 4/5 reftable/system: add abstraction to retrieve time in millisecondsPatrick Steinhardt, Apr 2, 2026
  27. Junio C HamanoApr 2, 2026
  28. 5/5 reftable/system: add abstraction to mmap filesPatrick Steinhardt, Apr 2, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.