git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/5] test-delta: read input into a heap buffer

From
Jeff King <peff@peff.net>
Date
Aug 30, 2018, 07:07 UTC
Message-ID
<20180830070751.GA15420@sigill.intra.peff.net>
In-Reply-To
<20180830070548.GA15081@sigill.intra.peff.net>

We currently read the input to test-delta by mmap()-ing it. However, memory-checking tools like valgrind and ASan are less able to detect reads/writes past the end of an mmap'd buffer, because the OS is likely to give us extra bytes to pad out the final page size. So instead, let's read into a heap buffer.

As a bonus, this also makes it possible to write tests with empty bases, as mmap() will complain about a zero-length map.

This is based on a patch by Jann Horn <jannh@google.com> which actually aligned the data at the end of a page, and followed it with another page marked with mprotect(). That would detect problems even without a tool like ASan, but it was significantly more complex and may have introduced portability problems. By comparison, this approach pushes the complexity onto existing memory-checking tools.

Note that this could be done even more simply by using strbuf_read_file(), but that would defeat the purpose: strbufs generally overallocate (and at the very least include a trailing NUL which we do not care about), which would defeat most memory checkers.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/helper/test-delta.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/t/helper/test-delta.c b/t/helper/test-delta.c
index 34c7259248..e749a49c88 100644
--- a/t/helper/test-delta.c
+++ b/t/helper/test-delta.c
@@ -34,8 +34,8 @@ int cmd__delta(int argc, const char **argv)
 		return 1;
 	}
 	from_size = st.st_size;
-	from_buf = mmap(NULL, from_size, PROT_READ, MAP_PRIVATE, fd, 0);
-	if (from_buf == MAP_FAILED) {
+	from_buf = xmalloc(from_size);
+	if (read_in_full(fd, from_buf, from_size) < 0) {
 		perror(argv[2]);
 		close(fd);
 		return 1;
@@ -48,8 +48,8 @@ int cmd__delta(int argc, const char **argv)
 		return 1;
 	}
 	data_size = st.st_size;
-	data_buf = mmap(NULL, data_size, PROT_READ, MAP_PRIVATE, fd, 0);
-	if (data_buf == MAP_FAILED) {
+	data_buf = xmalloc(data_size);
+	if (read_in_full(fd, data_buf, data_size) < 0) {
 		perror(argv[3]);
 		close(fd);
 		return 1;
-- 
2.19.0.rc1.539.g3876d0831e
Previous: Jeff KingNext: Jeff King
Message 13 of 31 in “patch-delta: fix oob read”
  1. 1/3 patch-delta: fix oob readJann Horn, Aug 29, 2018
  2. 2/3 t/helper/test-delta: segfault on OOB accessJann Horn, Aug 29, 2018
  3. Jeff KingAug 29, 2018
  4. Jann HornAug 29, 2018
  5. Jeff KingAug 29, 2018
  6. Jeff KingAug 29, 2018
  7. 3/3 t5303: add tests for corrupted deltasJann Horn, Aug 29, 2018
  8. Jeff KingAug 29, 2018
  9. Jeff KingAug 29, 2018
  10. Jeff KingAug 29, 2018
  11. Jeff KingAug 29, 2018
  12. 0/5 handle corruption in patch-deltaJeff King, Aug 30, 2018
  13. 1/5 test-delta: read input into a heap bufferJeff King, Aug 30, 2018
  14. 2/5 t5303: test some corrupt deltasJeff King, Aug 30, 2018
  15. Junio C HamanoAug 30, 2018
  16. Jeff KingAug 30, 2018
  17. Jeff KingAug 30, 2018
  18. Junio C HamanoAug 30, 2018
  19. Jeff KingAug 30, 2018
  20. Johannes SchindelinAug 31, 2018
  21. Junio C HamanoAug 31, 2018
  22. Jeff KingAug 31, 2018
  23. Junio C HamanoAug 31, 2018
  24. Johannes SchindelinAug 31, 2018
  25. Jeff KingAug 31, 2018
  26. Junio C HamanoAug 31, 2018
  27. 3/5 patch-delta: fix oob readJeff King, Aug 30, 2018
  28. 4/5 patch-delta: consistently report corruptionJeff King, Aug 30, 2018
  29. 5/5 patch-delta: handle truncated copy parametersJeff King, Aug 30, 2018
  30. Jann HornAug 30, 2018
  31. Nicolas PitreAug 30, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.