git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 05/20] midx: check size of oid lookup chunk

From
Jeff King <peff@peff.net>
Date
Oct 9, 2023, 21:02 UTC
Message-ID
<20231009210203.GE3282181@coredump.intra.peff.net>
In-Reply-To
<20231009205544.GA3281950@coredump.intra.peff.net>

When reading an on-disk multi-pack-index, we take the number of objects in the midx from the final value of the fanout table. But we just blindly assume that the chunk containing the actual oid entries is the correct size. This can lead to us reading out-of-bounds memory if the lookup chunk is too small (or if the fanout is corrupted; when they don't agree we cannot tell which one is wrong).

Note that we bump the assignment of m->num_objects into the fanout parser callback, so that it's set when we parse the lookup table (otherwise we'd have to manually record the lookup table size and check it later).

Signed-off-by: Jeff King <peff@peff.net>
---
As an aside, the first hunk of the diff annoyingly slides the "return 0"
into the wrong spot. I thought this was our heuristics gone wrong, but I
suspect it's actually the shortest diff because of the one-line change
in midx_read_oid_fanout(), which would otherwise require extra context
to split.
 midx.c                      | 18 +++++++++++++++---
 t/t5319-multi-pack-index.sh | 10 ++++++++++
 2 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/midx.c b/midx.c
index 21d7dd15ef..62e4c03e79 100644
--- a/midx.c
+++ b/midx.c
@@ -71,6 +71,20 @@ static int midx_read_oid_fanout(const unsigned char *chunk_start,
 		error(_("multi-pack-index OID fanout is of the wrong size"));
 		return 1;
 	}
+	m->num_objects = ntohl(m->chunk_oid_fanout[255]);
+	return 0;
+}
+
+static int midx_read_oid_lookup(const unsigned char *chunk_start,
+				size_t chunk_size, void *data)
+{
+	struct multi_pack_index *m = data;
+	m->chunk_oid_lookup = chunk_start;
+
+	if (chunk_size != st_mult(m->hash_len, m->num_objects)) {
+		error(_("multi-pack-index OID lookup chunk is the wrong size"));
+		return 1;
+	}
 	return 0;
 }
 
@@ -147,7 +161,7 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local
 		die(_("multi-pack-index required pack-name chunk missing or corrupted"));
 	if (read_chunk(cf, MIDX_CHUNKID_OIDFANOUT, midx_read_oid_fanout, m))
 		die(_("multi-pack-index required OID fanout chunk missing or corrupted"));
-	if (pair_chunk_unsafe(cf, MIDX_CHUNKID_OIDLOOKUP, &m->chunk_oid_lookup))
+	if (read_chunk(cf, MIDX_CHUNKID_OIDLOOKUP, midx_read_oid_lookup, m))
 		die(_("multi-pack-index required OID lookup chunk missing or corrupted"));
 	if (pair_chunk_unsafe(cf, MIDX_CHUNKID_OBJECTOFFSETS, &m->chunk_object_offsets))
 		die(_("multi-pack-index required object offsets chunk missing or corrupted"));
@@ -157,8 +171,6 @@ struct multi_pack_index *load_multi_pack_index(const char *object_dir, int local
 	if (git_env_bool("GIT_TEST_MIDX_READ_RIDX", 1))
 		pair_chunk_unsafe(cf, MIDX_CHUNKID_REVINDEX, &m->chunk_revindex);
 
-	m->num_objects = ntohl(m->chunk_oid_fanout[255]);
-
 	CALLOC_ARRAY(m->pack_names, m->num_packs);
 	CALLOC_ARRAY(m->packs, m->num_packs);
 
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index b8fe85aeba..2722e495b2 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -1073,4 +1073,14 @@ test_expect_success 'reader notices too-small oid fanout chunk' '
 	test_cmp expect err
 '
 
+test_expect_success 'reader notices too-small oid lookup chunk' '
+	corrupt_chunk OIDL clear 00000000 &&
+	test_must_fail git log 2>err &&
+	cat >expect <<-\EOF &&
+	error: multi-pack-index OID lookup chunk is the wrong size
+	fatal: multi-pack-index required OID lookup chunk missing or corrupted
+	EOF
+	test_cmp expect err
+'
+
 test_done
-- 
2.42.0.884.g35e1fe1a6a
Previous: Jeff KingNext: Jeff King
Message 14 of 67 in “bounds-checks for chunk-based files”
  1. 0/20 bounds-checks for chunk-based filesJeff King, Oct 9, 2023
  2. 01/20 chunk-format: note that pair_chunk() is unsafeJeff King, Oct 9, 2023
  3. Taylor BlauOct 10, 2023
  4. Jeff KingOct 11, 2023
  5. 02/20 t: add library for munging chunk-format filesJeff King, Oct 9, 2023
  6. Taylor BlauOct 10, 2023
  7. 03/20 midx: stop ignoring malformed oid fanout chunkJeff King, Oct 9, 2023
  8. Taylor BlauOct 10, 2023
  9. Jeff KingOct 11, 2023
  10. 04/20 commit-graph: check size of oid fanout chunkJeff King, Oct 9, 2023
  11. Taylor BlauOct 11, 2023
  12. Taylor BlauOct 11, 2023
  13. Jeff KingOct 11, 2023
  14. 05/20 midx: check size of oid lookup chunkJeff King, Oct 9, 2023
  15. 06/20 commit-graph: check consistency of fanout tableJeff King, Oct 9, 2023
  16. Taylor BlauOct 11, 2023
  17. 07/20 midx: check size of pack names chunkJeff King, Oct 9, 2023
  18. Taylor BlauOct 11, 2023
  19. Jeff KingOct 11, 2023
  20. 08/20 midx: enforce chunk alignment on readingJeff King, Oct 9, 2023
  21. Taylor BlauOct 11, 2023
  22. Taylor BlauOct 11, 2023
  23. Jeff KingOct 11, 2023
  24. 09/20 midx: check size of object offset chunkJeff King, Oct 9, 2023
  25. Taylor BlauOct 11, 2023
  26. 10/20 midx: bounds-check large offset chunkJeff King, Oct 9, 2023
  27. Taylor BlauOct 11, 2023
  28. Jeff KingOct 11, 2023
  29. 11/20 midx: check size of revindex chunkJeff King, Oct 9, 2023
  30. Taylor BlauOct 11, 2023
  31. 12/20 commit-graph: check size of commit data chunkJeff King, Oct 9, 2023
  32. Taylor BlauOct 11, 2023
  33. Jeff KingOct 11, 2023
  34. 13/20 commit-graph: detect out-of-bounds extra-edges pointersJeff King, Oct 9, 2023
  35. Taylor BlauOct 11, 2023
  36. 14/20 commit-graph: bounds-check base graphs chunkJeff King, Oct 9, 2023
  37. Taylor BlauOct 11, 2023
  38. 15/20 commit-graph: check size of generations chunkJeff King, Oct 9, 2023
  39. 16/20 commit-graph: bounds-check generation overflow chunkJeff King, Oct 9, 2023
  40. 17/20 commit-graph: check bounds when accessing BDAT chunkJeff King, Oct 9, 2023
  41. Taylor BlauOct 11, 2023
  42. Jeff KingOct 11, 2023
  43. 18/20 commit-graph: check bounds when accessing BIDX chunkJeff King, Oct 9, 2023
  44. Taylor BlauOct 11, 2023
  45. 19/20 commit-graph: detect out-of-order BIDX offsetsJeff King, Oct 9, 2023
  46. Taylor BlauOct 11, 2023
  47. 20/20 chunk-format: drop pair_chunk_unsafe()Jeff King, Oct 9, 2023
  48. Taylor BlauOct 11, 2023
  49. Jeff KingOct 11, 2023
  50. 0/8 chunk-format: introduce `pair_chunk_expect()` convenience APITaylor Blau, Oct 13, 2023
  51. 1/8 chunk-format: introduce `pair_chunk_expect()` helperTaylor Blau, Oct 13, 2023
  52. 2/8 commit-graph: read `OIDF` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  53. 3/8 commit-graph: read `CDAT` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  54. 4/8 commit-graph: read `GDAT` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  55. 5/8 commit-graph: read `BIDX` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  56. Taylor BlauOct 13, 2023
  57. Junio C HamanoOct 14, 2023
  58. Jeff KingOct 20, 2023
  59. 6/8 midx: read `OIDF` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  60. Junio C HamanoOct 13, 2023
  61. 7/8 midx: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  62. 8/8 midx: read `OOFF` chunk with `pair_chunk_expect()`Taylor Blau, Oct 13, 2023
  63. Jeff KingOct 20, 2023
  64. 21/20 t5319: make corrupted large-offset test more robustJeff King, Oct 14, 2023
  65. Junio C HamanoOct 14, 2023
  66. Jeff KingOct 15, 2023
  67. Junio C HamanoOct 15, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.