git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/9] commit-graph: handle overflow in chunk_size checks

From
Jeff King <peff@peff.net>
Date
Nov 9, 2023, 07:09 UTC
Message-ID
<20231109070948.GA2698043@coredump.intra.peff.net>
In-Reply-To
<20231109070310.GA2697602@coredump.intra.peff.net>

We check the size of chunks with fixed records by multiplying the width of each record by the number of commits in the file. Like:

  if (chunk_size != g->num_commits * GRAPH_DATA_WIDTH)

If this multiplication overflows, we may not notice a chunk is too small (which could later lead to out-of-bound reads).

In the current code this is only possible for the CDAT chunk, but the reasons are quite subtle. We compute g->num_commits by dividing the size of the OIDL chunk by the hash length (since it consists of a bunch of hashes). So we know that any size_t multiplication that uses a value smaller than the hash length cannot overflow. And the CDAT records are the only ones that are larger (the others are just 4-byte records). So it's worth fixing all of these, to make it clear that they're not subject to overflow (without having to reason about seemingly unrelated code).

The obvious thing to do is add an st_mult(), like:
  if (chunk_size != st_mult(g->num_commits, GRAPH_DATA_WIDTH))

And that certainly works, but it has one downside: if we detect an overflow, we'll immediately die(). But the commit graph is an optional file; if we run into other problems loading it, we'll generally return an error and fall back to accessing the full objects. Using st_mult() means a malformed file will abort the whole process.

So instead, we can do a division like this:
  if (chunk_size / GRAPH_DATA_WIDTH != g->num_commits)

where there's no possibility of overflow. We do lose a little bit of precision; due to integer division truncation we'd allow up to an extra GRAPH_DATA_WIDTH-1 bytes of data in the chunk. That's OK. Our main goal here is making sure we don't have too _few_ bytes, which would cause an out-of-bounds read (we could actually replace our "!=" with "<", but I think it's worth being a little pedantic, as a large mismatch could be a sign of other problems).

I didn't add a test here. We'd need to generate a very large graph file in order to get g->num_commits large enough to cause an overflow. And a later patch in this series will use this same division technique in a way that is much easier to trigger in the tests.

Signed-off-by: Jeff King <peff@peff.net>
---
There are still several st_mult() calls within commit-graph.c, unrelated
to my jk/chunk-bounds series. I suspect they're all redundant, as the
chunk-size checks give a stricter bound. But checking and removing them
is a separate topic.

I think the midx code has similar st_mult() problems, but it's quite happy to die() on any error already. So I've left auditing that for another day.

Another alternative to the division is introducing an st_mult() variant like:

  if (!st_mult(&out, &a, &b))
        return error(...);

We may want to go there in the long run as part of lib-ification, but I didn't want to get into it for this series.

 commit-graph.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/commit-graph.c b/commit-graph.c
index ee66098e07..5d7d7a89e5 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -344,7 +344,7 @@ static int graph_read_commit_data(const unsigned char *chunk_start,
 				  size_t chunk_size, void *data)
 {
 	struct commit_graph *g = data;
-	if (chunk_size != g->num_commits * GRAPH_DATA_WIDTH)
+	if (chunk_size / GRAPH_DATA_WIDTH != g->num_commits)
 		return error("commit-graph commit data chunk is wrong size");
 	g->chunk_commit_data = chunk_start;
 	return 0;
@@ -354,7 +354,7 @@ static int graph_read_generation_data(const unsigned char *chunk_start,
 				      size_t chunk_size, void *data)
 {
 	struct commit_graph *g = data;
-	if (chunk_size != g->num_commits * sizeof(uint32_t))
+	if (chunk_size / sizeof(uint32_t) != g->num_commits)
 		return error("commit-graph generations chunk is wrong size");
 	g->chunk_generation_data = chunk_start;
 	return 0;
@@ -364,7 +364,7 @@ static int graph_read_bloom_index(const unsigned char *chunk_start,
 				  size_t chunk_size, void *data)
 {
 	struct commit_graph *g = data;
-	if (chunk_size != g->num_commits * 4) {
+	if (chunk_size / 4 != g->num_commits) {
 		warning("commit-graph changed-path index chunk is too small");
 		return -1;
 	}
-- 
2.43.0.rc1.572.g273fc7bed6
Previous: Jeff KingNext: Taylor Blau
Message 2 of 39 in “some more chunk-file bounds-checks fixes”
  1. 0/9 some more chunk-file bounds-checks fixesJeff King, Nov 9, 2023
  2. 1/9 commit-graph: handle overflow in chunk_size checksJeff King, Nov 9, 2023
  3. Taylor BlauNov 9, 2023
  4. Jeff KingNov 9, 2023
  5. 2/9 midx: check consistency of fanout tableJeff King, Nov 9, 2023
  6. 3/9 commit-graph: drop redundant call to "lite" verificationJeff King, Nov 9, 2023
  7. 4/9 commit-graph: clarify missing-chunk error messagesJeff King, Nov 9, 2023
  8. 5/9 commit-graph: abort as soon as we see a bogus chunkJeff King, Nov 9, 2023
  9. Taylor BlauNov 9, 2023
  10. 6/9 commit-graph: use fanout value for graph sizeJeff King, Nov 9, 2023
  11. Taylor BlauNov 9, 2023
  12. Jeff KingNov 9, 2023
  13. Taylor BlauNov 9, 2023
  14. Jeff KingNov 10, 2023
  15. 7/9 commit-graph: check order while reading fanout chunkJeff King, Nov 9, 2023
  16. 8/9 commit-graph: drop verify_commit_graph_lite()Jeff King, Nov 9, 2023
  17. 9/9 commit-graph: mark chunk error messages for translationJeff King, Nov 9, 2023
  18. Taylor BlauNov 9, 2023
  19. 0/7 chunk-format: introduce `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  20. 1/7 chunk-format: introduce `pair_chunk_expect()` helperTaylor Blau, Nov 9, 2023
  21. Junio C HamanoNov 10, 2023
  22. Taylor BlauNov 10, 2023
  23. Jeff KingNov 10, 2023
  24. Junio C HamanoNov 10, 2023
  25. Junio C HamanoNov 10, 2023
  26. Jeff KingNov 10, 2023
  27. Jeff KingNov 10, 2023
  28. Jeff KingNov 10, 2023
  29. Linus ArverJan 15, 2024
  30. Linus ArverJan 15, 2024
  31. Jeff KingJan 16, 2024
  32. Linus ArverJan 18, 2024
  33. 2/7 commit-graph: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  34. Jeff KingNov 10, 2023
  35. 3/7 commit-graph: read `CDAT` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  36. 4/7 commit-graph: read `GDAT` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  37. 5/7 commit-graph: read `BIDX` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  38. 6/7 midx: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  39. 7/7 midx: read `OOFF` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.