git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/7] chunk-format: introduce `pair_chunk_expect()` helper

From
LALinus Arver <linusa@google.com>
Date
Jan 15, 2024, 22:31 UTC
Message-ID
<owly8r4qi5zj.fsf@fine.c.googlers.com>
In-Reply-To
<af5fe3b7237caeba8f970e967933db96c83a230e.1699569246.git.me@ttaylorr.com>
Taylor Blau <me@ttaylorr.com> writes:
Show 11 quoted lines
> +static int pair_chunk_expect_fn(const unsigned char *chunk_start,
> +				size_t chunk_size,
> +				void *data)
> +{
> +	struct pair_chunk_data *pcd = data;
> +	if (chunk_size / pcd->record_size != pcd->record_nr)
> +		return -1;
> +	*pcd->p = chunk_start;
> +	return 0;
> +}
> +

I don't think this function should assume anything about the inputs (chunk_size, record_size, nor record_nr). If we are asking the helper to be the common tool for multiple locations, it should assume even less about what these inputs could look like.

For example, if record_size is 0 this will lead to a divide-by-zero. Likewise, if record_nr is zero it doesn't make sense to check if chunk_size fits into record_size * record_nr.

And even if there are no (unexpected) zero-values involved, shouldn't we also check for nonsensical comparisons, such as if chunk_size is smaller than record_size?

I think there are several possibilities:

CHUNK_MISSING (chunk_size == 0) CHUNK_TOO_SMALL (chunk_size < record_size) CHUNK_OK (chunk_size == record_nr * record_size) CHUNK_TOO_BIG (chunk_size > record_size, record_nr * record_size < chunk_size)

And for the CHUNK_TOO_BIG case there are two possibilities --- the
leftover parts of chunk_size that are not accounted by "record_nr *
record_size" are (or are not) "aligned" such that increasing the
record_size by some positive increment could exactly match the
chunk_size. For example, chunk_size = 128, record_size = 8, record_nr =
8. In this case the records account for 64 bytes so we have 128 - 64 =
64 bytes left over, and simply increasing record_nr to 16 could account
for every bytes in chunk_size. If chunk_size in this example was 120 or
130, there would be bytes left over that cannot be accounted for by
adjusting record_size. This divisibility-of-leftover-bytes check could
be done with '%' as mentioned already by others.
So in summary there appear to be the following possibilities:

CHUNK_MISSING CHUNK_TOO_SMALL CHUNK_OK CHUNK_TOO_BIG_ALIGNED CHUNK_TOO_BIG_MISALIGNED

(on top of the cases where record_* inputs are zero).

And it seems prudent to treat each of the not-OK cases differently (including how we report error messages) once we know which category we fall into.

Previous: Jeff KingNext: Linus Arver
Message 29 of 39 in “some more chunk-file bounds-checks fixes”
  1. 0/9 some more chunk-file bounds-checks fixesJeff King, Nov 9, 2023
  2. 1/9 commit-graph: handle overflow in chunk_size checksJeff King, Nov 9, 2023
  3. Taylor BlauNov 9, 2023
  4. Jeff KingNov 9, 2023
  5. 2/9 midx: check consistency of fanout tableJeff King, Nov 9, 2023
  6. 3/9 commit-graph: drop redundant call to "lite" verificationJeff King, Nov 9, 2023
  7. 4/9 commit-graph: clarify missing-chunk error messagesJeff King, Nov 9, 2023
  8. 5/9 commit-graph: abort as soon as we see a bogus chunkJeff King, Nov 9, 2023
  9. Taylor BlauNov 9, 2023
  10. 6/9 commit-graph: use fanout value for graph sizeJeff King, Nov 9, 2023
  11. Taylor BlauNov 9, 2023
  12. Jeff KingNov 9, 2023
  13. Taylor BlauNov 9, 2023
  14. Jeff KingNov 10, 2023
  15. 7/9 commit-graph: check order while reading fanout chunkJeff King, Nov 9, 2023
  16. 8/9 commit-graph: drop verify_commit_graph_lite()Jeff King, Nov 9, 2023
  17. 9/9 commit-graph: mark chunk error messages for translationJeff King, Nov 9, 2023
  18. Taylor BlauNov 9, 2023
  19. 0/7 chunk-format: introduce `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  20. 1/7 chunk-format: introduce `pair_chunk_expect()` helperTaylor Blau, Nov 9, 2023
  21. Junio C HamanoNov 10, 2023
  22. Taylor BlauNov 10, 2023
  23. Jeff KingNov 10, 2023
  24. Junio C HamanoNov 10, 2023
  25. Junio C HamanoNov 10, 2023
  26. Jeff KingNov 10, 2023
  27. Jeff KingNov 10, 2023
  28. Jeff KingNov 10, 2023
  29. Linus ArverJan 15, 2024
  30. Linus ArverJan 15, 2024
  31. Jeff KingJan 16, 2024
  32. Linus ArverJan 18, 2024
  33. 2/7 commit-graph: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  34. Jeff KingNov 10, 2023
  35. 3/7 commit-graph: read `CDAT` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  36. 4/7 commit-graph: read `GDAT` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  37. 5/7 commit-graph: read `BIDX` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  38. 6/7 midx: read `OIDL` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023
  39. 7/7 midx: read `OOFF` chunk with `pair_chunk_expect()`Taylor Blau, Nov 9, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.