git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 02/20] packfile.c: prevent overflow in `load_idx()`

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 14, 2023, 16:29 UTC
Message-ID
<xmqqfs5qqww9.fsf@gitster.g>
In-Reply-To
<ZLCc3mRbdXPllpAp@nand.local>
Taylor Blau <me@ttaylorr.com> writes:
Show 12 quoted lines
> On Thu, Jul 13, 2023 at 10:24:53AM -0400, Taylor Blau wrote:
>> On Thu, Jul 13, 2023 at 09:21:55AM +0100, Phillip Wood wrote:
>> > p->crc_offset is a uint32_t so we're still prone to truncation here unless
>> > we change the crc_offset member of struct packed_git to be a size_t. I
>> > haven't checked if the other users of crc_offset would need adjusting if we
>> > change its type.
>>
>> Thanks for spotting. Luckily, this should be a straightforward change:
>
> Here's a replacement patch which changes the type of `crc_offset`. If
> there end up being other review comments, I'll fold this into the next
> round.

The code change to use st_add() and st_mult() is the same as before, and the type of .crc_offset member changes, both of which is not unexpected.

In the meantime I will replace the copy of [2/20] I have with this one.

Thanks, both.
Show 52 quoted lines
> --- 8< ---
> Subject: [PATCH] packfile.c: prevent overflow in `load_idx()`
>
> Prevent an overflow when locating a pack's CRC offset when the number
> of packed items is greater than 2^32-1/hashsz by guarding the
> computation with an `st_mult()`.
>
> Note that to avoid truncating the result, the `crc_offset` member must
> itself become a `size_t`. The only usage of this variable (besides the
> assignment in `load_idx()`) is in `read_v2_anomalous_offsets()` in the
> index-pack code. There we use the `crc_offset` as a pointer offset, so
> we are already equipped to handle the type change.
>
> Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> Signed-off-by: Taylor Blau <me@ttaylorr.com>
> ---
>  object-store-ll.h | 2 +-
>  packfile.c        | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/object-store-ll.h b/object-store-ll.h
> index e8f22cdb1b..26a3895c82 100644
> --- a/object-store-ll.h
> +++ b/object-store-ll.h
> @@ -106,7 +106,7 @@ struct packed_git {
>  	const void *index_data;
>  	size_t index_size;
>  	uint32_t num_objects;
> -	uint32_t crc_offset;
> +	size_t crc_offset;
>  	struct oidset bad_objects;
>  	int index_version;
>  	time_t mtime;
> diff --git a/packfile.c b/packfile.c
> index 89220f0e03..70acf1694b 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -186,7 +186,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map,
>  		     */
>  		    (sizeof(off_t) <= 4))
>  			return error("pack too large for current definition of off_t in %s", path);
> -		p->crc_offset = 8 + 4 * 256 + nr * hashsz;
> +		p->crc_offset = st_add(8 + 4 * 256, st_mult(nr, hashsz));
>  	}
>
>  	p->index_version = version;
> --
> 2.41.0.329.g0a1adfae833
> --- >8 ---
>
> Thanks,
> Taylor
Previous: Phillip WoodNext: Phillip Wood
Message 8 of 28 in “guard object lookups against 32-bit overflow”
  1. 00/20 guard object lookups against 32-bit overflowTaylor Blau, Jul 12, 2023
  2. 01/20 packfile.c: prevent overflow in `nth_packed_object_id()`Taylor Blau, Jul 12, 2023
  3. 02/20 packfile.c: prevent overflow in `load_idx()`Taylor Blau, Jul 12, 2023
  4. Phillip WoodJul 13, 2023
  5. Taylor BlauJul 13, 2023
  6. Taylor BlauJul 14, 2023
  7. Phillip WoodJul 14, 2023
  8. Junio C HamanoJul 14, 2023
  9. Phillip WoodJul 14, 2023
  10. Junio C HamanoJul 13, 2023
  11. 03/20 packfile.c: use checked arithmetic in `nth_packed_object_offset()`Taylor Blau, Jul 12, 2023
  12. 04/20 midx.c: use `size_t`'s for fanout nr and allocTaylor Blau, Jul 12, 2023
  13. 05/20 midx.c: prevent overflow in `nth_midxed_object_oid()`Taylor Blau, Jul 12, 2023
  14. 06/20 midx.c: prevent overflow in `nth_midxed_offset()`Taylor Blau, Jul 12, 2023
  15. 08/20 midx.c: prevent overflow in `write_midx_internal()`Taylor Blau, Jul 12, 2023
  16. 09/20 midx.c: prevent overflow in `fill_included_packs_batch()`Taylor Blau, Jul 12, 2023
  17. 10/20 pack-bitmap.c: ensure that eindex lookups don't overflowTaylor Blau, Jul 12, 2023
  18. 11/20 commit-graph.c: prevent overflow in `write_commit_graph_file()`Taylor Blau, Jul 12, 2023
  19. 12/20 commit-graph.c: prevent overflow in add_graph_to_chain()Taylor Blau, Jul 12, 2023
  20. 13/20 commit-graph.c: prevent overflow in `load_oid_from_graph()`Taylor Blau, Jul 12, 2023
  21. 14/20 commit-graph.c: prevent overflow in `fill_commit_graph_info()`Taylor Blau, Jul 12, 2023
  22. 15/20 commit-graph.c: prevent overflow in `fill_commit_in_graph()`Taylor Blau, Jul 12, 2023
  23. 16/20 commit-graph.c: prevent overflow in `load_tree_for_commit()`Taylor Blau, Jul 12, 2023
  24. 17/20 commit-graph.c: prevent overflow in `split_graph_merge_strategy()`Taylor Blau, Jul 12, 2023
  25. 18/20 commit-graph.c: prevent overflow in `merge_commit_graph()`Taylor Blau, Jul 12, 2023
  26. 19/20 commit-graph.c: prevent overflow in `write_commit_graph()`Taylor Blau, Jul 12, 2023
  27. 20/20 commit-graph.c: prevent overflow in `verify_commit_graph()`Taylor Blau, Jul 12, 2023
  28. 07/20 midx.c: store `nr`, `alloc` variables as `size_t`'sTaylor Blau, Jul 12, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.