git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/6] pack-bitmap.c: use `bitmap_index_seek()` where possible

From
Jeff King <peff@peff.net>
Date
Mar 24, 2023, 18:35 UTC
Message-ID
<20230324183514.GB536252@coredump.intra.peff.net>
In-Reply-To
<8a7cf287-421c-9a7f-0788-c5e219e6a8e3@github.com>
On Fri, Mar 24, 2023 at 02:06:43PM -0400, Derrick Stolee wrote:
Show 18 quoted lines
> >> +	bitmap_index_seek(index, header_size, SEEK_CUR);
> >>  	return 0;
> >>  }
> > 
> > Likewise this function already has bounds checks at the top:
> > 
> > 	if (index->map_size < header_size + the_hash_algo->rawsz)
> > 		return error(_("corrupted bitmap index (too small)"));
> > 
> > I'd be perfectly happy if we swapped that our for checking the bounds on
> > individual reads, but the extra checking in the seek step here just
> > seems redundant (and again, too late).
> 
> I think it would be nice to replace all of these custom bounds
> checks with a check within bitmap_index_seek() and error conditions
> done in response to an error code returned by that method. It keeps
> the code more consistent in the potential future of changing the
> amount to move the map_pos and the amount checked in these conditions.

Yeah, that's what I was getting at. But doing it at seek time is too late. We'll have just read off the end of the array.

You really need an interface more like "make sure there are N bytes for me to read at offset X". Then you can read and advance past them.

For individual items where you want to copy the bytes out anyway (like a be32) you can have a nice interface like:

  if (read_be32(bitmap_git, &commit_idx_pos) < 0 ||
      read_u8(bitmap_git, &xor_offset) < 0 ||
      read_u8(bitmap_git, &flags) < 0)
	return error("truncated bitmap entry");

But given that there is only one spot that calls these, that kind of refactoring might not be worth it (right now it just uses the magic number "6" right before grabbing the data).

-Peff
Previous: Derrick StoleeNext: Junio C Hamano
Message 20 of 29 in “pack-bitmap: miscellaneous mmap read hardening”
  1. 0/6 pack-bitmap: miscellaneous mmap read hardeningTaylor Blau, Mar 20, 2023
  2. 1/6 pack-bitmap.c: hide bitmap internals in `read_u8()`Taylor Blau, Mar 20, 2023
  3. Jeff KingMar 21, 2023
  4. Derrick StoleeMar 24, 2023
  5. 2/6 pack-bitmap.c: hide bitmap internals in `read_be32()`Taylor Blau, Mar 20, 2023
  6. 3/6 pack-bitmap.c: drop unnecessary 'inline'sTaylor Blau, Mar 20, 2023
  7. Jeff KingMar 21, 2023
  8. 4/6 pack-bitmap.c: factor out manual `map_pos` manipulationTaylor Blau, Mar 20, 2023
  9. Jeff KingMar 21, 2023
  10. Derrick StoleeMar 24, 2023
  11. Jeff KingMar 24, 2023
  12. Taylor BlauMar 24, 2023
  13. Jeff KingMar 25, 2023
  14. Taylor BlauMar 24, 2023
  15. Taylor BlauMar 24, 2023
  16. Taylor BlauMar 24, 2023
  17. 5/6 pack-bitmap.c: use `bitmap_index_seek()` where possibleTaylor Blau, Mar 20, 2023
  18. Jeff KingMar 21, 2023
  19. Derrick StoleeMar 24, 2023
  20. Jeff KingMar 24, 2023
  21. Junio C HamanoMar 24, 2023
  22. Jeff KingMar 24, 2023
  23. Junio C HamanoMar 24, 2023
  24. Taylor BlauMar 24, 2023
  25. 6/6 pack-bitmap.c: factor out `bitmap_index_seek_commit()`Taylor Blau, Mar 20, 2023
  26. Jeff KingMar 21, 2023
  27. Taylor BlauMar 21, 2023
  28. Jeff KingMar 21, 2023
  29. Derrick StoleeMar 24, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.