git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 4/9] cache-tree: avoid strtol() on non-string buffer

From
Junio C Hamano <gitster@pobox.com>
Date
Nov 23, 2025, 18:06 UTC
Message-ID
<xmqqh5ukzkqt.fsf@gitster.g>
In-Reply-To
<633f4d92-c258-45a8-9d32-116c94838e68@gmail.com>
Phillip Wood <phillip.wood123@gmail.com> writes:
Show 14 quoted lines
> All we need to do to accept a single minus sign is s/while/if/
> ...
> If we limit ourselves to accepting a single minus sign then this can become
> 	if (s == *ptr + (sign == -1))
>
> so we need very little in the way of extra code.
> ...
> A generic helper to replace strtol() that takes a length rather than 
> assuming the input is NUL terminated could be useful elsewhere but I'm 
> not sure we need something that complicated here. I do like the fact 
> that overflow does not cause undefined behavior though. Changing ret for 
> "int" to "unsigned" in peff's patch should fix that.
> 
> Thanks
Perhaps.
By the way, an interesting tangent is this.

The only reason why these fields under discussion are stored in textual decimal is pretty much the same as the reason why the object header expresses the byte-length of the payload in textual decimal, i.e., to be independent from the platform natural implementation of "int" type (e.g., endiannness and width), but unlike object files, the index is a local matter (we are prepared for the same directory accessed over NFS from two platforms with different endianness, but we do not recommend network access to a repository in the first place). And a lot more importantly, the total number of the index entries contained within an index file is capped to 2^32-1 (the header has 32-bit count in the network byte order). The total number of subdirectories within a directory or the total number of entries for a level of directory hierarchy that would form a tree object from a slice of the index cannot exceed that number anyway.

And thanks to the design that made cache-tree an optional index extension, we can make cache-tree version 2 where the in-core representation is exactly the same as the current one, but only uses different serialization when writing to and reading from the index file. The new serialization can use 32-bit network byte order integers, or use our own varint.{c,h,rs}, to record these numbers.

A version of Git that knows about that extension could be taught to read from the current cache-tree and convert to a new version, but better yet, it can simply ignore the current cache-tree data in the file, and write the new version when we do need to write the index out with a cache-tree. When such a transparent auto conversion happens, one single invocation of write_index_as_tree() would become more expensive than usual (because the last invocation of the current Git left cache-tree data in the index and usually the next invocation of Git would take advantage of it when it writes a tree, but a new version of Git that uses the v2 format would behave as if there is no cache-tree data in the index and build the tree from scratch. After that happens, the cache-tree data in the new format will be reused and things will continue to work. You could use an older version of Git on such an index file and the same transparent auto conversion will take care of the transition.

Previous: Phillip WoodNext: Jeff King
Message 40 of 64 in “asan bonanza”
  1. 0/9 asan bonanzaJeff King, Nov 12, 2025
  2. 1/9 compat/mmap: mark unused argument in git_munmap()Jeff King, Nov 12, 2025
  3. 2/9 pack-bitmap: handle name-hash lookups in incremental bitmapsJeff King, Nov 12, 2025
  4. Patrick SteinhardtNov 12, 2025
  5. Taylor BlauNov 13, 2025
  6. Jeff KingNov 18, 2025
  7. 3/9 Makefile: turn on NO_MMAP when building with ASanJeff King, Nov 12, 2025
  8. Collin FunkNov 12, 2025
  9. Jeff KingNov 12, 2025
  10. Collin FunkNov 12, 2025
  11. Patrick SteinhardtNov 12, 2025
  12. Taylor BlauNov 13, 2025
  13. Patrick SteinhardtNov 13, 2025
  14. Jeff KingNov 18, 2025
  15. Junio C HamanoNov 13, 2025
  16. Patrick SteinhardtNov 14, 2025
  17. Jeff KingNov 15, 2025
  18. 4/9 cache-tree: avoid strtol() on non-string bufferJeff King, Nov 12, 2025
  19. Patrick SteinhardtNov 12, 2025
  20. Taylor BlauNov 13, 2025
  21. Jeff KingNov 18, 2025
  22. Jeff KingNov 18, 2025
  23. 5/9 fsck: assert newline presence in fsck_ident()Jeff King, Nov 12, 2025
  24. 6/9 fsck: avoid strcspn() in fsck_ident()Jeff King, Nov 12, 2025
  25. 7/9 fsck: remove redundant date timestamp checkJeff King, Nov 12, 2025
  26. 8/9 fsck: avoid parse_timestamp() on buffer that isn't NUL-terminatedJeff King, Nov 12, 2025
  27. Patrick SteinhardtNov 12, 2025
  28. Junio C HamanoNov 12, 2025
  29. Jeff KingNov 15, 2025
  30. 9/9 t: enable ASan's strict_string_checks optionJeff King, Nov 12, 2025
  31. Taylor BlauNov 13, 2025
  32. 0/9 asan bonanzaJeff King, Nov 18, 2025
  33. 1/9 compat/mmap: mark unused argument in git_munmap()Jeff King, Nov 18, 2025
  34. 2/9 pack-bitmap: handle name-hash lookups in incremental bitmapsJeff King, Nov 18, 2025
  35. 3/9 Makefile: turn on NO_MMAP when building with ASanJeff King, Nov 18, 2025
  36. 4/9 cache-tree: avoid strtol() on non-string bufferJeff King, Nov 18, 2025
  37. Phillip WoodNov 18, 2025
  38. Junio C HamanoNov 23, 2025
  39. Phillip WoodNov 23, 2025
  40. Junio C HamanoNov 23, 2025
  41. Jeff KingNov 24, 2025
  42. Junio C HamanoNov 24, 2025
  43. Jeff KingNov 26, 2025
  44. Junio C HamanoNov 26, 2025
  45. 0/4 more robust functions for parsing int from bufJeff King, Nov 30, 2025
  46. 1/4 parse: prefer bool to int for boolean returnsJeff King, Nov 30, 2025
  47. Patrick SteinhardtDec 4, 2025
  48. 2/4 parse: add functions for parsing from non-string buffersJeff King, Nov 30, 2025
  49. my complaints with clarJeff King, Nov 30, 2025
  50. Phillip WoodDec 1, 2025
  51. Patrick SteinhardtDec 4, 2025
  52. Jeff KingDec 5, 2025
  53. Patrick SteinhardtDec 4, 2025
  54. Phillip WoodDec 5, 2025
  55. Junio C HamanoJan 20, 2026
  56. Jeff KingJan 21, 2026
  57. 3/4 cache-tree: use parse_int_from_buf()Jeff King, Nov 30, 2025
  58. 4/4 fsck: use parse_unsigned_from_buf() for parsing timestampJeff King, Nov 30, 2025
  59. 5/9 fsck: assert newline presence in fsck_ident()Jeff King, Nov 18, 2025
  60. 6/9 fsck: avoid strcspn() in fsck_ident()Jeff King, Nov 18, 2025
  61. 7/9 fsck: remove redundant date timestamp checkJeff King, Nov 18, 2025
  62. 8/9 fsck: avoid parse_timestamp() on buffer that isn't NUL-terminatedJeff King, Nov 18, 2025
  63. 9/9 t: enable ASan's strict_string_checks optionJeff King, Nov 18, 2025
  64. Junio C HamanoNov 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.