git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 0/4] use size_t for a few more object and buffer sizes

From
Elia Pinto <gitter.spiros@gmail.com>
Date
Oct 30, 2026, 09:58 UTC
Message-ID
<cover.1793352943.git.gitter.spiros@gmail.com>
In-Reply-To
<20261021101640.329990-1-gitter.spiros@gmail.com>

On 64-bit Windows "long" is still 32 bits wide, so a size_t that goes through an "unsigned long" loses anything above 4 GiB. Like v1, this series widens three such places: the length taken by the object write API (patch 1), the allocation size of the word-diff buffers (patch 2) and the buffer sizes in convert.c (patch 3). Patch 4 is new. The cover letter of v1 explains how these were found:

  https://lore.kernel.org/git/20261021101640.329990-1-gitter.spiros@gmail.com/
Changes since v1, all following Patrick's review (thanks!):
 - Patch 1 adds the test Patrick asked for. The existing "hash
   filtered files over 4GB correctly" test in t1007 only hashes; the
   new one writes the object with "git hash-object -w" and checks its
   object ID and size. It is EXPENSIVE and SIZE_T_IS_64BIT, like its
   neighbours. Truncating the length to 32 bits in
   odb_write_object_ext() on Linux, to mimic Windows, makes it fail.
 - Patch 4 is new and answers Patrick's question on patch 3: struct
   text_stat still counted in "unsigned", and count_ident() in an
   "int". Note that this one is not specific to LLP64, as "unsigned"
   is 32 bits everywhere; it is in this series because it came up
   in the review and touches the same code. A wrapped counter only
   leads to a wrong decision (e.g. no CRLF conversion for a file with
   exactly 2^32 lone LFs), not to a memory safety problem, because
   the counts are only size hints for strbuf_grow().
 - Rebased on current master; no conflicts.
Elia Pinto (4):
  odb: use size_t for the length of objects to write
  diff: use size_t for the allocation size of word-diff buffers
  convert: use size_t for buffer sizes
  convert: use size_t for text statistics and ident counts
 attr.c                 |  2 +-
 convert.c              | 24 ++++++++++++------------
 diff.c                 |  4 ++--
 odb.c                  |  2 +-
 odb.h                  |  4 ++--
 odb/source-inmemory.c  |  2 +-
 odb/source-loose.c     |  2 +-
 odb/source.h           |  2 +-
 read-cache-ll.h        |  2 +-
 read-cache.c           |  4 ++--
 t/t1007-hash-object.sh | 13 +++++++++++++
 11 files changed, 37 insertions(+), 24 deletions(-)
Range-diff against v1:
1:  0fff9532f9 ! 1:  8fcb9f10f1 odb: use size_t for the length of objects to write
    @@ Commit message
         source.
     
         The existing "hash filtered files over 4GB correctly" test in t1007
    -    only hashes. Running the same with "git hash-object -w" on Windows
    -    shows the problem, but needs another pass over a 5GiB file, so no new
    -    test is added here.
    +    only hashes, so add a variant that writes the object with "git
    +    hash-object -w" and checks both its object ID and its size. Like the
    +    other tests over 4GB, it needs EXPENSIVE and SIZE_T_IS_64BIT. Truncating
    +    the length to 32 bits in odb_write_object_ext(), as happens on Windows
    +    without this patch, makes it fail with a different object ID.
     
         Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
     
    @@ odb/source.h: static inline int odb_source_freshen_object(struct odb_source *sou
      					  enum object_type type,
      					  const struct object_id *oid,
      					  const struct object_id *compat_oid,
    +
    + ## t/t1007-hash-object.sh ##
    +@@ t/t1007-hash-object.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \
    + 	test_cmp expect actual
    + '
    + 
    ++test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \
    ++		'write filtered files over 4GB correctly' '
    ++	{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&
    ++	test_oid large5GB >expect &&
    ++	test_config filter.null-filter.clean "cat" &&
    ++	echo "big filter=null-filter" >.gitattributes &&
    ++	git hash-object -w -- big >actual &&
    ++	test_cmp expect actual &&
    ++	echo $((5*1024*1024*1024)) >expect &&
    ++	git cat-file -s $(cat actual) >actual &&
    ++	test_cmp expect actual
    ++'
    ++
    + test_done
2:  4ab9569f13 = 2:  42d4eae985 diff: use size_t for the allocation size of word-diff buffers
3:  7ea1a566b5 = 3:  88ca2ceb80 convert: use size_t for buffer sizes
-:  ---------- > 4:  84e9e38a3e convert: use size_t for text statistics and ident counts
base-commit: b9e5cff1803ab24582201ef059516eb16014b9d9
-- 
2.53.0
Previous: Patrick SteinhardtNext: Elia Pinto
Message 8 of 24 in “use size_t for a few more object and buffer sizes”
  1. 0/3 use size_t for a few more object and buffer sizesElia Pinto, Oct 21, 2026
  2. 1/3 odb: use size_t for the length of objects to writeElia Pinto, Oct 21, 2026
  3. Patrick SteinhardtOct 21, 2026
  4. 2/3 diff: use size_t for the allocation size of word-diff buffersElia Pinto, Oct 21, 2026
  5. Patrick SteinhardtOct 21, 2026
  6. 3/3 convert: use size_t for buffer sizesElia Pinto, Oct 21, 2026
  7. Patrick SteinhardtOct 21, 2026
  8. 0/4 use size_t for a few more object and buffer sizesElia Pinto, Oct 30, 2026
  9. 1/4 odb: use size_t for the length of objects to writeElia Pinto, Oct 30, 2026
  10. Patrick SteinhardtOct 30, 2026
  11. Elia PintoOct 31, 2026
  12. 2/4 diff: use size_t for the allocation size of word-diff buffersElia Pinto, Oct 30, 2026
  13. 3/4 convert: use size_t for buffer sizesElia Pinto, Oct 30, 2026
  14. 4/4 convert: use size_t for text statistics and ident countsElia Pinto, Oct 30, 2026
  15. Patrick SteinhardtOct 30, 2026
  16. Elia PintoOct 31, 2026
  17. Patrick SteinhardtNov 2, 2026
  18. Junio C HamanoNov 2, 2026
  19. Patrick SteinhardtNov 3, 2026
  20. 0/4 use size_t for a few more object and buffer sizesElia Pinto, Oct 31, 2026
  21. 1/4 odb: use size_t for the length of objects to writeElia Pinto, Oct 31, 2026
  22. 2/4 diff: use size_t for the allocation size of word-diff buffersElia Pinto, Oct 31, 2026
  23. 3/4 convert: use size_t for buffer sizesElia Pinto, Oct 31, 2026
  24. 4/4 convert: use size_t for text statistics and ident countsElia Pinto, Oct 31, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.