[PATCH v2 0/4] use size_t for a few more object and buffer sizes
- From
Elia Pinto <gitter.spiros@gmail.com>
- Date
- Oct 30, 2026, 09:58 UTC
- Message-ID
- <cover.1793352943.git.gitter.spiros@gmail.com>
- In-Reply-To
- <20261021101640.329990-1-gitter.spiros@gmail.com>
On 64-bit Windows "long" is still 32 bits wide, so a size_t that goes through an "unsigned long" loses anything above 4 GiB. Like v1, this series widens three such places: the length taken by the object write API (patch 1), the allocation size of the word-diff buffers (patch 2) and the buffer sizes in convert.c (patch 3). Patch 4 is new. The cover letter of v1 explains how these were found:
https://lore.kernel.org/git/20261021101640.329990-1-gitter.spiros@gmail.com/
Changes since v1, all following Patrick's review (thanks!):
- Patch 1 adds the test Patrick asked for. The existing "hash filtered files over 4GB correctly" test in t1007 only hashes; the new one writes the object with "git hash-object -w" and checks its object ID and size. It is EXPENSIVE and SIZE_T_IS_64BIT, like its neighbours. Truncating the length to 32 bits in odb_write_object_ext() on Linux, to mimic Windows, makes it fail.
- Patch 4 is new and answers Patrick's question on patch 3: struct text_stat still counted in "unsigned", and count_ident() in an "int". Note that this one is not specific to LLP64, as "unsigned" is 32 bits everywhere; it is in this series because it came up in the review and touches the same code. A wrapped counter only leads to a wrong decision (e.g. no CRLF conversion for a file with exactly 2^32 lone LFs), not to a memory safety problem, because the counts are only size hints for strbuf_grow().
- Rebased on current master; no conflicts.
Elia Pinto (4): odb: use size_t for the length of objects to write diff: use size_t for the allocation size of word-diff buffers convert: use size_t for buffer sizes convert: use size_t for text statistics and ident counts
attr.c | 2 +- convert.c | 24 ++++++++++++------------ diff.c | 4 ++-- odb.c | 2 +- odb.h | 4 ++-- odb/source-inmemory.c | 2 +- odb/source-loose.c | 2 +- odb/source.h | 2 +- read-cache-ll.h | 2 +- read-cache.c | 4 ++-- t/t1007-hash-object.sh | 13 +++++++++++++ 11 files changed, 37 insertions(+), 24 deletions(-)
Range-diff against v1:
1: 0fff9532f9 ! 1: 8fcb9f10f1 odb: use size_t for the length of objects to write
@@ Commit message
source.
The existing "hash filtered files over 4GB correctly" test in t1007
- only hashes. Running the same with "git hash-object -w" on Windows
- shows the problem, but needs another pass over a 5GiB file, so no new
- test is added here.
+ only hashes, so add a variant that writes the object with "git
+ hash-object -w" and checks both its object ID and its size. Like the
+ other tests over 4GB, it needs EXPENSIVE and SIZE_T_IS_64BIT. Truncating
+ the length to 32 bits in odb_write_object_ext(), as happens on Windows
+ without this patch, makes it fail with a different object ID.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
@@ odb/source.h: static inline int odb_source_freshen_object(struct odb_source *sou
enum object_type type,
const struct object_id *oid,
const struct object_id *compat_oid,
+
+ ## t/t1007-hash-object.sh ##
+@@ t/t1007-hash-object.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \
+ test_cmp expect actual
+ '
+
++test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \
++ 'write filtered files over 4GB correctly' '
++ { test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&
++ test_oid large5GB >expect &&
++ test_config filter.null-filter.clean "cat" &&
++ echo "big filter=null-filter" >.gitattributes &&
++ git hash-object -w -- big >actual &&
++ test_cmp expect actual &&
++ echo $((5*1024*1024*1024)) >expect &&
++ git cat-file -s $(cat actual) >actual &&
++ test_cmp expect actual
++'
++
+ test_done
2: 4ab9569f13 = 2: 42d4eae985 diff: use size_t for the allocation size of word-diff buffers
3: 7ea1a566b5 = 3: 88ca2ceb80 convert: use size_t for buffer sizes
-: ---------- > 4: 84e9e38a3e convert: use size_t for text statistics and ident countsbase-commit: b9e5cff1803ab24582201ef059516eb16014b9d9
-- 2.53.0