[PATCH v2 1/4] odb: use size_t for the length of objects to write
- From
Elia Pinto <gitter.spiros@gmail.com>
- Date
- Oct 30, 2026, 09:58 UTC
- Message-ID
- <8fcb9f10f13962acd5cdac652bea8fc4164f2e78.1793352943.git.gitter.spiros@gmail.com>
- In-Reply-To
- <cover.1793352943.git.gitter.spiros@gmail.com>
c6a4629e32 (odb: use size_t for object_info.sizep and the size APIs, 2026-06-15) widened the read side of the object database API, and the write_object() callback of odb_source already takes a size_t. But odb_write_object_ext(), odb_write_object() and odb_source_write_object() in front of it, as well as write_loose_object() behind it, still take an "unsigned long", which is only 32 bits wide on Windows.
Most callers hand them a size_t from a strbuf or from odb_read_object(), which silently gets truncated. The most visible case is index_mem() and index_stream_convert_blob(): when "git add" or "git hash-object -w" runs a file over 4GiB through a conversion (core.autocrlf, which is on by default in Git for Windows, an eol or ident attribute, or a clean filter), we write a blob made of the first "size mod 2^32" bytes. The index records it with the stat data of the real file, so "git status" stays clean and nobody notices until the file is checked out again. "git hash-object" without "-w" uses hash_object_file(), which takes a size_t, so the two disagree on the object ID of the same file.
Use size_t throughout, including for the size kept by the in-memory source.
The existing "hash filtered files over 4GB correctly" test in t1007 only hashes, so add a variant that writes the object with "git hash-object -w" and checks both its object ID and its size. Like the other tests over 4GB, it needs EXPENSIVE and SIZE_T_IS_64BIT. Truncating the length to 32 bits in odb_write_object_ext(), as happens on Windows without this patch, makes it fail with a different object ID.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> --- odb.c | 2 +- odb.h | 4 ++-- odb/source-inmemory.c | 2 +- odb/source-loose.c | 2 +- odb/source.h | 2 +- t/t1007-hash-object.sh | 13 +++++++++++++ 6 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/odb.c b/odb.c index 2c93a9490c..1f173625b7 100644 --- a/odb.c +++ b/odb.c @@ -640,7 +640,7 @@ void odb_assert_oid_type(struct object_database *odb, } int odb_write_object_ext(struct object_database *odb, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, struct object_id *oid, const struct object_id *compat_oid_in, diff --git a/odb.h b/odb.h index dd85b880a7..45182d28cb 100644 --- a/odb.h +++ b/odb.h @@ -621,14 +621,14 @@ enum odb_write_object_flags { * Returns 0 on success, a negative error code otherwise. */ int odb_write_object_ext(struct object_database *odb, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, struct object_id *oid, const struct object_id *compat_oid, enum odb_write_object_flags flags); static inline int odb_write_object(struct object_database *odb, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, struct object_id *oid) { diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c index 75cf29fa9d..c13cffc805 100644 --- a/odb/source-inmemory.c +++ b/odb/source-inmemory.c @@ -10,7 +10,7 @@ struct inmemory_object { enum object_type type; const void *buf; - unsigned long size; + size_t size; }; static const struct inmemory_object *find_cached_object(struct odb_source_inmemory *source, diff --git a/odb/source-loose.c b/odb/source-loose.c index 50cb388c35..60801e70e8 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -758,7 +758,7 @@ static int end_loose_object_common(struct odb_source_loose *loose, static int write_loose_object(struct odb_source_loose *loose, const struct object_id *oid, char *hdr, - int hdrlen, const void *buf, unsigned long len, + int hdrlen, const void *buf, size_t len, const time_t *mtime, unsigned flags) { int fd, ret; diff --git a/odb/source.h b/odb/source.h index 8b839caa39..07d5019aa5 100644 --- a/odb/source.h +++ b/odb/source.h @@ -451,7 +451,7 @@ static inline int odb_source_freshen_object(struct odb_source *source, * object ID and the compatibility object ID, if non-NULL. */ static inline int odb_source_write_object(struct odb_source *source, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, const struct object_id *oid, const struct object_id *compat_oid, diff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh index 463b38f990..0a89bbb455 100755 --- a/t/t1007-hash-object.sh +++ b/t/t1007-hash-object.sh @@ -297,4 +297,17 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \ test_cmp expect actual ' +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \ + 'write filtered files over 4GB correctly' ' + { test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } && + test_oid large5GB >expect && + test_config filter.null-filter.clean "cat" && + echo "big filter=null-filter" >.gitattributes && + git hash-object -w -- big >actual && + test_cmp expect actual && + echo $((5*1024*1024*1024)) >expect && + git cat-file -s $(cat actual) >actual && + test_cmp expect actual +' + test_done
-- 2.53.0