From: Patrick Steinhardt <ps@pks.im>
Date: Wed, 21 Oct 2026 11:24:25 GMT
Subject: Re: [PATCH 1/3] odb: use size_t for the length of objects to write
Message-ID: <atig6fUHbPqYdzXk@pks.im>
In-Reply-To: <20261021101640.329990-2-gitter.spiros@gmail.com>

On Wed, Oct 21, 2026 at 12:16:37PM +0200, Elia Pinto wrote:
> c6a4629e32 (odb: use size_t for object_info.sizep and the size APIs,
> 2026-06-15) widened the read side of the object database API, and the
> write_object() callback of odb_source already takes a size_t. But
> odb_write_object_ext(), odb_write_object() and odb_source_write_object()
> in front of it, as well as write_loose_object() behind it, still take
> an "unsigned long", which is only 32 bits wide on Windows.

Sometimes I wish that C would give us an easier way to detect such
mismatches. It of course does have optional warnings, but enabling those
now would probably lead to thousands of warnings.

> Most callers hand them a size_t from a strbuf or from odb_read_object(),
> which silently gets truncated. The most visible case is index_mem() and
> index_stream_convert_blob(): when "git add" or "git hash-object -w"
> runs a file over 4GiB through a conversion (core.autocrlf, which is on
> by default in Git for Windows, an eol or ident attribute, or a clean
> filter), we write a blob made of the first "size mod 2^32" bytes. The
> index records it with the stat data of the real file, so "git status"
> stays clean and nobody notices until the file is checked out again.
> "git hash-object" without "-w" uses hash_object_file(), which takes a
> size_t, so the two disagree on the object ID of the same file.

Awesome.

> Use size_t throughout, including for the size kept by the in-memory
> source.
> 
> The existing "hash filtered files over 4GB correctly" test in t1007
> only hashes. Running the same with "git hash-object -w" on Windows
> shows the problem, but needs another pass over a 5GiB file, so no new
> test is added here.

But wouldn't it make sense to add a test? It's a rather serious bug, and
the existing tests are marked with EXPENSIVE already anyway. So I don't
think this is a good enough argument to not add another EXPENSIVE test.

> diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
> index 868e723c81..ab7173cd2d 100644
> --- a/odb/source-inmemory.c
> +++ b/odb/source-inmemory.c
> @@ -10,7 +10,7 @@
>  struct inmemory_object {
>  	enum object_type type;
>  	const void *buf;
> -	unsigned long size;
> +	size_t size;
>  };

I really hope nobody writes 4GB blobs into memory :) But despite that,
this fix is correct anyway.

Patrick