git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/4] unpack-objects: use size_t for object sizes

From
Elia Pinto <gitter.spiros@gmail.com>
Date
Nov 18, 2026, 06:56 UTC
Message-ID
<b2c3d349a4ac73da7e1d066ac43754d3e6939b9f.1794984184.git.gitter.spiros@gmail.com>
In-Reply-To
<cover.1794984184.git.gitter.spiros@gmail.com>

Since f2063855fb (index-pack, unpack-objects: use size_t for object size, 2026-05-08), unpack_one() decodes the size of an object into a size_t, and e07ac16d1c (unpack-objects: pass the size of a streamed blob as size_t, 2026-11-06) passes it on as such to stream_blob(), which handles blobs larger than core.bigFileThreshold. The rest of builtin/unpack-objects.c still takes the size as an "unsigned long":

 - unpack_non_delta_entry() and get_data(), used for commits, trees,
   tags and blobs below core.bigFileThreshold, and for all blobs with
   "git unpack-objects -n", which never streams;
 - unpack_delta_entry() and the functions that keep and resolve
   deltas;
 - struct obj_buffer, which keeps objects around for --strict.

On platforms where "long" is 32 bits while size_t is 64, such as 64-bit Windows, the size of an object of 4GiB or more is therefore truncated:

 - For an object that is not streamed, get_data() allocates a buffer
   of the truncated size, inflate runs out of room in it, and
   unpack-objects fails with "inflate returned -5". Truncating the
   size to 16 bits in get_data() and unpack_non_delta_entry()
   reproduces this with a pack holding a 70000-byte blob, both with
   and without -n.
 - unpack_delta_entry() reads the base of a delta through
   cast_size_t_to_ulong(), so a base of 4GiB or more makes it die even
   when the result would be smaller.
 - The result of a delta reaches write_object() as a size_t, but
   write_object() takes it as an "unsigned long". A result of 4GiB or
   more is silently truncated there, and an object with the wrong
   contents, hence the wrong name, is written.

Use size_t across builtin/unpack-objects.c. The functions that write_object() hands objects to, odb_write_object() and parse_object_buffer(), still take an "unsigned long", as does fsck_object(); they are updated separately. Until then, move the size cast from unpack_delta_entry() to write_object(), where the size is passed to them. fsck_object() only sees objects that went through parse_object_buffer() first, and so does write_cached_object(), which writes them out later; neither needs a cast of its own.

As a result, on those platforms:
 - "git unpack-objects -n" handles objects of any size;
 - a delta with a base of 4GiB or more and a smaller result is
   resolved;
 - an object of 4GiB or more that is not streamed and has to be
   written, including the result of a delta, makes unpack-objects die
   with "object too large to read on this platform" instead of failing
   with "inflate returned -5" or being written truncated.

Truncating the size to 16 bits where write_object() passes it on confirms the last point for a 70000-byte blob: with the cast, unpack-objects dies with "object too large to read on this platform: 70000 is cut off to 4464"; without it, it exits with 0 and writes a truncated blob under another name.

Triggering this requires processing an object of 4GiB or more on a platform where "long" is 32 bits while size_t is 64, so no test is added.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
---
 builtin/unpack-objects.c | 42 ++++++++++++++++++++--------------------
 1 file changed, 21 insertions(+), 21 deletions(-)
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index d2f25bed49..9bd355b2c4 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -40,7 +40,7 @@ static struct progress *progress;
  */
 struct obj_buffer {
 	char *buffer;
-	unsigned long size;
+	size_t size;
 };
 
 static struct decoration obj_decorate;
@@ -50,7 +50,7 @@ static struct obj_buffer *lookup_object_buffer(struct object *base)
 	return lookup_decoration(&obj_decorate, base);
 }
 
-static void add_object_buffer(struct object *object, char *buffer, unsigned long size)
+static void add_object_buffer(struct object *object, char *buffer, size_t size)
 {
 	struct obj_buffer *obj;
 	CALLOC_ARRAY(obj, 1);
@@ -114,10 +114,10 @@ static void use(int bytes)
  * allocated buffer which is reused to hold temporary zstream output
  * and return NULL instead of returning garbage data.
  */
-static void *get_data(unsigned long size)
+static void *get_data(size_t size)
 {
 	git_zstream stream;
-	unsigned long bufsize = dry_run && size > 8192 ? 8192 : size;
+	size_t bufsize = dry_run && size > 8192 ? 8192 : size;
 	void *buf = xmallocz(bufsize);
 
 	memset(&stream, 0, sizeof(stream));
@@ -161,7 +161,7 @@ struct delta_info {
 	struct object_id base_oid;
 	unsigned nr;
 	off_t base_offset;
-	unsigned long size;
+	size_t size;
 	void *delta;
 	struct delta_info *next;
 };
@@ -170,7 +170,7 @@ static struct delta_info *delta_list;
 
 static void add_delta_to_list(unsigned nr, const struct object_id *base_oid,
 			      off_t base_offset,
-			      void *delta, unsigned long size)
+			      void *delta, size_t size)
 {
 	struct delta_info *info = xmalloc(sizeof(*info));
 
@@ -261,7 +261,7 @@ static void write_rest(void)
 }
 
 static void added_object(unsigned nr, enum object_type type,
-			 void *data, unsigned long size);
+			 void *data, size_t size);
 
 /*
  * Write out nr-th object from the list, now we know the contents
@@ -269,10 +269,11 @@ static void added_object(unsigned nr, enum object_type type,
  * to be checked at the end.
  */
 static void write_object(unsigned nr, enum object_type type,
-			 void *buf, unsigned long size)
+			 void *buf, size_t size)
 {
 	if (!strict) {
-		if (odb_write_object(the_repository->objects, buf, size, type,
+		if (odb_write_object(the_repository->objects, buf,
+				     cast_size_t_to_ulong(size), type,
 				     &obj_list[nr].oid) < 0)
 			die("failed to write object");
 		added_object(nr, type, buf, size);
@@ -280,7 +281,8 @@ static void write_object(unsigned nr, enum object_type type,
 		obj_list[nr].obj = NULL;
 	} else if (type == OBJ_BLOB) {
 		struct blob *blob;
-		if (odb_write_object(the_repository->objects, buf, size, type,
+		if (odb_write_object(the_repository->objects, buf,
+				     cast_size_t_to_ulong(size), type,
 				     &obj_list[nr].oid) < 0)
 			die("failed to write object");
 		added_object(nr, type, buf, size);
@@ -299,7 +301,7 @@ static void write_object(unsigned nr, enum object_type type,
 				 &obj_list[nr].oid);
 		added_object(nr, type, buf, size);
 		obj = parse_object_buffer(the_repository, &obj_list[nr].oid,
-					  type, size, buf,
+					  type, cast_size_t_to_ulong(size), buf,
 					  &eaten);
 		if (!obj)
 			die("invalid %s", type_name(type));
@@ -310,8 +312,8 @@ static void write_object(unsigned nr, enum object_type type,
 }
 
 static void resolve_delta(unsigned nr, enum object_type type,
-			  void *base, unsigned long base_size,
-			  void *delta, unsigned long delta_size)
+			  void *base, size_t base_size,
+			  void *delta, size_t delta_size)
 {
 	void *result;
 	size_t result_size;
@@ -330,7 +332,7 @@ static void resolve_delta(unsigned nr, enum object_type type,
  * resolve all the deltified objects that are based on it.
  */
 static void added_object(unsigned nr, enum object_type type,
-			 void *data, unsigned long size)
+			 void *data, size_t size)
 {
 	struct delta_info **p = &delta_list;
 	struct delta_info *info;
@@ -349,7 +351,7 @@ static void added_object(unsigned nr, enum object_type type,
 	}
 }
 
-static void unpack_non_delta_entry(enum object_type type, unsigned long size,
+static void unpack_non_delta_entry(enum object_type type, size_t size,
 				   unsigned nr)
 {
 	void *buf = get_data(size);
@@ -420,7 +422,7 @@ static void stream_blob(size_t size, unsigned nr)
 }
 
 static int resolve_against_held(unsigned nr, const struct object_id *base,
-				void *delta_data, unsigned long delta_size)
+				void *delta_data, size_t delta_size)
 {
 	struct object *obj;
 	struct obj_buffer *obj_buffer;
@@ -435,12 +437,11 @@ static int resolve_against_held(unsigned nr, const struct object_id *base,
 	return 1;
 }
 
-static void unpack_delta_entry(enum object_type type, unsigned long delta_size,
+static void unpack_delta_entry(enum object_type type, size_t delta_size,
 			       unsigned nr)
 {
 	void *delta_data, *base;
-	unsigned long base_size;
-	size_t base_size_st = 0;
+	size_t base_size = 0;
 	struct object_id base_oid;
 
 	if (type == OBJ_REF_DELTA) {
@@ -517,8 +518,7 @@ static void unpack_delta_entry(enum object_type type, unsigned long delta_size,
 		return;
 
 	base = odb_read_object(the_repository->objects, &base_oid,
-			       &type, &base_size_st);
-	base_size = cast_size_t_to_ulong(base_size_st);
+			       &type, &base_size);
 	if (!base) {
 		error("failed to read delta-pack base object %s",
 		      oid_to_hex(&base_oid));
-- 
2.53.0
Previous: Elia PintoNext: Patrick Steinhardt
Message 7 of 14 in “use size_t when receiving and verifying packs”
  1. 0/4 use size_t when receiving and verifying packsElia Pinto, Oct 31, 2026
  2. 1/4 unpack-objects: use size_t for object sizesElia Pinto, Oct 31, 2026
  3. 2/4 index-pack: use size_t in write_compressed()Elia Pinto, Oct 31, 2026
  4. 3/4 index-pack: use size_t for object sizesElia Pinto, Oct 31, 2026
  5. 4/4 pack-check: use size_t for the size passed to verify callbacksElia Pinto, Oct 31, 2026
  6. 0/4 use size_t when receiving and verifying packsElia Pinto, Nov 18, 2026
  7. 1/4 unpack-objects: use size_t for object sizesElia Pinto, Nov 18, 2026
  8. Patrick SteinhardtNov 18, 2026
  9. Junio C HamanoNov 18, 2026
  10. 2/4 index-pack: use size_t in write_compressed()Elia Pinto, Nov 18, 2026
  11. Patrick SteinhardtNov 18, 2026
  12. 3/4 index-pack: use size_t for object sizesElia Pinto, Nov 18, 2026
  13. 4/4 pack-check: use size_t for the size passed to verify callbacksElia Pinto, Nov 18, 2026
  14. Patrick SteinhardtNov 18, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.