git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 3/3] xdiff: drop LONG_MAX check on merge result size

From
Christian Couder <christian.couder@gmail.com>
Date
Nov 17, 2026, 17:43 UTC
Message-ID
<20261117174353.1158905-8-christian.couder@gmail.com>
In-Reply-To
<20261117174353.1158905-5-christian.couder@gmail.com>

Since 288266c11d (xdiff: use size_t for merge buffer size computations, 2026-10-16), a `size > LONG_MAX` check in xdl_do_merge() makes sure the size isn't truncated on LLP64 platforms like 64-bit Windows.

This check was useful because that `size` was assigned to mmbuffer_t's size which was a `long`. Since 81db32ddce (Merge branch 'jk/xdiff-size_t', 2026-10-15) though, `mmbuffer_t` has been replaced with `mmfile_t`, which uses `size_t` for its `size` member.

That size was still passed to odb_write_object() and write_buf_to_worktree() that accepted `unsigned long` arguments, but previous commits have changed the types of those arguments to `size_t`. So the `size > LONG_MAX` check in xdl_do_merge() is not preventing any truncation anymore.

Let's then get rid of this `size > LONG_MAX` check and allow sizes as big as `size_t` allows, so that merges with results over 2 GiB no longer fail on 64-bit Windows.

This also allows us to clean up related tests in "t4300-merge-tree.sh".

Note that overflows in the size computation itself, for example when `size_t` is 32 bits, are still caught by the st_add*() calls added by b1313872ce (xdiff: check for overflow when computing merge buffer size, 2026-10-16).

Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
 t/t4300-merge-tree.sh | 13 ++-----------
 xdiff/xmerge.c        |  4 ----
 2 files changed, 2 insertions(+), 15 deletions(-)
diff --git a/t/t4300-merge-tree.sh b/t/t4300-merge-tree.sh
index 41386d6025..54bf6c0327 100755
--- a/t/t4300-merge-tree.sh
+++ b/t/t4300-merge-tree.sh
@@ -430,15 +430,13 @@ test_expect_success EXPENSIVE 'setup: base commit with attributes' '
 	git commit -a -m side2
 '
 
-# We have LONG_IS_64BIT as a prerequisite since we guard against downcasts
-# from 64-bit size_t to 32-bit long (for example on Windows)
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'no buffer overflow in merge-tree --write-tree' '
+test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'no buffer overflow in merge-tree --write-tree' '
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
 	tree=$(head -n 1 out) &&
 	git cat-file -e "$tree^{tree}"
 '
 
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths are listed without buffer overflow' '
+test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'conflicted paths are listed without buffer overflow' '
 	test_expect_code 1 git merge-tree --write-tree --name-only \
 		side1 side2 >out &&
 	sed -n "2,\$p" out | sed "/^\$/,\$d" | sort -u >actual &&
@@ -449,13 +447,6 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths ar
 	test_cmp expect actual
 '
 
-# Ensure we properly identify a downcast from 64-bit size_t to
-# 32-bit long on Windows
-test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT 'merge-tree dies when merge buffer size does not fit long' '
-	test_must_fail git merge-tree --write-tree side1 side2 2>err &&
-	test_grep "failure to merge" err
-'
-
 test_expect_success EXPENSIVE,!SIZE_T_IS_64BIT 'merge-tree dies when merge buffer size overflows size_t' '
 	test_must_fail git merge-tree --write-tree side1 side2 2>err &&
 	test_grep "size_t overflow" err
diff --git a/xdiff/xmerge.c b/xdiff/xmerge.c
index 5a78b88fad..2ef9c049e5 100644
--- a/xdiff/xmerge.c
+++ b/xdiff/xmerge.c
@@ -669,10 +669,6 @@ static int xdl_do_merge(xdfenv_t *xe1, xdchange_t *xscr1,
 						    ancestor_name,
 						    favor, changes, NULL, style,
 						    marker_size);
-		if (size > LONG_MAX) {
-			xdl_cleanup_merge(changes);
-			return -1;
-		}
 		result->ptr = xdl_malloc(size);
 		if (!result->ptr) {
 			xdl_cleanup_merge(changes);
-- 
2.56.0
Previous: Christian Couder
Message 7 of 7 in “Allow merges with results over 2 GiB on 64-bit Windows”
  1. 0/3 Allow merges with results over 2 GiB on 64-bit WindowsChristian Couder, Nov 17, 2026
  2. 1/3 notes-merge: use size_t in write_buf_to_worktree()Christian Couder, Nov 17, 2026
  3. Patrick SteinhardtNov 18, 2026
  4. 2/3 odb: widen odb_write_object() length to size_tChristian Couder, Nov 17, 2026
  5. Patrick SteinhardtNov 18, 2026
  6. Christian CouderNov 18, 2026
  7. 3/3 xdiff: drop LONG_MAX check on merge result sizeChristian Couder, Nov 17, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.