[PATCH 3/3] xdiff: drop LONG_MAX check on merge result size
- From
Christian Couder <christian.couder@gmail.com>
- Date
- Nov 17, 2026, 17:43 UTC
- Message-ID
- <20261117174353.1158905-8-christian.couder@gmail.com>
- In-Reply-To
- <20261117174353.1158905-5-christian.couder@gmail.com>
Since 288266c11d (xdiff: use size_t for merge buffer size computations, 2026-10-16), a `size > LONG_MAX` check in xdl_do_merge() makes sure the size isn't truncated on LLP64 platforms like 64-bit Windows.
This check was useful because that `size` was assigned to mmbuffer_t's size which was a `long`. Since 81db32ddce (Merge branch 'jk/xdiff-size_t', 2026-10-15) though, `mmbuffer_t` has been replaced with `mmfile_t`, which uses `size_t` for its `size` member.
That size was still passed to odb_write_object() and write_buf_to_worktree() that accepted `unsigned long` arguments, but previous commits have changed the types of those arguments to `size_t`. So the `size > LONG_MAX` check in xdl_do_merge() is not preventing any truncation anymore.
Let's then get rid of this `size > LONG_MAX` check and allow sizes as big as `size_t` allows, so that merges with results over 2 GiB no longer fail on 64-bit Windows.
This also allows us to clean up related tests in "t4300-merge-tree.sh".
Note that overflows in the size computation itself, for example when `size_t` is 32 bits, are still caught by the st_add*() calls added by b1313872ce (xdiff: check for overflow when computing merge buffer size, 2026-10-16).
Signed-off-by: Christian Couder <christian.couder@gmail.com> --- t/t4300-merge-tree.sh | 13 ++----------- xdiff/xmerge.c | 4 ---- 2 files changed, 2 insertions(+), 15 deletions(-)
diff --git a/t/t4300-merge-tree.sh b/t/t4300-merge-tree.sh index 41386d6025..54bf6c0327 100755 --- a/t/t4300-merge-tree.sh +++ b/t/t4300-merge-tree.sh @@ -430,15 +430,13 @@ test_expect_success EXPENSIVE 'setup: base commit with attributes' ' git commit -a -m side2 ' -# We have LONG_IS_64BIT as a prerequisite since we guard against downcasts -# from 64-bit size_t to 32-bit long (for example on Windows) -test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'no buffer overflow in merge-tree --write-tree' ' +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'no buffer overflow in merge-tree --write-tree' ' test_expect_code 1 git merge-tree --write-tree side1 side2 >out && tree=$(head -n 1 out) && git cat-file -e "$tree^{tree}" ' -test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths are listed without buffer overflow' ' +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'conflicted paths are listed without buffer overflow' ' test_expect_code 1 git merge-tree --write-tree --name-only \ side1 side2 >out && sed -n "2,\$p" out | sed "/^\$/,\$d" | sort -u >actual && @@ -449,13 +447,6 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,LONG_IS_64BIT 'conflicted paths ar test_cmp expect actual ' -# Ensure we properly identify a downcast from 64-bit size_t to -# 32-bit long on Windows -test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT 'merge-tree dies when merge buffer size does not fit long' ' - test_must_fail git merge-tree --write-tree side1 side2 2>err && - test_grep "failure to merge" err -' - test_expect_success EXPENSIVE,!SIZE_T_IS_64BIT 'merge-tree dies when merge buffer size overflows size_t' ' test_must_fail git merge-tree --write-tree side1 side2 2>err && test_grep "size_t overflow" err diff --git a/xdiff/xmerge.c b/xdiff/xmerge.c index 5a78b88fad..2ef9c049e5 100644 --- a/xdiff/xmerge.c +++ b/xdiff/xmerge.c @@ -669,10 +669,6 @@ static int xdl_do_merge(xdfenv_t *xe1, xdchange_t *xscr1, ancestor_name, favor, changes, NULL, style, marker_size); - if (size > LONG_MAX) { - xdl_cleanup_merge(changes); - return -1; - } result->ptr = xdl_malloc(size); if (!result->ptr) { xdl_cleanup_merge(changes);
-- 2.56.0