git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/2] grep, diff: do not silently truncate sizes of 4GiB or more

From
Elia Pinto <gitter.spiros@gmail.com>
Date
Nov 1, 2026, 08:44 UTC
Message-ID
<cover.1793521916.git.gitter.spiros@gmail.com>

This series continues the search for truncated sizes on platforms where "long" is 32 bits but size_t is 64 bits (64-bit Windows). This time it covers grep and diff, where a file of 4GiB or more produced wrong results without any error.

- 1/2 grep: on the working tree and --no-index, it silently missed
  matches beyond the truncated size. Now it uses size_t and works,
  for blobs too.
- 2/2 diff: on the working tree, textconv, --no-index and combine-diff,
  it produced a diff of a prefix, and a patch that truncates the file
  when applied (as git add -p does). Now it dies with a clear error,
  like the blob side already did.

Of the remaining -Wshorten-64-to-32 warnings in the diff and grep code, these are the only reachable ones. The others are in xdiff internals (inputs limited to MAX_XDIFF_SIZE, 1023MiB), in buffer_is_binary() (which looks at 8000 bytes), or in places where the size can't exceed 4GiB. Widening diff_filespec.size to size_t, so that diff actually works above 4GiB, is a much bigger change, left for later.

No tests, as they would need 4GiB of input on such a platform. Both effects were verified by simulating the truncation at 16 bits, and the commit messages give the details.

It doesn't depend on the other series in flight and merges cleanly with them. Based on master 50fe5cc63f.

Elia Pinto (2):
  grep: use size_t for the size of buffers to search
  diff: die instead of truncating sizes of 4GiB or more
 combine-diff.c  | 11 +++++++----
 diff-no-index.c |  2 +-
 diff.c          |  8 ++++----
 grep.c          | 16 ++++++++--------
 grep.h          |  4 ++--
 5 files changed, 22 insertions(+), 19 deletions(-)
base-commit: 50fe5cc63f9cd6e89b87a2c9ccf96a444f4cfdb7
-- 
2.53.0
Next: Elia Pinto
Message 1 of 4 in “grep, diff: do not silently truncate sizes of 4GiB or more”
  1. 0/2 grep, diff: do not silently truncate sizes of 4GiB or moreElia Pinto, Nov 1, 2026
  2. 1/2 grep: use size_t for the size of buffers to searchElia Pinto, Nov 1, 2026
  3. 2/2 diff: die instead of truncating sizes of 4GiB or moreElia Pinto, Nov 1, 2026
  4. Elia PintoNov 19, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.