[PATCH v2 3/7] xdiff: use size_t for buffer sizes

An mmfile_t stores its size as a signed long, but the more natural type for a buffer size is size_t. This not only limits the size of entry we can hold, but also creates some possible integer overflow issues.

For example, read_mmfile() checks that the file size fits in a size_t before allocating, but then assigns it to a long. Likewise, read_mmblob() and fill_mmfile() copy sizes from other types without checking that they fit.

On LP64 systems like Linux, this is mostly academic. You could wrap to a negative long value, but you'd need an object that's 2^63 bytes, which is impractical.

But on an LLP64 system like Windows, a 2^31+1-byte blob could perhaps cause mischief. We do prevent large values from entering the xdiff code due to MAX_XDIFF_SIZE (which is itself marked as unsigned, so we'd convert any negative "long" back to a large unsigned value). But if you ask for binary diffs, that negative long value could instead be converted to a huge 64-bit size_t when passed to memcmp(), diff_delta(), etc. So probably there are paths that can cause an out-of-bounds read, given the right set of options, but I didn't really dig for them.

On a 32-bit system things are less clear. Because "long" and "size_t" have the same width, any time we implicitly convert to size_t, we should get back the original size (even if the intermediate "long" is itself negative). Probably iterating using a long could be a problem, but most of that happens inside xdiff, which is protected by MAX_XDIFF_SIZE (which, again, compares in the unsigned space).

Let's just use the obvious size_t type for counting the bytes. I suspect you could still find truncation problems on LLP64 systems due to the use of "unsigned long" throughout the code, but that's a larger problem. This should at least nudge us in the right direction.

Note that we have to update the printf format in emit_binary_diff_body() to accommodate the new type. Curiously it was using "%lu", even though the type was signed (I guess compiler printf-linting is happy enough if just the width of the format and the type match).

Signed-off-by: Jeff King <peff@peff.net>
---
 diff.c        | 2 +-
 xdiff/xdiff.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)