Re: read() MAX_IO_SIZE bytes, more than SSIZE_MAX?
- From
Junio C Hamano <gitster@pobox.com>
- Date
- Feb 11, 2015, 21:13 UTC
- Message-ID
- <xmqqmw4ktamx.fsf@gitster.dls.corp.google.com>
- In-Reply-To
- <CAPig+cTbLOV-0yFKp8wwVLSr4OJz7LUaLZgVGHUdFhj7xZEzrw@mail.gmail.com>
Eric Sunshine <sunshine@sunshineco.com> writes:
Show 10 quoted lines
> A bit cleaner: > > #ifndef(MAX_IO_SIZE) > # define MAX_IO_SIZE_DEFAULT (8*1024*1024) > # if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT) > # define MAX_IO_SIZE SSIZE_MAX > # else > # define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT > # endif > #endif
OK, then let's do this.
-- >8 -- Subject: xread/xwrite: clip MAX_IO_SIZE to SSIZE_MAX
Since 0b6806b9 (xread, xwrite: limit size of IO to 8MB, 2013-08-20), we chomp our calls to read(2) and write(2) into chunks of MAX_IO_SIZE bytes (8 MiB), because a large IO results in a bad latency when the program needs to be killed. This also brought our IO below SSIZE_MAX, which is a limit POSIX allows read(2) and write(2) to fail when the IO size exceeds it, for OS X, where a problem was originally reported.
However, there are other systems that define SSIZE_MAX smaller than our default X-<. Make sure we clip our calls to this as well.
Reported-by: Joachim Schmitz <jojo@schmitz-digital.de> Helped-by: Torsten Bögershausen <tboegi@web.de> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> --- wrapper.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/wrapper.c b/wrapper.c index 007ec0d..50e6697 100644 --- a/wrapper.c +++ b/wrapper.c @@ -172,8 +172,21 @@ void *xcalloc(size_t nmemb, size_t size) * 64-bit is buggy, returning EINVAL if len >= INT_MAX; and even in * the absence of bugs, large chunks can result in bad latencies when * you decide to kill the process. + * + * We pick 8 MiB as our default, but if the platform defines SSIZE_MAX + * that is smaller than that, clip it to SSIZE_MAX, as a call to + * read(2) or write(2) larger than taht is allowed to fail. As the last + * resort, we allow a port to pass via CFLAGS e.g. "-DMAX_IO_SIZE=value" + * to override this, if the definition of SSIZE_MAX platform is broken. */ -#define MAX_IO_SIZE (8*1024*1024) +#ifndef(MAX_IO_SIZE) +# define MAX_IO_SIZE_DEFAULT (8*1024*1024) +# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT) +# define MAX_IO_SIZE SSIZE_MAX +# else +# define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT +# endif +#endif /* * xread() is the same a read(), but it automatically restarts read()