Re: [PATCH] xread(): Fix read error when filtering >= 2GB on Mac OS X

From: Kyle J. McKay <mackyle@gmail.com>
Date: Aug 17, 2013, 20:25 UTC
Message-ID: <0EC822B9-E5BB-4FF5-B054-167866EA2075@gmail.com>
In-Reply-To: <1376743205-12618-1-git-send-email-prohaska@zib.de>

On Aug 17, 2013, at 05:40, Steffen Prohaska wrote:

Show 13 quoted lines

> Previously, filtering more than 2GB through an external filter (see
> test) failed on Mac OS X 10.8.4 (12E55) with:
>
>    error: read from external filter cat failed
>    error: cannot feed the input to external filter cat
>    error: cat died of signal 13
>    error: external filter cat failed 141
>    error: external filter cat failed
>
> The reason is that read() immediately returns with EINVAL if len >=  
> 2GB.
> I haven't found any information under which specific conditions this
> occurs.

According to POSIX [1] for read:

If the value of nbyte is greater than {SSIZE_MAX}, the result is implementation-defined.

The write function also has the same restriction [2].

Since OS X still supports running 32-bit executables, and SSIZE_MAX is 2GB - 1 when running 32-bit it would seem the same limit has been imposed on 64-bit executables. In any case, we should avoid "implementation-defined" behavior for portability unless we know the OS we were compiled on has acceptable "implementation-defined" behavior and otherwise never attempt to read or write more than SSIZE_MAX bytes.

[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/read.html [2] http://pubs.opengroup.org/onlinepubs/009695399/functions/write.html