Re: [PATCH v3] compat: Fix read() of 2GB and more on Mac OS X

From: Eric Sunshine <sunshine@sunshineco.com>
Date: Aug 19, 2013, 13:59 UTC
Message-ID: <CAPig+cTr_B+vtN4sFzepWeW4TpRPD9eKnjy08yJ2pf3KfVU1XA@mail.gmail.com>
In-Reply-To: <1376900499-662-1-git-send-email-prohaska@zib.de>

On Mon, Aug 19, 2013 at 4:21 AM, Steffen Prohaska <prohaska@zib.de> wrote:

Show 35 quoted lines

> Previously, filtering 2GB or more through an external filter (see test)
> failed on Mac OS X 10.8.4 (12E55) for a 64-bit executable with:
>
>     error: read from external filter cat failed
>     error: cannot feed the input to external filter cat
>     error: cat died of signal 13
>     error: external filter cat failed 141
>     error: external filter cat failed
>
>
> Signed-off-by: Steffen Prohaska <prohaska@zib.de>
> ---
>  Makefile              |  8 ++++++++
>  builtin/var.c         |  1 +
>  compat/clipped-read.c | 13 +++++++++++++
>  config.mak.uname      |  1 +
>  git-compat-util.h     |  5 +++++
>  streaming.c           |  1 +
>  t/t0021-conversion.sh | 14 ++++++++++++++
>  7 files changed, 43 insertions(+)
>  create mode 100644 compat/clipped-read.c
>
> diff --git a/Makefile b/Makefile
> index 3588ca1..0f69e24 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -69,6 +69,9 @@ all::
>  # Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt
>  # doesn't support GNU extensions like --check and --statistics
>  #
> +# Define NEEDS_CLIPPED_READ if your read(2) cannot read more than
> +# INT_MAX bytes at once (e.g. MacOS X).
> +#
>  # Define NEEDS_CLIPPED_WRITE if your write(2) cannot write more than
>  # INT_MAX bytes at once (e.g. MacOS X).

Is it likely that we would see a platform requiring only one or the other CLIPPED? Would it make sense to combine these into a single NEEDS_CLIPPED_IO?

Show 20 quoted lines

>  #
> @@ -1493,6 +1496,11 @@ ifndef NO_MSGFMT_EXTENDED_OPTIONS
>         MSGFMT += --check --statistics
>  endif
>
> +ifdef NEEDS_CLIPPED_READ
> +       BASIC_CFLAGS += -DNEEDS_CLIPPED_READ
> +       COMPAT_OBJS += compat/clipped-read.o
> +endif
> +
>  ifdef NEEDS_CLIPPED_WRITE
>         BASIC_CFLAGS += -DNEEDS_CLIPPED_WRITE
>         COMPAT_OBJS += compat/clipped-write.o
> diff --git a/builtin/var.c b/builtin/var.c
> index aedbb53..e59f5ba 100644
> --- a/builtin/var.c
> +++ b/builtin/var.c
> @@ -38,6 +38,7 @@ static struct git_var git_vars[] = {
>         { "", NULL },
>  };