Re: [RFC/PATCH 1/3] index-pack: add --max-input-size=<size> option

From: Jeff King <peff@peff.net>
Date: Aug 15, 2016, 20:10 UTC
Message-ID: <20160815201058.v2u24nnu6rjwlr2k@sigill.intra.peff.net>
In-Reply-To: <20160815195729.16826-2-chriscool@tuxfamily.org>

On Mon, Aug 15, 2016 at 09:57:27PM +0200, Christian Couder wrote:

> From: Jeff King <peff@peff.net>
> 
> When receiving a pack-file, it can be useful to abort the
> `git index-pack`, if the pack-file is too big.

Not much rationale here. I guess because it is all in the 3rd patch, which ties it into receive-pack. I'm not sure it's worth repeating. I guess it could all be squished back into one patch. I'm OK either way.

Show 18 quoted lines

> diff --git a/builtin/index-pack.c b/builtin/index-pack.c
> index 1d2ea58..1fd60bd 100644
> --- a/builtin/index-pack.c
> +++ b/builtin/index-pack.c
> @@ -87,6 +87,7 @@ static struct progress *progress;
>  static unsigned char input_buffer[4096];
>  static unsigned int input_offset, input_len;
>  static off_t consumed_bytes;
> +static off_t max_input_size;
>  static unsigned deepest_delta;
>  static git_SHA_CTX input_ctx;
>  static uint32_t input_crc32;
> @@ -297,6 +298,8 @@ static void use(int bytes)
>  	if (signed_add_overflows(consumed_bytes, bytes))
>  		die(_("pack too large for current definition of off_t"));
>  	consumed_bytes += bytes;
> +	if (max_input_size && consumed_bytes > max_input_size)
> +		die(_("pack exceeds maximum allowed size"));

Looks good. I see you marked it for translation, which makes sense.

On the original, I waffled on whether to share the size with the user in the message. I didn't want to encourage people with "oh, if it's under 2G it must be OK, then!". Because really 2G was meant to be a "you really shouldn't get this high, and we will unceremoniously dump your push if you do".

Show 7 quoted lines

>  static const char *open_pack_file(const char *pack_name)
> @@ -1714,6 +1717,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
>  					opts.off32_limit = strtoul(c+1, &c, 0);
>  				if (*c || opts.off32_limit & 0x80000000)
>  					die(_("bad %s"), arg);
> +			} else if (skip_prefix(arg, "--max-input-size=", &arg)) {
> +				max_input_size = strtoul(arg, NULL, 10);

max_input_size is an off_t, but your parse only up to ULONG_MAX here. For my purposes in the original patch, this was OK, as we set it at 2GB, which works everywhere (and also, GitHub systems all have 64-bit "long" these days). But somebody on a 32-bit system could not set this to 4GB, even though I think index-pack could otherwise handle it. We seem to use strtoumax() elsewhere, so that's probably a good match (technically it can overflow an off_t, but in practice this value comes from the admin and they will set something sane).

-Peff