Re: [PATCH v2] upload-pack.c: fix filter spec quoting bug

From: Junio C Hamano <gitster@pobox.com>
Date: Jan 25, 2021, 19:48 UTC
Message-ID: <xmqqlfcgyf0o.fsf@gitster.c.googlers.com>
In-Reply-To: <20210125170921.14291-1-jacob@gitlab.com>

Jacob Vosmaer <jacob@gitlab.com> writes:

Show 24 quoted lines

> This fixes a bug that occurs when you combine partial clone and
> uploadpack.packobjectshook. You can reproduce it as follows:
>
> git clone -u 'git -c uploadpack.allowfilter '\
> '-c uploadpack.packobjectshook=env '\
> 'upload-pack' --filter=blob:none --no-local \
> src.git dst.git
>
> Be careful with the line endings because this has a long quoted string
> as the -u argument.
>
> The error I get when I run this is:
>
> Cloning into '/tmp/broken'...
> remote: fatal: invalid filter-spec ''blob:none''
> error: git upload-pack: git-pack-objects died with error.
> fatal: git upload-pack: aborting due to possible repository corruption on the remote side.
> remote: aborting due to possible repository corruption on the remote side.
> fatal: early EOF
> fatal: index-pack failed
>
> The problem is an unnecessary and harmful layer of quoting. I tried
> digging through the history of this function and I think this quoting
> was there from the start.

Meaning that 10ac85c7 (upload-pack: add object filtering for partial clone, 2017-12-08) that added:

        if (filter_options.filter_spec) {
                struct strbuf buf = STRBUF_INIT;
                sq_quote_buf(&buf, filter_options.filter_spec);
                argv_array_pushf(&pack_objects.args, "--filter=%s", buf.buf);
                strbuf_release(&buf);
        }

Show 7 quoted lines

> My best guess is that it stems from a
> misunderstanding what use_shell=1 means. The code seems to assume it
> means "arguments get joined into one big string, then fed to /bin/sh".
> But that is not what it means: use_shell=1 means that the first
> argument in the arguments array may be a shell script and if so should
> be passed to /bin/sh. All other arguments are passed as normal
> arguments.

I noticed another thing that hasn't changed since that commit, which is that the setting of .use_shell is conditional. In today's code, at the beginning of create_pack_file(), we have

        if (!pack_data->pack_objects_hook)
                pack_objects.git_cmd = 1;
        else {
                strvec_push(&pack_objects.args, pack_data->pack_objects_hook);
                strvec_push(&pack_objects.args, "git");
                pack_objects.use_shell = 1;
        }

I suspect that 0b6069fe (fetch-pack: test support excluding large blobs, 2017-12-08) sort-of fixed half of the problem (i.e. the half when there is no hook used) while leaving the other half still broken as before.

But because .use_shell does not affect if we should or should not quote, we can unconditionally drop the use of sq_quote_buf().

> The solution is simple: never quote the filter spec.

Which makes sense.

> This commit removes the conditional quoting and adds a test for
> partial clone in t5544.
> ---

Thanks.  Missing sign-off.

Show 15 quoted lines

>  	if (pack_data->filter_options.choice) {
>  		const char *spec =
>  			expand_list_objects_filter_spec(&pack_data->filter_options);
> -		if (pack_objects.use_shell) {
> -			struct strbuf buf = STRBUF_INIT;
> -			sq_quote_buf(&buf, spec);
> -			strvec_pushf(&pack_objects.args, "--filter=%s", buf.buf);
> -			strbuf_release(&buf);
> -		} else {
> -			strvec_pushf(&pack_objects.args, "--filter=%s", spec);
> -		}
> +		strvec_pushf(&pack_objects.args, "--filter=%s", spec);
>  	}
>  	if (uri_protocols) {
>  		for (i = 0; i < uri_protocols->nr; i++)