Re: [PATCH v2 3/4] Makefile/coccicheck: allow for setting xargs concurrency

From: Jeff King <peff@peff.net>
Date: Mar 6, 2021, 10:51 UTC
Message-ID: <YENexlKkGa2btuBO@coredump.intra.peff.net>
In-Reply-To: <20210305170724.23859-4-avarab@gmail.com>

On Fri, Mar 05, 2021 at 06:07:23PM +0100, Ævar Arnfjörð Bjarmason wrote:

Show 9 quoted lines

> +# Setting SPATCH_XARGS overrides SPATCH_BATCH_SIZE. To get concurrency
> +# when targeting a single contrib/coccinelle/%.patch use e.g. "-P" if
> +# your xargs(1) supports it:
> +#
> +#    make contrib/coccinelle/strbuf.cocci.patch SPATCH_XARGS="xargs -P 8 -n 8"
> +#
> +# Or a combination -jN and "xargs -P":
> +#
> +#    make -j4 coccicheck SPATCH_XARGS="xargs -P 2 -n 8"

I don't think this is actually safe to do. At least not if you care about the output patch. All of the parallel processes will go to the same output file:

Show 10 quoted lines

> -	if test $(SPATCH_BATCH_SIZE) = 0; then \
> -		limit=; \
> -	else \
> -		limit='-n $(SPATCH_BATCH_SIZE)'; \
> -	fi; \
> -	if ! echo $(COCCI_SOURCES) | xargs $$limit \
> +	if ! echo $(COCCI_SOURCES) | $(SPATCH_XARGS) \
>  		$(SPATCH) --sp-file $< $(SPATCH_FLAGS) \
>  		>$@+ 2>$@.log; \
>  	then \

which means they run the risk of getting jumbled racily. It might work OK in practice if the patches are smaller than spatch puts into a single write() call, and what the OS will treat as an atomic file-write (which is really under-specified in POSIX for regular files).

-Peff