[PATCH v2 2/4] Makefile/coccicheck: speed up and fix bug with duplicate hunks

Change the coccicheck target to run on all of our *.c and *.h files with --no-includes, instead of only on the *.c files with --all-includes.

This speeds it up significantly and reduces its memory usage, since it doesn't need to parse N includes for every file it visits.

See [1] for a discussion thread about this commit with some timings for details, but briefly: This change speeds it up by ~2x and makes it use much less memory. Or a reduction of a max of around ~2GB per-process (under the old SPATCH_BATCH_SIZE=0) to around ~200MB.

Looking at the history of the coccicheck target I think we've always been running it in the wrong mode for what we wanted to achieve:

 - When it was added in 63f0a758a06 (add coccicheck make target,
   2016-09-15) it explicitly processed only the %.c files.

 - Then in a9a884aea57 (coccicheck: use --all-includes by default,
   2016-09-30) it started processing the %.h files by looking around for
   its own includes.

Let's instead just point it to both our *.c and *.h files, then there's no need to have it recursively look around for included files to change.

Setting --no-includes would not work if we expected to set COCCI_SOURCES to a subset of our source files, but that's not what we're doing here. If anyone manually tweaks COCCI_SOURCES they'll now need to tweak SPATCH_FLAGS too. The speed and correctness we gain is worth that small trade-off.

Using --no-includes also fixes a subtle bug introduced in 960154b9c17 (coccicheck: optionally batch spatch invocations, 2019-05-06) with duplicate hunks being written to the generated *.patch files.

This is because that change altered a file-at-a-time for-loop to an invocation of "xargs -n X". This would not matter for most other programs, but it matters for spatch.

This is because each spatch invocation will maintain shared lock files in /tmp, check if files being parsed were changed etc. I haven't dug into why exactly, but it's easy to reproduce the issue[2]. The issue goes away entirely if we just use --no-includes, which as noted above would have made sense even without that issue.

1. https://lore.kernel.org/git/20210302205103.12230-1-avarab@gmail.com/
2. A session showing racy spatch under xargs -n X:

    $ cat test.cocci
    @@
    expression E1;
    @@
    - strbuf_avail(E1)
    + strbuf_has(E1)
    $ for i in 1 2 4 16 64 128 512
    do
        echo with xargs -n $i: &&
	echo *.c | xargs -n $i spatch --sp-file \
            test.cocci --all-includes --patch . 2>/dev/null | \
	grep -F +++ | sort | uniq -c
    done
    with xargs -n 1:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
    with xargs -n 2:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
    with xargs -n 4:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
    with xargs -n 16:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
          2 +++ b/strbuf.h
    with xargs -n 64:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
          2 +++ b/strbuf.h
    with xargs -n 128:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
          2 +++ b/strbuf.h
    with xargs -n 512:
          1 +++ b/convert.c
          1 +++ b/strbuf.c
          1 +++ b/strbuf.h

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)