Re: New semantic patches vs. in-flight topics [was: Re: [PATCH 00/19] Bring more repository handles into our code base]

From: Stefan Beller <sbeller@google.com>
Date: Oct 23, 2018, 00:26 UTC
Message-ID: <CAGZ79kaWaY+oxJfoMbsCcq6MZDPoq1OgOwK0a9mkKR7sUEOHpg@mail.gmail.com>
In-Reply-To: <xmqqzhv5tx4m.fsf@gitster-ct.c.googlers.com>

Show 7 quoted lines

> Stepping back a bit, I'd imagine in an ideal world where "make
> coccicheck" can be done instantaneously _and_ the spatch machinery
> is just as reliable as C compilers.
> [...]
> Now we do not live in that ideal world and [...]
>  such a series will have zero
> chance of landing in 'pu', unless we freeze the world.

I wonder if we could approximate the ideal world with rerere+spatch a bit more:

1) I resend the series that includes "apply cocci patches"
    as the last patch and you queue it as usual

2) The first time such a series is merged, you'd merge
    HEAD^ (i.e. excluding the "apply the semantic patch)
    to pu instead. I view this as a yet-to-be invented mode
    '--theirs-is-stale-use-tree-instead=THEIRS~1^{tree}',
    then run spatch to reproduce the last patch into the
    dirty merge (which has pu and the last patch as parent)

    This step is done to 'pre-heat' the rerere cache.

3) Any further integration (e.g. rebuilding pu) would
    benefit from the hot rerere cache and very little work
    is actually required as the conflicts are resolved by rerere.

Am I overestimating or misunderstanding rerere here?

Show 10 quoted lines

> What I _could_ do (and what I did do only for one round of pushing
> out 'pu') is to queue a coccinelle transformation at the tip of
> integration branches.  If "make coccicheck" ran in subsecond, I
> could even automate it in the script that is used to rebuild 'pu'
> every day, so that after merging each and every topic, do the "make
> coccicheck" and apply resulting .cocci.patch files and squash that
> into the merge commit.
>
> But with the current "make coccicheck" performance, that is not
> realistic.

Would it be realistic for next and master branch instead of pu?

I'd be wary for the master branch, as we may not want to rely on spatch without review. (It can produce funny white space issues, but seems to produce working/correct code)

> I am wondering if it is feasible to do it at the tip of 'pu' (which
> is rebuilt two to three times a day), 'next' (which is updated once
> or twice a week) and 'master'.

We could even optimize that, by checking if contrib/cocci/ has changes for the new tip of next/master respectively.

Another thing I wonder is if we care about the distinction between the (a) pending changes as described by SZEDER, as we introduce these deliberately, whereas (b) undesirable code patterns (e.g. free and null instead of FREE_AND_NULL macro) might be caught and reported in pu/next and then someone learns from it. Automatic rewriting the (b) cases seems not just as desirable as (a), where we do it purely to avoid resolving merge conflicts by hand.

> I find that your "pending" idea may be nicer, as it distributes the
> load.  Whoever wants to change the world order by updating the .cocci
> rules is primarily responsible for making it happen without breaking
> the world during the transition.  That's more scalable.

... and I think SZEDER considers the current world broken as 'make coccicheck' returns non-empty, so it sounds to me as if the current transition is thought less-than-optimal.