threads / patch / 54205

patchblame.c: replace instance of !oidcmp for oideq

Subject: [PATCH] blame.c: replace instance of !oidcmp for oideq

## tl;dr

14 messages between Sep 7, 2020 and Sep 9, 2020. Diffs are folded; open one to read it.

replies: 13people: 6as markdown or json

Edmundo Carmona Antoranz· Sep 7, 2020, 17:16 UTC · lore
---
 blame.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to blame.c +2 −2
diff --git a/blame.c b/blame.c
index 1be1cd82a2..b475bfa1c0 100644
--- a/blame.c
+++ b/blame.c
@@ -1353,8 +1353,8 @@ static struct blame_origin *find_origin(struct repository *r,
 	else {
 		int compute_diff = 1;
 		if (origin->commit->parents &&
-		    !oidcmp(&parent->object.oid,
-			    &origin->commit->parents->item->object.oid))
+		    oideq(&parent->object.oid,
+			  &origin->commit->parents->item->object.oid))
 			compute_diff = maybe_changed_path(r, origin, bd);
 
 		if (compute_diff)

-- 
2.28.0
Edmundo Carmona Antoranz· Sep 8, 2020, 13:55 UTC · re: Edmundo Carmona Antoranz · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On Mon, Sep 7, 2020 at 11:21 AM Edmundo Carmona Antoranz <eantoranz@gmail.com> wrote:

> Blamed the wrong branch. I should have looped Derrick instead of Jeff.
> Sorry about that.

I realized I didn't sign it off. Should I send it again? Or given that it's an almost 1-liner, it's ok? If I send it again, I will provide just a little more context about having the !oidcmp calls replaced for oideq in previous versions.

Derrick Stolee· Sep 8, 2020, 19:07 UTC · re: Edmundo Carmona Antoranz · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On 9/7/2020 1:16 PM, Edmundo Carmona Antoranz wrote:
> ---

Please include sign-off. I saw you reported your intention there in another message, but it's probably best to just send it again.

This message could also mention 14438c4 (introduce hasheq() and oideq(), 2018-08-28) which introduced oideq().

This use of !oidcmp() was introduced by 0906ac2b (blame: use changed-path Bloom filters, 2020-04-16). My bad. There is no good reason to introduce this use since it is well after the oideq() method was introduced.

Show 9 quoted lines
> @@ -1353,8 +1353,8 @@ static struct blame_origin *find_origin(struct repository *r,
>  	else {
>  		int compute_diff = 1;
>  		if (origin->commit->parents &&
> -		    !oidcmp(&parent->object.oid,
> -			    &origin->commit->parents->item->object.oid))
> +		    oideq(&parent->object.oid,
> +			  &origin->commit->parents->item->object.oid))
>  			compute_diff = maybe_changed_path(r, origin, bd);
The code itself looks correct.

Thanks, -Stolee

Jeff King· Sep 9, 2020, 09:11 UTC · re: Derrick Stolee · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On Tue, Sep 08, 2020 at 03:07:34PM -0400, Derrick Stolee wrote:
Show 19 quoted lines
> This message could also mention 14438c4 (introduce hasheq() and
> oideq(), 2018-08-28) which introduced oideq().
> 
> This use of !oidcmp() was introduced by 0906ac2b (blame: use
> changed-path Bloom filters, 2020-04-16). My bad. There is no
> good reason to introduce this use since it is well after the
> oideq() method was introduced.
> 
> > @@ -1353,8 +1353,8 @@ static struct blame_origin *find_origin(struct repository *r,
> >  	else {
> >  		int compute_diff = 1;
> >  		if (origin->commit->parents &&
> > -		    !oidcmp(&parent->object.oid,
> > -			    &origin->commit->parents->item->object.oid))
> > +		    oideq(&parent->object.oid,
> > +			  &origin->commit->parents->item->object.oid))
> >  			compute_diff = maybe_changed_path(r, origin, bd);
> 
> The code itself looks correct.

Yeah, it looks obviously correct. I am puzzled why "make coccicheck" doesn't find this, though. +cc René, as my favorite target for coccinelle nerd-snipes. :)

(But clearly we should make the change with or without figuring out the coccinelle part).

-Peff
Edmundo Carmona Antoranz· Sep 9, 2020, 14:00 UTC · re: Jeff King · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On Wed, Sep 9, 2020 at 3:11 AM Jeff King <peff@peff.net> wrote:
Show 5 quoted lines
>
> Yeah, it looks obviously correct. I am puzzled why "make coccicheck"
> doesn't find this, though. +cc René, as my favorite target for
> coccinelle nerd-snipes. :)
>
I added this to contrib/coccinelle/object_id.cocci in v2.27.0
@@
identifier f != oideq;
expression E1, E2;
@@
- !oidcmp(E1, E2)
+ oideq(E1, E2)
And it found it:

$ cat contrib/coccinelle/object_id.cocci.patch diff -u -p a/blame.c b/blame.c

Show changes to blame.c +1 −2
--- a/blame.c
+++ b/blame.c
@@ -1352,8 +1352,7 @@ static struct blame_origin *find_origin(
       else {
               int compute_diff = 1;
               if (origin->commit->parents &&
-                   !oidcmp(&parent->object.oid,
-                           &origin->commit->parents->item->object.oid))
+                   oideq(&parent->object.oid,
&origin->commit->parents->item->object.oid))
                       compute_diff = maybe_changed_path(r, origin, bd);

               if (compute_diff)


Do I need to add more things into the coccinelle definition so that it
is more restrictive in terms of the
expression we are hunting down?
Jeff Smith· Sep 9, 2020, 17:48 UTC · re: Edmundo Carmona Antoranz · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

I haven't had a chance to look at the cocci script, but I did have one thought...

Derrick pointed out, 14438c4 added both oideq and hasheq. It might be good to have a similar check for hasheq, if there is not one already.

On Wed, Sep 9, 2020 at 9:01 AM Edmundo Carmona Antoranz <eantoranz@gmail.com> wrote:

Show 39 quoted lines
>
> On Wed, Sep 9, 2020 at 3:11 AM Jeff King <peff@peff.net> wrote:
> >
> > Yeah, it looks obviously correct. I am puzzled why "make coccicheck"
> > doesn't find this, though. +cc René, as my favorite target for
> > coccinelle nerd-snipes. :)
> >
>
> I added this to contrib/coccinelle/object_id.cocci in v2.27.0
>
> @@
> identifier f != oideq;
> expression E1, E2;
> @@
> - !oidcmp(E1, E2)
> + oideq(E1, E2)
>
> And it found it:
>
> $ cat contrib/coccinelle/object_id.cocci.patch
> diff -u -p a/blame.c b/blame.c
> --- a/blame.c
> +++ b/blame.c
> @@ -1352,8 +1352,7 @@ static struct blame_origin *find_origin(
>        else {
>                int compute_diff = 1;
>                if (origin->commit->parents &&
> -                   !oidcmp(&parent->object.oid,
> -                           &origin->commit->parents->item->object.oid))
> +                   oideq(&parent->object.oid,
> &origin->commit->parents->item->object.oid))
>                        compute_diff = maybe_changed_path(r, origin, bd);
>
>                if (compute_diff)
>
>
> Do I need to add more things into the coccinelle definition so that it
> is more restrictive in terms of the
> expression we are hunting down?
Jeff King· Sep 9, 2020, 19:13 UTC · re: Edmundo Carmona Antoranz · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On Wed, Sep 09, 2020 at 08:00:57AM -0600, Edmundo Carmona Antoranz wrote:
Show 17 quoted lines
> On Wed, Sep 9, 2020 at 3:11 AM Jeff King <peff@peff.net> wrote:
> >
> > Yeah, it looks obviously correct. I am puzzled why "make coccicheck"
> > doesn't find this, though. +cc René, as my favorite target for
> > coccinelle nerd-snipes. :)
> >
> 
> I added this to contrib/coccinelle/object_id.cocci in v2.27.0
> 
> @@
> identifier f != oideq;
> expression E1, E2;
> @@
> - !oidcmp(E1, E2)
> + oideq(E1, E2)
> 
> And it found it:
Interesting. The existing rule is:
  struct object_id *OIDPTR1;
  struct object_id *OIDPTR2;
  @@
  - oidcmp(OIDPTR1, OIDPTR2) == 0
  + oideq(OIDPTR1, OIDPTR2)

The "== 0" part looks like it might be significant, but it's not. Coccinelle knows that "!foo" is the same as "foo == 0" (and you can confirm by tweaking it).

The addition of "identifer f != oideq" here isn't necessary (we don't even define an "f" in the semantic patch part). And anyway, we use hasheq() inside oideq(), so no need to override the rule there.

So the relevant part is probably that our existing rule specifies the exact type, whereas your rule allows any expression.

And indeed, if I do this, it works:
Show changes to contrib/coccinelle/object_id.cocci +2 −3
diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci
index ddf4f22bd7..62a6cee0eb 100644
--- a/contrib/coccinelle/object_id.cocci
+++ b/contrib/coccinelle/object_id.cocci
@@ -55,8 +55,8 @@ struct object_id OID;
 + oidcmp(&OID, OIDPTR)
 
 @@
-struct object_id *OIDPTR1;
-struct object_id *OIDPTR2;
+expression OIDPTR1;
+expression OIDPTR2;
 @@
 - oidcmp(OIDPTR1, OIDPTR2) == 0
 + oideq(OIDPTR1, OIDPTR2)

Which really _seems_ like a bug in coccinelle, unless I am missing
something. Because both of those parameters look like object_id pointers
(and the compiler would be complaining if it were not the case).  But I
also wonder if giving the specific types in the coccinelle rule is
buying us anything. If you passed two void pointers or ints or whatever
to !oidcmp(), we'd still want to rewrite it as oideq().

-Peff
Jeff King· Sep 9, 2020, 19:17 UTC · re: Jeff King · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On Wed, Sep 09, 2020 at 03:13:46PM -0400, Jeff King wrote:
Show 6 quoted lines
> Which really _seems_ like a bug in coccinelle, unless I am missing
> something. Because both of those parameters look like object_id pointers
> (and the compiler would be complaining if it were not the case).  But I
> also wonder if giving the specific types in the coccinelle rule is
> buying us anything. If you passed two void pointers or ints or whatever
> to !oidcmp(), we'd still want to rewrite it as oideq().

And indeed, just blindly swapping out "struct object_id" for "expression" in the coccinelle file (patch below), shows another spot that was missed:

diff -u -p a/packfile.c b/packfile.c
Show changes to 2 files +15 −17

packfile.c, contrib/coccinelle/object_id.cocci

--- a/packfile.c
+++ b/packfile.c
@@ -735,7 +735,7 @@ struct packed_git *add_packed_git(const
 	p->mtime = st.st_mtime;
 	if (path_len < the_hash_algo->hexsz ||
 	    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))
-		hashclr(p->hash);
+		oidclr(p);
 	return p;
 }
 

Maybe it's worth being looser in our cocci patch definitions. I'm having
trouble thinking of a downside...

-Peff

-- >8 --
Here's the patch to loosen object_id.cocci. Perhaps we'd want to do the
same in other files.

diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci
index ddf4f22bd7..738c60923e 100644
--- a/contrib/coccinelle/object_id.cocci
+++ b/contrib/coccinelle/object_id.cocci
@@ -1,62 +1,62 @@
 @@
-struct object_id OID;
+expression OID;
 @@
 - is_null_sha1(OID.hash)
 + is_null_oid(&OID)
 
 @@
-struct object_id *OIDPTR;
+expression *OIDPTR;
 @@
 - is_null_sha1(OIDPTR->hash)
 + is_null_oid(OIDPTR)
 
 @@
-struct object_id OID;
+expression OID;
 @@
 - hashclr(OID.hash)
 + oidclr(&OID)
 
 @@
 identifier f != oidclr;
-struct object_id *OIDPTR;
+expression *OIDPTR;
 @@
   f(...) {<...
 - hashclr(OIDPTR->hash)
 + oidclr(OIDPTR)
   ...>}
 
 @@
-struct object_id OID1, OID2;
+expression OID1, OID2;
 @@
 - hashcmp(OID1.hash, OID2.hash)
 + oidcmp(&OID1, &OID2)
 
 @@
 identifier f != oidcmp;
-struct object_id *OIDPTR1, OIDPTR2;
+expression *OIDPTR1, OIDPTR2;
 @@
   f(...) {<...
 - hashcmp(OIDPTR1->hash, OIDPTR2->hash)
 + oidcmp(OIDPTR1, OIDPTR2)
   ...>}
 
 @@
-struct object_id *OIDPTR;
-struct object_id OID;
+expression *OIDPTR;
+expression OID;
 @@
 - hashcmp(OIDPTR->hash, OID.hash)
 + oidcmp(OIDPTR, &OID)
 
 @@
-struct object_id *OIDPTR;
-struct object_id OID;
+expression *OIDPTR;
+expression OID;
 @@
 - hashcmp(OID.hash, OIDPTR->hash)
 + oidcmp(&OID, OIDPTR)
 
 @@
-struct object_id *OIDPTR1;
-struct object_id *OIDPTR2;
+expression OIDPTR1;
+expression OIDPTR2;
 @@
 - oidcmp(OIDPTR1, OIDPTR2) == 0
 + oideq(OIDPTR1, OIDPTR2)
@@ -71,8 +71,8 @@ expression E1, E2;
   ...>}
 
 @@
-struct object_id *OIDPTR1;
-struct object_id *OIDPTR2;
+expression *OIDPTR1;
+expression *OIDPTR2;
 @@
 - oidcmp(OIDPTR1, OIDPTR2) != 0
 + !oideq(OIDPTR1, OIDPTR2)
René Scharfe· Sep 9, 2020, 19:54 UTC · re: Jeff King · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

Am 09.09.20 um 21:17 schrieb Jeff King:
Show 8 quoted lines
> On Wed, Sep 09, 2020 at 03:13:46PM -0400, Jeff King wrote:
>
>> Which really _seems_ like a bug in coccinelle, unless I am missing
>> something. Because both of those parameters look like object_id pointers
>> (and the compiler would be complaining if it were not the case).  But I
>> also wonder if giving the specific types in the coccinelle rule is
>> buying us anything. If you passed two void pointers or ints or whatever
>> to !oidcmp(), we'd still want to rewrite it as oideq().

Right, using expressions for such a like-for-like transformation is safe and practical in the sense that it won't break correct code, and broken code will be flagged by the compiler.

Show 20 quoted lines
>
> And indeed, just blindly swapping out "struct object_id" for
> "expression" in the coccinelle file (patch below), shows another spot
> that was missed:
>
> diff -u -p a/packfile.c b/packfile.c
> --- a/packfile.c
> +++ b/packfile.c
> @@ -735,7 +735,7 @@ struct packed_git *add_packed_git(const
>  	p->mtime = st.st_mtime;
>  	if (path_len < the_hash_algo->hexsz ||
>  	    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))
> -		hashclr(p->hash);
> +		oidclr(p);
>  	return p;
>  }
>
>
> Maybe it's worth being looser in our cocci patch definitions. I'm having
> trouble thinking of a downside...

For transformations that change the type as in the example above we should insist on getting the right one, otherwise we might introduce bugs -- like in the example above. p points to a struct packed_git and not to a struct object_id, so this introduces a type mismatch.

We better make sure our semantic patches are safe, otherwise we have to check all conversions very carefully, and then we might be better off doing them manually..

René
Jeff King· Sep 9, 2020, 19:58 UTC · re: René Scharfe · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

On Wed, Sep 09, 2020 at 09:54:55PM +0200, René Scharfe wrote:
Show 20 quoted lines
> > diff -u -p a/packfile.c b/packfile.c
> > --- a/packfile.c
> > +++ b/packfile.c
> > @@ -735,7 +735,7 @@ struct packed_git *add_packed_git(const
> >  	p->mtime = st.st_mtime;
> >  	if (path_len < the_hash_algo->hexsz ||
> >  	    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))
> > -		hashclr(p->hash);
> > +		oidclr(p);
> >  	return p;
> >  }
> >
> >
> > Maybe it's worth being looser in our cocci patch definitions. I'm having
> > trouble thinking of a downside...
> 
> For transformations that change the type as in the example above we
> should insist on getting the right one, otherwise we might introduce
> bugs -- like in the example above.  p points to a struct packed_git and
> not to a struct object_id, so this introduces a type mismatch.

Heh. You'd think that I would have applied that patch and run "make". Or even read it carefully.

Thanks for pointing that out. I guess now we have a real example of a downside (the compiler _would_ still catch it, but it means "make coccicheck" is useless if it's repeatedly suggesting a bad transformation).

-Peff
Junio C Hamano· Sep 9, 2020, 20:03 UTC · re: René Scharfe · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

René Scharfe <l.s.r@web.de> writes:
Show 20 quoted lines
>> diff -u -p a/packfile.c b/packfile.c
>> --- a/packfile.c
>> +++ b/packfile.c
>> @@ -735,7 +735,7 @@ struct packed_git *add_packed_git(const
>>  	p->mtime = st.st_mtime;
>>  	if (path_len < the_hash_algo->hexsz ||
>>  	    get_sha1_hex(path + path_len - the_hash_algo->hexsz, p->hash))
>> -		hashclr(p->hash);
>> +		oidclr(p);
>>  	return p;
>>  }
>>
>>
>> Maybe it's worth being looser in our cocci patch definitions. I'm having
>> trouble thinking of a downside...
>
> For transformations that change the type as in the example above we
> should insist on getting the right one, otherwise we might introduce
> bugs -- like in the example above.  p points to a struct packed_git and
> not to a struct object_id, so this introduces a type mismatch.
;-)  A good counter-example.
> We better make sure our semantic patches are safe, otherwise we have to
> check all conversions very carefully, and then we might be better off
> doing them manually..
Yes, that is a sensible suggestion.
Junio C Hamano· Sep 9, 2020, 20:06 UTC · re: Jeff King · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

Jeff King <peff@peff.net> writes:
Show 19 quoted lines
>  @@
> -struct object_id *OIDPTR1;
> -struct object_id *OIDPTR2;
> +expression OIDPTR1;
> +expression OIDPTR2;
>  @@
>  - oidcmp(OIDPTR1, OIDPTR2) == 0
>  + oideq(OIDPTR1, OIDPTR2)
> @@ -71,8 +71,8 @@ expression E1, E2;
>    ...>}
>  
>  @@
> -struct object_id *OIDPTR1;
> -struct object_id *OIDPTR2;
> +expression *OIDPTR1;
> +expression *OIDPTR2;
>  @@
>  - oidcmp(OIDPTR1, OIDPTR2) != 0
>  + !oideq(OIDPTR1, OIDPTR2)

With an extra insight from the counter-example Réne pointed out in your message, I think the above two are safe but all the others are unsafe.

René Scharfe· Sep 9, 2020, 20:43 UTC · re: Jeff King · lore

Re: [PATCH] blame.c: replace instance of !oidcmp for oideq

Am 09.09.20 um 21:13 schrieb Jeff King:
Show 31 quoted lines
> On Wed, Sep 09, 2020 at 08:00:57AM -0600, Edmundo Carmona Antoranz wrote:
>
>> On Wed, Sep 9, 2020 at 3:11 AM Jeff King <peff@peff.net> wrote:
>>>
>>> Yeah, it looks obviously correct. I am puzzled why "make coccicheck"
>>> doesn't find this, though. +cc René, as my favorite target for
>>> coccinelle nerd-snipes. :)
>>>
>>
>> I added this to contrib/coccinelle/object_id.cocci in v2.27.0
>>
>> @@
>> identifier f != oideq;
>> expression E1, E2;
>> @@
>> - !oidcmp(E1, E2)
>> + oideq(E1, E2)
>>
>> And it found it:
>
> Interesting. The existing rule is:
>
>   struct object_id *OIDPTR1;
>   struct object_id *OIDPTR2;
>   @@
>   - oidcmp(OIDPTR1, OIDPTR2) == 0
>   + oideq(OIDPTR1, OIDPTR2)
>
> The "== 0" part looks like it might be significant, but it's not.
> Coccinelle knows that "!foo" is the same as "foo == 0" (and you can
> confirm by tweaking it).

It is significant in the sense that "x == 0" in the semantic patch also matches "!x" in the code, but "!x" in the semantic patch doesn't match "x == 0". That's because coccinelle has this isomorphism built in (in /usr/lib/coccinelle/standard.iso on my machine):

Expression
@ not_int1 @
int X;
@@
 !X => X == 0

It's a one-way isomorphism (i.e. a rule that says that certain expressions have the same meaning). So we should use "x == 0" over "!x" in semantic patches to cover both cases.

Show 24 quoted lines
> So the relevant part is probably that our existing rule specifies the
> exact type, whereas your rule allows any expression.
>
> And indeed, if I do this, it works:
>
> diff --git a/contrib/coccinelle/object_id.cocci b/contrib/coccinelle/object_id.cocci
> index ddf4f22bd7..62a6cee0eb 100644
> --- a/contrib/coccinelle/object_id.cocci
> +++ b/contrib/coccinelle/object_id.cocci
> @@ -55,8 +55,8 @@ struct object_id OID;
>  + oidcmp(&OID, OIDPTR)
>
>  @@
> -struct object_id *OIDPTR1;
> -struct object_id *OIDPTR2;
> +expression OIDPTR1;
> +expression OIDPTR2;
>  @@
>  - oidcmp(OIDPTR1, OIDPTR2) == 0
>  + oideq(OIDPTR1, OIDPTR2)
>
> Which really _seems_ like a bug in coccinelle, unless I am missing
> something. Because both of those parameters look like object_id pointers
> (and the compiler would be complaining if it were not the case).

Yes, seems it looks like coccinelle gives up trying to determine the type of these things.

And while this one here matches the example in blame.c:
@@
expression A, B;
@@
- 0 == oidcmp(A, B)
+ oideq(A, B)
... and this one does as well:
@@
expression A, B;
@@
- !oidcmp
+ oideq
  (A, B)
... the following one doesn't:
@@
expression A, B;
@@
- 0 == oidcmp
+ oideq
  (A, B)
... and neither does this one:
@@
expression A, B;
@@
- oidcmp
+ oideq
  (A, B)
- == 0

So it helps to try some variants in the hope to bypass some of the restrictions/bugs/misunderstandings. O_o

René

← back to recent threads