threads / patch / 46802

patch, 4 partscache.h: hex2chr() - avoid -Wsign-compare warnings

Subject: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

## tl;dr

8 messages between Sep 21, 2017 and Sep 22, 2017. Diffs are folded; open one to read it.

replies: 7people: 3as markdown or json

Ramsay Jones· Sep 21, 2017, 16:48 UTC · lore
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
---
 cache.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to cache.h +2 −2
diff --git a/cache.h b/cache.h
index a916bc79e..a0e3e362c 100644
--- a/cache.h
+++ b/cache.h
@@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c)
  */
 static inline int hex2chr(const char *s)
 {
-	int val = hexval(s[0]);
-	return (val < 0) ? val : (val << 4) | hexval(s[1]);
+	unsigned int val = hexval(s[0]);
+	return (val & ~0xf) ? val : (val << 4) | hexval(s[1]);
 }
 
 /* Convert to/from hex/sha1 representation */

-- 
2.14.0
Jeff King· Sep 22, 2017, 05:47 UTC · re: Ramsay Jones · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

On Thu, Sep 21, 2017 at 05:48:38PM +0100, Ramsay Jones wrote:
Show 13 quoted lines
> diff --git a/cache.h b/cache.h
> index a916bc79e..a0e3e362c 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c)
>   */
>  static inline int hex2chr(const char *s)
>  {
> -	int val = hexval(s[0]);
> -	return (val < 0) ? val : (val << 4) | hexval(s[1]);
> +	unsigned int val = hexval(s[0]);
> +	return (val & ~0xf) ? val : (val << 4) | hexval(s[1]);
>  }

Ironically, the unsigned return from hexval() comes from internally converting the signed char in hexval_table. And then we again return it as a signed int from hex2chr().

Would it make sense to return a signed int from hexval()? That would make hex2chr just work as it tries to above. I admit that shifting signed values is a little funny, but it should be fine here since we know they're no larger than 8 bits in the first place.

As an aside, I also see some uses of hexval() that don't appear to be quite as rigorous in checking for invalid characters. A few unconditionally shift the first nibble and assume that there will still be high bits set. I think that's generally true for twos-complement negative numbers, but isn't shifting off the left side of a signed integer undefined behavior?

And mailinfo's decode_q_segment() does not seem to check for errors at all.

Handling those is getting far off your original patch, but I'm having trouble figuring out if it's saner for us to consistently stick to all-signed or all-unsigned here.

-Peff
Ramsay Jones· Sep 22, 2017, 16:05 UTC · re: Jeff King · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

On 22/09/17 06:47, Jeff King wrote:
Show 19 quoted lines
> On Thu, Sep 21, 2017 at 05:48:38PM +0100, Ramsay Jones wrote:
> 
>> diff --git a/cache.h b/cache.h
>> index a916bc79e..a0e3e362c 100644
>> --- a/cache.h
>> +++ b/cache.h
>> @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c)
>>   */
>>  static inline int hex2chr(const char *s)
>>  {
>> -	int val = hexval(s[0]);
>> -	return (val < 0) ? val : (val << 4) | hexval(s[1]);
>> +	unsigned int val = hexval(s[0]);
>> +	return (val & ~0xf) ? val : (val << 4) | hexval(s[1]);
>>  }
> 
> Ironically, the unsigned return from hexval() comes from internally
> converting the signed char in hexval_table. And then we again return it
> as a signed int from hex2chr().

Yep, my first inclination was to change the return type of hexval(), but after looking at all its callers, I decided against that (because it wouldn't help). ;-)

> Would it make sense to return a signed int from hexval()? That would
> make hex2chr just work as it tries to above. I admit that shifting
> signed values is a little funny, but it should be fine here since we
> know they're no larger than 8 bits in the first place.

Indeed, shifting signed values is a no-no, which is why I decided to go this route.

Show 6 quoted lines
> As an aside, I also see some uses of hexval() that don't appear to be
> quite as rigorous in checking for invalid characters. A few
> unconditionally shift the first nibble and assume that there will still
> be high bits set. I think that's generally true for twos-complement
> negative numbers, but isn't shifting off the left side of a signed
> integer undefined behavior?

All uses of hexval() that I can see are shifting an unsigned value. Have I missed something?

> And mailinfo's decode_q_segment() does not seem to check for errors at
> all.
Yes, I noticed that. (I put it on my TODO list).
> Handling those is getting far off your original patch, but I'm having
> trouble figuring out if it's saner for us to consistently stick to
> all-signed or all-unsigned here.
Oh, unsigned, without a doubt. :D

ATB, Ramsay Jones

Jeff King· Sep 22, 2017, 16:11 UTC · re: Ramsay Jones · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

On Fri, Sep 22, 2017 at 05:05:03PM +0100, Ramsay Jones wrote:
Show 9 quoted lines
> > As an aside, I also see some uses of hexval() that don't appear to be
> > quite as rigorous in checking for invalid characters. A few
> > unconditionally shift the first nibble and assume that there will still
> > be high bits set. I think that's generally true for twos-complement
> > negative numbers, but isn't shifting off the left side of a signed
> > integer undefined behavior?
> 
> All uses of hexval() that I can see are shifting an unsigned value.
> Have I missed something?
Hmm. get_hex_color() does:
  unsigned int val;
  val = (hexval(in[0]) << 4) | hexval(in[1]));

Isn't that shifting the signed return value of hexval(), and then converting it to unsigned afterwards?

I've been confused by C's integer conversion rules before, though, so perhaps I'm wrong.

I think if this function is fed an empty string that it will also read past the end of the buffer for in[1]. It shouldn't matter, since the NUL in in[0] would cause us to return an error regardless, but it's still undefined behavior.

In fact, this whole function is just hex2chr() implemented badly. Who is responsible for this terrible code? ;)

-Peff
Jeff King· Sep 22, 2017, 16:18 UTC · re: Jeff King · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

On Fri, Sep 22, 2017 at 12:11:59PM -0400, Jeff King wrote:
Show 22 quoted lines
> On Fri, Sep 22, 2017 at 05:05:03PM +0100, Ramsay Jones wrote:
> 
> > > As an aside, I also see some uses of hexval() that don't appear to be
> > > quite as rigorous in checking for invalid characters. A few
> > > unconditionally shift the first nibble and assume that there will still
> > > be high bits set. I think that's generally true for twos-complement
> > > negative numbers, but isn't shifting off the left side of a signed
> > > integer undefined behavior?
> > 
> > All uses of hexval() that I can see are shifting an unsigned value.
> > Have I missed something?
> 
> Hmm. get_hex_color() does:
> 
>   unsigned int val;
>   val = (hexval(in[0]) << 4) | hexval(in[1]));
> 
> Isn't that shifting the signed return value of hexval(), and then
> converting it to unsigned afterwards?
> 
> I've been confused by C's integer conversion rules before, though, so
> perhaps I'm wrong.

Oh, nevermind. I managed to confuse myself again. The return value from hexval() is already unsigned. It's hex2chr that has the funny signed return. So the signedness is fine.

> I think if this function is fed an empty string that it will also read
> past the end of the buffer for in[1]. It shouldn't matter, since the NUL
> in in[0] would cause us to return an error regardless, but it's still
> undefined behavior.
This is still a bug, though.
-Peff
Jeff King· Sep 22, 2017, 16:22 UTC · re: Jeff King · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

On Fri, Sep 22, 2017 at 12:18:17PM -0400, Jeff King wrote:
Show 6 quoted lines
> > I think if this function is fed an empty string that it will also read
> > past the end of the buffer for in[1]. It shouldn't matter, since the NUL
> > in in[0] would cause us to return an error regardless, but it's still
> > undefined behavior.
> 
> This is still a bug, though.
Last message, I promise. ;)

I started on the minimal fix for this, but actually it's OK by virtue of its sole caller first checking that we have enough length (because we're not parsing a string, in fact, but a ptr/len buffer).

So all is well, though I think get_hex_color() does serve as a poor example if somebody were to try to adapt it generally (hopefully they wouldn't, since hex2chr() is already globally available).

Sorry for the all the noise.
-Peff
Ramsay Jones· Sep 22, 2017, 16:24 UTC · re: Jeff King · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

On 22/09/17 17:11, Jeff King wrote:
Show 19 quoted lines
> On Fri, Sep 22, 2017 at 05:05:03PM +0100, Ramsay Jones wrote:
> 
>>> As an aside, I also see some uses of hexval() that don't appear to be
>>> quite as rigorous in checking for invalid characters. A few
>>> unconditionally shift the first nibble and assume that there will still
>>> be high bits set. I think that's generally true for twos-complement
>>> negative numbers, but isn't shifting off the left side of a signed
>>> integer undefined behavior?
>>
>> All uses of hexval() that I can see are shifting an unsigned value.
>> Have I missed something?
> 
> Hmm. get_hex_color() does:
> 
>   unsigned int val;
>   val = (hexval(in[0]) << 4) | hexval(in[1]));
> 
> Isn't that shifting the signed return value of hexval(), and then
> converting it to unsigned afterwards?

Err ... no. the return value of hexval() is *unsigned int*. (which is kinda the point!)

Show 7 quoted lines
> I've been confused by C's integer conversion rules before, though, so
> perhaps I'm wrong.
> 
> I think if this function is fed an empty string that it will also read
> past the end of the buffer for in[1]. It shouldn't matter, since the NUL
> in in[0] would cause us to return an error regardless, but it's still
> undefined behavior.
Correct.
> In fact, this whole function is just hex2chr() implemented badly. Who is
> responsible for this terrible code? ;)
;-)

ATB, Ramsay Jones

René Scharfe· Sep 22, 2017, 18:01 UTC · re: Jeff King · lore

Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings

Am 22.09.2017 um 07:47 schrieb Jeff King:
Show 24 quoted lines
> On Thu, Sep 21, 2017 at 05:48:38PM +0100, Ramsay Jones wrote:
> 
>> diff --git a/cache.h b/cache.h
>> index a916bc79e..a0e3e362c 100644
>> --- a/cache.h
>> +++ b/cache.h
>> @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c)
>>    */
>>   static inline int hex2chr(const char *s)
>>   {
>> -	int val = hexval(s[0]);
>> -	return (val < 0) ? val : (val << 4) | hexval(s[1]);
>> +	unsigned int val = hexval(s[0]);
>> +	return (val & ~0xf) ? val : (val << 4) | hexval(s[1]);
>>   }
> 
> Ironically, the unsigned return from hexval() comes from internally
> converting the signed char in hexval_table. And then we again return it
> as a signed int from hex2chr().
> 
> Would it make sense to return a signed int from hexval()? That would
> make hex2chr just work as it tries to above. I admit that shifting
> signed values is a little funny, but it should be fine here since we
> know they're no larger than 8 bits in the first place.

I'd say yes. We can replace that shift with a multiplication -- compilers will nevertheless use a shift even with -O0 (if beneficial).

Show 9 quoted lines
> As an aside, I also see some uses of hexval() that don't appear to be
> quite as rigorous in checking for invalid characters. A few
> unconditionally shift the first nibble and assume that there will still
> be high bits set. I think that's generally true for twos-complement
> negative numbers, but isn't shifting off the left side of a signed
> integer undefined behavior?
> 
> And mailinfo's decode_q_segment() does not seem to check for errors at
> all.
It also allocates and returns a strbuf, which seems odd as well.
René

← back to recent threads