# [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings 8 messages from 2017-09-21 to 2017-09-22. Participants: Ramsay Jones, Jeff King, René Scharfe. Thread: https://gitlist.dev/t/46802 ## Ramsay Jones, 2017-09-21 16:48 Subject: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: URL: https://gitlist.dev/e/ae537e7e-2c3d-f7f3-28f4-451c37c01bbb%40ramsayjones.plus.com ``` Signed-off-by: Ramsay Jones --- cache.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cache.h b/cache.h index a916bc79e..a0e3e362c 100644 --- a/cache.h +++ b/cache.h @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c) */ static inline int hex2chr(const char *s) { - int val = hexval(s[0]); - return (val < 0) ? val : (val << 4) | hexval(s[1]); + unsigned int val = hexval(s[0]); + return (val & ~0xf) ? val : (val << 4) | hexval(s[1]); } /* Convert to/from hex/sha1 representation */ -- 2.14.0 ``` ## Jeff King, 2017-09-22 05:47 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: <20170922054748.iseinawwwfw56vis@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170922054748.iseinawwwfw56vis%40sigill.intra.peff.net In-Reply-To: ``` On Thu, Sep 21, 2017 at 05:48:38PM +0100, Ramsay Jones wrote: > diff --git a/cache.h b/cache.h > index a916bc79e..a0e3e362c 100644 > --- a/cache.h > +++ b/cache.h > @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c) > */ > static inline int hex2chr(const char *s) > { > - int val = hexval(s[0]); > - return (val < 0) ? val : (val << 4) | hexval(s[1]); > + unsigned int val = hexval(s[0]); > + return (val & ~0xf) ? val : (val << 4) | hexval(s[1]); > } Ironically, the unsigned return from hexval() comes from internally converting the signed char in hexval_table. And then we again return it as a signed int from hex2chr(). Would it make sense to return a signed int from hexval()? That would make hex2chr just work as it tries to above. I admit that shifting signed values is a little funny, but it should be fine here since we know they're no larger than 8 bits in the first place. As an aside, I also see some uses of hexval() that don't appear to be quite as rigorous in checking for invalid characters. A few unconditionally shift the first nibble and assume that there will still be high bits set. I think that's generally true for twos-complement negative numbers, but isn't shifting off the left side of a signed integer undefined behavior? And mailinfo's decode_q_segment() does not seem to check for errors at all. Handling those is getting far off your original patch, but I'm having trouble figuring out if it's saner for us to consistently stick to all-signed or all-unsigned here. -Peff ``` ## Ramsay Jones, 2017-09-22 16:05 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: <89beb424-2cc0-bfe3-7847-c11035bfec70@ramsayjones.plus.com> URL: https://gitlist.dev/e/89beb424-2cc0-bfe3-7847-c11035bfec70%40ramsayjones.plus.com In-Reply-To: <20170922054748.iseinawwwfw56vis@sigill.intra.peff.net> ``` On 22/09/17 06:47, Jeff King wrote: > On Thu, Sep 21, 2017 at 05:48:38PM +0100, Ramsay Jones wrote: > >> diff --git a/cache.h b/cache.h >> index a916bc79e..a0e3e362c 100644 >> --- a/cache.h >> +++ b/cache.h >> @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c) >> */ >> static inline int hex2chr(const char *s) >> { >> - int val = hexval(s[0]); >> - return (val < 0) ? val : (val << 4) | hexval(s[1]); >> + unsigned int val = hexval(s[0]); >> + return (val & ~0xf) ? val : (val << 4) | hexval(s[1]); >> } > > Ironically, the unsigned return from hexval() comes from internally > converting the signed char in hexval_table. And then we again return it > as a signed int from hex2chr(). Yep, my first inclination was to change the return type of hexval(), but after looking at all its callers, I decided against that (because it wouldn't help). ;-) > Would it make sense to return a signed int from hexval()? That would > make hex2chr just work as it tries to above. I admit that shifting > signed values is a little funny, but it should be fine here since we > know they're no larger than 8 bits in the first place. Indeed, shifting signed values is a no-no, which is why I decided to go this route. > As an aside, I also see some uses of hexval() that don't appear to be > quite as rigorous in checking for invalid characters. A few > unconditionally shift the first nibble and assume that there will still > be high bits set. I think that's generally true for twos-complement > negative numbers, but isn't shifting off the left side of a signed > integer undefined behavior? All uses of hexval() that I can see are shifting an unsigned value. Have I missed something? > And mailinfo's decode_q_segment() does not seem to check for errors at > all. Yes, I noticed that. (I put it on my TODO list). > Handling those is getting far off your original patch, but I'm having > trouble figuring out if it's saner for us to consistently stick to > all-signed or all-unsigned here. Oh, unsigned, without a doubt. :D ATB, Ramsay Jones ``` ## Jeff King, 2017-09-22 16:11 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: <20170922161159.hkw4id4kpnwo7jou@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170922161159.hkw4id4kpnwo7jou%40sigill.intra.peff.net In-Reply-To: <89beb424-2cc0-bfe3-7847-c11035bfec70@ramsayjones.plus.com> ``` On Fri, Sep 22, 2017 at 05:05:03PM +0100, Ramsay Jones wrote: > > As an aside, I also see some uses of hexval() that don't appear to be > > quite as rigorous in checking for invalid characters. A few > > unconditionally shift the first nibble and assume that there will still > > be high bits set. I think that's generally true for twos-complement > > negative numbers, but isn't shifting off the left side of a signed > > integer undefined behavior? > > All uses of hexval() that I can see are shifting an unsigned value. > Have I missed something? Hmm. get_hex_color() does: unsigned int val; val = (hexval(in[0]) << 4) | hexval(in[1])); Isn't that shifting the signed return value of hexval(), and then converting it to unsigned afterwards? I've been confused by C's integer conversion rules before, though, so perhaps I'm wrong. I think if this function is fed an empty string that it will also read past the end of the buffer for in[1]. It shouldn't matter, since the NUL in in[0] would cause us to return an error regardless, but it's still undefined behavior. In fact, this whole function is just hex2chr() implemented badly. Who is responsible for this terrible code? ;) -Peff ``` ## Jeff King, 2017-09-22 16:18 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: <20170922161817.iy5tpexouxn5uwsb@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170922161817.iy5tpexouxn5uwsb%40sigill.intra.peff.net In-Reply-To: <20170922161159.hkw4id4kpnwo7jou@sigill.intra.peff.net> ``` On Fri, Sep 22, 2017 at 12:11:59PM -0400, Jeff King wrote: > On Fri, Sep 22, 2017 at 05:05:03PM +0100, Ramsay Jones wrote: > > > > As an aside, I also see some uses of hexval() that don't appear to be > > > quite as rigorous in checking for invalid characters. A few > > > unconditionally shift the first nibble and assume that there will still > > > be high bits set. I think that's generally true for twos-complement > > > negative numbers, but isn't shifting off the left side of a signed > > > integer undefined behavior? > > > > All uses of hexval() that I can see are shifting an unsigned value. > > Have I missed something? > > Hmm. get_hex_color() does: > > unsigned int val; > val = (hexval(in[0]) << 4) | hexval(in[1])); > > Isn't that shifting the signed return value of hexval(), and then > converting it to unsigned afterwards? > > I've been confused by C's integer conversion rules before, though, so > perhaps I'm wrong. Oh, nevermind. I managed to confuse myself again. The return value from hexval() is already unsigned. It's hex2chr that has the funny signed return. So the signedness is fine. > I think if this function is fed an empty string that it will also read > past the end of the buffer for in[1]. It shouldn't matter, since the NUL > in in[0] would cause us to return an error regardless, but it's still > undefined behavior. This is still a bug, though. -Peff ``` ## Jeff King, 2017-09-22 16:22 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: <20170922162251.kx2w5u4falztcpnh@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170922162251.kx2w5u4falztcpnh%40sigill.intra.peff.net In-Reply-To: <20170922161817.iy5tpexouxn5uwsb@sigill.intra.peff.net> ``` On Fri, Sep 22, 2017 at 12:18:17PM -0400, Jeff King wrote: > > I think if this function is fed an empty string that it will also read > > past the end of the buffer for in[1]. It shouldn't matter, since the NUL > > in in[0] would cause us to return an error regardless, but it's still > > undefined behavior. > > This is still a bug, though. Last message, I promise. ;) I started on the minimal fix for this, but actually it's OK by virtue of its sole caller first checking that we have enough length (because we're not parsing a string, in fact, but a ptr/len buffer). So all is well, though I think get_hex_color() does serve as a poor example if somebody were to try to adapt it generally (hopefully they wouldn't, since hex2chr() is already globally available). Sorry for the all the noise. -Peff ``` ## Ramsay Jones, 2017-09-22 16:24 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: URL: https://gitlist.dev/e/d47cdae8-b37f-f378-6df6-bf70a819dd5b%40ramsayjones.plus.com In-Reply-To: <20170922161159.hkw4id4kpnwo7jou@sigill.intra.peff.net> ``` On 22/09/17 17:11, Jeff King wrote: > On Fri, Sep 22, 2017 at 05:05:03PM +0100, Ramsay Jones wrote: > >>> As an aside, I also see some uses of hexval() that don't appear to be >>> quite as rigorous in checking for invalid characters. A few >>> unconditionally shift the first nibble and assume that there will still >>> be high bits set. I think that's generally true for twos-complement >>> negative numbers, but isn't shifting off the left side of a signed >>> integer undefined behavior? >> >> All uses of hexval() that I can see are shifting an unsigned value. >> Have I missed something? > > Hmm. get_hex_color() does: > > unsigned int val; > val = (hexval(in[0]) << 4) | hexval(in[1])); > > Isn't that shifting the signed return value of hexval(), and then > converting it to unsigned afterwards? Err ... no. the return value of hexval() is *unsigned int*. (which is kinda the point!) > I've been confused by C's integer conversion rules before, though, so > perhaps I'm wrong. > > I think if this function is fed an empty string that it will also read > past the end of the buffer for in[1]. It shouldn't matter, since the NUL > in in[0] would cause us to return an error regardless, but it's still > undefined behavior. Correct. > In fact, this whole function is just hex2chr() implemented badly. Who is > responsible for this terrible code? ;) ;-) ATB, Ramsay Jones ``` ## René Scharfe, 2017-09-22 18:01 Subject: Re: [PATCH 3/4] cache.h: hex2chr() - avoid -Wsign-compare warnings Message-ID: <5e21362b-7c42-37f5-73a8-907d4ad5db86@web.de> URL: https://gitlist.dev/e/5e21362b-7c42-37f5-73a8-907d4ad5db86%40web.de In-Reply-To: <20170922054748.iseinawwwfw56vis@sigill.intra.peff.net> ``` Am 22.09.2017 um 07:47 schrieb Jeff King: > On Thu, Sep 21, 2017 at 05:48:38PM +0100, Ramsay Jones wrote: > >> diff --git a/cache.h b/cache.h >> index a916bc79e..a0e3e362c 100644 >> --- a/cache.h >> +++ b/cache.h >> @@ -1243,8 +1243,8 @@ static inline unsigned int hexval(unsigned char c) >> */ >> static inline int hex2chr(const char *s) >> { >> - int val = hexval(s[0]); >> - return (val < 0) ? val : (val << 4) | hexval(s[1]); >> + unsigned int val = hexval(s[0]); >> + return (val & ~0xf) ? val : (val << 4) | hexval(s[1]); >> } > > Ironically, the unsigned return from hexval() comes from internally > converting the signed char in hexval_table. And then we again return it > as a signed int from hex2chr(). > > Would it make sense to return a signed int from hexval()? That would > make hex2chr just work as it tries to above. I admit that shifting > signed values is a little funny, but it should be fine here since we > know they're no larger than 8 bits in the first place. I'd say yes. We can replace that shift with a multiplication -- compilers will nevertheless use a shift even with -O0 (if beneficial). > As an aside, I also see some uses of hexval() that don't appear to be > quite as rigorous in checking for invalid characters. A few > unconditionally shift the first nibble and assume that there will still > be high bits set. I think that's generally true for twos-complement > negative numbers, but isn't shifting off the left side of a signed > integer undefined behavior? > > And mailinfo's decode_q_segment() does not seem to check for errors at > all. It also allocates and returns a strbuf, which seems odd as well. René ```