git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v12 04/10] utf8: teach same_encoding() alternative UTF encoding names

From
Lars Schneider <larsxschneider@gmail.com>
Date
Mar 15, 2018, 23:35 UTC
Message-ID
<328BE614-1927-450A-85D1-7391D0BB5C47@gmail.com>
In-Reply-To
<CAPig+cSvBi1pJYC-DND1_x9rHhmoAPs90B_3Fg53-X5FjtCQhA@mail.gmail.com>
Show 35 quoted lines
> On 16 Mar 2018, at 00:25, Eric Sunshine <sunshine@sunshineco.com> wrote:
> 
> On Thu, Mar 15, 2018 at 6:57 PM,  <lars.schneider@autodesk.com> wrote:
>> The function same_encoding() checked only for alternative UTF-8 encoding
>> names. Teach it to check for all kinds of alternative UTF encoding
>> names.
>> 
>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>> ---
>> diff --git a/utf8.c b/utf8.c
>> @@ -401,11 +401,27 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
>> +static int same_utf_encoding(const char *src, const char *dst)
>> +{
>> +       if (istarts_with(src, "utf") && istarts_with(dst, "utf")) {
>> +               /* src[3] or dst[3] might be '\0' */
>> +               int i = (src[3] == '-' ? 4 : 3);
>> +               int j = (dst[3] == '-' ? 4 : 3);
>> +               return !strcasecmp(src+i, dst+j);
>> +       }
>> +       return 0;
>> +}
> 
> Alternate implementation without magic numbers:
> 
>    if (iskip_prefix(src, "utf", &src) &&
>        iskip_prefix(dst, "utf", &dst)) {
>        if (*src == '-')
>            src++;
>        if (*dst == '-')
>            dst++;
>        return !strcasecmp(src, dst);
>    }
>    return 0;
> 
> ... assuming you add an iskip_prefix() function (patterned after skip_prefix()).
If a reroll is necessary, then I can do this.
Show 25 quoted lines
>> int is_encoding_utf8(const char *name)
>> {
>>        if (!name)
>>                return 1;
>> -       if (!strcasecmp(name, "utf-8") || !strcasecmp(name, "utf8"))
>> +       if (same_utf_encoding("utf-8", name))
>>                return 1;
>>        return 0;
>> }
>> @@ -414,6 +430,8 @@ int same_encoding(const char *src, const char *dst)
>> {
>>        if (is_encoding_utf8(src) && is_encoding_utf8(dst))
>>                return 1;
>> +       if (same_utf_encoding(src, dst))
>> +               return 1;
>>        return !strcasecmp(src, dst);
>> }
> 
> This seems odd. I would have expected the newly-added generalized
> conditional to replace the original UTF-8-specific conditional, not
> supplement it. That is, shouldn't the entire function body be:
> 
>    if (same_utf_encoding(src, dst))
>        return 1;
>    return !strcasecmp(src, dst);

No, because is_encoding_utf8() returns "true" (=1) if the encoding is NULL. That is not the case for UTF-16 et al. The caller of same_encoding() might expect that behavior.

I could have moved the "UTF-8" == NULL assumption into same_utf_encoding() but that did not feel right.

Does this make sense?
- Lars
Previous: Eric SunshineNext: Eric Sunshine
Message 9 of 22 in “convert: add support for different encodings”
  1. 00/10 convert: add support for different encodingslars.schneider@autodesk.com, Mar 15, 2018
  2. 01/10 strbuf: remove unnecessary NUL assignment in xstrdup_tolower()lars.schneider@autodesk.com, Mar 15, 2018
  3. 02/10 strbuf: add xstrdup_toupper()lars.schneider@autodesk.com, Mar 15, 2018
  4. 03/10 strbuf: add a case insensitive starts_with()lars.schneider@autodesk.com, Mar 15, 2018
  5. Morten WelinderMar 16, 2018
  6. Junio C HamanoMar 16, 2018
  7. 04/10 utf8: teach same_encoding() alternative UTF encoding nameslars.schneider@autodesk.com, Mar 15, 2018
  8. Eric SunshineMar 15, 2018
  9. Lars SchneiderMar 15, 2018
  10. Eric SunshineMar 15, 2018
  11. Junio C HamanoMar 16, 2018
  12. Eric SunshineMar 16, 2018
  13. Lars SchneiderApr 1, 2018
  14. 05/10 utf8: add function to detect prohibited UTF-16/32 BOMlars.schneider@autodesk.com, Mar 15, 2018
  15. 06/10 utf8: add function to detect a missing UTF-16/32 BOMlars.schneider@autodesk.com, Mar 15, 2018
  16. 07/10 convert: add 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 15, 2018
  17. 08/10 convert: check for detectable errors in UTF encodingslars.schneider@autodesk.com, Mar 15, 2018
  18. 10/10 convert: add round trip check based on 'core.checkRoundtripEncoding'lars.schneider@autodesk.com, Mar 15, 2018
  19. 09/10 convert: add tracing for 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 15, 2018
  20. Junio C HamanoMar 29, 2018
  21. Lars SchneiderApr 2, 2018
  22. Lars SchneiderApr 3, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.