git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v9 6/8] convert: check for detectable errors in UTF encodings

From
Lars Schneider <larsxschneider@gmail.com>
Date
Mar 6, 2018, 17:03 UTC
Message-ID
<A4240C40-A3DF-45DB-88A6-125FB2D3CE15@gmail.com>
In-Reply-To
<xmqqvaea6lpn.fsf@gitster-ct.c.googlers.com>
Show 41 quoted lines
> On 06 Mar 2018, at 02:23, Junio C Hamano <gitster@pobox.com> wrote:
> 
> Lars Schneider <larsxschneider@gmail.com> writes:
> 
>>> On 05 Mar 2018, at 22:50, Junio C Hamano <gitster@pobox.com> wrote:
>>> 
>>> lars.schneider@autodesk.com writes:
>>> 
>>>> +static int validate_encoding(const char *path, const char *enc,
>>>> +		      const char *data, size_t len, int die_on_error)
>>>> +{
>>>> +	if (!memcmp("UTF-", enc, 4)) {
>>> 
>>> Does the caller already know that enc is sufficiently long that
>>> using memcmp is safe?
>> 
>> No :-(
>> 
>> Would you be willing to squash that in?
>> 
>>    if (strlen(enc) > 4 && !memcmp("UTF-", enc, 4)) {
>> 
>> I deliberately used "> 4" as plain "UTF-" is not even valid.
> 
> I'd rather not.  The code does not have to even look at 6th and
> later bytes in the enc[] even if it wanted to reject "UTF-" followed
> by nothing, but use of strlen() forces it to look at everything.
> 
> Stepping back, shouldn't
> 
> 	if (starts_with(enc, "UTF-") 
> 
> be sufficient?  If you really care about the case where "UTF-" alone
> comes here, you could write
> 
> 	if (starts_with(enc, "UTF-") && enc[4])
> 
> but I do not think "&& enc[4]" is even needed.  The functions called
> from this block would not consider "UTF-" alone as something valid
> anyway, so with that "&& enf[4]" we would be piling more code only
> for invalid/rare case.

Agreed, "if (starts_with(enc, "UTF-"))" is sufficient. Can you squash that in? Thanks for pointing me to starts_with() as I forgot about this function!

- Lars
Previous: Junio C HamanoNext: Eric Sunshine
Message 23 of 26 in “convert: add support for different encodings”
  1. 0/8 convert: add support for different encodingslars.schneider@autodesk.com, Mar 4, 2018
  2. 1/8 strbuf: remove unnecessary NUL assignment in xstrdup_tolower()lars.schneider@autodesk.com, Mar 4, 2018
  3. 3/8 utf8: add function to detect prohibited UTF-16/32 BOMlars.schneider@autodesk.com, Mar 4, 2018
  4. 2/8 strbuf: add xstrdup_toupper()lars.schneider@autodesk.com, Mar 4, 2018
  5. 4/8 utf8: add function to detect a missing UTF-16/32 BOMlars.schneider@autodesk.com, Mar 4, 2018
  6. Junio C HamanoMar 6, 2018
  7. Lars SchneiderMar 6, 2018
  8. Junio C HamanoMar 6, 2018
  9. Lars SchneiderMar 6, 2018
  10. Junio C HamanoMar 6, 2018
  11. Lars SchneiderMar 6, 2018
  12. Junio C HamanoMar 6, 2018
  13. Torsten BögershausenMar 7, 2018
  14. 5/8 convert: add 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 4, 2018
  15. Eric SunshineMar 6, 2018
  16. Lars SchneiderMar 6, 2018
  17. Eric SunshineMar 6, 2018
  18. 8/8 convert: add round trip check based on 'core.checkRoundtripEncoding'lars.schneider@autodesk.com, Mar 4, 2018
  19. 6/8 convert: check for detectable errors in UTF encodingslars.schneider@autodesk.com, Mar 4, 2018
  20. Junio C HamanoMar 5, 2018
  21. Lars SchneiderMar 5, 2018
  22. Junio C HamanoMar 6, 2018
  23. Lars SchneiderMar 6, 2018
  24. Eric SunshineMar 6, 2018
  25. 7/8 convert: add tracing for 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 4, 2018
  26. Eric SunshineMar 6, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.