git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/2] commit: reject invalid UTF-8 codepoints

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 8, 2013, 19:36 UTC
Message-ID
<7vfvvozvx4.fsf@alter.siamese.dyndns.org>
In-Reply-To
<alpine.DEB.2.00.1307051345260.11814@ds9.cixit.se>
Peter Krefting <peter@softwolves.pp.se> writes:
Show 16 quoted lines
> brian m. carlson:
>
>> +		/* U+FFFE and U+FFFF are guaranteed non-characters. */
>> +		if ((codepoint & 0x1ffffe) == 0xfffe)
>> +			return bad_offset;
>
> I missed this the first time around: All Unicode characters whose
> lower 16-bits are FFFE or FFFF are non-characters, so you can re-write
> that to:
>
>   /* U+xxFFFE and U+xxFFFF are guaranteed non-characters. */
>   if ((codepoint & 0xfffe) == 0xfffe)
>    return bad_offset;
>
> Also, the range U+FDD0--U+FDEF are also non-characters, if you wish to
> be really pedantic.

Yeah, while we are at it, doing this may not hurt. I think Brian's two patches are in fairly good shape otherwise, so perhaps you can do this as a follow-up patch on top of the tip of the topic, e82bd6cc (commit: reject overlong UTF-8 sequences, 2013-07-04)?

Previous: Peter KreftingNext: Peter Krefting
Message 6 of 11 in “commit: improve UTF-8 validation”
  1. 0/2 commit: improve UTF-8 validationbrian m. carlson, Jul 4, 2013
  2. 1/2 commit: reject invalid UTF-8 codepointsbrian m. carlson, Jul 4, 2013
  3. Torsten BögershausenJul 4, 2013
  4. brian m. carlsonJul 4, 2013
  5. Peter KreftingJul 5, 2013
  6. Junio C HamanoJul 8, 2013
  7. commit: reject non-charactersPeter Krefting, Jul 9, 2013
  8. Peter KreftingAug 5, 2013
  9. Junio C HamanoAug 5, 2013
  10. Peter KreftingAug 6, 2013
  11. 2/2 commit: reject overlong UTF-8 sequencesbrian m. carlson, Jul 4, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.