git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] gitweb: Fix chop_str not to cut in middle of utf8 multibyte chars.

From
Jakub Narebski <jnareb@gmail.com>
Date
May 24, 2008, 13:34 UTC
Message-ID
<200805241534.25517.jnareb@gmail.com>
In-Reply-To
<4833D314.4010904@0x63.nu>
On Wed, 21 May 2008, Anders Waldenborg wrote:
> Junio C Hamano wrote:
Show 11 quoted lines
>> I haven't followed the codepath but what do the callers do to the string
>> returned from chop_str?  Don't they assume the string hasn't been decoded
>> (because the old implementation of chop_str did not do this to_utf8), and
>> emit the result directly to the output because it also assumes the
>> undecoded format is what the outside world wants?  In other words, don't
>> they now need to do different things because returned string has gone
>> through the to_utf8() processing already?
> 
> The to_utf8() (defined in gitweb.perl, not part of perl it self) is kind 
> of sneaky, it checks if the string already is valid utf8. (guess it 
> should be called ensure_utf8())
Perhaps it should...
> chop_str needs to work on decoded string, otherwise character count goes 
> all wrong. But maybe it is better to add the to_utf8() to the callsites?
Or do "binmode $fd, :utf8".

But yes, I guess converting to Perl internal form on input would be good idea. Gitweb currently does it partially...

-- 
Jakub Narebski
Poland
Previous: Anders WaldenborgNext: Anders Waldenborg
Message 5 of 6 in “gitweb: Fix chop_str not to cut in middle of utf8 multibyte chars.”
  1. gitweb: Fix chop_str not to cut in middle of utf8 multibyte chars.Anders Waldenborg, May 20, 2008
  2. Jakub NarebskiMay 20, 2008
  3. Junio C HamanoMay 21, 2008
  4. Anders WaldenborgMay 21, 2008
  5. Jakub NarebskiMay 24, 2008
  6. gitweb: Convert string to internal form before chopping in chop_strAnders Waldenborg, May 21, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.