Re: [PATCH/RFC] Gitweb: Convert UTF-8 encoded file names

From: Jakub Narębski <jnareb@gmail.com>
Date: May 15, 2014, 19:37 UTC
Message-ID: <CANQwDweFGwt5w2bJ1CesC50AAiJXBDT8uxuBEpFZ+nLbQaA_VQ@mail.gmail.com>
In-Reply-To: <CANQwDwe+GJ+yAYWdVfMaHq97zGXBoepCfUdLiaQD9LFoz3SiOA@mail.gmail.com>

On Thu, May 15, 2014 at 9:28 PM, Jakub Narębski <jnareb@gmail.com> wrote:

> On Thu, May 15, 2014 at 8:48 PM, Michael Wagner <accounts@mwagner.org> wrote:

[...]

Show 10 quoted lines

>> The subroutine "git tree" generates the tree view. It stores the output
>> of "git ls-tree -z ..." in an array named "@entries". Printing the content
>> of this array yields the following result:
>>
>> 00644 blob 6419cd06a9461c38d4f94d9705d97eaaa887156a     520 GÃ¼tekriterien.txt
>>
>> This leads to the "doubled" encoding. Declaring the encoding in the call
>> to open yields the following result:
>>
>> 100644 blob 6419cd06a9461c38d4f94d9705d97eaaa887156a     520 Gütekriterien.txt

Show 20 quoted lines

>> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
>> index a9f57d6..f1414e1 100755
>> --- a/gitweb/gitweb.perl
>> +++ b/gitweb/gitweb.perl
>> @@ -7138,7 +7138,7 @@ sub git_tree {
>>         my @entries = ();
>>         {
>>                 local $/ = "\0";
>> -               open my $fd, "-|", git_cmd(), "ls-tree", '-z',
>> +               open my $fd, "-|encoding(UTF-8)", git_cmd(), "ls-tree", '-z',
>>                         ($show_sizes ? '-l' : ()), @extra_options, $hash
>>                         or die_error(500, "Open git-ls-tree failed");
>
> Or put
>
>                    binmode $fd, ':utf8';
>
> like in the rest of the code.
>
>>                 @entries = map { chomp; $_ } <$fd>;

Though to be exact there isn't any mechanism that ensures that filenames in tree objects use utf-8 encoding, so perhaps a safer solution would be to use

   to_utf8($file_name)

(which respects $fallback_encoding) in appropriate places.

-- 
Jakub Narębski