Re: [PATCH/RFC] Gitweb: Convert UTF-8 encoded file names

From: Jakub Narębski <jnareb@gmail.com>
Date: May 14, 2014, 22:25 UTC
Message-ID: <CANQwDwdh1qQkYi9sB=22wbNnb+g5qv5prCzj2aWhHBbTZhVhdg@mail.gmail.com>
In-Reply-To: <xmqqd2fghvlf.fsf@gitster.dls.corp.google.com>

On Wed, May 14, 2014 at 11:57 PM, Junio C Hamano <gitster@pobox.com> wrote:

Show 6 quoted lines

> Michael Wagner <accounts@mwagner.org> writes:
>
>> Perl has an internal encoding used to store text strings. Currently, trying to
>> view files with UTF-8 encoded names results in an error (either "404 - Cannot
>> find file" [blob_plain] or "XML Parsing Error" [blob]). Converting these UTF-8
>> encoded file names into Perl's internal format resolves these errors.

Could you give us an example? What is important is whether filename is passed via path_info or via query string.

Because in evaluate_uri() there is

     our $path_info = decode_utf8($ENV{"PATH_INFO"});

and in evaluate_query_params() there is

    $input_params{$name} = decode_utf8($cgi->param($symbol));

Show 9 quoted lines

>> Signed-off-by: Michael Wagner <accounts@mwagner.org>
>> ---
>
> Cc'ing Jakub, who have been the area maintainer, for comments.
>
> One thing I wonder is that, if there are some additional calls to
> encode() necessary before we embed $file_name (which are now decoded
> to the internal string form, not a byte-sequence that happens to be
> in utf-8) in the generated pages, if we were to do this change.

There should be no problem with output encoding. esc_path(), which should be used for filenames, includes to_utf8, which in turn uses decode($fallback_encoding, $str, Encode::FB_DEFAULT);

Show 16 quoted lines

>>  gitweb/gitweb.perl | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
>> index a9f57d6..6046977 100755
>> --- a/gitweb/gitweb.perl
>> +++ b/gitweb/gitweb.perl
>> @@ -1056,7 +1056,7 @@ sub evaluate_and_validate_params {
>>               }
>>       }
>>
>> -     our $file_name = $input_params{'file_name'};
>> +     our $file_name = decode("utf-8", $input_params{'file_name'});
>>       if (defined $file_name) {
>>               if (!is_valid_pathname($file_name)) {
>>                       die_error(400, "Invalid file parameter");

Hmm... all %input_params should have been properly decoded already, how it was missed?

Also, branchname (hash_base etc.), search query, filename in file_parent, project name can be UTF-8 too, so it is at best partial fix.

-- 
Jakub Narębski