Re: [PATCH/RFC] Gitweb: Convert UTF-8 encoded file names
- From
- Michael Wagner <accounts@mwagner.org>
- Date
- May 15, 2014, 05:08 UTC
- Message-ID
- <20140515050820.GA30785@localhost.localdomain>
- In-Reply-To
- <CANQwDwdh1qQkYi9sB=22wbNnb+g5qv5prCzj2aWhHBbTZhVhdg@mail.gmail.com>
On Thu, May 15, 2014 at 12:25:45AM +0200, Jakub Narębski wrote:
Show 11 quoted lines
> On Wed, May 14, 2014 at 11:57 PM, Junio C Hamano <gitster@pobox.com> wrote: > > Michael Wagner <accounts@mwagner.org> writes: > > > >> Perl has an internal encoding used to store text strings. Currently, trying to > >> view files with UTF-8 encoded names results in an error (either "404 - Cannot > >> find file" [blob_plain] or "XML Parsing Error" [blob]). Converting these UTF-8 > >> encoded file names into Perl's internal format resolves these errors. > > Could you give us an example? What is important is whether filename > is passed via path_info or via query string. >
There is a file named "Gütekriterien.txt" in my repository. Trying to view this file as "blob_plain" produces an 404 error (displaying the file name with an additional print statement):
$ REQUEST_METHOD=GET QUERY_STRING='p=notes.git;a=blob_plain;f=work/G%C3%83%C2%BCtekriterien.txt;hb=HEAD' ./gitweb.cgi
work/Gütekriterien.txt Status: 404 Not Found
Decoding the UTF-8 encoded file name (again with an additional print statement):
$ REQUEST_METHOD=GET QUERY_STRING='p=notes.git;a=blob_plain;f=work/G%C3%83%C2%BCtekriterien.txt;hb=HEAD' ./gitweb.cgi
work/Gütekriterien.txt Content-disposition: inline; filename="work/Gütekriterien.txt"
Show 17 quoted lines
> Because in evaluate_uri() there is
>
> our $path_info = decode_utf8($ENV{"PATH_INFO"});
>
> and in evaluate_query_params() there is
>
> $input_params{$name} = decode_utf8($cgi->param($symbol));
>
> >> Signed-off-by: Michael Wagner <accounts@mwagner.org>
> >> ---
> >
> > Cc'ing Jakub, who have been the area maintainer, for comments.
> >
> > One thing I wonder is that, if there are some additional calls to
> > encode() necessary before we embed $file_name (which are now decoded
> > to the internal string form, not a byte-sequence that happens to be
> > in utf-8) in the generated pages, if we were to do this change.The generated pages show the correct file names.
Show 30 quoted lines
>
> There should be no problem with output encoding. esc_path(), which
> should be used for filenames, includes to_utf8, which in turn uses
> decode($fallback_encoding, $str, Encode::FB_DEFAULT);
>
> >> gitweb/gitweb.perl | 2 +-
> >> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> >> index a9f57d6..6046977 100755
> >> --- a/gitweb/gitweb.perl
> >> +++ b/gitweb/gitweb.perl
> >> @@ -1056,7 +1056,7 @@ sub evaluate_and_validate_params {
> >> }
> >> }
> >>
> >> - our $file_name = $input_params{'file_name'};
> >> + our $file_name = decode("utf-8", $input_params{'file_name'});
> >> if (defined $file_name) {
> >> if (!is_valid_pathname($file_name)) {
> >> die_error(400, "Invalid file parameter");
>
> Hmm... all %input_params should have been properly decoded
> already, how it was missed?
>
> Also, branchname (hash_base etc.), search query, filename in file_parent,
> project name can be UTF-8 too, so it is at best partial fix.
>
> --
> Jakub Narębski