git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 4/4] git-p4: use utf-8 encoding for file paths throughout

From
Elijah Newren <newren@gmail.com>
Date
Nov 28, 2019, 02:57 UTC
Message-ID
<CABPp-BEa883U+m1emhZHU4s5EqsJHyRiPvjnsyw6_0guKLUsLg@mail.gmail.com>
In-Reply-To
<20191128012807.3103-5-yang.zhao@skyboxlabs.com>
On Wed, Nov 27, 2019 at 5:32 PM Yang Zhao <yang.zhao@skyboxlabs.com> wrote:
Show 11 quoted lines
>
> Try to decode file paths in responses from p4 as soon as possible so
> that we are working with unicode string throughout the rest of the flow.
> This makes python 3 a lot happier.
>
> Signed-off-by: Yang Zhao <yang.zhao@skyboxlabs.com>
> ---
>
> This is probably the most risky patch out of the set. It's very likely
> that I've neglected to consider certain corner cases with decoding of
> path data.

Yes, this does seem somewhat risky to me. It may go well on platforms that require all filenames to be unicode. And it may work for users who happen to restrict their filenames to valid utf-8. But this abstraction doesn't fit the general problem, so some users may be left out in the cold.

I tried multiple times while switching git-filter-repo from python2 to python3, at different levels of pervasiveness, to use unicode more generally. But I mostly gave up; everyone knows files won't necessarily be unicode, but you just can't assume filenames or commit messages or branch or tag names (and perhaps a few other things I'm forgetting) are either. I ended up using bytestrings everywhere except messages displayed to the user, and I only decode at that point.

Of course, if perforce happens to only work with unicode filenames then you'll be fine. And perhaps you don't want or need to be as paranoid as I was about what people could do. So I don't know if my experience applies in your case (I've never used perforce myself), but I just thought I'd mention it in case it's useful.

Previous: Yang ZhaoNext: Johannes Schindelin
Message 6 of 7 in “git-p4: python 3 compatability”
  1. 0/4 git-p4: python 3 compatabilityYang Zhao, Nov 28, 2019
  2. 2/4 git-p4: properly encode/decode communication with git for python 3Yang Zhao, Nov 28, 2019
  3. 1/4 git-p4: decode response from p4 to str for python3Yang Zhao, Nov 28, 2019
  4. 3/4 git-p4: open .gitp4-usercache.txt in text modeYang Zhao, Nov 28, 2019
  5. 4/4 git-p4: use utf-8 encoding for file paths throughoutYang Zhao, Nov 28, 2019
  6. Elijah NewrenNov 28, 2019
  7. Johannes SchindelinNov 28, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.