git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] git-p4: import utf16 file properly

From
Luke Diamand <luke@diamand.org>
Date
Sep 14, 2011, 07:55 UTC
Message-ID
<4E705DF8.1040508@diamand.org>
In-Reply-To
<CANeU7QndA0yv1OzU3vta5B8r8nCRdBSqTy0Rboc_bbpst+1pcw@mail.gmail.com>
On 13/09/11 22:33, Chris Li wrote:
Show 11 quoted lines
> The current git-p4 does not handle utf16 files properly.
> The "p4 print" command, when output to stdout, converts the
> utf16 file into utf8. That effectively imported the utf16 file
> as utf8 for git. In other words, git-p4 import a different
> file compare to file check out by perforce. This breakes my
> windows build in the company project.
>
> The fix is simple, just ask perforce to print the depot
> file into a real file. This way perforce will not performe
> the utf16 to utf8 conversion. Git can import the exact same
> file as perforce checkout.
Does this change do the right thing with RCS keywords in UTF16 files?

If p4CmdList() fails, e.g. due to running out of diskspace, will this just happily import a truncated/corrupt file?

(And I could be wrong about this, but does you patch have newline damage? It didn't seem to apply for me).

Regards! Luke

Show 24 quoted lines
> ---
>   contrib/fast-import/git-p4 |    5 +++++
>   1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
> index 6b9de9e..5fb1ac7 100755
> --- a/contrib/fast-import/git-p4
> +++ b/contrib/fast-import/git-p4
> @@ -1239,6 +1239,11 @@ class P4Sync(Command, P4UserMap):
>               contents = map(lambda text:
> re.sub(r'(?i)\$(Id|Header):[^$]*\$',r'$\1$', text), contents)
>           elif file['type'] in ('text+k', 'ktext', 'kxtext',
> 'unicode+k', 'binary+k'):
>               contents = map(lambda text:
> re.sub(r'\$(Id|Header|Author|Date|DateTime|Change|File|Revision):[^$\n]*\$',r'$\1$',
> text), contents)
> +        elif file['type'] == 'utf16':
> +             tmpFile = tempfile.NamedTemporaryFile()
> +             p4CmdList("print -o %s %s"%(tmpFile.name, file['depotFile']))
> +             contents = [ open(tmpFile.name).read() ]
> +             tmpFile.close()
>
>           self.gitStream.write("M %s inline %s\n" % (mode, relPath))
>
Previous: Chris LiNext: Chris Li
Message 2 of 6 in “git-p4: import utf16 file properly”
  1. git-p4: import utf16 file properlyChris Li, Sep 13, 2011
  2. Luke DiamandSep 14, 2011
  3. Chris LiSep 14, 2011
  4. Chris LiSep 14, 2011
  5. Pete WyckoffSep 18, 2011
  6. Luke DiamandSep 14, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.