From: Luke Diamand <luke@diamand.org>
Date: Wed, 14 Sep 2011 07:55:36 GMT
Subject: Re: [PATCH] git-p4: import utf16 file properly
Message-ID: <4E705DF8.1040508@diamand.org>
In-Reply-To: <CANeU7QndA0yv1OzU3vta5B8r8nCRdBSqTy0Rboc_bbpst+1pcw@mail.gmail.com>

On 13/09/11 22:33, Chris Li wrote:
> The current git-p4 does not handle utf16 files properly.
> The "p4 print" command, when output to stdout, converts the
> utf16 file into utf8. That effectively imported the utf16 file
> as utf8 for git. In other words, git-p4 import a different
> file compare to file check out by perforce. This breakes my
> windows build in the company project.
>
> The fix is simple, just ask perforce to print the depot
> file into a real file. This way perforce will not performe
> the utf16 to utf8 conversion. Git can import the exact same
> file as perforce checkout.

Does this change do the right thing with RCS keywords in UTF16 files?

If p4CmdList() fails, e.g. due to running out of diskspace, will this 
just happily import a truncated/corrupt file?

(And I could be wrong about this, but does you patch have newline 
damage? It didn't seem to apply for me).

Regards!
Luke

> ---
>   contrib/fast-import/git-p4 |    5 +++++
>   1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
> index 6b9de9e..5fb1ac7 100755
> --- a/contrib/fast-import/git-p4
> +++ b/contrib/fast-import/git-p4
> @@ -1239,6 +1239,11 @@ class P4Sync(Command, P4UserMap):
>               contents = map(lambda text:
> re.sub(r'(?i)\$(Id|Header):[^$]*\$',r'$\1$', text), contents)
>           elif file['type'] in ('text+k', 'ktext', 'kxtext',
> 'unicode+k', 'binary+k'):
>               contents = map(lambda text:
> re.sub(r'\$(Id|Header|Author|Date|DateTime|Change|File|Revision):[^$\n]*\$',r'$\1$',
> text), contents)
> +        elif file['type'] == 'utf16':
> +             tmpFile = tempfile.NamedTemporaryFile()
> +             p4CmdList("print -o %s %s"%(tmpFile.name, file['depotFile']))
> +             contents = [ open(tmpFile.name).read() ]
> +             tmpFile.close()
>
>           self.gitStream.write("M %s inline %s\n" % (mode, relPath))
>