Re: [PATCH] git-p4: don't convert utf16 files.

From: Pete Wyckoff <pw@padd.com>
Date: Aug 21, 2011, 15:21 UTC
Message-ID: <20110821152156.GA30299@arf.padd.com>
In-Reply-To: <CANeU7QmGJTr4V-tWzuPnP2P=9eQbGDgQxv2phAVbGXjU5tqYPQ@mail.gmail.com>

christ.li@gmail.com wrote on Fri, 19 Aug 2011 15:50 -0700:

Show 8 quoted lines

> Some repository has some utf16 files git-p4 don't know
> how to convert. For those files, git-p4 just write the utf8
> files. That is wrong, because git get different file than
> perforce does, causing some windows resource file fail
> to compile.
> 
> Using the "p4 print -o tmpfile depotfile" can avoid this
> convertion (and possible failure) all together.

This isn't contrib/fast-import/git-p4. Searching around, I discovered a 2009 fork of git-p4 that is fairly active. CC-ing some of the names I found on github.

Here's one such repo:

    http://github.com/ermshiperete/git-p4

Git's git-p4 doesn't try to do anything special with utf-16. It does \r\n mangling, but not $Keyword$ removal, then just streams it to disk however p4 sends it. That's close to what you're trying to do here.

		-- Pete

Show 39 quoted lines

> Signed-off-by: Chris Li <git@chrisli.org>
> 
> ---
>  git-p4 |   11 +++++------
>  1 files changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/git-p4 b/git-p4
> index 672b0c2..0c6a5cc 100755
> --- a/git-p4
> +++ b/git-p4
> @@ -755,12 +755,11 @@ class P4FileReader:
>                      break
> 
>              if header['type'].startswith('utf16'):
> -                try:
> -                    text = textBuffer.getvalue().encode('utf_16')
> -                except UnicodeDecodeError:
> -                    # File checked in to Perforce has an error. Try
> without encoding
> -                    print "Corrupt UTF-16 file in Perforce: %s" %
> header['depotFile']
> -                    text = textBuffer.getvalue()
> +                # Don't even try to convert utf16. Ask p4 to write
> the file directly.
> +                tmpFile = tempfile.NamedTemporaryFile()
> +                P4Helper().p4_system("print -o %s %s"%(tmpFile.name,
> header['depotFile']))
> +                text = open(tmpFile.name).read()
> +      	        tmpFile.close()
>              else:
>                  text = textBuffer.getvalue()
>              textBuffer.close()
> -- 
> 1.7.6
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>