threads / patch / 40292

patch, 2 partsgit-p4: handle "Translation of file content failed"

Subject: [PATCH v1 0/2] git-p4: handle "Translation of file content failed"

## tl;dr

5 messages between Sep 7, 2015 and Sep 9, 2015. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

larsxschneider@gmail.com· Sep 7, 2015, 12:42 UTC · lore

From: Lars Schneider <larsxschneider@gmail.com>

Hi,

this patch fixes the P4 "Translation of file content failed" error. Unfortuantly I was not able to generate a P4 test repository to reproduce the error with a test case.

An Internet search shows that this error happens in the wild: https://stackoverflow.com/questions/5156909/translation-of-file-content-failed-error-in-perforce https://stackoverflow.com/questions/887006/perforce-translation-of-file-content-failed-error

Thanks, Lars

Lars Schneider (2):
  git-p4: print stderr if P4 read_pipe operation fails
  git-p4: handle "Translation of file content failed"

 git-p4.py | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

-- 2.5.1

larsxschneider@gmail.com· Sep 7, 2015, 12:42 UTC · re: larsxschneider@gmail.com · lore

[PATCH v1 1/2] git-p4: print stderr if P4 read_pipe operation fails

From: Lars Schneider <larsxschneider@gmail.com>

If read_pipe crashes then the caller can inspect the error and handle it appropriately.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 git-p4.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Show changes to git-p4.py +2 −2

diff --git a/git-p4.py b/git-p4.py
index 073f87b..36a4bcb 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -134,11 +134,11 @@ def read_pipe(c, ignore_error=False):
         sys.stderr.write('Reading pipe: %s\n' % str(c))
 
     expand = isinstance(c,basestring)
-    p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand)
+    p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand)
     pipe = p.stdout
     val = pipe.read()
     if p.wait() and not ignore_error:
-        die('Command failed: %s' % str(c))
+        die('Command failed: %s\nError: %s' % (str(c), p.stderr.read()))
 
     return val

-- 
2.5.1

Junio C Hamano· Sep 9, 2015, 16:00 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v1 1/2] git-p4: print stderr if P4 read_pipe operation fails

larsxschneider@gmail.com writes:

Show 25 quoted lines

> From: Lars Schneider <larsxschneider@gmail.com>
>
> If read_pipe crashes then the caller can inspect the error and handle
> it appropriately.
>
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
>  git-p4.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/git-p4.py b/git-p4.py
> index 073f87b..36a4bcb 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -134,11 +134,11 @@ def read_pipe(c, ignore_error=False):
>          sys.stderr.write('Reading pipe: %s\n' % str(c))
>  
>      expand = isinstance(c,basestring)
> -    p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand)
> +    p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand)
>      pipe = p.stdout
>      val = pipe.read()
>      if p.wait() and not ignore_error:
> -        die('Command failed: %s' % str(c))
> +        die('Command failed: %s\nError: %s' % (str(c), p.stderr.read()))

I don't know enough about the callers of this helper function to tell offhand if that is an issue, but this looks unsafe depending on what the process on the other side of these pipes are doing.

If it attempts to spew a lot on its standard error stream first and then write some to its standard output, I would not be surprised it would get stuck waiting for us to read and drain its standard error before it can proceed to write to its standard output, and in the meantime we would be waiting for it to say something on its standard output, no?

Lars Schneider· Sep 9, 2015, 18:50 UTC · re: Junio C Hamano · lore

Re: [PATCH v1 1/2] git-p4: print stderr if P4 read_pipe operation fails

On 09 Sep 2015, at 18:00, Junio C Hamano <gitster@pobox.com> wrote:

Show 39 quoted lines

> larsxschneider@gmail.com writes:
> 
>> From: Lars Schneider <larsxschneider@gmail.com>
>> 
>> If read_pipe crashes then the caller can inspect the error and handle
>> it appropriately.
>> 
>> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
>> ---
>> git-p4.py | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>> 
>> diff --git a/git-p4.py b/git-p4.py
>> index 073f87b..36a4bcb 100755
>> --- a/git-p4.py
>> +++ b/git-p4.py
>> @@ -134,11 +134,11 @@ def read_pipe(c, ignore_error=False):
>>         sys.stderr.write('Reading pipe: %s\n' % str(c))
>> 
>>     expand = isinstance(c,basestring)
>> -    p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand)
>> +    p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand)
>>     pipe = p.stdout
>>     val = pipe.read()
>>     if p.wait() and not ignore_error:
>> -        die('Command failed: %s' % str(c))
>> +        die('Command failed: %s\nError: %s' % (str(c), p.stderr.read()))
> 
> I don't know enough about the callers of this helper function to
> tell offhand if that is an issue, but this looks unsafe depending on
> what the process on the other side of these pipes are doing.
> 
> If it attempts to spew a lot on its standard error stream first and
> then write some to its standard output, I would not be surprised it
> would get stuck waiting for us to read and drain its standard error
> before it can proceed to write to its standard output, and in the
> meantime we would be waiting for it to say something on its standard
> output, no?
>

You are right. I will use the “communicate” function here as recommended in the Python docs: https://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait

Thanks!

larsxschneider@gmail.com· Sep 7, 2015, 12:42 UTC · re: larsxschneider@gmail.com · lore

[PATCH v1 2/2] git-p4: handle "Translation of file content failed"

From: Lars Schneider <larsxschneider@gmail.com>

A P4 repository can get into a state where it contains a file with file type UTF16 that does not not contain valid UTF16 characters. If git-p4 attempts to retrieve the file as UTF16 from P4 then the process crashes with a "Translation of file content failed" error.

Fix this by detecting this error and retrieving the file as binary
instead. The result in Git is the same.
---
 git-p4.py | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

Show changes to git-p4.py +11 −4

diff --git a/git-p4.py b/git-p4.py
index 36a4bcb..aaa0ad9 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2186,10 +2186,17 @@ class P4Sync(Command, P4UserMap):
             # them back too.  This is not needed to the cygwin windows version,
             # just the native "NT" type.
             #
-            text = p4_read_pipe(['print', '-q', '-o', '-', "%s@%s" % (file['depotFile'], file['change']) ])
-            if p4_version_string().find("/NT") >= 0:
-                text = text.replace("\r\n", "\n")
-            contents = [ text ]
+            try:
+                text = p4_read_pipe(['print', '-q', '-o', '-', '%s@%s' % (file['depotFile'], file['change'])])
+            except Exception as e:
+                if 'Translation of file content failed' in str(e):
+                    type_base = 'binary'
+                else:
+                    raise e
+            else:
+                if p4_version_string().find('/NT') >= 0:
+                    text = text.replace('\r\n', '\n')
+                contents = [ text ]
 
         if type_base == "apple":
             # Apple filetype files will be streamed as a concatenation of

-- 
2.5.1