Re: [PATCH] git-remote-testpy: fix patch hashing on Python 3

From: Junio C Hamano <gitster@pobox.com>
Date: Jan 27, 2013, 20:47 UTC
Message-ID: <7v622iuzea.fsf@alter.siamese.dyndns.org>
In-Reply-To: <7va9ruuzsf.fsf@alter.siamese.dyndns.org>

Junio C Hamano <gitster@pobox.com> writes:

Show 7 quoted lines

> John Keeping <john@keeping.me.uk> writes:
>
>> So I think the answer is "habit, but I probably shouldn't have put it
>> in in this case".
>
> OK, then I'll queue with a local amend to drop the leading
> underscore.

So this is what I will be queuing (I'd appreciate the second set of eyes, though), with the leading-underscore removal and log message typofixes.

I remember that I earlier asked somewhere if we want to say "Python 3.x that is older than 3.y is unsupported"

    http://thread.gmane.org/gmane.comp.version-control.git/213920/focus=213926

but I was told that we will support all versions in 3.x series, IIRC.

Does this patch contradict with that? If so I think we would need to revisit the update to CodingGuidelines in that thread.

I am perfectly fine with discarding early 3.x as "0.x releases of Python3", but I would want to see our document say so if that is what we do.

-- >8 --
From: John Keeping <john@keeping.me.uk>
Date: Sun, 27 Jan 2013 14:50:56 +0000
Subject: [PATCH] git-remote-testpy: fix path hashing on Python 3

When this change was originally made (0846b0c - git-remote-testpy: hash bytes explicitly , I didn't realise that the "hex" encoding we chose is a "bytes to bytes" encoding so it just fails with an error on Python 3 in the same way as the original code.

It is not possible to provide a single code path that works on Python 2 and Python 3 since Python 2.x will attempt to decode the string before encoding it, which fails for strings that are not valid in the default encoding. Python 3.1 introduced the "surrogateescape" error handler which handles this correctly and permits a bytes -> unicode -> bytes round-trip to be lossless.

At this point Python 3.0 is unsupported so we don't go out of our way to try to support it.

Helped-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 git-remote-testpy.py | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/git-remote-testpy.py b/git-remote-testpy.py
index c7a04ec..6098bdd 100644
--- a/git-remote-testpy.py
+++ b/git-remote-testpy.py
@@ -36,6 +36,22 @@ if sys.hexversion < 0x02000000:
     sys.stderr.write("git-remote-testgit: requires Python 2.0 or later.\n")
     sys.exit(1)
 
+
+def encode_filepath(path):
+    """Encodes a Unicode file path to a byte string.
+
+    On Python 2 this is a no-op; on Python 3 we encode the string as
+    suggested by [1] which allows an exact round-trip from the command line
+    to the filesystem.
+
+    [1] http://docs.python.org/3/c-api/unicode.html#file-system-encoding
+
+    """
+    if sys.hexversion < 0x03000000:
+        return path
+    return path.encode('utf-8', 'surrogateescape')
+
+
 def get_repo(alias, url):
     """Returns a git repository object initialized for usage.
     """
@@ -45,7 +61,7 @@ def get_repo(alias, url):
     repo.get_head()
 
     hasher = _digest()
-    hasher.update(repo.path.encode('hex'))
+    hasher.update(encode_filepath(repo.path))
     repo.hash = hasher.hexdigest()
 
     repo.get_base_path = lambda base: os.path.join(

-- 
1.8.1.1.550.g40037fd