git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 11/15] git-p4: add Py23File() - helper class for stream writing

From
Ben Keene via GitGitGadget <gitgitgadget@gmail.com>
Date
Dec 7, 2019, 17:47 UTC
Message-ID
<d22ada16143ac67551c089ff44ab1832b791dd46.1575740863.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.463.v5.git.1575740863.gitgitgadget@gmail.com>
From: Ben Keene <seraphire@gmail.com>

This is a preparatory commit that does not change current behavior. It adds a new class Py23File.

Following the Python recommendation of keeping text as unicode internally and only converting to and from bytes on input and output, this class provides an interface for the methods used for reading and writing files and file like streams.

A new class was implemented to avoid requiring additional dependencies.

Create a class that wraps the input and output functions used by the git-p4.py code for reading and writing to standard file handles.

The methods of this class should take a Unicode string for writing and return unicode strings in reads. This class should be a drop-in for existing file like streams

The following methods should be coded for supporting existing read/write
calls:
  * write - this should write a Unicode string to the underlying stream
  * read  - this should read from the underlying stream and cast the
            bytes as a unicode string
  * readline - this should read one line of text from the underlying
            stream and cast it as a unicode string
  * readline - this should read a number of lines, optionally hinted,
            and cast each line as a unicode string

The expression "cast as a unicode string" is used because the code should use the as_bytes() and as_string() functions instead of cohercing the data to actual unicode strings or bytes. This allows Python 2 code to continue to use the internal "str" data type instead of converting the data back and forth to actual unicode strings. This retains current Python 2 support while Python 3 support may be incomplete.

Signed-off-by: Ben Keene <seraphire@gmail.com>
---
 git-p4.py | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)
diff --git a/git-p4.py b/git-p4.py
index 1838045078..03829f796d 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -4187,6 +4187,72 @@ def run(self, args):
             print("%s <= %s (%s)" % (branch, ",".join(settings["depot-paths"]), settings["change"]))
         return True
 
+class Py23File():
+    """ Python2/3 Unicode File Wrapper
+    """
+
+    stream_handle = None
+    verbose       = False
+    debug_handle  = None
+
+    def __init__(self, stream_handle, verbose = False):
+        """ Create a Python3 compliant Unicode to Byte String
+            Windows compatible wrapper
+
+            stream_handle = the underlying file-like handle
+            verbose       = Boolean if content should be echoed
+        """
+        self.stream_handle = stream_handle
+        self.verbose       = verbose
+
+    def write(self, utf8string):
+        """ Writes the utf8 encoded string to the underlying
+            file stream
+        """
+        self.stream_handle.write(as_bytes(utf8string))
+        if self.verbose:
+            sys.stderr.write("Stream Output: %s" % utf8string)
+            sys.stderr.flush()
+
+    def read(self, size = None):
+        """ Reads int charcters from the underlying stream
+            and converts it to utf8.
+
+            Be aware, the size value is for reading the underlying
+            bytes so the value may be incorrect. Usage of the size
+            value is discouraged.
+        """
+        if size == None:
+            return as_string(self.stream_handle.read())
+        else:
+            return as_string(self.stream_handle.read(size))
+
+    def readline(self):
+        """ Reads a line from the underlying byte stream
+            and converts it to utf8
+        """
+        return as_string(self.stream_handle.readline())
+
+    def readlines(self, sizeHint = None):
+        """ Returns a list containing lines from the file converted to unicode.
+
+            sizehint - Optional. If the optional sizehint argument is
+            present, instead of reading up to EOF, whole lines totalling
+            approximately sizehint bytes are read.
+        """
+        lines = self.stream_handle.readlines(sizeHint)
+        for i in range(0, len(lines)):
+            lines[i] = as_string(lines[i])
+        return lines
+
+    def close(self):
+        """ Closes the underlying byte stream """
+        self.stream_handle.close()
+
+    def flush(self):
+        """ Flushes the underlying byte stream """
+        self.stream_handle.flush()
+
 class HelpFormatter(optparse.IndentedHelpFormatter):
     def __init__(self):
         optparse.IndentedHelpFormatter.__init__(self)
-- 
gitgitgadget
Previous: Ben Keene via GitGitGadgetNext: Ben Keene via GitGitGadget
Message 58 of 77 in “git-p4.py: Cast byte strings to unicode strings in python3”
  1. 0/1 git-p4.py: Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Nov 13, 2019
  2. 1/1 Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Nov 13, 2019
  3. Junio C HamanoNov 14, 2019
  4. Luke DiamandNov 14, 2019
  5. 0/3 git-p4.py: Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Nov 15, 2019
  6. 1/3 Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Nov 15, 2019
  7. 2/3 FIX: cast as unicode fails when a value is already unicodeBen Keene via GitGitGadget, Nov 15, 2019
  8. 3/3 FIX: wrap return for read_pipe_lines in ustring() and wrap GitLFS read of the pointer file in ustring()Ben Keene via GitGitGadget, Nov 15, 2019
  9. 0/1 git-p4.py: Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Dec 2, 2019
  10. 1/1 Python3 support for t9800 tests. Basic P4/Python3 supportBen Keene via GitGitGadget, Dec 2, 2019
  11. Denton LiuDec 3, 2019
  12. Ben KeeneDec 3, 2019
  13. Denton LiuDec 4, 2019
  14. 00/11 git-p4.py: Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Dec 4, 2019
  15. 01/11 git-p4: select p4 binary by operating-systemBen Keene via GitGitGadget, Dec 4, 2019
  16. Denton LiuDec 5, 2019
  17. Ben KeeneDec 5, 2019
  18. 02/11 git-p4: change the expansion test from basestring to listBen Keene via GitGitGadget, Dec 4, 2019
  19. Denton LiuDec 5, 2019
  20. Ben KeeneDec 5, 2019
  21. 03/11 git-p4: add new helper functions for python3 conversionBen Keene via GitGitGadget, Dec 4, 2019
  22. Denton LiuDec 5, 2019
  23. Ben KeeneDec 5, 2019
  24. 05/11 git-p4: Add new functions in preparation of usageBen Keene via GitGitGadget, Dec 4, 2019
  25. Denton LiuDec 5, 2019
  26. Ben KeeneDec 5, 2019
  27. 04/11 git-p4: python3 syntax changesBen Keene via GitGitGadget, Dec 4, 2019
  28. Denton LiuDec 5, 2019
  29. 08/11 git-p4: p4CmdList - support Unicode encodingBen Keene via GitGitGadget, Dec 4, 2019
  30. Junio C HamanoDec 5, 2019
  31. Ben KeeneDec 5, 2019
  32. 07/11 git-p4: Add a helper class for stream writingBen Keene via GitGitGadget, Dec 4, 2019
  33. Junio C HamanoDec 5, 2019
  34. Ben KeeneDec 5, 2019
  35. 06/11 git-p4: Fix assumed path separators to be more Windows friendlyBen Keene via GitGitGadget, Dec 4, 2019
  36. Junio C HamanoDec 5, 2019
  37. Ben KeeneDec 5, 2019
  38. 09/11 git-p4: Add usability enhancementsBen Keene via GitGitGadget, Dec 4, 2019
  39. Junio C HamanoDec 5, 2019
  40. Ben KeeneDec 5, 2019
  41. 10/11 git-p4: Support python3 for basic P4 clone, sync, and submitBen Keene via GitGitGadget, Dec 4, 2019
  42. 11/11 git-p4: Added --encoding parameter to p4 cloneBen Keene via GitGitGadget, Dec 4, 2019
  43. Luke DiamandDec 5, 2019
  44. Ben KeeneDec 5, 2019
  45. Denton LiuDec 5, 2019
  46. Ben KeeneDec 5, 2019
  47. 00/15 git-p4.py: Cast byte strings to unicode strings in python3Ben Keene via GitGitGadget, Dec 7, 2019
  48. 01/15 t/gitweb-lib.sh: drop confusing quotesJeff King via GitGitGadget, Dec 7, 2019
  49. 02/15 t/gitweb-lib.sh: set $REQUEST_URIJeff King via GitGitGadget, Dec 7, 2019
  50. 03/15 git-p4: select P4 binary by operating-systemBen Keene via GitGitGadget, Dec 7, 2019
  51. Junio C HamanoDec 9, 2019
  52. 05/15 git-p4: promote encodeWithUTF8() to a global functionBen Keene via GitGitGadget, Dec 7, 2019
  53. Junio C HamanoDec 11, 2019
  54. 04/15 git-p4: change the expansion test from basestring to listBen Keene via GitGitGadget, Dec 7, 2019
  55. Junio C HamanoDec 9, 2019
  56. Ben KeeneDec 13, 2019
  57. 06/15 git-p4: remove p4_write_pipe() and write_pipe() return valuesBen Keene via GitGitGadget, Dec 7, 2019
  58. 11/15 git-p4: add Py23File() - helper class for stream writingBen Keene via GitGitGadget, Dec 7, 2019
  59. 08/15 git-p4: add casting helper functions for python 3 conversionBen Keene via GitGitGadget, Dec 7, 2019
  60. 09/15 git-p4: python 3 syntax changesBen Keene via GitGitGadget, Dec 7, 2019
  61. 10/15 git-p4: fix assumed path separators to be more Windows friendlyBen Keene via GitGitGadget, Dec 7, 2019
  62. 15/15 git-p4: Add depot manipulation functionsBen Keene via GitGitGadget, Dec 7, 2019
  63. 14/15 git-p4: added --encoding parameter to p4 cloneBen Keene via GitGitGadget, Dec 7, 2019
  64. 07/15 git-p4: add new support function gitConfigSet()Ben Keene via GitGitGadget, Dec 7, 2019
  65. Junio C HamanoDec 11, 2019
  66. 13/15 git-p4: support Python 3 for basic P4 clone, sync, and submit (t9800)Ben Keene via GitGitGadget, Dec 7, 2019
  67. 12/15 git-p4: p4CmdList - support Unicode encodingBen Keene via GitGitGadget, Dec 7, 2019
  68. Jeff KingDec 7, 2019
  69. Ben KeeneDec 7, 2019
  70. Junio C HamanoDec 11, 2019
  71. Denton LiuDec 11, 2019
  72. Junio C HamanoDec 11, 2019
  73. Luke DiamandDec 11, 2019
  74. Junio C HamanoDec 11, 2019
  75. Yang ZhaoDec 11, 2019
  76. Ben KeeneDec 12, 2019
  77. Ben KeeneDec 13, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.