{"thread":{"id":"57477","subject":"Git-p4 UnicodeDecodeError on ascii 0x93 character","startedAt":"2022-02-26T05:22:07Z","lastAt":"2022-02-26T09:39:36Z","messageCount":2,"participants":["Hunter Laux","Torsten B��gershausen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"449652","messageId":"CAMQF8MhTmVs_LE8Te=9EL66Py8OtAMo7APf-cC=Ot6tykvaBpg@mail.gmail.com","threadId":"57477","inReplyTo":null,"subject":"Git-p4 UnicodeDecodeError on ascii 0x93 character","fromName":"Hunter Laux","fromEmail":"hunterlaux@gmail.com","sentAt":"2022-02-26T05:20:51Z","receivedAt":"2022-02-26T05:22:07Z","isPatch":false,"sender":{"key":"hunterlaux@gmail.com","avatar":null},"body":"One of my p4 commits has an “(0x93) character in the p4 description\nand I'm trying to export it to git.\nI'm using the latest git-p4.py as of c70bc33.\n\nI think python2 didn't throw an error, but python3 now treats this as an error.\nThe error is.\nTraceback (most recent call last):\n  File \"/home/hlaux/.local/bin/git-p4\", line 4292, in <module>\n    main()\n  File \"/home/hlaux/.local/bin/git-p4\", line 4286, in main\n    if not cmd.run(args):\n  File \"/home/hlaux/.local/bin/git-p4\", line 4043, in run\n    if not P4Sync.run(self, depotPaths):\n  File \"/home/hlaux/.local/bin/git-p4\", line 3910, in run\n    self.importRevisions(args, branch_arg_given)\n  File \"/home/hlaux/.local/bin/git-p4\", line 3658, in importRevisions\n    changes = p4ChangesForPaths(self.depotPaths, self.changeRange,\nself.changes_block_size)\n  File \"/home/hlaux/.local/bin/git-p4\", line 1080, in p4ChangesForPaths\n    result = p4CmdList(cmd, errors_as_exceptions=True)\n  File \"/home/hlaux/.local/bin/git-p4\", line 711, in p4CmdList\n    value = value.decode()\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position\n21: invalid start byte\n\nThis patch seems to work.\n\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -707,7 +707,7 @@ def p4CmdList(cmd, stdin=None, stdin_mode='w+b',\ncb=None, skip_info=False,\n                 for key, value in entry.items():\n                     key = key.decode()\n                     if isinstance(value, bytes) and not (key in\n('data', 'path', 'clientFile') or key.startswith('depotFile')):\n-                        value = value.decode()\n+                        value = value.decode('latin1')\n                     decoded_entry[key] = value\n                 # Parse out data if it's an error response\n                 if decoded_entry.get('code') == 'error' and 'data' in\ndecoded_entry:\n\n\nDoes anyone have a better fix?\n\nThanks,\n\nHunter Laux\n"},{"id":"449668","messageId":"20220226093929.uujvrauwatssl2mw@tb-raspi4","threadId":"57477","inReplyTo":"CAMQF8MhTmVs_LE8Te=9EL66Py8OtAMo7APf-cC=Ot6tykvaBpg@mail.gmail.com","subject":"Re: Git-p4 UnicodeDecodeError on ascii 0x93 character","fromName":"Torsten B��gershausen","fromEmail":"tboegi@web.de","sentAt":"2022-02-26T09:39:29Z","receivedAt":"2022-02-26T09:39:36Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Fri, Feb 25, 2022 at 09:20:51PM -0800, Hunter Laux wrote:\n> One of my p4 commits has an ???(0x93) character in the p4 description\n> and I'm trying to export it to git.\n> I'm using the latest git-p4.py as of c70bc33.\n>\n> I think python2 didn't throw an error, but python3 now treats this as an error.\n> The error is.\n> Traceback (most recent call last):\n>   File \"/home/hlaux/.local/bin/git-p4\", line 4292, in <module>\n>     main()\n>   File \"/home/hlaux/.local/bin/git-p4\", line 4286, in main\n>     if not cmd.run(args):\n>   File \"/home/hlaux/.local/bin/git-p4\", line 4043, in run\n>     if not P4Sync.run(self, depotPaths):\n>   File \"/home/hlaux/.local/bin/git-p4\", line 3910, in run\n>     self.importRevisions(args, branch_arg_given)\n>   File \"/home/hlaux/.local/bin/git-p4\", line 3658, in importRevisions\n>     changes = p4ChangesForPaths(self.depotPaths, self.changeRange,\n> self.changes_block_size)\n>   File \"/home/hlaux/.local/bin/git-p4\", line 1080, in p4ChangesForPaths\n>     result = p4CmdList(cmd, errors_as_exceptions=True)\n>   File \"/home/hlaux/.local/bin/git-p4\", line 711, in p4CmdList\n>     value = value.decode()\n> UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position\n> 21: invalid start byte\n>\n> This patch seems to work.\n>\n> --- a/git-p4.py\n> +++ b/git-p4.py\n> @@ -707,7 +707,7 @@ def p4CmdList(cmd, stdin=None, stdin_mode='w+b',\n> cb=None, skip_info=False,\n>                  for key, value in entry.items():\n>                      key = key.decode()\n>                      if isinstance(value, bytes) and not (key in\n> ('data', 'path', 'clientFile') or key.startswith('depotFile')):\n> -                        value = value.decode()\n> +                        value = value.decode('latin1')\n>                      decoded_entry[key] = value\n>                  # Parse out data if it's an error response\n>                  if decoded_entry.get('code') == 'error' and 'data' in\n> decoded_entry:\n>\n>\n> Does anyone have a better fix?\n>\n> Thanks,\n>\n> Hunter Laux\n\n\nThanks for reporting.\n\nI think there was a patch for this problem some time ago:\nhttps://lore.kernel.org/git/20210422050504.441-1-tzadik.vanderhoof@gmail.com/\n\n"}]}