git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/2] git-p4: do not decode data from perforce by default

From
TVTzadik Vanderhoof <tzadik.vanderhoof@gmail.com>
Date
Apr 29, 2021, 10:00 UTC
Message-ID
<CAKu1iLXRrsB4mRsDfhBH5aahWzDjpfqLuWP9t47RMB=RdpL1iA@mail.gmail.com>
In-Reply-To
<20210412085251.51475-3-andrew@adoakley.name>

I checked out "seen" and ran the test script from this patch (t9835-git-p4-message-encoding.sh) on my Windows machine and it fails.

I don't think the solution in this patch will solve the issue of non UTF-8 descriptions on Windows. The interaction between git-p4.py and p4 around non-ASCII descriptions is different on Linux and Windows (at least with the default code page settings). Unfortunately the CI on gitlab does not include any Windows test environments that have p4 installed.

As far as I can tell, non-ASCII strings passed to "p4 submit -d" pass unchanged to the Perforce database on Linux. As well, such data also passes unchanged in the other direction, when "p4" output is consumed by git-p4.py. Since this patch avoids decoding descriptions, and the test script uses binary data for descriptions, the tests pass on Linux.

However, on Windows, UTF-8 strings passed to "p4 submit -d" are somehow converted to the default Windows code page by the time they are stored in the Perforce database, probably as part of the process of passing the command line arguments to the Windows p4 executable. However, the "code page" data is *not* converted to UTF-8 on the way back from p4 to git-p4.py. The only way to get it into UTF-8 is to call string.decode(). As a result, this patch, which takes out the call to string.decode() will not work on Windows.

Previous: Andrew OakleyNext: Andrew Oakley
Message 3 of 13 in “git-p4: encoding of data from perforce”
  1. 0/2 git-p4: encoding of data from perforceAndrew Oakley, Apr 12, 2021
  2. 2/2 git-p4: do not decode data from perforce by defaultAndrew Oakley, Apr 12, 2021
  3. Tzadik VanderhoofApr 29, 2021
  4. Andrew OakleyApr 30, 2021
  5. Luke DiamandApr 30, 2021
  6. Tzadik VanderhoofApr 30, 2021
  7. Andrew OakleyMay 4, 2021
  8. Tzadik VanderhoofMay 4, 2021
  9. Junio C HamanoMay 5, 2021
  10. Tzadik VanderhoofMay 5, 2021
  11. Tzadik VanderhoofMay 5, 2021
  12. Junio C HamanoMay 5, 2021
  13. 1/2 git-p4: avoid decoding more data from perforceAndrew Oakley, Apr 12, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.