{"thread":{"id":"40251","subject":"[PATCH v5] git-p4: add config git-p4.pathEncoding","startedAt":"2015-09-02T20:39:14Z","lastAt":"2015-09-04T08:30:13Z","messageCount":6,"participants":["larsxschneider@gmail.com","Luke Diamand","Remi Galan Alfonso","Lars Schneider"],"isPatch":true,"patchVersion":5,"patchTotal":null},"messages":[{"id":"269234","messageId":"1441226355-61103-1-git-send-email-larsxschneider@gmail.com","threadId":"40251","inReplyTo":null,"subject":"[PATCH v5] git-p4: add config git-p4.pathEncoding","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-02T20:39:14Z","receivedAt":"2015-09-02T20:39:14Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nDiff to v4:\n* add verbose output\n* reword help text\n* remove comment how to generate encoded sequence\n* use ascii characters in encoded sequence where possible\n* wrap 'cd' in TC in a subshell\n\nThanks to Eric, Torsten, and Junio for feedback!\n\nCheers,\nLars\n\nLars Schneider (1):\n git-p4: add config git-p4.pathEncoding\n\n Documentation/git-p4.txt | 7 +++++\n git-p4.py | 11 ++++++++\n t/t9822-git-p4-path-encoding.sh | 60 +++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 78 insertions(+)\n create mode 100755 t/t9822-git-p4-path-encoding.sh\n\n--\n1.9.5 (Apple Git-50.3)\n"},{"id":"269235","messageId":"1441226355-61103-2-git-send-email-larsxschneider@gmail.com","threadId":"40251","inReplyTo":"1441226355-61103-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v5] git-p4: add config git-p4.pathEncoding","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-02T20:39:15Z","receivedAt":"2015-09-02T20:39:15Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nPerforce keeps the encoding of a path as given by the originating OS.\nGit expects paths encoded as UTF-8. Add a config to tell git-p4 what\nencoding Perforce had used for the paths. This encoding is used to\ntranscode the paths to UTF-8. As an example, Perforce on Windows often\nuses “cp1252” to encode path names.\n\nSigned-off-by: Lars Schneider \n---\n Documentation/git-p4.txt | 7 +++++\n git-p4.py | 11 ++++++++\n t/t9822-git-p4-path-encoding.sh | 60 +++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 78 insertions(+)\n create mode 100755 t/t9822-git-p4-path-encoding.sh\n\ndiff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt\nindex 82aa5d6..7b7a57d 100644\n--- a/Documentation/git-p4.txt\n+++ b/Documentation/git-p4.txt\n@@ -510,6 +510,13 @@ git-p4.useClientSpec::\n \toption '--use-client-spec'. See the \"CLIENT SPEC\" section above.\n \tThis variable is a boolean, not the name of a p4 client.\n \n+git-p4.pathEncoding::\n+\tPerforce keeps the encoding of a path as given by the originating OS.\n+\tGit expects paths encoded as UTF-8. Use this config to tell git-p4 \n+\twhat encoding Perforce had used for the paths. This encoding is used\n+\tto transcode the paths to UTF-8. As an example, Perforce on Windows\n+\toften uses “cp1252” to encode path names.\n+\n Submit variables\n ~~~~~~~~~~~~~~~~\n git-p4.detectRenames::\ndiff --git a/git-p4.py b/git-p4.py\nindex 073f87b..b1ad86d 100755\n--- a/git-p4.py\n+++ b/git-p4.py\n@@ -2213,6 +2213,17 @@ class P4Sync(Command, P4UserMap):\n text = regexp.sub(r'$\\1$', text)\n contents = [ text ]\n \n+ if gitConfig(\"git-p4.pathEncoding\"):\n+ relPath = relPath.decode(gitConfig(\"git-p4.pathEncoding\")).encode('utf8', 'replace')\n+ elif self.verbose:\n+ try:\n+ relPath.decode('ascii')\n+ except:\n+ print (\n+ \"Path with Non-ASCII characters detected and no path encoding defined. \"\n+ \"Please check the encoding: %s\" % relPath\n+ )\n+\n self.gitStream.write(\"M %s inline %s\\n\" % (git_mode, relPath))\n \n # total length...\ndiff --git a/t/t9822-git-p4-path-encoding.sh b/t/t9822-git-p4-path-encoding.sh\nnew file mode 100755\nindex 0000000..3646580\n--- /dev/null\n+++ b/t/t9822-git-p4-path-encoding.sh\n@@ -0,0 +1,60 @@\n+#!/bin/sh\n+\n+test_description='Clone repositories with non ASCII paths'\n+\n+. ./lib-git-p4.sh\n+\n+UTF8_ESCAPED=\"a-\\303\\244_o-\\303\\266_u-\\303\\274.txt\"\n+ISO8859_ESCAPED=\"a-\\344_o-\\366_u-\\374.txt\"\n+\n+test_expect_success 'start p4d' '\n+\tstart_p4d\n+'\n+\n+test_expect_success 'Create a repo containing iso8859-1 encoded paths' '\n+\t(\n+\t\tcd \"$cli\" &&\n+\t\tISO8859=\"$(printf \"$ISO8859_ESCAPED\")\" &&\n+\t\techo content123 >\"$ISO8859\" &&\n+\t\tp4 add \"$ISO8859\" &&\n+\t\tp4 submit -d \"test commit\"\n+\t)\n+'\n+\n+test_expect_success 'Clone repo containing iso8859-1 encoded paths without git-p4.pathEncoding' '\n+\tgit p4 clone --destination=\"$git\" //depot &&\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tUTF8=\"$(printf \"$UTF8_ESCAPED\")\" &&\n+\t\techo $UTF8 >expect &&\n+\t\tgit -c core.quotepath=false ls-files >actual &&\n+\t\ttest_must_fail test_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'Clone repo containing iso8859-1 encoded paths with git-p4.pathEncoding' '\n+\n+\ttest_when_finished cleanup_git &&\n+\t(\n+\t\tcd \"$git\" &&\n+\t\tgit init . &&\n+\t\tgit config git-p4.pathEncoding iso8859-1 &&\n+\t\tgit p4 clone --use-client-spec --destination=\"$git\" //depot &&\n+\t\tUTF8=\"$(printf \"$UTF8_ESCAPED\")\" &&\n+\t\techo $UTF8 >expect &&\n+\t\tgit -c core.quotepath=false ls-files >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tcat >expect <<-\\EOF &&\n+\t\tcontent123\n+\t\tEOF\n+\t\tcat $UTF8 >actual &&\n+\t\ttest_cmp expect actual\n+\t)\n+'\n+\n+test_expect_success 'kill p4d' '\n+\tkill_p4d\n+'\n+\n+test_done\n-- \n1.9.5 (Apple Git-50.3)\n"},{"id":"269239","messageId":"CAE5ih7_Kje_Qrz7zKH4uSGziVpdp6VqsZf7hSMwQkvta97CG9w@mail.gmail.com","threadId":"40251","inReplyTo":"1441226355-61103-2-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v5] git-p4: add config git-p4.pathEncoding","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-09-02T21:32:10Z","receivedAt":"2015-09-02T21:32:10Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 2 September 2015 at 21:39, wrote:\n> From: Lars Schneider \n>\n> Perforce keeps the encoding of a path as given by the originating OS.\n> Git expects paths encoded as UTF-8. Add a config to tell git-p4 what\n> encoding Perforce had used for the paths. This encoding is used to\n> transcode the paths to UTF-8. As an example, Perforce on Windows often\n> uses “cp1252” to encode path names.\n\nLooks good to me, and the test passes.\n\nNote that git diff --check complains about a trailing whitespace.\n"},{"id":"269296","messageId":"983771583.1014619.1441268816360.JavaMail.zimbra@ensimag.grenoble-inp.fr","threadId":"40251","inReplyTo":"1441226355-61103-2-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v5] git-p4: add config git-p4.pathEncoding","fromName":"Remi Galan Alfonso","fromEmail":"remi.galan-alfonso@ensimag.grenoble-inp.fr","sentAt":"2015-09-03T08:26:56Z","receivedAt":"2015-09-03T08:26:56Z","isPatch":true,"sender":{"key":"remi.galan-alfonso@ensimag.grenoble-inp.fr","avatar":"https://avatars.githubusercontent.com/u/12509162?v=4"},"body":"Hi Lars,\n\nLars Schneider writes:\n\n> +test_expect_success 'Clone repo containing iso8859-1 encoded paths with git-p4.pathEncoding' '\n> +\n> + test_when_finished cleanup_git &&\n> + (\n> + cd \"$git\" &&\n> + git init . &&\n> + git config git-p4.pathEncoding iso8859-1 &&\n\nWouldn't 'test_config' be better here?\n\nThanks,\nRémi\n"},{"id":"269324","messageId":"5E77893F-8E13-4696-B7A6-46E10FE41989@gmail.com","threadId":"40251","inReplyTo":"983771583.1014619.1441268816360.JavaMail.zimbra@ensimag.grenoble-inp.fr","subject":"Re: [PATCH v5] git-p4: add config git-p4.pathEncoding","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-09-03T15:20:35Z","receivedAt":"2015-09-03T15:20:35Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 03 Sep 2015, at 10:26, Remi Galan Alfonso wrote:\n\n> Hi Lars,\n> \n> Lars Schneider writes:\n> \n>> +test_expect_success 'Clone repo containing iso8859-1 encoded paths with git-p4.pathEncoding' '\n>> +\n>> + test_when_finished cleanup_git &&\n>> + (\n>> + cd \"$git\" &&\n>> + git init . &&\n>> + git config git-p4.pathEncoding iso8859-1 &&\n> \n> Wouldn't 'test_config' be better here?\n> \nIf I got Eric’s explanation right in \"[PATCH v1] t9821: use test_config” then we should not use “test_config” here?\n\nThanks,\nLars"},{"id":"269404","messageId":"1646294553.1067005.1441355413320.JavaMail.zimbra@ensimag.grenoble-inp.fr","threadId":"40251","inReplyTo":"5E77893F-8E13-4696-B7A6-46E10FE41989@gmail.com","subject":"Re: [PATCH v5] git-p4: add config git-p4.pathEncoding","fromName":"Remi Galan Alfonso","fromEmail":"remi.galan-alfonso@ensimag.grenoble-inp.fr","sentAt":"2015-09-04T08:30:13Z","receivedAt":"2015-09-04T08:30:13Z","isPatch":true,"sender":{"key":"remi.galan-alfonso@ensimag.grenoble-inp.fr","avatar":"https://avatars.githubusercontent.com/u/12509162?v=4"},"body":"Lars Schneider writes:\n\n> > Hi Lars,\n> >\n> > Lars Schneider writes:\n> >\n> >> +test_expect_success 'Clone repo containing iso8859-1 encoded paths with git-p4.pathEncoding' '\n> >> +\n> >> + test_when_finished cleanup_git &&\n> >> + (\n> >> + cd \"$git\" &&\n> >> + git init . &&\n> >> + git config git-p4.pathEncoding iso8859-1 &&\n> >\n> > Wouldn't 'test_config' be better here?\n> >\n> If I got Eric’s explanation right in \"[PATCH v1] t9821: use\n> test_config” then we should not use “test_config” here?\n\nAfter reading Eric's point of view, I agree with him about this.\n\nSo, sorry for the noise, now I'll know better in the future when\nto use and not use 'test_config'.\n\nThanks,\nRémi\n"}]}