{"thread":{"id":"62973","subject":"[PATCH 0/2] gitk: Fixing file name encoding issues.","startedAt":"2025-02-18T22:42:29Z","lastAt":"2025-02-19T18:11:55Z","messageCount":7,"participants":["Kazuhiro Kato via GitGitGadget","Junio C Hamano","Konstantin Khomoutov"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"512646","messageId":"pull.1886.git.git.1739918546.gitgitgadget@gmail.com","threadId":"62973","inReplyTo":null,"subject":"[PATCH 0/2] gitk: Fixing file name encoding issues.","fromName":"Kazuhiro Kato via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-02-18T22:42:24Z","receivedAt":"2025-02-18T22:42:29Z","isPatch":true,"sender":{"key":"name:Kazuhiro Kato","avatar":null},"body":"fix: file name encoding issues. fix: when resolving merge conflicts,\njapanese file names become garbled.\n\nKazuhiro Kato (2):\n Fixing file name encoding issues.\n fix: when resolving merge conflicts, japanese file names become\n garbled.\n\n gitk-git/gitk | 7 +++++--\n 1 file changed, 5 insertions(+), 2 deletions(-)\n\n\nbase-commit: bc204b742735ae06f65bb20291c95985c9633b7f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1886%2Fkkato233%2Ffix_filename_encoding-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1886/kkato233/fix_filename_encoding-v1\nPull-Request: https://github.com/git/git/pull/1886\n-- \ngitgitgadget\n"},{"id":"512647","messageId":"d33fb3d940cf97bdfe3dc544763a51df4874a5c0.1739918546.git.gitgitgadget@gmail.com","threadId":"62973","inReplyTo":"pull.1886.git.git.1739918546.gitgitgadget@gmail.com","subject":"[PATCH 1/2] Fixing file name encoding issues.","fromName":"Kazuhiro Kato via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-02-18T22:42:25Z","receivedAt":"2025-02-18T22:42:30Z","isPatch":true,"sender":{"key":"name:Kazuhiro Kato","avatar":null},"body":"From: Kazuhiro Kato \n\nSigned-off-by: Kazuhiro Kato \n---\n gitk-git/gitk | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/gitk-git/gitk b/gitk-git/gitk\nindex 47a7c1d29c4..88951ed2384 100755\n--- a/gitk-git/gitk\n+++ b/gitk-git/gitk\n@@ -12379,6 +12379,7 @@ catch {\n if {$gitencoding == \"\"} {\n set gitencoding \"utf-8\"\n }\n+encoding system utf-8\n set tclencoding [tcl_encoding $gitencoding]\n if {$tclencoding == {}} {\n puts stderr \"Warning: encoding $gitencoding is not supported by Tcl/Tk\"\n-- \ngitgitgadget\n\n"},{"id":"512648","messageId":"c698805f088e0643e5faf027d4eaa6de14d6c1ff.1739918546.git.gitgitgadget@gmail.com","threadId":"62973","inReplyTo":"pull.1886.git.git.1739918546.gitgitgadget@gmail.com","subject":"[PATCH 2/2] fix: when resolving merge conflicts, japanese file names become garbled.","fromName":"Kazuhiro Kato via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-02-18T22:42:26Z","receivedAt":"2025-02-18T22:42:30Z","isPatch":true,"sender":{"key":"name:Kazuhiro Kato","avatar":null},"body":"From: Kazuhiro Kato \n\nSigned-off-by: Kazuhiro Kato \n---\n gitk-git/gitk | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/gitk-git/gitk b/gitk-git/gitk\nindex 88951ed2384..f4f8dbd5fad 100755\n--- a/gitk-git/gitk\n+++ b/gitk-git/gitk\n@@ -8205,12 +8205,13 @@ proc parseblobdiffline {ids line} {\n \n if {$type eq \"--cc\"} {\n # start of a new file in a merge diff\n- set fname [string range $line 10 end]\n+ set fname_raw [string range $line 10 end]\n+ set fname [encoding convertfrom $fname_raw]\n if {[lsearch -exact $treediffs($ids) $fname] < 0} {\n lappend treediffs($ids) $fname\n add_flist [list $fname]\n }\n-\n+ set fname $fname_raw\n } else {\n set line [string range $line 11 end]\n # If the name hasn't changed the length will be odd,\n@@ -8310,6 +8311,7 @@ proc parseblobdiffline {ids line} {\n set diffinhdr 0\n return\n }\n+ set line [encoding convertfrom $line]\n $ctext insert end \"$line\\n\" filesep\n \n } else {\n-- \ngitgitgadget\n"},{"id":"512649","messageId":"xmqq7c5mj272.fsf@gitster.g","threadId":"62973","inReplyTo":"pull.1886.git.git.1739918546.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/2] gitk: Fixing file name encoding issues.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-18T22:52:17Z","receivedAt":"2025-02-18T22:52:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kazuhiro Kato via GitGitGadget\" writes:\n\n> fix: file name encoding issues. fix: when resolving merge conflicts,\n> japanese file names become garbled.\n>\n> Kazuhiro Kato (2):\n> Fixing file name encoding issues.\n> fix: when resolving merge conflicts, japanese file names become\n> garbled.\n>\n> gitk-git/gitk | 7 +++++--\n> 1 file changed, 5 insertions(+), 2 deletions(-)\n\nPlease\n\n - base your changes to j6t's gitk repository\n\n https://github.com/j6t/gitk\n\n where the file your patch would touch should appear at the top\n level of the working tree. This \"independent\" history is then\n merged into my tree with \"git pull -Xsubtree=gitk-git\".\n\n - have the e-mail address of the gitk maintainer on the Cc: line,\n i.e. \"Cc: Johannes Sixt \".\n\nThanks.\n\n\ncf.\n https://lore.kernel.org/git/b2038430-62dc-41fa-86c2-c0a14bd25e0f@kdbg.org/\n https://lore.kernel.org/git/7b826bba-11cf-4f45-8292-937522dbaf29@kdbg.org/\n https://lore.kernel.org/git/5ccc1943-c2a3-4896-a858-aa5fd6cdd426@kdbg.org/\n\n\n>\n>\n> base-commit: bc204b742735ae06f65bb20291c95985c9633b7f\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1886%2Fkkato233%2Ffix_filename_encoding-v1\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1886/kkato233/fix_filename_encoding-v1\n> Pull-Request: https://github.com/git/git/pull/1886\n"},{"id":"512722","messageId":"20250219173032.x5gzw6hsvsu4flba@carbon","threadId":"62973","inReplyTo":"d33fb3d940cf97bdfe3dc544763a51df4874a5c0.1739918546.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/2] Fixing file name encoding issues.","fromName":"Konstantin Khomoutov","fromEmail":"kostix@bswap.ru","sentAt":"2025-02-19T17:30:32Z","receivedAt":"2025-02-19T17:54:34Z","isPatch":true,"sender":{"key":"kostix@bswap.ru","avatar":null},"body":"On Tue, Feb 18, 2025 at 10:42:25PM +0000, Kazuhiro Kato via GitGitGadget wrote:\n\n[...]\n> diff --git a/gitk-git/gitk b/gitk-git/gitk\n> index 47a7c1d29c4..88951ed2384 100755\n> --- a/gitk-git/gitk\n> +++ b/gitk-git/gitk\n> @@ -12379,6 +12379,7 @@ catch {\n> if {$gitencoding == \"\"} {\n> set gitencoding \"utf-8\"\n> }\n> +encoding system utf-8\n> set tclencoding [tcl_encoding $gitencoding]\n> if {$tclencoding == {}} {\n> puts stderr \"Warning: encoding $gitencoding is not supported by Tcl/Tk\"\n\nI'm not sure one should sensibly do this - except to implement some\nwell-understood and well-documented kludge, simply because the \"system\nencoding\" is supposed to be set by the Tcl runtime.\n\nLet's demonstrate (on a Linux-based system with UTF-8 locale):\n\n tmp$ touch 'привет мир.txt'\n tmp$ tclsh\n % encoding system\n utf-8\n % glob привет*.txt\n {привет мир.txt}\n % encoding system cp1251\n % encoding system\n cp1251\n % glob привет*.txt\n no files matched glob pattern \"привет*.txt\"\n %\n\nHere, CP1251 is a Windows \"code page\" for Cyrillic; it's what\n[encoding system] reports on Windows systems.\n\nNote that I create a file whose name is two words in Cyrillic script encoded\nin UTF-8, and Tcl is fine finding and dispaying this file (via its \"glob\"\ncommand). But as soon as I change the Tcl's system encoding to another\n8-bit Cyrillic encoding, globbing stops working.\n\nMoreover, if I'd do [glob *.txt] so that my file would be matched anyway,\nits named would not be readable since Tcl would re-encode it from CP1251\nto Unicode, but the name is encoded in UTF-8, not CP1251.\n\nIn other words, your patch (supposedly) works on UTF-8-based systems\nwhich is common to Linux-based OSes and MacOS, but I'm afraid it won't work on\nWindows.\n\n"},{"id":"512723","messageId":"xmqqmsehg6ea.fsf@gitster.g","threadId":"62973","inReplyTo":"20250219173032.x5gzw6hsvsu4flba@carbon","subject":"Re: [PATCH 1/2] Fixing file name encoding issues.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-19T18:02:05Z","receivedAt":"2025-02-19T18:02:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Konstantin Khomoutov writes:\n\n> On Tue, Feb 18, 2025 at 10:42:25PM +0000, Kazuhiro Kato via GitGitGadget wrote:\n>\n> [...]\n>> diff --git a/gitk-git/gitk b/gitk-git/gitk\n>> index 47a7c1d29c4..88951ed2384 100755\n>> --- a/gitk-git/gitk\n>> +++ b/gitk-git/gitk\n>> @@ -12379,6 +12379,7 @@ catch {\n>> if {$gitencoding == \"\"} {\n>> set gitencoding \"utf-8\"\n>> }\n>> +encoding system utf-8\n>> set tclencoding [tcl_encoding $gitencoding]\n>> if {$tclencoding == {}} {\n>> puts stderr \"Warning: encoding $gitencoding is not supported by Tcl/Tk\"\n>\n> I'm not sure one should sensibly do this - except to implement some\n> well-understood and well-documented kludge, simply because the \"system\n> encoding\" is supposed to be set by the Tcl runtime.\n> ...\n> In other words, your patch (supposedly) works on UTF-8-based systems\n> which is common to Linux-based OSes and MacOS, but I'm afraid it won't work on\n> Windows.\n\nIn other words, things should work without \"encoding system blah\"\nforcing a particular encoding that Tcl may not agree with? Would\nthis mean perhaps in a \"curious\" repository with paths encoded in\nsomething Tcl does not expect to be used (e.g., on a UTF-8 system\nsomehow EUC-jp is used for paths containing Japanese characters), it\nneeds to be possible to specify a \"curious\" encoding either with an\nend-user on-demand action (e.g., menu items) or with an repository\nconfiguration (e.g., gitk.pathencoding = euc-jp)?\n\nThanks.\n"},{"id":"512724","messageId":"xmqqh64pg5xy.fsf@gitster.g","threadId":"62973","inReplyTo":"c698805f088e0643e5faf027d4eaa6de14d6c1ff.1739918546.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/2] fix: when resolving merge conflicts, japanese file names become garbled.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-19T18:11:53Z","receivedAt":"2025-02-19T18:11:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kazuhiro Kato via GitGitGadget\" writes:\n\n> From: Kazuhiro Kato \n\nHere is a place to give a bit more context. In what way the current\ncode is wrong, what end-user visible symptoms are brought due to\nthat wrongness, what is the correct way to implement it, etc.\n\n> Signed-off-by: Kazuhiro Kato \n> ---\n> gitk-git/gitk | 6 ++++--\n> 1 file changed, 4 insertions(+), 2 deletions(-)\n>\n> diff --git a/gitk-git/gitk b/gitk-git/gitk\n> index 88951ed2384..f4f8dbd5fad 100755\n> --- a/gitk-git/gitk\n> +++ b/gitk-git/gitk\n> @@ -8205,12 +8205,13 @@ proc parseblobdiffline {ids line} {\n> \n> if {$type eq \"--cc\"} {\n> # start of a new file in a merge diff\n> - set fname [string range $line 10 end]\n> + set fname_raw [string range $line 10 end]\n> + set fname [encoding convertfrom $fname_raw]\n\nIs this \"the Tcl read from git things as sequence of bytes, not\ncharacters, so somebody needs to pass the bytes to \"encoding\"\nfunction to turn them into a sequence of characters? Unless\neverything is US-ASCII, that is.\n\nIf that is the case, presumably the $line has a sequence of bytes,\nso it may be wrong to chop it at 10th position (presumably that's\n10th byte, not 10th character) when we are trying to teach the code\nto deal with non-ASCII data, no? \n\nI am reasonably sure that [string length \"diff --git\"] is where 10\ncomes from, and that prefix will always be in ASCII, but it feels\nsafer and kosher if we converted the whole line first and then\nchopped off the prefix.\n\nThe patch title says Japanese, but I would imagine this applies to\nanything non-ASCII, so it would be better to retitle the patch to\nsay \"non-ASCII\" instead to signal that the issue the patch fixes\napplies more widely.\n\nThanks.\n"}]}