{"thread":{"id":"16842","subject":"Git merge conflicts and encoding of logs","startedAt":"2008-12-23T04:48:44Z","lastAt":"2008-12-23T23:37:01Z","messageCount":5,"participants":["Junichi Uekawa","Junio C Hamano","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"98590","messageId":"87lju7h4yb.dancerj%dancer@netfort.gr.jp","threadId":"16842","inReplyTo":null,"subject":"Git merge conflicts and encoding of logs","fromName":"Junichi Uekawa","fromEmail":"dancer@netfort.gr.jp","sentAt":"2008-12-23T04:48:44Z","receivedAt":"2008-12-23T04:48:44Z","isPatch":false,"sender":{"key":"dancer@netfort.gr.jp","avatar":null},"body":"Hi,\n\n\nGit merge conflict will insert '<<< first line of commit log message'\n'===' '>>>' markers to the text file that is causing a conflict.\n\nUnfortunately, the encoding of the text file may be different from the\nlog message encoding, and that results in a file which has a mixed\nencoding (which is pretty hard to edit from any editor BTW).\n\nMy use case is editing platex files (iso-2022-jp encoded) with log\nmessages of utf-8.\n\nhttp://git.debian.org/?p=tokyodebian/monthly-report.git;a=summary\n\n\n\n\n... Thinking about it, it's probably the same encoding problem as git\nblame.  Is there already a good way to fix this or is this something\nthat needs fixing?\n\n\nregards,\n\tjunichi\n-- \ndancer@{netfort.gr.jp,debian.org}\n"},{"id":"98592","messageId":"7vprjjfgi7.fsf@gitster.siamese.dyndns.org","threadId":"16842","inReplyTo":"87lju7h4yb.dancerj%dancer@netfort.gr.jp","subject":"Re: Git merge conflicts and encoding of logs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-23T08:22:08Z","receivedAt":"2008-12-23T08:22:08Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junichi Uekawa <dancer@netfort.gr.jp> writes:\n\n> Git merge conflict will insert '<<< first line of commit log message'\n> '===' '>>>' markers to the text file that is causing a conflict.\n>\n> Unfortunately, the encoding of the text file may be different from the\n> log message encoding, and that results in a file which has a mixed\n> encoding (which is pretty hard to edit from any editor BTW).\n>\n> My use case is editing platex files (iso-2022-jp encoded) with log\n> messages of utf-8.\n>\n> ... Thinking about it, it's probably the same encoding problem as git\n> blame.\n\nWhat 69cd8f6 (builtin-blame: Reencode commit messages according to git-log\nrules., 2008-10-22) does to git-blame is to re-encode the data taken from\nthe commit log to i18n.logoutputencoding, and put that in the datastream.\n\nIf your commit object have names and messages in utf-8, and if you set\ni18n.logoutputencoding to iso-2022-jp, that would reencode data taken from\nthe commit object in iso-2022-jp and sprinkle them in the blame\ndatastream.\n\nThe issue would be certainly similar, *if* anything on your <<</===/>>>\nlines came from commit log message, but I couldn't trigger what you\ndescribe.  I prepared a history of this shape:\n\n   B\n  /\n o---A\n\nwith ISO-2022-JP payload and UTF-8 commit log message.  Then, I added:\n\n        [i18n]\n                logoutputencoding = iso-2022-jp\n\nwhich lets me read \"git log -p --all\" quite comfortably.  Everything comes\nout as good old JISX0208.  So far, so good.\n\nThen while on branch B, I tried to merge A, which resulted in conflicts\nthat looked like this:\n\n<<<<<<< HEAD:foo\nこれはサイドブランチの変更です。\nやはり JIS コードで書いてます。\n=======\n日本語のファイルです。\nJIS コードで書いてます。\n>>>>>>> master:foo\n\nThe above will probably come out as UTF-8 in this mail text, but the point\nis that the confict side markers do not have anything but filename and the\nbranch name.  I am still scratching my head trying to see where in the\nmerge-recursive codepath you got snippet of log message.\n\nA bundle from my test repository is attached.  You can use it to reproduce\nthe repository like this:\n \n    $ cd /var/tmp && mkdir test && cd test && git init\n    $ git pull ../x.bndl master\n    $ git fetch ../x.bndl side:side\n\n"},{"id":"98594","messageId":"4950A422.5030100@viscovery.net","threadId":"16842","inReplyTo":"7vprjjfgi7.fsf@gitster.siamese.dyndns.org","subject":"Re: Git merge conflicts and encoding of logs","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-12-23T08:41:06Z","receivedAt":"2008-12-23T08:41:06Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Junio C Hamano schrieb:\n> <<<<<<< HEAD:foo\n> これはサイドブランチの変更です。\n> やはり JIS コードで書いてます。\n> =======\n> 日本語のファイルです。\n> JIS コードで書いてます。\n>>>>>>>> master:foo\n> \n> The above will probably come out as UTF-8 in this mail text, but the point\n> is that the confict side markers do not have anything but filename and the\n> branch name.  I am still scratching my head trying to see where in the\n> merge-recursive codepath you got snippet of log message.\n\nTry rebase -i instead of merge: This should put summary lines onto the\nconflict markers.\n\n-- Hannes\n"},{"id":"98595","messageId":"7vlju7fbqu.fsf@gitster.siamese.dyndns.org","threadId":"16842","inReplyTo":"4950A422.5030100@viscovery.net","subject":"Re: Git merge conflicts and encoding of logs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-23T10:04:57Z","receivedAt":"2008-12-23T10:04:57Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt <j.sixt@viscovery.net> writes:\n\n> Junio C Hamano schrieb:\n>> <<<<<<< HEAD:foo\n>> これはサイドブランチの変更です。\n>> やはり JIS コードで書いてます。\n>> =======\n>> 日本語のファイルです。\n>> JIS コードで書いてます。\n>>>>>>>>> master:foo\n>> \n>> The above will probably come out as UTF-8 in this mail text, but the point\n>> is that the confict side markers do not have anything but filename and the\n>> branch name.  I am still scratching my head trying to see where in the\n>> merge-recursive codepath you got snippet of log message.\n>\n> Try rebase -i instead of merge: This should put summary lines onto the\n> conflict markers.\n\nAh, that's cherry-pick.\n\nThe fix should be around the area this weather-balloon patch touches.\n\nNote that this does not correctly work yet, and it seems that somewhere the\nstring is truncated.\n\nBut I won't be debugging it further for now...\n\n----\n builtin-revert.c |   15 ++++++++++++++-\n 1 files changed, 14 insertions(+), 1 deletions(-)\n\ndiff --git c/builtin-revert.c w/builtin-revert.c\nindex d48313c..47ff16f 100644\n--- c/builtin-revert.c\n+++ w/builtin-revert.c\n@@ -244,6 +244,19 @@ static struct tree *empty_tree(void)\n \treturn tree;\n }\n \n+static char *branch_label_to_output_encoding(char *oneline)\n+{\n+\tif (git_log_output_encoding &&\n+\t    strcmp(git_log_output_encoding, git_commit_encoding)) {\n+\t\tchar *it = reencode_string(oneline,\n+\t\t\t\t\t   git_log_output_encoding,\n+\t\t\t\t\t   git_commit_encoding);\n+\t\tif (it)\n+\t\t\treturn it;\n+\t}\n+\treturn oneline;\n+}\n+\n static int revert_or_cherry_pick(int argc, const char **argv)\n {\n \tunsigned char head[20];\n@@ -373,7 +386,7 @@ static int revert_or_cherry_pick(int argc, const char **argv)\n \tread_cache();\n \tinit_merge_options(&o);\n \to.branch1 = \"HEAD\";\n-\to.branch2 = oneline;\n+\to.branch2 = branch_label_to_output_encoding(oneline);\n \n \thead_tree = parse_tree_indirect(head);\n \tnext_tree = next ? next->tree : empty_tree();\n"},{"id":"98644","messageId":"877i5qh3aa.dancerj%dancer@netfort.gr.jp","threadId":"16842","inReplyTo":"7vlju7fbqu.fsf@gitster.siamese.dyndns.org","subject":"Re: Git merge conflicts and encoding of logs","fromName":"Junichi Uekawa","fromEmail":"dancer@netfort.gr.jp","sentAt":"2008-12-23T23:37:01Z","receivedAt":"2008-12-23T23:37:01Z","isPatch":false,"sender":{"key":"dancer@netfort.gr.jp","avatar":null},"body":"At Tue, 23 Dec 2008 02:04:57 -0800,\nJunio C Hamano wrote:\n> \n> Johannes Sixt <j.sixt@viscovery.net> writes:\n> \n> > Junio C Hamano schrieb:\n> >> <<<<<<< HEAD:foo\n> >> これはサイドブランチの変更です。\n> >> やはり JIS コードで書いてます。\n> >> =======\n> >> 日本語のファイルです。\n> >> JIS コードで書いてます。\n> >>>>>>>>> master:foo\n> >> \n> >> The above will probably come out as UTF-8 in this mail text, but the point\n> >> is that the confict side markers do not have anything but filename and the\n> >> branch name.  I am still scratching my head trying to see where in the\n> >> merge-recursive codepath you got snippet of log message.\n> >\n> > Try rebase -i instead of merge: This should put summary lines onto the\n> > conflict markers.\n> \n> Ah, that's cherry-pick.\n> \n> The fix should be around the area this weather-balloon patch touches.\n> \n> Note that this does not correctly work yet, and it seems that somewhere the\n> string is truncated.\n\n\nHi, I've read patches from Alexander Gavrilov which try to support\nper-file encoding in gitattributes (encoding), I assume such features\nare not in yet.\n\nMy git repository has mixture of iso-2022-jp(platex source), EUC-JP\n(some graph source code and other things), UTF-8, so log output format\nunification to iso-2022-jp is not ideal. That way, I'd have to guess\nwhich file is going to have a conflict through cherry-pick and define\nthe logoutput encoding.\n\n\n\nregards,\n\tjunichi\n-- \ndancer@{netfort.gr.jp,debian.org}\n"}]}