{"thread":{"id":"56385","subject":"[PATCH] pretty-options.txt: describe supported encoding","startedAt":"2021-08-26T21:34:34Z","lastAt":"2021-08-27T23:21:02Z","messageCount":7,"participants":["Christopher Yeleighton via GitGitGadget","Bagas Sanjaya","Krzysztof Żelechowski","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"433883","messageId":"pull.1079.git.git.1630013668862.gitgitgadget@gmail.com","threadId":"56385","inReplyTo":null,"subject":"[PATCH] pretty-options.txt: describe supported encoding","fromName":"Christopher Yeleighton via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-08-26T21:34:28Z","receivedAt":"2021-08-26T21:34:34Z","isPatch":true,"sender":{"key":"name:Christopher Yeleighton","avatar":null},"body":"From: Christopher Yeleighton \n\nPlease fix the manual for git log. It should say what encoding is recognised\n(namely if supported by iconv(1), except that POSIX character maps of\niconv(1p) are not supported), and that an unrecognised encoding is ignored.\n\nSigned-off-by: \n---\n log: describe supported encoding\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1079%2Fyecril71pl%2Fpatch-1-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1079/yecril71pl/patch-1-v1\nPull-Request: https://github.com/git/git/pull/1079\n\n Documentation/pretty-options.txt | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/pretty-options.txt b/Documentation/pretty-options.txt\nindex 27ddaf84a19..4f8376d681b 100644\n--- a/Documentation/pretty-options.txt\n+++ b/Documentation/pretty-options.txt\n@@ -36,9 +36,13 @@ people using 80-column terminals.\n \tThe commit objects record the encoding used for the log message\n \tin their encoding header; this option can be used to tell the\n \tcommand to re-code the commit log message in the encoding\n-\tpreferred by the user. For non plumbing commands this\n-\tdefaults to UTF-8. Note that if an object claims to be encoded\n-\tin `X` and we are outputting in `X`, we will output the object\n+\tpreferred by the user.\n+\tThe encoding must be a system encoding supported by iconv(1),\n+\totherwise this option will be ignored.\n+\tPOSIX character maps used by iconv(1p) are not supported.\n+\tFor non-plumbing commands this defaults to UTF-8.\n+\tNote that if an object claims to be encoded in `X`\n+\tand we are outputting in `X`, we shall output the object\n \tverbatim; this means that invalid sequences in the original\n \tcommit may be copied to the output.\n \n\nbase-commit: c4203212e360b25a1c69467b5a8437d45a373cac\n-- \ngitgitgadget\n"},{"id":"433922","messageId":"0877bb5d-da4b-125d-7beb-c3138903f468@gmail.com","threadId":"56385","inReplyTo":"pull.1079.git.git.1630013668862.gitgitgadget@gmail.com","subject":"Re: [PATCH] pretty-options.txt: describe supported encoding","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-08-27T10:46:22Z","receivedAt":"2021-08-27T10:46:29Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 27/08/21 04.34, Christopher Yeleighton via GitGitGadget wrote:\n> From: Christopher Yeleighton \n> \n> Please fix the manual for git log. It should say what encoding is recognised\n> (namely if supported by iconv(1), except that POSIX character maps of\n> iconv(1p) are not supported), and that an unrecognised encoding is ignored.\n> \n> Signed-off-by: \n> ---\n\nThe commit message should be:\n\"git log recognizes only system encodings supported by iconv(1), but not \nPOSIX character maps used by iconv(1p). Document it.\".\n\n> \tThe commit objects record the encoding used for the log message\n> \tin their encoding header; this option can be used to tell the\n> \tcommand to re-code the commit log message in the encoding\n> -\tpreferred by the user. For non plumbing commands this\n> -\tdefaults to UTF-8. Note that if an object claims to be encoded\n> -\tin `X` and we are outputting in `X`, we will output the object\n> +\tpreferred by the user.\n> +\tThe encoding must be a system encoding supported by iconv(1),\n> +\totherwise this option will be ignored.\n> +\tPOSIX character maps used by iconv(1p) are not supported.\n> +\tFor non-plumbing commands this defaults to UTF-8.\n> +\tNote that if an object claims to be encoded in `X`\n> +\tand we are outputting in `X`, we shall output the object\n> \tverbatim; this means that invalid sequences in the original\n> \tcommit may be copied to the output.\n> \n\nI think POSIX character maps and encoding are the same, what are their \ndifferences? Reading iconv(1p) [1] doesn't give definition of the former.\n\n[1]: https://man7.org/linux/man-pages/man1/iconv.1p.html\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"433926","messageId":"2623327.rHeH4EdoKx@localhost.localdomain","threadId":"56385","inReplyTo":"0877bb5d-da4b-125d-7beb-c3138903f468@gmail.com","subject":"Re: [PATCH] pretty-options.txt: describe supported encoding","fromName":"Krzysztof Żelechowski","fromEmail":"giecrilj@stegny.2a.pl","sentAt":"2021-08-27T11:47:44Z","receivedAt":"2021-08-27T11:47:54Z","isPatch":true,"sender":{"key":"giecrilj@stegny.2a.pl","avatar":"https://gravatar.com/avatar/419ea28e22b430cae87994078ab5203f95eb2146ac1fe20739b0521530b26a25?d=mp&s=160"},"body":"Dnia piątek, 27 sierpnia 2021 12:46:22 CEST Bagas Sanjaya pisze:\n> I think POSIX character maps and encoding are the same, what are their\n> differences? Reading iconv(1p) [1] doesn't give definition of the former.\n> \n> [1]: https://man7.org/linux/man-pages/man1/iconv.1p.html\n\nSystem encoding providers are code, POSIX character maps are data.\n\nChris\n\n\n\n"},{"id":"433927","messageId":"2247912.lYO0ccLKhl@localhost.localdomain","threadId":"56385","inReplyTo":"0877bb5d-da4b-125d-7beb-c3138903f468@gmail.com","subject":"[PATCH v2] pretty-options.txt: describe supported encoding","fromName":"Krzysztof Żelechowski","fromEmail":"giecrilj@stegny.2a.pl","sentAt":"2021-08-27T11:51:34Z","receivedAt":"2021-08-27T11:51:43Z","isPatch":true,"sender":{"key":"giecrilj@stegny.2a.pl","avatar":"https://gravatar.com/avatar/419ea28e22b430cae87994078ab5203f95eb2146ac1fe20739b0521530b26a25?d=mp&s=160"},"body":"git log recognises only system encodings supported by iconv(1), but not \nPOSIX character maps used by iconv(1p). Document it.\n\nSigned-off-by: \n\ndiff --git a/Documentation/pretty-options.txt b/Documentation/pretty-\noptions.txt\nindex 27ddaf84a19..4f8376d681b 100644\n--- a/Documentation/pretty-options.txt\n+++ b/Documentation/pretty-options.txt\n@@ -36,9 +36,13 @@ people using 80-column terminals.\n The commit objects record the encoding used for the log message\n in their encoding header; this option can be used to tell the\n command to re-code the commit log message in the encoding\n- preferred by the user. For non plumbing commands this\n- defaults to UTF-8. Note that if an object claims to be encoded\n- in `X` and we are outputting in `X`, we will output the object\n+ preferred by the user.\n+ The encoding must be a system encoding supported by iconv(1),\n+ otherwise this option will be ignored.\n+ POSIX character maps used by iconv(1p) are not supported.\n+ For non-plumbing commands this defaults to UTF-8.\n+ Note that if an object claims to be encoded in `X`\n+ and we are outputting in `X`, we shall output the object\n verbatim; this means that invalid sequences in the original\n commit may be copied to the output.\n \n\n\n"},{"id":"433936","messageId":"xmqq5yvqbz0j.fsf@gitster.g","threadId":"56385","inReplyTo":"2247912.lYO0ccLKhl@localhost.localdomain","subject":"Re: [PATCH v2] pretty-options.txt: describe supported encoding","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-08-27T17:03:56Z","receivedAt":"2021-08-27T17:04:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Krzysztof Żelechowski writes:\n\n> git log recognises only system encodings supported by iconv(1), but not \n> POSIX character maps used by iconv(1p). Document it.\n>\n> Signed-off-by: \n\nThe \"Human Readable Name \" on this line must match\nthe one on the \"From: \" line that records the author of the patch.\n\nIf you are forwarding somebody else's patch (with or without\nimprovement), we also need your sign off.\n\n> diff --git a/Documentation/pretty-options.txt b/Documentation/pretty-\n> options.txt\n> index 27ddaf84a19..4f8376d681b 100644\n> --- a/Documentation/pretty-options.txt\n> +++ b/Documentation/pretty-options.txt\n> @@ -36,9 +36,13 @@ people using 80-column terminals.\n> The commit objects record the encoding used for the log message\n> in their encoding header; this option can be used to tell the\n> command to re-code the commit log message in the encoding\n> - preferred by the user. For non plumbing commands this\n> - defaults to UTF-8. Note that if an object claims to be encoded\n> - in `X` and we are outputting in `X`, we will output the object\n> + preferred by the user.\n\n\n> + The encoding must be a system encoding supported by iconv(1),\n> + otherwise this option will be ignored.\n> + POSIX character maps used by iconv(1p) are not supported.\n\nThis paragraph is a bit hard to grok.\n\nI think it is saying that the \"-f frommap -t tomap\" form in [*1*]\nthat can use arbitrary character set description file is not\nsupported, but \"-f fromcode -t tocode\" form, which also is what\niconv_open() takes [*2*], is supported. Am I reading it correctly?\n\nIs there an easier-to-read way to explain the distinction to our\naverage reader?\n\nWhat I am getting at is this. Imagine average users who need to see\ntheir commits recoded to iso-8859-2. They see \"git log\" has\n\"--encoding=\" option, read the above paragraph and wonder\nif they are on the supported side or unsupported side of the above\nparagraph. I want to make it easy for them to stop wondering.\n\nFor that purpose, \"iconv(1) vs iconv(1p)\" would not help them very\nmuch, especially considering that not all Git users are UNIX users\n(they probably do not even know what (1) and (1p) means).\n\n> + For non-plumbing commands this defaults to UTF-8.\n\nI think I can guess why the patch wants to change \"non plumbing\" to\n\"non-plumbing\" (I do not strongly care either way, so I'd take the\npatch without complaint about that particular change). It would\nhave been nicer to mention this change in the proposed commit log\nmessage, though, but that is minor.\n\n> + Note that if an object claims to be encoded in `X`\n> + and we are outputting in `X`, we shall output the object\n> verbatim; this means that invalid sequences in the original\n> commit may be copied to the output.\n\nI probably wouldn't have noticed this if a new manual page used\n\"shall\" consistently, but since the original deliberately used\n\"will\" and the patch changes it to \"shall\", I have to ask: why?\n\nI think our end-user facing manual pages tend to avoid the latter.\nWe do use \"shall\" in the RFC2119/BCP14 sense on the technical side\nof our documentation where we give requirements to the third-party\nimplementations so that they can interoperate with us, but this is\nnot such a description.\n\nThanks.\n\n\n[References]\n\n*1* https://pubs.opengroup.org/onlinepubs/9699919799/utilities/iconv.html \n*2* https://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv_open.html\n"},{"id":"433941","messageId":"YSko++W+QHiDX81X@coredump.intra.peff.net","threadId":"56385","inReplyTo":"xmqq5yvqbz0j.fsf@gitster.g","subject":"Re: [PATCH v2] pretty-options.txt: describe supported encoding","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-08-27T18:03:39Z","receivedAt":"2021-08-27T18:03:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Aug 27, 2021 at 10:03:56AM -0700, Junio C Hamano wrote:\n\n> > + The encoding must be a system encoding supported by iconv(1),\n> > + otherwise this option will be ignored.\n> > + POSIX character maps used by iconv(1p) are not supported.\n> \n> This paragraph is a bit hard to grok.\n> \n> I think it is saying that the \"-f frommap -t tomap\" form in [*1*]\n> that can use arbitrary character set description file is not\n> supported, but \"-f fromcode -t tocode\" form, which also is what\n> iconv_open() takes [*2*], is supported. Am I reading it correctly?\n> \n> Is there an easier-to-read way to explain the distinction to our\n> average reader?\n> \n> What I am getting at is this. Imagine average users who need to see\n> their commits recoded to iso-8859-2. They see \"git log\" has\n> \"--encoding=\" option, read the above paragraph and wonder\n> if they are on the supported side or unsupported side of the above\n> paragraph. I want to make it easy for them to stop wondering.\n> \n> For that purpose, \"iconv(1) vs iconv(1p)\" would not help them very\n> much, especially considering that not all Git users are UNIX users\n> (they probably do not even know what (1) and (1p) means).\n\nI likewise found the mention of character maps confusing. If we were to\nrefer to anything, it would be iconv(3) or iconv_open(3). But really,\nall of the discussion that led to this patch seemed to be about the\ndistinction between \"character set conversion\" (or \"character encoding\",\nor \"codeset conversion\", all terms used by the POSIX pages) and the\nsyntactic encoding of HTML.\n\nIs there any version of iconv that would convert \"<\" to \"<\"?\n\nI guess that _conceptually_ one could think of that as a multi-byte\ncharacter conversion, but it seems to me that it is generally considered\na layer above (after all, the original \"<\" and characters in the HTML\nentity have to be in some character encoding; generally ASCII, but I\nthink you could have UTF-16 HTML, too).\n\nWhat I'm getting it is that maybe we just need to use a less generic\nword than \"encoding\". Perhaps just s/encoding/character &/ or something?\nAnd maybe add something like:\n\n Conversions are done using the system iconv(3) function. The set of\n available encodings will depend on your system.\n\nYou _can_ use \"iconv -l\" to get such a list on many systems, but it is\nnot even necessarily the same list.\n\nI also wonder if other mentions of encoding would want to use the same\nterm (e.g., gitattributes working-tree-encoding), and of course\ni18n.commitEncoding (though peeking at the latter, it seems to already\nsay \"Character encoding\", so maybe that is sufficient).\n\n-Peff\n"},{"id":"433978","messageId":"1659256.Cefek1SpHl@localhost.localdomain","threadId":"56385","inReplyTo":"xmqq5yvqbz0j.fsf@gitster.g","subject":"Re: [PATCH v2] pretty-options.txt: describe supported encoding","fromName":"Krzysztof Żelechowski","fromEmail":"giecrilj@stegny.2a.pl","sentAt":"2021-08-27T23:20:53Z","receivedAt":"2021-08-27T23:21:02Z","isPatch":true,"sender":{"key":"giecrilj@stegny.2a.pl","avatar":"https://gravatar.com/avatar/419ea28e22b430cae87994078ab5203f95eb2146ac1fe20739b0521530b26a25?d=mp&s=160"},"body":"Dnia piątek, 27 sierpnia 2021 19:03:56 CEST Junio C Hamano pisze:\n> > + The encoding must be a system encoding supported by iconv(1),\n> > + otherwise this option will be ignored.\n> > + POSIX character maps used by iconv(1p) are not supported.\n> \n> This paragraph is a bit hard to grok.\n> \n> I think it is saying that the \"-f frommap -t tomap\" form in [*1*]\n> that can use arbitrary character set description file is not\n> supported, but \"-f fromcode -t tocode\" form, which also is what\n> iconv_open() takes [*2*], is supported. Am I reading it correctly?\n\nYes\n\n> \n> Is there an easier-to-read way to explain the distinction to our\n> average reader?\n\nIt is not our job to explain what POSIX character maps are. The takeaway is \nthey are unsupported; if you do not know what they are, why should you bother?\n\n> \n> What I am getting at is this. Imagine average users who need to see\n> their commits recoded to iso-8859-2. They see \"git log\" has\n> \"--encoding=\" option, read the above paragraph and wonder\n> if they are on the supported side or unsupported side of the above\n> paragraph. I want to make it easy for them to stop wondering.\n> \n> For that purpose, \"iconv(1) vs iconv(1p)\" would not help them very\n> much, especially considering that not all Git users are UNIX users\n> (they probably do not even know what (1) and (1p) means).\n\nI am sorry, as a UNIX user I have no idea what iconv, being part of the GNU C \nlibrary, means and how it works on a non-UNIX system that does not contain \none. If you know, could you enlighten us please?\n\n> I think our end-user facing manual pages tend to avoid the latter.\n> We do use \"shall\" in the RFC2119/BCP14 sense on the technical side\n> of our documentation where we give requirements to the third-party\n> implementations so that they can interoperate with us, but this is\n> not such a description.\n> \n> Thanks.\n\nI shall revert it after we have come to an agreement about the POSIX stuff.\n\nBR,\nChris\n\n\n\n\n"}]}