{"thread":{"id":"50604","subject":"Need multibyte advice - Shift-JIS","startedAt":"2019-02-27T13:04:19Z","lastAt":"2019-02-27T19:54:08Z","messageCount":18,"participants":["Randall S. Becker","Michal Suchánek","Johannes Sixt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"370303","messageId":"001e01d4ce9c$eff07400$cfd15c00$@nexbridge.com","threadId":"50604","inReplyTo":null,"subject":"Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T13:04:08Z","receivedAt":"2019-02-27T13:04:19Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"Hi Git Team,\n\nI have to admit being perplexed by this one. I have been asked to support\nthe Shift-JIS character set in file contents, comments, and logs, for a\npartner of mine. I know there are a few ways to do this, but I'm looking for\nthe official non-hacky way to do this. This is CLI only, and our pager,\nless, does not support multi-byte, so I'm looking for options there also.\n\nI normally do all of my UTF-16 work from a workstation via ECLIPSE, with\nUTF-8 comments, so never really have encountered this as an issue. Although\nour UTF-16 HTML files look pretty ugly in a diff.\n\nThe platform (NonStop) does not have a lot of UTF-16 tooling by default\n(less does not support it), so I may have to write stuff, which is no issue.\nWe are on 2.21.0 officially as of yesterday.\n\nKind Regards,\nRandall\n\n-- Brief whoami:\nNonStop developer since approximately 211288444200000000\nUNIX developer since approximately 421664400\n-- In my real life, I talk too much.\n\n\n\n\n\n"},{"id":"370307","messageId":"20190227150836.495f1692@kitsune.suse.cz","threadId":"50604","inReplyTo":"001e01d4ce9c$eff07400$cfd15c00$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T14:08:36Z","receivedAt":"2019-02-27T14:08:39Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 08:04:08 -0500\n\"Randall S. Becker\" wrote:\n\n> Hi Git Team,\n> \n> I have to admit being perplexed by this one. I have been asked to support\n> the Shift-JIS character set in file contents, comments, and logs, for a\n> partner of mine. I know there are a few ways to do this, but I'm looking for\n> the official non-hacky way to do this. This is CLI only, and our pager,\n> less, does not support multi-byte, so I'm looking for options there also.\n\nSJIS is about as much multibyte as UTF-8.\n\nWhy do you think less does not support it?\n\nLast time I looked there was SJIS locale for libc so it is only matter\nof generating the correct locales and using them. Of course, if you are\nrunning in UTF-8 SJIS will look like garbage. \n\nHTH\n\nMichal\n"},{"id":"370315","messageId":"000001d4ceb4$b930df50$2b929df0$@nexbridge.com","threadId":"50604","inReplyTo":"20190227150836.495f1692@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T15:54:23Z","receivedAt":"2019-02-27T15:54:36Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 9:09, Michal Suchánek wrote:\n> On Wed, 27 Feb 2019 08:04:08 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > Hi Git Team,\n> >\n> > I have to admit being perplexed by this one. I have been asked to\n> > support the Shift-JIS character set in file contents, comments, and\n> > logs, for a partner of mine. I know there are a few ways to do this,\n> > but I'm looking for the official non-hacky way to do this. This is CLI\n> > only, and our pager, less, does not support multi-byte, so I'm looking\nfor\n> options there also.\n> \n> SJIS is about as much multibyte as UTF-8.\n> \n> Why do you think less does not support it?\n> \n> Last time I looked there was SJIS locale for libc so it is only matter of\n> generating the correct locales and using them. Of course, if you are\nrunning\n> in UTF-8 SJIS will look like garbage.\n\nSadly, I did not personally build less on this platform, and the libc used\ndid not include UTF-16, on the platform vendor supplied less. cat works\nfine, but the usual LESSCHARSET=utf-16 is unsupported, so I am looking for\nan alternative. THAT is why I think less does not support it. Sorry, I\nshould have made that more clear.\n\ncat works fine, so if I set GIT_PAGER=cat, I can at least see the diffs\ncleanly in SJIS, but this partner wants a pager that is usable.\n\n"},{"id":"370318","messageId":"20190227171103.4cbc735a@kitsune.suse.cz","threadId":"50604","inReplyTo":"000001d4ceb4$b930df50$2b929df0$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T16:11:03Z","receivedAt":"2019-02-27T16:11:06Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 10:54:23 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 9:09, Michal Suchánek wrote:\n> > On Wed, 27 Feb 2019 08:04:08 -0500\n> > \"Randall S. Becker\" wrote:\n> > \n> > > Hi Git Team,\n> > >\n> > > I have to admit being perplexed by this one. I have been asked to\n> > > support the Shift-JIS character set in file contents, comments, and\n> > > logs, for a partner of mine. I know there are a few ways to do this,\n> > > but I'm looking for the official non-hacky way to do this. This is CLI\n> > > only, and our pager, less, does not support multi-byte, so I'm looking \n> for\n> > options there also.\n> > \n> > SJIS is about as much multibyte as UTF-8.\n> > \n> > Why do you think less does not support it?\n> > \n> > Last time I looked there was SJIS locale for libc so it is only matter of\n> > generating the correct locales and using them. Of course, if you are \n> running\n> > in UTF-8 SJIS will look like garbage. \n> \n> Sadly, I did not personally build less on this platform, and the libc used\n> did not include UTF-16, on the platform vendor supplied less. cat works\n> fine, but the usual LESSCHARSET=utf-16 is unsupported, so I am looking for\n> an alternative. THAT is why I think less does not support it. Sorry, I\n> should have made that more clear.\n> \n> cat works fine, so if I set GIT_PAGER=cat, I can at least see the diffs\n> cleanly in SJIS, but this partner wants a pager that is usable.\n> \n\nSo you want to use SJIS because UTF-16 is not supported. So what is the\nproblem with SJIS (or UTF-8 for that matter)?\n\nThanks\n\nMichal\n"},{"id":"370319","messageId":"000a01d4ceb8$3cd19720$b674c560$@nexbridge.com","threadId":"50604","inReplyTo":"20190227171103.4cbc735a@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T16:19:33Z","receivedAt":"2019-02-27T16:19:44Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 11:11, Michal Suchánek wrote:\n> On Wed, 27 Feb 2019 10:54:23 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > On February 27, 2019 9:09, Michal Suchánek wrote:\n> > > On Wed, 27 Feb 2019 08:04:08 -0500\n> > > \"Randall S. Becker\" wrote:\n> > >\n> > > > Hi Git Team,\n> > > >\n> > > > I have to admit being perplexed by this one. I have been asked to\n> > > > support the Shift-JIS character set in file contents, comments,\n> > > > and logs, for a partner of mine. I know there are a few ways to do\n> > > > this, but I'm looking for the official non-hacky way to do this.\n> > > > This is CLI only, and our pager, less, does not support\n> > > > multi-byte, so I'm looking\n> > for\n> > > options there also.\n> > >\n> > > SJIS is about as much multibyte as UTF-8.\n> > >\n> > > Why do you think less does not support it?\n> > >\n> > > Last time I looked there was SJIS locale for libc so it is only\n> > > matter of generating the correct locales and using them. Of course,\n> > > if you are\n> > running\n> > > in UTF-8 SJIS will look like garbage.\n> >\n> > Sadly, I did not personally build less on this platform, and the libc\n> > used did not include UTF-16, on the platform vendor supplied less. cat\n> > works fine, but the usual LESSCHARSET=utf-16 is unsupported, so I am\n> > looking for an alternative. THAT is why I think less does not support\n> > it. Sorry, I should have made that more clear.\n> >\n> > cat works fine, so if I set GIT_PAGER=cat, I can at least see the\n> > diffs cleanly in SJIS, but this partner wants a pager that is usable.\n> >\n> \n> So you want to use SJIS because UTF-16 is not supported. So what is the\n> problem with SJIS (or UTF-8 for that matter)?\n\nThe partner I am working with is using multi-byte SJIS, which is also not supported by this incarnation of less. As a result, UTF-8 does not work either in this situation. The content is definitely multi-byte. I know this was fixed in RedHat's Less in 2016, but did not make this platform.\n\n"},{"id":"370320","messageId":"20190227172841.3a74fa60@kitsune.suse.cz","threadId":"50604","inReplyTo":"000a01d4ceb8$3cd19720$b674c560$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T16:28:41Z","receivedAt":"2019-02-27T16:28:44Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 11:19:33 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 11:11, Michal Suchánek wrote:\n> > On Wed, 27 Feb 2019 10:54:23 -0500\n> > \"Randall S. Becker\" wrote:\n> > \n> > > On February 27, 2019 9:09, Michal Suchánek wrote: \n> > > > On Wed, 27 Feb 2019 08:04:08 -0500\n> > > > \"Randall S. Becker\" wrote:\n> > > > \n> > > > > Hi Git Team,\n> > > > >\n> > > > > I have to admit being perplexed by this one. I have been asked to\n> > > > > support the Shift-JIS character set in file contents, comments,\n> > > > > and logs, for a partner of mine. I know there are a few ways to do\n> > > > > this, but I'm looking for the official non-hacky way to do this.\n> > > > > This is CLI only, and our pager, less, does not support\n> > > > > multi-byte, so I'm looking \n> > > for \n> > > > options there also.\n> > > >\n> > > > SJIS is about as much multibyte as UTF-8.\n> > > >\n> > > > Why do you think less does not support it?\n> > > >\n> > > > Last time I looked there was SJIS locale for libc so it is only\n> > > > matter of generating the correct locales and using them. Of course,\n> > > > if you are \n> > > running \n> > > > in UTF-8 SJIS will look like garbage. \n> > >\n> > > Sadly, I did not personally build less on this platform, and the libc\n> > > used did not include UTF-16, on the platform vendor supplied less. cat\n> > > works fine, but the usual LESSCHARSET=utf-16 is unsupported, so I am\n> > > looking for an alternative. THAT is why I think less does not support\n> > > it. Sorry, I should have made that more clear.\n> > >\n> > > cat works fine, so if I set GIT_PAGER=cat, I can at least see the\n> > > diffs cleanly in SJIS, but this partner wants a pager that is usable.\n> > > \n> > \n> > So you want to use SJIS because UTF-16 is not supported. So what is the\n> > problem with SJIS (or UTF-8 for that matter)? \n> \n> The partner I am working with is using multi-byte SJIS, which is also not supported by this incarnation of less. As a result, UTF-8 does not work either in this situation. The content is definitely multi-byte. I know this was fixed in RedHat's Less in 2016, but did not make this platform.\n> \n\nBoth UTF-8 and SJIS is multibyte and both is supported by less\nin general. If your particular less cannot support it then it is broken\nand you should fix it or get it fixed.\n\nHTH\n\nMichal\n"},{"id":"370321","messageId":"000b01d4ceba$39cfddf0$ad6f99d0$@nexbridge.com","threadId":"50604","inReplyTo":"20190227172841.3a74fa60@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T16:33:47Z","receivedAt":"2019-02-27T16:33:58Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 11:29 Michal Suchánek wrote:\n> On Wed, 27 Feb 2019 11:19:33 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > On February 27, 2019 11:11, Michal Suchánek wrote:\n> > > On Wed, 27 Feb 2019 10:54:23 -0500\n> > > \"Randall S. Becker\" wrote:\n> > >\n> > > > On February 27, 2019 9:09, Michal Suchánek wrote:\n> > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > wrote:\n> > > > >\n> > > > > > Hi Git Team,\n> > > > > >\n> > > > > > I have to admit being perplexed by this one. I have been asked\n> > > > > > to support the Shift-JIS character set in file contents,\n> > > > > > comments, and logs, for a partner of mine. I know there are a\n> > > > > > few ways to do this, but I'm looking for the official non-hacky way\n> to do this.\n> > > > > > This is CLI only, and our pager, less, does not support\n> > > > > > multi-byte, so I'm looking\n> > > > for\n> > > > > options there also.\n> > > > >\n> > > > > SJIS is about as much multibyte as UTF-8.\n> > > > >\n> > > > > Why do you think less does not support it?\n> > > > >\n> > > > > Last time I looked there was SJIS locale for libc so it is only\n> > > > > matter of generating the correct locales and using them. Of\n> > > > > course, if you are\n> > > > running\n> > > > > in UTF-8 SJIS will look like garbage.\n> > > >\n> > > > Sadly, I did not personally build less on this platform, and the\n> > > > libc used did not include UTF-16, on the platform vendor supplied\n> > > > less. cat works fine, but the usual LESSCHARSET=utf-16 is\n> > > > unsupported, so I am looking for an alternative. THAT is why I\n> > > > think less does not support it. Sorry, I should have made that more\n> clear.\n> > > >\n> > > > cat works fine, so if I set GIT_PAGER=cat, I can at least see the\n> > > > diffs cleanly in SJIS, but this partner wants a pager that is usable.\n> > > >\n> > >\n> > > So you want to use SJIS because UTF-16 is not supported. So what is\n> > > the problem with SJIS (or UTF-8 for that matter)?\n> >\n> > The partner I am working with is using multi-byte SJIS, which is also not\n> supported by this incarnation of less. As a result, UTF-8 does not work either\n> in this situation. The content is definitely multi-byte. I know this was fixed in\n> RedHat's Less in 2016, but did not make this platform.\n> >\n> \n> Both UTF-8 and SJIS is multibyte and both is supported by less in general. If\n> your particular less cannot support it then it is broken and you should fix it or\n> get it fixed.\n\nTo be more specific, the implementation of less' UTF-8 on this platform will present the data as unusable junk as expected. SJIS is multi-byte, but is not one of the allowed encodings in less. I am not empowered to \"get it fixed\". Thanks for your advice.\n\n"},{"id":"370324","messageId":"20190227175135.4392e9d7@kitsune.suse.cz","threadId":"50604","inReplyTo":"000b01d4ceba$39cfddf0$ad6f99d0$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T16:51:35Z","receivedAt":"2019-02-27T16:51:39Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 11:33:47 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 11:29 Michal Suchánek wrote:\n> > On Wed, 27 Feb 2019 11:19:33 -0500\n> > \"Randall S. Becker\" wrote:\n> > \n> > > On February 27, 2019 11:11, Michal Suchánek wrote: \n> > > > On Wed, 27 Feb 2019 10:54:23 -0500\n> > > > \"Randall S. Becker\" wrote:\n> > > > \n> > > > > On February 27, 2019 9:09, Michal Suchánek wrote: \n> > > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > > wrote:\n> > > > > > \n> > > > > > > Hi Git Team,\n> > > > > > >\n> > > > > > > I have to admit being perplexed by this one. I have been asked\n> > > > > > > to support the Shift-JIS character set in file contents,\n> > > > > > > comments, and logs, for a partner of mine. I know there are a\n> > > > > > > few ways to do this, but I'm looking for the official non-hacky way \n> > to do this. \n> > > > > > > This is CLI only, and our pager, less, does not support\n> > > > > > > multi-byte, so I'm looking \n> > > > > for \n> > > > > > options there also.\n> > > > > >\n> > > > > > SJIS is about as much multibyte as UTF-8.\n> > > > > >\n> > > > > > Why do you think less does not support it?\n> > > > > >\n> > > > > > Last time I looked there was SJIS locale for libc so it is only\n> > > > > > matter of generating the correct locales and using them. Of\n> > > > > > course, if you are \n> > > > > running \n> > > > > > in UTF-8 SJIS will look like garbage. \n> > > > >\n> > > > > Sadly, I did not personally build less on this platform, and the\n> > > > > libc used did not include UTF-16, on the platform vendor supplied\n> > > > > less. cat works fine, but the usual LESSCHARSET=utf-16 is\n> > > > > unsupported, so I am looking for an alternative. THAT is why I\n> > > > > think less does not support it. Sorry, I should have made that more \n> > clear. \n> > > > >\n> > > > > cat works fine, so if I set GIT_PAGER=cat, I can at least see the\n> > > > > diffs cleanly in SJIS, but this partner wants a pager that is usable.\n> > > > > \n> > > >\n> > > > So you want to use SJIS because UTF-16 is not supported. So what is\n> > > > the problem with SJIS (or UTF-8 for that matter)? \n> > >\n> > > The partner I am working with is using multi-byte SJIS, which is also not \n> > supported by this incarnation of less. As a result, UTF-8 does not work either\n> > in this situation. The content is definitely multi-byte. I know this was fixed in\n> > RedHat's Less in 2016, but did not make this platform. \n> > > \n> > \n> > Both UTF-8 and SJIS is multibyte and both is supported by less in general. If\n> > your particular less cannot support it then it is broken and you should fix it or\n> > get it fixed. \n> \n> To be more specific, the implementation of less' UTF-8 on this platform will present the data as unusable junk as expected. SJIS is multi-byte, but is not one of the allowed encodings in less. I am not empowered to \"get it fixed\". Thanks for your advice.\n> \n\nHow is this 'allowed encodings in less' defined?\n\nMichal\n"},{"id":"370325","messageId":"001101d4cebe$7119a080$534ce180$@nexbridge.com","threadId":"50604","inReplyTo":"20190227175135.4392e9d7@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T17:03:58Z","receivedAt":"2019-02-27T17:04:10Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 11:52, Michal Suchánek wrote:\n> To: Randall S. Becker \n> Cc: git@vger.kernel.org\n> Subject: Re: Need multibyte advice - Shift-JIS\n> \n> On Wed, 27 Feb 2019 11:33:47 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > On February 27, 2019 11:29 Michal Suchánek wrote:\n> > > On Wed, 27 Feb 2019 11:19:33 -0500\n> > > \"Randall S. Becker\" wrote:\n> > >\n> > > > On February 27, 2019 11:11, Michal Suchánek wrote:\n> > > > > On Wed, 27 Feb 2019 10:54:23 -0500 \"Randall S. Becker\"\n> > > > > wrote:\n> > > > >\n> > > > > > On February 27, 2019 9:09, Michal Suchánek wrote:\n> > > > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > > > wrote:\n> > > > > > >\n> > > > > > > > Hi Git Team,\n> > > > > > > >\n> > > > > > > > I have to admit being perplexed by this one. I have been\n> > > > > > > > asked to support the Shift-JIS character set in file\n> > > > > > > > contents, comments, and logs, for a partner of mine. I\n> > > > > > > > know there are a few ways to do this, but I'm looking for\n> > > > > > > > the official non-hacky way\n> > > to do this.\n> > > > > > > > This is CLI only, and our pager, less, does not support\n> > > > > > > > multi-byte, so I'm looking\n> > > > > > for\n> > > > > > > options there also.\n> > > > > > >\n> > > > > > > SJIS is about as much multibyte as UTF-8.\n> > > > > > >\n> > > > > > > Why do you think less does not support it?\n> > > > > > >\n> > > > > > > Last time I looked there was SJIS locale for libc so it is\n> > > > > > > only matter of generating the correct locales and using\n> > > > > > > them. Of course, if you are\n> > > > > > running\n> > > > > > > in UTF-8 SJIS will look like garbage.\n> > > > > >\n> > > > > > Sadly, I did not personally build less on this platform, and\n> > > > > > the libc used did not include UTF-16, on the platform vendor\n> > > > > > supplied less. cat works fine, but the usual\n> > > > > > LESSCHARSET=utf-16 is unsupported, so I am looking for an\n> > > > > > alternative. THAT is why I think less does not support it.\n> > > > > > Sorry, I should have made that more\n> > > clear.\n> > > > > >\n> > > > > > cat works fine, so if I set GIT_PAGER=cat, I can at least see\n> > > > > > the diffs cleanly in SJIS, but this partner wants a pager that is usable.\n> > > > > >\n> > > > >\n> > > > > So you want to use SJIS because UTF-16 is not supported. So what\n> > > > > is the problem with SJIS (or UTF-8 for that matter)?\n> > > >\n> > > > The partner I am working with is using multi-byte SJIS, which is\n> > > > also not\n> > > supported by this incarnation of less. As a result, UTF-8 does not\n> > > work either in this situation. The content is definitely multi-byte.\n> > > I know this was fixed in RedHat's Less in 2016, but did not make this\n> platform.\n> > > >\n> > >\n> > > Both UTF-8 and SJIS is multibyte and both is supported by less in\n> > > general. If your particular less cannot support it then it is broken\n> > > and you should fix it or get it fixed.\n> >\n> > To be more specific, the implementation of less' UTF-8 on this platform will\n> present the data as unusable junk as expected. SJIS is multi-byte, but is not\n> one of the allowed encodings in less. I am not empowered to \"get it fixed\".\n> Thanks for your advice.\n> >\n> \n> How is this 'allowed encodings in less' defined?\n\nWhen you run less with LESSCHARSET=encoding, if the encoding is not known, you get the error:\ninvalid charset name\n\nDoing the due diligence, I actually read the man page on the platform before asking the question, which listed the following as the only allowed encodings: ascii, iso8859, latin1, latin9, dos, IBM-1047, koi8-r, next, utf-8, windows. The utf-8 variant does not know how to display its multi-byte forms in SJIS, as with other platforms. Does that make sense now?\n\n"},{"id":"370326","messageId":"20190227181458.4f019d91@kitsune.suse.cz","threadId":"50604","inReplyTo":"001101d4cebe$7119a080$534ce180$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T17:14:58Z","receivedAt":"2019-02-27T17:15:02Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 12:03:58 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 11:52, Michal Suchánek wrote:\n> > To: Randall S. Becker \n> > Cc: git@vger.kernel.org\n> > Subject: Re: Need multibyte advice - Shift-JIS\n> > \n> > On Wed, 27 Feb 2019 11:33:47 -0500\n> > \"Randall S. Becker\" wrote:\n> > \n> > > On February 27, 2019 11:29 Michal Suchánek wrote: \n> > > > On Wed, 27 Feb 2019 11:19:33 -0500\n> > > > \"Randall S. Becker\" wrote:\n> > > > \n> > > > > On February 27, 2019 11:11, Michal Suchánek wrote: \n> > > > > > On Wed, 27 Feb 2019 10:54:23 -0500 \"Randall S. Becker\"\n> > > > > > wrote:\n> > > > > > \n> > > > > > > On February 27, 2019 9:09, Michal Suchánek wrote: \n> > > > > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > > > > wrote:\n> > > > > > > > \n> > > > > > > > > Hi Git Team,\n> > > > > > > > >\n> > > > > > > > > I have to admit being perplexed by this one. I have been\n> > > > > > > > > asked to support the Shift-JIS character set in file\n> > > > > > > > > contents, comments, and logs, for a partner of mine. I\n> > > > > > > > > know there are a few ways to do this, but I'm looking for\n> > > > > > > > > the official non-hacky way \n> > > > to do this. \n> > > > > > > > > This is CLI only, and our pager, less, does not support\n> > > > > > > > > multi-byte, so I'm looking \n> > > > > > > for \n> > > > > > > > options there also.\n> > > > > > > >\n> > > > > > > > SJIS is about as much multibyte as UTF-8.\n> > > > > > > >\n> > > > > > > > Why do you think less does not support it?\n> > > > > > > >\n> > > > > > > > Last time I looked there was SJIS locale for libc so it is\n> > > > > > > > only matter of generating the correct locales and using\n> > > > > > > > them. Of course, if you are \n> > > > > > > running \n> > > > > > > > in UTF-8 SJIS will look like garbage. \n> > > > > > >\n> > > > > > > Sadly, I did not personally build less on this platform, and\n> > > > > > > the libc used did not include UTF-16, on the platform vendor\n> > > > > > > supplied less. cat works fine, but the usual\n> > > > > > > LESSCHARSET=utf-16 is unsupported, so I am looking for an\n> > > > > > > alternative. THAT is why I think less does not support it.\n> > > > > > > Sorry, I should have made that more \n> > > > clear. \n> > > > > > >\n> > > > > > > cat works fine, so if I set GIT_PAGER=cat, I can at least see\n> > > > > > > the diffs cleanly in SJIS, but this partner wants a pager that is usable.\n> > > > > > > \n> > > > > >\n> > > > > > So you want to use SJIS because UTF-16 is not supported. So what\n> > > > > > is the problem with SJIS (or UTF-8 for that matter)? \n> > > > >\n> > > > > The partner I am working with is using multi-byte SJIS, which is\n> > > > > also not \n> > > > supported by this incarnation of less. As a result, UTF-8 does not\n> > > > work either in this situation. The content is definitely multi-byte.\n> > > > I know this was fixed in RedHat's Less in 2016, but did not make this \n> > platform. \n> > > > > \n> > > >\n> > > > Both UTF-8 and SJIS is multibyte and both is supported by less in\n> > > > general. If your particular less cannot support it then it is broken\n> > > > and you should fix it or get it fixed. \n> > >\n> > > To be more specific, the implementation of less' UTF-8 on this platform will \n> > present the data as unusable junk as expected. SJIS is multi-byte, but is not\n> > one of the allowed encodings in less. I am not empowered to \"get it fixed\".\n> > Thanks for your advice. \n> > > \n> > \n> > How is this 'allowed encodings in less' defined? \n> \n> When you run less with LESSCHARSET=encoding, if the encoding is not known, you get the error:\n> invalid charset name\n> \n> Doing the due diligence, I actually read the man page on the platform before asking the question, which listed the following as the only allowed encodings: ascii, iso8859, latin1, latin9, dos, IBM-1047, koi8-r, next, utf-8, windows. The utf-8 variant does not know how to display its multi-byte forms in SJIS, as with other platforms. Does that make sense now?\n> \n\nDoes the said man page also mention LESSCHARDEF or LESSOPEN?\n\nMichal\n"},{"id":"370329","messageId":"001b01d4cec3$364149e0$a2c3dda0$@nexbridge.com","threadId":"50604","inReplyTo":"20190227181458.4f019d91@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T17:38:06Z","receivedAt":"2019-02-27T17:38:18Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 12:15, Michal Suchánek wrote:\n> To: Randall S. Becker \n> Cc: git@vger.kernel.org\n> Subject: Re: Need multibyte advice - Shift-JIS\n> \n> On Wed, 27 Feb 2019 12:03:58 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > On February 27, 2019 11:52, Michal Suchánek wrote:\n> > > To: Randall S. Becker \n> > > Cc: git@vger.kernel.org\n> > > Subject: Re: Need multibyte advice - Shift-JIS\n> > >\n> > > On Wed, 27 Feb 2019 11:33:47 -0500\n> > > \"Randall S. Becker\" wrote:\n> > >\n> > > > On February 27, 2019 11:29 Michal Suchánek wrote:\n> > > > > On Wed, 27 Feb 2019 11:19:33 -0500 \"Randall S. Becker\"\n> > > > > wrote:\n> > > > >\n> > > > > > On February 27, 2019 11:11, Michal Suchánek wrote:\n> > > > > > > On Wed, 27 Feb 2019 10:54:23 -0500 \"Randall S. Becker\"\n> > > > > > > wrote:\n> > > > > > >\n> > > > > > > > On February 27, 2019 9:09, Michal Suchánek wrote:\n> > > > > > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > > > > > wrote:\n> > > > > > > > >\n> > > > > > > > > > Hi Git Team,\n> > > > > > > > > >\n> > > > > > > > > > I have to admit being perplexed by this one. I have\n> > > > > > > > > > been asked to support the Shift-JIS character set in\n> > > > > > > > > > file contents, comments, and logs, for a partner of\n> > > > > > > > > > mine. I know there are a few ways to do this, but I'm\n> > > > > > > > > > looking for the official non-hacky way\n> > > > > to do this.\n> > > > > > > > > > This is CLI only, and our pager, less, does not\n> > > > > > > > > > support multi-byte, so I'm looking\n> > > > > > > > for\n> > > > > > > > > options there also.\n> > > > > > > > >\n> > > > > > > > > SJIS is about as much multibyte as UTF-8.\n> > > > > > > > >\n> > > > > > > > > Why do you think less does not support it?\n> > > > > > > > >\n> > > > > > > > > Last time I looked there was SJIS locale for libc so it\n> > > > > > > > > is only matter of generating the correct locales and\n> > > > > > > > > using them. Of course, if you are\n> > > > > > > > running\n> > > > > > > > > in UTF-8 SJIS will look like garbage.\n> > > > > > > >\n> > > > > > > > Sadly, I did not personally build less on this platform,\n> > > > > > > > and the libc used did not include UTF-16, on the platform\n> > > > > > > > vendor supplied less. cat works fine, but the usual\n> > > > > > > > LESSCHARSET=utf-16 is unsupported, so I am looking for an\n> > > > > > > > alternative. THAT is why I think less does not support it.\n> > > > > > > > Sorry, I should have made that more\n> > > > > clear.\n> > > > > > > >\n> > > > > > > > cat works fine, so if I set GIT_PAGER=cat, I can at least\n> > > > > > > > see the diffs cleanly in SJIS, but this partner wants a pager that is\n> usable.\n> > > > > > > >\n> > > > > > >\n> > > > > > > So you want to use SJIS because UTF-16 is not supported. So\n> > > > > > > what is the problem with SJIS (or UTF-8 for that matter)?\n> > > > > >\n> > > > > > The partner I am working with is using multi-byte SJIS, which\n> > > > > > is also not\n> > > > > supported by this incarnation of less. As a result, UTF-8 does\n> > > > > not work either in this situation. The content is definitely multi-byte.\n> > > > > I know this was fixed in RedHat's Less in 2016, but did not make\n> > > > > this\n> > > platform.\n> > > > > >\n> > > > >\n> > > > > Both UTF-8 and SJIS is multibyte and both is supported by less\n> > > > > in general. If your particular less cannot support it then it is\n> > > > > broken and you should fix it or get it fixed.\n> > > >\n> > > > To be more specific, the implementation of less' UTF-8 on this\n> > > > platform will\n> > > present the data as unusable junk as expected. SJIS is multi-byte,\n> > > but is not one of the allowed encodings in less. I am not empowered to\n> \"get it fixed\".\n> > > Thanks for your advice.\n> > > >\n> > >\n> > > How is this 'allowed encodings in less' defined?\n> >\n> > When you run less with LESSCHARSET=encoding, if the encoding is not\n> known, you get the error:\n> > invalid charset name\n> >\n> > Doing the due diligence, I actually read the man page on the platform\n> before asking the question, which listed the following as the only allowed\n> encodings: ascii, iso8859, latin1, latin9, dos, IBM-1047, koi8-r, next, utf-8,\n> windows. The utf-8 variant does not know how to display its multi-byte\n> forms in SJIS, as with other platforms. Does that make sense now?\n> >\n> \n> Does the said man page also mention LESSCHARDEF or LESSOPEN?\n\nOf course it does.\n\n"},{"id":"370331","messageId":"20190227185044.120c0ba7@kitsune.suse.cz","threadId":"50604","inReplyTo":"001b01d4cec3$364149e0$a2c3dda0$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T17:50:44Z","receivedAt":"2019-02-27T17:50:48Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 12:38:06 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 12:15, Michal Suchánek wrote:\n> > To: Randall S. Becker \n> > Cc: git@vger.kernel.org\n> > Subject: Re: Need multibyte advice - Shift-JIS\n> > \n> > On Wed, 27 Feb 2019 12:03:58 -0500\n> > \"Randall S. Becker\" wrote:\n> > \n> > > On February 27, 2019 11:52, Michal Suchánek wrote: \n> > > > To: Randall S. Becker \n> > > > Cc: git@vger.kernel.org\n> > > > Subject: Re: Need multibyte advice - Shift-JIS\n> > > >\n> > > > On Wed, 27 Feb 2019 11:33:47 -0500\n> > > > \"Randall S. Becker\" wrote:\n> > > > \n> > > > > On February 27, 2019 11:29 Michal Suchánek wrote: \n> > > > > > On Wed, 27 Feb 2019 11:19:33 -0500 \"Randall S. Becker\"\n> > > > > > wrote:\n> > > > > > \n> > > > > > > On February 27, 2019 11:11, Michal Suchánek wrote: \n> > > > > > > > On Wed, 27 Feb 2019 10:54:23 -0500 \"Randall S. Becker\"\n> > > > > > > > wrote:\n> > > > > > > > \n> > > > > > > > > On February 27, 2019 9:09, Michal Suchánek wrote: \n> > > > > > > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > > > > > > wrote:\n> > > > > > > > > > \n> > > > > > > > > > > Hi Git Team,\n> > > > > > > > > > >\n> > > > > > > > > > > I have to admit being perplexed by this one. I have\n> > > > > > > > > > > been asked to support the Shift-JIS character set in\n> > > > > > > > > > > file contents, comments, and logs, for a partner of\n> > > > > > > > > > > mine. I know there are a few ways to do this, but I'm\n> > > > > > > > > > > looking for the official non-hacky way \n> > > > > > to do this. \n> > > > > > > > > > > This is CLI only, and our pager, less, does not\n> > > > > > > > > > > support multi-byte, so I'm looking \n> > > > > > > > > for \n> > > > > > > > > > options there also.\n> > > > > > > > > >\n> > > > > > > > > > SJIS is about as much multibyte as UTF-8.\n> > > > > > > > > >\n> > > > > > > > > > Why do you think less does not support it?\n> > > > > > > > > >\n> > > > > > > > > > Last time I looked there was SJIS locale for libc so it\n> > > > > > > > > > is only matter of generating the correct locales and\n> > > > > > > > > > using them. Of course, if you are \n> > > > > > > > > running \n> > > > > > > > > > in UTF-8 SJIS will look like garbage. \n> > > > > > > > >\n> > > > > > > > > Sadly, I did not personally build less on this platform,\n> > > > > > > > > and the libc used did not include UTF-16, on the platform\n> > > > > > > > > vendor supplied less. cat works fine, but the usual\n> > > > > > > > > LESSCHARSET=utf-16 is unsupported, so I am looking for an\n> > > > > > > > > alternative. THAT is why I think less does not support it.\n> > > > > > > > > Sorry, I should have made that more \n> > > > > > clear. \n> > > > > > > > >\n> > > > > > > > > cat works fine, so if I set GIT_PAGER=cat, I can at least\n> > > > > > > > > see the diffs cleanly in SJIS, but this partner wants a pager that is \n> > usable. \n> > > > > > > > > \n> > > > > > > >\n> > > > > > > > So you want to use SJIS because UTF-16 is not supported. So\n> > > > > > > > what is the problem with SJIS (or UTF-8 for that matter)? \n> > > > > > >\n> > > > > > > The partner I am working with is using multi-byte SJIS, which\n> > > > > > > is also not \n> > > > > > supported by this incarnation of less. As a result, UTF-8 does\n> > > > > > not work either in this situation. The content is definitely multi-byte.\n> > > > > > I know this was fixed in RedHat's Less in 2016, but did not make\n> > > > > > this \n> > > > platform. \n> > > > > > > \n> > > > > >\n> > > > > > Both UTF-8 and SJIS is multibyte and both is supported by less\n> > > > > > in general. If your particular less cannot support it then it is\n> > > > > > broken and you should fix it or get it fixed. \n> > > > >\n> > > > > To be more specific, the implementation of less' UTF-8 on this\n> > > > > platform will \n> > > > present the data as unusable junk as expected. SJIS is multi-byte,\n> > > > but is not one of the allowed encodings in less. I am not empowered to \n> > \"get it fixed\". \n> > > > Thanks for your advice. \n> > > > > \n> > > >\n> > > > How is this 'allowed encodings in less' defined? \n> > >\n> > > When you run less with LESSCHARSET=encoding, if the encoding is not \n> > known, you get the error: \n> > > invalid charset name\n> > >\n> > > Doing the due diligence, I actually read the man page on the platform \n> > before asking the question, which listed the following as the only allowed\n> > encodings: ascii, iso8859, latin1, latin9, dos, IBM-1047, koi8-r, next, utf-8,\n> > windows. The utf-8 variant does not know how to display its multi-byte\n> > forms in SJIS, as with other platforms. Does that make sense now? \n> > > \n> > \n> > Does the said man page also mention LESSCHARDEF or LESSOPEN? \n> \n> Of course it does.\n> \n\nSo what's the problem with displaying SJIS or even UTF-16 in less,\nexactly?\n\nAlso if you really don't like less there is lv.\n\nMichal\n"},{"id":"370333","messageId":"002501d4cec6$2a5e5a20$7f1b0e60$@nexbridge.com","threadId":"50604","inReplyTo":"20190227185044.120c0ba7@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T17:59:15Z","receivedAt":"2019-02-27T17:59:27Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 12:51, Michal Suchánek wrote:\n> To: Randall S. Becker \n> Cc: git@vger.kernel.org\n> Subject: Re: Need multibyte advice - Shift-JIS\n> \n> On Wed, 27 Feb 2019 12:38:06 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > On February 27, 2019 12:15, Michal Suchánek wrote:\n> > > To: Randall S. Becker \n> > > Cc: git@vger.kernel.org\n> > > Subject: Re: Need multibyte advice - Shift-JIS\n> > >\n> > > On Wed, 27 Feb 2019 12:03:58 -0500\n> > > \"Randall S. Becker\" wrote:\n> > >\n> > > > On February 27, 2019 11:52, Michal Suchánek wrote:\n> > > > > To: Randall S. Becker \n> > > > > Cc: git@vger.kernel.org\n> > > > > Subject: Re: Need multibyte advice - Shift-JIS\n> > > > >\n> > > > > On Wed, 27 Feb 2019 11:33:47 -0500 \"Randall S. Becker\"\n> > > > > wrote:\n> > > > >\n> > > > > > On February 27, 2019 11:29 Michal Suchánek wrote:\n> > > > > > > On Wed, 27 Feb 2019 11:19:33 -0500 \"Randall S. Becker\"\n> > > > > > > wrote:\n> > > > > > >\n> > > > > > > > On February 27, 2019 11:11, Michal Suchánek wrote:\n> > > > > > > > > On Wed, 27 Feb 2019 10:54:23 -0500 \"Randall S. Becker\"\n> > > > > > > > > wrote:\n> > > > > > > > >\n> > > > > > > > > > On February 27, 2019 9:09, Michal Suchánek wrote:\n> > > > > > > > > > > On Wed, 27 Feb 2019 08:04:08 -0500 \"Randall S. Becker\"\n> > > > > > > > > > > wrote:\n> > > > > > > > > > >\n> > > > > > > > > > > > Hi Git Team,\n> > > > > > > > > > > >\n> > > > > > > > > > > > I have to admit being perplexed by this one. I\n> > > > > > > > > > > > have been asked to support the Shift-JIS character\n> > > > > > > > > > > > set in file contents, comments, and logs, for a\n> > > > > > > > > > > > partner of mine. I know there are a few ways to do\n> > > > > > > > > > > > this, but I'm looking for the official non-hacky\n> > > > > > > > > > > > way\n> > > > > > > to do this.\n> > > > > > > > > > > > This is CLI only, and our pager, less, does not\n> > > > > > > > > > > > support multi-byte, so I'm looking\n> > > > > > > > > > for\n> > > > > > > > > > > options there also.\n> > > > > > > > > > >\n> > > > > > > > > > > SJIS is about as much multibyte as UTF-8.\n> > > > > > > > > > >\n> > > > > > > > > > > Why do you think less does not support it?\n> > > > > > > > > > >\n> > > > > > > > > > > Last time I looked there was SJIS locale for libc so\n> > > > > > > > > > > it is only matter of generating the correct locales\n> > > > > > > > > > > and using them. Of course, if you are\n> > > > > > > > > > running\n> > > > > > > > > > > in UTF-8 SJIS will look like garbage.\n> > > > > > > > > >\n> > > > > > > > > > Sadly, I did not personally build less on this\n> > > > > > > > > > platform, and the libc used did not include UTF-16, on\n> > > > > > > > > > the platform vendor supplied less. cat works fine, but\n> > > > > > > > > > the usual\n> > > > > > > > > > LESSCHARSET=utf-16 is unsupported, so I am looking for\n> > > > > > > > > > an alternative. THAT is why I think less does not support it.\n> > > > > > > > > > Sorry, I should have made that more\n> > > > > > > clear.\n> > > > > > > > > >\n> > > > > > > > > > cat works fine, so if I set GIT_PAGER=cat, I can at\n> > > > > > > > > > least see the diffs cleanly in SJIS, but this partner\n> > > > > > > > > > wants a pager that is\n> > > usable.\n> > > > > > > > > >\n> > > > > > > > >\n> > > > > > > > > So you want to use SJIS because UTF-16 is not supported.\n> > > > > > > > > So what is the problem with SJIS (or UTF-8 for that matter)?\n> > > > > > > >\n> > > > > > > > The partner I am working with is using multi-byte SJIS,\n> > > > > > > > which is also not\n> > > > > > > supported by this incarnation of less. As a result, UTF-8\n> > > > > > > does not work either in this situation. The content is definitely\n> multi-byte.\n> > > > > > > I know this was fixed in RedHat's Less in 2016, but did not\n> > > > > > > make this\n> > > > > platform.\n> > > > > > > >\n> > > > > > >\n> > > > > > > Both UTF-8 and SJIS is multibyte and both is supported by\n> > > > > > > less in general. If your particular less cannot support it\n> > > > > > > then it is broken and you should fix it or get it fixed.\n> > > > > >\n> > > > > > To be more specific, the implementation of less' UTF-8 on this\n> > > > > > platform will\n> > > > > present the data as unusable junk as expected. SJIS is\n> > > > > multi-byte, but is not one of the allowed encodings in less. I\n> > > > > am not empowered to\n> > > \"get it fixed\".\n> > > > > Thanks for your advice.\n> > > > > >\n> > > > >\n> > > > > How is this 'allowed encodings in less' defined?\n> > > >\n> > > > When you run less with LESSCHARSET=encoding, if the encoding is\n> > > > not\n> > > known, you get the error:\n> > > > invalid charset name\n> > > >\n> > > > Doing the due diligence, I actually read the man page on the\n> > > > platform\n> > > before asking the question, which listed the following as the only\n> > > allowed\n> > > encodings: ascii, iso8859, latin1, latin9, dos, IBM-1047, koi8-r,\n> > > next, utf-8, windows. The utf-8 variant does not know how to display\n> > > its multi-byte forms in SJIS, as with other platforms. Does that make\n> sense now?\n> > > >\n> > >\n> > > Does the said man page also mention LESSCHARDEF or LESSOPEN?\n> >\n> > Of course it does.\n> >\n> \n> So what's the problem with displaying SJIS or even UTF-16 in less, exactly?\n> \n> Also if you really don't like less there is lv.\n\nI'm sorry if I was not clear about all this. NonStop is not a Linux platform. It is POSIX. Not all utilities are available and not all utilities have all capabilities. lv is not available for the platform. less considers the data binary and displays what usually is displayed when you try to use it for binary multibyte. You get @^@- and such. It does not present the data in the correct character set for the user.\n\nThis was only one part of my original question. I am searching elsewhere for support on pagers, because this really is not an appropriate discussion for the git group to focus on, do let's drop this, please, as not worth continuing. My original request was more about how to set up the file attributes, difference engine, and the rest of the git infrastructure. The partner I am working with is doing this with git hooks, which I am not really happy about. Let's prune this discussion as not worthy.\n\n"},{"id":"370335","messageId":"20190227191814.054c2f3c@kitsune.suse.cz","threadId":"50604","inReplyTo":"002501d4cec6$2a5e5a20$7f1b0e60$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T18:18:14Z","receivedAt":"2019-02-27T18:18:18Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 12:59:15 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 12:51, Michal Suchánek wrote:\n> > To: Randall S. Becker \n> > Cc: git@vger.kernel.org\n> > Subject: Re: Need multibyte advice - Shift-JIS\n> > \n\n> I'm sorry if I was not clear about all this. NonStop is not a Linux platform. It is POSIX. Not all utilities are available and not all utilities have all capabilities. lv is not available for the platform. less considers the data binary and displays what usually is displayed when you try to use it for binary multibyte. You get @^@- and such. It does not present the data in the correct character set for the user.\n> \n> This was only one part of my original question. I am searching elsewhere for support on pagers, because this really is not an appropriate discussion for the git group to focus on, do let's drop this, please, as not worth continuing. My original request was more about how to set up the file attributes, difference engine, and the rest of the git infrastructure. The partner I am working with is doing this with git hooks, which I am not really happy about. Let's prune this discussion as not worthy.\n> \n\nYes, this is totally unclear. Setting git hooks is possible but setting\nLESSCHARDEF is not?\n\nIs patching git acceptable or is that out of question too?\n\nWhat are your requirements, exactly?\n\nIf your data is in fact UTF-16 how is SJIS involved?\n\nMichal\n"},{"id":"370340","messageId":"003301d4cecd$4bced250$e36c76f0$@nexbridge.com","threadId":"50604","inReplyTo":"20190227191814.054c2f3c@kitsune.suse.cz","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T18:50:18Z","receivedAt":"2019-02-27T18:50:30Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 13:18, Michal Suchánek wrote:\n> On Wed, 27 Feb 2019 12:59:15 -0500\n> \"Randall S. Becker\" wrote:\n> \n> > On February 27, 2019 12:51, Michal Suchánek wrote:\n> > > To: Randall S. Becker \n> > > Cc: git@vger.kernel.org\n> > > Subject: Re: Need multibyte advice - Shift-JIS\n> > >\n> \n> > I'm sorry if I was not clear about all this. NonStop is not a Linux platform. It\n> is POSIX. Not all utilities are available and not all utilities have all capabilities.\n> lv is not available for the platform. less considers the data binary and displays\n> what usually is displayed when you try to use it for binary multibyte. You get\n> @^@- and such. It does not present the data in the correct character set for\n> the user.\n> >\n> > This was only one part of my original question. I am searching elsewhere\n> for support on pagers, because this really is not an appropriate discussion for\n> the git group to focus on, do let's drop this, please, as not worth continuing.\n> My original request was more about how to set up the file attributes,\n> difference engine, and the rest of the git infrastructure. The partner I am\n> working with is doing this with git hooks, which I am not really happy about.\n> Let's prune this discussion as not worthy.\n> >\n> \n> Yes, this is totally unclear. Setting git hooks is possible but setting\n> LESSCHARDEF is not?\nIt can enter the environment simply through .profile, where we can change GIT_PAGER. We have established that separately.\n\n> Is patching git acceptable or is that out of question too?\nI have done a bunch of git patching, where specifically.\n\n> What are your requirements, exactly?\nSource code and comments contain SJIS content. The requirement is to be able to move seamlessly in and out of git, and have git show/diff/log display SJIS as well as ASCII content. How that happens is open. The UTF-16 is a red-herring, only as an attempt at getting at SJIS content differently than the limitation imposed by less.\n\n\n"},{"id":"370341","messageId":"20190227195934.60c45e5d@kitsune.suse.cz","threadId":"50604","inReplyTo":"003301d4cecd$4bced250$e36c76f0$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Michal Suchánek","fromEmail":"msuchanek@suse.de","sentAt":"2019-02-27T18:59:34Z","receivedAt":"2019-02-27T18:59:38Z","isPatch":false,"sender":{"key":"msuchanek@suse.de","avatar":"https://avatars.githubusercontent.com/u/787652?v=4"},"body":"On Wed, 27 Feb 2019 13:50:18 -0500\n\"Randall S. Becker\" wrote:\n\n> On February 27, 2019 13:18, Michal Suchánek wrote:\n> > On Wed, 27 Feb 2019 12:59:15 -0500\n> > \"Randall S. Becker\" wrote:\n> > \n> > > On February 27, 2019 12:51, Michal Suchánek wrote: \n> > > > To: Randall S. Becker \n> > > > Cc: git@vger.kernel.org\n> > > > Subject: Re: Need multibyte advice - Shift-JIS\n> > > > \n> > \n> > > I'm sorry if I was not clear about all this. NonStop is not a Linux platform. It \n> > is POSIX. Not all utilities are available and not all utilities have all capabilities.\n> > lv is not available for the platform. less considers the data binary and displays\n> > what usually is displayed when you try to use it for binary multibyte. You get\n> > @^@- and such. It does not present the data in the correct character set for\n> > the user. \n> > >\n> > > This was only one part of my original question. I am searching elsewhere \n> > for support on pagers, because this really is not an appropriate discussion for\n> > the git group to focus on, do let's drop this, please, as not worth continuing.\n> > My original request was more about how to set up the file attributes,\n> > difference engine, and the rest of the git infrastructure. The partner I am\n> > working with is doing this with git hooks, which I am not really happy about.\n> > Let's prune this discussion as not worthy. \n> > > \n> > \n> > Yes, this is totally unclear. Setting git hooks is possible but setting\n> > LESSCHARDEF is not? \n> It can enter the environment simply through .profile, where we can change GIT_PAGER. We have established that separately.\n> \n> > Is patching git acceptable or is that out of question too? \n> I have done a bunch of git patching, where specifically.\n> \n> > What are your requirements, exactly? \n> Source code and comments contain SJIS content. The requirement is to be able to move seamlessly in and out of git, and have git show/diff/log display SJIS as well as ASCII content. How that happens is open. The UTF-16 is a red-herring, only as an attempt at getting at SJIS content differently than the limitation imposed by less.\n> \n> \n\n - less is a very limited pager when it comes to international support.\n Patches to support Japanese specifically were at a time merged in\n RedHat but then reverted because they broke other languages. If\n 'cat' pager is good enough except for the paging 'less -r' provides\n same plus the paging. You can fine-tune what is supposed to be\n displayed verbatim and what is supposed to be treated specially with\n LESSCHARDEF but it might not work well for SJIS. You can also set\n the LESS variable to include -r by default.\n\n - if patching git is not out of question I don't see how adding a a\n full-featured pager such as lv is. \n\n - I don't think git itself needs any adjustments to handle SJIS if it\n can handle UTF-8 already. UTF-16 would be a different story.\n\nMichal\n"},{"id":"370342","messageId":"d3b76f4f-29b7-663e-ca3b-c15b1345a133@kdbg.org","threadId":"50604","inReplyTo":"003301d4cecd$4bced250$e36c76f0$@nexbridge.com","subject":"Re: Need multibyte advice - Shift-JIS","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2019-02-27T19:36:06Z","receivedAt":"2019-02-27T19:36:49Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 27.02.19 um 19:50 schrieb Randall S. Becker:\n> On February 27, 2019 13:18, Michal Suchánek wrote:\n>> What are your requirements, exactly?\n> Source code and comments contain SJIS content. The requirement is to\n> be able to move seamlessly in and out of git, and have git show/diff/log\n> display SJIS as well as ASCII content. How that happens is open. The\n> UTF-16 is a red-herring, only as an attempt at getting at SJIS content\n> differently than the limitation imposed by less.\n\nWhen your file content contains ShiftJIS, you should set an attribute in\n.gitattributes:\n\n*.sourcecode\tencoding=ShiftJIS\n\nWhen your git commits messages contain ShiftJIS, you should configure\n\n git config i18n.commitEncoding ShiftJIS\n\nMore precisely, this assumes that your editor that composes to commits\nmessages writes the .git/COMMIT_MSG in ShiftJIS.\n\nWhen your terminal or pager is configured that it interprets the byte\nstream that it receives from applications for display as ShiftJIS, then\nyou should configure\n\n git config i18n.logOutputEncoding ShiftJIS\n\nYou can set this independently from the other settings. In particular,\nwhen it is not set, UTF-8 is assumed. That is, if your terminal or pager\nsupports UTF-8, you should *NOT* set this configuration (or set it to\nUTF-8).\n\nAnd, of course, you must have built Git with iconv, which must have\nsupport for ShiftJIS.\n\n-- Hannes\n"},{"id":"370343","messageId":"004601d4ced6$2c6692f0$8533b8d0$@nexbridge.com","threadId":"50604","inReplyTo":"d3b76f4f-29b7-663e-ca3b-c15b1345a133@kdbg.org","subject":"RE: Need multibyte advice - Shift-JIS","fromName":"Randall S. Becker","fromEmail":"rsbecker@nexbridge.com","sentAt":"2019-02-27T19:53:50Z","receivedAt":"2019-02-27T19:54:08Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On February 27, 2019 14:36, Johannes Sixt wrote:\n> Am 27.02.19 um 19:50 schrieb Randall S. Becker:\n> > On February 27, 2019 13:18, Michal Suchánek wrote:\n> >> What are your requirements, exactly?\n> > Source code and comments contain SJIS content. The requirement is to\n> > be able to move seamlessly in and out of git, and have git\n> > show/diff/log display SJIS as well as ASCII content. How that happens\n> > is open. The\n> > UTF-16 is a red-herring, only as an attempt at getting at SJIS content\n> > differently than the limitation imposed by less.\n> \n> When your file content contains ShiftJIS, you should set an attribute in\n> .gitattributes:\n> \n> *.sourcecode\tencoding=ShiftJIS\n> \n> When your git commits messages contain ShiftJIS, you should configure\n> \n> git config i18n.commitEncoding ShiftJIS\n> \n> More precisely, this assumes that your editor that composes to commits\n> messages writes the .git/COMMIT_MSG in ShiftJIS.\n> \n> When your terminal or pager is configured that it interprets the byte stream\n> that it receives from applications for display as ShiftJIS, then you should\n> configure\n> \n> git config i18n.logOutputEncoding ShiftJIS\n> \n> You can set this independently from the other settings. In particular, when it\n> is not set, UTF-8 is assumed. That is, if your terminal or pager supports UTF-\n> 8, you should *NOT* set this configuration (or set it to UTF-8).\n> \n> And, of course, you must have built Git with iconv, which must have support\n> for ShiftJIS.\n\nThanks. I will forward this along and check the iconv build and rev-level we have used in git. Many thanks.\n\n\n"}]}