Re: [bug] git diff --word-diff gives wrong result for utf-8 chinese

From: Phillip Wood <phillip.wood123@gmail.com>
Date: Dec 1, 2022, 14:51 UTC
Message-ID: <4dac768f-1104-a565-db4c-8a1b7eb2870d@dunelm.org.uk>
In-Reply-To: <CACSwcnRDmiiJU8hL+ON6c+b4Q8UtLVbtku_rHSD+c+BwcNEX+Q@mail.gmail.com>

Hi Ping

On 01/12/2022 07:33, Ping Yin wrote:

>>> If the rule is "break on ascii whitespace",
> 
> Is there a way to achieve this: break english by word, and break
> chinese by utf-8 character

You could extend your current regex so that it matches whole utf-8 codepoints which is what git does for the builtin userdiff regexes. I've not tested it but I think

git config --global diff.wordregex "[[:alnum:]_]+|[^[:space:]]|$(printf '[\xc0-\xff][\x80-\xbf]+')"

should work. The downside is that you end up with a .gitconfig that is not valid utf-8. Perhaps someone else has a clever idea to get around that.

Best Wishes

Phillip