Re: [PATCH v5] add git-p4.fallbackEncoding config variable, to prevent git-p4 from crashing on non UTF-8 changeset descriptions

From: Torsten Bögershausen <tboegi@web.de>
Date: Apr 28, 2021, 14:58 UTC
Message-ID: <20210428145824.43c4t7hkjfqjyspb@tb-raspi4>
In-Reply-To: <xmqqr1ivauph.fsf@gitster.g>

On Wed, Apr 28, 2021 at 01:39:38PM +0900, Junio C Hamano wrote:

Show 7 quoted lines

> Tzadik Vanderhoof <tzadik.vanderhoof@gmail.com> writes:
>
> >  t/t9835-git-p4-config-fallback-encoding.sh | 87 ++++++++++++++++++++++
> >  3 files changed, 106 insertions(+), 1 deletion(-)
> >  create mode 100755 t/t9835-git-p4-config-fallback-encoding.sh
>
> 9835 is already taken (see 'seen').

In general, this looks good to me. There are two minor nitpicks to make the patch more the git-way:

> Subject: [PATCH v5] add git-p4.fallbackEncoding config variable, to prevent git-p4 from crashing on non UTF-8 changeset descriptionsw

The head line is somewhat too long. It should be much shorter, like 50-55 characters, if I recall it rigth. The first line of the commit message is what we see under PATCH in the email, followed by a blank line (that's what we have) and a detailed description (Which we have)

How abut this ?

git-p4: Add git-p4.fallbackEncoding

Add git-p4.fallbackEncoding config variable, to prevent git-p4 from crashing on non UTF-8 changeset descriptions.

When git-p4 reads the output from a p4 command, it assumes it will be 100% UTF-8. If even one character in the output of one p4 command is not UTF-8, git-p4 crashes with:

    File "C:/Program Files/Git/bin/git-p4.py", line 774, in p4CmdList
        value = value.decode() UnicodeDecodeError: 'utf-8' codec can't
        decode byte Ox93 in position 42: invalid start byte

This is especially a problem for the "git p4 clone ... @all" command, where git-p4 needs to read thousands of changeset descriptions, one of which may have a stray smart quote, causing the whole clone operation to fail.

Add a new config setting, allowing git-p4 to try a fallback encoding (for example, "cp1252") and/or use the Unicode replacement character, to prevent the whole program from crashing on such a minor problem.

[]

And then, somewhere in the test:

			cp /dev/null "$clone_fails" &&

This should create an empty file, right ? Then we can use a simple output-redirection:

			>"$clone_fails" &&