Re: [PATCH] git-p4: preserve utf8 BOM when importing from p4 to git

From: Tzadik Vanderhoof <tzadik.vanderhoof@gmail.com>
Date: Dec 14, 2022, 06:10 UTC
Message-ID: <CAKu1iLWvd+PVOK405Q+SNDyyYnhbi=LtovZvWw55y6eQqoRpaA@mail.gmail.com>

From: Tao Klerks <tao@klerks.biz>

Show 18 quoted lines

> Perforce has a file type "utf8" which represents a text file with
> explicit BOM. utf8-encoded files *without* BOM are stored as
> regular file type "text". The "utf8" file type behaves like text
> in all but one important way: it is stored, internally, without
> the leading 3 BOM bytes.
>
> git-p4 has historically imported utf8-with-BOM files (files stored,
> in Perforce, as type "utf8") the same way as regular text files -
> losing the BOM in the process.
>
> Under most circumstances this issue has little functional impact,
> as most systems consider the BOM to be optional and redundant, but
> this *is* a correctness failure, and can have lead to practical
> issues for example when BOMs are explicitly included in test files,
> for example in a file encoding test suite.
>
> Fix the handling of utf8-with-BOM files when importing changes from
> p4 to git, and introduce a test that checks it is working correctly

I don't really understand the problem that this patch addresses, but I feel it was incorrect to modify the behavior of git-p4 in this way. This change has made git-p4 behave differently than the native Perforce tools.

Perforce already has a "variable" (setting) to control this behavior. If P4CHARSET is set to "utf8-bom", the BOM will be added to files in the local workspace of type "utf8". If P4CHARSET is set to "utf8", the BOM will NOT be stored in the local workspace file, even if its type is "utf8"!

Whatever the problem was that motivated this change, it should have been solved using the Perforce variable P4CHARSET, not by modifying the behavior of git-p4. This new behavior has made it impossible for me to submit changes to files of type "utf8"! Any attempt fails with "patch does not apply" and the erroneously added BOM is the cause.

I propose rolling back the patch that introduced this behavior, and if that is not possible, I would like to submit a patch that adds a new setting in the "git-p4" section that will enable users to opt out of this new behavior (for example a boolean setting "git-p4.noUtf8Bom").

-- 
Tzadik