Re: [PATCH 0/3] [GSOC][RFC] ref-filter: add contents:raw atom

From: Junio C Hamano <gitster@pobox.com>
Date: May 26, 2021, 07:06 UTC
Message-ID: <xmqqpmxeas8p.fsf@gitster.g>
In-Reply-To: <CAOLTT8ReZffY5gznSDD=Fgbt7YTtA5aJWX+f8Q8npcj0OwcuFQ@mail.gmail.com>

ZheNing Hu <adlternative@gmail.com> writes:

> $ printf '%b' "name='a\\\0b\\\0c'\necho -e \"\$name\"" | sh | od -c
> 0000000   a  \0   b  \0   c  \n
> 0000006

This is wrong. In the above, the variable name does not have a NUL in it. It insead has 2-byte sequence "\" and "0" in it, and you are letting "echo -e" to convert it into binary, which is not portable at all.

I'd suggest instead to declare that some host languages, like shell, are not binary-clean and either refuse to process atoms whose values have NUL in them. Silently truncating strings at NUL or striping NULs in strings are also acceptable options if clearly documented. Claiming that we stuff binaries into variables of the host language, while not doing so and instead assigning a quoted form, is not good.

I have not thought about Python3 very much. For the purpose of most %(placeholders), it is vastly more preferrable to use str (i.e. text sequence type) as opposed to bytes, as you do not have to .decode() to use the resulting "string", but even for things like %(refname), it is not technically kosher to assume that the contents are UTF-8 encoded text, as filenames used to represent refnames are merely a sequence of bytes except NUL, but for consistency with binary gunk, we might have to emit everything as bytes. I dunno.

> In shell or python2/3, we can replace'\0' with "\\0".

Not for shell. We should declare that it is not supported to feed binary to shell.