Re: [PATCH 0/3] [GSOC][RFC] ref-filter: add contents:raw atom

From: ZheNing Hu <adlternative@gmail.com>
Date: May 26, 2021, 06:45 UTC
Message-ID: <CAOLTT8ReZffY5gznSDD=Fgbt7YTtA5aJWX+f8Q8npcj0OwcuFQ@mail.gmail.com>
In-Reply-To: <xmqq8s42cnyb.fsf@gitster.g>

Show 15 quoted lines

>
> Another thing to keep in mind is that not all host languages may be
> capable of expressing a string with NUL in it.  Most notably, shell.
> The --shell quoting rule used by for-each-ref would produce an
> equivalent of the "script" produced like this:
>
>         $ tr Q '\000' >script <<\EOF
>         #!/bin/sh
>         varname='varQname'
>         echo "$varname"
>         EOF
>
> but I do not think it would say 'var' followed by a NUL followed by
> 'name'.  The NUL is likely lost when assigned to the variable.
>

Yes, in the following example you mentioned earlier, I have also noticed the loss of '\0'.

Show 7 quoted lines

> >     git for-each-ref --format='
> >               name=%(refname)
> >               var=%(placeholder)
> >                 mkdir -p "$(dirname "$name")"
> >               printf "%%s" "$var" >"$name"
> >     ' --shell | /bin/sh
> >

Show 6 quoted lines

> So for some host languages, binaries may be useless with or without
> quoting.  But for ones that can use strings to hold arbitrary byte
> sequence, it should be OK to let for-each-ref to quote the byte
> sequence as a string literal for the language (so that the exact
> byte sequence will end up being in the variable after assignment).
>

I agree, and maybe some'\0' can be escaped appropriately to let host languages recognize it....

Show 5 quoted lines

> That reminds me of another thing.  The --python thing was written
> back when Python3 was still a distant dream and strings were the
> appropriate type for a random sequence of bytes (as opposed to
> unicode, which cannot have a random sequence of bytes).  Somebody
> needs to check if it needs any update to work with Python3.

$ printf '%b' "name='a\\\0b\\\0c'\nprint(name)" | python2.7 | od -c 0000000 a \0 b \0 c \n 0000006

$ printf '%b' "name='a\\\0b\\\0c'\necho -e \"\$name\"" | sh | od -c 0000000 a \0 b \0 c \n 0000006

In shell or python2/3, we can replace'\0' with "\\0".

In Tcl and perl, they are ok with '\0'.

$ printf '%b' "set name \"a\0b\0c\"\nputs \$name" | tclsh | od -c 0000000 ' a \0 b \0 c ' \n 0000010

$ printf '%b' "\$name = 'a\0b\0c';\n print \"\$name\"" | perl | od -c 0000000 a \0 b \0 c 0000005

So I currently think that a possible strategy is to modify `python_quote_buf_with_size()` and `sq_quote_buf_with_size()` from '\0' to "\\0".

Thanks!

-- ZheNing Hu