Re: t7005-editor.sh failure
On Wed, Sep 26, 2018 at 11:52:42AM +0200, Martin Ågren wrote:
Show 15 quoted lines
> On Wed, 26 Sep 2018 at 11:00, Alexander Pyhalov <apyhalov@gmail.com> wrote:
> > As for sign-off, do I understand correctly that you just want to know
> > that I'm the original author of the code? Yes, it's so.
>
> Right. Plus that you agree that the code (the commit) may be
> redistributed basically forever.
>
> > I see this on OpenIndiana in
> > https://github.com/OpenIndiana/oi-userland/pull/4456 , when running
> > test suite.
> > Not sure why it wasn't noticed earlier, as 'trash directory' is used in path.
>
> My first theory was that my shell and that of other developers was
> "modern" or "clever" enough to realize that the space belongs to the
> filename, so it just takes everything to the end of line.
(Note that redirections can occur anywhere in the command, i.e. these are all equivalent: 'echo foo >out', 'echo >out foo' '>out echo foo')
> Whereas your
> shell would be "dumber". I see now that you have a newer bash than I
> do... Maybe this cleverness can be configured (at compile-time?), or
> maybe something else is happening.
Let me put on my POSIX-lawyer hat for a moment to explain this :)
Long story short: Bash doesn't conform to POSIX in this respect.
So, the Shell Command Language specification section 2.6 Word Expansions [1] says among many other things the following:
Tilde expansions, parameter expansions, command substitutions,
arithmetic expansions, and quote removals that occur within a single
word expand to a single field. It is only field splitting or
pathname expansion that can create multiple fields from a single
word.
Later, in section 2.7 Redirection [2]:
[...] the word that follows the redirection operator shall be
subjected to tilde expansion, parameter expansion, command
substitution, arithmetic expansion, and quote removal. Pathname
expansion shall not be performed on the word by a non-interactive
shell; an interactive shell may perform it, but shall do so only
when the expansion would result in one word.
Note that this "word" is _not_ subject of field splitting, i.e. in a redirection like
echo foo >$file
it's not necessary to quote $file, because it will remain a single field even if it contains spaces.
Most shells I have at hand follow the specs:
$ for shell in dash mksh ksh ksh93 zsh ; do $shell ./e\ space.sh "output from $shell" ; done
$ ls -1 output\ from*
output from dash
output from ksh
output from ksh93
output from mksh
output from zsh
Bash doesn't:
$ bash ./e\ space.sh "output from bash"
./e space.sh: line 1: $1: ambiguous redirect
And this behaviour is documented in its man page (though the text calls "word splitting" what the specs call "field splitting"):
The word following the redirection operator in the following descrip
tions, unless otherwise noted, is subjected to brace expansion,
tilde expansion, parameter and variable expansion, command
substitution, arithmetic expansion, quote removal, pathname
expansion, and word splitting.
When run in posix mode, however, even Bash follows the specs:
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Sep 26 12:04 /bin/sh -> bash
$ sh ./e\ space.sh "output from bash as sh"
$ bash --posix ./e\ space.sh "output from bash --posix"
$ ls -1 output\ from\ bash*
output from bash as sh
output from bash --posix
That's why we didn't noticed it yet, not even on macOS, which uses Bash as /bin/sh. You have to build Git with 'SHELL_PATH=/bin/bash' to make t7005 fail because of this issue, and based on the trace that Alexander showed us it seems that OpenIndiana folks do build Git that way.
Good. *throws the POSIX-lawyer hat into the farthest corner*
Having said all that, I didn't omit the quotes in 4362da078e with the above in mind; in fact I tend to use quotes even when they are unnecessary (e.g. in variable assignments: var="$1"), because unquoted variables and command substitutions freak me out before I can think through whether its safe to omit the quotes or not :)
Sidenote: this test should use the write_script helper to create this
editor script.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06 [2] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07
Show 37 quoted lines
> > execve("/bin/bash", 0x007EA898, 0x007EA960) argc = 5
> > 2655: argv: /bin/bash -c ./e\ space.sh "$@" ./e\ space.sh
> > 2655: /export/home/alp/srcs/oi-userland/components/developer/git/build/amd64/t/trash
> > directory.t7005-editor/.git/COMMIT_EDITMSG
> > 2655: execve("./e space.sh", 0x005655C8, 0x00564008) Err#8 ENOEXEC
> > ./e space.sh: line 1: $1: ambiguous redirect
>
> > Shell is bash, as you can see (GNU bash, version 4.4.23(1)-release
> > (i386-pc-solaris2.11))
>
> I came up with the following commit message. What do you think about it?
>
> t7005-editor: quote filename to fix whitespace-issue
>
> Commit 4362da078e (t7005-editor: get rid of the SPACES_IN_FILENAMES
> prereq, 2018-05-14) removed code for detecting whether spaces in
> filenames work. Since we rely on spaces throughout the test suite
> ("trash directory.t1234-foo"), testing whether we can use the filename
> "e space.sh" was redundant and unnecessary.
>
> In simplifying the code, though, the commit introduced a regression around
> how spaces are handled, not in the /name/ of the script, but /in/ the
> script itself. The editor-script created looks like this:
>
> echo space >$1
>
> We will try to execute something like
>
> echo space >/foo/t/trash directory.t7005-editor/.git/COMMIT_EDITMSG
>
> Most shells seem to be able to figure out that the filename doesn't end
> with "trash" but continues all the way to "COMMIT_EDITMSG", but at least
> one shell chokes on this.
>
> Make sure that the editor-script quotes "$1".
>
> Martin
Alexander Pyhalov
