Re: t0008 test fails with ksh

From: Jeff King <peff@peff.net>
Date: May 12, 2016, 18:20 UTC
Message-ID: <20160512182055.GB13886@sigill.intra.peff.net>
In-Reply-To: <CALR6jEjWjJA0X2qXsxqObqc_yxrgX87LYf8cmJ0MmJFF6PkmTQ@mail.gmail.com>

On Thu, May 12, 2016 at 07:53:28PM +0200, Armin Kunaschik wrote:

Show 36 quoted lines

> The reason seems that the snippet
> cat <<-EOF >expected-all
>         .gitignore:1:one        ../one
>         ::      ../not-ignored
>         .gitignore:1:one        one
>         ::      not-ignored
>         a/b/.gitignore:8:!on*   b/on
>         a/b/.gitignore:8:!on*   b/one
>         a/b/.gitignore:8:!on*   b/one one
>         a/b/.gitignore:8:!on*   b/one two
>         a/b/.gitignore:8:!on*   "b/one\"three"
>         a/b/.gitignore:9:!two   b/two
>         ::      b/not-ignored
>         a/.gitignore:1:two*     b/twooo
>         $global_excludes:2:!globaltwo   ../globaltwo
>         $global_excludes:2:!globaltwo   globaltwo
>         $global_excludes:2:!globaltwo   b/globaltwo
>         $global_excludes:2:!globaltwo   ../b/globaltwo
>         ::      c/not-ignored
> EOF
> 
> behaves differently in bash and in ksh.
>         a/b/.gitignore:8:!on*   "b/one\"three" comes out unmodified
> with bash but with ksh it becomes
>         a/b/.gitignore:8:!on*   "b/one"three"
> I'm not sure what shell is "wrong" here, but when I modify the line to
>         a/b/.gitignore:8:!on*   "b/one\\"three"
> both shells generate the "right" output:
>         a/b/.gitignore:8:!on*   "b/one\"three"
> 
> From what I've learned I'd expect the double backslash to be the
> "right" way to escape one
> backslash. Do you agree or am I wrong?
> 
> This snippet appears twice in this test, generates expected-all and
> expected-verbose.

I think either is reasonable (there is no need to backslash-escape a double-quote inside a here-doc, but one assumes that backslash would generally have its usual behavior). I'm not quite sure how to interpret POSIX here (see below), but it seems clear that spelling it with two backslashes as you suggest is the best bet.

For the curious, here's what POSIX says (in the section on here-documents; "word" here refers to the EOF word):

  If no characters in word are quoted, all lines of the here-document
  shall be expanded for parameter expansion, command substitution, and
  arithmetic expansion. In this case, the backslash in the input behaves
  as the backslash inside double-quotes (see Double-Quotes). However,
  the double-quote character ( '"' ) shall not be treated specially
  within a here-document, except when the double-quote appears within
  "$()", "``", or "${}".

So OK, that sounds like ksh is doing the right thing. But what's that "specially" in the last sentence? Do they just mean it doesn't start a quoted section as it would elsewhere? Or do they mean in the section on Double-Quotes, when they say:

  \
      The backslash shall retain its special meaning as an escape
      character (see Escape Character (Backslash)) only when followed by
      one of the following characters when considered special:

              $   `   "   \   <newline>

Since the double-quote isn't special here, does that mean that backslash shouldn't retain its special meaning in this case? That would make bash right.

Anyway, it doesn't really matter what the standard says. We can spell this in a less ambiguous way, and it does not hurt too much to do so.

-Peff