threads / patch / 7981

patchRe: [PATCH] git-parse-remote: fix ambiguous shell bug in expand_refs_wildcard

Subject: Re: [PATCH] git-parse-remote: fix ambiguous shell bug in expand_refs_wildcard

## tl;dr

7 messages between Dec 18, 2006 and May 7, 2007. Diffs are folded; open one to read it.

replies: 6people: 2as markdown or json

Jeff King· Dec 18, 2006, 08:09 UTC · lore

[PATCH] git-parse-remote: fix ambiguous shell bug in expand_refs_wildcard

Bash and dash parse "${name%'^{}'}" differently, with bash quoting the ^{}, and dash assuming the first } is the end of the variable (and thus tacking '} to the end). Instead, use backslash to quote the closing brace.

Signed-off-by: Jeff King <peff@peff.net>
---
Please sanity check that I understand what the code is supposed to be
doing. The bug I was getting was this:
$ readlink /bin/sh
dash
$ git-clone git://git.kernel.org/pub/scm/git/git.git
[...]
$ cd git && git-pull
Warning: No merge candidate found because value of config option
         "branch.master.merge" does not match any remote branch fetched.

A shell trace showed lots of tests against strings like "refs/heads/master'}". Expand_refs_wildcard returned no entries, so it couldn't match the branch.master.merge field.

 git-parse-remote.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Show changes to git-parse-remote.sh +1 −1
diff --git a/git-parse-remote.sh b/git-parse-remote.sh
index f27c3c2..ab79042 100755
--- a/git-parse-remote.sh
+++ b/git-parse-remote.sh
@@ -116,7 +116,7 @@ expand_refs_wildcard () {
 			while read sha1 name
 			do
 				mapped=${name#"$from"}
-				if test "z$name" != "z${name%'^{}'}" ||
+				if test "z$name" != "z${name%'^{\}'}" ||
 					test "z$name" = "z$mapped"
 				then
 					continue
Jeff King· Dec 18, 2006, 08:16 UTC · re: Jeff King · lore
On Mon, Dec 18, 2006 at 03:09:41AM -0500, Jeff King wrote:
> -				if test "z$name" != "z${name%'^{}'}" ||
> +				if test "z$name" != "z${name%'^{\}'}" ||
Urgh, sorry, this is wrong. It should be:
  ${name%^{\}}
IOW, the \ replaces the quote, not in addition to.
Jeff King· Dec 18, 2006, 22:45 UTC · lore
On Mon, Dec 18, 2006 at 11:57:33AM -0800, Junio C Hamano wrote:
> Sounds like a dash bug, if my reading of 2.6.2 Parameter
> Expansion is correct:
> 
>     http://www.opengroup.org/onlinepubs/000095399/utilities/xcu_chap02.html
Interestingly, this works in dash:

$ foo=bar} $ echo ${foo%'}'} bar

but doing it inside an interpolated string doesn't:

$ foo=bar} $ echo "${foo%'}'}" bar}'}

Show 9 quoted lines
> This would be another way to work it around.  Both dash and bash
> say 'foo':
> 
>         $ suf='^{}'
>         $ name='foo^{}'
>         $ echo "${name%$suf}"
>         foo
> 
> I think this might be easier to read than using "^{\}".
That seems reasonable to me.
Herbert Xu· Dec 19, 2006, 00:35 UTC · re: Jeff King · lore
On Mon, Dec 18, 2006 at 05:45:05PM -0500, Jeff King wrote:
Show 6 quoted lines
> 
> but doing it inside an interpolated string doesn't:
> 
> $ foo=bar}
> $ echo "${foo%'}'}"
> bar}'}

Yes it's a bug in dash. Both quote marks (" and ') are represented by the same char internally before processing which is where the mix-up occurs.

I'll work on a fix.
Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
Herbert Xu· May 5, 2007, 08:03 UTC · re: Herbert Xu · lore
On Tue, Dec 19, 2006 at 11:35:57AM +1100, herbert wrote:
Show 13 quoted lines
> On Mon, Dec 18, 2006 at 05:45:05PM -0500, Jeff King wrote:
> > 
> > but doing it inside an interpolated string doesn't:
> > 
> > $ foo=bar}
> > $ echo "${foo%'}'}"
> > bar}'}
> 
> Yes it's a bug in dash.  Both quote marks (" and ') are represented
> by the same char internally before processing which is where the
> mix-up occurs.
> 
> I'll work on a fix.

Sorry for the delay. I've finally looked at fixing this. It turns out that dash's behaviour is actually correct and POSIX compliant.

It's correct because dash treats all single quotes within double quotes (except those within command substitutions) as literals. This interpretation is also supported by POSIX.

In fact the rationale (C.2.2.3) in the POSIX document explicitly disallows the aformentioned usage as it violates the rule that an even number of single quotes if any can occur in an ${...} expression enclosed by double quotes.

So the correct and portable expression in this case would be either
echo "${foo%\}}"
or

brace=} echo "${foo%$brace}"

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Jeff King· May 7, 2007, 06:36 UTC · re: Herbert Xu · lore
On Sat, May 05, 2007 at 06:03:13PM +1000, Herbert Xu wrote:
> In fact the rationale (C.2.2.3) in the POSIX document explicitly
> disallows the aformentioned usage as it violates the rule that an
> even number of single quotes if any can occur in an ${...} expression
> enclosed by double quotes.

Yes, there's not much room for interpretation; the old git code was clearly bogus (we are working around it by using sed instead). Thanks for tracking this down, Herbert.

It looks like bash is actually broken in POSIXLY_CORRECT mode, then:

$ echo $BASH_VERSION 3.1.17(1)-release $ POSIXLY_CORRECT=1 $ foo=bar} $ echo "${foo%'}'}" bar

My interpretation of the correct behavior is that it should remove a single quote from the end of foo, and then print '} literally (that is, single quote and brace).

-Peff
Herbert Xu· May 7, 2007, 08:01 UTC · re: Jeff King · lore
On Mon, May 07, 2007 at 02:36:22AM -0400, Jeff King wrote:
Show 13 quoted lines
>
> It looks like bash is actually broken in POSIXLY_CORRECT mode, then:
> 
> $ echo $BASH_VERSION
> 3.1.17(1)-release
> $ POSIXLY_CORRECT=1
> $ foo=bar}
> $ echo "${foo%'}'}"
> bar
> 
> My interpretation of the correct behavior is that it should remove a
> single quote from the end of foo, and then print '} literally (that is,
> single quote and brace).

Well strictly speaking this is allowed by the standard as this usage contains an odd number of single quotes inside an ${...} expression enclosed by double quotes, which behaves in an implementation-specific manner.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

← back to recent threads