[bug] git-check-ignore and file names with unicode chars in name - sys-out filename is corrupted

Reproduction:

  $ echo "*.ignoreme" >> .gitignore
  # (and commit)
  $ touch "fooo-€.ignoreme"
  $ find . -print | grep fooo | xargs git check-ignore
  "./fooo-\342\202\254.ignoreme"

You could view that git-check-ignore isn't corrupting anything, it is just outputting another form for the file name (octal escaped), but it doesn't need to change it at all, and its causing downstream problems in bash scripting.

Of course this may get munged by gmail or a list manager. In the text above, you should see unicode char "euro sign" to the right of a dash, and the left of .ignoreme

Git version is 2.9.2

-ph

On Tue, Aug 09, 2016 at 01:47:18AM -0400, Paul Hammant wrote:

It's not corrupted; like all git commands, check-ignore by default prints paths with a reversible quoting mechanism, so that odd filenames are not syntactically ambiguous (e.g., consider a filename with a newline in it), and so that you don't get binary spew on your terminal.

For robust scripting, you can either:

  - unquote the filenames in the receiving script (detect the presence
    of quoting by the double-quote in the first character, and then
    normal C-style dequoting).

or

  - use "-z" to get NUL-delimited filenames with no quoting. Your
    example above has problems in the find, grep, and xargs
    commands, too. A more careful version is:

      find . -print0 | grep -z fooo | git check-ignore --stdin -z

For human readability, you can do:

  git config core.quotepath false

to avoid quoting binary characters (here and in other tools like "git diff"), which is convenient if you use UTF8 filenames. It also will "unbreak" your scripts in the sense that it will avoid quoting in more situations. The scripts would still choke on more weird filenames (e.g., ones with embedded tabs or newlines), but in practice you'd probably never notice.

-Peff