Bug in git-filter-branch example

I've been working on importing my svn repo into git, and while moving some things around using git-filter-branch I ran into a bug in the example provided on the manpage:

    To move the whole tree into a subdirectory, or remove it from there:

    git filter-branch --index-filter \
            'git ls-files -s | sed "s-\t-&newsubdir/-" |
                    GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
                            git update-index --index-info &&
             mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD

The problem is with filenames that use complex utf8 (non-ascii?) chars:

-rw-r--r-- 1 knuckles users 109K 2010-01-31 18:57 Papers/A Java Fork ⁄ Join Framework -- Lea.pdf

as you can see, I'm using '⁄' as a replacement for '/' on that filename, since I obviously can't use '/' on a filename. The problem is, git ls-files -s lists that file wrapped in quotes and with the char value escaped

100644 c09309342037fa7d91f37651e2f16e981e4d739a 0    "Papers/A Java Fork \342\201\204 Join Framework -- Lea.pdf"

while other files aren't, so the simple sed line provided won't work, as it will output the new filename as newsubdir/"Papers...

I ended up using git filter-branch to remove the offending file ("if all you have is a hammer..."), and re-adding it after the move.

I'm also left wondering how many uses of git ls-files out there on the "internets" survive working on a filename such as this one. Maybe git ls-files should have an option to output the real utf8 file name instead of escaping the bytes...?

Ivo Anjo

On Mon, Feb 01, 2010 at 11:48:27AM +0000, Ivo Anjo wrote:

Yes, that is definitely the problem. update-index unquotes its input, as you would expect, but the sed munging does not take the quote into account. You can fix it by doing:

  sed "s-\t\"*-&newsubdir/-"

instead. Here is a patch to fix the documentation. I am slightly unsure of whether it should be applied. These examples are supposed to be simple and readable to help the user understand what the filters can do. And this makes it somewhat less simple for the sake of a special case. But at the same time, users are going to cut-and-paste these examples (I know I have), and the special case is not _that_ special, especially for non-English speakers. And having it in the example helps make people aware that quoted paths are a reality.

So I think on balance it is probably better to fix it.

Note also that another way of "fixing" this would be to set core.quotepath to false (which is something you probably want to do anyway if you are using utf8 characters in your filenames). And I put "fix" in quotes because you still may run across quoted paths, but they will be much less common; you will only see them if you have control characters or other insanity in your paths.

-- >8 --
Subject: [PATCH] docs: fix filter-branch example for quoted paths

If there is a quoted path, update-index will correctly unquote it. However, we must take care to put our new prefix inside the double-quote.

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-filter-branch.txt |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

Ivo Anjo schrieb:

Thanks for the report, but: Examples are just that: examples. IMO, it is OK that there are implicit assumptions (such as that only "common" file names are used in the repo).

Making the example universally applicable would greatly obfuscate the important messages.

It seems that the example has met its objective: It showed you how to use the hammer, and with your creativity mixed in, it enabled you to use the hammer in new ways.

-- Hannes