threads / bug / 12114

[BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Subject: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

## tl;dr

9 messages between Feb 15, 2008 and Feb 16, 2008.

replies: 8people: 3as markdown or json

Remi Vanicat· Feb 15, 2008, 16:56 UTC · lore

git filter-branch --tree-filter has a problem with filename with accentuated letter:

$ git add foo/baré
$ git commit -m "adding a file with an accent"
Created initial commit b27ae97: adding a file with an accent
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 "foo/bar\303\251"
$ git filter-branch --tree-filter "rm -rf foo"
Rewrite b27ae977459379e4e7eee1a3d523f908903ea6ae (1/1)
WARNING: Ref 'refs/heads/master' is unchanged

there the foo/baré file still exists, but: $ git filter-branch --tree-filter "rm -rf foo; git add -u" will suppress the said file from history.

The culprit seem to be those line of filter-branch: (around line 279) 
		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
			xargs -0 git update-index --add --replace --remove
git diff-index giving the filename as "foo/bar\303\251"
-- 
Rémi Vanicat
Junio C Hamano· Feb 15, 2008, 17:50 UTC · re: Remi Vanicat · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Remi Vanicat <vanicat@debian.org> writes:
> The culprit seem to be those line of filter-branch: (around line 279) 
> 		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
> 			xargs -0 git update-index --add --replace --remove
> git diff-index giving the filename as "foo/bar\303\251"
I have to wonder in what century filter-branch was written ;-)
Shouldn't those two lines be:
	git diff-index -r --name-only $commit |
        git update-index --add --replace --remove --stdin
these days, without any of the cut and cruft?
Johannes Schindelin· Feb 15, 2008, 18:12 UTC · re: Junio C Hamano · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Hi,
On Fri, 15 Feb 2008, Junio C Hamano wrote:
Show 15 quoted lines
> Remi Vanicat <vanicat@debian.org> writes:
> 
> > The culprit seem to be those line of filter-branch: (around line 279) 
> > 		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
> > 			xargs -0 git update-index --add --replace --remove
> > git diff-index giving the filename as "foo/bar\303\251"
> 
> I have to wonder in what century filter-branch was written ;-)
> 
> Shouldn't those two lines be:
> 
> 	git diff-index -r --name-only $commit |
>         git update-index --add --replace --remove --stdin
> 
> these days, without any of the cut and cruft?
Maybe even using "-z" in both cases?

Having said that, I do not understand why the old code did not work. Will have a look later today.

Ciao, Dscho

Junio C Hamano· Feb 15, 2008, 22:21 UTC · re: Johannes Schindelin · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 13 quoted lines
>> I have to wonder in what century filter-branch was written ;-)
>> 
>> Shouldn't those two lines be:
>> 
>> 	git diff-index -r --name-only $commit |
>>         git update-index --add --replace --remove --stdin
>> 
>> these days, without any of the cut and cruft?
>
> Maybe even using "-z" in both cases?
>
> Having said that, I do not understand why the old code did not work.  Will 
> have a look later today.

The reason mine does not have to use -z is because both end knows how to C-quote paths under non-z mode.

Now you mention it, it certainly is a bit puzzling why the old one did not work.

Johannes Schindelin· Feb 16, 2008, 03:09 UTC · re: Junio C Hamano · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Hi,
On Fri, 15 Feb 2008, Junio C Hamano wrote:
Show 7 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > Having said that, I do not understand why the old code did not work.  
> > Will have a look later today.
> 
> Now you mention it, it certainly is a bit puzzling why the old one did 
> not work.
Okay, so I will not manage today.  Tomorrow is another day.

Ciao, Dscho

Johannes Schindelin· Feb 16, 2008, 12:48 UTC · re: Johannes Schindelin · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Hi,
On Sat, 16 Feb 2008, Johannes Schindelin wrote:
Show 11 quoted lines
> On Fri, 15 Feb 2008, Junio C Hamano wrote:
> 
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > 
> > > Having said that, I do not understand why the old code did not work.  
> > > Will have a look later today.
> > 
> > Now you mention it, it certainly is a bit puzzling why the old one did 
> > not work.
> 
> Okay, so I will not manage today.  Tomorrow is another day.

Ah, now I understand. The accents are shown quoted, but xargs does not know how to unquote them. Cute.

Ciao, Dscho

Remi Vanicat· Feb 16, 2008, 06:34 UTC · re: Junio C Hamano · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Junio C Hamano <gitster@pobox.com> writes:
Show 13 quoted lines
> Remi Vanicat <vanicat@debian.org> writes:
>
>> The culprit seem to be those line of filter-branch: (around line 279) 
>> 		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
>> 			xargs -0 git update-index --add --replace --remove
>> git diff-index giving the filename as "foo/bar\303\251"
>
> I have to wonder in what century filter-branch was written ;-)
>
> Shouldn't those two lines be:
>
> 	git diff-index -r --name-only $commit |
>         git update-index --add --replace --remove --stdin
That solve the bug for me.
-- 
Rémi Vanicat
Junio C Hamano· Feb 16, 2008, 07:54 UTC · re: Remi Vanicat · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Not just those two lines but the next two lines were also very old fashioned.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 * Could you try this one?
 git-filter-branch.sh     |    9 +++++----
 t/t7003-filter-branch.sh |   14 ++++++++++++++
 2 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index ff716ca..49e13f0 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -276,10 +276,11 @@ while read commit parents; do
 		eval "$filter_tree" < /dev/null ||
 			die "tree filter failed: $filter_tree"
 
-		git diff-index -r $commit | cut -f 2- | tr '\012' '\000' | \
-			xargs -0 git update-index --add --replace --remove
-		git ls-files -z --others | \
-			xargs -0 git update-index --add --replace --remove
+		(
+			git diff-index -r --name-only $commit
+			git ls-files --others
+		) |
+		git update-index --add --replace --remove --stdin
 	fi
 
 	eval "$filter_index" < /dev/null ||
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 5f60b22..868babc 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -165,4 +165,18 @@ test_expect_success '"map" works in commit filter' '
 	git rev-parse --verify master
 '
 
+test_expect_success 'Name needing quotes' '
+
+	git checkout -b rerere A &&
+	mkdir foo &&
+	name="れれれ" &&
+	>foo/$name &&
+	git add foo &&
+	git commit -m "Adding a file" &&
+	git filter-branch --tree-filter "rm -fr foo" &&
+	! git ls-files --error-unmatch "foo/$name" &&
+	test $(git rev-parse --verify rerere) != $(git rev-parse --verify A)
+
+'
+
 test_done
Remi Vanicat· Feb 16, 2008, 08:26 UTC · re: Junio C Hamano · lore

Re: [BUG] git filter-branch failed to suppress a file with an accentuated letter in the filename

Junio C Hamano <gitster@pobox.com> writes:
Show 7 quoted lines
> Not just those two lines but the next two lines were also very
> old fashioned.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>
>  * Could you try this one?
It work both on my real word case and on my tests git repositories. 
-- 
Rémi Vanicat

← back to recent threads