Volume XXII, number 279Tuesday, October 6, 2026Latest message 21 minutes ago

The Git List

News and archive of git@vger.kernel.org, since April 2005

patchfiter-branch: fix commit map init from state branch

12 messages between Aug 1, 2026 and Oct 5, 2026, from Grant Moyer, Michele Locati, Patrick Steinhardt, Junio C Hamano.

Plain Markdown or JSON for tools and agents. Diffs are folded; open one to read it.

Grant MoyerAug 1, 2026, 03:31 UTC on lore

The commit map dir is populated from the state branch assuming a "to_commit:from_commit" format, but the state branch is written with a "from_commit:to_commit" format, resulting in an inverted mapping when the map is populated from the state branch. This is especially evident when --prune-empty is used and creates commits which map to nothing; when the map dir is populated from this state on subsequent runs, git-filter-branch outputs many errors while trying to create files with empty names, like:

> /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory

This change corrects the population of the commit map dir to match the "from_commit:to_commit" format.

Signed-off-by: Grant Moyer <dev@grantmoyer.com>
---
 git-filter-branch.sh     | 4 +++-
 t/t7003-filter-branch.sh | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)
Show changes to 2 files +4 −2

git-filter-branch.sh, t/t7003-filter-branch.sh

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 24fa317aaa..9aa07be6e1 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -302,7 +302,9 @@ then
 		do
 			case "$line" in
 			*:*)
-				echo "${line%:*}" >../map/"${line#*:}";;
+				from_commit=${line%:*}
+				to_commit=${line#*:}
+				echo "$to_commit" >../map/"$from_commit";;
 			*)
 				die "Unable to load state from $state_branch:filter.map";;
 			esac
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 86011e7b1f..3934cc4a11 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -121,7 +121,7 @@ W=$(git rev-parse HEAD)
 test_expect_success 'using --state-branch to skip already rewritten commits' '
 	test_when_finished git reset --hard $V &&
 	git reset --hard $V &&
-	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
+	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
 	test_cmp_rev $W HEAD
 '
 

base-commit: a97fcc37c2bc6340a8d7ce78dedf227aac4e9aa7
-- 
2.55.0
Michele LocatiSep 29, 2026, 14:31 UTC in reply to Grant Moyer on lore

Re: [PATCH] fiter-branch: fix commit map init from state branch

Hi Grant,
thanks for this patch: I hit the same bug, and I was about to report it.

Cc'ing Patrick and Junio, since the problem was introduced by f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), released in 2.50.0.

I tested your fix with git 2.56.0, running filter-branch incrementally with --state-branch, --prune-empty and --subdirectory-filter:

- without the fix, the new commits are attached to the original
  parents instead of the rewritten ones, and swapped entries are saved
  back to the state branch;
- with the fix, the parents are the rewritten commits and the state
  is saved correctly.
Tested-by: Michele Locati <michele@locati.it>

We know that the use of filter-branch is not recommended, but we rely on --state-branch to split a repository incrementally, that is, processing only the new commits instead of the whole history at every run.

It would be great to have this fix merged.
A minor note: there's a typo in the subject ("fiter-branch").

Thanks, Michele

Patrick SteinhardtSep 30, 2026, 14:40 UTC in reply to Grant Moyer on lore

Re: [PATCH] fiter-branch: fix commit map init from state branch

On Fri, Jul 31, 2026 at 11:31:27PM -0400, Grant Moyer wrote:
Sorry, this slipped may radar. Thanks for bumping this thread, Michele.
Nit: pointed out by Michele: the subject has a typo in "fiter-branch".
Show 12 quoted lines
> The commit map dir is populated from the state branch assuming a
> "to_commit:from_commit" format, but the state branch is written with a
> "from_commit:to_commit" format, resulting in an inverted mapping when the
> map is populated from the state branch. This is especially evident when
> --prune-empty is used and creates commits which map to nothing; when the
> map dir is populated from this state on subsequent runs, git-filter-branch
> outputs many errors while trying to create files with empty names, like:
> 
> > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory
> 
> This change corrects the population of the commit map dir to match the
> "from_commit:to_commit" format.

So... does this mean that we don't have test coverage for this case at all?

It might make sense to also mention f6d855091e (filter-branch: stop depending on Perl, 2025-04-16) for context, as this is where the issue was introduced.

Show 15 quoted lines
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index 24fa317aaa..9aa07be6e1 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -302,7 +302,9 @@ then
>  		do
>  			case "$line" in
>  			*:*)
> -				echo "${line%:*}" >../map/"${line#*:}";;
> +				from_commit=${line%:*}
> +				to_commit=${line#*:}
> +				echo "$to_commit" >../map/"$from_commit";;
>  			*)
>  				die "Unable to load state from $state_branch:filter.map";;
>  			esac
Okay. A simpler fix could've been the following diff:

- echo "${line%:*}" >../map/"${line#*:}";; + echo "${line#*:}" >../map/"${line%:*}";;

But I guess it doesn't hurt to have proper naming here.
Show 12 quoted lines
> diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
> index 86011e7b1f..3934cc4a11 100755
> --- a/t/t7003-filter-branch.sh
> +++ b/t/t7003-filter-branch.sh
> @@ -121,7 +121,7 @@ W=$(git rev-parse HEAD)
>  test_expect_success 'using --state-branch to skip already rewritten commits' '
>  	test_when_finished git reset --hard $V &&
>  	git reset --hard $V &&
> -	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
> +	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
>  	test_cmp_rev $W HEAD
>  '
So does this now detect the issue? If so, it feels somewhat roundabout.

Michele, it seems like you have already invested some time into reproducing the issue. Can this maybe be put into a proper test case that directly exercises your issues (that is, the swapped entries and such)?

Thanks!
Patrick
Grant MoyerSep 30, 2026, 15:48 UTC in reply to Patrick Steinhardt on lore

Re: [PATCH] fiter-branch: fix commit map init from state branch

On September 30, 2026 10:40:06 AM EDT, Patrick Steinhardt <ps@pks.im> wrote:
>Okay. A simpler fix could've been the following diff:
>
>- echo "${line%:*}" >../map/"${line#*:}";;
>+ echo "${line#*:}" >../map/"${line%:*}";;

Yep, that's equivalent and was my first version, but I decided to err on the side of clearer intent. Plus, the more verbose version mirrors how the state branch is constructed further down in the script.

>So... does this mean that we don't have test coverage for this case at
>all?
Yeah, the test for this case was broken.
Show 9 quoted lines
>>  test_expect_success 'using --state-branch to skip already rewritten commits' '
>>  test_when_finished git reset --hard $V &&
>>  git reset --hard $V &&
>> - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
>> + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
>>  test_cmp_rev $W HEAD
>>  '
>
>So does this now detect the issue? If so, it feels somewhat roundabout.

The current test seemingly intends to check if the --tree-filter filter was run on already processed commits, but it only checks that the filter produces the same final result. Since the filter is deterministic, the final result is the same whether or not the filter is re-run on already processed commits.

The proposed change makes the test fail immediately if the tree-filter is re-run. I looked around other tests for a test_* command or conventions to fail with a message, but I didn't find anything. I've checked that the test fails without the filter-branch change, and passes with it.

>Nit: pointed out by Michele: the subject has a typo in "fiter-branch".

I'm new here. Is this something I should fix by submitting a new version of the patch, or will the maintainer fix it up if/when they merge the patch?

Grant
Patrick SteinhardtSep 30, 2026, 16:08 UTC in reply to Grant Moyer on lore

Re: [PATCH] fiter-branch: fix commit map init from state branch

On Wed, Sep 30, 2026 at 11:48:25AM -0400, Grant Moyer wrote:
Show 36 quoted lines
> On September 30, 2026 10:40:06 AM EDT, Patrick Steinhardt <ps@pks.im> wrote:
> > Okay. A simpler fix could've been the following diff:
> > 
> > - echo "${line%:*}" >../map/"${line#*:}";;
> > + echo "${line#*:}" >../map/"${line%:*}";;
> 
> Yep, that's equivalent and was my first version, but I decided to err on the
> side of clearer intent. Plus, the more verbose version mirrors how the state
> branch is constructed further down in the script.
> 
> > So... does this mean that we don't have test coverage for this case at
> > all?
> 
> Yeah, the test for this case was broken.
> 
> > >  test_expect_success 'using --state-branch to skip already rewritten commits' '
> > >  test_when_finished git reset --hard $V &&
> > >  git reset --hard $V &&
> > > - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
> > > + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
> > >  test_cmp_rev $W HEAD
> > >  '
> > 
> > So does this now detect the issue? If so, it feels somewhat roundabout.
> 
> The current test seemingly intends to check if the --tree-filter filter was
> run on already processed commits, but it only checks that the filter
> produces the same final result. Since the filter is deterministic, the final
> result is the same whether or not the filter is re-run on already processed
> commits.
> 
> The proposed change makes the test fail immediately if the tree-filter
> is re-run. I looked around other tests for a test_* command or
> conventions to fail with a message, but I didn't find anything. I've
> checked that the test fails without the filter-branch change, and passes
> with it.

The thing that I'm worried about is that the test seems to only coincidentally exercise the code. I think it may be helpful to have a test that explicitly tests for the scenarios that were reported as broken by Michele. Let's maybe wait for them to respond -- if we're lucky they already have a test we can reuse here that more directly exercises a couple of the reportedly-broken scenarios.

> > Nit: pointed out by Michele: the subject has a typo in "fiter-branch".
> 
> I'm new here. Is this something I should fix by submitting a new version of
> the patch, or will the maintainer fix it  up if/when they merge the patch?

It depends. For a small fix like this the maintainer may make the change themselves if nothing else needs to be changed. But Junio will often tell you in that case, so if he stays quiet without the series getting merged down, then I'd eventually just send a new version. Also helps to bump up the patch series on the mailing list again :)

Thanks!
Patrick
Michele LocatiSep 30, 2026, 20:38 UTC in reply to Patrick Steinhardt on lore

Re: [PATCH] fiter-branch: fix commit map init from state branch

Hi Patrick,
> Michele, it seems like you have already invested some time into
> reproducing the issue. Can this maybe be put into a proper test case
> that directly exercises your issues (that is, the swapped entries and
> such)?

Sure. The test below performs an incremental rewrite in two runs: the second run must attach the new commit to the parent rewritten by the first run, and the saved state must contain "original:rewritten" entries. Without the fix the new commit is attached to the original parent, and the test fails at test_cmp_rev.

I checked it against master (a018953): it fails without the fix and passes with it, and the other tests of t7003 still pass. It also passes with v2.49.1, the last version before the regression.

Grant, feel free to squash it into your patch.
Show changes to t/t7003-filter-branch.sh +22 −0
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -126,6 +126,27 @@ test_expect_success 'using --state-branch to skip already rewritten commits' '
 	test_cmp_rev $W HEAD
 '
 
+test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
+	git init incremental &&
+	(
+		cd incremental &&
+		mkdir sub &&
+		test_commit first sub/file &&
+		test_commit outside root-file &&
+		git filter-branch --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- HEAD &&
+		rewritten_first=$(git rev-parse HEAD) &&
+		git reset --hard outside &&
+		test_commit second sub/file &&
+		git filter-branch -f --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- outside..HEAD &&
+		test_cmp_rev $rewritten_first HEAD^ &&
+		git show refs/state:filter.map >map &&
+		echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
+		grep "^$(git rev-parse second):" map >actual &&
+		test_cmp expect actual
+	)
+'
+
 git tag oldD HEAD~4
 test_expect_success 'rewrite one branch, keeping a side branch' '
 	git branch modD oldD &&

Thanks,
Michele
Grant MoyerOct 1, 2026, 01:23 UTC in reply to Grant Moyer on lore

[PATCH v2] filter-branch: fix commit map init from state branch

The commit map dir is populated from the state branch assuming a "to_commit:from_commit" format, but the state branch is written with a "from_commit:to_commit" format, resulting in an inverted mapping when the map is populated from the state branch. This is especially evident when --prune-empty is used and creates commits which map to nothing; when the map dir is populated from this state on subsequent runs, git-filter-branch outputs many errors while trying to create files with empty names, like:

> /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory

This change corrects the population of the commit map dir to match the "from_commit:to_commit" format and adds/updates tests to check that the state branch is written correctly.

Signed-off-by: Grant Moyer <dev@grantmoyer.com>
Tested-by: Michele Locati <michele@locati.it>
Co-authored-by: Michele Locati <michele@locati.it>
---
 git-filter-branch.sh     |  4 +++-
 t/t7003-filter-branch.sh | 24 +++++++++++++++++++++++-
 2 files changed, 26 insertions(+), 2 deletions(-)
Show changes to 2 files +26 −2

git-filter-branch.sh, t/t7003-filter-branch.sh

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 24fa317aaa..9aa07be6e1 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -302,7 +302,9 @@ then
 		do
 			case "$line" in
 			*:*)
-				echo "${line%:*}" >../map/"${line#*:}";;
+				from_commit=${line%:*}
+				to_commit=${line#*:}
+				echo "$to_commit" >../map/"$from_commit";;
 			*)
 				die "Unable to load state from $state_branch:filter.map";;
 			esac
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 86011e7b1f..cf225b0f0f 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -121,10 +121,32 @@ W=$(git rev-parse HEAD)
 test_expect_success 'using --state-branch to skip already rewritten commits' '
 	test_when_finished git reset --hard $V &&
 	git reset --hard $V &&
-	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
+	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
 	test_cmp_rev $W HEAD
 '
 
+test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
+	git init incremental &&
+	(
+		cd incremental &&
+		mkdir sub &&
+		test_commit first sub/file &&
+		test_commit outside root-file &&
+		git filter-branch --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- HEAD &&
+		rewritten_first=$(git rev-parse HEAD) &&
+		git reset --hard outside &&
+		test_commit second sub/file &&
+		git filter-branch -f --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- outside..HEAD &&
+		test_cmp_rev $rewritten_first HEAD^ &&
+		git show refs/state:filter.map >map &&
+		echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
+		grep "^$(git rev-parse second):" map >actual &&
+		test_cmp expect actual
+	)
+'
+
 git tag oldD HEAD~4
 test_expect_success 'rewrite one branch, keeping a side branch' '
 	git branch modD oldD &&
-- 
2.55.0
Patrick SteinhardtOct 1, 2026, 05:36 UTC in reply to Grant Moyer on lore

Re: [PATCH v2] filter-branch: fix commit map init from state branch

On Wed, Sep 30, 2026 at 09:23:47PM -0400, Grant Moyer wrote:
Show 7 quoted lines
> The commit map dir is populated from the state branch assuming a
> "to_commit:from_commit" format, but the state branch is written with a
> "from_commit:to_commit" format, resulting in an inverted mapping when the
> map is populated from the state branch. This is especially evident when
> --prune-empty is used and creates commits which map to nothing; when the
> map dir is populated from this state on subsequent runs, git-filter-branch
> outputs many errors while trying to create files with empty names, like:
This still doesn't mention the commit that introduced the regression.
Show 5 quoted lines
> > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory
> 
> This change corrects the population of the commit map dir to match the
> "from_commit:to_commit" format and adds/updates tests to check that the
> state branch is written correctly.
Nit: we typically write commit messages in imperative mood, as if you
were instructing the code to change.
> Signed-off-by: Grant Moyer <dev@grantmoyer.com>
> Tested-by: Michele Locati <michele@locati.it>
> Co-authored-by: Michele Locati <michele@locati.it>

Your Signed-off-by should always go last because you're signing off on everything that comes before it.

How about this message:
  The "--state-branch" option asks git-filter-branch(1) to write a
  mapping from old to new objects into the branch. This map is a
  simple blob stored in "$state_branch:filter.map" with the format
  "$to_commit:$from_commit".
  In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), we
  have refactored git-filter-branch(1) to no longer require Perl. But as
  part of that change we accidentally started to interpret the above
  format in reverse when populating the map directory. This of course
  breaks incremental rewrites that use the commit map multiple times.
  But we don't seem to  have any tests for this feature, and as a
  consequence we didn't notice the regression.
  Fix this bug by interpreting the mappings in the correct order again.
  Tested-by: Michele Locati <michele@locati.it>
  Co-authored-by: Michele Locati <michele@locati.it>
  Signed-off-by: Grant Moyer <dev@grantmoyer.com>
Show 14 quoted lines
> diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
> index 86011e7b1f..cf225b0f0f 100755
> --- a/t/t7003-filter-branch.sh
> +++ b/t/t7003-filter-branch.sh
> @@ -121,10 +121,32 @@ W=$(git rev-parse HEAD)
>  test_expect_success 'using --state-branch to skip already rewritten commits' '
>  	test_when_finished git reset --hard $V &&
>  	git reset --hard $V &&
> -	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
> +	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
>  	test_cmp_rev $W HEAD
>  '
>  
> +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
We could add `test_when_finished rm -rf incremental` here.
Show 20 quoted lines
> +	git init incremental &&
> +	(
> +		cd incremental &&
> +		mkdir sub &&
> +		test_commit first sub/file &&
> +		test_commit outside root-file &&
> +		git filter-branch --state-branch refs/state \
> +			--prune-empty --subdirectory-filter sub -- HEAD &&
> +		rewritten_first=$(git rev-parse HEAD) &&
> +		git reset --hard outside &&
> +		test_commit second sub/file &&
> +		git filter-branch -f --state-branch refs/state \
> +			--prune-empty --subdirectory-filter sub -- outside..HEAD &&
> +		test_cmp_rev $rewritten_first HEAD^ &&
> +		git show refs/state:filter.map >map &&
> +		echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
> +		grep "^$(git rev-parse second):" map >actual &&
> +		test_cmp expect actual
> +	)
> +'
Thanks!
Patrick
Michele LocatiOct 1, 2026, 06:33 UTC in reply to Patrick Steinhardt on lore

Re: [PATCH v2] filter-branch: fix commit map init from state branch

Hi Patrick,
> simple blob stored in "$state_branch:filter.map" with the format
> "$to_commit:$from_commit".

Small correction: the format is "$from_commit:$to_commit" (the original commit first, then the rewritten one), that's what the fix restores.

The test_when_finished cleanup sounds good to me.
Since I'm listed as co-author:
Signed-off-by: Michele Locati <michele@locati.it>

Thanks, Michele

Junio C HamanoOct 1, 2026, 16:10 UTC in reply to Grant Moyer on lore

Re: [PATCH v2] filter-branch: fix commit map init from state branch

Grant Moyer <dev@grantmoyer.com> writes:
Show 37 quoted lines
> The commit map dir is populated from the state branch assuming a
> "to_commit:from_commit" format, but the state branch is written with a
> "from_commit:to_commit" format, resulting in an inverted mapping when the
> map is populated from the state branch. This is especially evident when
> --prune-empty is used and creates commits which map to nothing; when the
> map dir is populated from this state on subsequent runs, git-filter-branch
> outputs many errors while trying to create files with empty names, like:
>
>> /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory
>
> This change corrects the population of the commit map dir to match the
> "from_commit:to_commit" format and adds/updates tests to check that the
> state branch is written correctly.
>
> Signed-off-by: Grant Moyer <dev@grantmoyer.com>
> Tested-by: Michele Locati <michele@locati.it>
> Co-authored-by: Michele Locati <michele@locati.it>
> ---
>  git-filter-branch.sh     |  4 +++-
>  t/t7003-filter-branch.sh | 24 +++++++++++++++++++++++-
>  2 files changed, 26 insertions(+), 2 deletions(-)
>
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index 24fa317aaa..9aa07be6e1 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -302,7 +302,9 @@ then
>  		do
>  			case "$line" in
>  			*:*)
> -				echo "${line%:*}" >../map/"${line#*:}";;
> +				from_commit=${line%:*}
> +				to_commit=${line#*:}
> +				echo "$to_commit" >../map/"$from_commit";;
>  			*)
>  				die "Unable to load state from $state_branch:filter.map";;
>  			esac

I very much appreciate the clear description in the proposed log message above that makes what this change corrects very easy to understand.

Show 12 quoted lines
> diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
> index 86011e7b1f..cf225b0f0f 100755
> --- a/t/t7003-filter-branch.sh
> +++ b/t/t7003-filter-branch.sh
> @@ -121,10 +121,32 @@ W=$(git rev-parse HEAD)
>  test_expect_success 'using --state-branch to skip already rewritten commits' '
>  	test_when_finished git reset --hard $V &&
>  	git reset --hard $V &&
> -	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
> +	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
>  	test_cmp_rev $W HEAD
>  '

I am not sure what this change is about. Care to explain in the commit log?

> +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
It is a minor nit, but do we want to have
	test_when_finished "rm -fr incremental" &&

here, before creating a new repository that is used only for this test, or do we expect that in the future we will add more pieces of tests that work inside this new repository (in which case leaving it there may indeed be a better choice)?

Show 24 quoted lines
> +	git init incremental &&
> +	(
> +		cd incremental &&
> +		mkdir sub &&
> +		test_commit first sub/file &&
> +		test_commit outside root-file &&
> +		git filter-branch --state-branch refs/state \
> +			--prune-empty --subdirectory-filter sub -- HEAD &&
> +		rewritten_first=$(git rev-parse HEAD) &&
> +		git reset --hard outside &&
> +		test_commit second sub/file &&
> +		git filter-branch -f --state-branch refs/state \
> +			--prune-empty --subdirectory-filter sub -- outside..HEAD &&
> +		test_cmp_rev $rewritten_first HEAD^ &&
> +		git show refs/state:filter.map >map &&
> +		echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
> +		grep "^$(git rev-parse second):" map >actual &&
> +		test_cmp expect actual
> +	)
> +'
> +
>  git tag oldD HEAD~4
>  test_expect_success 'rewrite one branch, keeping a side branch' '
>  	git branch modD oldD &&
Thanks.
Grant MoyerOct 3, 2026, 01:01 UTC in reply to Grant Moyer on lore

[PATCH v3] filter-branch: fix commit map init from state branch

The "--state-branch" option asks git-filter-branch(1) to write a mapping from old to new objects into a branch to enable incremental processing of large histories. This object mapping is stored as a simple blob at "$state_branch:filter.map" with one object pair per line in the format "$from_commit:$to_commit". Before processing commits in a subsequent run, the state branch is used to populate an object mapping directory, where each file is named "map/$from_commit" and has contents "$to_commit".

In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), we refactored git-filter-branch(1) to no longer require Perl, but accidentally started to interpret object pairs in reverse (as "$to_commit:$from_commit") when populating the map directory from the state branch. This can cause all kinds of bad behavior from the state-branch being effectively ignored to previously filtered objects accidentally being mapped back to unfiltered objects. One especially evident case occurs when "git filter-branch --prune-empty ..." maps some commits to nothing, then on subsequent runs outputs many errors while trying to create files with empty names, like:

> /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory

This regression went unnoticed because, since the introduction of the only test for "--state-branch" in 709cfe848a (filter-branch: skip commits present on --state-branch, 2018-06-26), the test accidentally passes even if commits are not skipped. The test checks that after populating a state branch with git-filter-branch(1), then running it again with that state branch, the resulting filtered commits for the first run and second run match. However since the filter used is deterministic, the commits always match, even if the commits in the state branch are re-filtered.

Fix the population of the object mapping dir by interpreting object pairs as "$from_commit:$to_commit". Also fix the existing "--state-branch" test by directly exiting with a non-zero code if any commits from the state branch aren't skipped. Finally, add a new "--state-branch" test which directly checks that a commit from the state branch is used when incrementally filtering a repo.

Tested-by: Michele Locati <michele@locati.it>
Co-authored-by: Michele Locati <michele@locati.it>
Signed-off-by: Michele Locati <michele@locati.it>
Signed-off-by: Grant Moyer <dev@grantmoyer.com>
---
 git-filter-branch.sh     |  4 +++-
 t/t7003-filter-branch.sh | 25 ++++++++++++++++++++++++-
 2 files changed, 27 insertions(+), 2 deletions(-)
Show changes to 2 files +27 −2

git-filter-branch.sh, t/t7003-filter-branch.sh

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 24fa317aaa..9aa07be6e1 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -302,7 +302,9 @@ then
 		do
 			case "$line" in
 			*:*)
-				echo "${line%:*}" >../map/"${line#*:}";;
+				from_commit=${line%:*}
+				to_commit=${line#*:}
+				echo "$to_commit" >../map/"$from_commit";;
 			*)
 				die "Unable to load state from $state_branch:filter.map";;
 			esac
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 86011e7b1f..801cc83e5e 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -121,10 +121,33 @@ W=$(git rev-parse HEAD)
 test_expect_success 'using --state-branch to skip already rewritten commits' '
 	test_when_finished git reset --hard $V &&
 	git reset --hard $V &&
-	git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
+	git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
 	test_cmp_rev $W HEAD
 '
 
+test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
+	test_when_finished "rm -fr incremental" &&
+	git init incremental &&
+	(
+		cd incremental &&
+		mkdir sub &&
+		test_commit first sub/file &&
+		test_commit outside root-file &&
+		git filter-branch --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- HEAD &&
+		rewritten_first=$(git rev-parse HEAD) &&
+		git reset --hard outside &&
+		test_commit second sub/file &&
+		git filter-branch -f --state-branch refs/state \
+			--prune-empty --subdirectory-filter sub -- outside..HEAD &&
+		test_cmp_rev $rewritten_first HEAD^ &&
+		git show refs/state:filter.map >map &&
+		echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
+		grep "^$(git rev-parse second):" map >actual &&
+		test_cmp expect actual
+	)
+'
+
 git tag oldD HEAD~4
 test_expect_success 'rewrite one branch, keeping a side branch' '
 	git branch modD oldD &&
-- 
2.55.0
Patrick SteinhardtOct 5, 2026, 06:11 UTC in reply to Grant Moyer on lore

Re: [PATCH v3] filter-branch: fix commit map init from state branch

On Fri, Oct 02, 2026 at 09:01:27PM -0400, Grant Moyer wrote:
Show 43 quoted lines
> The "--state-branch" option asks git-filter-branch(1) to write a
> mapping from old to new objects into a branch to enable incremental
> processing of large histories. This object mapping is stored as
> a simple blob at "$state_branch:filter.map" with one object pair
> per line in the format "$from_commit:$to_commit". Before processing
> commits in a subsequent run, the state branch is used to populate an
> object mapping directory, where each file is named "map/$from_commit"
> and has contents "$to_commit".
> 
> In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16),
> we refactored git-filter-branch(1) to no longer require Perl,
> but accidentally started to interpret object pairs in reverse (as
> "$to_commit:$from_commit") when populating the map directory from
> the state branch. This can cause all kinds of bad behavior from the
> state-branch being effectively ignored to previously filtered objects
> accidentally being mapped back to unfiltered objects. One especially
> evident case occurs when "git filter-branch --prune-empty ..." maps
> some commits to nothing, then on subsequent runs outputs many errors
> while trying to create files with empty names, like:
> 
> > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory
> 
> This regression went unnoticed because, since the introduction of
> the only test for "--state-branch" in 709cfe848a (filter-branch: skip
> commits present on --state-branch, 2018-06-26), the test accidentally
> passes even if commits are not skipped. The test checks that after
> populating a state branch with git-filter-branch(1), then running
> it again with that state branch, the resulting filtered commits for
> the first run and second run match. However since the filter used is
> deterministic, the commits always match, even if the commits in the
> state branch are re-filtered.
> 
> Fix the population of the object mapping dir by interpreting
> object pairs as "$from_commit:$to_commit". Also fix the existing
> "--state-branch" test by directly exiting with a non-zero code if
> any commits from the state branch aren't skipped. Finally, add a new
> "--state-branch" test which directly checks that a commit from the
> state branch is used when incrementally filtering a repo.
> 
> Tested-by: Michele Locati <michele@locati.it>
> Co-authored-by: Michele Locati <michele@locati.it>
> Signed-off-by: Michele Locati <michele@locati.it>
> Signed-off-by: Grant Moyer <dev@grantmoyer.com>
Thanks, I'm happy with this version.
Patrick

Back to recent threads