# [PATCH] fiter-branch: fix commit map init from state branch 12 messages from 2026-08-01 to 2026-10-05. Participants: Grant Moyer, Michele Locati, Patrick Steinhardt, Junio C Hamano. Thread: https://gitlist.dev/t/66098 ## Grant Moyer, 2026-08-01 03:31 Subject: [PATCH] fiter-branch: fix commit map init from state branch Message-ID: <20260801033127.10606-1-dev@grantmoyer.com> ``` The commit map dir is populated from the state branch assuming a "to_commit:from_commit" format, but the state branch is written with a "from_commit:to_commit" format, resulting in an inverted mapping when the map is populated from the state branch. This is especially evident when --prune-empty is used and creates commits which map to nothing; when the map dir is populated from this state on subsequent runs, git-filter-branch outputs many errors while trying to create files with empty names, like: > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory This change corrects the population of the commit map dir to match the "from_commit:to_commit" format. Signed-off-by: Grant Moyer --- git-filter-branch.sh | 4 +++- t/t7003-filter-branch.sh | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/git-filter-branch.sh b/git-filter-branch.sh index 24fa317aaa..9aa07be6e1 100755 --- a/git-filter-branch.sh +++ b/git-filter-branch.sh @@ -302,7 +302,9 @@ then do case "$line" in *:*) - echo "${line%:*}" >../map/"${line#*:}";; + from_commit=${line%:*} + to_commit=${line#*:} + echo "$to_commit" >../map/"$from_commit";; *) die "Unable to load state from $state_branch:filter.map";; esac diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh index 86011e7b1f..3934cc4a11 100755 --- a/t/t7003-filter-branch.sh +++ b/t/t7003-filter-branch.sh @@ -121,7 +121,7 @@ W=$(git rev-parse HEAD) test_expect_success 'using --state-branch to skip already rewritten commits' ' test_when_finished git reset --hard $V && git reset --hard $V && - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && test_cmp_rev $W HEAD ' base-commit: a97fcc37c2bc6340a8d7ce78dedf227aac4e9aa7 -- 2.55.0 ``` ## Michele Locati, 2026-09-29 14:31 Subject: Re: [PATCH] fiter-branch: fix commit map init from state branch Message-ID: <20260929143150.2420-1-michele@locati.it> In-Reply-To: <20260801033127.10606-1-dev@grantmoyer.com> ``` Hi Grant, thanks for this patch: I hit the same bug, and I was about to report it. Cc'ing Patrick and Junio, since the problem was introduced by f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), released in 2.50.0. I tested your fix with git 2.56.0, running filter-branch incrementally with --state-branch, --prune-empty and --subdirectory-filter: - without the fix, the new commits are attached to the original parents instead of the rewritten ones, and swapped entries are saved back to the state branch; - with the fix, the parents are the rewritten commits and the state is saved correctly. Tested-by: Michele Locati We know that the use of filter-branch is not recommended, but we rely on --state-branch to split a repository incrementally, that is, processing only the new commits instead of the whole history at every run. It would be great to have this fix merged. A minor note: there's a typo in the subject ("fiter-branch"). Thanks, Michele ``` ## Patrick Steinhardt, 2026-09-30 14:40 Subject: Re: [PATCH] fiter-branch: fix commit map init from state branch Message-ID: In-Reply-To: <20260801033127.10606-1-dev@grantmoyer.com> ``` On Fri, Jul 31, 2026 at 11:31:27PM -0400, Grant Moyer wrote: Sorry, this slipped may radar. Thanks for bumping this thread, Michele. Nit: pointed out by Michele: the subject has a typo in "fiter-branch". > The commit map dir is populated from the state branch assuming a > "to_commit:from_commit" format, but the state branch is written with a > "from_commit:to_commit" format, resulting in an inverted mapping when the > map is populated from the state branch. This is especially evident when > --prune-empty is used and creates commits which map to nothing; when the > map dir is populated from this state on subsequent runs, git-filter-branch > outputs many errors while trying to create files with empty names, like: > > > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory > > This change corrects the population of the commit map dir to match the > "from_commit:to_commit" format. So... does this mean that we don't have test coverage for this case at all? It might make sense to also mention f6d855091e (filter-branch: stop depending on Perl, 2025-04-16) for context, as this is where the issue was introduced. > diff --git a/git-filter-branch.sh b/git-filter-branch.sh > index 24fa317aaa..9aa07be6e1 100755 > --- a/git-filter-branch.sh > +++ b/git-filter-branch.sh > @@ -302,7 +302,9 @@ then > do > case "$line" in > *:*) > - echo "${line%:*}" >../map/"${line#*:}";; > + from_commit=${line%:*} > + to_commit=${line#*:} > + echo "$to_commit" >../map/"$from_commit";; > *) > die "Unable to load state from $state_branch:filter.map";; > esac Okay. A simpler fix could've been the following diff: - echo "${line%:*}" >../map/"${line#*:}";; + echo "${line#*:}" >../map/"${line%:*}";; But I guess it doesn't hurt to have proper naming here. > diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh > index 86011e7b1f..3934cc4a11 100755 > --- a/t/t7003-filter-branch.sh > +++ b/t/t7003-filter-branch.sh > @@ -121,7 +121,7 @@ W=$(git rev-parse HEAD) > test_expect_success 'using --state-branch to skip already rewritten commits' ' > test_when_finished git reset --hard $V && > git reset --hard $V && > - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && > + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && > test_cmp_rev $W HEAD > ' So does this now detect the issue? If so, it feels somewhat roundabout. Michele, it seems like you have already invested some time into reproducing the issue. Can this maybe be put into a proper test case that directly exercises your issues (that is, the swapped entries and such)? Thanks! Patrick ``` ## Grant Moyer, 2026-09-30 15:48 Subject: Re: [PATCH] fiter-branch: fix commit map init from state branch Message-ID: In-Reply-To: ``` On September 30, 2026 10:40:06 AM EDT, Patrick Steinhardt wrote: >Okay. A simpler fix could've been the following diff: > >- echo "${line%:*}" >../map/"${line#*:}";; >+ echo "${line#*:}" >../map/"${line%:*}";; Yep, that's equivalent and was my first version, but I decided to err on the side of clearer intent. Plus, the more verbose version mirrors how the state branch is constructed further down in the script. >So... does this mean that we don't have test coverage for this case at >all? Yeah, the test for this case was broken. >> test_expect_success 'using --state-branch to skip already rewritten commits' ' >> test_when_finished git reset --hard $V && >> git reset --hard $V && >> - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && >> + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && >> test_cmp_rev $W HEAD >> ' > >So does this now detect the issue? If so, it feels somewhat roundabout. The current test seemingly intends to check if the --tree-filter filter was run on already processed commits, but it only checks that the filter produces the same final result. Since the filter is deterministic, the final result is the same whether or not the filter is re-run on already processed commits. The proposed change makes the test fail immediately if the tree-filter is re-run. I looked around other tests for a test_* command or conventions to fail with a message, but I didn't find anything. I've checked that the test fails without the filter-branch change, and passes with it. >Nit: pointed out by Michele: the subject has a typo in "fiter-branch". I'm new here. Is this something I should fix by submitting a new version of the patch, or will the maintainer fix it up if/when they merge the patch? Grant ``` ## Patrick Steinhardt, 2026-09-30 16:08 Subject: Re: [PATCH] fiter-branch: fix commit map init from state branch Message-ID: In-Reply-To: ``` On Wed, Sep 30, 2026 at 11:48:25AM -0400, Grant Moyer wrote: > On September 30, 2026 10:40:06 AM EDT, Patrick Steinhardt wrote: > > Okay. A simpler fix could've been the following diff: > > > > - echo "${line%:*}" >../map/"${line#*:}";; > > + echo "${line#*:}" >../map/"${line%:*}";; > > Yep, that's equivalent and was my first version, but I decided to err on the > side of clearer intent. Plus, the more verbose version mirrors how the state > branch is constructed further down in the script. > > > So... does this mean that we don't have test coverage for this case at > > all? > > Yeah, the test for this case was broken. > > > > test_expect_success 'using --state-branch to skip already rewritten commits' ' > > > test_when_finished git reset --hard $V && > > > git reset --hard $V && > > > - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && > > > + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && > > > test_cmp_rev $W HEAD > > > ' > > > > So does this now detect the issue? If so, it feels somewhat roundabout. > > The current test seemingly intends to check if the --tree-filter filter was > run on already processed commits, but it only checks that the filter > produces the same final result. Since the filter is deterministic, the final > result is the same whether or not the filter is re-run on already processed > commits. > > The proposed change makes the test fail immediately if the tree-filter > is re-run. I looked around other tests for a test_* command or > conventions to fail with a message, but I didn't find anything. I've > checked that the test fails without the filter-branch change, and passes > with it. The thing that I'm worried about is that the test seems to only coincidentally exercise the code. I think it may be helpful to have a test that explicitly tests for the scenarios that were reported as broken by Michele. Let's maybe wait for them to respond -- if we're lucky they already have a test we can reuse here that more directly exercises a couple of the reportedly-broken scenarios. > > Nit: pointed out by Michele: the subject has a typo in "fiter-branch". > > I'm new here. Is this something I should fix by submitting a new version of > the patch, or will the maintainer fix it up if/when they merge the patch? It depends. For a small fix like this the maintainer may make the change themselves if nothing else needs to be changed. But Junio will often tell you in that case, so if he stays quiet without the series getting merged down, then I'd eventually just send a new version. Also helps to bump up the patch series on the mailing list again :) Thanks! Patrick ``` ## Michele Locati, 2026-09-30 20:38 Subject: Re: [PATCH] fiter-branch: fix commit map init from state branch Message-ID: <20260930203822.958-1-michele@locati.it> In-Reply-To: ``` Hi Patrick, > Michele, it seems like you have already invested some time into > reproducing the issue. Can this maybe be put into a proper test case > that directly exercises your issues (that is, the swapped entries and > such)? Sure. The test below performs an incremental rewrite in two runs: the second run must attach the new commit to the parent rewritten by the first run, and the saved state must contain "original:rewritten" entries. Without the fix the new commit is attached to the original parent, and the test fails at test_cmp_rev. I checked it against master (a018953): it fails without the fix and passes with it, and the other tests of t7003 still pass. It also passes with v2.49.1, the last version before the regression. Grant, feel free to squash it into your patch. --- a/t/t7003-filter-branch.sh +++ b/t/t7003-filter-branch.sh @@ -126,6 +126,27 @@ test_expect_success 'using --state-branch to skip already rewritten commits' ' test_cmp_rev $W HEAD ' +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' ' + git init incremental && + ( + cd incremental && + mkdir sub && + test_commit first sub/file && + test_commit outside root-file && + git filter-branch --state-branch refs/state \ + --prune-empty --subdirectory-filter sub -- HEAD && + rewritten_first=$(git rev-parse HEAD) && + git reset --hard outside && + test_commit second sub/file && + git filter-branch -f --state-branch refs/state \ + --prune-empty --subdirectory-filter sub -- outside..HEAD && + test_cmp_rev $rewritten_first HEAD^ && + git show refs/state:filter.map >map && + echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect && + grep "^$(git rev-parse second):" map >actual && + test_cmp expect actual + ) +' + git tag oldD HEAD~4 test_expect_success 'rewrite one branch, keeping a side branch' ' git branch modD oldD && Thanks, Michele ``` ## Grant Moyer, 2026-10-01 01:23 Subject: [PATCH v2] filter-branch: fix commit map init from state branch Message-ID: <20261001012347.3998801-1-dev@grantmoyer.com> In-Reply-To: <20260801033127.10606-1-dev@grantmoyer.com> ``` The commit map dir is populated from the state branch assuming a "to_commit:from_commit" format, but the state branch is written with a "from_commit:to_commit" format, resulting in an inverted mapping when the map is populated from the state branch. This is especially evident when --prune-empty is used and creates commits which map to nothing; when the map dir is populated from this state on subsequent runs, git-filter-branch outputs many errors while trying to create files with empty names, like: > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory This change corrects the population of the commit map dir to match the "from_commit:to_commit" format and adds/updates tests to check that the state branch is written correctly. Signed-off-by: Grant Moyer Tested-by: Michele Locati Co-authored-by: Michele Locati --- git-filter-branch.sh | 4 +++- t/t7003-filter-branch.sh | 24 +++++++++++++++++++++++- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/git-filter-branch.sh b/git-filter-branch.sh index 24fa317aaa..9aa07be6e1 100755 --- a/git-filter-branch.sh +++ b/git-filter-branch.sh @@ -302,7 +302,9 @@ then do case "$line" in *:*) - echo "${line%:*}" >../map/"${line#*:}";; + from_commit=${line%:*} + to_commit=${line#*:} + echo "$to_commit" >../map/"$from_commit";; *) die "Unable to load state from $state_branch:filter.map";; esac diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh index 86011e7b1f..cf225b0f0f 100755 --- a/t/t7003-filter-branch.sh +++ b/t/t7003-filter-branch.sh @@ -121,10 +121,32 @@ W=$(git rev-parse HEAD) test_expect_success 'using --state-branch to skip already rewritten commits' ' test_when_finished git reset --hard $V && git reset --hard $V && - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && test_cmp_rev $W HEAD ' +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' ' + git init incremental && + ( + cd incremental && + mkdir sub && + test_commit first sub/file && + test_commit outside root-file && + git filter-branch --state-branch refs/state \ + --prune-empty --subdirectory-filter sub -- HEAD && + rewritten_first=$(git rev-parse HEAD) && + git reset --hard outside && + test_commit second sub/file && + git filter-branch -f --state-branch refs/state \ + --prune-empty --subdirectory-filter sub -- outside..HEAD && + test_cmp_rev $rewritten_first HEAD^ && + git show refs/state:filter.map >map && + echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect && + grep "^$(git rev-parse second):" map >actual && + test_cmp expect actual + ) +' + git tag oldD HEAD~4 test_expect_success 'rewrite one branch, keeping a side branch' ' git branch modD oldD && -- 2.55.0 ``` ## Patrick Steinhardt, 2026-10-01 05:36 Subject: Re: [PATCH v2] filter-branch: fix commit map init from state branch Message-ID: In-Reply-To: <20261001012347.3998801-1-dev@grantmoyer.com> ``` On Wed, Sep 30, 2026 at 09:23:47PM -0400, Grant Moyer wrote: > The commit map dir is populated from the state branch assuming a > "to_commit:from_commit" format, but the state branch is written with a > "from_commit:to_commit" format, resulting in an inverted mapping when the > map is populated from the state branch. This is especially evident when > --prune-empty is used and creates commits which map to nothing; when the > map dir is populated from this state on subsequent runs, git-filter-branch > outputs many errors while trying to create files with empty names, like: This still doesn't mention the commit that introduced the regression. > > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory > > This change corrects the population of the commit map dir to match the > "from_commit:to_commit" format and adds/updates tests to check that the > state branch is written correctly. Nit: we typically write commit messages in imperative mood, as if you were instructing the code to change. > Signed-off-by: Grant Moyer > Tested-by: Michele Locati > Co-authored-by: Michele Locati Your Signed-off-by should always go last because you're signing off on everything that comes before it. How about this message: The "--state-branch" option asks git-filter-branch(1) to write a mapping from old to new objects into the branch. This map is a simple blob stored in "$state_branch:filter.map" with the format "$to_commit:$from_commit". In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), we have refactored git-filter-branch(1) to no longer require Perl. But as part of that change we accidentally started to interpret the above format in reverse when populating the map directory. This of course breaks incremental rewrites that use the commit map multiple times. But we don't seem to have any tests for this feature, and as a consequence we didn't notice the regression. Fix this bug by interpreting the mappings in the correct order again. Tested-by: Michele Locati Co-authored-by: Michele Locati Signed-off-by: Grant Moyer > diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh > index 86011e7b1f..cf225b0f0f 100755 > --- a/t/t7003-filter-branch.sh > +++ b/t/t7003-filter-branch.sh > @@ -121,10 +121,32 @@ W=$(git rev-parse HEAD) > test_expect_success 'using --state-branch to skip already rewritten commits' ' > test_when_finished git reset --hard $V && > git reset --hard $V && > - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && > + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && > test_cmp_rev $W HEAD > ' > > +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' ' We could add `test_when_finished rm -rf incremental` here. > + git init incremental && > + ( > + cd incremental && > + mkdir sub && > + test_commit first sub/file && > + test_commit outside root-file && > + git filter-branch --state-branch refs/state \ > + --prune-empty --subdirectory-filter sub -- HEAD && > + rewritten_first=$(git rev-parse HEAD) && > + git reset --hard outside && > + test_commit second sub/file && > + git filter-branch -f --state-branch refs/state \ > + --prune-empty --subdirectory-filter sub -- outside..HEAD && > + test_cmp_rev $rewritten_first HEAD^ && > + git show refs/state:filter.map >map && > + echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect && > + grep "^$(git rev-parse second):" map >actual && > + test_cmp expect actual > + ) > +' Thanks! Patrick ``` ## Michele Locati, 2026-10-01 06:33 Subject: Re: [PATCH v2] filter-branch: fix commit map init from state branch Message-ID: <20261001063306.616-1-michele@locati.it> In-Reply-To: ``` Hi Patrick, > simple blob stored in "$state_branch:filter.map" with the format > "$to_commit:$from_commit". Small correction: the format is "$from_commit:$to_commit" (the original commit first, then the rewritten one), that's what the fix restores. The test_when_finished cleanup sounds good to me. Since I'm listed as co-author: Signed-off-by: Michele Locati Thanks, Michele ``` ## Junio C Hamano, 2026-10-01 16:10 Subject: Re: [PATCH v2] filter-branch: fix commit map init from state branch Message-ID: In-Reply-To: <20261001012347.3998801-1-dev@grantmoyer.com> ``` Grant Moyer writes: > The commit map dir is populated from the state branch assuming a > "to_commit:from_commit" format, but the state branch is written with a > "from_commit:to_commit" format, resulting in an inverted mapping when the > map is populated from the state branch. This is especially evident when > --prune-empty is used and creates commits which map to nothing; when the > map dir is populated from this state on subsequent runs, git-filter-branch > outputs many errors while trying to create files with empty names, like: > >> /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory > > This change corrects the population of the commit map dir to match the > "from_commit:to_commit" format and adds/updates tests to check that the > state branch is written correctly. > > Signed-off-by: Grant Moyer > Tested-by: Michele Locati > Co-authored-by: Michele Locati > --- > git-filter-branch.sh | 4 +++- > t/t7003-filter-branch.sh | 24 +++++++++++++++++++++++- > 2 files changed, 26 insertions(+), 2 deletions(-) > > diff --git a/git-filter-branch.sh b/git-filter-branch.sh > index 24fa317aaa..9aa07be6e1 100755 > --- a/git-filter-branch.sh > +++ b/git-filter-branch.sh > @@ -302,7 +302,9 @@ then > do > case "$line" in > *:*) > - echo "${line%:*}" >../map/"${line#*:}";; > + from_commit=${line%:*} > + to_commit=${line#*:} > + echo "$to_commit" >../map/"$from_commit";; > *) > die "Unable to load state from $state_branch:filter.map";; > esac I very much appreciate the clear description in the proposed log message above that makes what this change corrects very easy to understand. > diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh > index 86011e7b1f..cf225b0f0f 100755 > --- a/t/t7003-filter-branch.sh > +++ b/t/t7003-filter-branch.sh > @@ -121,10 +121,32 @@ W=$(git rev-parse HEAD) > test_expect_success 'using --state-branch to skip already rewritten commits' ' > test_when_finished git reset --hard $V && > git reset --hard $V && > - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && > + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && > test_cmp_rev $W HEAD > ' I am not sure what this change is about. Care to explain in the commit log? > +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' ' It is a minor nit, but do we want to have test_when_finished "rm -fr incremental" && here, before creating a new repository that is used only for this test, or do we expect that in the future we will add more pieces of tests that work inside this new repository (in which case leaving it there may indeed be a better choice)? > + git init incremental && > + ( > + cd incremental && > + mkdir sub && > + test_commit first sub/file && > + test_commit outside root-file && > + git filter-branch --state-branch refs/state \ > + --prune-empty --subdirectory-filter sub -- HEAD && > + rewritten_first=$(git rev-parse HEAD) && > + git reset --hard outside && > + test_commit second sub/file && > + git filter-branch -f --state-branch refs/state \ > + --prune-empty --subdirectory-filter sub -- outside..HEAD && > + test_cmp_rev $rewritten_first HEAD^ && > + git show refs/state:filter.map >map && > + echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect && > + grep "^$(git rev-parse second):" map >actual && > + test_cmp expect actual > + ) > +' > + > git tag oldD HEAD~4 > test_expect_success 'rewrite one branch, keeping a side branch' ' > git branch modD oldD && Thanks. ``` ## Grant Moyer, 2026-10-03 01:01 Subject: [PATCH v3] filter-branch: fix commit map init from state branch Message-ID: <20261003010128.256757-1-dev@grantmoyer.com> In-Reply-To: <20261001012347.3998801-1-dev@grantmoyer.com> ``` The "--state-branch" option asks git-filter-branch(1) to write a mapping from old to new objects into a branch to enable incremental processing of large histories. This object mapping is stored as a simple blob at "$state_branch:filter.map" with one object pair per line in the format "$from_commit:$to_commit". Before processing commits in a subsequent run, the state branch is used to populate an object mapping directory, where each file is named "map/$from_commit" and has contents "$to_commit". In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), we refactored git-filter-branch(1) to no longer require Perl, but accidentally started to interpret object pairs in reverse (as "$to_commit:$from_commit") when populating the map directory from the state branch. This can cause all kinds of bad behavior from the state-branch being effectively ignored to previously filtered objects accidentally being mapped back to unfiltered objects. One especially evident case occurs when "git filter-branch --prune-empty ..." maps some commits to nothing, then on subsequent runs outputs many errors while trying to create files with empty names, like: > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory This regression went unnoticed because, since the introduction of the only test for "--state-branch" in 709cfe848a (filter-branch: skip commits present on --state-branch, 2018-06-26), the test accidentally passes even if commits are not skipped. The test checks that after populating a state branch with git-filter-branch(1), then running it again with that state branch, the resulting filtered commits for the first run and second run match. However since the filter used is deterministic, the commits always match, even if the commits in the state branch are re-filtered. Fix the population of the object mapping dir by interpreting object pairs as "$from_commit:$to_commit". Also fix the existing "--state-branch" test by directly exiting with a non-zero code if any commits from the state branch aren't skipped. Finally, add a new "--state-branch" test which directly checks that a commit from the state branch is used when incrementally filtering a repo. Tested-by: Michele Locati Co-authored-by: Michele Locati Signed-off-by: Michele Locati Signed-off-by: Grant Moyer --- git-filter-branch.sh | 4 +++- t/t7003-filter-branch.sh | 25 ++++++++++++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/git-filter-branch.sh b/git-filter-branch.sh index 24fa317aaa..9aa07be6e1 100755 --- a/git-filter-branch.sh +++ b/git-filter-branch.sh @@ -302,7 +302,9 @@ then do case "$line" in *:*) - echo "${line%:*}" >../map/"${line#*:}";; + from_commit=${line%:*} + to_commit=${line#*:} + echo "$to_commit" >../map/"$from_commit";; *) die "Unable to load state from $state_branch:filter.map";; esac diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh index 86011e7b1f..801cc83e5e 100755 --- a/t/t7003-filter-branch.sh +++ b/t/t7003-filter-branch.sh @@ -121,10 +121,33 @@ W=$(git rev-parse HEAD) test_expect_success 'using --state-branch to skip already rewritten commits' ' test_when_finished git reset --hard $V && git reset --hard $V && - git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD && + git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD && test_cmp_rev $W HEAD ' +test_expect_success '--state-branch incremental rewrite uses the rewritten parents' ' + test_when_finished "rm -fr incremental" && + git init incremental && + ( + cd incremental && + mkdir sub && + test_commit first sub/file && + test_commit outside root-file && + git filter-branch --state-branch refs/state \ + --prune-empty --subdirectory-filter sub -- HEAD && + rewritten_first=$(git rev-parse HEAD) && + git reset --hard outside && + test_commit second sub/file && + git filter-branch -f --state-branch refs/state \ + --prune-empty --subdirectory-filter sub -- outside..HEAD && + test_cmp_rev $rewritten_first HEAD^ && + git show refs/state:filter.map >map && + echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect && + grep "^$(git rev-parse second):" map >actual && + test_cmp expect actual + ) +' + git tag oldD HEAD~4 test_expect_success 'rewrite one branch, keeping a side branch' ' git branch modD oldD && -- 2.55.0 ``` ## Patrick Steinhardt, 2026-10-05 06:11 Subject: Re: [PATCH v3] filter-branch: fix commit map init from state branch Message-ID: In-Reply-To: <20261003010128.256757-1-dev@grantmoyer.com> ``` On Fri, Oct 02, 2026 at 09:01:27PM -0400, Grant Moyer wrote: > The "--state-branch" option asks git-filter-branch(1) to write a > mapping from old to new objects into a branch to enable incremental > processing of large histories. This object mapping is stored as > a simple blob at "$state_branch:filter.map" with one object pair > per line in the format "$from_commit:$to_commit". Before processing > commits in a subsequent run, the state branch is used to populate an > object mapping directory, where each file is named "map/$from_commit" > and has contents "$to_commit". > > In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16), > we refactored git-filter-branch(1) to no longer require Perl, > but accidentally started to interpret object pairs in reverse (as > "$to_commit:$from_commit") when populating the map directory from > the state branch. This can cause all kinds of bad behavior from the > state-branch being effectively ignored to previously filtered objects > accidentally being mapped back to unfiltered objects. One especially > evident case occurs when "git filter-branch --prune-empty ..." maps > some commits to nothing, then on subsequent runs outputs many errors > while trying to create files with empty names, like: > > > /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory > > This regression went unnoticed because, since the introduction of > the only test for "--state-branch" in 709cfe848a (filter-branch: skip > commits present on --state-branch, 2018-06-26), the test accidentally > passes even if commits are not skipped. The test checks that after > populating a state branch with git-filter-branch(1), then running > it again with that state branch, the resulting filtered commits for > the first run and second run match. However since the filter used is > deterministic, the commits always match, even if the commits in the > state branch are re-filtered. > > Fix the population of the object mapping dir by interpreting > object pairs as "$from_commit:$to_commit". Also fix the existing > "--state-branch" test by directly exiting with a non-zero code if > any commits from the state branch aren't skipped. Finally, add a new > "--state-branch" test which directly checks that a commit from the > state branch is used when incrementally filtering a repo. > > Tested-by: Michele Locati > Co-authored-by: Michele Locati > Signed-off-by: Michele Locati > Signed-off-by: Grant Moyer Thanks, I'm happy with this version. Patrick ```