# [PATCH] rerere: keep a background gc from killing a rebase 37 messages from 2026-09-02 to 2026-10-02. Participants: Thomas Bachem via GitGitGadget, Phillip Wood, Thomas Bachem, Patrick Steinhardt, Junio C Hamano. Thread: https://gitlist.dev/t/66250 ## Thomas Bachem via GitGitGadget, 2026-09-02 08:31 Subject: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: ``` From: Thomas Bachem Since 2.54 unscheduled maintenance uses the "geometric" strategy, so the "git maintenance run --auto --detach" behind every "git commit" runs "git rerere gc" in the background whenever rr-cache has an entry. That includes the "git commit" the sequencer runs for a resolved pick on "git rebase --continue". rerere_gc() takes MERGE_RR.lock through setup_rerere(), which uses LOCK_DIE_ON_ERROR, and so does the sequencer's repo_rerere() at the next conflict a few milliseconds later. Whichever comes second dies. When it is the rebase, it dies in do_pick_commit() with the index written but before make_patch() writes rebase-merge/{message,patch, stopped-sha}, and every later "git rebase --continue" refuses with "you have staged changes in your working tree". When it is the "git commit" of a later continue, that one dies in its post-commit repo_rerere() after the commit was made. Before 2.54 the same collision needed an auto gc to actually run, since gc runs "rerere gc" at its end. A rebase with two conflicts in a row shows it. The filler makes the pick slower than the ~5 ms the background task needs to take the lock, and keeps the lock held for about 0.4 s. It hit 6 of 6 runs here on 2.55.0, and a test suite driving rebases on toy repositories with a single rr-cache entry hit it in both runs that were traced: git init -q -b main r && cd r git config rerere.enabled true git config maintenance.auto false mkdir pad && seq 20000 | (cd pad && split -l 1 -a 5) echo base >f && git add -A && git commit -qm base git checkout -q -b topic echo b >f && git commit -qam B echo c >f && git commit -qam C git checkout -q main echo a >f && git commit -qam A git repack -adq seq 20000 | awk '{printf ".git/rr-cache/%040x\n", $1}' \ | xargs mkdir -p for d in .git/rr-cache/*/; do echo x >$d/preimage; done git config --unset maintenance.auto git checkout -q topic git rebase main echo ab >f && git add f GIT_EDITOR=true git rebase --continue The second continue dies with "Unable to create '.git/MERGE_RR.lock': File exists" while the gc spawned by its own commit holds the lock, and after resolving C every further continue refuses. Maintenance stays off during the setup so that no repack is pending: a repack due at that commit runs ahead of rerere-gc in the task list and would spend the window. The gc needs the lock: it removes every rr-cache directory it finds empty, and a rerere that has just created its directory but not yet written the preimage looks exactly like that. So keep the lock and fix both orders. When the gc finds the lock busy, let it warn and do nothing this time, the way "maintenance run" treats its own lock, so a manual "git rerere gc" sees the warning and the maintenance task and "git gc" see a clean exit. When the gc holds the lock, let every other caller wait it out instead of dying at once, for rerere.lockTimeout milliseconds with the semantics of core.packedRefsTimeout: 1000 by default, 0 for the old behaviour, -1 for an unbounded wait. Walking a 20000-entry rr-cache takes about 0.4 s here. That rebase now completes. The tests cover the gc under a held lock, directly and through the maintenance task, a merge that waits a lock out within a five second rerere.lockTimeout, and one that fails at once with a timeout of 0. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- rerere: keep a background gc from killing a rebase Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v1 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v1 Pull-Request: https://github.com/gitgitgadget/git/pull/2214 Documentation/config/rerere.adoc | 8 +++++++ Documentation/git-rerere.adoc | 4 +++- rerere.c | 27 +++++++++++++++++---- rerere.h | 1 + t/t4200-rerere.sh | 40 ++++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 8 +++++++ 6 files changed, 82 insertions(+), 6 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 3a78b5ebb1..8041a1587b 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -10,3 +10,11 @@ rerere.enabled:: enabled if there is an `rr-cache` directory under the `$GIT_DIR`, e.g. if "rerere" was previously used in the repository. + +rerere.lockTimeout:: + The length of time, in milliseconds, to retry when trying to + take the rerere lock while another process holds it, typically + a background `git rerere gc`. Value 0 means not to retry at + all; -1 means to try indefinitely. Default is 1000 (i.e., + retry for 1 second). `git rerere gc` itself does not wait and + skips its run instead. diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc index 4e6ab9a27c..05935b0603 100644 --- a/Documentation/git-rerere.adoc +++ b/Documentation/git-rerere.adoc @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older than 15 days and resolved conflicts older than 60 days are pruned. These defaults are controlled via the `gc.rerereUnresolved` and `gc.rerereResolved` configuration -variables respectively. +variables respectively. If another process holds the lock on the +recorded resolutions, for example a merge or rebase that is recording +a conflict, `gc` does nothing and reports so. DISCUSSION diff --git a/rerere.c b/rerere.c index 8232542585..22d114262b 100644 --- a/rerere.c +++ b/rerere.c @@ -32,6 +32,7 @@ static int rerere_enabled = -1; /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; +static int rerere_lock_timeout_ms = 1000; #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 @@ -876,6 +877,8 @@ static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); + repo_config_get_int(the_repository, "rerere.locktimeout", + &rerere_lock_timeout_ms); repo_config(the_repository, git_default_config, NULL); } @@ -908,12 +911,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) + if (flags & RERERE_READONLY) { fd = 0; - else + } else if (flags & RERERE_SKIP_LOCKED) { fd = hold_lock_file_for_update(&write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + git_path_merge_rr(r), 0); + if (fd < 0) { + warning_errno(_("unable to lock '%s', skipping"), + git_path_merge_rr(r)); + return -1; + } + } else { + /* + * A background "rerere gc" holds the lock for as long as it + * takes to walk rr-cache, so wait it out rather than die. + */ + fd = hold_lock_file_for_update_timeout(&write_lock, + git_path_merge_rr(r), + LOCK_DIE_ON_ERROR, + rerere_lock_timeout_ms); + } read_rr(r, merge_rr); return fd; } @@ -1237,7 +1254,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve = now - 60 * 86400; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) + if (setup_rerere(r, rr, RERERE_SKIP_LOCKED) < 0) return; repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", diff --git a/rerere.h b/rerere.h index d4b5f7c932..87964bb3c5 100644 --- a/rerere.h +++ b/rerere.h @@ -10,6 +10,7 @@ struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 +#define RERERE_SKIP_LOCKED 010 /* * Marks paths that have been hand-resolved and added to the diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 1717f407c8..6b90294435 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,46 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'gc does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc 2>err && + test_grep "MERGE_RR" err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && + git rerere gc && + test_path_is_missing $rr2/preimage +' + +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { + (sleep 1 && rm -f .git/MERGE_RR.lock) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && + test_grep ! "Unable to create" err && + grep "^=======\$" $rr/preimage +' + +test_expect_success 'rerere.lockTimeout=0 fails at once on a held lock' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && + test_grep "Unable to create" err && + test_path_is_missing $rr/preimage +' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index d7f82e1bec..a55ca2e829 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc ' +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && + mkdir .git/rr-cache && + : >.git/rr-cache/entry && + >.git/MERGE_RR.lock && + test_expect_rerere_gc git maintenance run --task=rerere-gc +' + test_expect_success '--auto and --schedule incompatible' ' test_must_fail git maintenance run --auto --schedule=daily 2>err && test_grep "cannot be used together" err base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc -- gitgitgadget ``` ## Phillip Wood, 2026-09-02 13:27 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Thomas On 02/09/2026 09:31, Thomas Bachem via GitGitGadget wrote: > From: Thomas Bachem > > Since 2.54 unscheduled maintenance uses the "geometric" strategy, so That change really is the gift that keeps on giving > the "git maintenance run --auto --detach" behind every "git commit" > runs "git rerere gc" in the background whenever rr-cache has an entry. > That includes the "git commit" the sequencer runs for a resolved pick > on "git rebase --continue". > > rerere_gc() takes MERGE_RR.lock through setup_rerere(), which uses > LOCK_DIE_ON_ERROR, and so does the sequencer's repo_rerere() at the > next conflict a few milliseconds later. Whichever comes second dies. To me this is another reason why we should disable gc.auto while rebasing. To do that we need to pass "-c gc.auto=false -c maintenance.auto=false" when running "git commit" in run_git_commit() and also when running "git merge" in do_merge(). We should also pass those settings via GIT_CONFIG_PARAMETERS when running a exec command in do_exec(). That is largly papering over the cracks but until we have a systematic solution it does at least stop exposing users to this bug. > When it is the rebase, it dies in do_pick_commit() That's a bug us well - we should be returning errors, not dying -rerere_setup() should be returning an error, so we can clean up and reschedule the pick. There is a lot of detail here about what causes the problem which is helpful, but there is very little discussion about the fix. As I understand it we now block the sequencer until the background maintenance has completed, or continue to die in an inconvenient state we timeout before the background maintenance finishes. That seems rather unfortunate as the idea of running the maintenance in the background is to prevent it from interfering with other commands. I think my preferred solution is to disable gc while rebasing. Returning an error from rerere_setup() would also help in the case where the user runs "git commit" and then continues the rebase. I'd be interested to hear what Junio and Patrick think about that. I'm also not clear why gc.auto has to fork a separate process just to check if it needs to run or not, I've not been following closely but my impression is that that is the cause of quite a lot of the lock contention bugs we've seen. Thanks Phillip > with the index > written but before make_patch() writes rebase-merge/{message,patch, > stopped-sha}, and every later "git rebase --continue" refuses with > "you have staged changes in your working tree". When it is the "git > commit" of a later continue, that one dies in its post-commit > repo_rerere() after the commit was made. Before 2.54 the same > collision needed an auto gc to actually run, since gc runs > "rerere gc" at its end. > > A rebase with two conflicts in a row shows it. The filler makes the > pick slower than the ~5 ms the background task needs to take the > lock, and keeps the lock held for about 0.4 s. It hit 6 of 6 runs > here on 2.55.0, and a test suite driving rebases on toy repositories > with a single rr-cache entry hit it in both runs that were traced: > > git init -q -b main r && cd r > git config rerere.enabled true > git config maintenance.auto false > mkdir pad && seq 20000 | (cd pad && split -l 1 -a 5) > echo base >f && git add -A && git commit -qm base > git checkout -q -b topic > echo b >f && git commit -qam B > echo c >f && git commit -qam C > git checkout -q main > echo a >f && git commit -qam A > git repack -adq > seq 20000 | awk '{printf ".git/rr-cache/%040x\n", $1}' \ > | xargs mkdir -p > for d in .git/rr-cache/*/; do echo x >$d/preimage; done > git config --unset maintenance.auto > git checkout -q topic > git rebase main > echo ab >f && git add f > GIT_EDITOR=true git rebase --continue > > The second continue dies with "Unable to create '.git/MERGE_RR.lock': > File exists" while the gc spawned by its own commit holds the lock, > and after resolving C every further continue refuses. Maintenance > stays off during the setup so that no repack is pending: a repack due > at that commit runs ahead of rerere-gc in the task list and would > spend the window. > > The gc needs the lock: it removes every rr-cache directory it finds > empty, and a rerere that has just created its directory but not yet > written the preimage looks exactly like that. So keep the lock and fix > both orders. When the gc finds the lock busy, let it warn and do > nothing this time, the way "maintenance run" treats its own lock, so a > manual "git rerere gc" sees the warning and the maintenance task and > "git gc" see a clean exit. When the gc holds the lock, let every other > caller wait it out instead of dying at once, for rerere.lockTimeout > milliseconds with the semantics of core.packedRefsTimeout: 1000 by > default, 0 for the old behaviour, -1 for an unbounded wait. Walking a > 20000-entry rr-cache takes about 0.4 s here. > > That rebase now completes. The tests cover the gc under a held lock, > directly and through the maintenance task, a merge that waits a lock > out within a five second rerere.lockTimeout, and one that fails at > once with a timeout of 0. > > Assisted-by: Claude Fable 5.1 > Signed-off-by: Thomas Bachem > --- > rerere: keep a background gc from killing a rebase > > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v1 > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v1 > Pull-Request: https://github.com/gitgitgadget/git/pull/2214 > > Documentation/config/rerere.adoc | 8 +++++++ > Documentation/git-rerere.adoc | 4 +++- > rerere.c | 27 +++++++++++++++++---- > rerere.h | 1 + > t/t4200-rerere.sh | 40 ++++++++++++++++++++++++++++++++ > t/t7900-maintenance.sh | 8 +++++++ > 6 files changed, 82 insertions(+), 6 deletions(-) > > diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc > index 3a78b5ebb1..8041a1587b 100644 > --- a/Documentation/config/rerere.adoc > +++ b/Documentation/config/rerere.adoc > @@ -10,3 +10,11 @@ rerere.enabled:: > enabled if there is an `rr-cache` directory under the > `$GIT_DIR`, e.g. if "rerere" was previously used in the > repository. > + > +rerere.lockTimeout:: > + The length of time, in milliseconds, to retry when trying to > + take the rerere lock while another process holds it, typically > + a background `git rerere gc`. Value 0 means not to retry at > + all; -1 means to try indefinitely. Default is 1000 (i.e., > + retry for 1 second). `git rerere gc` itself does not wait and > + skips its run instead. > diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc > index 4e6ab9a27c..05935b0603 100644 > --- a/Documentation/git-rerere.adoc > +++ b/Documentation/git-rerere.adoc > @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older > than 15 days and resolved conflicts older than 60 > days are pruned. These defaults are controlled via the > `gc.rerereUnresolved` and `gc.rerereResolved` configuration > -variables respectively. > +variables respectively. If another process holds the lock on the > +recorded resolutions, for example a merge or rebase that is recording > +a conflict, `gc` does nothing and reports so. > > > DISCUSSION > diff --git a/rerere.c b/rerere.c > index 8232542585..22d114262b 100644 > --- a/rerere.c > +++ b/rerere.c > @@ -32,6 +32,7 @@ static int rerere_enabled = -1; > > /* automatically update cleanly resolved paths to the index */ > static int rerere_autoupdate; > +static int rerere_lock_timeout_ms = 1000; > > #define RR_HAS_POSTIMAGE 1 > #define RR_HAS_PREIMAGE 2 > @@ -876,6 +877,8 @@ static void git_rerere_config(void) > { > repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); > repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); > + repo_config_get_int(the_repository, "rerere.locktimeout", > + &rerere_lock_timeout_ms); > repo_config(the_repository, git_default_config, NULL); > } > > @@ -908,12 +911,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) > > if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) > rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); > - if (flags & RERERE_READONLY) > + if (flags & RERERE_READONLY) { > fd = 0; > - else > + } else if (flags & RERERE_SKIP_LOCKED) { > fd = hold_lock_file_for_update(&write_lock, > - git_path_merge_rr(r), > - LOCK_DIE_ON_ERROR); > + git_path_merge_rr(r), 0); > + if (fd < 0) { > + warning_errno(_("unable to lock '%s', skipping"), > + git_path_merge_rr(r)); > + return -1; > + } > + } else { > + /* > + * A background "rerere gc" holds the lock for as long as it > + * takes to walk rr-cache, so wait it out rather than die. > + */ > + fd = hold_lock_file_for_update_timeout(&write_lock, > + git_path_merge_rr(r), > + LOCK_DIE_ON_ERROR, > + rerere_lock_timeout_ms); > + } > read_rr(r, merge_rr); > return fd; > } > @@ -1237,7 +1254,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) > timestamp_t cutoff_resolve = now - 60 * 86400; > struct strbuf buf = STRBUF_INIT; > > - if (setup_rerere(r, rr, 0) < 0) > + if (setup_rerere(r, rr, RERERE_SKIP_LOCKED) < 0) > return; > > repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", > diff --git a/rerere.h b/rerere.h > index d4b5f7c932..87964bb3c5 100644 > --- a/rerere.h > +++ b/rerere.h > @@ -10,6 +10,7 @@ struct repository; > #define RERERE_AUTOUPDATE 01 > #define RERERE_NOAUTOUPDATE 02 > #define RERERE_READONLY 04 > +#define RERERE_SKIP_LOCKED 010 > > /* > * Marks paths that have been hand-resolved and added to the > diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh > index 1717f407c8..6b90294435 100755 > --- a/t/t4200-rerere.sh > +++ b/t/t4200-rerere.sh > @@ -242,6 +242,46 @@ test_expect_success 'old records rest in peace' ' > test_path_is_missing $rr2/preimage > ' > > +test_expect_success 'gc does nothing while MERGE_RR is locked' ' > + mkdir -p $rr2 && > + echo Hello >$rr2/preimage && > + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && > + > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + git rerere gc 2>err && > + test_grep "MERGE_RR" err && > + test_path_is_file $rr2/preimage && > + > + rm .git/MERGE_RR.lock && > + git rerere gc && > + test_path_is_missing $rr2/preimage > +' > + > +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' > + git reset --hard && > + rm -rf $rr && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + { > + (sleep 1 && rm -f .git/MERGE_RR.lock) & > + } && > + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && > + wait && > + test_grep ! "Unable to create" err && > + grep "^=======\$" $rr/preimage > +' > + > +test_expect_success 'rerere.lockTimeout=0 fails at once on a held lock' ' > + git reset --hard && > + rm -rf $rr && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && > + test_grep "Unable to create" err && > + test_path_is_missing $rr/preimage > +' > + > rerere_gc_custom_expiry_test () { > five_days="$1" right_now="$2" > test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' > diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh > index d7f82e1bec..a55ca2e829 100755 > --- a/t/t7900-maintenance.sh > +++ b/t/t7900-maintenance.sh > @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut > test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc > ' > > +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' > + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && > + mkdir .git/rr-cache && > + : >.git/rr-cache/entry && > + >.git/MERGE_RR.lock && > + test_expect_rerere_gc git maintenance run --task=rerere-gc > +' > + > test_expect_success '--auto and --schedule incompatible' ' > test_must_fail git maintenance run --auto --schedule=daily 2>err && > test_grep "cannot be used together" err > > base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc ``` ## Thomas Bachem, 2026-09-02 15:07 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Phillip, On 02/09/2026 15:27, Phillip Wood wrote: > To me this is another reason why we should disable gc.auto while > rebasing. To do that we need to pass "-c gc.auto=false -c > maintenance.auto=false" when running "git commit" in run_git_commit() > and also when running "git merge" in do_merge(). We should also pass > those settings via GIT_CONFIG_PARAMETERS when running a exec command in > do_exec(). That is largly papering over the cracks but until we have a > systematic solution it does at least stop exposing users to this bug. OK, I'll do that. It is also more consistent than it looks: the commits the sequencer creates in-process via try_to_commit() don't run auto maintenance at all, only the "git commit" child does (for a resolved, reworded or squashed commit). What surprised me is that a rebase with the merge backend then never runs maintenance, not even at the end, because it doesn't go through finish_rebase() where the apply backend runs it. Do you want a single run at the end of the sequence in that patch, or keep it minimal? FWIW, the tool I hit this with has been setting both for its whole process tree since, and the failures stopped. >> When it is the rebase, it dies in do_pick_commit() > > That's a bug us well - we should be returning errors, not dying > -rerere_setup() should be returning an error, so we can clean up and > reschedule the pick. Yes. I don't think we even need to reschedule: when repo_rerere() is called there, the merge result is already in the index and worktree, the error and advice have been printed, and the return value is ignored. If setup_rerere() reports the lock and returns -1, the pick just stops at the conflict like any other, minus rerere's recording and replay, and --continue works. I went through the callers of setup_rerere(): all of them handle a negative return, because that is what a disabled rerere returns, so this is close to a one-branch change. It also fixes the stale-lock case (crashed process), which disabling gc can't. > As I understand it we now block the sequencer until the background > maintenance has completed, or continue to die in an inconvenient state > we timeout before the background maintenance finishes. That seems rather > unfortunate as the idea of running the maintenance in the background is > to prevent it from interfering with other commands. Right, that's what it does. I copied the timeout from core.packedRefsTimeout, but a ref update can't be skipped and a rerere can, so the wait buys little. I'll drop rerere.lockTimeout. What it did buy: the gc spawned by the continue's own commit needs ~5ms to take the lock, the next pick usually longer to reach its rerere, so the gc is normally holding it by then. With only the gc-side skip my repro still died 3 of 3 times; with the error return those runs would survive but lose rerere at that stop. Tolerable, but it is why I'd rather have the sequencer patch in the same series than leave it for later. > I think my preferred solution is to disable gc while rebasing. Returning > an error from rerere_setup() would also help in the case where the user > runs "git commit" and then continues the rebase. I'd be interested to > hear what Junio and Patrick think about that. So v2 would be two patches: rerere returning an error on a busy lock (with "rerere gc" still warning and skipping as in v1, and a commit message that talks about the fix instead of the trace), and the sequencer disabling gc.auto/maintenance.auto for "git commit", "git merge" and exec. I'll wait for Junio and Patrick before rerolling in case they see it differently. Patrick, one thing I noticed on the way: since 452b12c2e0 (builtin/maintenance: use "geometric" strategy by default, 2026-02-24) every "maintenance run --auto" runs rerere-gc as soon as rr-cache has even a single entry, stale or not. The doc for maintenance.rerere-gc.auto says the heuristic may be refined; that would make this rare for every command, not only the sequencer. Not touching it in this series, just mentioning it. Thanks, Tom Am Mi., 2. Sept. 2026 um 15:27 Uhr schrieb Phillip Wood : > > Hi Thomas > > On 02/09/2026 09:31, Thomas Bachem via GitGitGadget wrote: > > From: Thomas Bachem > > > > Since 2.54 unscheduled maintenance uses the "geometric" strategy, so > > That change really is the gift that keeps on giving > > > the "git maintenance run --auto --detach" behind every "git commit" > > runs "git rerere gc" in the background whenever rr-cache has an entry. > > That includes the "git commit" the sequencer runs for a resolved pick > > on "git rebase --continue". > > > > rerere_gc() takes MERGE_RR.lock through setup_rerere(), which uses > > LOCK_DIE_ON_ERROR, and so does the sequencer's repo_rerere() at the > > next conflict a few milliseconds later. Whichever comes second dies. > > To me this is another reason why we should disable gc.auto while > rebasing. To do that we need to pass "-c gc.auto=false -c > maintenance.auto=false" when running "git commit" in run_git_commit() > and also when running "git merge" in do_merge(). We should also pass > those settings via GIT_CONFIG_PARAMETERS when running a exec command in > do_exec(). That is largly papering over the cracks but until we have a > systematic solution it does at least stop exposing users to this bug. > > > When it is the rebase, it dies in do_pick_commit() > > That's a bug us well - we should be returning errors, not dying > -rerere_setup() should be returning an error, so we can clean up and > reschedule the pick. > > There is a lot of detail here about what causes the problem which is > helpful, but there is very little discussion about the fix. As I > understand it we now block the sequencer until the background > maintenance has completed, or continue to die in an inconvenient state > we timeout before the background maintenance finishes. That seems rather > unfortunate as the idea of running the maintenance in the background is > to prevent it from interfering with other commands. > > I think my preferred solution is to disable gc while rebasing. Returning > an error from rerere_setup() would also help in the case where the user > runs "git commit" and then continues the rebase. I'd be interested to > hear what Junio and Patrick think about that. I'm also not clear why > gc.auto has to fork a separate process just to check if it needs to run > or not, I've not been following closely but my impression is that that > is the cause of quite a lot of the lock contention bugs we've seen. > > Thanks > > Phillip > > > with the index > > written but before make_patch() writes rebase-merge/{message,patch, > > stopped-sha}, and every later "git rebase --continue" refuses with > > "you have staged changes in your working tree". When it is the "git > > commit" of a later continue, that one dies in its post-commit > > repo_rerere() after the commit was made. Before 2.54 the same > > collision needed an auto gc to actually run, since gc runs > > "rerere gc" at its end. > > > > A rebase with two conflicts in a row shows it. The filler makes the > > pick slower than the ~5 ms the background task needs to take the > > lock, and keeps the lock held for about 0.4 s. It hit 6 of 6 runs > > here on 2.55.0, and a test suite driving rebases on toy repositories > > with a single rr-cache entry hit it in both runs that were traced: > > > > git init -q -b main r && cd r > > git config rerere.enabled true > > git config maintenance.auto false > > mkdir pad && seq 20000 | (cd pad && split -l 1 -a 5) > > echo base >f && git add -A && git commit -qm base > > git checkout -q -b topic > > echo b >f && git commit -qam B > > echo c >f && git commit -qam C > > git checkout -q main > > echo a >f && git commit -qam A > > git repack -adq > > seq 20000 | awk '{printf ".git/rr-cache/%040x\n", $1}' \ > > | xargs mkdir -p > > for d in .git/rr-cache/*/; do echo x >$d/preimage; done > > git config --unset maintenance.auto > > git checkout -q topic > > git rebase main > > echo ab >f && git add f > > GIT_EDITOR=true git rebase --continue > > > > The second continue dies with "Unable to create '.git/MERGE_RR.lock': > > File exists" while the gc spawned by its own commit holds the lock, > > and after resolving C every further continue refuses. Maintenance > > stays off during the setup so that no repack is pending: a repack due > > at that commit runs ahead of rerere-gc in the task list and would > > spend the window. > > > > The gc needs the lock: it removes every rr-cache directory it finds > > empty, and a rerere that has just created its directory but not yet > > written the preimage looks exactly like that. So keep the lock and fix > > both orders. When the gc finds the lock busy, let it warn and do > > nothing this time, the way "maintenance run" treats its own lock, so a > > manual "git rerere gc" sees the warning and the maintenance task and > > "git gc" see a clean exit. When the gc holds the lock, let every other > > caller wait it out instead of dying at once, for rerere.lockTimeout > > milliseconds with the semantics of core.packedRefsTimeout: 1000 by > > default, 0 for the old behaviour, -1 for an unbounded wait. Walking a > > 20000-entry rr-cache takes about 0.4 s here. > > > > That rebase now completes. The tests cover the gc under a held lock, > > directly and through the maintenance task, a merge that waits a lock > > out within a five second rerere.lockTimeout, and one that fails at > > once with a timeout of 0. > > > > Assisted-by: Claude Fable 5.1 > > Signed-off-by: Thomas Bachem > > --- > > rerere: keep a background gc from killing a rebase > > > > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v1 > > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v1 > > Pull-Request: https://github.com/gitgitgadget/git/pull/2214 > > > > Documentation/config/rerere.adoc | 8 +++++++ > > Documentation/git-rerere.adoc | 4 +++- > > rerere.c | 27 +++++++++++++++++---- > > rerere.h | 1 + > > t/t4200-rerere.sh | 40 ++++++++++++++++++++++++++++++++ > > t/t7900-maintenance.sh | 8 +++++++ > > 6 files changed, 82 insertions(+), 6 deletions(-) > > > > diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc > > index 3a78b5ebb1..8041a1587b 100644 > > --- a/Documentation/config/rerere.adoc > > +++ b/Documentation/config/rerere.adoc > > @@ -10,3 +10,11 @@ rerere.enabled:: > > enabled if there is an `rr-cache` directory under the > > `$GIT_DIR`, e.g. if "rerere" was previously used in the > > repository. > > + > > +rerere.lockTimeout:: > > + The length of time, in milliseconds, to retry when trying to > > + take the rerere lock while another process holds it, typically > > + a background `git rerere gc`. Value 0 means not to retry at > > + all; -1 means to try indefinitely. Default is 1000 (i.e., > > + retry for 1 second). `git rerere gc` itself does not wait and > > + skips its run instead. > > diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc > > index 4e6ab9a27c..05935b0603 100644 > > --- a/Documentation/git-rerere.adoc > > +++ b/Documentation/git-rerere.adoc > > @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older > > than 15 days and resolved conflicts older than 60 > > days are pruned. These defaults are controlled via the > > `gc.rerereUnresolved` and `gc.rerereResolved` configuration > > -variables respectively. > > +variables respectively. If another process holds the lock on the > > +recorded resolutions, for example a merge or rebase that is recording > > +a conflict, `gc` does nothing and reports so. > > > > > > DISCUSSION > > diff --git a/rerere.c b/rerere.c > > index 8232542585..22d114262b 100644 > > --- a/rerere.c > > +++ b/rerere.c > > @@ -32,6 +32,7 @@ static int rerere_enabled = -1; > > > > /* automatically update cleanly resolved paths to the index */ > > static int rerere_autoupdate; > > +static int rerere_lock_timeout_ms = 1000; > > > > #define RR_HAS_POSTIMAGE 1 > > #define RR_HAS_PREIMAGE 2 > > @@ -876,6 +877,8 @@ static void git_rerere_config(void) > > { > > repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); > > repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); > > + repo_config_get_int(the_repository, "rerere.locktimeout", > > + &rerere_lock_timeout_ms); > > repo_config(the_repository, git_default_config, NULL); > > } > > > > @@ -908,12 +911,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) > > > > if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) > > rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); > > - if (flags & RERERE_READONLY) > > + if (flags & RERERE_READONLY) { > > fd = 0; > > - else > > + } else if (flags & RERERE_SKIP_LOCKED) { > > fd = hold_lock_file_for_update(&write_lock, > > - git_path_merge_rr(r), > > - LOCK_DIE_ON_ERROR); > > + git_path_merge_rr(r), 0); > > + if (fd < 0) { > > + warning_errno(_("unable to lock '%s', skipping"), > > + git_path_merge_rr(r)); > > + return -1; > > + } > > + } else { > > + /* > > + * A background "rerere gc" holds the lock for as long as it > > + * takes to walk rr-cache, so wait it out rather than die. > > + */ > > + fd = hold_lock_file_for_update_timeout(&write_lock, > > + git_path_merge_rr(r), > > + LOCK_DIE_ON_ERROR, > > + rerere_lock_timeout_ms); > > + } > > read_rr(r, merge_rr); > > return fd; > > } > > @@ -1237,7 +1254,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) > > timestamp_t cutoff_resolve = now - 60 * 86400; > > struct strbuf buf = STRBUF_INIT; > > > > - if (setup_rerere(r, rr, 0) < 0) > > + if (setup_rerere(r, rr, RERERE_SKIP_LOCKED) < 0) > > return; > > > > repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", > > diff --git a/rerere.h b/rerere.h > > index d4b5f7c932..87964bb3c5 100644 > > --- a/rerere.h > > +++ b/rerere.h > > @@ -10,6 +10,7 @@ struct repository; > > #define RERERE_AUTOUPDATE 01 > > #define RERERE_NOAUTOUPDATE 02 > > #define RERERE_READONLY 04 > > +#define RERERE_SKIP_LOCKED 010 > > > > /* > > * Marks paths that have been hand-resolved and added to the > > diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh > > index 1717f407c8..6b90294435 100755 > > --- a/t/t4200-rerere.sh > > +++ b/t/t4200-rerere.sh > > @@ -242,6 +242,46 @@ test_expect_success 'old records rest in peace' ' > > test_path_is_missing $rr2/preimage > > ' > > > > +test_expect_success 'gc does nothing while MERGE_RR is locked' ' > > + mkdir -p $rr2 && > > + echo Hello >$rr2/preimage && > > + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && > > + > > + test_when_finished "rm -f .git/MERGE_RR.lock" && > > + >.git/MERGE_RR.lock && > > + git rerere gc 2>err && > > + test_grep "MERGE_RR" err && > > + test_path_is_file $rr2/preimage && > > + > > + rm .git/MERGE_RR.lock && > > + git rerere gc && > > + test_path_is_missing $rr2/preimage > > +' > > + > > +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' > > + git reset --hard && > > + rm -rf $rr && > > + test_when_finished "rm -f .git/MERGE_RR.lock" && > > + >.git/MERGE_RR.lock && > > + { > > + (sleep 1 && rm -f .git/MERGE_RR.lock) & > > + } && > > + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && > > + wait && > > + test_grep ! "Unable to create" err && > > + grep "^=======\$" $rr/preimage > > +' > > + > > +test_expect_success 'rerere.lockTimeout=0 fails at once on a held lock' ' > > + git reset --hard && > > + rm -rf $rr && > > + test_when_finished "rm -f .git/MERGE_RR.lock" && > > + >.git/MERGE_RR.lock && > > + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && > > + test_grep "Unable to create" err && > > + test_path_is_missing $rr/preimage > > +' > > + > > rerere_gc_custom_expiry_test () { > > five_days="$1" right_now="$2" > > test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' > > diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh > > index d7f82e1bec..a55ca2e829 100755 > > --- a/t/t7900-maintenance.sh > > +++ b/t/t7900-maintenance.sh > > @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut > > test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc > > ' > > > > +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' > > + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && > > + mkdir .git/rr-cache && > > + : >.git/rr-cache/entry && > > + >.git/MERGE_RR.lock && > > + test_expect_rerere_gc git maintenance run --task=rerere-gc > > +' > > + > > test_expect_success '--auto and --schedule incompatible' ' > > test_must_fail git maintenance run --auto --schedule=daily 2>err && > > test_grep "cannot be used together" err > > > > base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc > ``` ## Patrick Steinhardt, 2026-09-03 07:40 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` On Wed, Sep 02, 2026 at 08:31:37AM +0000, Thomas Bachem via GitGitGadget wrote: > From: Thomas Bachem > > Since 2.54 unscheduled maintenance uses the "geometric" strategy, so > the "git maintenance run --auto --detach" behind every "git commit" > runs "git rerere gc" in the background whenever rr-cache has an entry. > That includes the "git commit" the sequencer runs for a resolved pick > on "git rebase --continue". I think this hints that we should tweak the default value of "maintenance.rerere-gc.auto". The way it's currently written we indeed are quite aggressive with spawning `git rerere gc`, and I agree that we should tweak it. And in the best case we'd not only respect whether we have a specific number of entries, but we should also respect whether those would be garbage collected in the first place. I'll send a patch series later today to do this. [snip] > The gc needs the lock: it removes every rr-cache directory it finds > empty, and a rerere that has just created its directory but not yet > written the preimage looks exactly like that. So keep the lock and fix > both orders. When the gc finds the lock busy, let it warn and do > nothing this time, the way "maintenance run" treats its own lock, so a > manual "git rerere gc" sees the warning and the maintenance task and > "git gc" see a clean exit. When the gc holds the lock, let every other > caller wait it out instead of dying at once, for rerere.lockTimeout > milliseconds with the semantics of core.packedRefsTimeout: 1000 by > default, 0 for the old behaviour, -1 for an unbounded wait. Walking a > 20000-entry rr-cache takes about 0.4 s here. Having a locking timeout is sensible anyway, I think. It does not only solve races with a concurrent maintenance run, but also with concurrent writers. > diff --git a/rerere.c b/rerere.c > index 8232542585..22d114262b 100644 > --- a/rerere.c > +++ b/rerere.c > @@ -32,6 +32,7 @@ static int rerere_enabled = -1; > > /* automatically update cleanly resolved paths to the index */ > static int rerere_autoupdate; > +static int rerere_lock_timeout_ms = 1000; > > #define RR_HAS_POSTIMAGE 1 > #define RR_HAS_PREIMAGE 2 > @@ -876,6 +877,8 @@ static void git_rerere_config(void) > { > repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); > repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); > + repo_config_get_int(the_repository, "rerere.locktimeout", > + &rerere_lock_timeout_ms); > repo_config(the_repository, git_default_config, NULL); > } > > @@ -908,12 +911,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) > > if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) > rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); > - if (flags & RERERE_READONLY) > + if (flags & RERERE_READONLY) { > fd = 0; > - else > + } else if (flags & RERERE_SKIP_LOCKED) { > fd = hold_lock_file_for_update(&write_lock, > - git_path_merge_rr(r), > - LOCK_DIE_ON_ERROR); > + git_path_merge_rr(r), 0); > + if (fd < 0) { > + warning_errno(_("unable to lock '%s', skipping"), > + git_path_merge_rr(r)); > + return -1; > + } We should instead pass `LOCK_REPORT_ON_ERROR`, as the lockfile machinery knows better why exactly locking has failed. > + } else { > + /* > + * A background "rerere gc" holds the lock for as long as it > + * takes to walk rr-cache, so wait it out rather than die. > + */ > + fd = hold_lock_file_for_update_timeout(&write_lock, > + git_path_merge_rr(r), > + LOCK_DIE_ON_ERROR, > + rerere_lock_timeout_ms); > + } I think we can easily combine those two branches and simply set the timeout value to 0 in case we see the flag. Patrick ``` ## Thomas Bachem, 2026-09-03 08:11 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Patrick, On Thu, Sep 03, 2026 at 09:40:04AM +0200, Patrick Steinhardt wrote: > I think this hints that we should tweak the default value of > "maintenance.rerere-gc.auto". The way it's currently written we indeed > are quite aggressive with spawning `git rerere gc`, and I agree that we > should tweak it. And in the best case we'd not only respect whether we > have a specific number of entries, but we should also respect whether > those would be garbage collected in the first place. > > I'll send a patch series later today to do this. Thanks. Checking whether anything would actually be pruned sounds right to me. It takes the frequency away, not the race, so I'd still do the sequencer part Phillip asked for. > Having a locking timeout is sensible anyway, I think. It does not only > solve races with a concurrent maintenance run, but also with concurrent > writers. Phillip found the wait unfortunate and I offered to drop it. You would keep it. I think the two fit together: wait up to rerere.lockTimeout, then warn and return -1 instead of dying, so the caller goes on without rerere this once. The gc passes 0 and does not wait. That takes the die out, which is what broke the rebase. The wait stays, bounded to a second, but skipping rerere is not free either: it can mean resolving a conflict again that rerere had already recorded, and a second is cheap next to that. With the sequencer no longer spawning the gc and your heuristic change, it should rarely come to either. Phillip, would that work for you? > We should instead pass `LOCK_REPORT_ON_ERROR`, as the lockfile machinery > knows better why exactly locking has failed. Agreed on the text, which also names a stale lock. But the callers that go on without rerere then exit as if it were disabled, "git commit" with 0, so for them I'd print it as a warning through unable_to_lock_message() rather than let LOCK_REPORT_ON_ERROR call it an error. An explicit "git rerere forget" or "clear" fails as before. > I think we can easily combine those two branches and simply set the > timeout value to 0 in case we see the flag. Yes, that folds into one call. So v2: setup_rerere() waits up to rerere.lockTimeout, 0 for the gc, then warns and returns -1 where the caller can go on, with the sequencer patch on top. I'll reroll once Phillip has had a look. Thanks, Tom ``` ## Patrick Steinhardt, 2026-09-03 08:32 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` On Thu, Sep 03, 2026 at 10:11:05AM +0200, Thomas Bachem wrote: > Hi Patrick, > > On Thu, Sep 03, 2026 at 09:40:04AM +0200, Patrick Steinhardt wrote: > > I think this hints that we should tweak the default value of > > "maintenance.rerere-gc.auto". The way it's currently written we indeed > > are quite aggressive with spawning `git rerere gc`, and I agree that we > > should tweak it. And in the best case we'd not only respect whether we > > have a specific number of entries, but we should also respect whether > > those would be garbage collected in the first place. > > > > I'll send a patch series later today to do this. > > Thanks. Checking whether anything would actually be pruned sounds > right to me. It takes the frequency away, not the race, so I'd still > do the sequencer part Phillip asked for. Yes. Ideally, I'd think that we should both introduce the grace period for locking the file and adapting the heuristic used by the maintenance strategy. Whether we should completely disable auto-maintenance when in the sequencer... I dunno. In any case, that feels like another separate topic that should probably be discussed in its own series. > > Having a locking timeout is sensible anyway, I think. It does not only > > solve races with a concurrent maintenance run, but also with concurrent > > writers. > > Phillip found the wait unfortunate and I offered to drop it. You would > keep it. I think the two fit together: wait up to rerere.lockTimeout, > then warn and return -1 instead of dying, so the caller goes on > without rerere this once. The gc passes 0 and does not wait. That > takes the die out, which is what broke the rebase. The wait stays, > bounded to a second, but skipping rerere is not free either: it can > mean resolving a conflict again that rerere had already recorded, and > a second is cheap next to that. With the sequencer no longer spawning > the gc and your heuristic change, it should rarely come to either. > Phillip, would that work for you? I think that having the wait is a sensible thing to do, as the race was a preexisting one that was only uncovered by the change to the default maintenance strategy. It can also happen with two concurrent processes that both happen to write rerere entries. You wouldn't normally see the wait anyway, so in the happy path nobody will really care. And in the cases where you would see it the user is probably more happy to wait a bit than having Git die (or just not write a rerere entry at all). Patrick ``` ## Thomas Bachem, 2026-09-03 12:12 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Patrick, On Thu, Sep 03, 2026 at 10:32:36AM +0200, Patrick Steinhardt wrote: > Yes. Ideally, I'd think that we should both introduce the grace period > for locking the file and adapting the heuristic used by the maintenance > strategy. Whether we should completely disable auto-maintenance when in > the sequencer... I dunno. In any case, that feels like another separate > topic that should probably be discussed in its own series. Phillip, this is the part I said I'd do in this series, so I'd rather answer it here than just drop it. I think Patrick is right that it's a topic of its own. My reason for wanting it in the same series was the recording lost at a stop while the gc holds the lock, and that was for the variant without the wait. With the wait kept, the next pick waits the gc out and records as before, so the sequencer patch no longer buys the rebase anything the rerere patch doesn't, short of a prune that outlasts the timeout. What it would still decide is whether a rebase with the merge backend runs maintenance at all, the question from my last mail, and that is a discussion of its own. So I'd make v2 the rerere patch alone and send the sequencer change separately if you still want it. Say if you would rather keep them together. > I think that having the wait is a sensible thing to do, as the race was > a preexisting one that was only uncovered by the change to the default > maintenance strategy. It can also happen with two concurrent processes > that both happen to write rerere entries. You wouldn't normally see the > wait anyway, so in the happy path nobody will really care. And in the > cases where you would see it the user is probably more happy to wait a > bit than having Git die (or just not write a rerere entry at all). Agreed, and that is the order v2 keeps: wait first, skip only once the wait has run out. Since your series means the gc now only runs when there is something to prune, I measured how long that wait can get: pruning 20000 stale entries holds the lock for 2.7 s here, walking 20000 fresh ones takes 0.4 s, so the one second default covers a prune of roughly 7000 entries if it scales. I'd keep the default. A backlog that size is a one-off, and where it does hit, the timeout now skips one recording where it used to kill the rebase. My patch is based on maint since the bug is there, and I'd keep it that way unless Junio would rather have it on master. Merged up it conflicts with d43f701d32 (lockfile: add repo_hold_lock_file_for_update{,_timeout}{,_mode}(), 2026-07-14) in setup_rerere(). The resolution is to take the repo-scoped helper, and with that t4200 and t7900 pass on top of your series. I'll wait a day or two for Phillip before rerolling. Thanks, Tom ``` ## Phillip Wood, 2026-09-03 13:50 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Patrick and Thomas On 03/09/2026 09:32, Patrick Steinhardt wrote: > On Thu, Sep 03, 2026 at 10:11:05AM +0200, Thomas Bachem wrote: >> Hi Patrick, >> >> On Thu, Sep 03, 2026 at 09:40:04AM +0200, Patrick Steinhardt wrote: >>> I think this hints that we should tweak the default value of >>> "maintenance.rerere-gc.auto". The way it's currently written we indeed >>> are quite aggressive with spawning `git rerere gc`, and I agree that we >>> should tweak it. And in the best case we'd not only respect whether we >>> have a specific number of entries, but we should also respect whether >>> those would be garbage collected in the first place. >>> >>> I'll send a patch series later today to do this. >> >> Thanks. Checking whether anything would actually be pruned sounds >> right to me. It takes the frequency away, not the race, so I'd still >> do the sequencer part Phillip asked for. > > Yes. Ideally, I'd think that we should both introduce the grace period > for locking the file and adapting the heuristic used by the maintenance > strategy. I agree > Whether we should completely disable auto-maintenance when in > the sequencer... I dunno. In any case, that feels like another separate > topic that should probably be discussed in its own series. We've seen other bugs reported related to auto-maintenance triggered during a rebase such as the one dscho fixed recently. While I can see repacking might be helpful during a very large rebase, I do not think garbage collection is useful - all the objects and rerere entries that are created during the rebase are going to be too fresh to be collected. So I think it would be a good idea to disable auto maintenance in a rebase and see if anyone complains. If it turns out to be a problem we can figure out how to make it repack incrementally. >>> Having a locking timeout is sensible anyway, I think. It does not only >>> solve races with a concurrent maintenance run, but also with concurrent >>> writers. >> >> Phillip found the wait unfortunate and I offered to drop it. You would >> keep it. I think the two fit together: wait up to rerere.lockTimeout, >> then warn and return -1 instead of dying, so the caller goes on >> without rerere this once. The gc passes 0 and does not wait. That >> takes the die out, which is what broke the rebase. The wait stays, >> bounded to a second, but skipping rerere is not free either: it can >> mean resolving a conflict again that rerere had already recorded, and >> a second is cheap next to that. With the sequencer no longer spawning >> the gc and your heuristic change, it should rarely come to either. >> Phillip, would that work for you? > > I think that having the wait is a sensible thing to do, as the race was > a preexisting one that was only uncovered by the change to the default > maintenance strategy. It can also happen with two concurrent processes > that both happen to write rerere entries. You wouldn't normally see the > wait anyway, so in the happy path nobody will really care. And in the > cases where you would see it the user is probably more happy to wait a > bit than having Git die (or just not write a rerere entry at all). I don't object to the timeout as part of the solution. My objection was based on it being the only solution as it is inconvenient to the user if they have to wait for background maintenance jobs and it does not stop the rebase from failing if the timeout is too short. Thanks Phillip ``` ## Phillip Wood, 2026-09-03 13:50 Subject: Re: [PATCH] rerere: keep a background gc from killing a rebase Message-ID: <86efb07c-a0ce-49b0-b4eb-7d6b4bbaeccc@gmail.com> In-Reply-To: ``` Hi Thomas On 02/09/2026 16:07, Thomas Bachem wrote: > On 02/09/2026 15:27, Phillip Wood wrote: >> To me this is another reason why we should disable gc.auto while >> rebasing. To do that we need to pass "-c gc.auto=false -c >> maintenance.auto=false" when running "git commit" in run_git_commit() >> and also when running "git merge" in do_merge(). We should also pass >> those settings via GIT_CONFIG_PARAMETERS when running a exec command in >> do_exec(). That is largly papering over the cracks but until we have a >> systematic solution it does at least stop exposing users to this bug. > > OK, I'll do that. It is also more consistent than it looks: the > commits the sequencer creates in-process via try_to_commit() don't run > auto maintenance at all, only the "git commit" child does (for a > resolved, reworded or squashed commit). What surprised me is that a > rebase with the merge backend then never runs maintenance, not even at > the end, because it doesn't go through finish_rebase() where the apply > backend runs it. Do you want a single run at the end of the sequence > in that patch, or keep it minimal? We should be consistent between the backends, so yes we should be calling run_auto_maintenance() at the end of a rebase with the merge backend. > FWIW, the tool I hit this with has been setting both for its whole > process tree since, and the failures stopped. > >>> When it is the rebase, it dies in do_pick_commit() >> >> That's a bug us well - we should be returning errors, not dying >> -rerere_setup() should be returning an error, so we can clean up and >> reschedule the pick. > > Yes. I don't think we even need to reschedule: when repo_rerere() is > called there, the merge result is already in the index and worktree, > the error and advice have been printed, and the return value is > ignored. If setup_rerere() reports the lock and returns -1, the pick > just stops at the conflict like any other, minus rerere's recording > and replay, and --continue works. Oh good point, if we get an error then we'll write the files to get "git rebase --continue" to commit the conflict resolution so we don't need to reschedule. Thanks Phillip >> I think my preferred solution is to disable gc while rebasing. Returning >> an error from rerere_setup() would also help in the case where the user >> runs "git commit" and then continues the rebase. I'd be interested to >> hear what Junio and Patrick think about that. > > So v2 would be two patches: rerere returning an error on a busy lock > (with "rerere gc" still warning and skipping as in v1, and a commit > message that talks about the fix instead of the trace), and the > sequencer disabling gc.auto/maintenance.auto for "git commit", "git > merge" and exec. I'll wait for Junio and Patrick before rerolling in > case they see it differently. > > Patrick, one thing I noticed on the way: since 452b12c2e0 > (builtin/maintenance: use "geometric" strategy by default, 2026-02-24) > every "maintenance run --auto" runs rerere-gc as soon as rr-cache has > even a single entry, stale or not. The doc for > maintenance.rerere-gc.auto says the heuristic may be refined; that > would make this rare for every command, not only the sequencer. Not > touching it in this series, just mentioning it. > > Thanks, > Tom > > > Am Mi., 2. Sept. 2026 um 15:27 Uhr schrieb Phillip Wood > : >> >> Hi Thomas >> >> On 02/09/2026 09:31, Thomas Bachem via GitGitGadget wrote: >>> From: Thomas Bachem >>> >>> Since 2.54 unscheduled maintenance uses the "geometric" strategy, so >> >> That change really is the gift that keeps on giving >> >>> the "git maintenance run --auto --detach" behind every "git commit" >>> runs "git rerere gc" in the background whenever rr-cache has an entry. >>> That includes the "git commit" the sequencer runs for a resolved pick >>> on "git rebase --continue". >>> >>> rerere_gc() takes MERGE_RR.lock through setup_rerere(), which uses >>> LOCK_DIE_ON_ERROR, and so does the sequencer's repo_rerere() at the >>> next conflict a few milliseconds later. Whichever comes second dies. >> >> To me this is another reason why we should disable gc.auto while >> rebasing. To do that we need to pass "-c gc.auto=false -c >> maintenance.auto=false" when running "git commit" in run_git_commit() >> and also when running "git merge" in do_merge(). We should also pass >> those settings via GIT_CONFIG_PARAMETERS when running a exec command in >> do_exec(). That is largly papering over the cracks but until we have a >> systematic solution it does at least stop exposing users to this bug. >> >>> When it is the rebase, it dies in do_pick_commit() >> >> That's a bug us well - we should be returning errors, not dying >> -rerere_setup() should be returning an error, so we can clean up and >> reschedule the pick. >> >> There is a lot of detail here about what causes the problem which is >> helpful, but there is very little discussion about the fix. As I >> understand it we now block the sequencer until the background >> maintenance has completed, or continue to die in an inconvenient state >> we timeout before the background maintenance finishes. That seems rather >> unfortunate as the idea of running the maintenance in the background is >> to prevent it from interfering with other commands. >> >> I think my preferred solution is to disable gc while rebasing. Returning >> an error from rerere_setup() would also help in the case where the user >> runs "git commit" and then continues the rebase. I'd be interested to >> hear what Junio and Patrick think about that. I'm also not clear why >> gc.auto has to fork a separate process just to check if it needs to run >> or not, I've not been following closely but my impression is that that >> is the cause of quite a lot of the lock contention bugs we've seen. >> >> Thanks >> >> Phillip >> >>> with the index >>> written but before make_patch() writes rebase-merge/{message,patch, >>> stopped-sha}, and every later "git rebase --continue" refuses with >>> "you have staged changes in your working tree". When it is the "git >>> commit" of a later continue, that one dies in its post-commit >>> repo_rerere() after the commit was made. Before 2.54 the same >>> collision needed an auto gc to actually run, since gc runs >>> "rerere gc" at its end. >>> >>> A rebase with two conflicts in a row shows it. The filler makes the >>> pick slower than the ~5 ms the background task needs to take the >>> lock, and keeps the lock held for about 0.4 s. It hit 6 of 6 runs >>> here on 2.55.0, and a test suite driving rebases on toy repositories >>> with a single rr-cache entry hit it in both runs that were traced: >>> >>> git init -q -b main r && cd r >>> git config rerere.enabled true >>> git config maintenance.auto false >>> mkdir pad && seq 20000 | (cd pad && split -l 1 -a 5) >>> echo base >f && git add -A && git commit -qm base >>> git checkout -q -b topic >>> echo b >f && git commit -qam B >>> echo c >f && git commit -qam C >>> git checkout -q main >>> echo a >f && git commit -qam A >>> git repack -adq >>> seq 20000 | awk '{printf ".git/rr-cache/%040x\n", $1}' \ >>> | xargs mkdir -p >>> for d in .git/rr-cache/*/; do echo x >$d/preimage; done >>> git config --unset maintenance.auto >>> git checkout -q topic >>> git rebase main >>> echo ab >f && git add f >>> GIT_EDITOR=true git rebase --continue >>> >>> The second continue dies with "Unable to create '.git/MERGE_RR.lock': >>> File exists" while the gc spawned by its own commit holds the lock, >>> and after resolving C every further continue refuses. Maintenance >>> stays off during the setup so that no repack is pending: a repack due >>> at that commit runs ahead of rerere-gc in the task list and would >>> spend the window. >>> >>> The gc needs the lock: it removes every rr-cache directory it finds >>> empty, and a rerere that has just created its directory but not yet >>> written the preimage looks exactly like that. So keep the lock and fix >>> both orders. When the gc finds the lock busy, let it warn and do >>> nothing this time, the way "maintenance run" treats its own lock, so a >>> manual "git rerere gc" sees the warning and the maintenance task and >>> "git gc" see a clean exit. When the gc holds the lock, let every other >>> caller wait it out instead of dying at once, for rerere.lockTimeout >>> milliseconds with the semantics of core.packedRefsTimeout: 1000 by >>> default, 0 for the old behaviour, -1 for an unbounded wait. Walking a >>> 20000-entry rr-cache takes about 0.4 s here. >>> >>> That rebase now completes. The tests cover the gc under a held lock, >>> directly and through the maintenance task, a merge that waits a lock >>> out within a five second rerere.lockTimeout, and one that fails at >>> once with a timeout of 0. >>> >>> Assisted-by: Claude Fable 5.1 >>> Signed-off-by: Thomas Bachem >>> --- >>> rerere: keep a background gc from killing a rebase >>> >>> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v1 >>> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v1 >>> Pull-Request: https://github.com/gitgitgadget/git/pull/2214 >>> >>> Documentation/config/rerere.adoc | 8 +++++++ >>> Documentation/git-rerere.adoc | 4 +++- >>> rerere.c | 27 +++++++++++++++++---- >>> rerere.h | 1 + >>> t/t4200-rerere.sh | 40 ++++++++++++++++++++++++++++++++ >>> t/t7900-maintenance.sh | 8 +++++++ >>> 6 files changed, 82 insertions(+), 6 deletions(-) >>> >>> diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc >>> index 3a78b5ebb1..8041a1587b 100644 >>> --- a/Documentation/config/rerere.adoc >>> +++ b/Documentation/config/rerere.adoc >>> @@ -10,3 +10,11 @@ rerere.enabled:: >>> enabled if there is an `rr-cache` directory under the >>> `$GIT_DIR`, e.g. if "rerere" was previously used in the >>> repository. >>> + >>> +rerere.lockTimeout:: >>> + The length of time, in milliseconds, to retry when trying to >>> + take the rerere lock while another process holds it, typically >>> + a background `git rerere gc`. Value 0 means not to retry at >>> + all; -1 means to try indefinitely. Default is 1000 (i.e., >>> + retry for 1 second). `git rerere gc` itself does not wait and >>> + skips its run instead. >>> diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc >>> index 4e6ab9a27c..05935b0603 100644 >>> --- a/Documentation/git-rerere.adoc >>> +++ b/Documentation/git-rerere.adoc >>> @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older >>> than 15 days and resolved conflicts older than 60 >>> days are pruned. These defaults are controlled via the >>> `gc.rerereUnresolved` and `gc.rerereResolved` configuration >>> -variables respectively. >>> +variables respectively. If another process holds the lock on the >>> +recorded resolutions, for example a merge or rebase that is recording >>> +a conflict, `gc` does nothing and reports so. >>> >>> >>> DISCUSSION >>> diff --git a/rerere.c b/rerere.c >>> index 8232542585..22d114262b 100644 >>> --- a/rerere.c >>> +++ b/rerere.c >>> @@ -32,6 +32,7 @@ static int rerere_enabled = -1; >>> >>> /* automatically update cleanly resolved paths to the index */ >>> static int rerere_autoupdate; >>> +static int rerere_lock_timeout_ms = 1000; >>> >>> #define RR_HAS_POSTIMAGE 1 >>> #define RR_HAS_PREIMAGE 2 >>> @@ -876,6 +877,8 @@ static void git_rerere_config(void) >>> { >>> repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); >>> repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); >>> + repo_config_get_int(the_repository, "rerere.locktimeout", >>> + &rerere_lock_timeout_ms); >>> repo_config(the_repository, git_default_config, NULL); >>> } >>> >>> @@ -908,12 +911,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) >>> >>> if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) >>> rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); >>> - if (flags & RERERE_READONLY) >>> + if (flags & RERERE_READONLY) { >>> fd = 0; >>> - else >>> + } else if (flags & RERERE_SKIP_LOCKED) { >>> fd = hold_lock_file_for_update(&write_lock, >>> - git_path_merge_rr(r), >>> - LOCK_DIE_ON_ERROR); >>> + git_path_merge_rr(r), 0); >>> + if (fd < 0) { >>> + warning_errno(_("unable to lock '%s', skipping"), >>> + git_path_merge_rr(r)); >>> + return -1; >>> + } >>> + } else { >>> + /* >>> + * A background "rerere gc" holds the lock for as long as it >>> + * takes to walk rr-cache, so wait it out rather than die. >>> + */ >>> + fd = hold_lock_file_for_update_timeout(&write_lock, >>> + git_path_merge_rr(r), >>> + LOCK_DIE_ON_ERROR, >>> + rerere_lock_timeout_ms); >>> + } >>> read_rr(r, merge_rr); >>> return fd; >>> } >>> @@ -1237,7 +1254,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) >>> timestamp_t cutoff_resolve = now - 60 * 86400; >>> struct strbuf buf = STRBUF_INIT; >>> >>> - if (setup_rerere(r, rr, 0) < 0) >>> + if (setup_rerere(r, rr, RERERE_SKIP_LOCKED) < 0) >>> return; >>> >>> repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", >>> diff --git a/rerere.h b/rerere.h >>> index d4b5f7c932..87964bb3c5 100644 >>> --- a/rerere.h >>> +++ b/rerere.h >>> @@ -10,6 +10,7 @@ struct repository; >>> #define RERERE_AUTOUPDATE 01 >>> #define RERERE_NOAUTOUPDATE 02 >>> #define RERERE_READONLY 04 >>> +#define RERERE_SKIP_LOCKED 010 >>> >>> /* >>> * Marks paths that have been hand-resolved and added to the >>> diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh >>> index 1717f407c8..6b90294435 100755 >>> --- a/t/t4200-rerere.sh >>> +++ b/t/t4200-rerere.sh >>> @@ -242,6 +242,46 @@ test_expect_success 'old records rest in peace' ' >>> test_path_is_missing $rr2/preimage >>> ' >>> >>> +test_expect_success 'gc does nothing while MERGE_RR is locked' ' >>> + mkdir -p $rr2 && >>> + echo Hello >$rr2/preimage && >>> + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && >>> + >>> + test_when_finished "rm -f .git/MERGE_RR.lock" && >>> + >.git/MERGE_RR.lock && >>> + git rerere gc 2>err && >>> + test_grep "MERGE_RR" err && >>> + test_path_is_file $rr2/preimage && >>> + >>> + rm .git/MERGE_RR.lock && >>> + git rerere gc && >>> + test_path_is_missing $rr2/preimage >>> +' >>> + >>> +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' >>> + git reset --hard && >>> + rm -rf $rr && >>> + test_when_finished "rm -f .git/MERGE_RR.lock" && >>> + >.git/MERGE_RR.lock && >>> + { >>> + (sleep 1 && rm -f .git/MERGE_RR.lock) & >>> + } && >>> + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && >>> + wait && >>> + test_grep ! "Unable to create" err && >>> + grep "^=======\$" $rr/preimage >>> +' >>> + >>> +test_expect_success 'rerere.lockTimeout=0 fails at once on a held lock' ' >>> + git reset --hard && >>> + rm -rf $rr && >>> + test_when_finished "rm -f .git/MERGE_RR.lock" && >>> + >.git/MERGE_RR.lock && >>> + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && >>> + test_grep "Unable to create" err && >>> + test_path_is_missing $rr/preimage >>> +' >>> + >>> rerere_gc_custom_expiry_test () { >>> five_days="$1" right_now="$2" >>> test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' >>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh >>> index d7f82e1bec..a55ca2e829 100755 >>> --- a/t/t7900-maintenance.sh >>> +++ b/t/t7900-maintenance.sh >>> @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut >>> test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc >>> ' >>> >>> +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' >>> + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && >>> + mkdir .git/rr-cache && >>> + : >.git/rr-cache/entry && >>> + >.git/MERGE_RR.lock && >>> + test_expect_rerere_gc git maintenance run --task=rerere-gc >>> +' >>> + >>> test_expect_success '--auto and --schedule incompatible' ' >>> test_must_fail git maintenance run --auto --schedule=daily 2>err && >>> test_grep "cannot be used together" err >>> >>> base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc >> ``` ## Thomas Bachem via GitGitGadget, 2026-09-04 07:44 Subject: [PATCH v2] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` From: Thomas Bachem Since 2.54 unscheduled maintenance uses the "geometric" strategy, so the "git maintenance run --auto --detach" behind every "git commit" runs "git rerere gc" in the background whenever rr-cache has an entry. That includes the "git commit" the sequencer runs for a resolved pick on "git rebase --continue". rerere_gc() takes MERGE_RR.lock through setup_rerere(), which uses LOCK_DIE_ON_ERROR, and so does the sequencer's repo_rerere() at the next conflict a few milliseconds later. Whichever comes second dies. When it is the rebase, it dies in do_pick_commit() with the index written but before make_patch() writes rebase-merge/{message,patch, stopped-sha}, and every later "git rebase --continue" refuses with "you have staged changes in your working tree". When it is the "git commit" of a later continue, that one dies in its post-commit repo_rerere() after the commit was made. Before 2.54 the same collision needed an auto gc to actually run, since gc runs "rerere gc" at its end. A rebase with two conflicts in a row shows it. The filler makes the pick slower than the ~5 ms the background task needs to take the lock, and the stale entries give the gc something to prune, which keeps the lock held for about half a second. It hit 6 of 6 runs here on 2.55.0, and a test suite driving rebases on toy repositories with a single rr-cache entry hit it in both runs that were traced: git init -q -b main r && cd r git config rerere.enabled true git config maintenance.auto false mkdir pad && seq 20000 | (cd pad && split -l 1 -a 5) echo base >f && git add -A && git commit -qm base git checkout -q -b topic echo b >f && git commit -qam B echo c >f && git commit -qam C git checkout -q main echo a >f && git commit -qam A git repack -adq git ls-files -s pad | head -n 5000 | awk '{print ".git/rr-cache/" $2}' | xargs mkdir -p for d in .git/rr-cache/*/; do echo x >$d/preimage; done touch -t 202001010000 .git/rr-cache/*/preimage git config --unset maintenance.auto git checkout -q topic git rebase main echo ab >f && git add f GIT_EDITOR=true git rebase --continue The continue dies with "Unable to create '.git/MERGE_RR.lock': File exists" while the gc spawned by its own commit holds the lock, and after resolving C every further continue refuses. Maintenance stays off during the setup so that no repack is pending: a repack due at that commit runs ahead of rerere-gc in the task list and would spend the window. The gc needs the lock: it removes every rr-cache directory it finds empty, and a rerere that has just created its directory but not yet written the preimage looks exactly like that. So keep the lock and stop dying over it. A caller that finds the lock held now waits for rerere.lockTimeout milliseconds, with the semantics of core.packedRefsTimeout and the same default of 1000, and then warns and goes on without rerere: a merge, a commit or a pick loses one recording or replay, which is nothing next to a rebase that cannot continue. The gc itself never waits, since it has nothing to lose from a skipped run, and "git rerere", "git rerere forget" and "git rerere clear" keep dying, since they exist for nothing but the state behind the lock. The clearing "git am" and "git rebase" do on --abort and --skip goes on without it: the cleanup that follows removes MERGE_RR anyway, and the unresolved entries it would have dropped are left for the gc. A stale MERGE_RR.lock, which used to stop every merge, now costs each command a second and a warning until it is removed. That rebase now stops at C the normal way, with its preimage recorded once the prune is over, and continues once C is resolved. The tests cover the gc under a held lock, directly and through the maintenance task, a merge that waits a lock out within rerere.lockTimeout, a merge, a commit and a rebase that go on without rerere once it is up, "git rebase --abort" doing the same, and the three explicit commands failing. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- rerere: keep a background gc from killing a rebase Changes since v1: * Once rerere.lockTimeout is up, setup_rerere() warns and returns -1 instead of dying, so a merge, commit or pick goes on without rerere (Phillip). Only "git rerere", "git rerere forget" and "git rerere clear" keep dying, through a RERERE_LOCK_OR_DIE flag. rerere_clear() takes flags for that, since "am" and "rebase" call it too and go on. * RERERE_SKIP_LOCKED is now RERERE_NOWAIT: skipping is what everyone does, not waiting is what sets the gc apart. The wait and its default stay (Patrick). * The warning stays hand-rolled rather than LOCK_REPORT_ON_ERROR (Patrick): the lockfile message tells the user to terminate other git processes and try again, which a command that goes on without rerere should not say. It names the lock file, says that rerere is skipped, and carries the errno, so the reason still shows. * The repro uses 5000 stale entries named after blobs, so the gc has something to prune and the run still triggers a gc that only prunes what is due, as with Patrick's heuristic series. The prune holds the lock for about half a second here, within the default timeout, so the fixed rebase records the second conflict rather than skip it. * Tests: the timeout=0 test now checks that the merge goes on, and new ones cover a commit and a rebase going on under a held lock, "git rebase --abort", and the three explicit commands. test_grep throughout. * Commit message: the fix gets the discussion, the repro one paragraph and its script (Phillip). What waits, what goes on and what keeps dying is spelled out, and the script has one continue, not two. Still based on maint, where the bug ships (2.54.0 and 2.55.0). Merged up it conflicts with d43f701d32 (lockfile: add repo_hold_lock_file_for_update{,_timeout}{,_mode}(), 2026-07-14) in setup_rerere(), where the resolution takes the repo-scoped helper. With that, t4200 and t7900 pass on top of Patrick's series and with the sequencer series that keeps auto maintenance out of a rebase, sent separately. Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/2214 Range-diff vs v1: 1: ecd9e5b4ec ! 1: bbb2338572 rerere: keep a background gc from killing a rebase @@ Commit message A rebase with two conflicts in a row shows it. The filler makes the pick slower than the ~5 ms the background task needs to take the - lock, and keeps the lock held for about 0.4 s. It hit 6 of 6 runs - here on 2.55.0, and a test suite driving rebases on toy repositories - with a single rr-cache entry hit it in both runs that were traced: + lock, and the stale entries give the gc something to prune, which + keeps the lock held for about half a second. It hit 6 of 6 runs here + on 2.55.0, and a test suite driving rebases on toy repositories with + a single rr-cache entry hit it in both runs that were traced: git init -q -b main r && cd r git config rerere.enabled true @@ Commit message git checkout -q main echo a >f && git commit -qam A git repack -adq - seq 20000 | awk '{printf ".git/rr-cache/%040x\n", $1}' \ - | xargs mkdir -p + git ls-files -s pad | head -n 5000 | + awk '{print ".git/rr-cache/" $2}' | xargs mkdir -p for d in .git/rr-cache/*/; do echo x >$d/preimage; done + touch -t 202001010000 .git/rr-cache/*/preimage git config --unset maintenance.auto git checkout -q topic git rebase main echo ab >f && git add f GIT_EDITOR=true git rebase --continue - The second continue dies with "Unable to create '.git/MERGE_RR.lock': - File exists" while the gc spawned by its own commit holds the lock, - and after resolving C every further continue refuses. Maintenance - stays off during the setup so that no repack is pending: a repack due - at that commit runs ahead of rerere-gc in the task list and would - spend the window. + The continue dies with "Unable to create '.git/MERGE_RR.lock': File + exists" while the gc spawned by its own commit holds the lock, and + after resolving C every further continue refuses. Maintenance stays + off during the setup so that no repack is pending: a repack due at + that commit runs ahead of rerere-gc in the task list and would spend + the window. The gc needs the lock: it removes every rr-cache directory it finds empty, and a rerere that has just created its directory but not yet - written the preimage looks exactly like that. So keep the lock and fix - both orders. When the gc finds the lock busy, let it warn and do - nothing this time, the way "maintenance run" treats its own lock, so a - manual "git rerere gc" sees the warning and the maintenance task and - "git gc" see a clean exit. When the gc holds the lock, let every other - caller wait it out instead of dying at once, for rerere.lockTimeout - milliseconds with the semantics of core.packedRefsTimeout: 1000 by - default, 0 for the old behaviour, -1 for an unbounded wait. Walking a - 20000-entry rr-cache takes about 0.4 s here. - - That rebase now completes. The tests cover the gc under a held lock, - directly and through the maintenance task, a merge that waits a lock - out within a five second rerere.lockTimeout, and one that fails at - once with a timeout of 0. + written the preimage looks exactly like that. So keep the lock and + stop dying over it. A caller that finds the lock held now waits for + rerere.lockTimeout milliseconds, with the semantics of + core.packedRefsTimeout and the same default of 1000, and then warns + and goes on without rerere: a merge, a commit or a pick loses one + recording or replay, which is nothing next to a rebase that cannot + continue. The gc itself never waits, since it has nothing to lose + from a skipped run, and "git rerere", "git rerere forget" and "git + rerere clear" keep dying, since they exist for nothing but the state + behind the lock. The clearing "git am" and "git rebase" do on --abort + and --skip goes on without it: the cleanup that follows removes + MERGE_RR anyway, and the unresolved entries it would have dropped are + left for the gc. A stale MERGE_RR.lock, which used to stop every + merge, now costs each command a second and a warning until it is + removed. + + That rebase now stops at C the normal way, with its preimage recorded + once the prune is over, and continues once C is resolved. The tests + cover the gc under a held lock, directly and through the maintenance + task, a merge that waits a lock out within rerere.lockTimeout, a + merge, a commit and a rebase that go on without rerere once it is + up, "git rebase --abort" doing the same, and the three explicit + commands failing. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem @@ Documentation/config/rerere.adoc: rerere.enabled:: +rerere.lockTimeout:: + The length of time, in milliseconds, to retry when trying to + take the rerere lock while another process holds it, typically -+ a background `git rerere gc`. Value 0 means not to retry at -+ all; -1 means to try indefinitely. Default is 1000 (i.e., -+ retry for 1 second). `git rerere gc` itself does not wait and -+ skips its run instead. ++ a background `git rerere gc`. When the time is up, the command ++ warns and goes on without rerere. Value 0 means not to retry ++ at all; -1 means to try indefinitely. Default is 1000 (i.e., ++ retry for 1 second). `git rerere gc` does not retry, and ++ `git rerere`, `git rerere forget` and `git rerere clear` fail ++ instead of going on. ## Documentation/git-rerere.adoc ## @@ Documentation/git-rerere.adoc: occurred a long time ago. By default, unresolved conflicts older @@ Documentation/git-rerere.adoc: occurred a long time ago. By default, unresolved DISCUSSION + ## builtin/am.c ## +@@ builtin/am.c: static int clean_index(const struct object_id *head, const struct object_id *rem + static void am_rerere_clear(void) + { + struct string_list merge_rr = STRING_LIST_INIT_DUP; +- rerere_clear(the_repository, &merge_rr); ++ rerere_clear(the_repository, &merge_rr, 0); + string_list_clear(&merge_rr, 1); + } + + + ## builtin/rebase.c ## +@@ builtin/rebase.c: static int run_sequencer_rebase(struct rebase_options *opts) + case ACTION_SKIP: { + struct string_list merge_rr = STRING_LIST_INIT_DUP; + +- rerere_clear(the_repository, &merge_rr); ++ rerere_clear(the_repository, &merge_rr, 0); + } + /* fallthrough */ + case ACTION_CONTINUE: { +@@ builtin/rebase.c: int cmd_rebase(int argc, + case ACTION_SKIP: { + struct string_list merge_rr = STRING_LIST_INIT_DUP; + +- rerere_clear(the_repository, &merge_rr); ++ rerere_clear(the_repository, &merge_rr, 0); + string_list_clear(&merge_rr, 1); + ropts.flags = RESET_HEAD_HARD; + if (reset_head(the_repository, &ropts) < 0) +@@ builtin/rebase.c: int cmd_rebase(int argc, + struct string_list merge_rr = STRING_LIST_INIT_DUP; + struct strbuf head_msg = STRBUF_INIT; + +- rerere_clear(the_repository, &merge_rr); ++ rerere_clear(the_repository, &merge_rr, 0); + string_list_clear(&merge_rr, 1); + + if (read_basic_state(&options)) + + ## builtin/rerere.c ## +@@ builtin/rerere.c: int cmd_rerere(int argc, + flags = RERERE_NOAUTOUPDATE; + + if (argc < 1) +- return repo_rerere(the_repository, flags); ++ return repo_rerere(the_repository, flags | RERERE_LOCK_OR_DIE); + + if (!strcmp(argv[0], "forget")) { + struct pathspec pathspec; +@@ builtin/rerere.c: int cmd_rerere(int argc, + parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD, + prefix, argv + 1); + +- ret = rerere_forget(the_repository, &pathspec); ++ ret = rerere_forget(the_repository, &pathspec, ++ RERERE_LOCK_OR_DIE); + + clear_pathspec(&pathspec); + return ret; + } + + if (!strcmp(argv[0], "clear")) { +- rerere_clear(the_repository, &merge_rr); ++ rerere_clear(the_repository, &merge_rr, RERERE_LOCK_OR_DIE); + } else if (!strcmp(argv[0], "gc")) + rerere_gc(the_repository, &merge_rr); + else if (!strcmp(argv[0], "status")) { + ## rerere.c ## @@ rerere.c: static int rerere_enabled = -1; @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i + if (flags & RERERE_READONLY) { fd = 0; - else -+ } else if (flags & RERERE_SKIP_LOCKED) { - fd = hold_lock_file_for_update(&write_lock, +- fd = hold_lock_file_for_update(&write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); -+ git_path_merge_rr(r), 0); -+ if (fd < 0) { -+ warning_errno(_("unable to lock '%s', skipping"), -+ git_path_merge_rr(r)); -+ return -1; -+ } + } else { ++ int lock_flags = 0; ++ long timeout_ms = rerere_lock_timeout_ms; ++ ++ if (flags & RERERE_LOCK_OR_DIE) ++ lock_flags = LOCK_DIE_ON_ERROR; ++ if (flags & RERERE_NOWAIT) ++ timeout_ms = 0; + /* + * A background "rerere gc" holds the lock for as long as it -+ * takes to walk rr-cache, so wait it out rather than die. ++ * takes to prune rr-cache, so wait it out rather than fail ++ * at once. The gc itself has nothing to lose from a skipped ++ * run and never waits. + */ + fd = hold_lock_file_for_update_timeout(&write_lock, + git_path_merge_rr(r), -+ LOCK_DIE_ON_ERROR, -+ rerere_lock_timeout_ms); ++ lock_flags, timeout_ms); ++ if (fd < 0) { ++ warning_errno(_("skipping rerere, unable to create '%s.lock'"), ++ git_path_merge_rr(r)); ++ return -1; ++ } + } read_rr(r, merge_rr); return fd; } +@@ rerere.c: fail_exit: + return -1; + } + +-int rerere_forget(struct repository *r, struct pathspec *pathspec) ++int rerere_forget(struct repository *r, struct pathspec *pathspec, int flags) + { + int i, fd, ret; + struct string_list conflict = STRING_LIST_INIT_DUP; +@@ rerere.c: int rerere_forget(struct repository *r, struct pathspec *pathspec) + if (repo_read_index(r) < 0) + return error(_("index file corrupt")); + +- fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE); ++ fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE | flags); + if (fd < 0) + return 0; + @@ rerere.c: void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve = now - 60 * 86400; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) -+ if (setup_rerere(r, rr, RERERE_SKIP_LOCKED) < 0) ++ if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) return; repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", +@@ rerere.c: void rerere_gc(struct repository *r, struct string_list *rr) + * + * NEEDSWORK: shouldn't we be calling this from "reset --hard"? + */ +-void rerere_clear(struct repository *r, struct string_list *merge_rr) ++void rerere_clear(struct repository *r, struct string_list *merge_rr, int flags) + { + int i; + +- if (setup_rerere(r, merge_rr, 0) < 0) ++ if (setup_rerere(r, merge_rr, flags) < 0) + return; + + for (i = 0; i < merge_rr->nr; i++) { ## rerere.h ## @@ rerere.h: struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 -+#define RERERE_SKIP_LOCKED 010 ++/* Do not wait for the lock when another process holds it */ ++#define RERERE_NOWAIT 010 ++/* Die on a lock that cannot be taken instead of going on without rerere */ ++#define RERERE_LOCK_OR_DIE 020 /* * Marks paths that have been hand-resolved and added to the +@@ rerere.h: int repo_rerere(struct repository *, int); + */ + const char *rerere_path(struct strbuf *buf, const struct rerere_id *, + const char *file); +-int rerere_forget(struct repository *, struct pathspec *); ++int rerere_forget(struct repository *, struct pathspec *, int); + int rerere_remaining(struct repository *, struct string_list *); +-void rerere_clear(struct repository *, struct string_list *); ++void rerere_clear(struct repository *, struct string_list *, int); + void rerere_gc(struct repository *, struct string_list *); + + #define OPT_RERERE_AUTOUPDATE(v) OPT_UYN(0, "rerere-autoupdate", (v), \ ## t/t4200-rerere.sh ## @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc 2>err && -+ test_grep "MERGE_RR" err && ++ test_grep "MERGE_RR.lock" err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { -+ (sleep 1 && rm -f .git/MERGE_RR.lock) & ++ ( sleep 1 && rm -f .git/MERGE_RR.lock ) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && -+ test_grep ! "Unable to create" err && -+ grep "^=======\$" $rr/preimage ++ test_grep ! "MERGE_RR" err && ++ test_grep "^=======\$" $rr/preimage +' + -+test_expect_success 'rerere.lockTimeout=0 fails at once on a held lock' ' ++test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && ++ test_grep "skipping rerere" err && ++ test_grep "^=======\$" a1 && ++ test_path_is_missing $rr/preimage ++' ++ ++test_expect_success 'commit goes on without rerere once rerere.lockTimeout is up' ' ++ git reset --hard && ++ rm -rf $rr && ++ git checkout -b lock-held-commit third && ++ test_when_finished "git checkout third && git branch -D lock-held-commit" && ++ test_must_fail git merge first && ++ test_path_is_file $rr/preimage && ++ test_when_finished "rm -f .git/MERGE_RR.lock" && ++ >.git/MERGE_RR.lock && ++ echo resolved >a1 && ++ git add a1 && ++ git -c rerere.lockTimeout=0 commit -qm resolved 2>err && ++ test_grep "skipping rerere" err && ++ test_path_is_missing $rr/postimage ++' ++ ++test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' ++ test_when_finished "rm -f .git/MERGE_RR.lock" && ++ >.git/MERGE_RR.lock && ++ test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && ++ test_grep "Unable to create" err && ++ test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && ++ test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && ++ test_grep "Unable to create" err ++' ++ ++test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' ++ git reset --hard && ++ rm -rf $rr && ++ git checkout -b lock-held third && ++ test_when_finished "git checkout third && git branch -D lock-held" && ++ test_when_finished "rm -f .git/MERGE_RR.lock" && ++ >.git/MERGE_RR.lock && ++ test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && ++ test_grep "skipping rerere" err && ++ test_path_is_file .git/rebase-merge/stopped-sha && ++ echo resolved >a1 && ++ git add a1 && ++ git -c rerere.lockTimeout=0 rebase --continue && ++ test_path_is_missing .git/rebase-merge && + test_path_is_missing $rr/preimage +' ++ ++test_expect_success 'rebase --abort goes on without rerere on a held lock' ' ++ git checkout -b lock-held-abort third && ++ test_when_finished "git checkout third && git branch -D lock-held-abort" && ++ test_must_fail git rebase first && ++ test_when_finished "rm -f .git/MERGE_RR.lock" && ++ >.git/MERGE_RR.lock && ++ git -c rerere.lockTimeout=0 rebase --abort 2>err && ++ test_grep "skipping rerere" err && ++ test_path_is_missing .git/rebase-merge ++' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" Documentation/config/rerere.adoc | 10 ++++ Documentation/git-rerere.adoc | 4 +- builtin/am.c | 2 +- builtin/rebase.c | 6 +- builtin/rerere.c | 7 ++- rerere.c | 42 ++++++++++---- rerere.h | 8 ++- t/t4200-rerere.sh | 96 ++++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 8 +++ 9 files changed, 163 insertions(+), 20 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 3a78b5ebb1..b67323fc46 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -10,3 +10,13 @@ rerere.enabled:: enabled if there is an `rr-cache` directory under the `$GIT_DIR`, e.g. if "rerere" was previously used in the repository. + +rerere.lockTimeout:: + The length of time, in milliseconds, to retry when trying to + take the rerere lock while another process holds it, typically + a background `git rerere gc`. When the time is up, the command + warns and goes on without rerere. Value 0 means not to retry + at all; -1 means to try indefinitely. Default is 1000 (i.e., + retry for 1 second). `git rerere gc` does not retry, and + `git rerere`, `git rerere forget` and `git rerere clear` fail + instead of going on. diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc index 4e6ab9a27c..05935b0603 100644 --- a/Documentation/git-rerere.adoc +++ b/Documentation/git-rerere.adoc @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older than 15 days and resolved conflicts older than 60 days are pruned. These defaults are controlled via the `gc.rerereUnresolved` and `gc.rerereResolved` configuration -variables respectively. +variables respectively. If another process holds the lock on the +recorded resolutions, for example a merge or rebase that is recording +a conflict, `gc` does nothing and reports so. DISCUSSION diff --git a/builtin/am.c b/builtin/am.c index e9623b8307..32f11161b4 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -2112,7 +2112,7 @@ static int clean_index(const struct object_id *head, const struct object_id *rem static void am_rerere_clear(void) { struct string_list merge_rr = STRING_LIST_INIT_DUP; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); string_list_clear(&merge_rr, 1); } diff --git a/builtin/rebase.c b/builtin/rebase.c index fa4f5d9306..363d177472 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -367,7 +367,7 @@ static int run_sequencer_rebase(struct rebase_options *opts) case ACTION_SKIP: { struct string_list merge_rr = STRING_LIST_INIT_DUP; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); } /* fallthrough */ case ACTION_CONTINUE: { @@ -1382,7 +1382,7 @@ int cmd_rebase(int argc, case ACTION_SKIP: { struct string_list merge_rr = STRING_LIST_INIT_DUP; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); string_list_clear(&merge_rr, 1); ropts.flags = RESET_HEAD_HARD; if (reset_head(the_repository, &ropts) < 0) @@ -1396,7 +1396,7 @@ int cmd_rebase(int argc, struct string_list merge_rr = STRING_LIST_INIT_DUP; struct strbuf head_msg = STRBUF_INIT; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); string_list_clear(&merge_rr, 1); if (read_basic_state(&options)) diff --git a/builtin/rerere.c b/builtin/rerere.c index a056cb791b..70a4bd1683 100644 --- a/builtin/rerere.c +++ b/builtin/rerere.c @@ -74,7 +74,7 @@ int cmd_rerere(int argc, flags = RERERE_NOAUTOUPDATE; if (argc < 1) - return repo_rerere(the_repository, flags); + return repo_rerere(the_repository, flags | RERERE_LOCK_OR_DIE); if (!strcmp(argv[0], "forget")) { struct pathspec pathspec; @@ -85,14 +85,15 @@ int cmd_rerere(int argc, parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD, prefix, argv + 1); - ret = rerere_forget(the_repository, &pathspec); + ret = rerere_forget(the_repository, &pathspec, + RERERE_LOCK_OR_DIE); clear_pathspec(&pathspec); return ret; } if (!strcmp(argv[0], "clear")) { - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, RERERE_LOCK_OR_DIE); } else if (!strcmp(argv[0], "gc")) rerere_gc(the_repository, &merge_rr); else if (!strcmp(argv[0], "status")) { diff --git a/rerere.c b/rerere.c index 8232542585..4e2ececc09 100644 --- a/rerere.c +++ b/rerere.c @@ -32,6 +32,7 @@ static int rerere_enabled = -1; /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; +static int rerere_lock_timeout_ms = 1000; #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 @@ -876,6 +877,8 @@ static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); + repo_config_get_int(the_repository, "rerere.locktimeout", + &rerere_lock_timeout_ms); repo_config(the_repository, git_default_config, NULL); } @@ -908,12 +911,31 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) + if (flags & RERERE_READONLY) { fd = 0; - else - fd = hold_lock_file_for_update(&write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + } else { + int lock_flags = 0; + long timeout_ms = rerere_lock_timeout_ms; + + if (flags & RERERE_LOCK_OR_DIE) + lock_flags = LOCK_DIE_ON_ERROR; + if (flags & RERERE_NOWAIT) + timeout_ms = 0; + /* + * A background "rerere gc" holds the lock for as long as it + * takes to prune rr-cache, so wait it out rather than fail + * at once. The gc itself has nothing to lose from a skipped + * run and never waits. + */ + fd = hold_lock_file_for_update_timeout(&write_lock, + git_path_merge_rr(r), + lock_flags, timeout_ms); + if (fd < 0) { + warning_errno(_("skipping rerere, unable to create '%s.lock'"), + git_path_merge_rr(r)); + return -1; + } + } read_rr(r, merge_rr); return fd; } @@ -1124,7 +1146,7 @@ fail_exit: return -1; } -int rerere_forget(struct repository *r, struct pathspec *pathspec) +int rerere_forget(struct repository *r, struct pathspec *pathspec, int flags) { int i, fd, ret; struct string_list conflict = STRING_LIST_INIT_DUP; @@ -1133,7 +1155,7 @@ int rerere_forget(struct repository *r, struct pathspec *pathspec) if (repo_read_index(r) < 0) return error(_("index file corrupt")); - fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE); + fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE | flags); if (fd < 0) return 0; @@ -1237,7 +1259,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve = now - 60 * 86400; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) + if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) return; repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", @@ -1289,11 +1311,11 @@ void rerere_gc(struct repository *r, struct string_list *rr) * * NEEDSWORK: shouldn't we be calling this from "reset --hard"? */ -void rerere_clear(struct repository *r, struct string_list *merge_rr) +void rerere_clear(struct repository *r, struct string_list *merge_rr, int flags) { int i; - if (setup_rerere(r, merge_rr, 0) < 0) + if (setup_rerere(r, merge_rr, flags) < 0) return; for (i = 0; i < merge_rr->nr; i++) { diff --git a/rerere.h b/rerere.h index d4b5f7c932..3a9f58acd9 100644 --- a/rerere.h +++ b/rerere.h @@ -10,6 +10,10 @@ struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 +/* Do not wait for the lock when another process holds it */ +#define RERERE_NOWAIT 010 +/* Die on a lock that cannot be taken instead of going on without rerere */ +#define RERERE_LOCK_OR_DIE 020 /* * Marks paths that have been hand-resolved and added to the @@ -34,9 +38,9 @@ int repo_rerere(struct repository *, int); */ const char *rerere_path(struct strbuf *buf, const struct rerere_id *, const char *file); -int rerere_forget(struct repository *, struct pathspec *); +int rerere_forget(struct repository *, struct pathspec *, int); int rerere_remaining(struct repository *, struct string_list *); -void rerere_clear(struct repository *, struct string_list *); +void rerere_clear(struct repository *, struct string_list *, int); void rerere_gc(struct repository *, struct string_list *); #define OPT_RERERE_AUTOUPDATE(v) OPT_UYN(0, "rerere-autoupdate", (v), \ diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 1717f407c8..243b3ebed3 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,102 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'gc does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc 2>err && + test_grep "MERGE_RR.lock" err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && + git rerere gc && + test_path_is_missing $rr2/preimage +' + +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { + ( sleep 1 && rm -f .git/MERGE_RR.lock ) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && + test_grep ! "MERGE_RR" err && + test_grep "^=======\$" $rr/preimage +' + +test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 && + test_path_is_missing $rr/preimage +' + +test_expect_success 'commit goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-commit third && + test_when_finished "git checkout third && git branch -D lock-held-commit" && + test_must_fail git merge first && + test_path_is_file $rr/preimage && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + echo resolved >a1 && + git add a1 && + git -c rerere.lockTimeout=0 commit -qm resolved 2>err && + test_grep "skipping rerere" err && + test_path_is_missing $rr/postimage +' + +test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && + test_grep "Unable to create" err +' + +test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held third && + test_when_finished "git checkout third && git branch -D lock-held" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && + test_grep "skipping rerere" err && + test_path_is_file .git/rebase-merge/stopped-sha && + echo resolved >a1 && + git add a1 && + git -c rerere.lockTimeout=0 rebase --continue && + test_path_is_missing .git/rebase-merge && + test_path_is_missing $rr/preimage +' + +test_expect_success 'rebase --abort goes on without rerere on a held lock' ' + git checkout -b lock-held-abort third && + test_when_finished "git checkout third && git branch -D lock-held-abort" && + test_must_fail git rebase first && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git -c rerere.lockTimeout=0 rebase --abort 2>err && + test_grep "skipping rerere" err && + test_path_is_missing .git/rebase-merge +' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index d7f82e1bec..a55ca2e829 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc ' +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && + mkdir .git/rr-cache && + : >.git/rr-cache/entry && + >.git/MERGE_RR.lock && + test_expect_rerere_gc git maintenance run --task=rerere-gc +' + test_expect_success '--auto and --schedule incompatible' ' test_must_fail git maintenance run --auto --schedule=daily 2>err && test_grep "cannot be used together" err base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc -- gitgitgadget ``` ## Phillip Wood, 2026-09-04 15:21 Subject: Re: [PATCH v2] rerere: keep a background gc from killing a rebase Message-ID: <5e613735-60e2-429d-a5bb-1a4f03578604@gmail.com> In-Reply-To: ``` Hi Thomas On 04/09/2026 08:44, Thomas Bachem via GitGitGadget wrote: > From: Thomas Bachem > > Since 2.54 unscheduled maintenance uses the "geometric" strategy, so > the "git maintenance run --auto --detach" behind every "git commit" > runs "git rerere gc" in the background whenever rr-cache has an entry. With Patricks patches that's no-longer true I think. I think a better motivation, as the cache is per-repository, rather than per-worktree, is concurrent writers running in different worktrees. That makes the timeout much more sensible as we expect writing a conflict resolution to be much faster than gc. Overall, this commit message is rather long and it would be helpful if you could distill it to remove unnecessary and unrelated details. > > Documentation/config/rerere.adoc | 10 ++++ > Documentation/git-rerere.adoc | 4 +- > builtin/am.c | 2 +- > builtin/rebase.c | 6 +- > builtin/rerere.c | 7 ++- > rerere.c | 42 ++++++++++---- > rerere.h | 8 ++- > t/t4200-rerere.sh | 96 ++++++++++++++++++++++++++++++++ > t/t7900-maintenance.sh | 8 +++ > 9 files changed, 163 insertions(+), 20 deletions(-) > > diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc > index 3a78b5ebb1..b67323fc46 100644 > --- a/Documentation/config/rerere.adoc > +++ b/Documentation/config/rerere.adoc > @@ -10,3 +10,13 @@ rerere.enabled:: > enabled if there is an `rr-cache` directory under the > `$GIT_DIR`, e.g. if "rerere" was previously used in the > repository. > + > +rerere.lockTimeout:: > + The length of time, in milliseconds, to retry when trying to > + take the rerere lock while another process holds it, typically > + a background `git rerere gc`. When the time is up, the command > + warns and goes on without rerere. Value 0 means not to retry > + at all; -1 means to try indefinitely. Default is 1000 (i.e., > + retry for 1 second). `git rerere gc` does not retry, and > + `git rerere`, `git rerere forget` and `git rerere clear` fail > + instead of going on. Why do those commands fail rather than wait? > @@ -908,12 +911,31 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) > > if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) > rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); > - if (flags & RERERE_READONLY) > + if (flags & RERERE_READONLY) { > fd = 0; > - else > - fd = hold_lock_file_for_update(&write_lock, > - git_path_merge_rr(r), > - LOCK_DIE_ON_ERROR); > + } else { > + int lock_flags = 0; > + long timeout_ms = rerere_lock_timeout_ms; > + > + if (flags & RERERE_LOCK_OR_DIE) > + lock_flags = LOCK_DIE_ON_ERROR; > + if (flags & RERERE_NOWAIT) > + timeout_ms = 0; It might be worth adding a check above here that BUG()s out if the caller passes an incompatible set of flags. > + /* > + * A background "rerere gc" holds the lock for as long as it > + * takes to prune rr-cache, so wait it out rather than fail > + * at once. The gc itself has nothing to lose from a skipped > + * run and never waits. > + */ > + fd = hold_lock_file_for_update_timeout(&write_lock, > + git_path_merge_rr(r), > + lock_flags, timeout_ms); > + if (fd < 0) { > + warning_errno(_("skipping rerere, unable to create '%s.lock'"), > + git_path_merge_rr(r)); A background job that the user did not explicitly start printing to the terminal is rather confusing as it is likely to get mixed in with the output of whatever is running in the foreground. Thanks Phillip > + return -1; > + } > + } > read_rr(r, merge_rr); > return fd; > } > @@ -1124,7 +1146,7 @@ fail_exit: > return -1; > } > > -int rerere_forget(struct repository *r, struct pathspec *pathspec) > +int rerere_forget(struct repository *r, struct pathspec *pathspec, int flags) > { > int i, fd, ret; > struct string_list conflict = STRING_LIST_INIT_DUP; > @@ -1133,7 +1155,7 @@ int rerere_forget(struct repository *r, struct pathspec *pathspec) > if (repo_read_index(r) < 0) > return error(_("index file corrupt")); > > - fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE); > + fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE | flags); > if (fd < 0) > return 0; > > @@ -1237,7 +1259,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) > timestamp_t cutoff_resolve = now - 60 * 86400; > struct strbuf buf = STRBUF_INIT; > > - if (setup_rerere(r, rr, 0) < 0) > + if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) > return; > > repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", > @@ -1289,11 +1311,11 @@ void rerere_gc(struct repository *r, struct string_list *rr) > * > * NEEDSWORK: shouldn't we be calling this from "reset --hard"? > */ > -void rerere_clear(struct repository *r, struct string_list *merge_rr) > +void rerere_clear(struct repository *r, struct string_list *merge_rr, int flags) > { > int i; > > - if (setup_rerere(r, merge_rr, 0) < 0) > + if (setup_rerere(r, merge_rr, flags) < 0) > return; > > for (i = 0; i < merge_rr->nr; i++) { > diff --git a/rerere.h b/rerere.h > index d4b5f7c932..3a9f58acd9 100644 > --- a/rerere.h > +++ b/rerere.h > @@ -10,6 +10,10 @@ struct repository; > #define RERERE_AUTOUPDATE 01 > #define RERERE_NOAUTOUPDATE 02 > #define RERERE_READONLY 04 > +/* Do not wait for the lock when another process holds it */ > +#define RERERE_NOWAIT 010 > +/* Die on a lock that cannot be taken instead of going on without rerere */ > +#define RERERE_LOCK_OR_DIE 020 > > /* > * Marks paths that have been hand-resolved and added to the > @@ -34,9 +38,9 @@ int repo_rerere(struct repository *, int); > */ > const char *rerere_path(struct strbuf *buf, const struct rerere_id *, > const char *file); > -int rerere_forget(struct repository *, struct pathspec *); > +int rerere_forget(struct repository *, struct pathspec *, int); > int rerere_remaining(struct repository *, struct string_list *); > -void rerere_clear(struct repository *, struct string_list *); > +void rerere_clear(struct repository *, struct string_list *, int); > void rerere_gc(struct repository *, struct string_list *); > > #define OPT_RERERE_AUTOUPDATE(v) OPT_UYN(0, "rerere-autoupdate", (v), \ > diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh > index 1717f407c8..243b3ebed3 100755 > --- a/t/t4200-rerere.sh > +++ b/t/t4200-rerere.sh > @@ -242,6 +242,102 @@ test_expect_success 'old records rest in peace' ' > test_path_is_missing $rr2/preimage > ' > > +test_expect_success 'gc does nothing while MERGE_RR is locked' ' > + mkdir -p $rr2 && > + echo Hello >$rr2/preimage && > + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && > + > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + git rerere gc 2>err && > + test_grep "MERGE_RR.lock" err && > + test_path_is_file $rr2/preimage && > + > + rm .git/MERGE_RR.lock && > + git rerere gc && > + test_path_is_missing $rr2/preimage > +' > + > +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' > + git reset --hard && > + rm -rf $rr && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + { > + ( sleep 1 && rm -f .git/MERGE_RR.lock ) & > + } && > + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && > + wait && > + test_grep ! "MERGE_RR" err && > + test_grep "^=======\$" $rr/preimage > +' > + > +test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' > + git reset --hard && > + rm -rf $rr && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && > + test_grep "skipping rerere" err && > + test_grep "^=======\$" a1 && > + test_path_is_missing $rr/preimage > +' > + > +test_expect_success 'commit goes on without rerere once rerere.lockTimeout is up' ' > + git reset --hard && > + rm -rf $rr && > + git checkout -b lock-held-commit third && > + test_when_finished "git checkout third && git branch -D lock-held-commit" && > + test_must_fail git merge first && > + test_path_is_file $rr/preimage && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + echo resolved >a1 && > + git add a1 && > + git -c rerere.lockTimeout=0 commit -qm resolved 2>err && > + test_grep "skipping rerere" err && > + test_path_is_missing $rr/postimage > +' > + > +test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && > + test_grep "Unable to create" err && > + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && > + test_grep "Unable to create" err && > + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && > + test_grep "Unable to create" err > +' > + > +test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' > + git reset --hard && > + rm -rf $rr && > + git checkout -b lock-held third && > + test_when_finished "git checkout third && git branch -D lock-held" && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && > + test_grep "skipping rerere" err && > + test_path_is_file .git/rebase-merge/stopped-sha && > + echo resolved >a1 && > + git add a1 && > + git -c rerere.lockTimeout=0 rebase --continue && > + test_path_is_missing .git/rebase-merge && > + test_path_is_missing $rr/preimage > +' > + > +test_expect_success 'rebase --abort goes on without rerere on a held lock' ' > + git checkout -b lock-held-abort third && > + test_when_finished "git checkout third && git branch -D lock-held-abort" && > + test_must_fail git rebase first && > + test_when_finished "rm -f .git/MERGE_RR.lock" && > + >.git/MERGE_RR.lock && > + git -c rerere.lockTimeout=0 rebase --abort 2>err && > + test_grep "skipping rerere" err && > + test_path_is_missing .git/rebase-merge > +' > + > rerere_gc_custom_expiry_test () { > five_days="$1" right_now="$2" > test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' > diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh > index d7f82e1bec..a55ca2e829 100755 > --- a/t/t7900-maintenance.sh > +++ b/t/t7900-maintenance.sh > @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut > test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc > ' > > +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' > + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && > + mkdir .git/rr-cache && > + : >.git/rr-cache/entry && > + >.git/MERGE_RR.lock && > + test_expect_rerere_gc git maintenance run --task=rerere-gc > +' > + > test_expect_success '--auto and --schedule incompatible' ' > test_must_fail git maintenance run --auto --schedule=daily 2>err && > test_grep "cannot be used together" err > > base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc ``` ## Thomas Bachem via GitGitGadget, 2026-09-04 15:51 Subject: [PATCH v3] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` From: Thomas Bachem A "git rerere gc" holds MERGE_RR.lock for as long as pruning rr-cache takes, and since 2.54 the auto maintenance after every commit runs one whenever rr-cache has an entry. The commit a rebase spawns for a resolved pick starts it too, and the sequencer's repo_rerere() at the next conflict wants the lock a few milliseconds later. Both take it with LOCK_DIE_ON_ERROR, so whichever comes second dies. When it is the rebase, the index is written but the state for "git rebase --continue" is not, and every later continue refuses with "you have staged changes". The gc needs the lock, since a rerere that has just created its directory looks like the empty ones it prunes. So wait for it instead, rerere.lockTimeout milliseconds, 1000 by default with the semantics of core.packedRefsTimeout, then warn and go on without rerere: a lost recording or replay is nothing next to a rebase that cannot continue. The gc itself never waits, and "git rerere", "git rerere forget" and "git rerere clear" wait but then die, since the state behind the lock is all they are for. The clearing "am" and "rebase" do on --abort, --skip and --quit goes on without it, and leaves the entries for the gc. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- rerere: keep a background gc from killing a rebase Changes since v2: * The description is a quarter of the size and says what the patch does and why (Junio, Phillip). * setup_rerere() BUG()s on RERERE_NOWAIT combined with RERERE_LOCK_OR_DIE, and on RERERE_READONLY combined with either (Phillip). * The rerere.lockTimeout entry now says that "git rerere", "git rerere forget" and "git rerere clear" retry like everything else and only fail once the time is up (Phillip). Patrick's heuristic in ps/tune-rerere-gc narrows the auto trigger from "rr-cache has an entry" to "enough entries are stale". A gc that does run races the same way, so the fix stands on its own. Still based on maint, where the bug ships (2.54.0 and 2.55.0). Merged up it conflicts with 2e486bfbf7 (use repo_hold_lock_file_for_update{,_mode,_timeout}() with custom repos, 2026-07-14) in setup_rerere(), where the resolution takes the repo-scoped helper. Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v3 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v3 Pull-Request: https://github.com/gitgitgadget/git/pull/2214 Range-diff vs v2: 1: bbb2338572 ! 1: 5bfda65baa rerere: keep a background gc from killing a rebase @@ Metadata ## Commit message ## rerere: keep a background gc from killing a rebase - Since 2.54 unscheduled maintenance uses the "geometric" strategy, so - the "git maintenance run --auto --detach" behind every "git commit" - runs "git rerere gc" in the background whenever rr-cache has an entry. - That includes the "git commit" the sequencer runs for a resolved pick - on "git rebase --continue". + A "git rerere gc" holds MERGE_RR.lock for as long as pruning rr-cache + takes, and since 2.54 the auto maintenance after every commit runs + one whenever rr-cache has an entry. The commit a rebase spawns for a + resolved pick starts it too, and the sequencer's repo_rerere() at the + next conflict wants the lock a few milliseconds later. Both take it + with LOCK_DIE_ON_ERROR, so whichever comes second dies. When it is + the rebase, the index is written but the state for "git rebase + --continue" is not, and every later continue refuses with "you have + staged changes". - rerere_gc() takes MERGE_RR.lock through setup_rerere(), which uses - LOCK_DIE_ON_ERROR, and so does the sequencer's repo_rerere() at the - next conflict a few milliseconds later. Whichever comes second dies. - When it is the rebase, it dies in do_pick_commit() with the index - written but before make_patch() writes rebase-merge/{message,patch, - stopped-sha}, and every later "git rebase --continue" refuses with - "you have staged changes in your working tree". When it is the "git - commit" of a later continue, that one dies in its post-commit - repo_rerere() after the commit was made. Before 2.54 the same - collision needed an auto gc to actually run, since gc runs - "rerere gc" at its end. - - A rebase with two conflicts in a row shows it. The filler makes the - pick slower than the ~5 ms the background task needs to take the - lock, and the stale entries give the gc something to prune, which - keeps the lock held for about half a second. It hit 6 of 6 runs here - on 2.55.0, and a test suite driving rebases on toy repositories with - a single rr-cache entry hit it in both runs that were traced: - - git init -q -b main r && cd r - git config rerere.enabled true - git config maintenance.auto false - mkdir pad && seq 20000 | (cd pad && split -l 1 -a 5) - echo base >f && git add -A && git commit -qm base - git checkout -q -b topic - echo b >f && git commit -qam B - echo c >f && git commit -qam C - git checkout -q main - echo a >f && git commit -qam A - git repack -adq - git ls-files -s pad | head -n 5000 | - awk '{print ".git/rr-cache/" $2}' | xargs mkdir -p - for d in .git/rr-cache/*/; do echo x >$d/preimage; done - touch -t 202001010000 .git/rr-cache/*/preimage - git config --unset maintenance.auto - git checkout -q topic - git rebase main - echo ab >f && git add f - GIT_EDITOR=true git rebase --continue - - The continue dies with "Unable to create '.git/MERGE_RR.lock': File - exists" while the gc spawned by its own commit holds the lock, and - after resolving C every further continue refuses. Maintenance stays - off during the setup so that no repack is pending: a repack due at - that commit runs ahead of rerere-gc in the task list and would spend - the window. - - The gc needs the lock: it removes every rr-cache directory it finds - empty, and a rerere that has just created its directory but not yet - written the preimage looks exactly like that. So keep the lock and - stop dying over it. A caller that finds the lock held now waits for - rerere.lockTimeout milliseconds, with the semantics of - core.packedRefsTimeout and the same default of 1000, and then warns - and goes on without rerere: a merge, a commit or a pick loses one - recording or replay, which is nothing next to a rebase that cannot - continue. The gc itself never waits, since it has nothing to lose - from a skipped run, and "git rerere", "git rerere forget" and "git - rerere clear" keep dying, since they exist for nothing but the state - behind the lock. The clearing "git am" and "git rebase" do on --abort - and --skip goes on without it: the cleanup that follows removes - MERGE_RR anyway, and the unresolved entries it would have dropped are - left for the gc. A stale MERGE_RR.lock, which used to stop every - merge, now costs each command a second and a warning until it is - removed. - - That rebase now stops at C the normal way, with its preimage recorded - once the prune is over, and continues once C is resolved. The tests - cover the gc under a held lock, directly and through the maintenance - task, a merge that waits a lock out within rerere.lockTimeout, a - merge, a commit and a rebase that go on without rerere once it is - up, "git rebase --abort" doing the same, and the three explicit - commands failing. + The gc needs the lock, since a rerere that has just created its + directory looks like the empty ones it prunes. So wait for it + instead, rerere.lockTimeout milliseconds, 1000 by default with the + semantics of core.packedRefsTimeout, then warn and go on without + rerere: a lost recording or replay is nothing next to a rebase that + cannot continue. The gc itself never waits, and "git rerere", "git + rerere forget" and "git rerere clear" wait but then die, since the + state behind the lock is all they are for. The clearing "am" and + "rebase" do on --abort, --skip and --quit goes on without it, and + leaves the entries for the gc. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem @@ Documentation/config/rerere.adoc: rerere.enabled:: + a background `git rerere gc`. When the time is up, the command + warns and goes on without rerere. Value 0 means not to retry + at all; -1 means to try indefinitely. Default is 1000 (i.e., -+ retry for 1 second). `git rerere gc` does not retry, and -+ `git rerere`, `git rerere forget` and `git rerere clear` fail -+ instead of going on. ++ retry for 1 second). `git rerere gc` does not retry at all. ++ `git rerere`, `git rerere forget` and `git rerere clear` retry ++ the same way, but fail when the time is up instead of going on. ## Documentation/git-rerere.adoc ## @@ Documentation/git-rerere.adoc: occurred a long time ago. By default, unresolved conflicts older @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) ++ if ((flags & RERERE_NOWAIT) && (flags & RERERE_LOCK_OR_DIE)) ++ BUG("RERERE_NOWAIT and RERERE_LOCK_OR_DIE are mutually exclusive"); ++ if ((flags & RERERE_READONLY) && ++ (flags & (RERERE_NOWAIT | RERERE_LOCK_OR_DIE))) ++ BUG("RERERE_READONLY takes no lock, so no lock flag applies"); + if (flags & RERERE_READONLY) { fd = 0; - else Documentation/config/rerere.adoc | 10 ++++ Documentation/git-rerere.adoc | 4 +- builtin/am.c | 2 +- builtin/rebase.c | 6 +- builtin/rerere.c | 7 ++- rerere.c | 47 ++++++++++++---- rerere.h | 8 ++- t/t4200-rerere.sh | 96 ++++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 8 +++ 9 files changed, 168 insertions(+), 20 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 3a78b5ebb1..14ef193545 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -10,3 +10,13 @@ rerere.enabled:: enabled if there is an `rr-cache` directory under the `$GIT_DIR`, e.g. if "rerere" was previously used in the repository. + +rerere.lockTimeout:: + The length of time, in milliseconds, to retry when trying to + take the rerere lock while another process holds it, typically + a background `git rerere gc`. When the time is up, the command + warns and goes on without rerere. Value 0 means not to retry + at all; -1 means to try indefinitely. Default is 1000 (i.e., + retry for 1 second). `git rerere gc` does not retry at all. + `git rerere`, `git rerere forget` and `git rerere clear` retry + the same way, but fail when the time is up instead of going on. diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc index 4e6ab9a27c..05935b0603 100644 --- a/Documentation/git-rerere.adoc +++ b/Documentation/git-rerere.adoc @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older than 15 days and resolved conflicts older than 60 days are pruned. These defaults are controlled via the `gc.rerereUnresolved` and `gc.rerereResolved` configuration -variables respectively. +variables respectively. If another process holds the lock on the +recorded resolutions, for example a merge or rebase that is recording +a conflict, `gc` does nothing and reports so. DISCUSSION diff --git a/builtin/am.c b/builtin/am.c index e9623b8307..32f11161b4 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -2112,7 +2112,7 @@ static int clean_index(const struct object_id *head, const struct object_id *rem static void am_rerere_clear(void) { struct string_list merge_rr = STRING_LIST_INIT_DUP; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); string_list_clear(&merge_rr, 1); } diff --git a/builtin/rebase.c b/builtin/rebase.c index fa4f5d9306..363d177472 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -367,7 +367,7 @@ static int run_sequencer_rebase(struct rebase_options *opts) case ACTION_SKIP: { struct string_list merge_rr = STRING_LIST_INIT_DUP; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); } /* fallthrough */ case ACTION_CONTINUE: { @@ -1382,7 +1382,7 @@ int cmd_rebase(int argc, case ACTION_SKIP: { struct string_list merge_rr = STRING_LIST_INIT_DUP; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); string_list_clear(&merge_rr, 1); ropts.flags = RESET_HEAD_HARD; if (reset_head(the_repository, &ropts) < 0) @@ -1396,7 +1396,7 @@ int cmd_rebase(int argc, struct string_list merge_rr = STRING_LIST_INIT_DUP; struct strbuf head_msg = STRBUF_INIT; - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, 0); string_list_clear(&merge_rr, 1); if (read_basic_state(&options)) diff --git a/builtin/rerere.c b/builtin/rerere.c index a056cb791b..70a4bd1683 100644 --- a/builtin/rerere.c +++ b/builtin/rerere.c @@ -74,7 +74,7 @@ int cmd_rerere(int argc, flags = RERERE_NOAUTOUPDATE; if (argc < 1) - return repo_rerere(the_repository, flags); + return repo_rerere(the_repository, flags | RERERE_LOCK_OR_DIE); if (!strcmp(argv[0], "forget")) { struct pathspec pathspec; @@ -85,14 +85,15 @@ int cmd_rerere(int argc, parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD, prefix, argv + 1); - ret = rerere_forget(the_repository, &pathspec); + ret = rerere_forget(the_repository, &pathspec, + RERERE_LOCK_OR_DIE); clear_pathspec(&pathspec); return ret; } if (!strcmp(argv[0], "clear")) { - rerere_clear(the_repository, &merge_rr); + rerere_clear(the_repository, &merge_rr, RERERE_LOCK_OR_DIE); } else if (!strcmp(argv[0], "gc")) rerere_gc(the_repository, &merge_rr); else if (!strcmp(argv[0], "status")) { diff --git a/rerere.c b/rerere.c index 8232542585..bae780f584 100644 --- a/rerere.c +++ b/rerere.c @@ -32,6 +32,7 @@ static int rerere_enabled = -1; /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; +static int rerere_lock_timeout_ms = 1000; #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 @@ -876,6 +877,8 @@ static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); + repo_config_get_int(the_repository, "rerere.locktimeout", + &rerere_lock_timeout_ms); repo_config(the_repository, git_default_config, NULL); } @@ -908,12 +911,36 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) + if ((flags & RERERE_NOWAIT) && (flags & RERERE_LOCK_OR_DIE)) + BUG("RERERE_NOWAIT and RERERE_LOCK_OR_DIE are mutually exclusive"); + if ((flags & RERERE_READONLY) && + (flags & (RERERE_NOWAIT | RERERE_LOCK_OR_DIE))) + BUG("RERERE_READONLY takes no lock, so no lock flag applies"); + if (flags & RERERE_READONLY) { fd = 0; - else - fd = hold_lock_file_for_update(&write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + } else { + int lock_flags = 0; + long timeout_ms = rerere_lock_timeout_ms; + + if (flags & RERERE_LOCK_OR_DIE) + lock_flags = LOCK_DIE_ON_ERROR; + if (flags & RERERE_NOWAIT) + timeout_ms = 0; + /* + * A background "rerere gc" holds the lock for as long as it + * takes to prune rr-cache, so wait it out rather than fail + * at once. The gc itself has nothing to lose from a skipped + * run and never waits. + */ + fd = hold_lock_file_for_update_timeout(&write_lock, + git_path_merge_rr(r), + lock_flags, timeout_ms); + if (fd < 0) { + warning_errno(_("skipping rerere, unable to create '%s.lock'"), + git_path_merge_rr(r)); + return -1; + } + } read_rr(r, merge_rr); return fd; } @@ -1124,7 +1151,7 @@ fail_exit: return -1; } -int rerere_forget(struct repository *r, struct pathspec *pathspec) +int rerere_forget(struct repository *r, struct pathspec *pathspec, int flags) { int i, fd, ret; struct string_list conflict = STRING_LIST_INIT_DUP; @@ -1133,7 +1160,7 @@ int rerere_forget(struct repository *r, struct pathspec *pathspec) if (repo_read_index(r) < 0) return error(_("index file corrupt")); - fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE); + fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE | flags); if (fd < 0) return 0; @@ -1237,7 +1264,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve = now - 60 * 86400; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) + if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) return; repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", @@ -1289,11 +1316,11 @@ void rerere_gc(struct repository *r, struct string_list *rr) * * NEEDSWORK: shouldn't we be calling this from "reset --hard"? */ -void rerere_clear(struct repository *r, struct string_list *merge_rr) +void rerere_clear(struct repository *r, struct string_list *merge_rr, int flags) { int i; - if (setup_rerere(r, merge_rr, 0) < 0) + if (setup_rerere(r, merge_rr, flags) < 0) return; for (i = 0; i < merge_rr->nr; i++) { diff --git a/rerere.h b/rerere.h index d4b5f7c932..3a9f58acd9 100644 --- a/rerere.h +++ b/rerere.h @@ -10,6 +10,10 @@ struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 +/* Do not wait for the lock when another process holds it */ +#define RERERE_NOWAIT 010 +/* Die on a lock that cannot be taken instead of going on without rerere */ +#define RERERE_LOCK_OR_DIE 020 /* * Marks paths that have been hand-resolved and added to the @@ -34,9 +38,9 @@ int repo_rerere(struct repository *, int); */ const char *rerere_path(struct strbuf *buf, const struct rerere_id *, const char *file); -int rerere_forget(struct repository *, struct pathspec *); +int rerere_forget(struct repository *, struct pathspec *, int); int rerere_remaining(struct repository *, struct string_list *); -void rerere_clear(struct repository *, struct string_list *); +void rerere_clear(struct repository *, struct string_list *, int); void rerere_gc(struct repository *, struct string_list *); #define OPT_RERERE_AUTOUPDATE(v) OPT_UYN(0, "rerere-autoupdate", (v), \ diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 1717f407c8..243b3ebed3 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,102 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'gc does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc 2>err && + test_grep "MERGE_RR.lock" err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && + git rerere gc && + test_path_is_missing $rr2/preimage +' + +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { + ( sleep 1 && rm -f .git/MERGE_RR.lock ) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && + test_grep ! "MERGE_RR" err && + test_grep "^=======\$" $rr/preimage +' + +test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 && + test_path_is_missing $rr/preimage +' + +test_expect_success 'commit goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-commit third && + test_when_finished "git checkout third && git branch -D lock-held-commit" && + test_must_fail git merge first && + test_path_is_file $rr/preimage && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + echo resolved >a1 && + git add a1 && + git -c rerere.lockTimeout=0 commit -qm resolved 2>err && + test_grep "skipping rerere" err && + test_path_is_missing $rr/postimage +' + +test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && + test_grep "Unable to create" err +' + +test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held third && + test_when_finished "git checkout third && git branch -D lock-held" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && + test_grep "skipping rerere" err && + test_path_is_file .git/rebase-merge/stopped-sha && + echo resolved >a1 && + git add a1 && + git -c rerere.lockTimeout=0 rebase --continue && + test_path_is_missing .git/rebase-merge && + test_path_is_missing $rr/preimage +' + +test_expect_success 'rebase --abort goes on without rerere on a held lock' ' + git checkout -b lock-held-abort third && + test_when_finished "git checkout third && git branch -D lock-held-abort" && + test_must_fail git rebase first && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git -c rerere.lockTimeout=0 rebase --abort 2>err && + test_grep "skipping rerere" err && + test_path_is_missing .git/rebase-merge +' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index d7f82e1bec..a55ca2e829 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -885,6 +885,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc ' +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && + mkdir .git/rr-cache && + : >.git/rr-cache/entry && + >.git/MERGE_RR.lock && + test_expect_rerere_gc git maintenance run --task=rerere-gc +' + test_expect_success '--auto and --schedule incompatible' ' test_must_fail git maintenance run --auto --schedule=daily 2>err && test_grep "cannot be used together" err base-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc -- gitgitgadget ``` ## Thomas Bachem, 2026-09-04 15:55 Subject: Re: [PATCH v2] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: <5e613735-60e2-429d-a5bb-1a4f03578604@gmail.com> ``` Hi Phillip, On 04/09/2026 16:21, Phillip Wood wrote: > With Patricks patches that's no-longer true I think. I think a better > motivation, as the cache is per-repository, rather than per-worktree, is > concurrent writers running in different worktrees. MERGE_RR is per worktree, though, and so is its lock: $ git -C linked rev-parse --git-path MERGE_RR /path/to/main/.git/worktrees/linked/MERGE_RR so writers in different worktrees never meet on it. What they share is rr-cache, which a gc in one worktree prunes under its own worktree's lock only. That is a gap of its own, and not one this patch closes. What remains after Patrick's series is any "git rerere gc" that runs while a command records a conflict, from "git gc", from a maintenance run, or from auto maintenance once enough entries are stale. The v3 message says it that way. > Overall, this commit message is rather long and it would be helpful if > you could distill it to remove unnecessary and unrelated details. Done, it is a quarter of the size now. > Why do those commands fail rather than wait? They wait like everything else, and once the time is up they fail instead of going on without rerere, which is all they are for. That way a stale lock gets the usual advice to remove it. The config text said otherwise, fixed. > It might be worth adding a check above here that BUG()s out if the > caller passes an incompatible set of flags. Added, for RERERE_NOWAIT with RERERE_LOCK_OR_DIE and for RERERE_READONLY with either. > A background job that the user did not explicitly start printing to the > terminal is rather confusing as it is likely to get mixed in with the > output of whatever is running in the foreground. The detached maintenance run has no terminal: daemonize() closes the standard descriptors and reopens them on /dev/null, so the gc's warning goes nowhere when it loses the lock. Where it cannot detach, on Windows, it runs in the foreground of the commit that started it and there is no race to lose. The warning the user does see is the foreground command's own, when it gives up waiting. Thanks, Thomas ``` ## Junio C Hamano, 2026-09-04 17:06 Subject: Re: [PATCH v2] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: <5e613735-60e2-429d-a5bb-1a4f03578604@gmail.com> ``` Phillip Wood writes: > Overall, this commit message is rather long and it would be helpful if > you could distill it to remove unnecessary and unrelated details. Hear hear. >> +rerere.lockTimeout:: >> + The length of time, in milliseconds, to retry when trying to >> + take the rerere lock while another process holds it, typically >> + a background `git rerere gc`. When the time is up, the command >> + warns and goes on without rerere. Value 0 means not to retry >> + at all; -1 means to try indefinitely. Default is 1000 (i.e., >> + retry for 1 second). `git rerere gc` does not retry, and >> + `git rerere`, `git rerere forget` and `git rerere clear` fail >> + instead of going on. > > Why do those commands fail rather than wait? Isn't locktimeout about waiting? After waiting enough, why should it not fail but proceed? When there is somebody holding the lock, they acquired the lock exactly because they did not want to see others (including ourselves) to touch the rerere database until they are done. The description "`git rerere gc` does not retry" is highly questionable. None of the others retries, either. What makes `git rerere gc` different among all is not that it does not retry. It just does not insist doing a GC and instead leaves without doing anything (and without failing). I think this is justifyable as the actions visible to end-users of "rerere gc" is a vague "discard old enough crufts to gain the diskspace back" (as opposed to "I know this particular entry is old enough and I want to see it gone right now"). Compared to that, with "git rerere forget", the end-user explicitly says "I know the specific rerere entry i just saw reused is *wrong* and I want to get rid of it". If another process holding the lock prevents it from being carried out, I'd prefer to see it fail loudly and let me know that the entry I wanted to remove is still there (so if I retried the same merge, I'll see the same mistaken resolution). >> + if (fd < 0) { >> + warning_errno(_("skipping rerere, unable to create '%s.lock'"), >> + git_path_merge_rr(r)); > > A background job that the user did not explicitly start printing to the > terminal is rather confusing as it is likely to get mixed in with the > output of whatever is running in the foreground. Very good point. Thanks. ``` ## Thomas Bachem, 2026-09-04 18:17 Subject: Re: [PATCH v2] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Junio, On 04/09/2026 19:06, Junio C Hamano wrote: > Phillip Wood writes: > >> Overall, this commit message is rather long and it would be helpful if >> you could distill it to remove unnecessary and unrelated details. > > Hear hear. The v3 log message is down to 23 lines from 81: > When there is somebody holding the lock, they acquired the lock > exactly because they did not want to see others (including > ourselves) to touch the rerere database until they are done. That caught one more case I got wrong in v3. The rerere_clear() that --abort and --skip run also waits and then goes on, and that leaves MERGE_RR behind. The next rerere run then takes each path in it as resolved by the user and records whatever the reset left there. The clear is the first thing --abort and --skip do, so I'll let it fail like "git rerere clear" does, from every caller. That also drops the flag from rerere_clear() and rerere_forget() again and leaves am.c and rebase.c untouched. > What makes `git rerere gc` different among all is not that it does > not retry. It just does not insist doing a GC and instead leaves > without doing anything (and without failing). Right, and I'll say it that way in the config text. All of them wait for the lock except the gc, which loses nothing by giving up at once. When the time is up, "git rerere", "git rerere forget" and "git rerere clear" fail, and a merge or commit that would record or reuse a resolution on the way warns and goes on without it. >> A background job that the user did not explicitly start printing to the >> terminal is rather confusing as it is likely to get mixed in with the >> output of whatever is running in the foreground. > > Very good point. I don't think it can happen, though. The detached run has no terminal: daemonize() reopens the standard descriptors on /dev/null before the gc runs. Where it doesn't detach, on Windows or with autoDetach off, the command that started it waits for it, so it isn't in the background either. The warning a user sees comes from the command in the foreground, once it has given up waiting. I'll wait for the rest of the v3 comments before rerolling. Thanks, Thomas ``` ## Junio C Hamano, 2026-09-04 19:08 Subject: Re: [PATCH v3] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` "Thomas Bachem via GitGitGadget" writes: > semantics of core.packedRefsTimeout, then warn and go on without > rerere: a lost recording or replay is nothing next to a rebase that > cannot continue. I do not understand the logic at the latter half of the above sentence. During a rebase, any and all opportunity to reuse a conflict resolution you made earlier is preferrable. I would be fine if "we cannot grab the lock so let's skip without doing 'rerere gc' at all". But if a conflicted step in rebase that stops and leaves conflicts in the working tree fails to record the preimage of a conflicted path, and makes the user realize that was what happened only after the user spends significant amount of work to resolve the conflicts and the resolution is not added to the rerere database, that is a huge loss. Perhaps it is just the way the above three lines is stated and what the code actually does may not be problematic, but I am not sure if that is what the latter half of the above sentence is trying to say. Thanks. ``` ## Thomas Bachem, 2026-09-05 05:41 Subject: Re: [PATCH v3] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Junio, On 04/09/2026 21:08, Junio C Hamano wrote: >> semantics of core.packedRefsTimeout, then warn and go on without >> rerere: a lost recording or replay is nothing next to a rebase that >> cannot continue. > > Perhaps it is just the way the above three lines is stated and what > the code actually does may not be problematic, but I am not sure if > that is what the latter half of the above sentence is trying to say. No, it's what the code does. Once the timeout is up, the conflicted step goes on without recording the preimage, and the resolution the user makes after that is lost, as you say. The rebase that cannot continue is the one from the message's first paragraph. It dies inside rerere, which do_pick_commit() runs before error_with_patch() writes the state "git rebase --continue" needs, so I traded the recording for a rebase that survives. You've convinced me that's the wrong trade. So in v4 every caller waits rerere.lockTimeout and then fails as it does today, and only "git rerere gc" gives up at once. That drops the RERERE_LOCK_OR_DIE flag and the hunks in the callers, and it takes back what I said in my reply to your other mail about merge and commit going on without rerere. A gc that outlasts the timeout still stops the rebase where it does today. With the gc giving way whenever it comes second and the sequencer series keeping a rebase's own commits from starting one, that should be rare. Whoever would rather wait it out can set rerere.lockTimeout to -1, but I'd keep the default finite so a lock left behind by a crash fails like every other lock instead of hanging. Writing the stop state before rerere runs would let such a rebase continue, which I can look at separately. Thanks, Thomas ``` ## Junio C Hamano, 2026-09-05 16:10 Subject: Re: [PATCH v3] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Thomas Bachem writes: >> Perhaps it is just the way the above three lines is stated and what >> the code actually does may not be problematic, but I am not sure if >> that is what the latter half of the above sentence is trying to say. > > No, it's what the code does. Once the timeout is up, the conflicted > step goes on without recording the preimage, and the resolution the > user makes after that is lost, as you say. It is a hard-to-accept regression without a good justification, though, especially with the other efforts to tame "rerere gc" from hogging the lock too often going on. If you have a 100-commit "rebase" that is interrupted in the middle, say at commit #70, at worst you should be able to hard reset and abort it, and then restart it starting on top of the result of applying up to commit #69 (with "rebase --onto") to finish the rest, so failing in the middle is not like throwing the effort you made so far away. > A gc that outlasts the timeout still stops the rebase where it does > today. With the gc giving way whenever it comes second and the > sequencer series keeping a rebase's own commits from starting one, > that should be rare. Whoever would rather wait it out can set > rerere.lockTimeout to -1, but I'd keep the default finite so a lock > left behind by a crash fails like every other lock instead of > hanging. Writing the stop state before rerere runs would let such a > rebase continue, which I can look at separately. Stepping back a bit, what does a "conflicted step goes on without recording the preimage" exactly look like? "git rebase" goes on chugging, and hits a commit that does not cleanly apply. It leaves a conflict and in a normal case immediately before returning the control back to the user, its "git rerere" invocation creates a preimage. Even if we make "git rerere" fail to do so, it would not be unrecoverable. The end user has control at that point, and it is not like the rest of rebase goes on without giving a chance to the user to intervene and recover. Would it make sense to LOUDLY tell the user when "git rerere" fails to do what the user expects to do? The output at the point of time on the terminal would end with something like CONFLICT (content): Merge conflict in t/t0123-frotz.sh Auto-merging nitfol.c error: could not apply 8c7b68a8bf... nitfol: remove frotz Recorded preimage for 't/t0123-frotz.sh' Could not apply 8c7b68a8bf... # nitfol: remove frotz if "git rerere" kicked in correctly, so if we said CONFLICT (content): Merge conflict in t/t0123-frotz.sh Auto-merging nitfol.c error: could not apply 8c7b68a8bf... nitfol: remove frotz FAILED TO RECORD PREIMAGE FOR 't/t0123-frotz.sh' Could not apply 8c7b68a8bf... # nitfol: remove frotz *** RUN "git rerere" MANUALLY BEFORE DOING ANYTHING ELSE *** *** IF YOU DO NOT WANT TO MAKE YOUR EFFORT IN RESOLVING *** *** THIS CONFLICT WASTED *** or something similar, would that help the user? I do not expect the "silent failure" to run "rerere" would not be followed by automated applications of many subsequent commits that makes it too late when the user notices what happened. An attempt to invoke "rerere" will always be followed by a stopped automation and the user will have the control at that point. So in that sense, as long as the user is told clearly that some step that usually happens and the user has learned to rely on did *not* happen, and also told how to recover from the failure, it is not too bad. Thanks. ``` ## Thomas Bachem, 2026-09-06 10:29 Subject: Re: [PATCH v3] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` Hi Junio, On 05/09/2026 18:10, Junio C Hamano wrote: > Would it make sense to LOUDLY tell the user when "git rerere" fails > to do what the user expects to do? It would, and the user gets everything back. At the stop the conflict is still in the index, so "git rerere" run by hand right there records the preimage that was skipped, "git rebase --continue" then records the resolution, and replaying the same conflict resolves it from the cache. A resolution it would have replayed comes back the same way, so one command covers whatever was skipped. I checked all of it. Your "BEFORE DOING ANYTHING ELSE" has to be in there, though. Resolve the file first and the conflict markers go with it, and "git rerere" records nothing at all. Failing doesn't record anything either, and the conflict is still sitting there to record by hand afterwards, so what it buys over a warning is that the user can't miss it. I agreed to that trade without checking what it costs. > If you have a 100-commit "rebase" that is interrupted in the middle, > say at commit #70, at worst you should be able to hard reset and > abort it, and then restart it starting on top of the result of > applying up to commit #69 (with "rebase --onto") to finish the rest, > so failing in the middle is not like throwing the effort you made so > far away. It's rougher than that. The "you have staged changes" refusal from the log message offers "git commit --amend" or "git commit", and neither gets commit #70 back as it was. The first folds its changes into #69 and drops its message, and the rebase then runs to the end, one commit short. The second keeps the message but not the author. Aborting does work, but the user first has to ignore what git just told them to do. That's 2.55 today, not something this patch adds, and waiting only makes it rarer. Writing the stop state before rerere runs would let the rebase continue, but the recording is still gone, so the message is needed there too. I'd rather not stop a rebase the user can otherwise finish over a recording we can tell them how to get back, so I'd keep v3's behavior at a conflict, where the command stops anyway and the warning can say to run "git rerere" before resolving. The recording after the resolution is different: "git am --continue" and a rebase's "git commit" go on with the rest, and a leftover MERGE_RR entry then records whatever the file holds at the next rerere run, as with the clears. So those keep failing after the wait. "git rerere", "forget" and "clear" fail after it too, so do the clears that "am" and "rebase" run, and the gc gives up at once. Say if you'd still rather all of it failed and I'll build that instead. Thanks, Thomas ``` ## Patrick Steinhardt, 2026-09-07 07:41 Subject: Re: [PATCH v3] rerere: keep a background gc from killing a rebase Message-ID: In-Reply-To: ``` On Fri, Sep 04, 2026 at 03:51:21PM +0000, Thomas Bachem via GitGitGadget wrote: > From: Thomas Bachem > > A "git rerere gc" holds MERGE_RR.lock for as long as pruning rr-cache > takes, and since 2.54 the auto maintenance after every commit runs > one whenever rr-cache has an entry. The commit a rebase spawns for a > resolved pick starts it too, and the sequencer's repo_rerere() at the > next conflict wants the lock a few milliseconds later. Both take it > with LOCK_DIE_ON_ERROR, so whichever comes second dies. When it is > the rebase, the index is written but the state for "git rebase > --continue" is not, and every later continue refuses with "you have > staged changes". Haven't we said that this race is not exclusive to `git rerere gc` with a concurrent writer though? It also happens between two normal writers. So it's good to have the context that we discovered this race because of the changed heuristics in maintenance, but we should clarify that it's a longer-standing conceptual issue. > diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc > index 3a78b5ebb1..14ef193545 100644 > --- a/Documentation/config/rerere.adoc > +++ b/Documentation/config/rerere.adoc > @@ -10,3 +10,13 @@ rerere.enabled:: > enabled if there is an `rr-cache` directory under the > `$GIT_DIR`, e.g. if "rerere" was previously used in the > repository. > + > +rerere.lockTimeout:: > + The length of time, in milliseconds, to retry when trying to > + take the rerere lock while another process holds it, typically > + a background `git rerere gc`. When the time is up, the command > + warns and goes on without rerere. Value 0 means not to retry > + at all; -1 means to try indefinitely. Default is 1000 (i.e., > + retry for 1 second). `git rerere gc` does not retry at all. > + `git rerere`, `git rerere forget` and `git rerere clear` retry > + the same way, but fail when the time is up instead of going on. I'm not a 100% sold that it's sensible to just skip writing the rerere entry. But maybe it's more sensible to regress gracefully compared to just aborting the whole command? In any case, I feel like this change warrants its own preparatory commit so that we can discuss separately why it's a good idea to ignore those failures. Patrick ``` ## Phillip Wood, 2026-09-07 10:07 Subject: Re: [PATCH v2] rerere: keep a background gc from killing a rebase Message-ID: <595d0d45-7000-4c52-8430-f18ce8f99c71@gmail.com> In-Reply-To: ``` Hi Thomas On 04/09/2026 16:55, Thomas Bachem wrote: > On 04/09/2026 16:21, Phillip Wood wrote: >> With Patricks patches that's no-longer true I think. I think a better >> motivation, as the cache is per-repository, rather than per-worktree, is >> concurrent writers running in different worktrees. > > MERGE_RR is per worktree, though, and so is its lock: > > $ git -C linked rev-parse --git-path MERGE_RR > /path/to/main/.git/worktrees/linked/MERGE_RR > > so writers in different worktrees never meet on it. What they share is > rr-cache, which a gc in one worktree prunes under its own worktree's > lock only. That is a gap of its own, and not one this patch closes. Oh, I didn't realize the lock was per-worktree. So the lock "rerere gc" takes does not actually stop another process running in a different worktree from altering the rerere cache. > What remains after Patrick's series is any "git rerere gc" that runs > while a command records a conflict, from "git gc", from a maintenance > run, or from auto maintenance once enough entries are stale. The v3 > message says it that way. > >> Overall, this commit message is rather long and it would be helpful if >> you could distill it to remove unnecessary and unrelated details. > > Done, it is a quarter of the size now. > >> Why do those commands fail rather than wait? > > They wait like everything else, and once the time is up they fail > instead of going on without rerere, which is all they are for. That > way a stale lock gets the usual advice to remove it. The config text > said otherwise, fixed. That's good, I think I'd maybe misunderstood what the original patch was trying to say. >> It might be worth adding a check above here that BUG()s out if the >> caller passes an incompatible set of flags. > > Added, for RERERE_NOWAIT with RERERE_LOCK_OR_DIE and for > RERERE_READONLY with either. > >> A background job that the user did not explicitly start printing to the >> terminal is rather confusing as it is likely to get mixed in with the >> output of whatever is running in the foreground. > > The detached maintenance run has no terminal: daemonize() closes the > standard descriptors and reopens them on /dev/null, so the gc's > warning goes nowhere when it loses the lock. Where it cannot detach, > on Windows, it runs in the foreground of the commit that started it > and there is no race to lose. The warning the user does see is the > foreground command's own, when it gives up waiting. Thanks for clarifying that Phillip > > Thanks, > Thomas ``` ## Thomas Bachem via GitGitGadget, 2026-09-14 08:04 Subject: [PATCH v4 0/2] rerere: wait for MERGE_RR.lock, and go on at a conflict Message-ID: In-Reply-To: ``` This is now two patches, as Patrick asked: the wait first, the skip on top, so that the second can be discussed on its own. Patch 1 makes every writer wait rerere.lockTimeout for MERGE_RR.lock before it fails as it does today, and lets "git rerere gc" skip its run instead of waiting. That alone keeps a rebase alive through the gc that a commit's auto maintenance starts: the gc gives way whenever it comes second, and the one second default is longer than pruning a few thousand entries takes. Patch 2 is what I proposed in my 6 September reply to Junio: a command that stops at a conflict warns and goes on when the lock is still held after the wait, and the warning tells the user to run "git rerere" before resolving. Everything else keeps failing: the recordings after a resolution, the user's own rerere commands, and the clears that am and rebase run. Changes since v3: * Based on master instead of maint. On master setup_rerere() already uses the repository-scoped lock helpers, so the series merges cleanly into next and seen, including ps/tune-rerere-gc (which makes the gc rarer, not the race) and tb/rerere-lock-grace (which only stops the commands a rebase spawns from starting the gc). * The first log message says that the race is as old as the lock and that the 2.54 maintenance default only made it easy to hit (Patrick). * Only the six callers that stop at a conflict go on without rerere: the sequencer's pick and merge, "git merge", the 3-way fallback of "git am", "git stash" and "git apply --3way". "git commit" and "git am --continue" fail after the wait, as in 2.55 (Junio). Merge, rebase, am, stash and apply each have a test for this. * The config entry describes the outcome per command and no longer says "retry" (Junio). * The "rerere clear" that am and rebase run for --skip and --abort waits and then fails like every other caller, so a held lock cannot leave a stale MERGE_RR entry behind. The signatures of rerere_clear() and rerere_forget() are back to what upstream has. * A new test runs "git rerere" at the stop after a skipped recording and checks that the preimage, and after the resolution the postimage, get recorded. Thomas Bachem (2): rerere: wait for MERGE_RR.lock, and let the gc skip it rerere: go on at a conflict when the lock stays busy Documentation/config/rerere.adoc | 13 +++ Documentation/git-rerere.adoc | 4 +- apply.c | 2 +- builtin/am.c | 3 +- builtin/merge.c | 2 +- builtin/stash.c | 2 +- rerere.c | 49 +++++++-- rerere.h | 4 + sequencer.c | 4 +- t/t4200-rerere.sh | 168 +++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 8 ++ 11 files changed, 246 insertions(+), 13 deletions(-) base-commit: 47ce80527c56f462cb97db4ca8125342204d3783 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v4 Pull-Request: https://github.com/gitgitgadget/git/pull/2214 Range-diff vs v3: 1: 5bfda65baa ! 1: 8a7a74d6aa rerere: keep a background gc from killing a rebase @@ Metadata Author: Thomas Bachem ## Commit message ## - rerere: keep a background gc from killing a rebase + rerere: wait for MERGE_RR.lock, and let the gc skip it - A "git rerere gc" holds MERGE_RR.lock for as long as pruning rr-cache - takes, and since 2.54 the auto maintenance after every commit runs - one whenever rr-cache has an entry. The commit a rebase spawns for a - resolved pick starts it too, and the sequencer's repo_rerere() at the - next conflict wants the lock a few milliseconds later. Both take it - with LOCK_DIE_ON_ERROR, so whichever comes second dies. When it is - the rebase, the index is written but the state for "git rebase - --continue" is not, and every later continue refuses with "you have - staged changes". + setup_rerere() takes MERGE_RR.lock with LOCK_DIE_ON_ERROR. When two + processes want the lock at the same time, the second one dies. This + was always the case, but since 452b12c2e0 (builtin/maintenance: use + "geometric" strategy by default, 2026-02-24) it is easy to hit: auto + maintenance now runs "git rerere gc" after every commit whenever + rr-cache contains at least one entry, and the gc holds the lock + while it prunes. - The gc needs the lock, since a rerere that has just created its - directory looks like the empty ones it prunes. So wait for it - instead, rerere.lockTimeout milliseconds, 1000 by default with the - semantics of core.packedRefsTimeout, then warn and go on without - rerere: a lost recording or replay is nothing next to a rebase that - cannot continue. The gc itself never waits, and "git rerere", "git - rerere forget" and "git rerere clear" wait but then die, since the - state behind the lock is all they are for. The clearing "am" and - "rebase" do on --abort, --skip and --quit goes on without it, and - leaves the entries for the gc. + A rebase whose next pick conflicts while the gc holds the lock dies + inside repo_rerere(). That runs before the sequencer writes the state + that "git rebase --continue" needs, so every later "git rebase + --continue" fails with "you have staged changes". + + Instead of dying right away, wait for the lock for up to + rerere.lockTimeout milliseconds, 1000 by default, and only then fail + as before. The gc itself does not wait: when the lock is held, it + skips this run and leaves the pruning to the next one. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem @@ Documentation/config/rerere.adoc: rerere.enabled:: repository. + +rerere.lockTimeout:: -+ The length of time, in milliseconds, to retry when trying to -+ take the rerere lock while another process holds it, typically -+ a background `git rerere gc`. When the time is up, the command -+ warns and goes on without rerere. Value 0 means not to retry -+ at all; -1 means to try indefinitely. Default is 1000 (i.e., -+ retry for 1 second). `git rerere gc` does not retry at all. -+ `git rerere`, `git rerere forget` and `git rerere clear` retry -+ the same way, but fail when the time is up instead of going on. ++ The length of time, in milliseconds, to wait for the rerere ++ lock when another process holds it, typically a background ++ `git rerere gc`. Value 0 means not to wait at all; -1 means ++ to wait indefinitely. Default is 1000 (i.e., wait for 1 ++ second). When the time is up, the command fails as it does ++ for any other lock it cannot take. `git rerere gc` never ++ waits and skips its run while the lock is held. ## Documentation/git-rerere.adoc ## @@ Documentation/git-rerere.adoc: occurred a long time ago. By default, unresolved conflicts older @@ Documentation/git-rerere.adoc: occurred a long time ago. By default, unresolved days are pruned. These defaults are controlled via the `gc.rerereUnresolved` and `gc.rerereResolved` configuration -variables respectively. -+variables respectively. If another process holds the lock on the -+recorded resolutions, for example a merge or rebase that is recording -+a conflict, `gc` does nothing and reports so. ++variables respectively. If another process holds the rerere lock, ++for example a merge or rebase that is recording a conflict, `gc` ++does nothing and says so. DISCUSSION - ## builtin/am.c ## -@@ builtin/am.c: static int clean_index(const struct object_id *head, const struct object_id *rem - static void am_rerere_clear(void) - { - struct string_list merge_rr = STRING_LIST_INIT_DUP; -- rerere_clear(the_repository, &merge_rr); -+ rerere_clear(the_repository, &merge_rr, 0); - string_list_clear(&merge_rr, 1); - } - - - ## builtin/rebase.c ## -@@ builtin/rebase.c: static int run_sequencer_rebase(struct rebase_options *opts) - case ACTION_SKIP: { - struct string_list merge_rr = STRING_LIST_INIT_DUP; - -- rerere_clear(the_repository, &merge_rr); -+ rerere_clear(the_repository, &merge_rr, 0); - } - /* fallthrough */ - case ACTION_CONTINUE: { -@@ builtin/rebase.c: int cmd_rebase(int argc, - case ACTION_SKIP: { - struct string_list merge_rr = STRING_LIST_INIT_DUP; - -- rerere_clear(the_repository, &merge_rr); -+ rerere_clear(the_repository, &merge_rr, 0); - string_list_clear(&merge_rr, 1); - ropts.flags = RESET_HEAD_HARD; - if (reset_head(the_repository, &ropts) < 0) -@@ builtin/rebase.c: int cmd_rebase(int argc, - struct string_list merge_rr = STRING_LIST_INIT_DUP; - struct strbuf head_msg = STRBUF_INIT; - -- rerere_clear(the_repository, &merge_rr); -+ rerere_clear(the_repository, &merge_rr, 0); - string_list_clear(&merge_rr, 1); - - if (read_basic_state(&options)) - - ## builtin/rerere.c ## -@@ builtin/rerere.c: int cmd_rerere(int argc, - flags = RERERE_NOAUTOUPDATE; - - if (argc < 1) -- return repo_rerere(the_repository, flags); -+ return repo_rerere(the_repository, flags | RERERE_LOCK_OR_DIE); - - if (!strcmp(argv[0], "forget")) { - struct pathspec pathspec; -@@ builtin/rerere.c: int cmd_rerere(int argc, - parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD, - prefix, argv + 1); - -- ret = rerere_forget(the_repository, &pathspec); -+ ret = rerere_forget(the_repository, &pathspec, -+ RERERE_LOCK_OR_DIE); - - clear_pathspec(&pathspec); - return ret; - } - - if (!strcmp(argv[0], "clear")) { -- rerere_clear(the_repository, &merge_rr); -+ rerere_clear(the_repository, &merge_rr, RERERE_LOCK_OR_DIE); - } else if (!strcmp(argv[0], "gc")) - rerere_gc(the_repository, &merge_rr); - else if (!strcmp(argv[0], "status")) { - ## rerere.c ## @@ rerere.c: static int rerere_enabled = -1; - /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; -+static int rerere_lock_timeout_ms = 1000; ++/* how long to wait for MERGE_RR.lock, in milliseconds */ ++static int rerere_lock_timeout_ms = 1000; ++ #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 + struct rerere_dir { @@ rerere.c: static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) -+ if ((flags & RERERE_NOWAIT) && (flags & RERERE_LOCK_OR_DIE)) -+ BUG("RERERE_NOWAIT and RERERE_LOCK_OR_DIE are mutually exclusive"); -+ if ((flags & RERERE_READONLY) && -+ (flags & (RERERE_NOWAIT | RERERE_LOCK_OR_DIE))) -+ BUG("RERERE_READONLY takes no lock, so no lock flag applies"); ++ if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) ++ BUG("RERERE_READONLY takes no lock, so RERERE_NOWAIT does not apply"); + if (flags & RERERE_READONLY) { fd = 0; - else -- fd = hold_lock_file_for_update(&write_lock, -- git_path_merge_rr(r), -- LOCK_DIE_ON_ERROR); +- fd = repo_hold_lock_file_for_update(r, &write_lock, +- git_path_merge_rr(r), +- LOCK_DIE_ON_ERROR); + } else { -+ int lock_flags = 0; ++ const char *path = git_path_merge_rr(r); ++ int lock_flags = LOCK_DIE_ON_ERROR; + long timeout_ms = rerere_lock_timeout_ms; + -+ if (flags & RERERE_LOCK_OR_DIE) -+ lock_flags = LOCK_DIE_ON_ERROR; -+ if (flags & RERERE_NOWAIT) -+ timeout_ms = 0; + /* -+ * A background "rerere gc" holds the lock for as long as it -+ * takes to prune rr-cache, so wait it out rather than fail -+ * at once. The gc itself has nothing to lose from a skipped -+ * run and never waits. ++ * Another process may hold the lock for a while, e.g. ++ * "git rerere gc" while it prunes rr-cache, so wait for ++ * it instead of dying right away. The gc itself never ++ * waits: skipping one of its runs costs nothing. + */ -+ fd = hold_lock_file_for_update_timeout(&write_lock, -+ git_path_merge_rr(r), -+ lock_flags, timeout_ms); ++ if (flags & RERERE_NOWAIT) { ++ lock_flags = 0; ++ timeout_ms = 0; ++ } ++ fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, ++ path, lock_flags, ++ timeout_ms); + if (fd < 0) { -+ warning_errno(_("skipping rerere, unable to create '%s.lock'"), -+ git_path_merge_rr(r)); ++ warning_errno(_("skipping rerere, " ++ "unable to create '%s.lock'"), path); + return -1; + } + } read_rr(r, merge_rr); return fd; } -@@ rerere.c: fail_exit: - return -1; - } - --int rerere_forget(struct repository *r, struct pathspec *pathspec) -+int rerere_forget(struct repository *r, struct pathspec *pathspec, int flags) - { - int i, fd, ret; - struct string_list conflict = STRING_LIST_INIT_DUP; -@@ rerere.c: int rerere_forget(struct repository *r, struct pathspec *pathspec) - if (repo_read_index(r) < 0) - return error(_("index file corrupt")); - -- fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE); -+ fd = setup_rerere(r, &merge_rr, RERERE_NOAUTOUPDATE | flags); - if (fd < 0) - return 0; - @@ rerere.c: void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve = now - 60 * 86400; struct strbuf buf = STRBUF_INIT; @@ rerere.c: void rerere_gc(struct repository *r, struct string_list *rr) return; repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", -@@ rerere.c: void rerere_gc(struct repository *r, struct string_list *rr) - * - * NEEDSWORK: shouldn't we be calling this from "reset --hard"? - */ --void rerere_clear(struct repository *r, struct string_list *merge_rr) -+void rerere_clear(struct repository *r, struct string_list *merge_rr, int flags) - { - int i; - -- if (setup_rerere(r, merge_rr, 0) < 0) -+ if (setup_rerere(r, merge_rr, flags) < 0) - return; - - for (i = 0; i < merge_rr->nr; i++) { ## rerere.h ## @@ rerere.h: struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 -+/* Do not wait for the lock when another process holds it */ ++/* Never wait for MERGE_RR.lock, and skip the run when it is held */ +#define RERERE_NOWAIT 010 -+/* Die on a lock that cannot be taken instead of going on without rerere */ -+#define RERERE_LOCK_OR_DIE 020 /* * Marks paths that have been hand-resolved and added to the -@@ rerere.h: int repo_rerere(struct repository *, int); - */ - const char *rerere_path(struct strbuf *buf, const struct rerere_id *, - const char *file); --int rerere_forget(struct repository *, struct pathspec *); -+int rerere_forget(struct repository *, struct pathspec *, int); - int rerere_remaining(struct repository *, struct string_list *); --void rerere_clear(struct repository *, struct string_list *); -+void rerere_clear(struct repository *, struct string_list *, int); - void rerere_gc(struct repository *, struct string_list *); - - #define OPT_RERERE_AUTOUPDATE(v) OPT_UYN(0, "rerere-autoupdate", (v), \ ## t/t4200-rerere.sh ## @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' + test_grep "^=======\$" $rr/preimage +' + -+test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' ++test_expect_success 'merge fails once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && -+ test_grep "skipping rerere" err && ++ test_grep "Unable to create" err && + test_grep "^=======\$" a1 && + test_path_is_missing $rr/preimage +' + -+test_expect_success 'commit goes on without rerere once rerere.lockTimeout is up' ' -+ git reset --hard && -+ rm -rf $rr && -+ git checkout -b lock-held-commit third && -+ test_when_finished "git checkout third && git branch -D lock-held-commit" && -+ test_must_fail git merge first && -+ test_path_is_file $rr/preimage && -+ test_when_finished "rm -f .git/MERGE_RR.lock" && -+ >.git/MERGE_RR.lock && -+ echo resolved >a1 && -+ git add a1 && -+ git -c rerere.lockTimeout=0 commit -qm resolved 2>err && -+ test_grep "skipping rerere" err && -+ test_path_is_missing $rr/postimage -+' -+ +test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' + test_grep "Unable to create" err +' + -+test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' ++test_expect_success 'rebase --abort fails on a lock it cannot take' ' + git reset --hard && -+ rm -rf $rr && -+ git checkout -b lock-held third && -+ test_when_finished "git checkout third && git branch -D lock-held" && -+ test_when_finished "rm -f .git/MERGE_RR.lock" && -+ >.git/MERGE_RR.lock && -+ test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && -+ test_grep "skipping rerere" err && -+ test_path_is_file .git/rebase-merge/stopped-sha && -+ echo resolved >a1 && -+ git add a1 && -+ git -c rerere.lockTimeout=0 rebase --continue && -+ test_path_is_missing .git/rebase-merge && -+ test_path_is_missing $rr/preimage -+' -+ -+test_expect_success 'rebase --abort goes on without rerere on a held lock' ' + git checkout -b lock-held-abort third && + test_when_finished "git checkout third && git branch -D lock-held-abort" && + test_must_fail git rebase first && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && -+ git -c rerere.lockTimeout=0 rebase --abort 2>err && -+ test_grep "skipping rerere" err && ++ test_must_fail git -c rerere.lockTimeout=0 rebase --abort 2>err && ++ test_grep "Unable to create" err && ++ test_path_is_dir .git/rebase-merge && ++ rm .git/MERGE_RR.lock && ++ git rebase --abort && + test_path_is_missing .git/rebase-merge +' + -: ---------- > 2: 1cce403113 rerere: go on at a conflict when the lock stays busy -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-09-14 08:04 Subject: [PATCH v4 1/2] rerere: wait for MERGE_RR.lock, and let the gc skip it Message-ID: <8a7a74d6aa359844a49593538ef6178cd1b02031.1789373061.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem setup_rerere() takes MERGE_RR.lock with LOCK_DIE_ON_ERROR. When two processes want the lock at the same time, the second one dies. This was always the case, but since 452b12c2e0 (builtin/maintenance: use "geometric" strategy by default, 2026-02-24) it is easy to hit: auto maintenance now runs "git rerere gc" after every commit whenever rr-cache contains at least one entry, and the gc holds the lock while it prunes. A rebase whose next pick conflicts while the gc holds the lock dies inside repo_rerere(). That runs before the sequencer writes the state that "git rebase --continue" needs, so every later "git rebase --continue" fails with "you have staged changes". Instead of dying right away, wait for the lock for up to rerere.lockTimeout milliseconds, 1000 by default, and only then fail as before. The gc itself does not wait: when the lock is held, it skips this run and leaves the pruning to the next one. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 9 +++++ Documentation/git-rerere.adoc | 4 +- rerere.c | 39 ++++++++++++++++--- rerere.h | 2 + t/t4200-rerere.sh | 67 ++++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 8 ++++ 6 files changed, 122 insertions(+), 7 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 3a78b5ebb1..cc9dd0c37b 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -10,3 +10,12 @@ rerere.enabled:: enabled if there is an `rr-cache` directory under the `$GIT_DIR`, e.g. if "rerere" was previously used in the repository. + +rerere.lockTimeout:: + The length of time, in milliseconds, to wait for the rerere + lock when another process holds it, typically a background + `git rerere gc`. Value 0 means not to wait at all; -1 means + to wait indefinitely. Default is 1000 (i.e., wait for 1 + second). When the time is up, the command fails as it does + for any other lock it cannot take. `git rerere gc` never + waits and skips its run while the lock is held. diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc index 4e6ab9a27c..4df653367e 100644 --- a/Documentation/git-rerere.adoc +++ b/Documentation/git-rerere.adoc @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older than 15 days and resolved conflicts older than 60 days are pruned. These defaults are controlled via the `gc.rerereUnresolved` and `gc.rerereResolved` configuration -variables respectively. +variables respectively. If another process holds the rerere lock, +for example a merge or rebase that is recording a conflict, `gc` +does nothing and says so. DISCUSSION diff --git a/rerere.c b/rerere.c index 3d3bd0db16..7d44f3937c 100644 --- a/rerere.c +++ b/rerere.c @@ -33,6 +33,9 @@ static int rerere_enabled = -1; /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; +/* how long to wait for MERGE_RR.lock, in milliseconds */ +static int rerere_lock_timeout_ms = 1000; + #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 struct rerere_dir { @@ -850,6 +853,8 @@ static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); + repo_config_get_int(the_repository, "rerere.locktimeout", + &rerere_lock_timeout_ms); repo_config(the_repository, git_default_config, NULL); } @@ -882,12 +887,34 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) + if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) + BUG("RERERE_READONLY takes no lock, so RERERE_NOWAIT does not apply"); + if (flags & RERERE_READONLY) { fd = 0; - else - fd = repo_hold_lock_file_for_update(r, &write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + } else { + const char *path = git_path_merge_rr(r); + int lock_flags = LOCK_DIE_ON_ERROR; + long timeout_ms = rerere_lock_timeout_ms; + + /* + * Another process may hold the lock for a while, e.g. + * "git rerere gc" while it prunes rr-cache, so wait for + * it instead of dying right away. The gc itself never + * waits: skipping one of its runs costs nothing. + */ + if (flags & RERERE_NOWAIT) { + lock_flags = 0; + timeout_ms = 0; + } + fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, + path, lock_flags, + timeout_ms); + if (fd < 0) { + warning_errno(_("skipping rerere, " + "unable to create '%s.lock'"), path); + return -1; + } + } read_rr(r, merge_rr); return fd; } @@ -1211,7 +1238,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve = now - 60 * 86400; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) + if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) return; repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", diff --git a/rerere.h b/rerere.h index d4b5f7c932..a2712d543e 100644 --- a/rerere.h +++ b/rerere.h @@ -10,6 +10,8 @@ struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 +/* Never wait for MERGE_RR.lock, and skip the run when it is held */ +#define RERERE_NOWAIT 010 /* * Marks paths that have been hand-resolved and added to the diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 7bb601e117..27082a7676 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,73 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'gc does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc 2>err && + test_grep "MERGE_RR.lock" err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && + git rerere gc && + test_path_is_missing $rr2/preimage +' + +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { + ( sleep 1 && rm -f .git/MERGE_RR.lock ) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && + test_grep ! "MERGE_RR" err && + test_grep "^=======\$" $rr/preimage +' + +test_expect_success 'merge fails once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && + test_grep "Unable to create" err && + test_grep "^=======\$" a1 && + test_path_is_missing $rr/preimage +' + +test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && + test_grep "Unable to create" err +' + +test_expect_success 'rebase --abort fails on a lock it cannot take' ' + git reset --hard && + git checkout -b lock-held-abort third && + test_when_finished "git checkout third && git branch -D lock-held-abort" && + test_must_fail git rebase first && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase --abort 2>err && + test_grep "Unable to create" err && + test_path_is_dir .git/rebase-merge && + rm .git/MERGE_RR.lock && + git rebase --abort && + test_path_is_missing .git/rebase-merge +' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 5fbb16f0f0..4a27767817 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -1051,6 +1051,14 @@ test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc ' +test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && + mkdir .git/rr-cache && + : >.git/rr-cache/entry && + >.git/MERGE_RR.lock && + test_expect_rerere_gc git maintenance run --task=rerere-gc +' + test_expect_success '--auto and --schedule incompatible' ' test_must_fail git maintenance run --auto --schedule=daily 2>err && test_grep "cannot be used together" err -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-09-14 08:04 Subject: [PATCH v4 2/2] rerere: go on at a conflict when the lock stays busy Message-ID: <1cce403113833c14a1c4a0da0db0772c5abdeb1c.1789373061.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem When a merge, rebase, cherry-pick, revert, am, stash or apply stops at a conflict, it runs rerere right before it returns to the user. If MERGE_RR.lock is still held when rerere.lockTimeout runs out, the command dies there. For a rebase that is worse than a lost recording: the sequencer has not yet written the state that "git rebase --continue" needs, so the rebase cannot continue, and following the "git commit --amend" advice folds the conflicted pick into the previous commit. So print a warning and go on instead. The conflict is still in place, and the warning tells the user to run "git rerere" before resolving it. That records the preimage, or replays a known resolution, just as the command would have done. All other callers still fail when the timeout runs out. "git commit" and "git am --continue" record the resolution and then move on to the next commit or patch. A warning would come too late there, and the next rerere run would record whatever the file contains by then. The "rerere clear" that am and rebase run for --skip and --abort would leave the same stale entry behind. "git rerere", "git rerere forget" and "git rerere clear" fail because the user asked for that state explicitly. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 10 ++- apply.c | 2 +- builtin/am.c | 3 +- builtin/merge.c | 2 +- builtin/stash.c | 2 +- rerere.c | 16 ++++- rerere.h | 2 + sequencer.c | 4 +- t/t4200-rerere.sh | 105 ++++++++++++++++++++++++++++++- 9 files changed, 132 insertions(+), 14 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index cc9dd0c37b..81deefa006 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -16,6 +16,10 @@ rerere.lockTimeout:: lock when another process holds it, typically a background `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 - second). When the time is up, the command fails as it does - for any other lock it cannot take. `git rerere gc` never - waits and skips its run while the lock is held. + second). When the time is up, a command that stops at a + conflict, such as `git merge` or `git rebase`, prints a + warning and goes on without rerere; run `git rerere` before + resolving the conflict to record it after all. Any other + command fails, as it does for any other lock it cannot take. + `git rerere gc` never waits and skips its run while the lock + is held. diff --git a/apply.c b/apply.c index f00b7ba4d3..3b8502535b 100644 --- a/apply.c +++ b/apply.c @@ -4865,7 +4865,7 @@ static int write_out_results(struct apply_state *state, struct patch *list) * tree with conflict markers, but that isn't written with --cached. */ if (!state->cached) - repo_rerere(state->repo, 0); + repo_rerere(state->repo, RERERE_SKIP_LOCKED); } return errs; diff --git a/builtin/am.c b/builtin/am.c index e9623b8307..bcb93db515 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1649,7 +1649,8 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa o.verbosity = 0; if (merge_ort_generic(&o, &our_tree, &their_tree, 1, bases, &result)) { - repo_rerere(the_repository, state->allow_rerere_autoupdate); + repo_rerere(the_repository, state->allow_rerere_autoupdate | + RERERE_SKIP_LOCKED); free(their_tree_name); return error(_("Failed to merge in the changes.")); } diff --git a/builtin/merge.c b/builtin/merge.c index 5b4eb23a83..1ed5959bfd 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -1061,7 +1061,7 @@ static int suggest_conflicts(void) fputs(msgbuf.buf, fp); strbuf_release(&msgbuf); fclose(fp); - repo_rerere(the_repository, allow_rerere_auto); + repo_rerere(the_repository, allow_rerere_auto | RERERE_SKIP_LOCKED); printf(_("Automatic merge failed; " "fix conflicts and then commit the result.\n")); return 1; diff --git a/builtin/stash.c b/builtin/stash.c index 72c52571f8..83b6e1be72 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -729,7 +729,7 @@ static int do_apply_stash(const char *prefix, struct stash_info *info, ret = error(_("could not write index")); if (ret) { - repo_rerere(the_repository, 0); + repo_rerere(the_repository, RERERE_SKIP_LOCKED); if (index) fprintf_ln(stderr, _("Index was not unstashed.")); diff --git a/rerere.c b/rerere.c index 7d44f3937c..a996d39159 100644 --- a/rerere.c +++ b/rerere.c @@ -3,6 +3,7 @@ #include "git-compat-util.h" #include "abspath.h" +#include "advice.h" #include "config.h" #include "copy.h" #include "environment.h" @@ -887,8 +888,9 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) - BUG("RERERE_READONLY takes no lock, so RERERE_NOWAIT does not apply"); + if ((flags & RERERE_READONLY) && + (flags & (RERERE_NOWAIT | RERERE_SKIP_LOCKED))) + BUG("RERERE_READONLY takes no lock, so no lock flag applies"); if (flags & RERERE_READONLY) { fd = 0; } else { @@ -900,18 +902,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) * Another process may hold the lock for a while, e.g. * "git rerere gc" while it prunes rr-cache, so wait for * it instead of dying right away. The gc itself never - * waits: skipping one of its runs costs nothing. + * waits: skipping one of its runs costs nothing. A + * command that stops at a conflict must not die here + * either, so it warns and goes on without rerere. */ if (flags & RERERE_NOWAIT) { lock_flags = 0; timeout_ms = 0; } + if (flags & RERERE_SKIP_LOCKED) + lock_flags = 0; fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, path, lock_flags, timeout_ms); if (fd < 0) { warning_errno(_("skipping rerere, " "unable to create '%s.lock'"), path); + if (flags & RERERE_SKIP_LOCKED) + advise(_("run \"git rerere\" before resolving " + "the conflict to record or replay " + "its resolution")); return -1; } } diff --git a/rerere.h b/rerere.h index a2712d543e..e91f3f4afa 100644 --- a/rerere.h +++ b/rerere.h @@ -12,6 +12,8 @@ struct repository; #define RERERE_READONLY 04 /* Never wait for MERGE_RR.lock, and skip the run when it is held */ #define RERERE_NOWAIT 010 +/* Warn and go on without rerere if MERGE_RR.lock cannot be taken in time */ +#define RERERE_SKIP_LOCKED 020 /* * Marks paths that have been hand-resolved and added to the diff --git a/sequencer.c b/sequencer.c index 65afd100d9..49776e7c5a 100644 --- a/sequencer.c +++ b/sequencer.c @@ -2519,7 +2519,7 @@ static enum pick_result do_pick_commit(struct repository *r, : _("could not apply %s... %s"), short_commit_name(r, commit), msg.subject); print_advice(r, res == 1, opts); - repo_rerere(r, opts->allow_rerere_auto); + repo_rerere(r, opts->allow_rerere_auto | RERERE_SKIP_LOCKED); goto leave; } @@ -4448,7 +4448,7 @@ static int do_merge(struct repository *r, rollback_lock_file(&lock); if (ret) - repo_rerere(r, opts->allow_rerere_auto); + repo_rerere(r, opts->allow_rerere_auto | RERERE_SKIP_LOCKED); else /* * In case of problems, we now want to return a positive diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 27082a7676..d0a9363a1c 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -272,17 +272,118 @@ test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' test_grep "^=======\$" $rr/preimage ' -test_expect_success 'merge fails once rerere.lockTimeout is up' ' +test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' git reset --hard && rm -rf $rr && test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && - test_grep "Unable to create" err && + test_grep "skipping rerere" err && + test_grep "hint: .*git rerere" err && test_grep "^=======\$" a1 && test_path_is_missing $rr/preimage ' +test_expect_success 'rerere run at the stop records what was skipped' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-catch-up third && + test_when_finished "git checkout third && git branch -D lock-held-catch-up" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first && + test_path_is_missing $rr/preimage && + rm .git/MERGE_RR.lock && + git rerere && + test_grep "^=======\$" $rr/preimage && + echo resolved >a1 && + git add a1 && + git commit -qm resolved && + test_path_is_file $rr/postimage +' + +test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held third && + test_when_finished "git checkout third && git branch -D lock-held" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && + test_grep "skipping rerere" err && + test_path_is_file .git/rebase-merge/stopped-sha && + rm .git/MERGE_RR.lock && + echo resolved >a1 && + git add a1 && + git rebase --continue && + test_path_is_missing .git/rebase-merge && + test_path_is_missing $rr/preimage +' + +test_expect_success 'commit fails on a lock it cannot take' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-commit third && + test_when_finished "git checkout third && git branch -D lock-held-commit" && + test_must_fail git merge first && + test_path_is_file $rr/preimage && + echo resolved >a1 && + git add a1 && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 commit -qm resolved 2>err && + test_grep "Unable to create" err && + test_path_is_missing $rr/postimage +' + +test_expect_success 'am goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-am third && + test_when_finished "test_might_fail git am --abort && + git checkout third && git branch -D lock-held-am" && + git format-patch -1 --stdout first >first.patch && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 am --3way first.patch 2>err && + test_grep "skipping rerere" err && + test_path_is_dir .git/rebase-apply && + rm .git/MERGE_RR.lock && + git am --abort && + test_path_is_missing .git/rebase-apply +' + +test_expect_success 'stash pop goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-stash third && + test_when_finished "git reset --hard && git stash drop && + git checkout third && git branch -D lock-held-stash" && + echo stashed >>a1 && + git stash && + echo committed >>a1 && + git commit -qam committed && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 stash pop 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 +' + +test_expect_success 'apply --3way goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-apply third && + test_when_finished "git reset --hard && + git checkout third && git branch -D lock-held-apply" && + git format-patch -1 --stdout first >first.patch && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 apply --3way first.patch 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 +' + test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && -- gitgitgadget ``` ## Patrick Steinhardt, 2026-09-28 08:18 Subject: Re: [PATCH v4 1/2] rerere: wait for MERGE_RR.lock, and let the gc skip it Message-ID: In-Reply-To: <8a7a74d6aa359844a49593538ef6178cd1b02031.1789373061.git.gitgitgadget@gmail.com> ``` On Mon, Sep 14, 2026 at 08:04:20AM +0000, Thomas Bachem via GitGitGadget wrote: > diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc > index 4e6ab9a27c..4df653367e 100644 > --- a/Documentation/git-rerere.adoc > +++ b/Documentation/git-rerere.adoc > @@ -70,7 +70,9 @@ occurred a long time ago. By default, unresolved conflicts older > than 15 days and resolved conflicts older than 60 > days are pruned. These defaults are controlled via the > `gc.rerereUnresolved` and `gc.rerereResolved` configuration > -variables respectively. > +variables respectively. If another process holds the rerere lock, > +for example a merge or rebase that is recording a conflict, `gc` > +does nothing and says so. Do we maybe want to drop these examples? I don't feel like they add any value. > diff --git a/rerere.c b/rerere.c > index 3d3bd0db16..7d44f3937c 100644 > --- a/rerere.c > +++ b/rerere.c > @@ -33,6 +33,9 @@ static int rerere_enabled = -1; > /* automatically update cleanly resolved paths to the index */ > static int rerere_autoupdate; > > +/* how long to wait for MERGE_RR.lock, in milliseconds */ > +static int rerere_lock_timeout_ms = 1000; > + > #define RR_HAS_POSTIMAGE 1 > #define RR_HAS_PREIMAGE 2 > struct rerere_dir { > @@ -850,6 +853,8 @@ static void git_rerere_config(void) > { > repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); > repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); > + repo_config_get_int(the_repository, "rerere.locktimeout", > + &rerere_lock_timeout_ms); > repo_config(the_repository, git_default_config, NULL); > } > > @@ -882,12 +887,34 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) > > if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) > rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); > - if (flags & RERERE_READONLY) > + if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) > + BUG("RERERE_READONLY takes no lock, so RERERE_NOWAIT does not apply"); > + if (flags & RERERE_READONLY) { > fd = 0; > - else > - fd = repo_hold_lock_file_for_update(r, &write_lock, > - git_path_merge_rr(r), > - LOCK_DIE_ON_ERROR); > + } else { > + const char *path = git_path_merge_rr(r); > + int lock_flags = LOCK_DIE_ON_ERROR; > + long timeout_ms = rerere_lock_timeout_ms; Here you're using a `long` whereas `rerere_lock_timeout_ms` is an `int`. Of course we'd ideally use a `long` consistently as that's also what `repo_hold_lock_file_for_update_timeout()` accepts. But I guess the reason you didn't is that we don't have `repo_config_get_long()`. So I guess this is good enough for now. > @@ -1211,7 +1238,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) > timestamp_t cutoff_resolve = now - 60 * 86400; > struct strbuf buf = STRBUF_INIT; > > - if (setup_rerere(r, rr, 0) < 0) > + if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) > return; > > repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", I'm not a 100% sold on this change. There's two different scenarios under which we want to perform garbage collection: - As part of auto-maintenance, triggered by Git automatically. Here I'm fully aligned that it makes sense to just silently ignore the case where we couldn't acquire the lock, as auto-maintenance is done on a best-effort basis anyway. - As part of `git rerere gc`, which is invoked manually by the user. Here I'm less so, as the user has explicitly asked us to garbage collect. Sure, we print a warning now, but the exit code does not signal that we failed garbage collecting. So I'd argue that we should discern those two use cases. I think that in the second use case, we'd probably want to use a timeout and if we fail to acquire the lock, we should make `git rerere gc` fail with a non-zero exit code. Patrick ``` ## Patrick Steinhardt, 2026-09-28 08:18 Subject: Re: [PATCH v4 2/2] rerere: go on at a conflict when the lock stays busy Message-ID: In-Reply-To: <1cce403113833c14a1c4a0da0db0772c5abdeb1c.1789373061.git.gitgitgadget@gmail.com> ``` On Mon, Sep 14, 2026 at 08:04:21AM +0000, Thomas Bachem via GitGitGadget wrote: > diff --git a/rerere.c b/rerere.c > index 7d44f3937c..a996d39159 100644 > --- a/rerere.c > +++ b/rerere.c > @@ -900,18 +902,26 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) > * Another process may hold the lock for a while, e.g. > * "git rerere gc" while it prunes rr-cache, so wait for > * it instead of dying right away. The gc itself never > - * waits: skipping one of its runs costs nothing. > + * waits: skipping one of its runs costs nothing. A > + * command that stops at a conflict must not die here > + * either, so it warns and goes on without rerere. > */ > if (flags & RERERE_NOWAIT) { > lock_flags = 0; > timeout_ms = 0; > } > + if (flags & RERERE_SKIP_LOCKED) > + lock_flags = 0; > fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, > path, lock_flags, > timeout_ms); > if (fd < 0) { > warning_errno(_("skipping rerere, " > "unable to create '%s.lock'"), path); > + if (flags & RERERE_SKIP_LOCKED) > + advise(_("run \"git rerere\" before resolving " > + "the conflict to record or replay " > + "its resolution")); > return -1; > } > } Should this use `advise_if_enabled()`? Patrick ``` ## Thomas Bachem via GitGitGadget, 2026-09-28 11:58 Subject: [PATCH v5 1/3] rerere: wait for MERGE_RR.lock before giving up Message-ID: <3dc3d02f12a3118ac9e270c19960815f6b8170cb.1790596702.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem setup_rerere() takes MERGE_RR.lock with LOCK_DIE_ON_ERROR, so of two processes that want it at the same time the second one dies. That used to be rare. Since 452b12c2e0 (builtin/maintenance: use "geometric" strategy by default, 2026-02-24) the auto maintenance after a commit runs "git rerere gc" whenever rr-cache has enough stale entries, and the gc holds the lock while it prunes. A rebase whose next pick conflicts while that happens dies inside repo_rerere(), before the sequencer has written the state that "git rebase --continue" needs. Every later "git rebase --continue" then fails with "you have staged changes in your working tree". Wait for the lock for up to rerere.lockTimeout milliseconds, 1000 by default, and only then fail as before. Pruning a few thousand entries takes well under a second, so the default covers a rebase that runs into the gc. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 8 +++++ rerere.c | 22 ++++++++++--- t/t4200-rerere.sh | 53 ++++++++++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 5 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 3a78b5ebb1..30e827f32b 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -10,3 +10,11 @@ rerere.enabled:: enabled if there is an `rr-cache` directory under the `$GIT_DIR`, e.g. if "rerere" was previously used in the repository. + +rerere.lockTimeout:: + The length of time, in milliseconds, to wait for the rerere + lock when another process holds it, typically a background + `git rerere gc`. Value 0 means not to wait at all; -1 means + to wait indefinitely. Default is 1000 (i.e., wait for 1 + second). When the time is up, the command fails as it does + for any other lock it cannot take. diff --git a/rerere.c b/rerere.c index 1c3745d9e3..64fac07c71 100644 --- a/rerere.c +++ b/rerere.c @@ -33,6 +33,9 @@ static int rerere_enabled = -1; /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; +/* how long to wait for MERGE_RR.lock, in milliseconds */ +static int rerere_lock_timeout_ms = 1000; + #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 struct rerere_dir { @@ -850,6 +853,8 @@ static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); + repo_config_get_int(the_repository, "rerere.locktimeout", + &rerere_lock_timeout_ms); repo_config(the_repository, git_default_config, NULL); } @@ -882,12 +887,19 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) + if (flags & RERERE_READONLY) { fd = 0; - else - fd = repo_hold_lock_file_for_update(r, &write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + } else { + /* + * Another process may hold the lock for a while, e.g. + * "git rerere gc" while it prunes rr-cache, so wait for + * it instead of dying right away. + */ + fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, + git_path_merge_rr(r), + LOCK_DIE_ON_ERROR, + rerere_lock_timeout_ms); + } read_rr(r, merge_rr); return fd; } diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 7bb601e117..7bd92235dc 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,59 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { + ( sleep 1 && rm -f .git/MERGE_RR.lock ) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && + test_grep ! "MERGE_RR" err && + test_grep "^=======\$" $rr/preimage +' + +test_expect_success 'merge fails once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && + test_grep "Unable to create" err && + test_grep "^=======\$" a1 && + test_path_is_missing $rr/preimage +' + +test_expect_success 'rerere, forget, clear and gc fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere gc 2>err && + test_grep "Unable to create" err +' + +test_expect_success 'rebase --abort fails on a lock it cannot take' ' + git reset --hard && + git checkout -b lock-held-abort third && + test_when_finished "git checkout third && git branch -D lock-held-abort" && + test_must_fail git rebase first && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase --abort 2>err && + test_grep "Unable to create" err && + test_path_is_dir .git/rebase-merge && + rm .git/MERGE_RR.lock && + git rebase --abort && + test_path_is_missing .git/rebase-merge +' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-09-28 11:58 Subject: [PATCH v5 0/3] rerere: wait for MERGE_RR.lock, and go on at a conflict Message-ID: In-Reply-To: ``` This is now three patches: the wait on its own, the gc's skip as "git rerere gc --auto", and the conflict-time callers on top, as before. Patch 1 makes every writer wait rerere.lockTimeout for MERGE_RR.lock before it fails as it does today. That alone keeps a rebase alive through the gc that a commit's auto maintenance starts, since pruning a few thousand entries takes well under the one second default. Patch 2 is the split Patrick asked for: a "git rerere gc" run by hand waits and then fails as it does today, like every other rerere command, and only "git rerere gc --auto" skips a held lock, quietly, as "git gc --auto" does with its own lock. "git maintenance run --auto" and "git gc --auto" pass the option, as they do for "git pack-refs --auto". A manual or scheduled maintenance run does not and fails on a held lock like the command itself. I left scheduled runs on that side since maintenance treats them like manual runs for pack-refs and gc too. Patch 3 is v4's second patch: a command that stops at a conflict warns and goes on when the lock is still held after the wait, and the warning says to run "git rerere" before resolving. Changes since v4: * Based on today's master, which has ps/tune-rerere-gc, and merges cleanly into next. In seen it only conflicts with its own v4. * "git rerere gc" no longer skips a held lock on its own. Only "git rerere gc --auto" does, and silently, so a background run prints nothing (Patrick, Phillip). * The git-rerere(1) sentence with the merge and rebase examples is gone. The gc paragraph describes --auto instead (Patrick). * I kept the timeout an int, like core.filesRefLockTimeout and core.packedRefsTimeout, and dropped the long local (Patrick). * The hint to run "git rerere" goes through advise_if_enabled() under advice.mergeConflict (Patrick). * Tests: the old gc test became the "gc --auto" test of patch 2, "git rerere gc" joined the commands that fail on a held lock in patch 1, and t7900 checks that maintenance passes --auto and that a run without it fails on a held lock. Thomas Bachem (3): rerere: wait for MERGE_RR.lock before giving up rerere: add "gc --auto" that skips a held lock rerere: go on at a conflict when the lock stays busy Documentation/config/rerere.adoc | 13 +++ Documentation/git-rerere.adoc | 9 +- apply.c | 2 +- builtin/am.c | 3 +- builtin/gc.c | 4 +- builtin/merge.c | 2 +- builtin/rerere.c | 13 ++- builtin/stash.c | 2 +- rerere.c | 56 ++++++++-- rerere.h | 6 +- sequencer.c | 4 +- t/t4200-rerere.sh | 175 +++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 25 ++++- 13 files changed, 292 insertions(+), 22 deletions(-) base-commit: 34f06850c16c7f7ac822b1adc71354f11b0f2ca3 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v5 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v5 Pull-Request: https://github.com/gitgitgadget/git/pull/2214 Range-diff vs v4: 1: 8a7a74d6aa ! 1: 3dc3d02f12 rerere: wait for MERGE_RR.lock, and let the gc skip it @@ Metadata Author: Thomas Bachem ## Commit message ## - rerere: wait for MERGE_RR.lock, and let the gc skip it + rerere: wait for MERGE_RR.lock before giving up - setup_rerere() takes MERGE_RR.lock with LOCK_DIE_ON_ERROR. When two - processes want the lock at the same time, the second one dies. This - was always the case, but since 452b12c2e0 (builtin/maintenance: use - "geometric" strategy by default, 2026-02-24) it is easy to hit: auto - maintenance now runs "git rerere gc" after every commit whenever - rr-cache contains at least one entry, and the gc holds the lock - while it prunes. + setup_rerere() takes MERGE_RR.lock with LOCK_DIE_ON_ERROR, so of two + processes that want it at the same time the second one dies. That + used to be rare. Since 452b12c2e0 (builtin/maintenance: use + "geometric" strategy by default, 2026-02-24) the auto maintenance + after a commit runs "git rerere gc" whenever rr-cache has enough + stale entries, and the gc holds the lock while it prunes. - A rebase whose next pick conflicts while the gc holds the lock dies - inside repo_rerere(). That runs before the sequencer writes the state - that "git rebase --continue" needs, so every later "git rebase - --continue" fails with "you have staged changes". + A rebase whose next pick conflicts while that happens dies inside + repo_rerere(), before the sequencer has written the state that + "git rebase --continue" needs. Every later "git rebase --continue" + then fails with "you have staged changes in your working tree". - Instead of dying right away, wait for the lock for up to - rerere.lockTimeout milliseconds, 1000 by default, and only then fail - as before. The gc itself does not wait: when the lock is held, it - skips this run and leaves the pruning to the next one. + Wait for the lock for up to rerere.lockTimeout milliseconds, 1000 by + default, and only then fail as before. Pruning a few thousand entries + takes well under a second, so the default covers a rebase that runs + into the gc. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem @@ Documentation/config/rerere.adoc: rerere.enabled:: + `git rerere gc`. Value 0 means not to wait at all; -1 means + to wait indefinitely. Default is 1000 (i.e., wait for 1 + second). When the time is up, the command fails as it does -+ for any other lock it cannot take. `git rerere gc` never -+ waits and skips its run while the lock is held. - - ## Documentation/git-rerere.adoc ## -@@ Documentation/git-rerere.adoc: occurred a long time ago. By default, unresolved conflicts older - than 15 days and resolved conflicts older than 60 - days are pruned. These defaults are controlled via the - `gc.rerereUnresolved` and `gc.rerereResolved` configuration --variables respectively. -+variables respectively. If another process holds the rerere lock, -+for example a merge or rebase that is recording a conflict, `gc` -+does nothing and says so. - - - DISCUSSION ++ for any other lock it cannot take. ## rerere.c ## @@ rerere.c: static int rerere_enabled = -1; @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) -+ if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) -+ BUG("RERERE_READONLY takes no lock, so RERERE_NOWAIT does not apply"); + if (flags & RERERE_READONLY) { fd = 0; - else @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + } else { -+ const char *path = git_path_merge_rr(r); -+ int lock_flags = LOCK_DIE_ON_ERROR; -+ long timeout_ms = rerere_lock_timeout_ms; -+ + /* + * Another process may hold the lock for a while, e.g. + * "git rerere gc" while it prunes rr-cache, so wait for -+ * it instead of dying right away. The gc itself never -+ * waits: skipping one of its runs costs nothing. ++ * it instead of dying right away. + */ -+ if (flags & RERERE_NOWAIT) { -+ lock_flags = 0; -+ timeout_ms = 0; -+ } + fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, -+ path, lock_flags, -+ timeout_ms); -+ if (fd < 0) { -+ warning_errno(_("skipping rerere, " -+ "unable to create '%s.lock'"), path); -+ return -1; -+ } ++ git_path_merge_rr(r), ++ LOCK_DIE_ON_ERROR, ++ rerere_lock_timeout_ms); + } read_rr(r, merge_rr); return fd; } -@@ rerere.c: void rerere_gc(struct repository *r, struct string_list *rr) - timestamp_t cutoff_resolve = now - 60 * 86400; - struct strbuf buf = STRBUF_INIT; - -- if (setup_rerere(r, rr, 0) < 0) -+ if (setup_rerere(r, rr, RERERE_NOWAIT) < 0) - return; - - repo_config_get_expiry_in_days(the_repository, "gc.rerereresolved", - - ## rerere.h ## -@@ rerere.h: struct repository; - #define RERERE_AUTOUPDATE 01 - #define RERERE_NOAUTOUPDATE 02 - #define RERERE_READONLY 04 -+/* Never wait for MERGE_RR.lock, and skip the run when it is held */ -+#define RERERE_NOWAIT 010 - - /* - * Marks paths that have been hand-resolved and added to the ## t/t4200-rerere.sh ## @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' -+test_expect_success 'gc does nothing while MERGE_RR is locked' ' -+ mkdir -p $rr2 && -+ echo Hello >$rr2/preimage && -+ test-tool chmtime =$just_over_15_days_ago $rr2/preimage && -+ -+ test_when_finished "rm -f .git/MERGE_RR.lock" && -+ >.git/MERGE_RR.lock && -+ git rerere gc 2>err && -+ test_grep "MERGE_RR.lock" err && -+ test_path_is_file $rr2/preimage && -+ -+ rm .git/MERGE_RR.lock && -+ git rerere gc && -+ test_path_is_missing $rr2/preimage -+' -+ +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' + test_path_is_missing $rr/preimage +' + -+test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' ++test_expect_success 'rerere, forget, clear and gc fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && ++ test_grep "Unable to create" err && ++ test_must_fail git -c rerere.lockTimeout=0 rerere gc 2>err && + test_grep "Unable to create" err +' + @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' - - ## t/t7900-maintenance.sh ## -@@ t/t7900-maintenance.sh: test_expect_success 'rerere-gc task with --auto honors maintenance.rerere-gc.aut - test_expect_rerere_gc ! git -c maintenance.rerere-gc.auto=0 maintenance run --auto --task=rerere-gc - ' - -+test_expect_success 'rerere-gc task succeeds while MERGE_RR is locked' ' -+ test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && -+ mkdir .git/rr-cache && -+ : >.git/rr-cache/entry && -+ >.git/MERGE_RR.lock && -+ test_expect_rerere_gc git maintenance run --task=rerere-gc -+' -+ - test_expect_success '--auto and --schedule incompatible' ' - test_must_fail git maintenance run --auto --schedule=daily 2>err && - test_grep "cannot be used together" err -: ---------- > 2: 27673137aa rerere: add "gc --auto" that skips a held lock 2: 1cce403113 ! 3: 3984c7666b rerere: go on at a conflict when the lock stays busy @@ Commit message When a merge, rebase, cherry-pick, revert, am, stash or apply stops at a conflict, it runs rerere right before it returns to the user. If MERGE_RR.lock is still held when rerere.lockTimeout runs out, the - command dies there. For a rebase that is worse than a lost - recording: the sequencer has not yet written the state that - "git rebase --continue" needs, so the rebase cannot continue, and - following the "git commit --amend" advice folds the conflicted pick - into the previous commit. + command dies there. A rebase loses more than a recording that way. + The sequencer has not yet written the state that "git rebase + --continue" needs, so the rebase cannot go on, and the "git commit + --amend" it suggests instead folds the conflicted pick into the + previous commit. - So print a warning and go on instead. The conflict is still in - place, and the warning tells the user to run "git rerere" before - resolving it. That records the preimage, or replays a known - resolution, just as the command would have done. + So warn and go on. The conflict is still in place, and the warning + tells the user to run "git rerere" before resolving it, which records + the preimage or replays a known resolution as the command would have. + The hint is under advice.mergeConflict like the other hints printed + at a conflict stop. - All other callers still fail when the timeout runs out. "git commit" - and "git am --continue" record the resolution and then move on to - the next commit or patch. A warning would come too late there, and - the next rerere run would record whatever the file contains by then. - The "rerere clear" that am and rebase run for --skip and --abort - would leave the same stale entry behind. "git rerere", "git rerere - forget" and "git rerere clear" fail because the user asked for that - state explicitly. + Callers that run rerere after a resolution, like "git commit" and + "git am --continue", still fail when the wait is up. They move on + right away, and a leftover MERGE_RR entry would make the next rerere + run record whatever the file holds by then. The user's own rerere + commands and the "rerere clear" of --skip and --abort fail as well. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem @@ Documentation/config/rerere.adoc: rerere.lockTimeout:: `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 - second). When the time is up, the command fails as it does -- for any other lock it cannot take. `git rerere gc` never -- waits and skips its run while the lock is held. +- for any other lock it cannot take. `git rerere gc --auto` +- does not wait and does nothing while the lock is held. + second). When the time is up, a command that stops at a + conflict, such as `git merge` or `git rebase`, prints a + warning and goes on without rerere; run `git rerere` before + resolving the conflict to record it after all. Any other + command fails, as it does for any other lock it cannot take. -+ `git rerere gc` never waits and skips its run while the lock -+ is held. ++ `git rerere gc --auto` does not wait and does nothing while ++ the lock is held. ## apply.c ## @@ apply.c: static int write_out_results(struct apply_state *state, struct patch *list) @@ builtin/merge.c: static int suggest_conflicts(void) return 1; ## builtin/stash.c ## -@@ builtin/stash.c: static int do_apply_stash(const char *prefix, struct stash_info *info, +@@ builtin/stash.c: static enum stash_apply_result do_apply_stash(const char *prefix, ret = error(_("could not write index")); if (ret) { @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) -- BUG("RERERE_READONLY takes no lock, so RERERE_NOWAIT does not apply"); +- BUG("RERERE_NOWAIT does not apply with RERERE_READONLY"); + if ((flags & RERERE_READONLY) && + (flags & (RERERE_NOWAIT | RERERE_SKIP_LOCKED))) + BUG("RERERE_READONLY takes no lock, so no lock flag applies"); if (flags & RERERE_READONLY) { fd = 0; } else { ++ const char *path = git_path_merge_rr(r); + int lock_flags = LOCK_DIE_ON_ERROR; + int timeout_ms = rerere_lock_timeout_ms; + @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) - * Another process may hold the lock for a while, e.g. * "git rerere gc" while it prunes rr-cache, so wait for - * it instead of dying right away. The gc itself never -- * waits: skipping one of its runs costs nothing. -+ * waits: skipping one of its runs costs nothing. A -+ * command that stops at a conflict must not die here -+ * either, so it warns and goes on without rerere. + * it instead of dying right away. The gc of an automatic + * maintenance run does not wait, since skipping one of +- * its runs costs nothing. ++ * its runs costs nothing. A command that stops at a ++ * conflict must not die here either, so it warns and ++ * goes on without rerere. */ if (flags & RERERE_NOWAIT) { lock_flags = 0; @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i + if (flags & RERERE_SKIP_LOCKED) + lock_flags = 0; fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, - path, lock_flags, - timeout_ms); - if (fd < 0) { - warning_errno(_("skipping rerere, " - "unable to create '%s.lock'"), path); -+ if (flags & RERERE_SKIP_LOCKED) -+ advise(_("run \"git rerere\" before resolving " -+ "the conflict to record or replay " -+ "its resolution")); +- git_path_merge_rr(r), +- lock_flags, timeout_ms); +- if (fd < 0) ++ path, lock_flags, ++ timeout_ms); ++ if (fd < 0) { ++ if (flags & RERERE_SKIP_LOCKED) { ++ warning_errno(_("skipping rerere, " ++ "unable to create '%s.lock'"), ++ path); ++ advise_if_enabled(ADVICE_MERGE_CONFLICT, ++ _("run \"git rerere\" before " ++ "resolving the conflict to " ++ "record or replay its " ++ "resolution")); ++ } return -1; - } ++ } } + read_rr(r, merge_rr); + return fd; ## rerere.h ## @@ rerere.h: struct repository; #define RERERE_READONLY 04 - /* Never wait for MERGE_RR.lock, and skip the run when it is held */ + /* Take MERGE_RR.lock only if it is free, and return quietly otherwise */ #define RERERE_NOWAIT 010 +/* Warn and go on without rerere if MERGE_RR.lock cannot be taken in time */ +#define RERERE_SKIP_LOCKED 020 @@ t/t4200-rerere.sh: test_expect_success 'a held lock is waited out within rerere. + test_grep "^=======\$" a1 +' + - test_expect_success 'rerere, forget and clear fail on a lock they cannot take' ' + test_expect_success 'rerere, forget, clear and gc fail on a lock they cannot take' ' test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-09-28 11:58 Subject: [PATCH v5 2/3] rerere: add "gc --auto" that skips a held lock Message-ID: <27673137aae961105a3d3b6ea615879e0686cc65.1790596702.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem The previous commit made "git rerere gc" wait for MERGE_RR.lock like every other rerere command before it fails, as it always has. That suits a user who runs it by hand and wants to know when nothing was pruned. It does not suit the gc that auto maintenance starts after a commit: nobody is waiting for that run, and the next commit starts another one. So add "--auto", with which "git rerere gc" quietly does nothing when the lock is taken, and pass it from "git maintenance run --auto" and "git gc --auto", as they already do for "git pack-refs --auto". A run without it, from the command line or a maintenance schedule, keeps waiting and failing. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 3 ++- Documentation/git-rerere.adoc | 9 ++++++--- builtin/gc.c | 4 +++- builtin/rerere.c | 13 ++++++++++--- rerere.c | 22 +++++++++++++++++----- rerere.h | 4 +++- t/t4200-rerere.sh | 21 +++++++++++++++++++++ t/t7900-maintenance.sh | 25 ++++++++++++++++++++++++- 8 files changed, 86 insertions(+), 15 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 30e827f32b..a58c2ff684 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -17,4 +17,5 @@ rerere.lockTimeout:: `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 second). When the time is up, the command fails as it does - for any other lock it cannot take. + for any other lock it cannot take. `git rerere gc --auto` + does not wait and does nothing while the lock is held. diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc index 4e6ab9a27c..da7a1d093e 100644 --- a/Documentation/git-rerere.adoc +++ b/Documentation/git-rerere.adoc @@ -8,7 +8,7 @@ git-rerere - Reuse recorded resolution of conflicted merges SYNOPSIS -------- [verse] -'git rerere' [clear | forget ... | diff | status | remaining | gc] +'git rerere' [clear | forget ... | diff | status | remaining | gc [--auto]] DESCRIPTION ----------- @@ -63,14 +63,17 @@ Print paths with conflicts that have not been autoresolved by rerere. This includes paths whose resolutions cannot be tracked by rerere, such as conflicting submodules. -'gc':: +'gc' [--auto]:: Prune records of conflicted merges that occurred a long time ago. By default, unresolved conflicts older than 15 days and resolved conflicts older than 60 days are pruned. These defaults are controlled via the `gc.rerereUnresolved` and `gc.rerereResolved` configuration -variables respectively. +variables respectively. With `--auto`, which `git maintenance run +--auto` and `git gc --auto` pass, `gc` does nothing while another +process holds the rerere lock. Without it, `gc` waits for the lock +as long as `rerere.lockTimeout` allows and then fails. DISCUSSION diff --git a/builtin/gc.c b/builtin/gc.c index 57a3520263..5c33082b5b 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -385,12 +385,14 @@ out: return should_prune; } -static int maintenance_task_rerere_gc(struct maintenance_run_opts *opts UNUSED, +static int maintenance_task_rerere_gc(struct maintenance_run_opts *opts, struct gc_config *cfg UNUSED) { struct child_process rerere_cmd = CHILD_PROCESS_INIT; rerere_cmd.git_cmd = 1; strvec_pushl(&rerere_cmd.args, "rerere", "gc", NULL); + if (opts->auto_flag) + strvec_push(&rerere_cmd.args, "--auto"); return run_command(&rerere_cmd); } diff --git a/builtin/rerere.c b/builtin/rerere.c index a056cb791b..2a8871df41 100644 --- a/builtin/rerere.c +++ b/builtin/rerere.c @@ -12,7 +12,8 @@ #include "pathspec.h" static const char * const rerere_usage[] = { - N_("git rerere [clear | forget ... | diff | status | remaining | gc]"), + N_("git rerere [clear | forget ... | diff | status | " + "remaining | gc [--auto]]"), NULL, }; @@ -56,16 +57,21 @@ int cmd_rerere(int argc, struct repository *repo UNUSED) { struct string_list merge_rr = STRING_LIST_INIT_DUP; - int autoupdate = -1, flags = 0; + int autoupdate = -1, auto_flag = 0, flags = 0; struct option options[] = { OPT_SET_INT(0, "rerere-autoupdate", &autoupdate, N_("register clean resolutions in index"), 1), + OPT_BOOL(0, "auto", &auto_flag, + N_("skip gc while another process holds the lock")), OPT_END(), }; argc = parse_options(argc, argv, prefix, options, rerere_usage, 0); + if (auto_flag && (argc < 1 || strcmp(argv[0], "gc"))) + die(_("the option '%s' requires '%s'"), "--auto", "gc"); + repo_config(the_repository, git_xmerge_config, NULL); if (autoupdate == 1) @@ -94,7 +100,8 @@ int cmd_rerere(int argc, if (!strcmp(argv[0], "clear")) { rerere_clear(the_repository, &merge_rr); } else if (!strcmp(argv[0], "gc")) - rerere_gc(the_repository, &merge_rr); + rerere_gc(the_repository, &merge_rr, + auto_flag ? RERERE_NOWAIT : 0); else if (!strcmp(argv[0], "status")) { if (setup_rerere(the_repository, &merge_rr, flags | RERERE_READONLY) < 0) diff --git a/rerere.c b/rerere.c index 64fac07c71..43c8eb04db 100644 --- a/rerere.c +++ b/rerere.c @@ -887,18 +887,30 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); + if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) + BUG("RERERE_NOWAIT does not apply with RERERE_READONLY"); if (flags & RERERE_READONLY) { fd = 0; } else { + int lock_flags = LOCK_DIE_ON_ERROR; + int timeout_ms = rerere_lock_timeout_ms; + /* * Another process may hold the lock for a while, e.g. * "git rerere gc" while it prunes rr-cache, so wait for - * it instead of dying right away. + * it instead of dying right away. The gc of an automatic + * maintenance run does not wait, since skipping one of + * its runs costs nothing. */ + if (flags & RERERE_NOWAIT) { + lock_flags = 0; + timeout_ms = 0; + } fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, git_path_merge_rr(r), - LOCK_DIE_ON_ERROR, - rerere_lock_timeout_ms); + lock_flags, timeout_ms); + if (fd < 0) + return -1; } read_rr(r, merge_rr); return fd; @@ -1284,7 +1296,7 @@ out: return needed; } -void rerere_gc(struct repository *r, struct string_list *rr) +void rerere_gc(struct repository *r, struct string_list *rr, int flags) { struct string_list to_remove = STRING_LIST_INIT_DUP; DIR *dir; @@ -1294,7 +1306,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) + if (setup_rerere(r, rr, flags) < 0) return; rerere_gc_cutoffs(r, &cutoff_resolve, &cutoff_noresolve); diff --git a/rerere.h b/rerere.h index feeb0e2c9f..d54c53d0d4 100644 --- a/rerere.h +++ b/rerere.h @@ -10,6 +10,8 @@ struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 +/* Take MERGE_RR.lock only if it is free, and return quietly otherwise */ +#define RERERE_NOWAIT 010 /* * Marks paths that have been hand-resolved and added to the @@ -37,7 +39,7 @@ const char *rerere_path(struct strbuf *buf, const struct rerere_id *, int rerere_forget(struct repository *, struct pathspec *); int rerere_remaining(struct repository *, struct string_list *); void rerere_clear(struct repository *, struct string_list *); -void rerere_gc(struct repository *, struct string_list *); +void rerere_gc(struct repository *, struct string_list *, int); /* * Check whether garbage collection for rerere entries is needed, which is diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 7bd92235dc..9435b0ed58 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,27 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'gc --auto does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc --auto 2>err && + test_must_be_empty err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && + git rerere gc --auto && + test_path_is_missing $rr2/preimage +' + +test_expect_success '--auto is only accepted by gc' ' + test_must_fail git rerere --auto clear 2>err && + test_grep "option .--auto. requires .gc." err +' + test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' git reset --hard && rm -rf $rr && diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 4f65fa9439..8842340236 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -1007,9 +1007,17 @@ test_expect_rerere_gc () { shift fi + # an automatic run passes --auto on to "git rerere gc" + auto= + case " $* " in + *" --auto "*) + auto=--auto + ;; + esac + rm -f "rerere-gc.txt" && GIT_TRACE2_EVENT="$(pwd)/rerere-gc.txt" "$@" && - test_subcommand $negate git rerere gc .git/MERGE_RR.lock && + test_expect_rerere_gc git -c maintenance.rerere-gc.auto=-1 maintenance run --auto --task=rerere-gc +' + +test_expect_success 'rerere-gc task without --auto fails while MERGE_RR is locked' ' + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && + mkdir .git/rr-cache && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 maintenance run --task=rerere-gc 2>err && + test_grep "Unable to create" err +' + test_expect_success '--auto and --schedule incompatible' ' test_must_fail git maintenance run --auto --schedule=daily 2>err && test_grep "cannot be used together" err -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-09-28 11:58 Subject: [PATCH v5 3/3] rerere: go on at a conflict when the lock stays busy Message-ID: <3984c7666bc3ee1e7670f4e092f1d59c0935c2bc.1790596702.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem When a merge, rebase, cherry-pick, revert, am, stash or apply stops at a conflict, it runs rerere right before it returns to the user. If MERGE_RR.lock is still held when rerere.lockTimeout runs out, the command dies there. A rebase loses more than a recording that way. The sequencer has not yet written the state that "git rebase --continue" needs, so the rebase cannot go on, and the "git commit --amend" it suggests instead folds the conflicted pick into the previous commit. So warn and go on. The conflict is still in place, and the warning tells the user to run "git rerere" before resolving it, which records the preimage or replays a known resolution as the command would have. The hint is under advice.mergeConflict like the other hints printed at a conflict stop. Callers that run rerere after a resolution, like "git commit" and "git am --continue", still fail when the wait is up. They move on right away, and a leftover MERGE_RR entry would make the next rerere run record whatever the file holds by then. The user's own rerere commands and the "rerere clear" of --skip and --abort fail as well. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 10 ++- apply.c | 2 +- builtin/am.c | 3 +- builtin/merge.c | 2 +- builtin/stash.c | 2 +- rerere.c | 30 +++++++-- rerere.h | 2 + sequencer.c | 4 +- t/t4200-rerere.sh | 105 ++++++++++++++++++++++++++++++- 9 files changed, 143 insertions(+), 17 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index a58c2ff684..d9ea56f5b2 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -16,6 +16,10 @@ rerere.lockTimeout:: lock when another process holds it, typically a background `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 - second). When the time is up, the command fails as it does - for any other lock it cannot take. `git rerere gc --auto` - does not wait and does nothing while the lock is held. + second). When the time is up, a command that stops at a + conflict, such as `git merge` or `git rebase`, prints a + warning and goes on without rerere; run `git rerere` before + resolving the conflict to record it after all. Any other + command fails, as it does for any other lock it cannot take. + `git rerere gc --auto` does not wait and does nothing while + the lock is held. diff --git a/apply.c b/apply.c index f00b7ba4d3..3b8502535b 100644 --- a/apply.c +++ b/apply.c @@ -4865,7 +4865,7 @@ static int write_out_results(struct apply_state *state, struct patch *list) * tree with conflict markers, but that isn't written with --cached. */ if (!state->cached) - repo_rerere(state->repo, 0); + repo_rerere(state->repo, RERERE_SKIP_LOCKED); } return errs; diff --git a/builtin/am.c b/builtin/am.c index e9623b8307..bcb93db515 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1649,7 +1649,8 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa o.verbosity = 0; if (merge_ort_generic(&o, &our_tree, &their_tree, 1, bases, &result)) { - repo_rerere(the_repository, state->allow_rerere_autoupdate); + repo_rerere(the_repository, state->allow_rerere_autoupdate | + RERERE_SKIP_LOCKED); free(their_tree_name); return error(_("Failed to merge in the changes.")); } diff --git a/builtin/merge.c b/builtin/merge.c index 5b4eb23a83..1ed5959bfd 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -1061,7 +1061,7 @@ static int suggest_conflicts(void) fputs(msgbuf.buf, fp); strbuf_release(&msgbuf); fclose(fp); - repo_rerere(the_repository, allow_rerere_auto); + repo_rerere(the_repository, allow_rerere_auto | RERERE_SKIP_LOCKED); printf(_("Automatic merge failed; " "fix conflicts and then commit the result.\n")); return 1; diff --git a/builtin/stash.c b/builtin/stash.c index 7a9843413b..3f9fd20569 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -732,7 +732,7 @@ static enum stash_apply_result do_apply_stash(const char *prefix, ret = error(_("could not write index")); if (ret) { - repo_rerere(the_repository, 0); + repo_rerere(the_repository, RERERE_SKIP_LOCKED); if (index) fprintf_ln(stderr, _("Index was not unstashed.")); diff --git a/rerere.c b/rerere.c index 43c8eb04db..5a036fe456 100644 --- a/rerere.c +++ b/rerere.c @@ -3,6 +3,7 @@ #include "git-compat-util.h" #include "abspath.h" +#include "advice.h" #include "config.h" #include "copy.h" #include "environment.h" @@ -887,11 +888,13 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) - BUG("RERERE_NOWAIT does not apply with RERERE_READONLY"); + if ((flags & RERERE_READONLY) && + (flags & (RERERE_NOWAIT | RERERE_SKIP_LOCKED))) + BUG("RERERE_READONLY takes no lock, so no lock flag applies"); if (flags & RERERE_READONLY) { fd = 0; } else { + const char *path = git_path_merge_rr(r); int lock_flags = LOCK_DIE_ON_ERROR; int timeout_ms = rerere_lock_timeout_ms; @@ -900,17 +903,32 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) * "git rerere gc" while it prunes rr-cache, so wait for * it instead of dying right away. The gc of an automatic * maintenance run does not wait, since skipping one of - * its runs costs nothing. + * its runs costs nothing. A command that stops at a + * conflict must not die here either, so it warns and + * goes on without rerere. */ if (flags & RERERE_NOWAIT) { lock_flags = 0; timeout_ms = 0; } + if (flags & RERERE_SKIP_LOCKED) + lock_flags = 0; fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, - git_path_merge_rr(r), - lock_flags, timeout_ms); - if (fd < 0) + path, lock_flags, + timeout_ms); + if (fd < 0) { + if (flags & RERERE_SKIP_LOCKED) { + warning_errno(_("skipping rerere, " + "unable to create '%s.lock'"), + path); + advise_if_enabled(ADVICE_MERGE_CONFLICT, + _("run \"git rerere\" before " + "resolving the conflict to " + "record or replay its " + "resolution")); + } return -1; + } } read_rr(r, merge_rr); return fd; diff --git a/rerere.h b/rerere.h index d54c53d0d4..ed8a878f8f 100644 --- a/rerere.h +++ b/rerere.h @@ -12,6 +12,8 @@ struct repository; #define RERERE_READONLY 04 /* Take MERGE_RR.lock only if it is free, and return quietly otherwise */ #define RERERE_NOWAIT 010 +/* Warn and go on without rerere if MERGE_RR.lock cannot be taken in time */ +#define RERERE_SKIP_LOCKED 020 /* * Marks paths that have been hand-resolved and added to the diff --git a/sequencer.c b/sequencer.c index 6dae43e4db..dabba729c7 100644 --- a/sequencer.c +++ b/sequencer.c @@ -2520,7 +2520,7 @@ static enum pick_result do_pick_commit(struct repository *r, : _("could not apply %s... %s"), short_commit_name(r, commit), msg.subject); print_advice(r, res == 1, opts); - repo_rerere(r, opts->allow_rerere_auto); + repo_rerere(r, opts->allow_rerere_auto | RERERE_SKIP_LOCKED); goto leave; } @@ -4449,7 +4449,7 @@ static int do_merge(struct repository *r, rollback_lock_file(&lock); if (ret) - repo_rerere(r, opts->allow_rerere_auto); + repo_rerere(r, opts->allow_rerere_auto | RERERE_SKIP_LOCKED); else /* * In case of problems, we now want to return a positive diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 9435b0ed58..b5209d8c04 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -277,17 +277,118 @@ test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' test_grep "^=======\$" $rr/preimage ' -test_expect_success 'merge fails once rerere.lockTimeout is up' ' +test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' git reset --hard && rm -rf $rr && test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && - test_grep "Unable to create" err && + test_grep "skipping rerere" err && + test_grep "hint: .*git rerere" err && test_grep "^=======\$" a1 && test_path_is_missing $rr/preimage ' +test_expect_success 'rerere run at the stop records what was skipped' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-catch-up third && + test_when_finished "git checkout third && git branch -D lock-held-catch-up" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first && + test_path_is_missing $rr/preimage && + rm .git/MERGE_RR.lock && + git rerere && + test_grep "^=======\$" $rr/preimage && + echo resolved >a1 && + git add a1 && + git commit -qm resolved && + test_path_is_file $rr/postimage +' + +test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held third && + test_when_finished "git checkout third && git branch -D lock-held" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && + test_grep "skipping rerere" err && + test_path_is_file .git/rebase-merge/stopped-sha && + rm .git/MERGE_RR.lock && + echo resolved >a1 && + git add a1 && + git rebase --continue && + test_path_is_missing .git/rebase-merge && + test_path_is_missing $rr/preimage +' + +test_expect_success 'commit fails on a lock it cannot take' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-commit third && + test_when_finished "git checkout third && git branch -D lock-held-commit" && + test_must_fail git merge first && + test_path_is_file $rr/preimage && + echo resolved >a1 && + git add a1 && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 commit -qm resolved 2>err && + test_grep "Unable to create" err && + test_path_is_missing $rr/postimage +' + +test_expect_success 'am goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-am third && + test_when_finished "test_might_fail git am --abort && + git checkout third && git branch -D lock-held-am" && + git format-patch -1 --stdout first >first.patch && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 am --3way first.patch 2>err && + test_grep "skipping rerere" err && + test_path_is_dir .git/rebase-apply && + rm .git/MERGE_RR.lock && + git am --abort && + test_path_is_missing .git/rebase-apply +' + +test_expect_success 'stash pop goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-stash third && + test_when_finished "git reset --hard && git stash drop && + git checkout third && git branch -D lock-held-stash" && + echo stashed >>a1 && + git stash && + echo committed >>a1 && + git commit -qam committed && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 stash pop 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 +' + +test_expect_success 'apply --3way goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-apply third && + test_when_finished "git reset --hard && + git checkout third && git branch -D lock-held-apply" && + git format-patch -1 --stdout first >first.patch && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 apply --3way first.patch 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 +' + test_expect_success 'rerere, forget, clear and gc fail on a lock they cannot take' ' test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && -- gitgitgadget ``` ## Patrick Steinhardt, 2026-09-30 15:00 Subject: Re: [PATCH v5 2/3] rerere: add "gc --auto" that skips a held lock Message-ID: In-Reply-To: <27673137aae961105a3d3b6ea615879e0686cc65.1790596702.git.gitgitgadget@gmail.com> ``` On Mon, Sep 28, 2026 at 11:58:21AM +0000, Thomas Bachem via GitGitGadget wrote: > diff --git a/Documentation/git-rerere.adoc b/Documentation/git-rerere.adoc > index 4e6ab9a27c..da7a1d093e 100644 > --- a/Documentation/git-rerere.adoc > +++ b/Documentation/git-rerere.adoc > @@ -63,14 +63,17 @@ Print paths with conflicts that have not been autoresolved by rerere. > This includes paths whose resolutions cannot be tracked by rerere, > such as conflicting submodules. > > -'gc':: > +'gc' [--auto]:: > > Prune records of conflicted merges that > occurred a long time ago. By default, unresolved conflicts older > than 15 days and resolved conflicts older than 60 > days are pruned. These defaults are controlled via the > `gc.rerereUnresolved` and `gc.rerereResolved` configuration > -variables respectively. > +variables respectively. With `--auto`, which `git maintenance run > +--auto` and `git gc --auto` pass, `gc` does nothing while another > +process holds the rerere lock. Without it, `gc` waits for the lock > +as long as `rerere.lockTimeout` allows and then fails. It's a bit weird to have git-rerere(1) document who calls it. We may want to document why specifically this is useful though. > diff --git a/builtin/rerere.c b/builtin/rerere.c > index a056cb791b..2a8871df41 100644 > --- a/builtin/rerere.c > +++ b/builtin/rerere.c > @@ -56,16 +57,21 @@ int cmd_rerere(int argc, > struct repository *repo UNUSED) > { > struct string_list merge_rr = STRING_LIST_INIT_DUP; > - int autoupdate = -1, flags = 0; > + int autoupdate = -1, auto_flag = 0, flags = 0; > > struct option options[] = { > OPT_SET_INT(0, "rerere-autoupdate", &autoupdate, > N_("register clean resolutions in index"), 1), > + OPT_BOOL(0, "auto", &auto_flag, > + N_("skip gc while another process holds the lock")), > OPT_END(), > }; Thinking about this a bit... I know it was my suggestion, but I wonder whether "auto" is misnamed. We don't let any heuristics kick in like we typically do for other commands like `git pack-refs --auto`, we only know to skip garbage collection if the lock is taken. So there is a bit of a mismatch here. How about we instead call this "--skip-locked"? We could even mark it as a hidden option and not even document it, as it feels very specific to how git-maintenance(1) wants to invoke it. If so, we could maybe remove it again at a later point. An alternative could be to instead call `rerere_gc()` directly, and if so we wouldn't have to add this flag at all. But that may result in some bigger changes, so I'll leave it up to you to decide. > argc = parse_options(argc, argv, prefix, options, rerere_usage, 0); > > + if (auto_flag && (argc < 1 || strcmp(argv[0], "gc"))) > + die(_("the option '%s' requires '%s'"), "--auto", "gc"); > + > repo_config(the_repository, git_xmerge_config, NULL); > > if (autoupdate == 1) Oh dear, this is a mess. The file could really use a refactoring to use proper subcommands. But anyway, that's certainly outside the scope of this patch series. Patrick ``` ## Thomas Bachem, 2026-10-01 08:08 Subject: Re: [PATCH v5 2/3] rerere: add "gc --auto" that skips a held lock Message-ID: In-Reply-To: ``` Hi Patrick, On 30/09/2026 17:00, Patrick Steinhardt wrote: > It's a bit weird to have git-rerere(1) document who calls it. We may > want to document why specifically this is useful though. I'll take that out of git-rerere(1) again. I'd keep the last sentence of the rerere.lockTimeout entry, since that is where I say what each command does when the time is up, but name the two commands there instead of the option: "A `git rerere gc` run by `git maintenance run --auto` or `git gc --auto` does not wait and does nothing while the lock is held." > How about we instead call this "--skip-locked"? We could even mark it as > a hidden option and not even document it, as it feels very specific to > how git-maintenance(1) wants to invoke it. If so, we could maybe remove > it again at a later point. I'll take both, the name and hiding it. Patch 3 has a RERERE_SKIP_LOCKED flag for the conflict-time callers. I'll rename that one to RERERE_WARN_LOCKED so it doesn't look like the option's flag, which stays RERERE_NOWAIT. > An alternative could be to instead call `rerere_gc()` directly, and if > so we wouldn't have to add this flag at all. But that may result in some > bigger changes, so I'll leave it up to you to decide. I tried it. It is six lines in builtin/gc.c, but rerere_gc() dies when it can't take the lock. A manual or scheduled "git maintenance run" then dies with the lockfile's message and exit code 128, where it now reports "task 'rerere-gc' failed" and exits with 1. The rerere-gc tests in t7900 fail too, since their helper looks for the "git rerere gc" child. So I'd keep the option for this series. Say if you'd rather have the direct call. I'll wait a day or two for other comments before I send v6. Thanks, Thomas ``` ## Patrick Steinhardt, 2026-10-01 11:19 Subject: Re: [PATCH v5 2/3] rerere: add "gc --auto" that skips a held lock Message-ID: In-Reply-To: ``` On Thu, Oct 01, 2026 at 10:08:16AM +0200, Thomas Bachem wrote: > Hi Patrick, > > On 30/09/2026 17:00, Patrick Steinhardt wrote: > > It's a bit weird to have git-rerere(1) document who calls it. We may > > want to document why specifically this is useful though. > > I'll take that out of git-rerere(1) again. I'd keep the last sentence > of the rerere.lockTimeout entry, since that is where I say what each > command does when the time is up, but name the two commands there > instead of the option: > > "A `git rerere gc` run by `git maintenance run --auto` or > `git gc --auto` does not wait and does nothing while the lock is held." > > > How about we instead call this "--skip-locked"? We could even mark it as > > a hidden option and not even document it, as it feels very specific to > > how git-maintenance(1) wants to invoke it. If so, we could maybe remove > > it again at a later point. > > I'll take both, the name and hiding it. > > Patch 3 has a RERERE_SKIP_LOCKED flag for the conflict-time callers. > I'll rename that one to RERERE_WARN_LOCKED so it doesn't look like the > option's flag, which stays RERERE_NOWAIT. > > > An alternative could be to instead call `rerere_gc()` directly, and if > > so we wouldn't have to add this flag at all. But that may result in some > > bigger changes, so I'll leave it up to you to decide. > > I tried it. It is six lines in builtin/gc.c, but rerere_gc() dies when > it can't take the lock. A manual or scheduled "git maintenance run" > then dies with the lockfile's message and exit code 128, where it now > reports "task 'rerere-gc' failed" and exits with 1. The rerere-gc > tests in t7900 fail too, since their helper looks for the > "git rerere gc" child. So I'd keep the option for this series. Say if > you'd rather have the direct call. Ah, right, that makes sense. Let's keep the hidden option in that case. Thanks! Patrick ``` ## Thomas Bachem via GitGitGadget, 2026-10-02 11:11 Subject: [PATCH v6 0/3] rerere: wait for MERGE_RR.lock, and go on at a conflict Message-ID: In-Reply-To: ``` A rebase dies at a conflict when a background "git rerere gc" holds MERGE_RR.lock at that moment. With the first patch, rerere waits for the lock instead of dying at once. With the second, the "git rerere gc" started by auto maintenance does nothing while the lock is held. With the third, a command that stops at a conflict goes on without rerere if the lock is still held after the wait. For v6 I took Patrick's suggestions for the second patch. Changes since v5: * Patch 2's option is "--skip-locked" instead of "--auto", since there is no heuristic behind it (Patrick). * The option is hidden and undocumented, like the "--skip-foreground-tasks" that maintenance passes to "git gc", and I no longer touch git-rerere(1) (Patrick). I kept the last sentence of the rerere.lockTimeout entry and named "git maintenance run --auto" and "git gc --auto" in it instead of the option. * I renamed patch 3's flag from RERERE_SKIP_LOCKED to RERERE_WARN_LOCKED, so that it does not look like the flag of "--skip-locked", which is still RERERE_NOWAIT. * I added a test to patch 3 for the merge that "git rebase -r" re-creates, the only call site without one. * I corrected and reworded the log messages of patches 2 and 3. Both implied that every rerere command takes the lock, but status, diff and remaining do not. Patch 2's also said that nobody waits for the gc of auto maintenance, but where maintenance does not detach, the command that starts it does. From patch 3's I dropped my explanation of why "git commit" and "git am --continue" still fail, which does not hold for "git commit": it has made its commit by the time it runs rerere. The message now gives the plain reason, that the rebase or am can still be continued when they fail. I kept the option rather than have maintenance call rerere_gc() directly. rerere_gc() dies when it cannot take the lock, so a manual or scheduled "git maintenance run" would die where it now reports the failed task (Patrick agreed). Thomas Bachem (3): rerere: wait for MERGE_RR.lock before giving up rerere: add "gc --skip-locked" for auto maintenance rerere: go on at a conflict when the lock stays busy Documentation/config/rerere.adoc | 14 +++ apply.c | 2 +- builtin/am.c | 3 +- builtin/gc.c | 4 +- builtin/merge.c | 2 +- builtin/rerere.c | 10 +- builtin/stash.c | 2 +- rerere.c | 56 +++++++-- rerere.h | 6 +- sequencer.c | 4 +- t/t4200-rerere.sh | 190 +++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 25 +++- 12 files changed, 300 insertions(+), 18 deletions(-) base-commit: 34f06850c16c7f7ac822b1adc71354f11b0f2ca3 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2214%2Fthomasbachem%2Frerere-gc-lock-v6 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2214/thomasbachem/rerere-gc-lock-v6 Pull-Request: https://github.com/gitgitgadget/git/pull/2214 Range-diff vs v5: 1: 3dc3d02f12 = 1: 3dc3d02f12 rerere: wait for MERGE_RR.lock before giving up 2: 27673137aa ! 2: 2ef141410a rerere: add "gc --auto" that skips a held lock @@ Metadata Author: Thomas Bachem ## Commit message ## - rerere: add "gc --auto" that skips a held lock + rerere: add "gc --skip-locked" for auto maintenance - The previous commit made "git rerere gc" wait for MERGE_RR.lock like - every other rerere command before it fails, as it always has. That - suits a user who runs it by hand and wants to know when nothing was - pruned. It does not suit the gc that auto maintenance starts after - a commit: nobody is waiting for that run, and the next commit starts - another one. + Since the previous commit, "git rerere gc" waits for MERGE_RR.lock + like every other command that takes it, and fails only if the wait + times out. That suits a user who runs it by hand and wants to know + when nothing was pruned. But the user did not ask for the gc that auto + maintenance starts after a commit, and the next commit starts another + one. - So add "--auto", with which "git rerere gc" quietly does nothing when - the lock is taken, and pass it from "git maintenance run --auto" and - "git gc --auto", as they already do for "git pack-refs --auto". A run - without it, from the command line or a maintenance schedule, keeps - waiting and failing. + So add "--skip-locked", with which "git rerere gc" quietly does + nothing while the lock is held, and pass it from + "git maintenance run --auto" and "git gc --auto". Only these two need + the option, so hide it and leave it undocumented, like the + "--skip-foreground-tasks" that "git maintenance run" passes to + "git gc". A run without it, from the command line or a maintenance + schedule, still waits for the lock and fails if the wait times out. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem @@ Documentation/config/rerere.adoc: rerere.lockTimeout:: to wait indefinitely. Default is 1000 (i.e., wait for 1 second). When the time is up, the command fails as it does - for any other lock it cannot take. -+ for any other lock it cannot take. `git rerere gc --auto` -+ does not wait and does nothing while the lock is held. - - ## Documentation/git-rerere.adoc ## -@@ Documentation/git-rerere.adoc: git-rerere - Reuse recorded resolution of conflicted merges - SYNOPSIS - -------- - [verse] --'git rerere' [clear | forget ... | diff | status | remaining | gc] -+'git rerere' [clear | forget ... | diff | status | remaining | gc [--auto]] - - DESCRIPTION - ----------- -@@ Documentation/git-rerere.adoc: Print paths with conflicts that have not been autoresolved by rerere. - This includes paths whose resolutions cannot be tracked by rerere, - such as conflicting submodules. - --'gc':: -+'gc' [--auto]:: - - Prune records of conflicted merges that - occurred a long time ago. By default, unresolved conflicts older - than 15 days and resolved conflicts older than 60 - days are pruned. These defaults are controlled via the - `gc.rerereUnresolved` and `gc.rerereResolved` configuration --variables respectively. -+variables respectively. With `--auto`, which `git maintenance run -+--auto` and `git gc --auto` pass, `gc` does nothing while another -+process holds the rerere lock. Without it, `gc` waits for the lock -+as long as `rerere.lockTimeout` allows and then fails. - - - DISCUSSION ++ for any other lock it cannot take. A `git rerere gc` run by ++ `git maintenance run --auto` or `git gc --auto` does not wait ++ and does nothing while the lock is held. ## builtin/gc.c ## @@ builtin/gc.c: out: @@ builtin/gc.c: out: rerere_cmd.git_cmd = 1; strvec_pushl(&rerere_cmd.args, "rerere", "gc", NULL); + if (opts->auto_flag) -+ strvec_push(&rerere_cmd.args, "--auto"); ++ strvec_push(&rerere_cmd.args, "--skip-locked"); return run_command(&rerere_cmd); } ## builtin/rerere.c ## -@@ - #include "pathspec.h" - - static const char * const rerere_usage[] = { -- N_("git rerere [clear | forget ... | diff | status | remaining | gc]"), -+ N_("git rerere [clear | forget ... | diff | status | " -+ "remaining | gc [--auto]]"), - NULL, - }; - @@ builtin/rerere.c: int cmd_rerere(int argc, struct repository *repo UNUSED) { struct string_list merge_rr = STRING_LIST_INIT_DUP; - int autoupdate = -1, flags = 0; -+ int autoupdate = -1, auto_flag = 0, flags = 0; ++ int autoupdate = -1, skip_locked = 0, flags = 0; struct option options[] = { OPT_SET_INT(0, "rerere-autoupdate", &autoupdate, N_("register clean resolutions in index"), 1), -+ OPT_BOOL(0, "auto", &auto_flag, -+ N_("skip gc while another process holds the lock")), ++ OPT_HIDDEN_BOOL(0, "skip-locked", &skip_locked, ++ N_("skip gc while another process holds the lock")), OPT_END(), }; argc = parse_options(argc, argv, prefix, options, rerere_usage, 0); -+ if (auto_flag && (argc < 1 || strcmp(argv[0], "gc"))) -+ die(_("the option '%s' requires '%s'"), "--auto", "gc"); ++ if (skip_locked && (argc < 1 || strcmp(argv[0], "gc"))) ++ die(_("the option '%s' requires '%s'"), "--skip-locked", "gc"); + repo_config(the_repository, git_xmerge_config, NULL); @@ builtin/rerere.c: int cmd_rerere(int argc, } else if (!strcmp(argv[0], "gc")) - rerere_gc(the_repository, &merge_rr); + rerere_gc(the_repository, &merge_rr, -+ auto_flag ? RERERE_NOWAIT : 0); ++ skip_locked ? RERERE_NOWAIT : 0); else if (!strcmp(argv[0], "status")) { if (setup_rerere(the_repository, &merge_rr, flags | RERERE_READONLY) < 0) @@ t/t4200-rerere.sh: test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' -+test_expect_success 'gc --auto does nothing while MERGE_RR is locked' ' ++test_expect_success 'gc --skip-locked does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && -+ git rerere gc --auto 2>err && ++ git rerere gc --skip-locked 2>err && + test_must_be_empty err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && -+ git rerere gc --auto && ++ git rerere gc --skip-locked && + test_path_is_missing $rr2/preimage +' + -+test_expect_success '--auto is only accepted by gc' ' -+ test_must_fail git rerere --auto clear 2>err && -+ test_grep "option .--auto. requires .gc." err ++test_expect_success '--skip-locked is only accepted by gc' ' ++ test_must_fail git rerere --skip-locked clear 2>err && ++ test_grep "option .--skip-locked. requires .gc." err +' + test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' @@ t/t7900-maintenance.sh: test_expect_rerere_gc () { shift fi -+ # an automatic run passes --auto on to "git rerere gc" -+ auto= ++ # An automatic run passes --skip-locked to "git rerere gc". ++ skip_locked= + case " $* " in + *" --auto "*) -+ auto=--auto ++ skip_locked=--skip-locked + ;; + esac + rm -f "rerere-gc.txt" && GIT_TRACE2_EVENT="$(pwd)/rerere-gc.txt" "$@" && - test_subcommand $negate git rerere gc @@ Documentation/config/rerere.adoc: rerere.lockTimeout:: `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 - second). When the time is up, the command fails as it does -- for any other lock it cannot take. `git rerere gc --auto` -- does not wait and does nothing while the lock is held. +- for any other lock it cannot take. A `git rerere gc` run by +- `git maintenance run --auto` or `git gc --auto` does not wait +- and does nothing while the lock is held. + second). When the time is up, a command that stops at a + conflict, such as `git merge` or `git rebase`, prints a + warning and goes on without rerere; run `git rerere` before + resolving the conflict to record it after all. Any other + command fails, as it does for any other lock it cannot take. -+ `git rerere gc --auto` does not wait and does nothing while -+ the lock is held. ++ A `git rerere gc` run by `git maintenance run --auto` or ++ `git gc --auto` does not wait and does nothing while the lock ++ is held. ## apply.c ## @@ apply.c: static int write_out_results(struct apply_state *state, struct patch *list) @@ apply.c: static int write_out_results(struct apply_state *state, struct patch *l */ if (!state->cached) - repo_rerere(state->repo, 0); -+ repo_rerere(state->repo, RERERE_SKIP_LOCKED); ++ repo_rerere(state->repo, RERERE_WARN_LOCKED); } return errs; @@ builtin/am.c: static int fall_back_threeway(const struct am_state *state, const if (merge_ort_generic(&o, &our_tree, &their_tree, 1, bases, &result)) { - repo_rerere(the_repository, state->allow_rerere_autoupdate); + repo_rerere(the_repository, state->allow_rerere_autoupdate | -+ RERERE_SKIP_LOCKED); ++ RERERE_WARN_LOCKED); free(their_tree_name); return error(_("Failed to merge in the changes.")); } @@ builtin/merge.c: static int suggest_conflicts(void) strbuf_release(&msgbuf); fclose(fp); - repo_rerere(the_repository, allow_rerere_auto); -+ repo_rerere(the_repository, allow_rerere_auto | RERERE_SKIP_LOCKED); ++ repo_rerere(the_repository, allow_rerere_auto | RERERE_WARN_LOCKED); printf(_("Automatic merge failed; " "fix conflicts and then commit the result.\n")); return 1; @@ builtin/stash.c: static enum stash_apply_result do_apply_stash(const char *prefi if (ret) { - repo_rerere(the_repository, 0); -+ repo_rerere(the_repository, RERERE_SKIP_LOCKED); ++ repo_rerere(the_repository, RERERE_WARN_LOCKED); if (index) fprintf_ln(stderr, _("Index was not unstashed.")); @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i - if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) - BUG("RERERE_NOWAIT does not apply with RERERE_READONLY"); + if ((flags & RERERE_READONLY) && -+ (flags & (RERERE_NOWAIT | RERERE_SKIP_LOCKED))) ++ (flags & (RERERE_NOWAIT | RERERE_WARN_LOCKED))) + BUG("RERERE_READONLY takes no lock, so no lock flag applies"); if (flags & RERERE_READONLY) { fd = 0; @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i lock_flags = 0; timeout_ms = 0; } -+ if (flags & RERERE_SKIP_LOCKED) ++ if (flags & RERERE_WARN_LOCKED) + lock_flags = 0; fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, - git_path_merge_rr(r), @@ rerere.c: int setup_rerere(struct repository *r, struct string_list *merge_rr, i + path, lock_flags, + timeout_ms); + if (fd < 0) { -+ if (flags & RERERE_SKIP_LOCKED) { ++ if (flags & RERERE_WARN_LOCKED) { + warning_errno(_("skipping rerere, " + "unable to create '%s.lock'"), + path); @@ rerere.h: struct repository; /* Take MERGE_RR.lock only if it is free, and return quietly otherwise */ #define RERERE_NOWAIT 010 +/* Warn and go on without rerere if MERGE_RR.lock cannot be taken in time */ -+#define RERERE_SKIP_LOCKED 020 ++#define RERERE_WARN_LOCKED 020 /* * Marks paths that have been hand-resolved and added to the @@ sequencer.c: static enum pick_result do_pick_commit(struct repository *r, short_commit_name(r, commit), msg.subject); print_advice(r, res == 1, opts); - repo_rerere(r, opts->allow_rerere_auto); -+ repo_rerere(r, opts->allow_rerere_auto | RERERE_SKIP_LOCKED); ++ repo_rerere(r, opts->allow_rerere_auto | RERERE_WARN_LOCKED); goto leave; } @@ sequencer.c: static int do_merge(struct repository *r, rollback_lock_file(&lock); if (ret) - repo_rerere(r, opts->allow_rerere_auto); -+ repo_rerere(r, opts->allow_rerere_auto | RERERE_SKIP_LOCKED); ++ repo_rerere(r, opts->allow_rerere_auto | RERERE_WARN_LOCKED); else /* * In case of problems, we now want to return a positive @@ t/t4200-rerere.sh: test_expect_success 'a held lock is waited out within rerere. + test_path_is_missing $rr/preimage +' + ++test_expect_success 'rebase -r goes on without rerere once rerere.lockTimeout is up' ' ++ git reset --hard && ++ git checkout -b lock-held-merge second && ++ test_when_finished "test_might_fail git rebase --abort && ++ git checkout third && git branch -D lock-held-merge" && ++ test_when_finished "rm -f .git/MERGE_RR.lock" && ++ >.git/MERGE_RR.lock && ++ test_must_fail git -c rerere.lockTimeout=0 rebase -r --force-rebase main 2>err && ++ test_grep "skipping rerere" err && ++ test_cmp_rev REBASE_HEAD second && ++ rm .git/MERGE_RR.lock && ++ git rebase --abort && ++ test_path_is_missing .git/rebase-merge ++' ++ +test_expect_success 'commit fails on a lock it cannot take' ' + git reset --hard && + rm -rf $rr && -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-10-02 11:11 Subject: [PATCH v6 1/3] rerere: wait for MERGE_RR.lock before giving up Message-ID: <3dc3d02f12a3118ac9e270c19960815f6b8170cb.1790939492.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem setup_rerere() takes MERGE_RR.lock with LOCK_DIE_ON_ERROR, so of two processes that want it at the same time the second one dies. That used to be rare. Since 452b12c2e0 (builtin/maintenance: use "geometric" strategy by default, 2026-02-24) the auto maintenance after a commit runs "git rerere gc" whenever rr-cache has enough stale entries, and the gc holds the lock while it prunes. A rebase whose next pick conflicts while that happens dies inside repo_rerere(), before the sequencer has written the state that "git rebase --continue" needs. Every later "git rebase --continue" then fails with "you have staged changes in your working tree". Wait for the lock for up to rerere.lockTimeout milliseconds, 1000 by default, and only then fail as before. Pruning a few thousand entries takes well under a second, so the default covers a rebase that runs into the gc. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 8 +++++ rerere.c | 22 ++++++++++--- t/t4200-rerere.sh | 53 ++++++++++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 5 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 3a78b5ebb1..30e827f32b 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -10,3 +10,11 @@ rerere.enabled:: enabled if there is an `rr-cache` directory under the `$GIT_DIR`, e.g. if "rerere" was previously used in the repository. + +rerere.lockTimeout:: + The length of time, in milliseconds, to wait for the rerere + lock when another process holds it, typically a background + `git rerere gc`. Value 0 means not to wait at all; -1 means + to wait indefinitely. Default is 1000 (i.e., wait for 1 + second). When the time is up, the command fails as it does + for any other lock it cannot take. diff --git a/rerere.c b/rerere.c index 1c3745d9e3..64fac07c71 100644 --- a/rerere.c +++ b/rerere.c @@ -33,6 +33,9 @@ static int rerere_enabled = -1; /* automatically update cleanly resolved paths to the index */ static int rerere_autoupdate; +/* how long to wait for MERGE_RR.lock, in milliseconds */ +static int rerere_lock_timeout_ms = 1000; + #define RR_HAS_POSTIMAGE 1 #define RR_HAS_PREIMAGE 2 struct rerere_dir { @@ -850,6 +853,8 @@ static void git_rerere_config(void) { repo_config_get_bool(the_repository, "rerere.enabled", &rerere_enabled); repo_config_get_bool(the_repository, "rerere.autoupdate", &rerere_autoupdate); + repo_config_get_int(the_repository, "rerere.locktimeout", + &rerere_lock_timeout_ms); repo_config(the_repository, git_default_config, NULL); } @@ -882,12 +887,19 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if (flags & RERERE_READONLY) + if (flags & RERERE_READONLY) { fd = 0; - else - fd = repo_hold_lock_file_for_update(r, &write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + } else { + /* + * Another process may hold the lock for a while, e.g. + * "git rerere gc" while it prunes rr-cache, so wait for + * it instead of dying right away. + */ + fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, + git_path_merge_rr(r), + LOCK_DIE_ON_ERROR, + rerere_lock_timeout_ms); + } read_rr(r, merge_rr); return fd; } diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 7bb601e117..7bd92235dc 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,59 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + { + ( sleep 1 && rm -f .git/MERGE_RR.lock ) & + } && + test_must_fail git -c rerere.lockTimeout=5000 merge first 2>err && + wait && + test_grep ! "MERGE_RR" err && + test_grep "^=======\$" $rr/preimage +' + +test_expect_success 'merge fails once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && + test_grep "Unable to create" err && + test_grep "^=======\$" a1 && + test_path_is_missing $rr/preimage +' + +test_expect_success 'rerere, forget, clear and gc fail on a lock they cannot take' ' + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rerere 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere forget a1 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere clear 2>err && + test_grep "Unable to create" err && + test_must_fail git -c rerere.lockTimeout=0 rerere gc 2>err && + test_grep "Unable to create" err +' + +test_expect_success 'rebase --abort fails on a lock it cannot take' ' + git reset --hard && + git checkout -b lock-held-abort third && + test_when_finished "git checkout third && git branch -D lock-held-abort" && + test_must_fail git rebase first && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase --abort 2>err && + test_grep "Unable to create" err && + test_path_is_dir .git/rebase-merge && + rm .git/MERGE_RR.lock && + git rebase --abort && + test_path_is_missing .git/rebase-merge +' + rerere_gc_custom_expiry_test () { five_days="$1" right_now="$2" test_expect_success "rerere gc with custom expiry ($five_days, $right_now)" ' -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-10-02 11:11 Subject: [PATCH v6 2/3] rerere: add "gc --skip-locked" for auto maintenance Message-ID: <2ef141410a1508477976f9e57cec05f1a7603264.1790939492.git.gitgitgadget@gmail.com> In-Reply-To: ``` From: Thomas Bachem Since the previous commit, "git rerere gc" waits for MERGE_RR.lock like every other command that takes it, and fails only if the wait times out. That suits a user who runs it by hand and wants to know when nothing was pruned. But the user did not ask for the gc that auto maintenance starts after a commit, and the next commit starts another one. So add "--skip-locked", with which "git rerere gc" quietly does nothing while the lock is held, and pass it from "git maintenance run --auto" and "git gc --auto". Only these two need the option, so hide it and leave it undocumented, like the "--skip-foreground-tasks" that "git maintenance run" passes to "git gc". A run without it, from the command line or a maintenance schedule, still waits for the lock and fails if the wait times out. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 4 +++- builtin/gc.c | 4 +++- builtin/rerere.c | 10 ++++++++-- rerere.c | 22 +++++++++++++++++----- rerere.h | 4 +++- t/t4200-rerere.sh | 21 +++++++++++++++++++++ t/t7900-maintenance.sh | 25 ++++++++++++++++++++++++- 7 files changed, 79 insertions(+), 11 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 30e827f32b..80c38ee951 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -17,4 +17,6 @@ rerere.lockTimeout:: `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 second). When the time is up, the command fails as it does - for any other lock it cannot take. + for any other lock it cannot take. A `git rerere gc` run by + `git maintenance run --auto` or `git gc --auto` does not wait + and does nothing while the lock is held. diff --git a/builtin/gc.c b/builtin/gc.c index 57a3520263..7ad3987b71 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -385,12 +385,14 @@ out: return should_prune; } -static int maintenance_task_rerere_gc(struct maintenance_run_opts *opts UNUSED, +static int maintenance_task_rerere_gc(struct maintenance_run_opts *opts, struct gc_config *cfg UNUSED) { struct child_process rerere_cmd = CHILD_PROCESS_INIT; rerere_cmd.git_cmd = 1; strvec_pushl(&rerere_cmd.args, "rerere", "gc", NULL); + if (opts->auto_flag) + strvec_push(&rerere_cmd.args, "--skip-locked"); return run_command(&rerere_cmd); } diff --git a/builtin/rerere.c b/builtin/rerere.c index a056cb791b..445f1df032 100644 --- a/builtin/rerere.c +++ b/builtin/rerere.c @@ -56,16 +56,21 @@ int cmd_rerere(int argc, struct repository *repo UNUSED) { struct string_list merge_rr = STRING_LIST_INIT_DUP; - int autoupdate = -1, flags = 0; + int autoupdate = -1, skip_locked = 0, flags = 0; struct option options[] = { OPT_SET_INT(0, "rerere-autoupdate", &autoupdate, N_("register clean resolutions in index"), 1), + OPT_HIDDEN_BOOL(0, "skip-locked", &skip_locked, + N_("skip gc while another process holds the lock")), OPT_END(), }; argc = parse_options(argc, argv, prefix, options, rerere_usage, 0); + if (skip_locked && (argc < 1 || strcmp(argv[0], "gc"))) + die(_("the option '%s' requires '%s'"), "--skip-locked", "gc"); + repo_config(the_repository, git_xmerge_config, NULL); if (autoupdate == 1) @@ -94,7 +99,8 @@ int cmd_rerere(int argc, if (!strcmp(argv[0], "clear")) { rerere_clear(the_repository, &merge_rr); } else if (!strcmp(argv[0], "gc")) - rerere_gc(the_repository, &merge_rr); + rerere_gc(the_repository, &merge_rr, + skip_locked ? RERERE_NOWAIT : 0); else if (!strcmp(argv[0], "status")) { if (setup_rerere(the_repository, &merge_rr, flags | RERERE_READONLY) < 0) diff --git a/rerere.c b/rerere.c index 64fac07c71..43c8eb04db 100644 --- a/rerere.c +++ b/rerere.c @@ -887,18 +887,30 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); + if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) + BUG("RERERE_NOWAIT does not apply with RERERE_READONLY"); if (flags & RERERE_READONLY) { fd = 0; } else { + int lock_flags = LOCK_DIE_ON_ERROR; + int timeout_ms = rerere_lock_timeout_ms; + /* * Another process may hold the lock for a while, e.g. * "git rerere gc" while it prunes rr-cache, so wait for - * it instead of dying right away. + * it instead of dying right away. The gc of an automatic + * maintenance run does not wait, since skipping one of + * its runs costs nothing. */ + if (flags & RERERE_NOWAIT) { + lock_flags = 0; + timeout_ms = 0; + } fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, git_path_merge_rr(r), - LOCK_DIE_ON_ERROR, - rerere_lock_timeout_ms); + lock_flags, timeout_ms); + if (fd < 0) + return -1; } read_rr(r, merge_rr); return fd; @@ -1284,7 +1296,7 @@ out: return needed; } -void rerere_gc(struct repository *r, struct string_list *rr) +void rerere_gc(struct repository *r, struct string_list *rr, int flags) { struct string_list to_remove = STRING_LIST_INIT_DUP; DIR *dir; @@ -1294,7 +1306,7 @@ void rerere_gc(struct repository *r, struct string_list *rr) timestamp_t cutoff_resolve; struct strbuf buf = STRBUF_INIT; - if (setup_rerere(r, rr, 0) < 0) + if (setup_rerere(r, rr, flags) < 0) return; rerere_gc_cutoffs(r, &cutoff_resolve, &cutoff_noresolve); diff --git a/rerere.h b/rerere.h index feeb0e2c9f..d54c53d0d4 100644 --- a/rerere.h +++ b/rerere.h @@ -10,6 +10,8 @@ struct repository; #define RERERE_AUTOUPDATE 01 #define RERERE_NOAUTOUPDATE 02 #define RERERE_READONLY 04 +/* Take MERGE_RR.lock only if it is free, and return quietly otherwise */ +#define RERERE_NOWAIT 010 /* * Marks paths that have been hand-resolved and added to the @@ -37,7 +39,7 @@ const char *rerere_path(struct strbuf *buf, const struct rerere_id *, int rerere_forget(struct repository *, struct pathspec *); int rerere_remaining(struct repository *, struct string_list *); void rerere_clear(struct repository *, struct string_list *); -void rerere_gc(struct repository *, struct string_list *); +void rerere_gc(struct repository *, struct string_list *, int); /* * Check whether garbage collection for rerere entries is needed, which is diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 7bd92235dc..28152bf456 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -242,6 +242,27 @@ test_expect_success 'old records rest in peace' ' test_path_is_missing $rr2/preimage ' +test_expect_success 'gc --skip-locked does nothing while MERGE_RR is locked' ' + mkdir -p $rr2 && + echo Hello >$rr2/preimage && + test-tool chmtime =$just_over_15_days_ago $rr2/preimage && + + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + git rerere gc --skip-locked 2>err && + test_must_be_empty err && + test_path_is_file $rr2/preimage && + + rm .git/MERGE_RR.lock && + git rerere gc --skip-locked && + test_path_is_missing $rr2/preimage +' + +test_expect_success '--skip-locked is only accepted by gc' ' + test_must_fail git rerere --skip-locked clear 2>err && + test_grep "option .--skip-locked. requires .gc." err +' + test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' git reset --hard && rm -rf $rr && diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 4f65fa9439..f0f9b37d4f 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -1007,9 +1007,17 @@ test_expect_rerere_gc () { shift fi + # An automatic run passes --skip-locked to "git rerere gc". + skip_locked= + case " $* " in + *" --auto "*) + skip_locked=--skip-locked + ;; + esac + rm -f "rerere-gc.txt" && GIT_TRACE2_EVENT="$(pwd)/rerere-gc.txt" "$@" && - test_subcommand $negate git rerere gc .git/MERGE_RR.lock && + test_expect_rerere_gc git -c maintenance.rerere-gc.auto=-1 maintenance run --auto --task=rerere-gc +' + +test_expect_success 'rerere-gc task without --auto fails while MERGE_RR is locked' ' + test_when_finished "rm -rf .git/rr-cache .git/MERGE_RR.lock" && + mkdir .git/rr-cache && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 maintenance run --task=rerere-gc 2>err && + test_grep "Unable to create" err +' + test_expect_success '--auto and --schedule incompatible' ' test_must_fail git maintenance run --auto --schedule=daily 2>err && test_grep "cannot be used together" err -- gitgitgadget ``` ## Thomas Bachem via GitGitGadget, 2026-10-02 11:11 Subject: [PATCH v6 3/3] rerere: go on at a conflict when the lock stays busy Message-ID: In-Reply-To: ``` From: Thomas Bachem When a merge, rebase, cherry-pick, revert, am, stash or apply stops at a conflict, it runs rerere right before it returns to the user. If MERGE_RR.lock is still held when rerere.lockTimeout runs out, the command dies there. In a rebase, the sequencer has not yet written the state that "git rebase --continue" needs. A later "git rebase --continue" fails, and the "git commit --amend" that its message offers first folds the conflicted pick into the previous commit. So warn and go on without rerere. The conflict is still in place, and a hint tells the user to run "git rerere" before resolving it. That records the preimage or replays a known resolution, as the command would have. The hint is under advice.mergeConflict like other hints printed at a conflict stop. Everything else that waits for the lock is left as it is and still fails if the wait times out. That includes "git commit" and "git am --continue", which run rerere after a resolution. When they fail, the rebase or am can still be continued. Assisted-by: Claude Fable 5.1 Signed-off-by: Thomas Bachem --- Documentation/config/rerere.adoc | 12 ++-- apply.c | 2 +- builtin/am.c | 3 +- builtin/merge.c | 2 +- builtin/stash.c | 2 +- rerere.c | 30 ++++++-- rerere.h | 2 + sequencer.c | 4 +- t/t4200-rerere.sh | 120 ++++++++++++++++++++++++++++++- 9 files changed, 159 insertions(+), 18 deletions(-) diff --git a/Documentation/config/rerere.adoc b/Documentation/config/rerere.adoc index 80c38ee951..08fb1cd37e 100644 --- a/Documentation/config/rerere.adoc +++ b/Documentation/config/rerere.adoc @@ -16,7 +16,11 @@ rerere.lockTimeout:: lock when another process holds it, typically a background `git rerere gc`. Value 0 means not to wait at all; -1 means to wait indefinitely. Default is 1000 (i.e., wait for 1 - second). When the time is up, the command fails as it does - for any other lock it cannot take. A `git rerere gc` run by - `git maintenance run --auto` or `git gc --auto` does not wait - and does nothing while the lock is held. + second). When the time is up, a command that stops at a + conflict, such as `git merge` or `git rebase`, prints a + warning and goes on without rerere; run `git rerere` before + resolving the conflict to record it after all. Any other + command fails, as it does for any other lock it cannot take. + A `git rerere gc` run by `git maintenance run --auto` or + `git gc --auto` does not wait and does nothing while the lock + is held. diff --git a/apply.c b/apply.c index f00b7ba4d3..f2b896af9d 100644 --- a/apply.c +++ b/apply.c @@ -4865,7 +4865,7 @@ static int write_out_results(struct apply_state *state, struct patch *list) * tree with conflict markers, but that isn't written with --cached. */ if (!state->cached) - repo_rerere(state->repo, 0); + repo_rerere(state->repo, RERERE_WARN_LOCKED); } return errs; diff --git a/builtin/am.c b/builtin/am.c index e9623b8307..aa09211461 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1649,7 +1649,8 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa o.verbosity = 0; if (merge_ort_generic(&o, &our_tree, &their_tree, 1, bases, &result)) { - repo_rerere(the_repository, state->allow_rerere_autoupdate); + repo_rerere(the_repository, state->allow_rerere_autoupdate | + RERERE_WARN_LOCKED); free(their_tree_name); return error(_("Failed to merge in the changes.")); } diff --git a/builtin/merge.c b/builtin/merge.c index 5b4eb23a83..68511614c7 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -1061,7 +1061,7 @@ static int suggest_conflicts(void) fputs(msgbuf.buf, fp); strbuf_release(&msgbuf); fclose(fp); - repo_rerere(the_repository, allow_rerere_auto); + repo_rerere(the_repository, allow_rerere_auto | RERERE_WARN_LOCKED); printf(_("Automatic merge failed; " "fix conflicts and then commit the result.\n")); return 1; diff --git a/builtin/stash.c b/builtin/stash.c index 7a9843413b..08062512c1 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -732,7 +732,7 @@ static enum stash_apply_result do_apply_stash(const char *prefix, ret = error(_("could not write index")); if (ret) { - repo_rerere(the_repository, 0); + repo_rerere(the_repository, RERERE_WARN_LOCKED); if (index) fprintf_ln(stderr, _("Index was not unstashed.")); diff --git a/rerere.c b/rerere.c index 43c8eb04db..bb7052d3a1 100644 --- a/rerere.c +++ b/rerere.c @@ -3,6 +3,7 @@ #include "git-compat-util.h" #include "abspath.h" +#include "advice.h" #include "config.h" #include "copy.h" #include "environment.h" @@ -887,11 +888,13 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & (RERERE_AUTOUPDATE|RERERE_NOAUTOUPDATE)) rerere_autoupdate = !!(flags & RERERE_AUTOUPDATE); - if ((flags & RERERE_READONLY) && (flags & RERERE_NOWAIT)) - BUG("RERERE_NOWAIT does not apply with RERERE_READONLY"); + if ((flags & RERERE_READONLY) && + (flags & (RERERE_NOWAIT | RERERE_WARN_LOCKED))) + BUG("RERERE_READONLY takes no lock, so no lock flag applies"); if (flags & RERERE_READONLY) { fd = 0; } else { + const char *path = git_path_merge_rr(r); int lock_flags = LOCK_DIE_ON_ERROR; int timeout_ms = rerere_lock_timeout_ms; @@ -900,17 +903,32 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) * "git rerere gc" while it prunes rr-cache, so wait for * it instead of dying right away. The gc of an automatic * maintenance run does not wait, since skipping one of - * its runs costs nothing. + * its runs costs nothing. A command that stops at a + * conflict must not die here either, so it warns and + * goes on without rerere. */ if (flags & RERERE_NOWAIT) { lock_flags = 0; timeout_ms = 0; } + if (flags & RERERE_WARN_LOCKED) + lock_flags = 0; fd = repo_hold_lock_file_for_update_timeout(r, &write_lock, - git_path_merge_rr(r), - lock_flags, timeout_ms); - if (fd < 0) + path, lock_flags, + timeout_ms); + if (fd < 0) { + if (flags & RERERE_WARN_LOCKED) { + warning_errno(_("skipping rerere, " + "unable to create '%s.lock'"), + path); + advise_if_enabled(ADVICE_MERGE_CONFLICT, + _("run \"git rerere\" before " + "resolving the conflict to " + "record or replay its " + "resolution")); + } return -1; + } } read_rr(r, merge_rr); return fd; diff --git a/rerere.h b/rerere.h index d54c53d0d4..12ff4a8adb 100644 --- a/rerere.h +++ b/rerere.h @@ -12,6 +12,8 @@ struct repository; #define RERERE_READONLY 04 /* Take MERGE_RR.lock only if it is free, and return quietly otherwise */ #define RERERE_NOWAIT 010 +/* Warn and go on without rerere if MERGE_RR.lock cannot be taken in time */ +#define RERERE_WARN_LOCKED 020 /* * Marks paths that have been hand-resolved and added to the diff --git a/sequencer.c b/sequencer.c index 6dae43e4db..17a775806d 100644 --- a/sequencer.c +++ b/sequencer.c @@ -2520,7 +2520,7 @@ static enum pick_result do_pick_commit(struct repository *r, : _("could not apply %s... %s"), short_commit_name(r, commit), msg.subject); print_advice(r, res == 1, opts); - repo_rerere(r, opts->allow_rerere_auto); + repo_rerere(r, opts->allow_rerere_auto | RERERE_WARN_LOCKED); goto leave; } @@ -4449,7 +4449,7 @@ static int do_merge(struct repository *r, rollback_lock_file(&lock); if (ret) - repo_rerere(r, opts->allow_rerere_auto); + repo_rerere(r, opts->allow_rerere_auto | RERERE_WARN_LOCKED); else /* * In case of problems, we now want to return a positive diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh index 28152bf456..a9dfe74091 100755 --- a/t/t4200-rerere.sh +++ b/t/t4200-rerere.sh @@ -277,17 +277,133 @@ test_expect_success 'a held lock is waited out within rerere.lockTimeout' ' test_grep "^=======\$" $rr/preimage ' -test_expect_success 'merge fails once rerere.lockTimeout is up' ' +test_expect_success 'merge goes on without rerere once rerere.lockTimeout is up' ' git reset --hard && rm -rf $rr && test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && test_must_fail git -c rerere.lockTimeout=0 merge first 2>err && - test_grep "Unable to create" err && + test_grep "skipping rerere" err && + test_grep "hint: .*git rerere" err && test_grep "^=======\$" a1 && test_path_is_missing $rr/preimage ' +test_expect_success 'rerere run at the stop records what was skipped' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-catch-up third && + test_when_finished "git checkout third && git branch -D lock-held-catch-up" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 merge first && + test_path_is_missing $rr/preimage && + rm .git/MERGE_RR.lock && + git rerere && + test_grep "^=======\$" $rr/preimage && + echo resolved >a1 && + git add a1 && + git commit -qm resolved && + test_path_is_file $rr/postimage +' + +test_expect_success 'rebase goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held third && + test_when_finished "git checkout third && git branch -D lock-held" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase first 2>err && + test_grep "skipping rerere" err && + test_path_is_file .git/rebase-merge/stopped-sha && + rm .git/MERGE_RR.lock && + echo resolved >a1 && + git add a1 && + git rebase --continue && + test_path_is_missing .git/rebase-merge && + test_path_is_missing $rr/preimage +' + +test_expect_success 'rebase -r goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + git checkout -b lock-held-merge second && + test_when_finished "test_might_fail git rebase --abort && + git checkout third && git branch -D lock-held-merge" && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 rebase -r --force-rebase main 2>err && + test_grep "skipping rerere" err && + test_cmp_rev REBASE_HEAD second && + rm .git/MERGE_RR.lock && + git rebase --abort && + test_path_is_missing .git/rebase-merge +' + +test_expect_success 'commit fails on a lock it cannot take' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-commit third && + test_when_finished "git checkout third && git branch -D lock-held-commit" && + test_must_fail git merge first && + test_path_is_file $rr/preimage && + echo resolved >a1 && + git add a1 && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 commit -qm resolved 2>err && + test_grep "Unable to create" err && + test_path_is_missing $rr/postimage +' + +test_expect_success 'am goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-am third && + test_when_finished "test_might_fail git am --abort && + git checkout third && git branch -D lock-held-am" && + git format-patch -1 --stdout first >first.patch && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 am --3way first.patch 2>err && + test_grep "skipping rerere" err && + test_path_is_dir .git/rebase-apply && + rm .git/MERGE_RR.lock && + git am --abort && + test_path_is_missing .git/rebase-apply +' + +test_expect_success 'stash pop goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-stash third && + test_when_finished "git reset --hard && git stash drop && + git checkout third && git branch -D lock-held-stash" && + echo stashed >>a1 && + git stash && + echo committed >>a1 && + git commit -qam committed && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 stash pop 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 +' + +test_expect_success 'apply --3way goes on without rerere once rerere.lockTimeout is up' ' + git reset --hard && + rm -rf $rr && + git checkout -b lock-held-apply third && + test_when_finished "git reset --hard && + git checkout third && git branch -D lock-held-apply" && + git format-patch -1 --stdout first >first.patch && + test_when_finished "rm -f .git/MERGE_RR.lock" && + >.git/MERGE_RR.lock && + test_must_fail git -c rerere.lockTimeout=0 apply --3way first.patch 2>err && + test_grep "skipping rerere" err && + test_grep "^=======\$" a1 +' + test_expect_success 'rerere, forget, clear and gc fail on a lock they cannot take' ' test_when_finished "rm -f .git/MERGE_RR.lock" && >.git/MERGE_RR.lock && -- gitgitgadget ```