{"thread":{"id":"66869","subject":"[PATCH 0/4] Improve performance of reflog expiration","startedAt":"2026-11-18T09:30:59Z","lastAt":"2026-11-19T11:59:45Z","messageCount":18,"participants":["Patrick Steinhardt","Karthik Nayak","Toon Claes","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"558685","messageId":"20261118-pks-reflog-expire-performance-v1-0-10b9263566a8@pks.im","threadId":"66869","inReplyTo":null,"subject":"[PATCH 0/4] Improve performance of reflog expiration","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T09:30:59Z","receivedAt":"2026-11-18T09:30:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nI recently noticed that a bunch of commands started hanging for almost a\nsecond after they're essentially done already. As it turns out, this was\ncaused by auto-maintenance, which was blocking in the foreground for\n~800ms.\n\nThe root cause of this is that the auto-condition of \"reflog-expire\"\ndoesn't properly evaluate reachability of reflog entries, so it instead\nconsidered every single reflog entry to point to an unreachable commit.\nHence, all entries were evaluated against \"gc.reflogExpireUnreachable\"\ninstead of \"gc.reflogExpire\", and thus the auto-condition significantly\novercounts the number of entries that we'd prune. So every single run of\ngit-maintenance(1) ultimately ended up spawning `git reflog expire`.\n\nThis is fixed by the last patch in this series. But I was a bit\nsurprised that a no-op `git reflog expire` took this long, so the first\ntwo patches optimize the reftable backend to be more efficient, too.\nIn my copy of \"git.git\":\n\n  Benchmark 1: git maintenance run --auto (commit = master) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    853.2 ms  ±    99.9 ms     570.9 ms  …   901.9 ms\n    Memory       117.5 MiB ±     0.3 MiB    117.2 MiB …   118.2 MiB\n\n  Benchmark 2: git maintenance run --auto (commit = reftable-optimizations) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    392.3 ms  ±     2.3 ms     388.0 ms  …   396.6 ms           -54.0%  (2.2x faster)\n    Memory       117.2 MiB ±     0.2 MiB    116.8 MiB …   117.4 MiB           -0.2%\n\n  Benchmark 3: git maintenance run --auto (commit = auto-condition-optimizations) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time     18.6 ms  ±     1.1 ms      17.4 ms  …    20.9 ms           -97.8%  (45.8x faster)\n    Memory        81.2 MiB ±     0.1 MiB     80.9 MiB …    81.4 MiB          -30.9%  (1.4x less)\n\nSo yeah, it's a bit more efficient now. :)\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (4):\n      refs/reftable: don't write new table on dry-run expire\n      refs/reftable: don't write new table when expire is a no-op\n      builtin/maintenance: only consider `gc.reflogExpire` for \"--auto\"\n      Documentation/config: fix typo in \"maintenance.reflog-expire.auto\"\n\n Documentation/config/maintenance.adoc |  2 +-\n builtin/gc.c                          | 33 +++++++++++++++++++++------------\n refs/reftable-backend.c               | 21 +++++++++++++++++----\n t/t0610-reftable-basics.sh            | 30 ++++++++++++++++++++++++++++++\n 4 files changed, 69 insertions(+), 17 deletions(-)\n\n\n---\nbase-commit: e1ec09b23c8e0deb21a0b95ac5996422277395d4\nchange-id: 20261118-pks-reflog-expire-performance-e051b93f901a\n\n\n"},{"id":"558686","messageId":"20261118-pks-reflog-expire-performance-v1-1-10b9263566a8@pks.im","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-0-10b9263566a8@pks.im","subject":"[PATCH 1/4] refs/reftable: don't write new table on dry-run expire","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T09:31:00Z","receivedAt":"2026-11-18T09:31:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The `git reflog expire` subcommand supports a dry-run flag. When set, we\nprint what would be pruned without actually pruning anything on disk.\n\nBut even though the expectation is that the on-disk state won't change,\nthe reftable backend still ends up writing a new reftable to disk that\ncontains the changes. This is of course rather pointless, as we would\nhave already printed the entries that we're about to prune at that point\nin time.\n\nSkip writing the data, which provides a small speedup:\n\n  Benchmark 1: git reflog expire --all --dry-run (commit = HEAD~) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    414.2 ms  ±     2.2 ms     411.8 ms  …   418.0 ms\n    Memory       116.9 MiB ±     0.1 MiB    116.6 MiB …   117.0 MiB\n\n  Benchmark 2: git reflog expire --all --dry-run (commit = HEAD) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    369.1 ms  ±     3.2 ms     361.9 ms  …   371.9 ms           -10.9%\n    Memory       116.8 MiB ±     0.1 MiB    116.7 MiB …   117.0 MiB           -0.1%\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n refs/reftable-backend.c | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 3ba6a9ac0db..4216a18684d 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2706,6 +2706,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n \tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n \n+\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n+\t\tret = 0;\n+\t\tgoto done;\n+\t}\n+\n \targ.refs = refs;\n \targ.records = rewritten;\n \targ.len = logs_nr;\n@@ -2720,8 +2725,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n \t * Future improvement: we could skip writing records that were\n \t * not changed.\n \t */\n-\tif (!(flags & EXPIRE_REFLOGS_DRY_RUN))\n-\t\tret = reftable_addition_commit(add);\n+\tret = reftable_addition_commit(add);\n \n done:\n \tif (add)\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558687","messageId":"20261118-pks-reflog-expire-performance-v1-2-10b9263566a8@pks.im","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-0-10b9263566a8@pks.im","subject":"[PATCH 2/4] refs/reftable: don't write new table when expire is a no-op","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T09:31:01Z","receivedAt":"2026-11-18T09:31:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When executing `git reflog expire` we may of course end up not expiring\nany reflog entries because none of them are older than \"gc.reflogExpire\"\nor \"gc.reflogExpireUnreachable\". But even so, the reftable backend would\nend up writing a new table to disk anyway that contains basically an\nexact copy of the current reflog. And with the \"--all\" flag, that would\nhappen once per existing reflog. This doesn't result in any correctness\nissues, but it is of course a lot of pointless busywork.\n\nSkip writing and committing the new reftable in the case where there\naren't any changes, which results in a nice speedup:\n\n  Benchmark 1: git reflog expire --all (commit = HEAD~) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    736.0 ms  ±    11.7 ms     720.0 ms  …   752.3 ms\n    Memory       117.2 MiB ±     0.1 MiB    117.1 MiB …   117.3 MiB\n\n  Benchmark 2: git reflog expire --all (commit = HEAD) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    370.3 ms  ±     3.7 ms     363.4 ms  …   375.6 ms           -49.7%  (2.0x faster)\n    Memory       116.8 MiB ±     0.2 MiB    116.5 MiB …   117.1 MiB           -0.3%\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n refs/reftable-backend.c    | 15 ++++++++++++---\n t/t0610-reftable-basics.sh | 30 ++++++++++++++++++++++++++++++\n 2 files changed, 42 insertions(+), 3 deletions(-)\n\ndiff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\nindex 4216a18684d..e2027bb9c02 100644\n--- a/refs/reftable-backend.c\n+++ b/refs/reftable-backend.c\n@@ -2607,6 +2607,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n \tstruct strbuf referent = STRBUF_INIT;\n \tuint8_t *last_hash = NULL;\n \tsize_t logs_nr = 0, logs_alloc = 0, i;\n+\tbool has_changes = false;\n \tunsigned int type = 0;\n \tint ret;\n \n@@ -2696,17 +2697,25 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n \t\t\t\t    logs[i].value.update.message,\n \t\t\t\t    policy_cb_data)) {\n \t\t\tdest->value_type = REFTABLE_LOG_DELETION;\n+\t\t\thas_changes = true;\n \t\t} else {\n-\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash)\n+\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash &&\n+\t\t\t    memcmp(dest->value.update.old_hash, last_hash,\n+\t\t\t\t   ref_store->repo->hash_algo->rawsz)) {\n \t\t\t\tmemcpy(dest->value.update.old_hash, last_hash, GIT_MAX_RAWSZ);\n+\t\t\t\thas_changes = true;\n+\t\t\t}\n \t\t\tlast_hash = logs[i].value.update.new_hash;\n \t\t}\n \t}\n \n-\tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n+\tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid)) {\n \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n+\t\tif (!oideq(&arg.update_oid, &oid))\n+\t\t\thas_changes = true;\n+\t}\n \n-\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n+\tif (!has_changes || (flags & EXPIRE_REFLOGS_DRY_RUN)) {\n \t\tret = 0;\n \t\tgoto done;\n \t}\ndiff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh\nindex a7027458395..d808f1887f6 100755\n--- a/t/t0610-reftable-basics.sh\n+++ b/t/t0610-reftable-basics.sh\n@@ -912,6 +912,36 @@ test_expect_success 'reflog: expiry empties reflog' '\n \t)\n '\n \n+test_expect_success 'reflog: expiry without changes does not write tables' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\n+\t\ttest_commit A &&\n+\t\ttest_commit B &&\n+\t\ttest_commit C &&\n+\t\tcp .git/reftable/tables.list expect &&\n+\n+\t\tgit reflog expire --all --expire=never --expire-unreachable=never &&\n+\t\ttest_cmp expect .git/reftable/tables.list &&\n+\n+\t\tgit reflog expire --all --rewrite \\\n+\t\t\t--expire=never --expire-unreachable=never &&\n+\t\ttest_cmp expect .git/reftable/tables.list &&\n+\n+\t\tgit reflog expire --all --updateref \\\n+\t\t\t--expire=never --expire-unreachable=never &&\n+\t\ttest_cmp expect .git/reftable/tables.list &&\n+\n+\t\tgit reflog expire --all --dry-run --expire=all &&\n+\t\ttest_cmp expect .git/reftable/tables.list &&\n+\n+\t\tgit reflog expire --all --expire=all &&\n+\t\t! test_cmp expect .git/reftable/tables.list\n+\t)\n+'\n+\n test_expect_success 'reflog: can be deleted' '\n \ttest_when_finished \"rm -rf repo\" &&\n \tgit init repo &&\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558688","messageId":"20261118-pks-reflog-expire-performance-v1-3-10b9263566a8@pks.im","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-0-10b9263566a8@pks.im","subject":"[PATCH 3/4] builtin/maintenance: only consider `gc.reflogExpire` for \"--auto\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T09:31:02Z","receivedAt":"2026-11-18T09:31:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The \"reflog-expire\" maintenance task is responsible for expiring reflog\nentries that are older than either \"gc.reflogExpire\" for entries that\nare still reachable, or \"gc.reflogExpireUnreachable\" for those that are\nnot reachable.\n\nWhen executed with \"--auto\", the condition that we use to trigger this\ntask verifies whether the number of reflog entries that we would prune\nis larger than \"maintenance.reflog-expire.auto\". This is evaluated by\nwalking the \"HEAD\" reflog, where we then check every entry by calling\n`should_expire_reflog_ent()`. That function will then check whether the\nreflog entry refers to a commit that is reachable or not, and depending\non the outcome compares against either \"gc.reflogExpire\", or against\nboth \"gc.reflogExpire\" and \"gc.reflogExpireUnreachable\".\n\nTo decide reachability the caller has to call `reflog_expiry_prepare()`,\nwhich amongst other things also performs a reachability walk for a given\nrefname. But we don't do that in the \"reflog-expire\" maintenance task,\nand hence every single reflog entry will be considered unreachable and\nthus we'd consider many reachable entries prunable even though they\nwon't be pruned. So we overcount the number of prunable entries quite a\nbit and will thus trigger `git reflog expire` even though it won't do\nanything.\n\nNow we could easiliy fix that by calling `reflog_expiry_prepare()`. But\nin the case of auto-maintenance it has the big downside that we now have\nto perform a reachability walk in the foreground, and that may be quite\nexpensive. The whole point of those auto-conditions is to use heuristics\nthat are fast to evaluate though so that the time we have to block is\nreduced as much as possible.\n\nInstead, adapt the heuristic to only evaluate reflog entries against\n\"gc.reflogExpire\". This of course means that we're now undercounting the\nnumber of prunable entries. But as part of background maintenance that\nfeels like a reasonable tradeoff, and a user that really wants to prune\nunreachable reflog entries can either execute git-maintenance(1) without\n\"--auto\", or they can run `git reflog expire` directly.\n\nIn any case, the consequence is that we only spawn git-reflog(1) now in\nthose cases where it would indeed prune entries. And in my own working\ncopy of \"git.git\" that I use daily that results in quite a performance\nimprovement:\n\n  Benchmark 1: git maintenance run --auto (commit = HEAD~) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time    385.2 ms  ±     1.9 ms     382.7 ms  …   388.6 ms\n    Memory       116.8 MiB ±     0.1 MiB    116.7 MiB …   117.0 MiB\n\n  Benchmark 2: git maintenance run --auto (commit = HEAD) (10 runs)\n                  mean     ±       σ          min     …     max\n    Wall Time     17.9 ms  ±     0.7 ms      17.2 ms  …    19.6 ms           -95.4%  (21.5x faster)\n    Memory        81.0 MiB ±     0.1 MiB     80.7 MiB …    81.2 MiB          -30.7%  (1.4x less)\n\nNote that this improvement can be observed after every single command\nthat triggers auto-maintenance, so this is overall quite an important\noptimization.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/gc.c | 33 +++++++++++++++++++++------------\n 1 file changed, 21 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 68cb04a672d..d4a1f337a29 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -282,19 +282,31 @@ static int maintenance_task_pack_refs(struct maintenance_run_opts *opts,\n }\n \n struct count_reflog_entries_data {\n-\tstruct expire_reflog_policy_cb policy;\n+\tstruct reflog_expire_options opts;\n \tsize_t count;\n \tsize_t limit;\n };\n \n static int count_reflog_entries(const char *refname UNUSED,\n-\t\t\t\tstruct object_id *old_oid, struct object_id *new_oid,\n-\t\t\t\tconst char *committer, timestamp_t timestamp,\n-\t\t\t\tint tz, const char *msg, void *cb_data)\n+\t\t\t\tstruct object_id *old_oid UNUSED,\n+\t\t\t\tstruct object_id *new_oid UNUSED,\n+\t\t\t\tconst char *committer UNUSED,\n+\t\t\t\ttimestamp_t timestamp,\n+\t\t\t\tint tz UNUSED,\n+\t\t\t\tconst char *msg UNUSED,\n+\t\t\t\tvoid *cb_data)\n {\n \tstruct count_reflog_entries_data *data = cb_data;\n-\tif (should_expire_reflog_ent(old_oid, new_oid, committer, timestamp, tz, msg, &data->policy))\n+\n+\t/*\n+\t * We only evaluate whether the entry is older than `gc.reflogExpire`\n+\t * and ignore `gc.reflogExpireUnreachable`. Evaluating the latter would\n+\t * require a reachability walk, and that's simply too expensive in this\n+\t * context.\n+\t */\n+\tif (timestamp < data->opts.expire_total)\n \t\tdata->count++;\n+\n \treturn data->count >= data->limit;\n }\n \n@@ -302,9 +314,7 @@ static int reflog_expire_condition(struct gc_config *cfg UNUSED)\n {\n \ttimestamp_t now = time(NULL);\n \tstruct count_reflog_entries_data data = {\n-\t\t.policy = {\n-\t\t\t.opts = REFLOG_EXPIRE_OPTIONS_INIT(now),\n-\t\t},\n+\t\t.opts = REFLOG_EXPIRE_OPTIONS_INIT(now),\n \t};\n \tint limit = 100;\n \n@@ -315,14 +325,13 @@ static int reflog_expire_condition(struct gc_config *cfg UNUSED)\n \t\treturn 1;\n \tdata.limit = limit;\n \n-\trepo_config(the_repository, reflog_expire_config, &data.policy.opts);\n+\trepo_config(the_repository, reflog_expire_config, &data.opts);\n \n-\treflog_expire_options_set_refname(&data.policy.opts, \"HEAD\");\n+\treflog_expire_options_set_refname(&data.opts, \"HEAD\");\n \trefs_for_each_reflog_ent(get_main_ref_store(the_repository), \"HEAD\",\n \t\t\t\t count_reflog_entries, &data);\n \n-\treflog_expiry_cleanup(&data.policy);\n-\treflog_clear_expire_config(&data.policy.opts);\n+\treflog_clear_expire_config(&data.opts);\n \treturn data.count >= data.limit;\n }\n \n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558689","messageId":"20261118-pks-reflog-expire-performance-v1-4-10b9263566a8@pks.im","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-0-10b9263566a8@pks.im","subject":"[PATCH 4/4] Documentation/config: fix typo in \"maintenance.reflog-expire.auto\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T09:31:03Z","receivedAt":"2026-11-18T09:31:03Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In the documentation of \"maintenance.reflog-expire.auto\" we accidentally\nrefer to \"maintenance.loose-objects.auto\" instead. Fix this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n Documentation/config/maintenance.adoc | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/Documentation/config/maintenance.adoc b/Documentation/config/maintenance.adoc\nindex 165102518ce..7066fe0e168 100644\n--- a/Documentation/config/maintenance.adoc\n+++ b/Documentation/config/maintenance.adoc\n@@ -116,7 +116,7 @@ maintenance.reflog-expire.auto::\n \tnegative value will force the task to run every time. Otherwise, a\n \tpositive value implies the command should run when the number of\n \texpired reflog entries in the \"HEAD\" reflog is at least the value of\n-\t`maintenance.loose-objects.auto`. The default value is 100.\n+\t`maintenance.reflog-expire.auto`. The default value is 100.\n \n maintenance.rerere-gc.auto::\n \tThis integer config option controls how often the `rerere-gc` task\n\n-- \n2.56.1.831.g720f3f5937.dirty\n\n\n"},{"id":"558695","messageId":"CAOLa=ZSx=m=5+Jw8JzKXutaYp_mKOWbSsBVTHtMT3rDUk3a=eA@mail.gmail.com","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-1-10b9263566a8@pks.im","subject":"Re: [PATCH 1/4] refs/reftable: don't write new table on dry-run expire","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2026-11-18T11:28:25Z","receivedAt":"2026-11-18T11:28:25Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The `git reflog expire` subcommand supports a dry-run flag. When set, we\n> print what would be pruned without actually pruning anything on disk.\n>\n> But even though the expectation is that the on-disk state won't change,\n> the reftable backend still ends up writing a new reftable to disk that\n> contains the changes. This is of course rather pointless, as we would\n> have already printed the entries that we're about to prune at that point\n> in time.\n>\n> Skip writing the data, which provides a small speedup:\n>\n>   Benchmark 1: git reflog expire --all --dry-run (commit = HEAD~) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    414.2 ms  ±     2.2 ms     411.8 ms  …   418.0 ms\n>     Memory       116.9 MiB ±     0.1 MiB    116.6 MiB …   117.0 MiB\n>\n>   Benchmark 2: git reflog expire --all --dry-run (commit = HEAD) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    369.1 ms  ±     3.2 ms     361.9 ms  …   371.9 ms           -10.9%\n>     Memory       116.8 MiB ±     0.1 MiB    116.7 MiB …   117.0 MiB           -0.1%\n>\n\nNice find. Obviously the previous flow was wrong.\n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  refs/reftable-backend.c | 8 ++++++--\n>  1 file changed, 6 insertions(+), 2 deletions(-)\n>\n> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> index 3ba6a9ac0db..4216a18684d 100644\n> --- a/refs/reftable-backend.c\n> +++ b/refs/reftable-backend.c\n> @@ -2706,6 +2706,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n>  \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n>\n> +\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n> +\t\tret = 0;\n> +\t\tgoto done;\n> +\t}\n> +\n>  \targ.refs = refs;\n>  \targ.records = rewritten;\n>  \targ.len = logs_nr;\n> @@ -2720,8 +2725,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \t * Future improvement: we could skip writing records that were\n>  \t * not changed.\n>  \t */\n\nThis comment doesn't seem to get removed in the following patch.\n\n> -\tif (!(flags & EXPIRE_REFLOGS_DRY_RUN))\n> -\t\tret = reftable_addition_commit(add);\n> +\tret = reftable_addition_commit(add);\n>\n\nThe changes in this patch look sensible.\n\n>  done:\n>  \tif (add)\n>\n> --\n> 2.56.1.831.g720f3f5937.dirty\n"},{"id":"558696","messageId":"CAOLa=ZQGJkyVY=e94hJVmLa2fuPVUfC3uFRsQTeoa95sUe3UcA@mail.gmail.com","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-2-10b9263566a8@pks.im","subject":"Re: [PATCH 2/4] refs/reftable: don't write new table when expire is a no-op","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2026-11-18T11:33:04Z","receivedAt":"2026-11-18T11:33:04Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> index 4216a18684d..e2027bb9c02 100644\n> --- a/refs/reftable-backend.c\n> +++ b/refs/reftable-backend.c\n> @@ -2607,6 +2607,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \tstruct strbuf referent = STRBUF_INIT;\n>  \tuint8_t *last_hash = NULL;\n>  \tsize_t logs_nr = 0, logs_alloc = 0, i;\n> +\tbool has_changes = false;\n>  \tunsigned int type = 0;\n>  \tint ret;\n>\n> @@ -2696,17 +2697,25 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \t\t\t\t    logs[i].value.update.message,\n>  \t\t\t\t    policy_cb_data)) {\n>  \t\t\tdest->value_type = REFTABLE_LOG_DELETION;\n> +\t\t\thas_changes = true;\n>  \t\t} else {\n> -\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash)\n> +\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash &&\n> +\t\t\t    memcmp(dest->value.update.old_hash, last_hash,\n> +\t\t\t\t   ref_store->repo->hash_algo->rawsz)) {\n>  \t\t\t\tmemcpy(dest->value.update.old_hash, last_hash, GIT_MAX_RAWSZ);\n\nWe can skip this second copy.\n\n> +\t\t\t\thas_changes = true;\n> +\t\t\t}\n>  \t\t\tlast_hash = logs[i].value.update.new_hash;\n>  \t\t}\n>  \t}\n>\n> -\tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n> +\tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid)) {\n>  \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n> +\t\tif (!oideq(&arg.update_oid, &oid))\n\nWe now check if its changed, makes sense.\n\n> +\t\t\thas_changes = true;\n> +\t}\n>\n> -\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n> +\tif (!has_changes || (flags & EXPIRE_REFLOGS_DRY_RUN)) {\n>  \t\tret = 0;\n>  \t\tgoto done;\n>  \t}\n\nNit: it would be nice to mention that we exit early to skip writing any\nnew tables.\n\n> diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh\n> index a7027458395..d808f1887f6 100755\n> --- a/t/t0610-reftable-basics.sh\n> +++ b/t/t0610-reftable-basics.sh\n> @@ -912,6 +912,36 @@ test_expect_success 'reflog: expiry empties reflog' '\n>  \t)\n>  '\n>\n> +test_expect_success 'reflog: expiry without changes does not write tables' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\n> +\t\ttest_commit A &&\n> +\t\ttest_commit B &&\n> +\t\ttest_commit C &&\n> +\t\tcp .git/reftable/tables.list expect &&\n> +\n> +\t\tgit reflog expire --all --expire=never --expire-unreachable=never &&\n> +\t\ttest_cmp expect .git/reftable/tables.list &&\n> +\n> +\t\tgit reflog expire --all --rewrite \\\n> +\t\t\t--expire=never --expire-unreachable=never &&\n> +\t\ttest_cmp expect .git/reftable/tables.list &&\n> +\n> +\t\tgit reflog expire --all --updateref \\\n> +\t\t\t--expire=never --expire-unreachable=never &&\n> +\t\ttest_cmp expect .git/reftable/tables.list &&\n> +\n> +\t\tgit reflog expire --all --dry-run --expire=all &&\n> +\t\ttest_cmp expect .git/reftable/tables.list &&\n> +\n> +\t\tgit reflog expire --all --expire=all &&\n> +\t\t! test_cmp expect .git/reftable/tables.list\n> +\t)\n> +'\n\nNice. Could we add a similar test for the previous patch too?\n"},{"id":"558702","messageId":"av2Ui2uOeUoES0-a@pks.im","threadId":"66869","inReplyTo":"CAOLa=ZSx=m=5+Jw8JzKXutaYp_mKOWbSsBVTHtMT3rDUk3a=eA@mail.gmail.com","subject":"Re: [PATCH 1/4] refs/reftable: don't write new table on dry-run expire","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T12:05:31Z","receivedAt":"2026-11-18T12:05:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 18, 2026 at 03:28:25AM -0800, Karthik Nayak wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> > diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> > index 3ba6a9ac0db..4216a18684d 100644\n> > --- a/refs/reftable-backend.c\n> > +++ b/refs/reftable-backend.c\n> > @@ -2706,6 +2706,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n> >  \tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n> >  \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n> >\n> > +\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n> > +\t\tret = 0;\n> > +\t\tgoto done;\n> > +\t}\n> > +\n> >  \targ.refs = refs;\n> >  \targ.records = rewritten;\n> >  \targ.len = logs_nr;\n> > @@ -2720,8 +2725,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n> >  \t * Future improvement: we could skip writing records that were\n> >  \t * not changed.\n> >  \t */\n> \n> This comment doesn't seem to get removed in the following patch.\n\nThat's because I don't fully consider this comment to be addressed. If I\nunderstand correctly, this also speaks about the fact that we could drop\nrewriting entries in a finer-grained way. All I'm changing is that we\nnow don't write any entries in case none of them need to be rewritten.\n\nPatrick\n\n"},{"id":"558703","messageId":"av2UlJLpSDOPKbua@pks.im","threadId":"66869","inReplyTo":"CAOLa=ZQGJkyVY=e94hJVmLa2fuPVUfC3uFRsQTeoa95sUe3UcA@mail.gmail.com","subject":"Re: [PATCH 2/4] refs/reftable: don't write new table when expire is a no-op","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-18T12:05:40Z","receivedAt":"2026-11-18T12:05:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 18, 2026 at 03:33:04AM -0800, Karthik Nayak wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> > diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> > index 4216a18684d..e2027bb9c02 100644\n> > --- a/refs/reftable-backend.c\n> > +++ b/refs/reftable-backend.c\n> > @@ -2696,17 +2697,25 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n> >  \t\t\t\t    logs[i].value.update.message,\n> >  \t\t\t\t    policy_cb_data)) {\n> >  \t\t\tdest->value_type = REFTABLE_LOG_DELETION;\n> > +\t\t\thas_changes = true;\n> >  \t\t} else {\n> > -\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash)\n> > +\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash &&\n> > +\t\t\t    memcmp(dest->value.update.old_hash, last_hash,\n> > +\t\t\t\t   ref_store->repo->hash_algo->rawsz)) {\n> >  \t\t\t\tmemcpy(dest->value.update.old_hash, last_hash, GIT_MAX_RAWSZ);\n> \n> We can skip this second copy.\n> \n> > +\t\t\t\thas_changes = true;\n> > +\t\t\t}\n> >  \t\t\tlast_hash = logs[i].value.update.new_hash;\n> >  \t\t}\n> >  \t}\n> >\n> > -\tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n> > +\tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid)) {\n> >  \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n> > +\t\tif (!oideq(&arg.update_oid, &oid))\n> \n> We now check if its changed, makes sense.\n> \n> > +\t\t\thas_changes = true;\n> > +\t}\n> >\n> > -\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n> > +\tif (!has_changes || (flags & EXPIRE_REFLOGS_DRY_RUN)) {\n> >  \t\tret = 0;\n> >  \t\tgoto done;\n> >  \t}\n> \n> Nit: it would be nice to mention that we exit early to skip writing any\n> new tables.\n\nHm. I thought that this was obvious from the code itself, so I didn't\nthink there was much value in adding a comment. I wouldn't really know\nwhat information to put here that doesn't just repeat the code exactly.\n\n> > diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh\n> > index a7027458395..d808f1887f6 100755\n> > --- a/t/t0610-reftable-basics.sh\n> > +++ b/t/t0610-reftable-basics.sh\n> > @@ -912,6 +912,36 @@ test_expect_success 'reflog: expiry empties reflog' '\n> >  \t)\n> >  '\n> >\n> > +test_expect_success 'reflog: expiry without changes does not write tables' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\n> > +\t\ttest_commit A &&\n> > +\t\ttest_commit B &&\n> > +\t\ttest_commit C &&\n> > +\t\tcp .git/reftable/tables.list expect &&\n> > +\n> > +\t\tgit reflog expire --all --expire=never --expire-unreachable=never &&\n> > +\t\ttest_cmp expect .git/reftable/tables.list &&\n> > +\n> > +\t\tgit reflog expire --all --rewrite \\\n> > +\t\t\t--expire=never --expire-unreachable=never &&\n> > +\t\ttest_cmp expect .git/reftable/tables.list &&\n> > +\n> > +\t\tgit reflog expire --all --updateref \\\n> > +\t\t\t--expire=never --expire-unreachable=never &&\n> > +\t\ttest_cmp expect .git/reftable/tables.list &&\n> > +\n> > +\t\tgit reflog expire --all --dry-run --expire=all &&\n> > +\t\ttest_cmp expect .git/reftable/tables.list &&\n> > +\n> > +\t\tgit reflog expire --all --expire=all &&\n> > +\t\t! test_cmp expect .git/reftable/tables.list\n> > +\t)\n> > +'\n> \n> Nice. Could we add a similar test for the previous patch too?\n\nThis test also exercises the preceding patch as we use \"--dry-run\" here.\nI could've split this up across the two commits, but I was lazy and just\nadded it here.\n\nPatrick\n\n"},{"id":"558713","messageId":"87jymajgqw.fsf@dev.null.iotcl.com.invalid","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-2-10b9263566a8@pks.im","subject":"Re: [PATCH 2/4] refs/reftable: don't write new table when expire is a no-op","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-11-18T15:34:15Z","receivedAt":"2026-11-18T15:34:15Z","isPatch":true,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> When executing `git reflog expire` we may of course end up not expiring\n> any reflog entries because none of them are older than \"gc.reflogExpire\"\n> or \"gc.reflogExpireUnreachable\". But even so, the reftable backend would\n> end up writing a new table to disk anyway that contains basically an\n> exact copy of the current reflog. And with the \"--all\" flag, that would\n> happen once per existing reflog. This doesn't result in any correctness\n> issues, but it is of course a lot of pointless busywork.\n>\n> Skip writing and committing the new reftable in the case where there\n> aren't any changes, which results in a nice speedup:\n>\n>   Benchmark 1: git reflog expire --all (commit = HEAD~) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    736.0 ms  ±    11.7 ms     720.0 ms  …   752.3 ms\n>     Memory       117.2 MiB ±     0.1 MiB    117.1 MiB …   117.3 MiB\n>\n>   Benchmark 2: git reflog expire --all (commit = HEAD) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    370.3 ms  ±     3.7 ms     363.4 ms  …   375.6 ms           -49.7%  (2.0x faster)\n>     Memory       116.8 MiB ±     0.2 MiB    116.5 MiB …   117.1 MiB           -0.3%\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  refs/reftable-backend.c    | 15 ++++++++++++---\n>  t/t0610-reftable-basics.sh | 30 ++++++++++++++++++++++++++++++\n>  2 files changed, 42 insertions(+), 3 deletions(-)\n>\n> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> index 4216a18684d..e2027bb9c02 100644\n> --- a/refs/reftable-backend.c\n> +++ b/refs/reftable-backend.c\n> @@ -2607,6 +2607,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \tstruct strbuf referent = STRBUF_INIT;\n>  \tuint8_t *last_hash = NULL;\n>  \tsize_t logs_nr = 0, logs_alloc = 0, i;\n> +\tbool has_changes = false;\n>  \tunsigned int type = 0;\n>  \tint ret;\n>  \n> @@ -2696,17 +2697,25 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \t\t\t\t    logs[i].value.update.message,\n>  \t\t\t\t    policy_cb_data)) {\n>  \t\t\tdest->value_type = REFTABLE_LOG_DELETION;\n> +\t\t\thas_changes = true;\n>  \t\t} else {\n> -\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash)\n> +\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash &&\n> +\t\t\t    memcmp(dest->value.update.old_hash, last_hash,\n> +\t\t\t\t   ref_store->repo->hash_algo->rawsz)) {\n>  \t\t\t\tmemcpy(dest->value.update.old_hash, last_hash, GIT_MAX_RAWSZ);\n\nI cannot help being bothered by using GIT_MAX_RAWSZ here, and\nref_store->repo->hash_algo->rawsz the line above. Would it make sense to\nbe consistent?\n\n-- \nLaters,\nToon\n\n"},{"id":"558715","messageId":"87h5hejfw2.fsf@dev.null.iotcl.com.invalid","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-3-10b9263566a8@pks.im","subject":"Re: [PATCH 3/4] builtin/maintenance: only consider `gc.reflogExpire` for \"--auto\"","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-11-18T15:52:45Z","receivedAt":"2026-11-18T15:52:45Z","isPatch":true,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The \"reflog-expire\" maintenance task is responsible for expiring reflog\n> entries that are older than either \"gc.reflogExpire\" for entries that\n> are still reachable, or \"gc.reflogExpireUnreachable\" for those that are\n> not reachable.\n>\n> When executed with \"--auto\", the condition that we use to trigger this\n> task verifies whether the number of reflog entries that we would prune\n> is larger than \"maintenance.reflog-expire.auto\". This is evaluated by\n> walking the \"HEAD\" reflog, where we then check every entry by calling\n> `should_expire_reflog_ent()`. That function will then check whether the\n> reflog entry refers to a commit that is reachable or not, and depending\n> on the outcome compares against either \"gc.reflogExpire\", or against\n> both \"gc.reflogExpire\" and \"gc.reflogExpireUnreachable\".\n>\n> To decide reachability the caller has to call `reflog_expiry_prepare()`,\n> which amongst other things also performs a reachability walk for a given\n> refname. But we don't do that in the \"reflog-expire\" maintenance task,\n> and hence every single reflog entry will be considered unreachable and\n> thus we'd consider many reachable entries prunable even though they\n> won't be pruned. So we overcount the number of prunable entries quite a\n> bit and will thus trigger `git reflog expire` even though it won't do\n> anything.\n>\n> Now we could easiliy fix that by calling `reflog_expiry_prepare()`.\n\nDo you mean \"not calling\"?\n\nAnyhow, from reading the diff and the surrounding code, it's not clear\nto me where this is/was called?\n\nThe fix itself though makes sense. You basically pull out the first\ncondition from should_expire_reflog_ent() and skip the reachability\ncheck. \n\n> But in the case of auto-maintenance it has the big downside that we now have\n> to perform a reachability walk in the foreground, and that may be quite\n> expensive. The whole point of those auto-conditions is to use heuristics\n> that are fast to evaluate though so that the time we have to block is\n> reduced as much as possible.\n>\n> Instead, adapt the heuristic to only evaluate reflog entries against\n> \"gc.reflogExpire\". This of course means that we're now undercounting the\n> number of prunable entries. But as part of background maintenance that\n> feels like a reasonable tradeoff, and a user that really wants to prune\n> unreachable reflog entries can either execute git-maintenance(1) without\n> \"--auto\", or they can run `git reflog expire` directly.\n\nYeah, makes sense to me to undercount.\n\n> In any case, the consequence is that we only spawn git-reflog(1) now in\n> those cases where it would indeed prune entries. And in my own working\n> copy of \"git.git\" that I use daily that results in quite a performance\n> improvement:\n>\n>   Benchmark 1: git maintenance run --auto (commit = HEAD~) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    385.2 ms  ±     1.9 ms     382.7 ms  …   388.6 ms\n>     Memory       116.8 MiB ±     0.1 MiB    116.7 MiB …   117.0 MiB\n>\n>   Benchmark 2: git maintenance run --auto (commit = HEAD) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time     17.9 ms  ±     0.7 ms      17.2 ms  …    19.6 ms           -95.4%  (21.5x faster)\n>     Memory        81.0 MiB ±     0.1 MiB     80.7 MiB …    81.2 MiB          -30.7%  (1.4x less)\n>\n> Note that this improvement can be observed after every single command\n> that triggers auto-maintenance, so this is overall quite an important\n> optimization.\n\nSeems to me the tradeoff is worthwhile.\n\n-- \nLaters,\nToon\n\n"},{"id":"558716","messageId":"87eccijftu.fsf@dev.null.iotcl.com.invalid","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-4-10b9263566a8@pks.im","subject":"Re: [PATCH 4/4] Documentation/config: fix typo in \"maintenance.reflog-expire.auto\"","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-11-18T15:54:05Z","receivedAt":"2026-11-18T15:54:05Z","isPatch":true,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> In the documentation of \"maintenance.reflog-expire.auto\" we accidentally\n> refer to \"maintenance.loose-objects.auto\" instead. Fix this.\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  Documentation/config/maintenance.adoc | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/Documentation/config/maintenance.adoc b/Documentation/config/maintenance.adoc\n> index 165102518ce..7066fe0e168 100644\n> --- a/Documentation/config/maintenance.adoc\n> +++ b/Documentation/config/maintenance.adoc\n> @@ -116,7 +116,7 @@ maintenance.reflog-expire.auto::\n>  \tnegative value will force the task to run every time. Otherwise, a\n>  \tpositive value implies the command should run when the number of\n>  \texpired reflog entries in the \"HEAD\" reflog is at least the value of\n> -\t`maintenance.loose-objects.auto`. The default value is 100.\n> +\t`maintenance.reflog-expire.auto`. The default value is 100.\n\n:) Whoops. Good find.\n\n-- \nLaters,\nToon\n\n"},{"id":"558717","messageId":"87bj7mjfp2.fsf@dev.null.iotcl.com.invalid","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-0-10b9263566a8@pks.im","subject":"Re: [PATCH 0/4] Improve performance of reflog expiration","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-11-18T15:56:57Z","receivedAt":"2026-11-18T15:56:57Z","isPatch":true,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> Hi,\n>\n> I recently noticed that a bunch of commands started hanging for almost a\n> second after they're essentially done already. As it turns out, this was\n> caused by auto-maintenance, which was blocking in the foreground for\n> ~800ms.\n>\n> The root cause of this is that the auto-condition of \"reflog-expire\"\n> doesn't properly evaluate reachability of reflog entries, so it instead\n> considered every single reflog entry to point to an unreachable commit.\n> Hence, all entries were evaluated against \"gc.reflogExpireUnreachable\"\n> instead of \"gc.reflogExpire\", and thus the auto-condition significantly\n> overcounts the number of entries that we'd prune. So every single run of\n> git-maintenance(1) ultimately ended up spawning `git reflog expire`.\n>\n> This is fixed by the last patch in this series.\n\nWell, the last one is a doc change, but I get what you mean.\n\n> But I was a bit\n> surprised that a no-op `git reflog expire` took this long, so the first\n> two patches optimize the reftable backend to be more efficient, too.\n> In my copy of \"git.git\":\n>\n>   Benchmark 1: git maintenance run --auto (commit = master) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    853.2 ms  ±    99.9 ms     570.9 ms  …   901.9 ms\n>     Memory       117.5 MiB ±     0.3 MiB    117.2 MiB …   118.2 MiB\n>\n>   Benchmark 2: git maintenance run --auto (commit = reftable-optimizations) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    392.3 ms  ±     2.3 ms     388.0 ms  …   396.6 ms           -54.0%  (2.2x faster)\n>     Memory       117.2 MiB ±     0.2 MiB    116.8 MiB …   117.4 MiB           -0.2%\n>\n>   Benchmark 3: git maintenance run --auto (commit = auto-condition-optimizations) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time     18.6 ms  ±     1.1 ms      17.4 ms  …    20.9 ms           -97.8%  (45.8x faster)\n>     Memory        81.2 MiB ±     0.1 MiB     80.9 MiB …    81.4 MiB          -30.9%  (1.4x less)\n>\n> So yeah, it's a bit more efficient now. :)\n\nGreat work. I only had one remark on consistency, but not really worth a\nreroll in itself if you ask me.\n\n-- \nLaters,\nToon\n\n"},{"id":"558724","messageId":"xmqq1p8i84ma.fsf@gitster.g","threadId":"66869","inReplyTo":"20261118-pks-reflog-expire-performance-v1-1-10b9263566a8@pks.im","subject":"Re: [PATCH 1/4] refs/reftable: don't write new table on dry-run expire","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-18T16:51:41Z","receivedAt":"2026-11-18T16:51:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The `git reflog expire` subcommand supports a dry-run flag. When set, we\n> print what would be pruned without actually pruning anything on disk.\n>\n> But even though the expectation is that the on-disk state won't change,\n> the reftable backend still ends up writing a new reftable to disk that\n> contains the changes. This is of course rather pointless, as we would\n> have already printed the entries that we're about to prune at that point\n> in time.\n\nI hope I am right in assuming that the current code already has\nprovisions to discard (or \"not to use\") this data written to the\ndisk.  If so, that means --dry-run is working correctly, and this\nchange is a performance-only improvement.  Is that the untold piece\nof information here?\n\n> Skip writing the data, which provides a small speedup:\n>\n>   Benchmark 1: git reflog expire --all --dry-run (commit = HEAD~) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    414.2 ms  ±     2.2 ms     411.8 ms  …   418.0 ms\n>     Memory       116.9 MiB ±     0.1 MiB    116.6 MiB …   117.0 MiB\n>\n>   Benchmark 2: git reflog expire --all --dry-run (commit = HEAD) (10 runs)\n>                   mean     ±       σ          min     …     max\n>     Wall Time    369.1 ms  ±     3.2 ms     361.9 ms  …   371.9 ms           -10.9%\n>     Memory       116.8 MiB ±     0.1 MiB    116.7 MiB …   117.0 MiB           -0.1%\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  refs/reftable-backend.c | 8 ++++++--\n>  1 file changed, 6 insertions(+), 2 deletions(-)\n>\n> diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> index 3ba6a9ac0db..4216a18684d 100644\n> --- a/refs/reftable-backend.c\n> +++ b/refs/reftable-backend.c\n> @@ -2706,6 +2706,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \tif (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid))\n>  \t\toidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo);\n>  \n> +\tif ((flags & EXPIRE_REFLOGS_DRY_RUN)) {\n> +\t\tret = 0;\n> +\t\tgoto done;\n> +\t}\n> +\n>  \targ.refs = refs;\n>  \targ.records = rewritten;\n>  \targ.len = logs_nr;\n> @@ -2720,8 +2725,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n>  \t * Future improvement: we could skip writing records that were\n>  \t * not changed.\n>  \t */\n> -\tif (!(flags & EXPIRE_REFLOGS_DRY_RUN))\n> -\t\tret = reftable_addition_commit(add);\n> +\tret = reftable_addition_commit(add);\n>  \n>  done:\n>  \tif (add)\n\n"},{"id":"558835","messageId":"av7kRjkcrmW_FtQK@pks.im","threadId":"66869","inReplyTo":"87jymajgqw.fsf@dev.null.iotcl.com.invalid","subject":"Re: [PATCH 2/4] refs/reftable: don't write new table when expire is a no-op","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-19T11:57:58Z","receivedAt":"2026-11-19T11:57:58Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 18, 2026 at 04:34:15PM +0100, Toon Claes wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> > diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c\n> > index 4216a18684d..e2027bb9c02 100644\n> > --- a/refs/reftable-backend.c\n> > +++ b/refs/reftable-backend.c\n> > @@ -2696,17 +2697,25 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store,\n> >  \t\t\t\t    logs[i].value.update.message,\n> >  \t\t\t\t    policy_cb_data)) {\n> >  \t\t\tdest->value_type = REFTABLE_LOG_DELETION;\n> > +\t\t\thas_changes = true;\n> >  \t\t} else {\n> > -\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash)\n> > +\t\t\tif ((flags & EXPIRE_REFLOGS_REWRITE) && last_hash &&\n> > +\t\t\t    memcmp(dest->value.update.old_hash, last_hash,\n> > +\t\t\t\t   ref_store->repo->hash_algo->rawsz)) {\n> >  \t\t\t\tmemcpy(dest->value.update.old_hash, last_hash, GIT_MAX_RAWSZ);\n> \n> I cannot help being bothered by using GIT_MAX_RAWSZ here, and\n> ref_store->repo->hash_algo->rawsz the line above. Would it make sense to\n> be consistent?\n\nFair point, will fix.\n\nPatrick\n\n"},{"id":"558836","messageId":"av7kbFW33jKrm7xb@pks.im","threadId":"66869","inReplyTo":"87h5hejfw2.fsf@dev.null.iotcl.com.invalid","subject":"Re: [PATCH 3/4] builtin/maintenance: only consider `gc.reflogExpire` for \"--auto\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-19T11:58:36Z","receivedAt":"2026-11-19T11:58:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 18, 2026 at 04:52:45PM +0100, Toon Claes wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > The \"reflog-expire\" maintenance task is responsible for expiring reflog\n> > entries that are older than either \"gc.reflogExpire\" for entries that\n> > are still reachable, or \"gc.reflogExpireUnreachable\" for those that are\n> > not reachable.\n> >\n> > When executed with \"--auto\", the condition that we use to trigger this\n> > task verifies whether the number of reflog entries that we would prune\n> > is larger than \"maintenance.reflog-expire.auto\". This is evaluated by\n> > walking the \"HEAD\" reflog, where we then check every entry by calling\n> > `should_expire_reflog_ent()`. That function will then check whether the\n> > reflog entry refers to a commit that is reachable or not, and depending\n> > on the outcome compares against either \"gc.reflogExpire\", or against\n> > both \"gc.reflogExpire\" and \"gc.reflogExpireUnreachable\".\n> >\n> > To decide reachability the caller has to call `reflog_expiry_prepare()`,\n> > which amongst other things also performs a reachability walk for a given\n> > refname. But we don't do that in the \"reflog-expire\" maintenance task,\n> > and hence every single reflog entry will be considered unreachable and\n> > thus we'd consider many reachable entries prunable even though they\n> > won't be pruned. So we overcount the number of prunable entries quite a\n> > bit and will thus trigger `git reflog expire` even though it won't do\n> > anything.\n> >\n> > Now we could easiliy fix that by calling `reflog_expiry_prepare()`.\n> \n> Do you mean \"not calling\"?\n> \n> Anyhow, from reading the diff and the surrounding code, it's not clear\n> to me where this is/was called?\n\nIt isn't called, and that's exactly the problem because we need to call\nit to compute reachability. So we could fix this by starting to call\nthe function, but we instead pick a different route to fix this issue.\n\nPatrick\n\n"},{"id":"558837","messageId":"av7kgOuPOugFsCbr@pks.im","threadId":"66869","inReplyTo":"87bj7mjfp2.fsf@dev.null.iotcl.com.invalid","subject":"Re: [PATCH 0/4] Improve performance of reflog expiration","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-19T11:58:56Z","receivedAt":"2026-11-19T11:58:56Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 18, 2026 at 04:56:57PM +0100, Toon Claes wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > Hi,\n> >\n> > I recently noticed that a bunch of commands started hanging for almost a\n> > second after they're essentially done already. As it turns out, this was\n> > caused by auto-maintenance, which was blocking in the foreground for\n> > ~800ms.\n> >\n> > The root cause of this is that the auto-condition of \"reflog-expire\"\n> > doesn't properly evaluate reachability of reflog entries, so it instead\n> > considered every single reflog entry to point to an unreachable commit.\n> > Hence, all entries were evaluated against \"gc.reflogExpireUnreachable\"\n> > instead of \"gc.reflogExpire\", and thus the auto-condition significantly\n> > overcounts the number of entries that we'd prune. So every single run of\n> > git-maintenance(1) ultimately ended up spawning `git reflog expire`.\n> >\n> > This is fixed by the last patch in this series.\n> \n> Well, the last one is a doc change, but I get what you mean.\n\nOops, true, this one is stale. I only added that last commit after\nhaving written the cover letter.\n\nPatrick\n\n"},{"id":"558838","messageId":"av7ksRi3bQtx3pGf@pks.im","threadId":"66869","inReplyTo":"xmqq1p8i84ma.fsf@gitster.g","subject":"Re: [PATCH 1/4] refs/reftable: don't write new table on dry-run expire","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-19T11:59:45Z","receivedAt":"2026-11-19T11:59:45Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Nov 18, 2026 at 08:51:41AM -0800, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > The `git reflog expire` subcommand supports a dry-run flag. When set, we\n> > print what would be pruned without actually pruning anything on disk.\n> >\n> > But even though the expectation is that the on-disk state won't change,\n> > the reftable backend still ends up writing a new reftable to disk that\n> > contains the changes. This is of course rather pointless, as we would\n> > have already printed the entries that we're about to prune at that point\n> > in time.\n> \n> I hope I am right in assuming that the current code already has\n> provisions to discard (or \"not to use\") this data written to the\n> disk.  If so, that means --dry-run is working correctly, and this\n> change is a performance-only improvement.  Is that the untold piece\n> of information here?\n\nYes. We write everything to disk, but instead of committing we then call\n`reftable_addition_destroy()`, which calls `reftable_addition_close()`,\nand that function in turn unlinks the table from disk again.\n\nI'll adapt the commit message to make that explicit.\n\nPatrick\n\n"}]}