[PATCH 3/4] builtin/maintenance: only consider `gc.reflogExpire` for "--auto"
The "reflog-expire" maintenance task is responsible for expiring reflog entries that are older than either "gc.reflogExpire" for entries that are still reachable, or "gc.reflogExpireUnreachable" for those that are not reachable.
When executed with "--auto", the condition that we use to trigger this task verifies whether the number of reflog entries that we would prune is larger than "maintenance.reflog-expire.auto". This is evaluated by walking the "HEAD" reflog, where we then check every entry by calling `should_expire_reflog_ent()`. That function will then check whether the reflog entry refers to a commit that is reachable or not, and depending on the outcome compares against either "gc.reflogExpire", or against both "gc.reflogExpire" and "gc.reflogExpireUnreachable".
To decide reachability the caller has to call `reflog_expiry_prepare()`, which amongst other things also performs a reachability walk for a given refname. But we don't do that in the "reflog-expire" maintenance task, and hence every single reflog entry will be considered unreachable and thus we'd consider many reachable entries prunable even though they won't be pruned. So we overcount the number of prunable entries quite a bit and will thus trigger `git reflog expire` even though it won't do anything.
Now we could easiliy fix that by calling `reflog_expiry_prepare()`. But in the case of auto-maintenance it has the big downside that we now have to perform a reachability walk in the foreground, and that may be quite expensive. The whole point of those auto-conditions is to use heuristics that are fast to evaluate though so that the time we have to block is reduced as much as possible.
Instead, adapt the heuristic to only evaluate reflog entries against "gc.reflogExpire". This of course means that we're now undercounting the number of prunable entries. But as part of background maintenance that feels like a reasonable tradeoff, and a user that really wants to prune unreachable reflog entries can either execute git-maintenance(1) without "--auto", or they can run `git reflog expire` directly.
In any case, the consequence is that we only spawn git-reflog(1) now in those cases where it would indeed prune entries. And in my own working copy of "git.git" that I use daily that results in quite a performance improvement:
Benchmark 1: git maintenance run --auto (commit = HEAD~) (10 runs)
mean ± σ min … max
Wall Time 385.2 ms ± 1.9 ms 382.7 ms … 388.6 ms
Memory 116.8 MiB ± 0.1 MiB 116.7 MiB … 117.0 MiB Benchmark 2: git maintenance run --auto (commit = HEAD) (10 runs)
mean ± σ min … max
Wall Time 17.9 ms ± 0.7 ms 17.2 ms … 19.6 ms -95.4% (21.5x faster)
Memory 81.0 MiB ± 0.1 MiB 80.7 MiB … 81.2 MiB -30.7% (1.4x less)Note that this improvement can be observed after every single command that triggers auto-maintenance, so this is overall quite an important optimization.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/gc.c | 33 +++++++++++++++++++++------------
1 file changed, 21 insertions(+), 12 deletions(-)
Show changes to builtin/gc.c +21 −12
diff --git a/builtin/gc.c b/builtin/gc.c
index 68cb04a672d..d4a1f337a29 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -282,19 +282,31 @@ static int maintenance_task_pack_refs(struct maintenance_run_opts *opts,
}
struct count_reflog_entries_data {
- struct expire_reflog_policy_cb policy;
+ struct reflog_expire_options opts;
size_t count;
size_t limit;
};
static int count_reflog_entries(const char *refname UNUSED,
- struct object_id *old_oid, struct object_id *new_oid,
- const char *committer, timestamp_t timestamp,
- int tz, const char *msg, void *cb_data)
+ struct object_id *old_oid UNUSED,
+ struct object_id *new_oid UNUSED,
+ const char *committer UNUSED,
+ timestamp_t timestamp,
+ int tz UNUSED,
+ const char *msg UNUSED,
+ void *cb_data)
{
struct count_reflog_entries_data *data = cb_data;
- if (should_expire_reflog_ent(old_oid, new_oid, committer, timestamp, tz, msg, &data->policy))
+
+ /*
+ * We only evaluate whether the entry is older than `gc.reflogExpire`
+ * and ignore `gc.reflogExpireUnreachable`. Evaluating the latter would
+ * require a reachability walk, and that's simply too expensive in this
+ * context.
+ */
+ if (timestamp < data->opts.expire_total)
data->count++;
+
return data->count >= data->limit;
}
@@ -302,9 +314,7 @@ static int reflog_expire_condition(struct gc_config *cfg UNUSED)
{
timestamp_t now = time(NULL);
struct count_reflog_entries_data data = {
- .policy = {
- .opts = REFLOG_EXPIRE_OPTIONS_INIT(now),
- },
+ .opts = REFLOG_EXPIRE_OPTIONS_INIT(now),
};
int limit = 100;
@@ -315,14 +325,13 @@ static int reflog_expire_condition(struct gc_config *cfg UNUSED)
return 1;
data.limit = limit;
- repo_config(the_repository, reflog_expire_config, &data.policy.opts);
+ repo_config(the_repository, reflog_expire_config, &data.opts);
- reflog_expire_options_set_refname(&data.policy.opts, "HEAD");
+ reflog_expire_options_set_refname(&data.opts, "HEAD");
refs_for_each_reflog_ent(get_main_ref_store(the_repository), "HEAD",
count_reflog_entries, &data);
- reflog_expiry_cleanup(&data.policy);
- reflog_clear_expire_config(&data.policy.opts);
+ reflog_clear_expire_config(&data.opts);
return data.count >= data.limit;
}
--
2.56.1.831.g720f3f5937.dirty