[PATCH v3 00/10] builtin/maintenance: introduce "geometric" strategy
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Oct 24, 2025, 06:57 UTC
- Message-ID
- <20251024-pks-maintenance-geometric-strategy-v3-0-9b5b3bdb4387@pks.im>
- In-Reply-To
- <20251016-pks-maintenance-geometric-strategy-v1-0-18943d474203@pks.im>
Hi,
by default, git-maintenance(1) uses git-gc(1) to perform repository housekeeping. This tool has a couple of shortcomings, most importantly that it regularly does all-into-one repacks. This doesn't really work all that well in the context of monorepos, where you really want to avoid repacking all objects regularly.
An alternative maintenance strategy is the "incremental" strategy, but this strategy has two downsides:
- Strategies in general only apply to scheduled maintenance. So if you
run git-maintenance(1), you still end up with git-gc(1). - The strategy is designed to not ever delete any data, but a full
replacment for git-gc(1) needs to also prune reflogs, rereree caches
and vanished worktrees.This patch series aims to fix both of these issues.
First, the series introduces a new "geometric" maintenance task, which makes use of geometric repacking as exposed by git-repack(1) in the general case. In the case where a geometric repack ends up merging all packfiles into one we instead do an all-into-one repack with cruft packs so that we can still phase out objects over time.
Second, the series extends maintenance strategies to also cover normal maintenance. If the user has configured the "geometric" strategy, we'll thus use it for both manual and scheduled maintenance. For backwards compatibility, the "incremental" strategy is changed so that it uses git-gc(1) for manual maintenance and the other tasks for scheduled maintenance.
The series is built on top of b660e2dcb9 (Sync with 'maint', 2025-10-14) with tb/incremental-midx-part-3.1 at c886af90f8 (SQUASH??? play well with other topics by preemptively including "repository.h", 2025-09-29) merged into it.
Changes in v3:
- More line wrapping.
- Improve readability of maintenance strategies by using nested
designated initializers.
- Use git-count-object(1) to count loose objects.
- Link to v2: https://lore.kernel.org/r/20251021-pks-maintenance-geometric-strategy-v2-0-f0d727832b80@pks.imChanges in v2:
- Make the geometric factor configurable via
"maintenance.geometric-repack.splitFactor".
- Wrap some overly long lines in our tests.
- Link to v1: https://lore.kernel.org/r/20251016-pks-maintenance-geometric-strategy-v1-0-18943d474203@pks.imThanks!
Patrick
---
Patrick Steinhardt (10):
builtin/gc: remove global `repack` variable
builtin/gc: make `too_many_loose_objects()` reusable without GC config
builtin/maintenance: introduce "geometric-repack" task
builtin/maintenance: make the geometric factor configurable
builtin/maintenance: don't silently ignore invalid strategy
builtin/maintenance: improve readability of strategies
builtin/maintenance: run maintenance tasks depending on type
builtin/maintenance: extend "maintenance.strategy" to manual maintenance
builtin/maintenance: make "gc" strategy accessible
builtin/maintenance: introduce "geometric" strategyDocumentation/config/maintenance.adoc | 49 +++++- builtin/gc.c | 313 ++++++++++++++++++++++++++++------ t/t7900-maintenance.sh | 245 ++++++++++++++++++++++++++ 3 files changed, 544 insertions(+), 63 deletions(-)
Range-diff versus v2:
1: b853ba54dca = 1: c35408a33d0 builtin/gc: remove global `repack` variable
2: 9bbdfe1b9e5 = 2: be572fe1542 builtin/gc: make `too_many_loose_objects()` reusable without GC config
3: bcd82ad038e ! 3: 5290f6d3e0f builtin/maintenance: introduce "geometric-repack" task
@@ t/t7900-maintenance.sh: test_expect_success 'maintenance.incremental-repack.auto
+ test_line_count = "$EXPECTED_PACKS" packfiles &&
+
+ # And verify that there are no loose objects anymore.
-+ cat >expect <<-\EOF &&
-+ info
-+ pack
-+ EOF
-+ ls .git/objects >actual &&
-+ test_cmp expect actual
++ git count-objects -v >count &&
++ test_grep '^count: 0$' count
+}
+
+test_expect_success 'geometric repacking task' '
@@ t/t7900-maintenance.sh: test_expect_success 'maintenance.incremental-repack.auto
+ # The initial repack causes an all-into-one repack.
+ GIT_TRACE2_EVENT="$(pwd)/initial-repack.txt" \
+ git maintenance run --task=geometric-repack 2>/dev/null &&
-+ test_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago --quiet --write-midx <initial-repack.txt &&
++ test_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \
++ --quiet --write-midx <initial-repack.txt &&
+
+ # Repacking should now cause a no-op geometric repack because
+ # no packfiles need to be combined.
@@ t/t7900-maintenance.sh: test_expect_success 'maintenance.incremental-repack.auto
+ # an all-into-one-repack.
+ GIT_TRACE2_EVENT="$(pwd)/all-into-one-repack.txt" \
+ git maintenance run --task=geometric-repack 2>/dev/null &&
-+ test_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago --quiet --write-midx <all-into-one-repack.txt &&
++ test_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \
++ --quiet --write-midx <all-into-one-repack.txt &&
+
+ # The geometric repack soaks up unreachable objects.
+ echo blob-1 | git hash-object -w --stdin -t blob &&
@@ t/t7900-maintenance.sh: test_expect_success 'maintenance.incremental-repack.auto
+ run_and_verify_geometric_pack 3 &&
+ GIT_TRACE2_EVENT="$(pwd)/cruft-repack.txt" \
+ git maintenance run --task=geometric-repack 2>/dev/null &&
-+ test_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago --quiet --write-midx <cruft-repack.txt &&
++ test_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \
++ --quiet --write-midx <cruft-repack.txt &&
+ ls .git/objects/pack/*.pack >packs &&
+ test_line_count = 2 packs &&
+ ls .git/objects/pack/*.mtimes >cruft &&
4: cb10031cc7c = 4: 7f2067fa4ec builtin/maintenance: make the geometric factor configurable
5: 7e8f83d4753 = 5: 7a76003215e builtin/maintenance: don't silently ignore invalid strategy
-: ----------- > 6: a6383d121b2 builtin/maintenance: improve readability of strategies
6: 4217c37c0bf ! 7: e25c878a3ff builtin/maintenance: run maintenance tasks depending on type
@@ builtin/gc.c: static int maintenance_run_tasks(struct maintenance_run_opts *opts
enum schedule_priority schedule;
} tasks[TASK__COUNT];
};
-
- static const struct maintenance_strategy none_strategy = { 0 };
-+
+@@ builtin/gc.c: static const struct maintenance_strategy none_strategy = { 0 };
static const struct maintenance_strategy default_strategy = {
.tasks = {
-- [TASK_GC].enabled = 1,
-+ [TASK_GC].type = MAINTENANCE_TYPE_MANUAL,
+ [TASK_GC] = {
+- .enabled = 1,
++ .type = MAINTENANCE_TYPE_MANUAL,
+ },
},
};
-+
+@@ builtin/gc.c: static const struct maintenance_strategy default_strategy = {
static const struct maintenance_strategy incremental_strategy = {
.tasks = {
-- [TASK_COMMIT_GRAPH].enabled = 1,
-+ [TASK_COMMIT_GRAPH].type = MAINTENANCE_TYPE_SCHEDULED,
- [TASK_COMMIT_GRAPH].schedule = SCHEDULE_HOURLY,
-- [TASK_PREFETCH].enabled = 1,
-+ [TASK_PREFETCH].type = MAINTENANCE_TYPE_SCHEDULED,
- [TASK_PREFETCH].schedule = SCHEDULE_HOURLY,
-- [TASK_INCREMENTAL_REPACK].enabled = 1,
-+ [TASK_INCREMENTAL_REPACK].type = MAINTENANCE_TYPE_SCHEDULED,
- [TASK_INCREMENTAL_REPACK].schedule = SCHEDULE_DAILY,
-- [TASK_LOOSE_OBJECTS].enabled = 1,
-+ [TASK_LOOSE_OBJECTS].type = MAINTENANCE_TYPE_SCHEDULED,
- [TASK_LOOSE_OBJECTS].schedule = SCHEDULE_DAILY,
-- [TASK_PACK_REFS].enabled = 1,
-+ [TASK_PACK_REFS].type = MAINTENANCE_TYPE_SCHEDULED,
- [TASK_PACK_REFS].schedule = SCHEDULE_WEEKLY,
+ [TASK_COMMIT_GRAPH] = {
+- .enabled = 1,
++ .type = MAINTENANCE_TYPE_SCHEDULED,
+ .schedule = SCHEDULE_HOURLY,
+ },
+ [TASK_PREFETCH] = {
+- .enabled = 1,
++ .type = MAINTENANCE_TYPE_SCHEDULED,
+ .schedule = SCHEDULE_HOURLY,
+ },
+ [TASK_INCREMENTAL_REPACK] = {
+- .enabled = 1,
++ .type = MAINTENANCE_TYPE_SCHEDULED,
+ .schedule = SCHEDULE_DAILY,
+ },
+ [TASK_LOOSE_OBJECTS] = {
+- .enabled = 1,
++ .type = MAINTENANCE_TYPE_SCHEDULED,
+ .schedule = SCHEDULE_DAILY,
+ },
+ [TASK_PACK_REFS] = {
+- .enabled = 1,
++ .type = MAINTENANCE_TYPE_SCHEDULED,
+ .schedule = SCHEDULE_WEEKLY,
+ },
},
- };
@@ builtin/gc.c: static void initialize_task_config(struct maintenance_run_opts *opts,
{
struct strbuf config_name = STRBUF_INIT;
7: 422b16a62a2 ! 8: ba147c3bf33 builtin/maintenance: extend "maintenance.strategy" to manual maintenance
@@ Documentation/config/maintenance.adoc: detach.
## builtin/gc.c ##
@@ builtin/gc.c: static const struct maintenance_strategy incremental_strategy = {
- [TASK_LOOSE_OBJECTS].schedule = SCHEDULE_DAILY,
- [TASK_PACK_REFS].type = MAINTENANCE_TYPE_SCHEDULED,
- [TASK_PACK_REFS].schedule = SCHEDULE_WEEKLY,
-+
+ .type = MAINTENANCE_TYPE_SCHEDULED,
+ .schedule = SCHEDULE_WEEKLY,
+ },
+ /*
+ * Historically, the "incremental" strategy was only available
+ * in the context of scheduled maintenance when set up via
@@ builtin/gc.c: static const struct maintenance_strategy incremental_strategy = {
+ * requested. This is the same as the default strategy, which
+ * would have been in use beforehand.
+ */
-+ [TASK_GC].type = MAINTENANCE_TYPE_MANUAL,
++ [TASK_GC] = {
++ .type = MAINTENANCE_TYPE_MANUAL,
++ },
},
};
8: 07f5b32a22e ! 9: eebfab4acda builtin/maintenance: make "gc" strategy accessible
@@ builtin/gc.c: struct maintenance_strategy {
-static const struct maintenance_strategy default_strategy = {
+static const struct maintenance_strategy gc_strategy = {
.tasks = {
-- [TASK_GC].type = MAINTENANCE_TYPE_MANUAL,
-+ [TASK_GC].type = MAINTENANCE_TYPE_MANUAL | MAINTENANCE_TYPE_SCHEDULED,
-+ [TASK_GC].schedule = SCHEDULE_DAILY,
+ [TASK_GC] = {
+- .type = MAINTENANCE_TYPE_MANUAL,
++ .type = MAINTENANCE_TYPE_MANUAL | MAINTENANCE_TYPE_SCHEDULED,
++ .schedule = SCHEDULE_DAILY,
+ },
},
};
-
@@ builtin/gc.c: static struct maintenance_strategy parse_maintenance_strategy(const char *name)
{
if (!strcasecmp(name, "incremental"))
9: c597ae7f94a ! 10: 936358736f3 builtin/maintenance: introduce "geometric" strategy
@@ builtin/gc.c: static const struct maintenance_strategy incremental_strategy = {
+static const struct maintenance_strategy geometric_strategy = {
+ .tasks = {
-+ [TASK_COMMIT_GRAPH].type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
-+ [TASK_COMMIT_GRAPH].schedule = SCHEDULE_HOURLY,
-+ [TASK_GEOMETRIC_REPACK].type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
-+ [TASK_GEOMETRIC_REPACK].schedule = SCHEDULE_DAILY,
-+ [TASK_PACK_REFS].type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
-+ [TASK_PACK_REFS].schedule = SCHEDULE_DAILY,
-+ [TASK_RERERE_GC].type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
-+ [TASK_RERERE_GC].schedule = SCHEDULE_WEEKLY,
-+ [TASK_REFLOG_EXPIRE].type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
-+ [TASK_REFLOG_EXPIRE].schedule = SCHEDULE_WEEKLY,
-+ [TASK_WORKTREE_PRUNE].type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
-+ [TASK_WORKTREE_PRUNE].schedule = SCHEDULE_WEEKLY,
++ [TASK_COMMIT_GRAPH] = {
++ .type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
++ .schedule = SCHEDULE_HOURLY,
++ },
++ [TASK_GEOMETRIC_REPACK] = {
++ .type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
++ .schedule = SCHEDULE_DAILY,
++ },
++ [TASK_PACK_REFS] = {
++ .type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
++ .schedule = SCHEDULE_DAILY,
++ },
++ [TASK_RERERE_GC] = {
++ .type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
++ .schedule = SCHEDULE_WEEKLY,
++ },
++ [TASK_REFLOG_EXPIRE] = {
++ .type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
++ .schedule = SCHEDULE_WEEKLY,
++ },
++ [TASK_WORKTREE_PRUNE] = {
++ .type = MAINTENANCE_TYPE_SCHEDULED | MAINTENANCE_TYPE_MANUAL,
++ .schedule = SCHEDULE_WEEKLY,
++ },
+ },
+};
+--- base-commit: 0bb2c786c2349dd6700727153c13d81cbfb41710 change-id: 20251015-pks-maintenance-geometric-strategy-580c58581b01