{"thread":{"id":"65942","subject":"[PATCH 00/11] odb: make optimizations pluggable","startedAt":"2026-07-07T15:32:57Z","lastAt":"2026-07-26T16:25:36Z","messageCount":33,"participants":["Patrick Steinhardt","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":11},"messages":[{"id":"547341","messageId":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","threadId":"65942","inReplyTo":null,"subject":"[PATCH 00/11] odb: make optimizations pluggable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:32Z","receivedAt":"2026-07-07T15:32:57Z","isPatch":true,"body":"Hi,\n\nthis patch series converts object housekeeping to become pluggable.\nThere isn't really anything else to say about this.\n\nThe series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n2026-07-06).\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (11):\n odb: run \"pre-auto-gc\" hook for all maintenance tasks\n builtin/gc: move worktree and rerere tasks before object optimizations\n builtin/gc: extract object database optimizations into separate function\n builtin/gc: make repack arguments self-contained\n builtin/gc: inline config values specific to the \"files\" backend\n builtin/gc: introduce object database optimization options\n builtin/gc: move geometric repacking into `odb_optimize()`\n builtin/gc: introduce `odb_optimize_required()`\n builtin/gc: refactor ODB optimizations to operate on \"files\" source\n builtin/gc: fix signedness issues in ODB-related functionality\n odb: make optimizations pluggable\n\n builtin/gc.c | 534 ++++++++-----------------------------------------\n odb.c | 12 ++\n odb.h | 45 +++++\n odb/source-files.c | 470 +++++++++++++++++++++++++++++++++++++++++++\n odb/source-files.h | 15 ++\n odb/source.h | 36 ++++\n t/t7900-maintenance.sh | 143 ++++++++++++-\n 7 files changed, 789 insertions(+), 466 deletions(-)\n\n\n---\nbase-commit: f85a7e662054a7b0d9070e432508831afa214b47\nchange-id: 20260612-b4-pks-odb-optimize-3426c57e5c30\n\n"},{"id":"547342","messageId":"20260707-b4-pks-odb-optimize-v1-1-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 01/11] odb: run \"pre-auto-gc\" hook for all maintenance tasks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:33Z","receivedAt":"2026-07-07T15:32:58Z","isPatch":true,"body":"The \"pre-auto-gc\" hook is supposed to run before auto-maintenance\nstarts. The intent of this is to give users the ability to intercept\nrunning maintenance in case there's for example an event that is not\nsupposed to run in parallel with repository maintenance.\n\nThis hook runs via `need_to_gc()`, which is invoked via two paths:\n\n - It is called directly by git-gc(1).\n\n - It is called indirectly by git-maintenance(1) via the \"gc\" task.\n\nWhile the former makes sense, the latter is somewhat off. While the hook\nis indeed strongly tied to gc'ing a repository, the original intent of\nthe hook is rather to inhibit any kind of automated garbage collection.\nThat noticeably also includes all the other maintenance tasks that our\nnew infrastructure may run, but those aren't getting intercepted at all.\nThe move towards our new maintenance strategy has thus somewhat neutered\nthe effectiveness of the hook.\n\nFix this issue by running the hook before the first auto-maintenance\ntask that would run as determined by the tasks's auto condition. Note\nthat this requires us to lift the call to `run_hooks()` out of\n`needs_to_gc()`, as the hook would otherwise potentially run multiple\ntimes.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 35 ++++++++++----\n t/t7900-maintenance.sh | 121 +++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 147 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex d32af422af..77d0a5c948 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -709,8 +709,6 @@ static int need_to_gc(struct gc_config *cfg, struct strvec *repack_args)\n \telse\n \t\treturn 0;\n \n-\tif (run_hooks(the_repository, \"pre-auto-gc\"))\n-\t\treturn 0;\n \treturn 1;\n }\n \n@@ -933,7 +931,8 @@ int cmd_gc(int argc,\n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(&cfg, &repack_args)) {\n+\t\tif (!need_to_gc(&cfg, &repack_args) ||\n+\t\t run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -1755,11 +1754,18 @@ enum task_phase {\n \tTASK_PHASE_BACKGROUND,\n };\n \n+enum auto_gc_hook_result {\n+\tAUTO_GC_HOOK_UNDECIDED = 0,\n+\tAUTO_GC_HOOK_RUN = 1,\n+\tAUTO_GC_HOOK_SKIP = 2,\n+};\n+\n static int maybe_run_task(const struct maintenance_task *task,\n \t\t\t struct repository *repo,\n \t\t\t struct maintenance_run_opts *opts,\n \t\t\t struct gc_config *cfg,\n-\t\t\t enum task_phase phase)\n+\t\t\t enum task_phase phase,\n+\t\t\t enum auto_gc_hook_result *auto_gc_hook_result)\n {\n \tint foreground = (phase == TASK_PHASE_FOREGROUND);\n \tmaintenance_task_fn fn = foreground ? task->foreground : task->background;\n@@ -1768,9 +1774,19 @@ static int maybe_run_task(const struct maintenance_task *task,\n \n \tif (!fn)\n \t\treturn 0;\n-\tif (opts->auto_flag &&\n-\t (!task->auto_condition || !task->auto_condition(cfg)))\n-\t\treturn 0;\n+\tif (opts->auto_flag) {\n+\t\tif (*auto_gc_hook_result == AUTO_GC_HOOK_SKIP)\n+\t\t\treturn 0;\n+\n+\t\tif (!task->auto_condition || !task->auto_condition(cfg))\n+\t\t\treturn 0;\n+\n+\t\tif (*auto_gc_hook_result == AUTO_GC_HOOK_UNDECIDED)\n+\t\t\t*auto_gc_hook_result = run_hooks(repo, \"pre-auto-gc\") ?\n+\t\t\t\tAUTO_GC_HOOK_SKIP : AUTO_GC_HOOK_RUN;\n+\t\tif (*auto_gc_hook_result == AUTO_GC_HOOK_SKIP)\n+\t\t\treturn 0;\n+\t}\n \n \ttrace2_region_enter(region, task->name, repo);\n \tif (fn(opts, cfg)) {\n@@ -1789,6 +1805,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,\n \tstruct lock_file lk;\n \tstruct repository *r = the_repository;\n \tchar *lock_path = xstrfmt(\"%s/maintenance\", r->objects->sources->path);\n+\tenum auto_gc_hook_result auto_gc_hook_result = AUTO_GC_HOOK_UNDECIDED;\n \n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {\n \t\t/*\n@@ -1808,7 +1825,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,\n \n \tfor (size_t i = 0; i < opts->tasks_nr; i++)\n \t\tif (maybe_run_task(&tasks[opts->tasks[i]], r, opts, cfg,\n-\t\t\t\t TASK_PHASE_FOREGROUND))\n+\t\t\t\t TASK_PHASE_FOREGROUND, &auto_gc_hook_result))\n \t\t\tresult = 1;\n \n \t/* Failure to daemonize is ok, we'll continue in foreground. */\n@@ -1820,7 +1837,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,\n \n \tfor (size_t i = 0; i < opts->tasks_nr; i++)\n \t\tif (maybe_run_task(&tasks[opts->tasks[i]], r, opts, cfg,\n-\t\t\t\t TASK_PHASE_BACKGROUND))\n+\t\t\t\t TASK_PHASE_BACKGROUND, &auto_gc_hook_result))\n \t\t\tresult = 1;\n \n \trollback_lock_file(&lk);\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex d7f82e1bec..1212b306b6 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -740,6 +740,127 @@ test_expect_success 'geometric repacking honors configured split factor' '\n \t)\n '\n \n+test_expect_success 'pre-auto-gc hook runs exactly once' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\tEOF\n+\n+\t\t# Satisfy the auto condition for multiple tasks, both in the\n+\t\t# foreground and in the background phase.\n+\t\tgit config set maintenance.reflog-expire.auto -1 &&\n+\t\tgit config set maintenance.geometric-repack.auto -1 &&\n+\t\tgit config set maintenance.rerere-gc.auto -1 &&\n+\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run --auto 2>/dev/null &&\n+\n+\t\t# The successful hook does not inhibit any of the tasks...\n+\t\ttest_subcommand git reflog expire --all >hook.log\n+\t\texit 1\n+\t\tEOF\n+\n+\t\tgit config set maintenance.reflog-expire.auto -1 &&\n+\t\tgit config set maintenance.geometric-repack.auto -1 &&\n+\t\tgit config set maintenance.rerere-gc.auto -1 &&\n+\n+\t\t# Maintenance would be required...\n+\t\tgit maintenance is-needed --auto &&\n+\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run --auto 2>/dev/null &&\n+\n+\t\t# ... but the failing hook inhibits all tasks. The hook itself\n+\t\t# is expected to be the only child process being spawned, and\n+\t\t# it must only run a single time.\n+\t\ttest_grep \"child_start.*pre-auto-gc\" trace2.txt &&\n+\t\ttest_subcommand_flex ! git trace2 &&\n+\t\ttest_line_count = 1 hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook can inhibit gc strategy' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\texit 1\n+\t\tEOF\n+\n+\t\tgit config set maintenance.strategy gc &&\n+\t\tgit config set maintenance.auto false &&\n+\t\tgit config set gc.auto 3 &&\n+\n+\t\ttest_oid_init &&\n+\n+\t\t# We need to create two objects whose hashes start with 17\n+\t\t# since this is what the gc task counts.\n+\t\ttest_commit \"$(test_oid blob17_1)\" &&\n+\t\ttest_commit \"$(test_oid blob17_2)\" &&\n+\n+\t\t# Maintenance would be required...\n+\t\tgit maintenance is-needed --auto &&\n+\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run --auto 2>/dev/null &&\n+\n+\t\t# ... but the failing hook inhibits all tasks. The hook itself\n+\t\t# is expected to be the only child process being spawned, and\n+\t\t# it must only run a single time.\n+\t\ttest_grep \"child_start.*pre-auto-gc\" trace2.txt &&\n+\t\ttest_subcommand_flex ! git trace2 &&\n+\t\ttest_line_count = 1 hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook does not run when no maintenance is needed' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\tEOF\n+\t\ttest_must_fail git maintenance is-needed --auto &&\n+\t\tgit maintenance run --auto 2>/dev/null &&\n+\t\ttest_path_is_missing hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook does not run without --auto' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\ttest_hook -C repo pre-auto-gc <<-\\EOF &&\n+\techo hook >>hook.log\n+\tEOF\n+\t(\n+\t\tcd repo &&\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run 2>/dev/null &&\n+\t\ttest_grep \"\\[\\\"git\\\",\\\"repack\\\",\" trace2.txt &&\n+\t\ttest_path_is_missing hook.log\n+\t)\n+'\n+\n test_expect_success 'pack-refs task' '\n \tfor n in $(test_seq 1 5)\n \tdo\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547343","messageId":"20260707-b4-pks-odb-optimize-v1-2-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 02/11] builtin/gc: move worktree and rerere tasks before object optimizations","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:34Z","receivedAt":"2026-07-07T15:33:07Z","isPatch":true,"body":"In subsequent patches we'll consolidate all tasks that relate to\nmaintenance of the object database and move it into the \"files\" backend.\nThe relevant code is somewhat scattered though, as several other tasks\nare interspersed between.\n\nRefactor the code so that all object database optimizations are grouped\ntogether, which requires us to move worktree pruning and rerere garbage\ncollection around. In theory, rearranging this code can have an effect\non the object database optimizations:\n\n - Rerere entries really shouldn't impact garbage collection at all, as\n these entries are not stored in the object database.\n\n - The index and HEAD reference of pruned worktrees may reference\n objects that become unreachable.\n\nThat being said, the impact should be overall rather negligible. If the\nuser was asking us to prune objects with immediate expiration time then\nwe might now prune objects that were previously still kept alive by the\nworktree. But besides being a very specific edge case, it's arguably not\neven the wrong thing to also prune any potentially-unreachable objects\nimmediately.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 77d0a5c948..8f568003ee 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1011,6 +1011,13 @@ int cmd_gc(int argc,\n \tif (opts.detach <= 0 && !skip_foreground_tasks)\n \t\tgc_foreground_tasks(&opts, &cfg);\n \n+\tif (cfg.prune_worktrees_expire &&\n+\t maintenance_task_worktree_prune(&opts, &cfg))\n+\t\tdie(FAILED_RUN, \"worktree\");\n+\n+\tif (maintenance_task_rerere_gc(&opts, &cfg))\n+\t\tdie(FAILED_RUN, \"rerere\");\n+\n \tif (!the_repository->repository_format_precious_objects) {\n \t\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \n@@ -1038,13 +1045,6 @@ int cmd_gc(int argc,\n \t\t}\n \t}\n \n-\tif (cfg.prune_worktrees_expire &&\n-\t maintenance_task_worktree_prune(&opts, &cfg))\n-\t\tdie(FAILED_RUN, \"worktree\");\n-\n-\tif (maintenance_task_rerere_gc(&opts, &cfg))\n-\t\tdie(FAILED_RUN, \"rerere\");\n-\n \treport_garbage = report_pack_garbage;\n \todb_reprepare(the_repository->objects);\n \tif (pack_garbage.nr > 0) {\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547344","messageId":"20260707-b4-pks-odb-optimize-v1-3-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 03/11] builtin/gc: extract object database optimizations into separate function","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:35Z","receivedAt":"2026-07-07T15:33:08Z","isPatch":true,"body":"Extract the object database optimization logic from `cmd_gc()` into a\nnew `maintenance_task_odb()` helper function. This is a pure refactoring\nwith no intended functional change.\n\nNote that the message that notifies the user about too many loose\nobjects is moved into the new function, as well. It is inherently an\nimplementation detail of how the \"files\" source works, and as a\nconsequence we'll move it around in a later commit, as well. This\nreordering means that the warning may now be printed at a different\npoint in time, but it's not expected that this will have any practical\nimplications.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 79 +++++++++++++++++++++++++++++++++++++-----------------------\n 1 file changed, 49 insertions(+), 30 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 8f568003ee..2ff98fa727 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -839,6 +839,53 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \treturn 0;\n }\n \n+static int maintenance_task_odb(struct maintenance_run_opts *opts,\n+\t\t\t\tstruct gc_config *cfg,\n+\t\t\t\tstruct strvec *repack_args)\n+{\n+\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n+\tint ret;\n+\n+\tif (the_repository->repository_format_precious_objects)\n+\t\treturn 0;\n+\n+\trepack_cmd.git_cmd = 1;\n+\trepack_cmd.odb_to_close = the_repository->objects;\n+\tstrvec_pushv(&repack_cmd.args, repack_args->v);\n+\tif (run_command(&repack_cmd)) {\n+\t\tret = error(FAILED_RUN, repack_args->v[0]);\n+\t\tgoto out;\n+\t}\n+\n+\tif (cfg->prune_expire) {\n+\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n+\n+\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n+\t\t/* run `git prune` even if using cruft packs */\n+\t\tstrvec_push(&prune_cmd.args, cfg->prune_expire);\n+\t\tif (opts->quiet)\n+\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n+\t\tif (repo_has_promisor_remote(the_repository))\n+\t\t\tstrvec_push(&prune_cmd.args,\n+\t\t\t\t \"--exclude-promisor-objects\");\n+\t\tprune_cmd.git_cmd = 1;\n+\n+\t\tif (run_command(&prune_cmd)) {\n+\t\t\tret = error(FAILED_RUN, prune_cmd.args.v[0]);\n+\t\t\tgoto out;\n+\t\t}\n+\t}\n+\n+\tif (opts->auto_flag && too_many_loose_objects(cfg->gc_auto_threshold))\n+\t\twarning(_(\"There are too many unreachable loose objects; \"\n+\t\t\t\"run 'git prune' to remove them.\"));\n+\n+\tret = 0;\n+\n+out:\n+\treturn ret;\n+}\n+\n int cmd_gc(int argc,\n \t const char **argv,\n \t const char *prefix,\n@@ -1018,32 +1065,8 @@ int cmd_gc(int argc,\n \tif (maintenance_task_rerere_gc(&opts, &cfg))\n \t\tdie(FAILED_RUN, \"rerere\");\n \n-\tif (!the_repository->repository_format_precious_objects) {\n-\t\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n-\n-\t\trepack_cmd.git_cmd = 1;\n-\t\trepack_cmd.odb_to_close = the_repository->objects;\n-\t\tstrvec_pushv(&repack_cmd.args, repack_args.v);\n-\t\tif (run_command(&repack_cmd))\n-\t\t\tdie(FAILED_RUN, repack_args.v[0]);\n-\n-\t\tif (cfg.prune_expire) {\n-\t\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n-\n-\t\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n-\t\t\t/* run `git prune` even if using cruft packs */\n-\t\t\tstrvec_push(&prune_cmd.args, cfg.prune_expire);\n-\t\t\tif (opts.quiet)\n-\t\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n-\t\t\tif (repo_has_promisor_remote(the_repository))\n-\t\t\t\tstrvec_push(&prune_cmd.args,\n-\t\t\t\t\t \"--exclude-promisor-objects\");\n-\t\t\tprune_cmd.git_cmd = 1;\n-\n-\t\t\tif (run_command(&prune_cmd))\n-\t\t\t\tdie(FAILED_RUN, prune_cmd.args.v[0]);\n-\t\t}\n-\t}\n+\tif (maintenance_task_odb(&opts, &cfg, &repack_args))\n+\t\tdie(NULL);\n \n \treport_garbage = report_pack_garbage;\n \todb_reprepare(the_repository->objects);\n@@ -1057,10 +1080,6 @@ int cmd_gc(int argc,\n \t\t\t\t\t !opts.quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0,\n \t\t\t\t\t NULL);\n \n-\tif (opts.auto_flag && too_many_loose_objects(cfg.gc_auto_threshold))\n-\t\twarning(_(\"There are too many unreachable loose objects; \"\n-\t\t\t\"run 'git prune' to remove them.\"));\n-\n \tif (!daemonized) {\n \t\tchar *path = repo_git_path(the_repository, \"gc.log\");\n \t\tunlink(path);\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547346","messageId":"20260707-b4-pks-odb-optimize-v1-4-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 04/11] builtin/gc: make repack arguments self-contained","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:36Z","receivedAt":"2026-07-07T15:33:08Z","isPatch":true,"body":"When optimizing the object database most of the heavy-lifting is done by\ngit-repack(1). The arguments we pass to this function are assembled in\nglobal scope, which is hard to follow.\n\nRefactor the logic by moving the vector into `maintenance_task_odb()`.\nWhile that means we have to pass more arguments to this function, it has\nthe upside that the logic becomes self-contained without any kind of\nglobal interdependencies.\n\nThis is a pure refactoring with no intended functional change.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 156 ++++++++++++++++++++++++++++-------------------------------\n 1 file changed, 75 insertions(+), 81 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 2ff98fa727..25a59caea6 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -661,7 +661,7 @@ static void add_repack_incremental_option(struct strvec *args)\n \tstrvec_push(args, \"--no-write-bitmap-index\");\n }\n \n-static int need_to_gc(struct gc_config *cfg, struct strvec *repack_args)\n+static int need_to_gc(struct gc_config *cfg)\n {\n \t/*\n \t * Setting gc.auto to 0 or negative can disable the\n@@ -669,46 +669,8 @@ static int need_to_gc(struct gc_config *cfg, struct strvec *repack_args)\n \t */\n \tif (cfg->gc_auto_threshold <= 0)\n \t\treturn 0;\n-\n-\t/*\n-\t * If there are too many loose objects, but not too many\n-\t * packs, we run \"repack -d -l\". If there are too many packs,\n-\t * we run \"repack -A -d -l\". Otherwise we tell the caller\n-\t * there is no need.\n-\t */\n-\tif (too_many_packs(cfg)) {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (cfg->big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n-\t\t\tif (keep_pack.nr >= cfg->gc_auto_pack_limit) {\n-\t\t\t\tcfg->big_pack_threshold = 0;\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t\t}\n-\t\t} else {\n-\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n-\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\tmem_have = total_ram();\n-\t\t\tmem_want = estimate_repack_memory(cfg, p);\n-\n-\t\t\t/*\n-\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t * the rest for the OS and other processes in the\n-\t\t\t * system.\n-\t\t\t */\n-\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t}\n-\n-\t\tadd_repack_all_option(cfg, &keep_pack, repack_args);\n-\t\tstring_list_clear(&keep_pack, 0);\n-\t} else if (too_many_loose_objects(cfg->gc_auto_threshold))\n-\t\tadd_repack_incremental_option(repack_args);\n-\telse\n+\tif (!too_many_packs(cfg) && !too_many_loose_objects(cfg->gc_auto_threshold))\n \t\treturn 0;\n-\n \treturn 1;\n }\n \n@@ -841,7 +803,8 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \n static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tstruct gc_config *cfg,\n-\t\t\t\tstruct strvec *repack_args)\n+\t\t\t\tint keep_largest_pack,\n+\t\t\t\tint aggressive)\n {\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \tint ret;\n@@ -851,9 +814,75 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \n \trepack_cmd.git_cmd = 1;\n \trepack_cmd.odb_to_close = the_repository->objects;\n-\tstrvec_pushv(&repack_cmd.args, repack_args->v);\n+\n+\tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n+\tif (aggressive) {\n+\t\tstrvec_push(&repack_cmd.args, \"-f\");\n+\t\tif (cfg->aggressive_depth > 0)\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", cfg->aggressive_depth);\n+\t\tif (cfg->aggressive_window > 0)\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", cfg->aggressive_window);\n+\t}\n+\tif (opts->quiet)\n+\t\tstrvec_push(&repack_cmd.args, \"-q\");\n+\n+\t/*\n+\t * There's three cases we need to consider:\n+\t *\n+\t * - If we're invoked without `--auto` we'll need to perform a full\n+\t * repack.\n+\t *\n+\t * - If we're invoked with `--auto` and there's too many packs, then\n+\t * we perform a full repack, as well.\n+\t *\n+\t * - Otherwise we perform an incremental repack.\n+\t */\n+\tif (!opts->auto_flag) {\n+\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\tif (keep_largest_pack != -1) {\n+\t\t\tif (keep_largest_pack)\n+\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t} else if (cfg->big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n+\t\t}\n+\n+\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n+\t\tstring_list_clear(&keep_pack, 0);\n+\t} else if (too_many_packs(cfg)) {\n+\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\tif (cfg->big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n+\t\t\tif (keep_pack.nr >= cfg->gc_auto_pack_limit) {\n+\t\t\t\tcfg->big_pack_threshold = 0;\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\tmem_have = total_ram();\n+\t\t\tmem_want = estimate_repack_memory(cfg, p);\n+\n+\t\t\t/*\n+\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t * the rest for the OS and other processes in the\n+\t\t\t * system.\n+\t\t\t */\n+\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t}\n+\n+\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n+\t\tstring_list_clear(&keep_pack, 0);\n+\t} else {\n+\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t}\n+\n \tif (run_command(&repack_cmd)) {\n-\t\tret = error(FAILED_RUN, repack_args->v[0]);\n+\t\tret = error(FAILED_RUN, repack_cmd.args.v[0]);\n \t\tgoto out;\n \t}\n \n@@ -899,7 +928,6 @@ int cmd_gc(int argc,\n \tint keep_largest_pack = -1;\n \tint skip_foreground_tasks = 0;\n \ttimestamp_t dummy;\n-\tstruct strvec repack_args = STRVEC_INIT;\n \tstruct maintenance_run_opts opts = MAINTENANCE_RUN_OPTS_INIT;\n \tstruct gc_config cfg = GC_CONFIG_INIT;\n \tconst char *prune_expire_sentinel = \"sentinel\";\n@@ -939,8 +967,6 @@ int cmd_gc(int argc,\n \tshow_usage_with_options_if_asked(argc, argv,\n \t\t\t\t\t builtin_gc_usage, builtin_gc_options);\n \n-\tstrvec_pushl(&repack_args, \"repack\", \"-d\", \"-l\", NULL);\n-\n \tgc_config(&cfg);\n \n \tif (parse_expiry_date(cfg.gc_log_expire, &gc_log_expire_time))\n@@ -961,16 +987,6 @@ int cmd_gc(int argc,\n \tif (cfg.prune_expire && parse_expiry_date(cfg.prune_expire, &dummy))\n \t\tdie(_(\"failed to parse prune expiry value %s\"), cfg.prune_expire);\n \n-\tif (aggressive) {\n-\t\tstrvec_push(&repack_args, \"-f\");\n-\t\tif (cfg.aggressive_depth > 0)\n-\t\t\tstrvec_pushf(&repack_args, \"--depth=%d\", cfg.aggressive_depth);\n-\t\tif (cfg.aggressive_window > 0)\n-\t\t\tstrvec_pushf(&repack_args, \"--window=%d\", cfg.aggressive_window);\n-\t}\n-\tif (opts.quiet)\n-\t\tstrvec_push(&repack_args, \"-q\");\n-\n \tif (opts.auto_flag) {\n \t\tif (cfg.detach_auto && opts.detach < 0)\n \t\t\topts.detach = 1;\n@@ -978,8 +994,7 @@ int cmd_gc(int argc,\n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(&cfg, &repack_args) ||\n-\t\t run_hooks(the_repository, \"pre-auto-gc\")) {\n+\t\tif (!need_to_gc(&cfg) || run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -991,18 +1006,6 @@ int cmd_gc(int argc,\n \t\t\t\tfprintf(stderr, _(\"Auto packing the repository for optimum performance.\\n\"));\n \t\t\tfprintf(stderr, _(\"See \\\"git help gc\\\" for manual housekeeping.\\n\"));\n \t\t}\n-\t} else {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (keep_largest_pack != -1) {\n-\t\t\tif (keep_largest_pack)\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t} else if (cfg.big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg.big_pack_threshold);\n-\t\t}\n-\n-\t\tadd_repack_all_option(&cfg, &keep_pack, &repack_args);\n-\t\tstring_list_clear(&keep_pack, 0);\n \t}\n \n \tif (opts.detach > 0) {\n@@ -1065,7 +1068,7 @@ int cmd_gc(int argc,\n \tif (maintenance_task_rerere_gc(&opts, &cfg))\n \t\tdie(FAILED_RUN, \"rerere\");\n \n-\tif (maintenance_task_odb(&opts, &cfg, &repack_args))\n+\tif (maintenance_task_odb(&opts, &cfg, keep_largest_pack, aggressive))\n \t\tdie(NULL);\n \n \treport_garbage = report_pack_garbage;\n@@ -1088,7 +1091,6 @@ int cmd_gc(int argc,\n \n out:\n \tmaintenance_run_opts_release(&opts);\n-\tstrvec_clear(&repack_args);\n \tgc_config_release(&cfg);\n \treturn 0;\n }\n@@ -1291,15 +1293,7 @@ static int maintenance_task_gc_background(struct maintenance_run_opts *opts,\n \n static int gc_condition(struct gc_config *cfg)\n {\n-\t/*\n-\t * Note that it's fine to drop the repack arguments here, as we execute\n-\t * git-gc(1) as a separate child process anyway. So it knows to compute\n-\t * these arguments again.\n-\t */\n-\tstruct strvec repack_args = STRVEC_INIT;\n-\tint ret = need_to_gc(cfg, &repack_args);\n-\tstrvec_clear(&repack_args);\n-\treturn ret;\n+\treturn need_to_gc(cfg);\n }\n \n static int prune_packed(struct maintenance_run_opts *opts)\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547345","messageId":"20260707-b4-pks-odb-optimize-v1-5-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 05/11] builtin/gc: inline config values specific to the \"files\" backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:37Z","receivedAt":"2026-07-07T15:33:09Z","isPatch":true,"body":"The `struct gc_config` contains a set of values that we read via the Git\nrepository's configuration. Several of those values that are consumed by\nthe object database optimization logic are inherently specific to the\n\"files\" config.\n\nIn a later commit we'll make the logic to optimize object databases\npluggable. So by carrying these \"files\"-backend specific values in the\ngeneric config struct means that other backends would have to worry\nabout these values, too. This feels somewhat dirty, as implementation-\nspecific details should live with the backends themselves.\n\nInline these values directly at the call sites that need them instead.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 115 +++++++++++++++++++++++++++--------------------------------\n 1 file changed, 53 insertions(+), 62 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 25a59caea6..5d445edaa0 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -130,22 +130,11 @@ struct gc_config {\n \tunsigned long max_cruft_size;\n \tint aggressive_depth;\n \tint aggressive_window;\n-\tint gc_auto_threshold;\n-\tint gc_auto_pack_limit;\n \tint detach_auto;\n \tchar *gc_log_expire;\n \tchar *prune_expire;\n \tchar *prune_worktrees_expire;\n-\tchar *repack_filter;\n-\tchar *repack_filter_to;\n \tchar *repack_expire_to;\n-\tunsigned long big_pack_threshold;\n-\tunsigned long max_delta_cache_size;\n-\t/*\n-\t * Remove this member from gc_config once repo_settings is passed\n-\t * through the callchain.\n-\t */\n-\tsize_t delta_base_cache_limit;\n };\n \n #define GC_CONFIG_INIT { \\\n@@ -154,14 +143,10 @@ struct gc_config {\n \t.cruft_packs = 1, \\\n \t.aggressive_depth = 50, \\\n \t.aggressive_window = 250, \\\n-\t.gc_auto_threshold = 6700, \\\n-\t.gc_auto_pack_limit = 50, \\\n \t.detach_auto = 1, \\\n \t.gc_log_expire = xstrdup(\"1.day.ago\"), \\\n \t.prune_expire = xstrdup(\"2.weeks.ago\"), \\\n \t.prune_worktrees_expire = xstrdup(\"3.months.ago\"), \\\n-\t.max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE, \\\n-\t.delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT, \\\n }\n \n static void gc_config_release(struct gc_config *cfg)\n@@ -169,15 +154,12 @@ static void gc_config_release(struct gc_config *cfg)\n \tfree(cfg->gc_log_expire);\n \tfree(cfg->prune_expire);\n \tfree(cfg->prune_worktrees_expire);\n-\tfree(cfg->repack_filter);\n-\tfree(cfg->repack_filter_to);\n }\n \n static void gc_config(struct gc_config *cfg)\n {\n \tconst char *value;\n \tchar *owned = NULL;\n-\tunsigned long ulongval;\n \n \tif (!repo_config_get_value(the_repository, \"gc.packrefs\", &value)) {\n \t\tif (value && !strcmp(value, \"notbare\"))\n@@ -192,8 +174,6 @@ static void gc_config(struct gc_config *cfg)\n \n \trepo_config_get_int(the_repository, \"gc.aggressivewindow\", &cfg->aggressive_window);\n \trepo_config_get_int(the_repository, \"gc.aggressivedepth\", &cfg->aggressive_depth);\n-\trepo_config_get_int(the_repository, \"gc.auto\", &cfg->gc_auto_threshold);\n-\trepo_config_get_int(the_repository, \"gc.autopacklimit\", &cfg->gc_auto_pack_limit);\n \trepo_config_get_bool(the_repository, \"gc.autodetach\", &cfg->detach_auto);\n \trepo_config_get_bool(the_repository, \"gc.cruftpacks\", &cfg->cruft_packs);\n \trepo_config_get_ulong(the_repository, \"gc.maxcruftsize\", &cfg->max_cruft_size);\n@@ -213,22 +193,6 @@ static void gc_config(struct gc_config *cfg)\n \t\tcfg->gc_log_expire = owned;\n \t}\n \n-\trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &cfg->big_pack_threshold);\n-\trepo_config_get_ulong(the_repository, \"pack.deltacachesize\", &cfg->max_delta_cache_size);\n-\n-\tif (!repo_config_get_ulong(the_repository, \"core.deltabasecachelimit\", &ulongval))\n-\t\tcfg->delta_base_cache_limit = ulongval;\n-\n-\tif (!repo_config_get_string(the_repository, \"gc.repackfilter\", &owned)) {\n-\t\tfree(cfg->repack_filter);\n-\t\tcfg->repack_filter = owned;\n-\t}\n-\n-\tif (!repo_config_get_string(the_repository, \"gc.repackfilterto\", &owned)) {\n-\t\tfree(cfg->repack_filter_to);\n-\t\tcfg->repack_filter_to = owned;\n-\t}\n-\n \trepo_config(the_repository, git_default_config, NULL);\n }\n \n@@ -504,12 +468,12 @@ static struct packed_git *find_base_packs(struct string_list *packs,\n \treturn base;\n }\n \n-static int too_many_packs(struct gc_config *cfg)\n+static int too_many_packs(int gc_auto_pack_limit)\n {\n \tstruct packed_git *p;\n \tint cnt = 0;\n \n-\tif (cfg->gc_auto_pack_limit <= 0)\n+\tif (gc_auto_pack_limit <= 0)\n \t\treturn 0;\n \n \trepo_for_each_pack(the_repository, p) {\n@@ -523,7 +487,7 @@ static int too_many_packs(struct gc_config *cfg)\n \t\t */\n \t\tcnt++;\n \t}\n-\treturn cfg->gc_auto_pack_limit < cnt;\n+\treturn gc_auto_pack_limit < cnt;\n }\n \n static uint64_t total_ram(void)\n@@ -571,9 +535,10 @@ static uint64_t total_ram(void)\n \treturn 0;\n }\n \n-static uint64_t estimate_repack_memory(struct gc_config *cfg,\n-\t\t\t\t struct packed_git *pack)\n+static uint64_t estimate_repack_memory(struct packed_git *pack)\n {\n+\tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n+\tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n \tunsigned long nr_objects;\n \tsize_t os_cache, heap;\n \n@@ -584,6 +549,9 @@ static uint64_t estimate_repack_memory(struct gc_config *cfg,\n \tif (!pack || !nr_objects)\n \t\treturn 0;\n \n+\trepo_config_get_ulong(the_repository, \"pack.deltacachesize\", &max_delta_cache_size);\n+\trepo_config_get_ulong(the_repository, \"core.deltabasecachelimit\", &delta_base_cache_limit);\n+\n \t/*\n \t * First we have to scan through at least one pack.\n \t * Assume enough room in OS file cache to keep the entire pack\n@@ -611,9 +579,9 @@ static uint64_t estimate_repack_memory(struct gc_config *cfg,\n \t * read_sha1_file() (either at delta calculation phase, or\n \t * writing phase) also fills up the delta base cache\n \t */\n-\theap += cfg->delta_base_cache_limit;\n+\theap += delta_base_cache_limit;\n \t/* and of course pack-objects has its own delta cache */\n-\theap += cfg->max_delta_cache_size;\n+\theap += max_delta_cache_size;\n \n \treturn os_cache + heap;\n }\n@@ -629,6 +597,12 @@ static void add_repack_all_option(struct gc_config *cfg,\n \t\t\t\t struct string_list *keep_pack,\n \t\t\t\t struct strvec *args)\n {\n+\tchar *repack_filter = NULL;\n+\tchar *repack_filter_to = NULL;\n+\n+\trepo_config_get_string(the_repository, \"gc.repackfilter\", &repack_filter);\n+\trepo_config_get_string(the_repository, \"gc.repackfilterto\", &repack_filter_to);\n+\n \tif (cfg->prune_expire && !strcmp(cfg->prune_expire, \"now\")\n \t\t&& !(cfg->cruft_packs && cfg->repack_expire_to))\n \t\tstrvec_push(args, \"-a\");\n@@ -650,10 +624,13 @@ static void add_repack_all_option(struct gc_config *cfg,\n \tif (keep_pack)\n \t\tfor_each_string_list(keep_pack, keep_one_pack, args);\n \n-\tif (cfg->repack_filter && *cfg->repack_filter)\n-\t\tstrvec_pushf(args, \"--filter=%s\", cfg->repack_filter);\n-\tif (cfg->repack_filter_to && *cfg->repack_filter_to)\n-\t\tstrvec_pushf(args, \"--filter-to=%s\", cfg->repack_filter_to);\n+\tif (repack_filter && *repack_filter)\n+\t\tstrvec_pushf(args, \"--filter=%s\", repack_filter);\n+\tif (repack_filter_to && *repack_filter_to)\n+\t\tstrvec_pushf(args, \"--filter-to=%s\", repack_filter_to);\n+\n+\tfree(repack_filter);\n+\tfree(repack_filter_to);\n }\n \n static void add_repack_incremental_option(struct strvec *args)\n@@ -661,16 +638,24 @@ static void add_repack_incremental_option(struct strvec *args)\n \tstrvec_push(args, \"--no-write-bitmap-index\");\n }\n \n-static int need_to_gc(struct gc_config *cfg)\n+static int need_to_gc(struct repository *repo)\n {\n+\tint gc_auto_threshold = 6700;\n+\tint gc_auto_pack_limit = 50;\n+\n+\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\n \t/*\n \t * Setting gc.auto to 0 or negative can disable the\n \t * automatic gc.\n \t */\n-\tif (cfg->gc_auto_threshold <= 0)\n+\tif (gc_auto_threshold <= 0)\n \t\treturn 0;\n-\tif (!too_many_packs(cfg) && !too_many_loose_objects(cfg->gc_auto_threshold))\n+\tif (!too_many_packs(gc_auto_pack_limit) &&\n+\t !too_many_loose_objects(gc_auto_threshold))\n \t\treturn 0;\n+\n \treturn 1;\n }\n \n@@ -807,8 +792,15 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tint aggressive)\n {\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n+\tunsigned long big_pack_threshold = 0;\n+\tint gc_auto_threshold = 6700;\n+\tint gc_auto_pack_limit = 50;\n \tint ret;\n \n+\trepo_config_get_int(the_repository, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(the_repository, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &big_pack_threshold);\n+\n \tif (the_repository->repository_format_precious_objects)\n \t\treturn 0;\n \n@@ -843,19 +835,18 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\tif (keep_largest_pack != -1) {\n \t\t\tif (keep_largest_pack)\n \t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t} else if (cfg->big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n+\t\t} else if (big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n \t\t}\n \n \t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n \t\tstring_list_clear(&keep_pack, 0);\n-\t} else if (too_many_packs(cfg)) {\n+\t} else if (too_many_packs(gc_auto_pack_limit)) {\n \t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n-\t\tif (cfg->big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n-\t\t\tif (keep_pack.nr >= cfg->gc_auto_pack_limit) {\n-\t\t\t\tcfg->big_pack_threshold = 0;\n+\t\tif (big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n \t\t\t\tstring_list_clear(&keep_pack, 0);\n \t\t\t\tfind_base_packs(&keep_pack, 0);\n \t\t\t}\n@@ -864,7 +855,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\tuint64_t mem_have, mem_want;\n \n \t\t\tmem_have = total_ram();\n-\t\t\tmem_want = estimate_repack_memory(cfg, p);\n+\t\t\tmem_want = estimate_repack_memory(p);\n \n \t\t\t/*\n \t\t\t * Only allow 1/2 of memory for pack-objects, leave\n@@ -905,7 +896,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t}\n \t}\n \n-\tif (opts->auto_flag && too_many_loose_objects(cfg->gc_auto_threshold))\n+\tif (opts->auto_flag && too_many_loose_objects(gc_auto_threshold))\n \t\twarning(_(\"There are too many unreachable loose objects; \"\n \t\t\t\"run 'git prune' to remove them.\"));\n \n@@ -994,7 +985,7 @@ int cmd_gc(int argc,\n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(&cfg) || run_hooks(the_repository, \"pre-auto-gc\")) {\n+\t\tif (!need_to_gc(the_repository) || run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -1291,9 +1282,9 @@ static int maintenance_task_gc_background(struct maintenance_run_opts *opts,\n \treturn run_command(&child);\n }\n \n-static int gc_condition(struct gc_config *cfg)\n+static int gc_condition(struct gc_config *cfg UNUSED)\n {\n-\treturn need_to_gc(cfg);\n+\treturn need_to_gc(the_repository);\n }\n \n static int prune_packed(struct maintenance_run_opts *opts)\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547347","messageId":"20260707-b4-pks-odb-optimize-v1-6-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 06/11] builtin/gc: introduce object database optimization options","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:38Z","receivedAt":"2026-07-07T15:33:10Z","isPatch":true,"body":"Introduce `struct odb_optimize_options` to decouple the options that are\nspecific to optimizing the object database from `struct gc_config`. This\nstructure will be moved into the object database layer in a subsequent\ncommit.\n\nNote that there are a small set of backend-specific options in this\nstructure. In an ideal world those of course wouldn't exist, but as\nwe're introducing the object database abstractions retroactively we are\nsomewhat forced to keep them.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 181 +++++++++++++++++++++++++++++++++++++++--------------------\n 1 file changed, 120 insertions(+), 61 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 5d445edaa0..17490106fc 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -593,7 +593,39 @@ static int keep_one_pack(struct string_list_item *item, void *data)\n \treturn 0;\n }\n \n-static void add_repack_all_option(struct gc_config *cfg,\n+enum odb_optimize_flags {\n+\t/* Enable verbose logging and progress reporting. */\n+\tODB_OPTIMIZE_VERBOSE = (1 << 0),\n+\n+\t/* Perform auto-maintenance, only optimizing objects as required. */\n+\tODB_OPTIMIZE_AUTO = (1 << 1),\n+\n+\t/* Recompute existing deltas. */\n+\tODB_OPTIMIZE_NO_REUSE_DELTAS = (1 << 2),\n+};\n+\n+struct odb_optimize_options {\n+\tenum odb_optimize_flags flags;\n+\tconst char *prune_expire;\n+\tconst char *expire_to;\n+\tint depth;\n+\tint window;\n+\n+\t/* Backend-specific options. */\n+\tint keep_largest_pack;\n+\tint cruft_packs;\n+\tunsigned long max_cruft_size;\n+};\n+\n+#define OPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive) \\\n+\t.prune_expire = (cfg)->prune_expire, \\\n+\t.expire_to = (cfg)->repack_expire_to, \\\n+\t.cruft_packs = (cfg)->cruft_packs, \\\n+\t.max_cruft_size = (cfg)->max_cruft_size, \\\n+\t.window = (aggressive) ? (cfg)->aggressive_window : 0, \\\n+\t.depth = (aggressive) ? (cfg)->aggressive_depth : 0\n+\n+static void add_repack_all_option(const struct odb_optimize_options *opts,\n \t\t\t\t struct string_list *keep_pack,\n \t\t\t\t struct strvec *args)\n {\n@@ -603,22 +635,22 @@ static void add_repack_all_option(struct gc_config *cfg,\n \trepo_config_get_string(the_repository, \"gc.repackfilter\", &repack_filter);\n \trepo_config_get_string(the_repository, \"gc.repackfilterto\", &repack_filter_to);\n \n-\tif (cfg->prune_expire && !strcmp(cfg->prune_expire, \"now\")\n-\t\t&& !(cfg->cruft_packs && cfg->repack_expire_to))\n+\tif (opts->prune_expire && !strcmp(opts->prune_expire, \"now\") &&\n+\t !(opts->cruft_packs && opts->expire_to))\n \t\tstrvec_push(args, \"-a\");\n-\telse if (cfg->cruft_packs) {\n+\telse if (opts->cruft_packs) {\n \t\tstrvec_push(args, \"--cruft\");\n-\t\tif (cfg->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", cfg->prune_expire);\n-\t\tif (cfg->max_cruft_size)\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", opts->prune_expire);\n+\t\tif (opts->max_cruft_size)\n \t\t\tstrvec_pushf(args, \"--max-cruft-size=%lu\",\n-\t\t\t\t cfg->max_cruft_size);\n-\t\tif (cfg->repack_expire_to)\n-\t\t\tstrvec_pushf(args, \"--expire-to=%s\", cfg->repack_expire_to);\n+\t\t\t\t opts->max_cruft_size);\n+\t\tif (opts->expire_to)\n+\t\t\tstrvec_pushf(args, \"--expire-to=%s\", opts->expire_to);\n \t} else {\n \t\tstrvec_push(args, \"-A\");\n-\t\tif (cfg->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", cfg->prune_expire);\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", opts->prune_expire);\n \t}\n \n \tif (keep_pack)\n@@ -786,10 +818,8 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \treturn 0;\n }\n \n-static int maintenance_task_odb(struct maintenance_run_opts *opts,\n-\t\t\t\tstruct gc_config *cfg,\n-\t\t\t\tint keep_largest_pack,\n-\t\t\t\tint aggressive)\n+static int odb_optimize(struct object_database *odb,\n+\t\t\tconst struct odb_optimize_options *opts)\n {\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \tunsigned long big_pack_threshold = 0;\n@@ -801,21 +831,20 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \trepo_config_get_int(the_repository, \"gc.autopacklimit\", &gc_auto_pack_limit);\n \trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &big_pack_threshold);\n \n-\tif (the_repository->repository_format_precious_objects)\n+\tif (odb->repo->repository_format_precious_objects)\n \t\treturn 0;\n \n \trepack_cmd.git_cmd = 1;\n \trepack_cmd.odb_to_close = the_repository->objects;\n \n \tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (aggressive) {\n+\tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n \t\tstrvec_push(&repack_cmd.args, \"-f\");\n-\t\tif (cfg->aggressive_depth > 0)\n-\t\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", cfg->aggressive_depth);\n-\t\tif (cfg->aggressive_window > 0)\n-\t\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", cfg->aggressive_window);\n-\t}\n-\tif (opts->quiet)\n+\tif (opts->depth > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", opts->depth);\n+\tif (opts->window > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", opts->window);\n+\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n \t\tstrvec_push(&repack_cmd.args, \"-q\");\n \n \t/*\n@@ -829,47 +858,49 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t *\n \t * - Otherwise we perform an incremental repack.\n \t */\n-\tif (!opts->auto_flag) {\n+\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n \t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n-\t\tif (keep_largest_pack != -1) {\n-\t\t\tif (keep_largest_pack)\n+\t\tif (opts->keep_largest_pack != -1) {\n+\t\t\tif (opts->keep_largest_pack)\n \t\t\t\tfind_base_packs(&keep_pack, 0);\n \t\t} else if (big_pack_threshold) {\n \t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n \t\t}\n \n-\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n+\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n \t\tstring_list_clear(&keep_pack, 0);\n-\t} else if (too_many_packs(gc_auto_pack_limit)) {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n-\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t} else {\n+\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\tif (big_pack_threshold) {\n+\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n+\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t}\n+\t\t\t} else {\n+\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\t\tmem_have = total_ram();\n+\t\t\t\tmem_want = estimate_repack_memory(p);\n+\n+\t\t\t\t/*\n+\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t\t * the rest for the OS and other processes in the\n+\t\t\t\t * system.\n+\t\t\t\t */\n+\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n \t\t\t}\n-\t\t} else {\n-\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n-\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\tmem_have = total_ram();\n-\t\t\tmem_want = estimate_repack_memory(p);\n \n-\t\t\t/*\n-\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t * the rest for the OS and other processes in the\n-\t\t\t * system.\n-\t\t\t */\n-\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n+\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t} else {\n+\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n \t\t}\n-\n-\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n-\t\tstring_list_clear(&keep_pack, 0);\n-\t} else {\n-\t\tadd_repack_incremental_option(&repack_cmd.args);\n \t}\n \n \tif (run_command(&repack_cmd)) {\n@@ -877,13 +908,13 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\tgoto out;\n \t}\n \n-\tif (cfg->prune_expire) {\n+\tif (opts->prune_expire) {\n \t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n \n \t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n \t\t/* run `git prune` even if using cruft packs */\n-\t\tstrvec_push(&prune_cmd.args, cfg->prune_expire);\n-\t\tif (opts->quiet)\n+\t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n \t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n \t\tif (repo_has_promisor_remote(the_repository))\n \t\t\tstrvec_push(&prune_cmd.args,\n@@ -896,7 +927,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t}\n \t}\n \n-\tif (opts->auto_flag && too_many_loose_objects(gc_auto_threshold))\n+\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(gc_auto_threshold))\n \t\twarning(_(\"There are too many unreachable loose objects; \"\n \t\t\t\"run 'git prune' to remove them.\"));\n \n@@ -906,6 +937,26 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \treturn ret;\n }\n \n+static int maintenance_task_odb(struct maintenance_run_opts *opts,\n+\t\t\t\tstruct gc_config *cfg,\n+\t\t\t\tint keep_largest_pack,\n+\t\t\t\tint aggressive)\n+{\n+\tstruct odb_optimize_options odb_opts = {\n+\t\t.keep_largest_pack = keep_largest_pack,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive),\n+\t};\n+\n+\tif (opts->auto_flag)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_AUTO;\n+\tif (!opts->quiet)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n+\tif (aggressive)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_NO_REUSE_DELTAS;\n+\n+\treturn odb_optimize(the_repository->objects, &odb_opts);\n+}\n+\n int cmd_gc(int argc,\n \t const char **argv,\n \t const char *prefix,\n@@ -1596,11 +1647,19 @@ static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,\n \tchild.odb_to_close = the_repository->objects;\n \n \tstrvec_pushl(&child.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (geometry.split < geometry.pack_nr)\n+\tif (geometry.split < geometry.pack_nr) {\n \t\tstrvec_pushf(&child.args, \"--geometric=%d\",\n \t\t\t geometry.split_factor);\n-\telse\n-\t\tadd_repack_all_option(cfg, NULL, &child.args);\n+\t} else {\n+\t\tstruct odb_optimize_options odb_opts = {\n+\t\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n+\t\t};\n+\n+\t\tif (!opts->quiet)\n+\t\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n+\n+\t\tadd_repack_all_option(&odb_opts, NULL, &child.args);\n+\t}\n \tif (opts->quiet)\n \t\tstrvec_push(&child.args, \"--quiet\");\n \tif (the_repository->settings.core_multi_pack_index)\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547348","messageId":"20260707-b4-pks-odb-optimize-v1-7-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 07/11] builtin/gc: move geometric repacking into `odb_optimize()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:39Z","receivedAt":"2026-07-07T15:33:13Z","isPatch":true,"body":"We have two major object database optimization strategies:\n\n - The legacy strategy used by git-gc(1), which absorbs loose objects\n into packfiles, and eventually merges all packfiles once we have too\n many of them.\n\n - The more recent \"geometric\" strategy used by git-maintenance(1),\n which merges packfiles using a geometric sequence.\n\nThese two strategies are still using completely separate code paths. In\na subsequent commit we'll want to make both strategies pluggable though.\n\nPrepare for this change by merging the \"geometric\" strategy into\n`odb_optimize()`. This also allows us to reuse some of the logic we have\nin that function.\n\nNote that this change requires us to adapt tests because we're now using\n\"-q\" instead of \"--quiet\". Naturally though, these invocations are of\ncourse equivalent to one another.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 171 +++++++++++++++++++++++++------------------------\n t/t7900-maintenance.sh | 22 +++----\n 2 files changed, 98 insertions(+), 95 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 17490106fc..c8504f4456 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -593,6 +593,11 @@ static int keep_one_pack(struct string_list_item *item, void *data)\n \treturn 0;\n }\n \n+enum odb_optimize_strategy {\n+\tODB_OPTIMIZE_INCREMENTAL,\n+\tODB_OPTIMIZE_GEOMETRIC,\n+};\n+\n enum odb_optimize_flags {\n \t/* Enable verbose logging and progress reporting. */\n \tODB_OPTIMIZE_VERBOSE = (1 << 0),\n@@ -605,6 +610,7 @@ enum odb_optimize_flags {\n };\n \n struct odb_optimize_options {\n+\tenum odb_optimize_strategy strategy;\n \tenum odb_optimize_flags flags;\n \tconst char *prune_expire;\n \tconst char *expire_to;\n@@ -858,49 +864,87 @@ static int odb_optimize(struct object_database *odb,\n \t *\n \t * - Otherwise we perform an incremental repack.\n \t */\n-\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (opts->keep_largest_pack != -1) {\n-\t\t\tif (opts->keep_largest_pack)\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t} else if (big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n-\t\t}\n-\n-\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n-\t\tstring_list_clear(&keep_pack, 0);\n-\t} else {\n-\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL:\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n \t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n-\t\t\tif (big_pack_threshold) {\n-\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n-\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n-\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\tif (opts->keep_largest_pack != -1) {\n+\t\t\t\tif (opts->keep_largest_pack)\n \t\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t\t\t}\n-\t\t\t} else {\n-\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n-\t\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\t\tmem_have = total_ram();\n-\t\t\t\tmem_want = estimate_repack_memory(p);\n-\n-\t\t\t\t/*\n-\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t\t * the rest for the OS and other processes in the\n-\t\t\t\t * system.\n-\t\t\t\t */\n-\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t} else if (big_pack_threshold) {\n+\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n \t\t\t}\n \n \t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n \t\t\tstring_list_clear(&keep_pack, 0);\n \t\t} else {\n-\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\t\tif (big_pack_threshold) {\n+\t\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t\t}\n+\t\t\t\t} else {\n+\t\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\t\t\tmem_have = total_ram();\n+\t\t\t\t\tmem_want = estimate_repack_memory(p);\n+\n+\t\t\t\t\t/*\n+\t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t\t\t * the rest for the OS and other processes in the\n+\t\t\t\t\t * system.\n+\t\t\t\t\t */\n+\t\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t}\n+\n+\t\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t} else {\n+\t\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t\t\t}\n \t\t}\n+\n+\t\tbreak;\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\n+\t\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = the_repository;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\tif (geometry.split < geometry.pack_nr) {\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--geometric=%d\",\n+\t\t\t\t geometry.split_factor);\n+\t\t} else {\n+\t\t\tadd_repack_all_option(opts, NULL, &repack_cmd.args);\n+\t\t}\n+\t\tif (the_repository->settings.core_multi_pack_index)\n+\t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n+\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tdie(\"unknown maintenance strategy '%d'\", opts->strategy);\n \t}\n \n \tif (run_command(&repack_cmd)) {\n@@ -908,7 +952,8 @@ static int odb_optimize(struct object_database *odb,\n \t\tgoto out;\n \t}\n \n-\tif (opts->prune_expire) {\n+\t/* Geometric repacking uses cruft packs, so we don't have to prune separately. */\n+\tif (opts->strategy != ODB_OPTIMIZE_GEOMETRIC && opts->prune_expire) {\n \t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n \n \t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n@@ -943,6 +988,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tint aggressive)\n {\n \tstruct odb_optimize_options odb_opts = {\n+\t\t.strategy = ODB_OPTIMIZE_INCREMENTAL,\n \t\t.keep_largest_pack = keep_largest_pack,\n \t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive),\n \t};\n@@ -1624,58 +1670,15 @@ static int maintenance_task_incremental_repack(struct maintenance_run_opts *opts\n static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,\n \t\t\t\t\t struct gc_config *cfg)\n {\n-\tstruct pack_geometry geometry = {\n-\t\t.split_factor = 2,\n-\t};\n-\tstruct pack_objects_args po_args = {\n-\t\t.local = 1,\n+\tstruct odb_optimize_options odb_opts = {\n+\t\t.strategy = ODB_OPTIMIZE_GEOMETRIC,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n \t};\n-\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\tstruct child_process child = CHILD_PROCESS_INIT;\n-\tint ret;\n-\n-\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t &geometry.split_factor);\n-\n-\texisting_packs.repo = the_repository;\n-\texisting_packs_collect(&existing_packs, &kept_packs);\n-\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\tpack_geometry_split(&geometry);\n-\n-\tchild.git_cmd = 1;\n-\tchild.odb_to_close = the_repository->objects;\n-\n-\tstrvec_pushl(&child.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (geometry.split < geometry.pack_nr) {\n-\t\tstrvec_pushf(&child.args, \"--geometric=%d\",\n-\t\t\t geometry.split_factor);\n-\t} else {\n-\t\tstruct odb_optimize_options odb_opts = {\n-\t\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n-\t\t};\n \n-\t\tif (!opts->quiet)\n-\t\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n-\n-\t\tadd_repack_all_option(&odb_opts, NULL, &child.args);\n-\t}\n-\tif (opts->quiet)\n-\t\tstrvec_push(&child.args, \"--quiet\");\n-\tif (the_repository->settings.core_multi_pack_index)\n-\t\tstrvec_push(&child.args, \"--write-midx\");\n-\n-\tif (run_command(&child)) {\n-\t\tret = error(_(\"failed to perform geometric repack\"));\n-\t\tgoto out;\n-\t}\n-\n-\tret = 0;\n+\tif (!opts->quiet)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n \n-out:\n-\texisting_packs_release(&existing_packs);\n-\tpack_geometry_release(&geometry);\n-\treturn ret;\n+\treturn odb_optimize(the_repository->objects, &odb_opts);\n }\n \n static int geometric_repack_auto_condition(struct gc_config *cfg UNUSED)\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 1212b306b6..e0251064c7 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -556,8 +556,8 @@ run_and_verify_geometric_pack () {\n \trm -f \"trace2.txt\" &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n \t\tgit maintenance run --task=geometric-repack 2>/dev/null &&\n-\ttest_subcommand git repack -d -l --geometric=2 \\\n-\t\t--quiet --write-midx packfiles &&\n@@ -588,8 +588,8 @@ test_expect_success 'geometric repacking task' '\n \t\t# The initial repack causes an all-into-one repack.\n \t\tGIT_TRACE2_EVENT=\"$(pwd)/initial-repack.txt\" \\\n \t\t\tgit maintenance run --task=geometric-repack 2>/dev/null &&\n-\t\ttest_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \\\n-\t\t\t--quiet --write-midx /dev/null &&\n-\t\ttest_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \\\n-\t\t\t--quiet --write-midx /dev/null &&\n-\t\ttest_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \\\n-\t\t\t--quiet --write-midx packs &&\n \t\ttest_line_count = 2 packs &&\n \t\tls .git/objects/pack/*.mtimes >cruft &&\n@@ -736,7 +736,7 @@ test_expect_success 'geometric repacking honors configured split factor' '\n \n \t\ttest_geometric_repack_needed false splitFactor=2 &&\n \t\ttest_geometric_repack_needed true splitFactor=3 &&\n-\t\ttest_subcommand git repack -d -l --geometric=3 --quiet --write-midx \n---\n builtin/gc.c | 160 ++++++++++++++++++++++++++++++++++-------------------------\n 1 file changed, 92 insertions(+), 68 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex c8504f4456..e119930adc 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -676,25 +676,84 @@ static void add_repack_incremental_option(struct strvec *args)\n \tstrvec_push(args, \"--no-write-bitmap-index\");\n }\n \n-static int need_to_gc(struct repository *repo)\n+static bool odb_optimize_required(struct object_database *odb,\n+\t\t\t\t const struct odb_optimize_options *opts)\n {\n-\tint gc_auto_threshold = 6700;\n-\tint gc_auto_pack_limit = 50;\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL: {\n+\t\tint gc_auto_threshold = 6700;\n+\t\tint gc_auto_pack_limit = 50;\n \n-\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n-\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\t\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n+\t\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n \n-\t/*\n-\t * Setting gc.auto to 0 or negative can disable the\n-\t * automatic gc.\n-\t */\n-\tif (gc_auto_threshold <= 0)\n-\t\treturn 0;\n-\tif (!too_many_packs(gc_auto_pack_limit) &&\n-\t !too_many_loose_objects(gc_auto_threshold))\n-\t\treturn 0;\n+\t\t/*\n+\t\t * Setting gc.auto to 0 or negative can disable the\n+\t\t * automatic gc.\n+\t\t */\n+\t\tif (gc_auto_threshold <= 0)\n+\t\t\treturn false;\n+\t\tif (!too_many_packs(gc_auto_pack_limit) &&\n+\t\t !too_many_loose_objects(gc_auto_threshold))\n+\t\t\treturn false;\n \n-\treturn 1;\n+\t\treturn true;\n+\t}\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\t\tint auto_value = 100;\n+\t\tbool ret;\n+\n+\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.auto\",\n+\t\t\t\t &auto_value);\n+\t\tif (!auto_value)\n+\t\t\treturn false;\n+\t\tif (auto_value < 0)\n+\t\t\treturn true;\n+\n+\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = odb->repo;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\t/*\n+\t\t * When we'd merge at least two packs with one another we always\n+\t\t * perform the repack.\n+\t\t */\n+\t\tif (geometry.split) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\t/*\n+\t\t * Otherwise, we estimate the number of loose objects to determine\n+\t\t * whether we want to create a new packfile or not.\n+\t\t */\n+\t\tif (too_many_loose_objects(auto_value)) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tret = false;\n+\n+\tout:\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\treturn ret;\n+\t}\n+\tdefault:\n+\t\tBUG(\"unknown maintenance strategy '%d'\", opts->strategy);\n+\t}\n }\n \n /* return NULL on success, else hostname running the gc */\n@@ -1076,13 +1135,19 @@ int cmd_gc(int argc,\n \t\tdie(_(\"failed to parse prune expiry value %s\"), cfg.prune_expire);\n \n \tif (opts.auto_flag) {\n+\t\tstruct odb_optimize_options optimize_opts = {\n+\t\t\t.strategy = ODB_OPTIMIZE_INCREMENTAL,\n+\t\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(&cfg, 0),\n+\t\t};\n+\n \t\tif (cfg.detach_auto && opts.detach < 0)\n \t\t\topts.detach = 1;\n \n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(the_repository) || run_hooks(the_repository, \"pre-auto-gc\")) {\n+\t\tif (!odb_optimize_required(the_repository->objects, &optimize_opts) ||\n+\t\t run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -1379,9 +1444,13 @@ static int maintenance_task_gc_background(struct maintenance_run_opts *opts,\n \treturn run_command(&child);\n }\n \n-static int gc_condition(struct gc_config *cfg UNUSED)\n+static int gc_condition(struct gc_config *cfg)\n {\n-\treturn need_to_gc(the_repository);\n+\tstruct odb_optimize_options opts = {\n+\t\t.strategy = ODB_OPTIMIZE_INCREMENTAL,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n+\t};\n+\treturn odb_optimize_required(the_repository->objects, &opts);\n }\n \n static int prune_packed(struct maintenance_run_opts *opts)\n@@ -1681,58 +1750,13 @@ static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,\n \treturn odb_optimize(the_repository->objects, &odb_opts);\n }\n \n-static int geometric_repack_auto_condition(struct gc_config *cfg UNUSED)\n+static int geometric_repack_auto_condition(struct gc_config *cfg)\n {\n-\tstruct pack_geometry geometry = {\n-\t\t.split_factor = 2,\n-\t};\n-\tstruct pack_objects_args po_args = {\n-\t\t.local = 1,\n+\tstruct odb_optimize_options opts = {\n+\t\t.strategy = ODB_OPTIMIZE_GEOMETRIC,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n \t};\n-\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\tint auto_value = 100;\n-\tint ret;\n-\n-\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.auto\",\n-\t\t\t &auto_value);\n-\tif (!auto_value)\n-\t\treturn 0;\n-\tif (auto_value < 0)\n-\t\treturn 1;\n-\n-\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t &geometry.split_factor);\n-\n-\texisting_packs.repo = the_repository;\n-\texisting_packs_collect(&existing_packs, &kept_packs);\n-\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\tpack_geometry_split(&geometry);\n-\n-\t/*\n-\t * When we'd merge at least two packs with one another we always\n-\t * perform the repack.\n-\t */\n-\tif (geometry.split) {\n-\t\tret = 1;\n-\t\tgoto out;\n-\t}\n-\n-\t/*\n-\t * Otherwise, we estimate the number of loose objects to determine\n-\t * whether we want to create a new packfile or not.\n-\t */\n-\tif (too_many_loose_objects(auto_value)) {\n-\t\tret = 1;\n-\t\tgoto out;\n-\t}\n-\n-\tret = 0;\n-\n-out:\n-\texisting_packs_release(&existing_packs);\n-\tpack_geometry_release(&geometry);\n-\treturn ret;\n+\treturn odb_optimize_required(the_repository->objects, &opts);\n }\n \n typedef int (*maintenance_task_fn)(struct maintenance_run_opts *opts,\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547350","messageId":"20260707-b4-pks-odb-optimize-v1-9-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 09/11] builtin/gc: refactor ODB optimizations to operate on \"files\" source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:41Z","receivedAt":"2026-07-07T15:33:18Z","isPatch":true,"body":"We have a couple of functions that are implementation details of how the\n\"files\" object database source performs optimizations. These functions\noften use global state like `the_repository` and implicitly derive the\nsource they are supposed to optimize.\n\nRefactor these interfaces to accept a \"files\" source directly. This will\nmake it easier to move around the whole logic into \"odb/source-files.c\"\nin a subsequent step.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 79 +++++++++++++++++++++++++++++++-----------------------------\n 1 file changed, 41 insertions(+), 38 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e119930adc..3207182488 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -428,9 +428,8 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n \treturn should_gc;\n }\n \n-static int too_many_loose_objects(int limit)\n+static int too_many_loose_objects(struct odb_source_files *files, int limit)\n {\n-\tstruct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);\n \t/*\n \t * This is weird, but stems from legacy behaviour: the GC auto\n \t * threshold was always essentially interpreted as if it was rounded up\n@@ -446,19 +445,21 @@ static int too_many_loose_objects(int limit)\n \treturn loose_count > auto_threshold;\n }\n \n-static struct packed_git *find_base_packs(struct string_list *packs,\n+static struct packed_git *find_base_packs(struct odb_source_files *files,\n+\t\t\t\t\t struct string_list *packs,\n \t\t\t\t\t unsigned long limit)\n {\n-\tstruct packed_git *p, *base = NULL;\n+\tstruct packfile_list_entry *e;\n+\tstruct packed_git *base = NULL;\n \n-\trepo_for_each_pack(the_repository, p) {\n-\t\tif (!p->pack_local || p->is_cruft)\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->is_cruft)\n \t\t\tcontinue;\n \t\tif (limit) {\n-\t\t\tif (p->pack_size >= limit)\n-\t\t\t\tstring_list_append(packs, p->pack_name);\n-\t\t} else if (!base || base->pack_size < p->pack_size) {\n-\t\t\tbase = p;\n+\t\t\tif (e->pack->pack_size >= limit)\n+\t\t\t\tstring_list_append(packs, e->pack->pack_name);\n+\t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n+\t\t\tbase = e->pack;\n \t\t}\n \t}\n \n@@ -468,18 +469,16 @@ static struct packed_git *find_base_packs(struct string_list *packs,\n \treturn base;\n }\n \n-static int too_many_packs(int gc_auto_pack_limit)\n+static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit)\n {\n-\tstruct packed_git *p;\n+\tstruct packfile_list_entry *e;\n \tint cnt = 0;\n \n \tif (gc_auto_pack_limit <= 0)\n \t\treturn 0;\n \n-\trepo_for_each_pack(the_repository, p) {\n-\t\tif (!p->pack_local)\n-\t\t\tcontinue;\n-\t\tif (p->pack_keep)\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->pack_keep)\n \t\t\tcontinue;\n \t\t/*\n \t\t * Perhaps check the size of the pack and count only\n@@ -535,15 +534,16 @@ static uint64_t total_ram(void)\n \treturn 0;\n }\n \n-static uint64_t estimate_repack_memory(struct packed_git *pack)\n+static uint64_t estimate_repack_memory(struct odb_source_files *files,\n+\t\t\t\t struct packed_git *pack)\n {\n \tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n \tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n \tunsigned long nr_objects;\n \tsize_t os_cache, heap;\n \n-\tif (odb_count_objects(the_repository->objects,\n-\t\t\t ODB_COUNT_OBJECTS_APPROXIMATE, &nr_objects) < 0)\n+\tif (odb_source_count_objects(&files->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n+\t\t\t\t &nr_objects) < 0)\n \t\treturn 0;\n \n \tif (!pack || !nr_objects)\n@@ -679,6 +679,8 @@ static void add_repack_incremental_option(struct strvec *args)\n static bool odb_optimize_required(struct object_database *odb,\n \t\t\t\t const struct odb_optimize_options *opts)\n {\n+\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n+\n \tswitch (opts->strategy) {\n \tcase ODB_OPTIMIZE_INCREMENTAL: {\n \t\tint gc_auto_threshold = 6700;\n@@ -693,8 +695,8 @@ static bool odb_optimize_required(struct object_database *odb,\n \t\t */\n \t\tif (gc_auto_threshold <= 0)\n \t\t\treturn false;\n-\t\tif (!too_many_packs(gc_auto_pack_limit) &&\n-\t\t !too_many_loose_objects(gc_auto_threshold))\n+\t\tif (!too_many_packs(files, gc_auto_pack_limit) &&\n+\t\t !too_many_loose_objects(files, gc_auto_threshold))\n \t\t\treturn false;\n \n \t\treturn true;\n@@ -739,7 +741,7 @@ static bool odb_optimize_required(struct object_database *odb,\n \t\t * Otherwise, we estimate the number of loose objects to determine\n \t\t * whether we want to create a new packfile or not.\n \t\t */\n-\t\tif (too_many_loose_objects(auto_value)) {\n+\t\tif (too_many_loose_objects(files, auto_value)) {\n \t\t\tret = true;\n \t\t\tgoto out;\n \t\t}\n@@ -886,21 +888,22 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n static int odb_optimize(struct object_database *odb,\n \t\t\tconst struct odb_optimize_options *opts)\n {\n+\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \tunsigned long big_pack_threshold = 0;\n \tint gc_auto_threshold = 6700;\n \tint gc_auto_pack_limit = 50;\n \tint ret;\n \n-\trepo_config_get_int(the_repository, \"gc.auto\", &gc_auto_threshold);\n-\trepo_config_get_int(the_repository, \"gc.autopacklimit\", &gc_auto_pack_limit);\n-\trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &big_pack_threshold);\n+\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\trepo_config_get_ulong(odb->repo, \"gc.bigpackthreshold\", &big_pack_threshold);\n \n \tif (odb->repo->repository_format_precious_objects)\n \t\treturn 0;\n \n \trepack_cmd.git_cmd = 1;\n-\trepack_cmd.odb_to_close = the_repository->objects;\n+\trepack_cmd.odb_to_close = odb->repo->objects;\n \n \tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n \tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n@@ -930,29 +933,29 @@ static int odb_optimize(struct object_database *odb,\n \n \t\t\tif (opts->keep_largest_pack != -1) {\n \t\t\t\tif (opts->keep_largest_pack)\n-\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n \t\t\t} else if (big_pack_threshold) {\n-\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n \t\t\t}\n \n \t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n \t\t\tstring_list_clear(&keep_pack, 0);\n \t\t} else {\n-\t\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\t\t\tif (too_many_packs(files, gc_auto_pack_limit)) {\n \t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n \t\t\t\tif (big_pack_threshold) {\n-\t\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n \t\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n \t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n \t\t\t\t\t}\n \t\t\t\t} else {\n-\t\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\t\t\tstruct packed_git *p = find_base_packs(files, &keep_pack, 0);\n \t\t\t\t\tuint64_t mem_have, mem_want;\n \n \t\t\t\t\tmem_have = total_ram();\n-\t\t\t\t\tmem_want = estimate_repack_memory(p);\n+\t\t\t\t\tmem_want = estimate_repack_memory(files, p);\n \n \t\t\t\t\t/*\n \t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n@@ -981,10 +984,10 @@ static int odb_optimize(struct object_database *odb,\n \t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n \t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n \n-\t\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n+\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n \t\t\t\t &geometry.split_factor);\n \n-\t\texisting_packs.repo = the_repository;\n+\t\texisting_packs.repo = odb->repo;\n \t\texisting_packs_collect(&existing_packs, &kept_packs);\n \t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n \t\tpack_geometry_split(&geometry);\n@@ -995,7 +998,7 @@ static int odb_optimize(struct object_database *odb,\n \t\t} else {\n \t\t\tadd_repack_all_option(opts, NULL, &repack_cmd.args);\n \t\t}\n-\t\tif (the_repository->settings.core_multi_pack_index)\n+\t\tif (odb->repo->settings.core_multi_pack_index)\n \t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n \n \t\texisting_packs_release(&existing_packs);\n@@ -1020,7 +1023,7 @@ static int odb_optimize(struct object_database *odb,\n \t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n \t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n \t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n-\t\tif (repo_has_promisor_remote(the_repository))\n+\t\tif (repo_has_promisor_remote(odb->repo))\n \t\t\tstrvec_push(&prune_cmd.args,\n \t\t\t\t \"--exclude-promisor-objects\");\n \t\tprune_cmd.git_cmd = 1;\n@@ -1031,7 +1034,7 @@ static int odb_optimize(struct object_database *odb,\n \t\t}\n \t}\n \n-\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(gc_auto_threshold))\n+\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(files, gc_auto_threshold))\n \t\twarning(_(\"There are too many unreachable loose objects; \"\n \t\t\t\"run 'git prune' to remove them.\"));\n \n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547351","messageId":"20260707-b4-pks-odb-optimize-v1-10-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 10/11] builtin/gc: fix signedness issues in ODB-related functionality","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:42Z","receivedAt":"2026-07-07T15:33:21Z","isPatch":true,"body":"There are a couple of signedness issues in ODB-related functionality.\nThese are not a problem because we disable -Wsign-compare in this file,\nbut once we move these functions into \"odb/source-files.c\" they will\nresult in warnings.\n\nFix those issues:\n\n - In `too_many_loose_objects()` we receive a signed limit, but compare\n it with the unsigned actual number of loose objects. This is fixed\n by bailing out immediately when the limit is smaller than or equal\n to zero, which we also do similarly in other places. The warning is\n then squelched via a cast.\n\n - In `find_base_packs()` we compare the signed size of the pack\n against the unsigned limit. As the pack size is always going to be a\n positive file size it's safe to cast it to an unsigned value.\n\n - In `odb_optimize()` we compare the unsigned `keep_pack.nr` value\n against the signed `gc_auto_pack_limit`. We only reach this code\n when `too_many_packs()` returns true-ish, and that can only happen\n when `gc_auto_pack_limit > 0`. Consequently, we can fix the warning\n by casting the limit to an unsigned value.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 20 +++++++++++---------\n 1 file changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 3207182488..8cf3781313 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -430,19 +430,21 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n \n static int too_many_loose_objects(struct odb_source_files *files, int limit)\n {\n-\t/*\n-\t * This is weird, but stems from legacy behaviour: the GC auto\n-\t * threshold was always essentially interpreted as if it was rounded up\n-\t * to the next multiple 256 of, so we retain this behaviour for now.\n-\t */\n-\tint auto_threshold = DIV_ROUND_UP(limit, 256) * 256;\n \tunsigned long loose_count;\n \n+\tif (limit <= 0)\n+\t\treturn 0;\n+\n \tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n \t\t\t\t &loose_count) < 0)\n \t\treturn 0;\n \n-\treturn loose_count > auto_threshold;\n+\t/*\n+\t * This is weird, but stems from legacy behaviour: the GC auto\n+\t * threshold was always essentially interpreted as if it was rounded up\n+\t * to the next multiple 256 of, so we retain this behaviour for now.\n+\t */\n+\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n }\n \n static struct packed_git *find_base_packs(struct odb_source_files *files,\n@@ -456,7 +458,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,\n \t\tif (e->pack->is_cruft)\n \t\t\tcontinue;\n \t\tif (limit) {\n-\t\t\tif (e->pack->pack_size >= limit)\n+\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n \t\t\t\tstring_list_append(packs, e->pack->pack_name);\n \t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n \t\t\tbase = e->pack;\n@@ -946,7 +948,7 @@ static int odb_optimize(struct object_database *odb,\n \n \t\t\t\tif (big_pack_threshold) {\n \t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n-\t\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n+\t\t\t\t\tif (keep_pack.nr >= (unsigned long) gc_auto_pack_limit) {\n \t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n \t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n \t\t\t\t\t}\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547352","messageId":"20260707-b4-pks-odb-optimize-v1-11-aae607667be4@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH 11/11] odb: make optimizations pluggable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-07T15:32:43Z","receivedAt":"2026-07-07T15:33:24Z","isPatch":true,"body":"Move `odb_optimize()` and `odb_optimize_required()` from \"builtin/gc.c\"\ninto the \"files\" source and wire them up via newly introduced vtable\npointers for the object database sources. This makes the logic pluggable\nand thus allows other backends to have their own, custom implementation.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 490 +----------------------------------------------------\n odb.c | 12 ++\n odb.h | 45 +++++\n odb/source-files.c | 470 ++++++++++++++++++++++++++++++++++++++++++++++++++\n odb/source-files.h | 15 ++\n odb/source.h | 36 ++++\n 6 files changed, 579 insertions(+), 489 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 8cf3781313..ac1a21e912 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -30,16 +30,11 @@\n #include \"commit-graph.h\"\n #include \"packfile.h\"\n #include \"object-file.h\"\n-#include \"pack.h\"\n-#include \"pack-objects.h\"\n+#include \"odb.h\"\n #include \"path.h\"\n #include \"reflog.h\"\n-#include \"repack.h\"\n #include \"rerere.h\"\n #include \"revision.h\"\n-#include \"blob.h\"\n-#include \"tree.h\"\n-#include \"promisor-remote.h\"\n #include \"refs.h\"\n #include \"remote.h\"\n #include \"exec-cmd.h\"\n@@ -428,203 +423,6 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n \treturn should_gc;\n }\n \n-static int too_many_loose_objects(struct odb_source_files *files, int limit)\n-{\n-\tunsigned long loose_count;\n-\n-\tif (limit <= 0)\n-\t\treturn 0;\n-\n-\tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n-\t\t\t\t &loose_count) < 0)\n-\t\treturn 0;\n-\n-\t/*\n-\t * This is weird, but stems from legacy behaviour: the GC auto\n-\t * threshold was always essentially interpreted as if it was rounded up\n-\t * to the next multiple 256 of, so we retain this behaviour for now.\n-\t */\n-\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n-}\n-\n-static struct packed_git *find_base_packs(struct odb_source_files *files,\n-\t\t\t\t\t struct string_list *packs,\n-\t\t\t\t\t unsigned long limit)\n-{\n-\tstruct packfile_list_entry *e;\n-\tstruct packed_git *base = NULL;\n-\n-\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n-\t\tif (e->pack->is_cruft)\n-\t\t\tcontinue;\n-\t\tif (limit) {\n-\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n-\t\t\t\tstring_list_append(packs, e->pack->pack_name);\n-\t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n-\t\t\tbase = e->pack;\n-\t\t}\n-\t}\n-\n-\tif (base)\n-\t\tstring_list_append(packs, base->pack_name);\n-\n-\treturn base;\n-}\n-\n-static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit)\n-{\n-\tstruct packfile_list_entry *e;\n-\tint cnt = 0;\n-\n-\tif (gc_auto_pack_limit <= 0)\n-\t\treturn 0;\n-\n-\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n-\t\tif (e->pack->pack_keep)\n-\t\t\tcontinue;\n-\t\t/*\n-\t\t * Perhaps check the size of the pack and count only\n-\t\t * very small ones here?\n-\t\t */\n-\t\tcnt++;\n-\t}\n-\treturn gc_auto_pack_limit < cnt;\n-}\n-\n-static uint64_t total_ram(void)\n-{\n-#if defined(HAVE_SYSINFO)\n-\tstruct sysinfo si;\n-\n-\tif (!sysinfo(&si)) {\n-\t\tuint64_t total = si.totalram;\n-\n-\t\tif (si.mem_unit > 1)\n-\t\t\ttotal *= (uint64_t)si.mem_unit;\n-\t\treturn total;\n-\t}\n-#elif defined(HAVE_BSD_SYSCTL) && (defined(HW_MEMSIZE) || defined(HW_PHYSMEM) || defined(HW_PHYSMEM64))\n-\tuint64_t physical_memory;\n-\tint mib[2];\n-\tsize_t length;\n-\n-\tmib[0] = CTL_HW;\n-# if defined(HW_MEMSIZE)\n-\tmib[1] = HW_MEMSIZE;\n-# elif defined(HW_PHYSMEM64)\n-\tmib[1] = HW_PHYSMEM64;\n-# else\n-\tmib[1] = HW_PHYSMEM;\n-# endif\n-\tlength = sizeof(physical_memory);\n-\tif (!sysctl(mib, 2, &physical_memory, &length, NULL, 0)) {\n-\t\tif (length == 4) {\n-\t\t\tuint32_t mem;\n-\n-\t\t\tif (!sysctl(mib, 2, &mem, &length, NULL, 0))\n-\t\t\t\tphysical_memory = mem;\n-\t\t}\n-\t\treturn physical_memory;\n-\t}\n-#elif defined(GIT_WINDOWS_NATIVE)\n-\tMEMORYSTATUSEX memInfo;\n-\n-\tmemInfo.dwLength = sizeof(MEMORYSTATUSEX);\n-\tif (GlobalMemoryStatusEx(&memInfo))\n-\t\treturn memInfo.ullTotalPhys;\n-#endif\n-\treturn 0;\n-}\n-\n-static uint64_t estimate_repack_memory(struct odb_source_files *files,\n-\t\t\t\t struct packed_git *pack)\n-{\n-\tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n-\tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n-\tunsigned long nr_objects;\n-\tsize_t os_cache, heap;\n-\n-\tif (odb_source_count_objects(&files->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n-\t\t\t\t &nr_objects) < 0)\n-\t\treturn 0;\n-\n-\tif (!pack || !nr_objects)\n-\t\treturn 0;\n-\n-\trepo_config_get_ulong(the_repository, \"pack.deltacachesize\", &max_delta_cache_size);\n-\trepo_config_get_ulong(the_repository, \"core.deltabasecachelimit\", &delta_base_cache_limit);\n-\n-\t/*\n-\t * First we have to scan through at least one pack.\n-\t * Assume enough room in OS file cache to keep the entire pack\n-\t * or we may accidentally evict data of other processes from\n-\t * the cache.\n-\t */\n-\tos_cache = pack->pack_size + pack->index_size;\n-\t/* then pack-objects needs lots more for book keeping */\n-\theap = sizeof(struct object_entry) * nr_objects;\n-\t/*\n-\t * internal rev-list --all --objects takes up some memory too,\n-\t * let's say half of it is for blobs\n-\t */\n-\theap += sizeof(struct blob) * nr_objects / 2;\n-\t/*\n-\t * and the other half is for trees (commits and tags are\n-\t * usually insignificant)\n-\t */\n-\theap += sizeof(struct tree) * nr_objects / 2;\n-\t/* and then obj_hash[], underestimated in fact */\n-\theap += sizeof(struct object *) * nr_objects;\n-\t/* revindex is used also */\n-\theap += (sizeof(off_t) + sizeof(uint32_t)) * nr_objects;\n-\t/*\n-\t * read_sha1_file() (either at delta calculation phase, or\n-\t * writing phase) also fills up the delta base cache\n-\t */\n-\theap += delta_base_cache_limit;\n-\t/* and of course pack-objects has its own delta cache */\n-\theap += max_delta_cache_size;\n-\n-\treturn os_cache + heap;\n-}\n-\n-static int keep_one_pack(struct string_list_item *item, void *data)\n-{\n-\tstruct strvec *args = data;\n-\tstrvec_pushf(args, \"--keep-pack=%s\", basename(item->string));\n-\treturn 0;\n-}\n-\n-enum odb_optimize_strategy {\n-\tODB_OPTIMIZE_INCREMENTAL,\n-\tODB_OPTIMIZE_GEOMETRIC,\n-};\n-\n-enum odb_optimize_flags {\n-\t/* Enable verbose logging and progress reporting. */\n-\tODB_OPTIMIZE_VERBOSE = (1 << 0),\n-\n-\t/* Perform auto-maintenance, only optimizing objects as required. */\n-\tODB_OPTIMIZE_AUTO = (1 << 1),\n-\n-\t/* Recompute existing deltas. */\n-\tODB_OPTIMIZE_NO_REUSE_DELTAS = (1 << 2),\n-};\n-\n-struct odb_optimize_options {\n-\tenum odb_optimize_strategy strategy;\n-\tenum odb_optimize_flags flags;\n-\tconst char *prune_expire;\n-\tconst char *expire_to;\n-\tint depth;\n-\tint window;\n-\n-\t/* Backend-specific options. */\n-\tint keep_largest_pack;\n-\tint cruft_packs;\n-\tunsigned long max_cruft_size;\n-};\n-\n #define OPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive) \\\n \t.prune_expire = (cfg)->prune_expire, \\\n \t.expire_to = (cfg)->repack_expire_to, \\\n@@ -633,133 +431,6 @@ struct odb_optimize_options {\n \t.window = (aggressive) ? (cfg)->aggressive_window : 0, \\\n \t.depth = (aggressive) ? (cfg)->aggressive_depth : 0\n \n-static void add_repack_all_option(const struct odb_optimize_options *opts,\n-\t\t\t\t struct string_list *keep_pack,\n-\t\t\t\t struct strvec *args)\n-{\n-\tchar *repack_filter = NULL;\n-\tchar *repack_filter_to = NULL;\n-\n-\trepo_config_get_string(the_repository, \"gc.repackfilter\", &repack_filter);\n-\trepo_config_get_string(the_repository, \"gc.repackfilterto\", &repack_filter_to);\n-\n-\tif (opts->prune_expire && !strcmp(opts->prune_expire, \"now\") &&\n-\t !(opts->cruft_packs && opts->expire_to))\n-\t\tstrvec_push(args, \"-a\");\n-\telse if (opts->cruft_packs) {\n-\t\tstrvec_push(args, \"--cruft\");\n-\t\tif (opts->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", opts->prune_expire);\n-\t\tif (opts->max_cruft_size)\n-\t\t\tstrvec_pushf(args, \"--max-cruft-size=%lu\",\n-\t\t\t\t opts->max_cruft_size);\n-\t\tif (opts->expire_to)\n-\t\t\tstrvec_pushf(args, \"--expire-to=%s\", opts->expire_to);\n-\t} else {\n-\t\tstrvec_push(args, \"-A\");\n-\t\tif (opts->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", opts->prune_expire);\n-\t}\n-\n-\tif (keep_pack)\n-\t\tfor_each_string_list(keep_pack, keep_one_pack, args);\n-\n-\tif (repack_filter && *repack_filter)\n-\t\tstrvec_pushf(args, \"--filter=%s\", repack_filter);\n-\tif (repack_filter_to && *repack_filter_to)\n-\t\tstrvec_pushf(args, \"--filter-to=%s\", repack_filter_to);\n-\n-\tfree(repack_filter);\n-\tfree(repack_filter_to);\n-}\n-\n-static void add_repack_incremental_option(struct strvec *args)\n-{\n-\tstrvec_push(args, \"--no-write-bitmap-index\");\n-}\n-\n-static bool odb_optimize_required(struct object_database *odb,\n-\t\t\t\t const struct odb_optimize_options *opts)\n-{\n-\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n-\n-\tswitch (opts->strategy) {\n-\tcase ODB_OPTIMIZE_INCREMENTAL: {\n-\t\tint gc_auto_threshold = 6700;\n-\t\tint gc_auto_pack_limit = 50;\n-\n-\t\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n-\t\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n-\n-\t\t/*\n-\t\t * Setting gc.auto to 0 or negative can disable the\n-\t\t * automatic gc.\n-\t\t */\n-\t\tif (gc_auto_threshold <= 0)\n-\t\t\treturn false;\n-\t\tif (!too_many_packs(files, gc_auto_pack_limit) &&\n-\t\t !too_many_loose_objects(files, gc_auto_threshold))\n-\t\t\treturn false;\n-\n-\t\treturn true;\n-\t}\n-\tcase ODB_OPTIMIZE_GEOMETRIC: {\n-\t\tstruct pack_geometry geometry = {\n-\t\t\t.split_factor = 2,\n-\t\t};\n-\t\tstruct pack_objects_args po_args = {\n-\t\t\t.local = 1,\n-\t\t};\n-\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\t\tint auto_value = 100;\n-\t\tbool ret;\n-\n-\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.auto\",\n-\t\t\t\t &auto_value);\n-\t\tif (!auto_value)\n-\t\t\treturn false;\n-\t\tif (auto_value < 0)\n-\t\t\treturn true;\n-\n-\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t\t &geometry.split_factor);\n-\n-\t\texisting_packs.repo = odb->repo;\n-\t\texisting_packs_collect(&existing_packs, &kept_packs);\n-\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\t\tpack_geometry_split(&geometry);\n-\n-\t\t/*\n-\t\t * When we'd merge at least two packs with one another we always\n-\t\t * perform the repack.\n-\t\t */\n-\t\tif (geometry.split) {\n-\t\t\tret = true;\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\t/*\n-\t\t * Otherwise, we estimate the number of loose objects to determine\n-\t\t * whether we want to create a new packfile or not.\n-\t\t */\n-\t\tif (too_many_loose_objects(files, auto_value)) {\n-\t\t\tret = true;\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tret = false;\n-\n-\tout:\n-\t\texisting_packs_release(&existing_packs);\n-\t\tpack_geometry_release(&geometry);\n-\t\treturn ret;\n-\t}\n-\tdefault:\n-\t\tBUG(\"unknown maintenance strategy '%d'\", opts->strategy);\n-\t}\n-}\n-\n /* return NULL on success, else hostname running the gc */\n static const char *lock_repo_for_gc(int force, pid_t* ret_pid)\n {\n@@ -887,165 +558,6 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \treturn 0;\n }\n \n-static int odb_optimize(struct object_database *odb,\n-\t\t\tconst struct odb_optimize_options *opts)\n-{\n-\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n-\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n-\tunsigned long big_pack_threshold = 0;\n-\tint gc_auto_threshold = 6700;\n-\tint gc_auto_pack_limit = 50;\n-\tint ret;\n-\n-\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n-\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n-\trepo_config_get_ulong(odb->repo, \"gc.bigpackthreshold\", &big_pack_threshold);\n-\n-\tif (odb->repo->repository_format_precious_objects)\n-\t\treturn 0;\n-\n-\trepack_cmd.git_cmd = 1;\n-\trepack_cmd.odb_to_close = odb->repo->objects;\n-\n-\tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n-\t\tstrvec_push(&repack_cmd.args, \"-f\");\n-\tif (opts->depth > 0)\n-\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", opts->depth);\n-\tif (opts->window > 0)\n-\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", opts->window);\n-\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n-\t\tstrvec_push(&repack_cmd.args, \"-q\");\n-\n-\t/*\n-\t * There's three cases we need to consider:\n-\t *\n-\t * - If we're invoked without `--auto` we'll need to perform a full\n-\t * repack.\n-\t *\n-\t * - If we're invoked with `--auto` and there's too many packs, then\n-\t * we perform a full repack, as well.\n-\t *\n-\t * - Otherwise we perform an incremental repack.\n-\t */\n-\tswitch (opts->strategy) {\n-\tcase ODB_OPTIMIZE_INCREMENTAL:\n-\t\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n-\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\t\tif (opts->keep_largest_pack != -1) {\n-\t\t\t\tif (opts->keep_largest_pack)\n-\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n-\t\t\t} else if (big_pack_threshold) {\n-\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n-\t\t\t}\n-\n-\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n-\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t} else {\n-\t\t\tif (too_many_packs(files, gc_auto_pack_limit)) {\n-\t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\t\t\tif (big_pack_threshold) {\n-\t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n-\t\t\t\t\tif (keep_pack.nr >= (unsigned long) gc_auto_pack_limit) {\n-\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n-\t\t\t\t\t}\n-\t\t\t\t} else {\n-\t\t\t\t\tstruct packed_git *p = find_base_packs(files, &keep_pack, 0);\n-\t\t\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\t\t\tmem_have = total_ram();\n-\t\t\t\t\tmem_want = estimate_repack_memory(files, p);\n-\n-\t\t\t\t\t/*\n-\t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t\t\t * the rest for the OS and other processes in the\n-\t\t\t\t\t * system.\n-\t\t\t\t\t */\n-\t\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\t}\n-\n-\t\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t} else {\n-\t\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n-\t\t\t}\n-\t\t}\n-\n-\t\tbreak;\n-\tcase ODB_OPTIMIZE_GEOMETRIC: {\n-\t\tstruct pack_geometry geometry = {\n-\t\t\t.split_factor = 2,\n-\t\t};\n-\t\tstruct pack_objects_args po_args = {\n-\t\t\t.local = 1,\n-\t\t};\n-\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\n-\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t\t &geometry.split_factor);\n-\n-\t\texisting_packs.repo = odb->repo;\n-\t\texisting_packs_collect(&existing_packs, &kept_packs);\n-\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\t\tpack_geometry_split(&geometry);\n-\n-\t\tif (geometry.split < geometry.pack_nr) {\n-\t\t\tstrvec_pushf(&repack_cmd.args, \"--geometric=%d\",\n-\t\t\t\t geometry.split_factor);\n-\t\t} else {\n-\t\t\tadd_repack_all_option(opts, NULL, &repack_cmd.args);\n-\t\t}\n-\t\tif (odb->repo->settings.core_multi_pack_index)\n-\t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n-\n-\t\texisting_packs_release(&existing_packs);\n-\t\tpack_geometry_release(&geometry);\n-\t\tbreak;\n-\t}\n-\tdefault:\n-\t\tdie(\"unknown maintenance strategy '%d'\", opts->strategy);\n-\t}\n-\n-\tif (run_command(&repack_cmd)) {\n-\t\tret = error(FAILED_RUN, repack_cmd.args.v[0]);\n-\t\tgoto out;\n-\t}\n-\n-\t/* Geometric repacking uses cruft packs, so we don't have to prune separately. */\n-\tif (opts->strategy != ODB_OPTIMIZE_GEOMETRIC && opts->prune_expire) {\n-\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n-\n-\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n-\t\t/* run `git prune` even if using cruft packs */\n-\t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n-\t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n-\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n-\t\tif (repo_has_promisor_remote(odb->repo))\n-\t\t\tstrvec_push(&prune_cmd.args,\n-\t\t\t\t \"--exclude-promisor-objects\");\n-\t\tprune_cmd.git_cmd = 1;\n-\n-\t\tif (run_command(&prune_cmd)) {\n-\t\t\tret = error(FAILED_RUN, prune_cmd.args.v[0]);\n-\t\t\tgoto out;\n-\t\t}\n-\t}\n-\n-\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(files, gc_auto_threshold))\n-\t\twarning(_(\"There are too many unreachable loose objects; \"\n-\t\t\t\"run 'git prune' to remove them.\"));\n-\n-\tret = 0;\n-\n-out:\n-\treturn ret;\n-}\n-\n static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tstruct gc_config *cfg,\n \t\t\t\tint keep_largest_pack,\ndiff --git a/odb.c b/odb.c\nindex 7d555be09f..89660981fe 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -1003,6 +1003,18 @@ int odb_write_object_stream(struct object_database *odb,\n \treturn odb_source_write_object_stream(odb->sources, stream, len, oid);\n }\n \n+int odb_optimize(struct object_database *odb,\n+\t\t const struct odb_optimize_options *opts)\n+{\n+\treturn odb_source_optimize(odb->sources, opts);\n+}\n+\n+bool odb_optimize_required(struct object_database *odb,\n+\t\t\t const struct odb_optimize_options *opts)\n+{\n+\treturn odb_source_optimize_required(odb->sources, opts);\n+}\n+\n struct object_database *odb_new(struct repository *repo,\n \t\t\t\tconst char *primary_source,\n \t\t\t\tconst char *secondary_sources)\ndiff --git a/odb.h b/odb.h\nindex 3834a0dcbf..7e1c85c22e 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -117,6 +117,51 @@ struct object_database *odb_new(struct repository *repo,\n /* Free the object database and release all resources. */\n void odb_free(struct object_database *o);\n \n+enum odb_optimize_strategy {\n+\tODB_OPTIMIZE_INCREMENTAL,\n+\tODB_OPTIMIZE_GEOMETRIC,\n+};\n+\n+enum odb_optimize_flags {\n+\t/* Enable verbose logging and progress reporting. */\n+\tODB_OPTIMIZE_VERBOSE = (1 << 0),\n+\n+\t/* Perform auto-maintenance, only optimizing objects as required. */\n+\tODB_OPTIMIZE_AUTO = (1 << 1),\n+\n+\t/* Recompute existing deltas. */\n+\tODB_OPTIMIZE_NO_REUSE_DELTAS = (1 << 2),\n+};\n+\n+struct odb_optimize_options {\n+\tenum odb_optimize_strategy strategy;\n+\tenum odb_optimize_flags flags;\n+\tconst char *prune_expire;\n+\tconst char *expire_to;\n+\tint depth;\n+\tint window;\n+\n+\t/* Backend-specific options. */\n+\tint keep_largest_pack;\n+\tint cruft_packs;\n+\tunsigned long max_cruft_size;\n+};\n+\n+/*\n+ * Optimize the object database. Returns 0 on success, a negative error code\n+ * otherwise.\n+ */\n+int odb_optimize(struct object_database *odb,\n+\t\t const struct odb_optimize_options *opts);\n+\n+/*\n+ * Check whether optimization of the object database is required given the\n+ * provided options. Returns true if optimization should be performed, false\n+ * otherwise.\n+ */\n+bool odb_optimize_required(struct object_database *odb,\n+\t\t\t const struct odb_optimize_options *opts);\n+\n /*\n * Close the object database and all of its sources so that any held resources\n * will be released. The database can still be used after closing it, in which\ndiff --git a/odb/source-files.c b/odb/source-files.c\nindex bbd1784b33..82cf61da4a 100644\n--- a/odb/source-files.c\n+++ b/odb/source-files.c\n@@ -1,6 +1,8 @@\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n+#include \"blob.h\"\n #include \"chdir-notify.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"lockfile.h\"\n #include \"object-file.h\"\n@@ -8,8 +10,16 @@\n #include \"odb/source.h\"\n #include \"odb/source-files.h\"\n #include \"odb/source-loose.h\"\n+#include \"pack-objects.h\"\n #include \"packfile.h\"\n+#include \"path.h\"\n+#include \"promisor-remote.h\"\n+#include \"repack.h\"\n+#include \"run-command.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n+#include \"strvec.h\"\n+#include \"tree.h\"\n #include \"write-or-die.h\"\n \n static void odb_source_files_reparent(const char *name UNUSED,\n@@ -260,6 +270,464 @@ static int odb_source_files_write_alternate(struct odb_source *source,\n \treturn ret;\n }\n \n+static int too_many_loose_objects(struct odb_source_files *files, int limit)\n+{\n+\tunsigned long loose_count;\n+\n+\tif (limit <= 0)\n+\t\treturn 0;\n+\n+\tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n+\t\t\t\t &loose_count) < 0)\n+\t\treturn 0;\n+\n+\t/*\n+\t * This is weird, but stems from legacy behaviour: the GC auto\n+\t * threshold was always essentially interpreted as if it was rounded up\n+\t * to the next multiple 256 of, so we retain this behaviour for now.\n+\t */\n+\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n+}\n+\n+static struct packed_git *find_base_packs(struct odb_source_files *files,\n+\t\t\t\t\t struct string_list *packs,\n+\t\t\t\t\t unsigned long limit)\n+{\n+\tstruct packfile_list_entry *e;\n+\tstruct packed_git *base = NULL;\n+\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->is_cruft)\n+\t\t\tcontinue;\n+\t\tif (limit) {\n+\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n+\t\t\t\tstring_list_append(packs, e->pack->pack_name);\n+\t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n+\t\t\tbase = e->pack;\n+\t\t}\n+\t}\n+\n+\tif (base)\n+\t\tstring_list_append(packs, base->pack_name);\n+\n+\treturn base;\n+}\n+\n+static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit)\n+{\n+\tstruct packfile_list_entry *e;\n+\tint cnt = 0;\n+\n+\tif (gc_auto_pack_limit <= 0)\n+\t\treturn 0;\n+\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->pack_keep)\n+\t\t\tcontinue;\n+\t\t/*\n+\t\t * Perhaps check the size of the pack and count only\n+\t\t * very small ones here?\n+\t\t */\n+\t\tcnt++;\n+\t}\n+\treturn gc_auto_pack_limit < cnt;\n+}\n+\n+static uint64_t total_ram(void)\n+{\n+#if defined(HAVE_SYSINFO)\n+\tstruct sysinfo si;\n+\n+\tif (!sysinfo(&si)) {\n+\t\tuint64_t total = si.totalram;\n+\n+\t\tif (si.mem_unit > 1)\n+\t\t\ttotal *= (uint64_t)si.mem_unit;\n+\t\treturn total;\n+\t}\n+#elif defined(HAVE_BSD_SYSCTL) && (defined(HW_MEMSIZE) || defined(HW_PHYSMEM) || defined(HW_PHYSMEM64))\n+\tuint64_t physical_memory;\n+\tint mib[2];\n+\tsize_t length;\n+\n+\tmib[0] = CTL_HW;\n+# if defined(HW_MEMSIZE)\n+\tmib[1] = HW_MEMSIZE;\n+# elif defined(HW_PHYSMEM64)\n+\tmib[1] = HW_PHYSMEM64;\n+# else\n+\tmib[1] = HW_PHYSMEM;\n+# endif\n+\tlength = sizeof(physical_memory);\n+\tif (!sysctl(mib, 2, &physical_memory, &length, NULL, 0)) {\n+\t\tif (length == 4) {\n+\t\t\tuint32_t mem;\n+\n+\t\t\tif (!sysctl(mib, 2, &mem, &length, NULL, 0))\n+\t\t\t\tphysical_memory = mem;\n+\t\t}\n+\t\treturn physical_memory;\n+\t}\n+#elif defined(GIT_WINDOWS_NATIVE)\n+\tMEMORYSTATUSEX memInfo;\n+\n+\tmemInfo.dwLength = sizeof(MEMORYSTATUSEX);\n+\tif (GlobalMemoryStatusEx(&memInfo))\n+\t\treturn memInfo.ullTotalPhys;\n+#endif\n+\treturn 0;\n+}\n+\n+static uint64_t estimate_repack_memory(struct odb_source_files *files,\n+\t\t\t\t struct packed_git *pack)\n+{\n+\tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n+\tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n+\tunsigned long nr_objects;\n+\tsize_t os_cache, heap;\n+\n+\tif (odb_source_count_objects(&files->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n+\t\t\t\t &nr_objects) < 0)\n+\t\treturn 0;\n+\n+\tif (!pack || !nr_objects)\n+\t\treturn 0;\n+\n+\trepo_config_get_ulong(files->base.odb->repo, \"pack.deltacachesize\",\n+\t\t\t &max_delta_cache_size);\n+\trepo_config_get_ulong(files->base.odb->repo, \"core.deltabasecachelimit\",\n+\t\t\t &delta_base_cache_limit);\n+\n+\t/*\n+\t * First we have to scan through at least one pack.\n+\t * Assume enough room in OS file cache to keep the entire pack\n+\t * or we may accidentally evict data of other processes from\n+\t * the cache.\n+\t */\n+\tos_cache = pack->pack_size + pack->index_size;\n+\t/* then pack-objects needs lots more for book keeping */\n+\theap = sizeof(struct object_entry) * nr_objects;\n+\t/*\n+\t * internal rev-list --all --objects takes up some memory too,\n+\t * let's say half of it is for blobs\n+\t */\n+\theap += sizeof(struct blob) * nr_objects / 2;\n+\t/*\n+\t * and the other half is for trees (commits and tags are\n+\t * usually insignificant)\n+\t */\n+\theap += sizeof(struct tree) * nr_objects / 2;\n+\t/* and then obj_hash[], underestimated in fact */\n+\theap += sizeof(struct object *) * nr_objects;\n+\t/* revindex is used also */\n+\theap += (sizeof(off_t) + sizeof(uint32_t)) * nr_objects;\n+\t/*\n+\t * read_sha1_file() (either at delta calculation phase, or\n+\t * writing phase) also fills up the delta base cache\n+\t */\n+\theap += delta_base_cache_limit;\n+\t/* and of course pack-objects has its own delta cache */\n+\theap += max_delta_cache_size;\n+\n+\treturn os_cache + heap;\n+}\n+\n+static int keep_one_pack(struct string_list_item *item, void *data)\n+{\n+\tstruct strvec *args = data;\n+\tstrvec_pushf(args, \"--keep-pack=%s\", basename(item->string));\n+\treturn 0;\n+}\n+\n+static void add_repack_all_option(struct repository *repo,\n+\t\t\t\t const struct odb_optimize_options *opts,\n+\t\t\t\t struct string_list *keep_pack,\n+\t\t\t\t struct strvec *args)\n+{\n+\tchar *repack_filter = NULL;\n+\tchar *repack_filter_to = NULL;\n+\n+\trepo_config_get_string(repo, \"gc.repackfilter\", &repack_filter);\n+\trepo_config_get_string(repo, \"gc.repackfilterto\", &repack_filter_to);\n+\n+\tif (opts->prune_expire && !strcmp(opts->prune_expire, \"now\") &&\n+\t !(opts->cruft_packs && opts->expire_to))\n+\t\tstrvec_push(args, \"-a\");\n+\telse if (opts->cruft_packs) {\n+\t\tstrvec_push(args, \"--cruft\");\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", opts->prune_expire);\n+\t\tif (opts->max_cruft_size)\n+\t\t\tstrvec_pushf(args, \"--max-cruft-size=%lu\",\n+\t\t\t\t opts->max_cruft_size);\n+\t\tif (opts->expire_to)\n+\t\t\tstrvec_pushf(args, \"--expire-to=%s\", opts->expire_to);\n+\t} else {\n+\t\tstrvec_push(args, \"-A\");\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", opts->prune_expire);\n+\t}\n+\n+\tif (keep_pack)\n+\t\tfor_each_string_list(keep_pack, keep_one_pack, args);\n+\n+\tif (repack_filter && *repack_filter)\n+\t\tstrvec_pushf(args, \"--filter=%s\", repack_filter);\n+\tif (repack_filter_to && *repack_filter_to)\n+\t\tstrvec_pushf(args, \"--filter-to=%s\", repack_filter_to);\n+\n+\tfree(repack_filter);\n+\tfree(repack_filter_to);\n+}\n+\n+static void add_repack_incremental_option(struct strvec *args)\n+{\n+\tstrvec_push(args, \"--no-write-bitmap-index\");\n+}\n+\n+bool odb_source_files_optimize_required(struct odb_source *source,\n+\t\t\t\t\tconst struct odb_optimize_options *opts)\n+{\n+\tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\tstruct repository *repo = source->odb->repo;\n+\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL: {\n+\t\tint gc_auto_threshold = 6700;\n+\t\tint gc_auto_pack_limit = 50;\n+\n+\t\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n+\t\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\n+\t\t/*\n+\t\t * Setting gc.auto to 0 or negative can disable the\n+\t\t * automatic gc.\n+\t\t */\n+\t\tif (gc_auto_threshold <= 0)\n+\t\t\treturn false;\n+\t\tif (!too_many_packs(files, gc_auto_pack_limit) &&\n+\t\t !too_many_loose_objects(files, gc_auto_threshold))\n+\t\t\treturn false;\n+\n+\t\treturn true;\n+\t}\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\t\tint auto_value = 100;\n+\t\tbool ret;\n+\n+\t\trepo_config_get_int(repo, \"maintenance.geometric-repack.auto\",\n+\t\t\t\t &auto_value);\n+\t\tif (!auto_value)\n+\t\t\treturn false;\n+\t\tif (auto_value < 0)\n+\t\t\treturn true;\n+\n+\t\trepo_config_get_int(repo, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = repo;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\t/*\n+\t\t * When we'd merge at least two packs with one another we always\n+\t\t * perform the repack.\n+\t\t */\n+\t\tif (geometry.split) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\t/*\n+\t\t * Otherwise, we estimate the number of loose objects to determine\n+\t\t * whether we want to create a new packfile or not.\n+\t\t */\n+\t\tif (too_many_loose_objects(files, auto_value)) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tret = false;\n+\n+\tout:\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\treturn ret;\n+\t}\n+\tdefault:\n+\t\tBUG(\"unknown maintenance strategy '%d'\", opts->strategy);\n+\t}\n+}\n+\n+int odb_source_files_optimize(struct odb_source *source,\n+\t\t\t const struct odb_optimize_options *opts)\n+{\n+\tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\tstruct repository *repo = source->odb->repo;\n+\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n+\tunsigned long big_pack_threshold = 0;\n+\tint gc_auto_threshold = 6700;\n+\tint gc_auto_pack_limit = 50;\n+\tint ret;\n+\n+\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\trepo_config_get_ulong(repo, \"gc.bigpackthreshold\", &big_pack_threshold);\n+\n+\tif (repo->repository_format_precious_objects)\n+\t\treturn 0;\n+\n+\trepack_cmd.git_cmd = 1;\n+\trepack_cmd.odb_to_close = repo->objects;\n+\n+\tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n+\tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n+\t\tstrvec_push(&repack_cmd.args, \"-f\");\n+\tif (opts->depth > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", opts->depth);\n+\tif (opts->window > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", opts->window);\n+\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n+\t\tstrvec_push(&repack_cmd.args, \"-q\");\n+\n+\t/*\n+\t * There's three cases we need to consider:\n+\t *\n+\t * - If we're invoked without `--auto` we'll need to perform a full\n+\t * repack.\n+\t *\n+\t * - If we're invoked with `--auto` and there's too many packs, then\n+\t * we perform a full repack, as well.\n+\t *\n+\t * - Otherwise we perform an incremental repack.\n+\t */\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL:\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n+\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\tif (opts->keep_largest_pack != -1) {\n+\t\t\t\tif (opts->keep_largest_pack)\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n+\t\t\t} else if (big_pack_threshold) {\n+\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n+\t\t\t}\n+\n+\t\t\tadd_repack_all_option(repo, opts, &keep_pack, &repack_cmd.args);\n+\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t} else {\n+\t\t\tif (too_many_packs(files, gc_auto_pack_limit)) {\n+\t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\t\tif (big_pack_threshold) {\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n+\t\t\t\t\tif (keep_pack.nr >= (unsigned long) gc_auto_pack_limit) {\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n+\t\t\t\t\t}\n+\t\t\t\t} else {\n+\t\t\t\t\tstruct packed_git *p = find_base_packs(files, &keep_pack, 0);\n+\t\t\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\t\t\tmem_have = total_ram();\n+\t\t\t\t\tmem_want = estimate_repack_memory(files, p);\n+\n+\t\t\t\t\t/*\n+\t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t\t\t * the rest for the OS and other processes in the\n+\t\t\t\t\t * system.\n+\t\t\t\t\t */\n+\t\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t}\n+\n+\t\t\t\tadd_repack_all_option(repo, opts, &keep_pack, &repack_cmd.args);\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t} else {\n+\t\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t\t\t}\n+\t\t}\n+\n+\t\tbreak;\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\n+\t\trepo_config_get_int(repo, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = repo;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\tif (geometry.split < geometry.pack_nr) {\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--geometric=%d\",\n+\t\t\t\t geometry.split_factor);\n+\t\t} else {\n+\t\t\tadd_repack_all_option(repo, opts, NULL, &repack_cmd.args);\n+\t\t}\n+\t\tif (repo->settings.core_multi_pack_index)\n+\t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n+\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tdie(\"unknown maintenance strategy '%d'\", opts->strategy);\n+\t}\n+\n+\tif (run_command(&repack_cmd)) {\n+\t\tret = error(\"failed to run %s\", repack_cmd.args.v[0]);\n+\t\tgoto out;\n+\t}\n+\n+\t/* Geometric repacking uses cruft packs, so we don't have to prune separately. */\n+\tif (opts->strategy != ODB_OPTIMIZE_GEOMETRIC && opts->prune_expire) {\n+\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n+\n+\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n+\t\t/* run `git prune` even if using cruft packs */\n+\t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n+\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n+\t\tif (repo_has_promisor_remote(repo))\n+\t\t\tstrvec_push(&prune_cmd.args,\n+\t\t\t\t \"--exclude-promisor-objects\");\n+\t\tprune_cmd.git_cmd = 1;\n+\n+\t\tif (run_command(&prune_cmd)) {\n+\t\t\tret = error(\"failed to run %s\", prune_cmd.args.v[0]);\n+\t\t\tgoto out;\n+\t\t}\n+\t}\n+\n+\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(files, gc_auto_threshold))\n+\t\twarning(_(\"There are too many unreachable loose objects; \"\n+\t\t\t\"run 'git prune' to remove them.\"));\n+\n+\tret = 0;\n+\n+out:\n+\treturn ret;\n+}\n+\n struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \t\t\t\t\t const char *path,\n \t\t\t\t\t bool local)\n@@ -285,6 +753,8 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \tfiles->base.begin_transaction = odb_source_files_begin_transaction;\n \tfiles->base.read_alternates = odb_source_files_read_alternates;\n \tfiles->base.write_alternate = odb_source_files_write_alternate;\n+\tfiles->base.optimize = odb_source_files_optimize;\n+\tfiles->base.optimize_required = odb_source_files_optimize_required;\n \n \t/*\n \t * Ideally, we would only ever store absolute paths in the source. This\ndiff --git a/odb/source-files.h b/odb/source-files.h\nindex d7ac3c1c81..044242bc36 100644\n--- a/odb/source-files.h\n+++ b/odb/source-files.h\n@@ -21,6 +21,21 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \t\t\t\t\t const char *path,\n \t\t\t\t\t bool local);\n \n+/*\n+ * Optimize the files object database source by repacking loose objects and\n+ * packfiles as needed. Returns 0 on success, a negative error code otherwise.\n+ */\n+int odb_source_files_optimize(struct odb_source *source,\n+\t\t\t const struct odb_optimize_options *opts);\n+\n+/*\n+ * Check whether optimization of the files object database source is required\n+ * given the provided options. Returns true if optimization should be\n+ * performed, false otherwise.\n+ */\n+bool odb_source_files_optimize_required(struct odb_source *source,\n+\t\t\t\t\tconst struct odb_optimize_options *opts);\n+\n /*\n * Cast the given object database source to the files backend. This will cause\n * a BUG in case the source doesn't use this backend.\ndiff --git a/odb/source.h b/odb/source.h\nindex 8767708c9c..88a48ba3c3 100644\n--- a/odb/source.h\n+++ b/odb/source.h\n@@ -258,6 +258,21 @@ struct odb_source {\n \t */\n \tint (*write_alternate)(struct odb_source *source,\n \t\t\t const char *alternate);\n+\n+\t/*\n+\t * This callback is expected to optimize the object database source.\n+\t * Returns 0 on success, a negative error code otherwise.\n+\t */\n+\tint (*optimize)(struct odb_source *source,\n+\t\t\tconst struct odb_optimize_options *opts);\n+\n+\t/*\n+\t * This callback is expected to check whether optimization of the\n+\t * object database source is required given the provided options.\n+\t * Returns true if optimization should be performed, false otherwise.\n+\t */\n+\tbool (*optimize_required)(struct odb_source *source,\n+\t\t\t\t const struct odb_optimize_options *opts);\n };\n \n /*\n@@ -475,4 +490,25 @@ static inline int odb_source_begin_transaction(struct odb_source *source,\n \treturn source->begin_transaction(source, out);\n }\n \n+/*\n+ * Optimize the object database source. Returns 0 on success, a negative error\n+ * code otherwise.\n+ */\n+static inline int odb_source_optimize(struct odb_source *source,\n+\t\t\t\t const struct odb_optimize_options *opts)\n+{\n+\treturn source->optimize(source, opts);\n+}\n+\n+/*\n+ * Check whether optimization of the object database source is required given\n+ * the provided options. Returns true if optimization should be performed,\n+ * false otherwise.\n+ */\n+static inline bool odb_source_optimize_required(struct odb_source *source,\n+\t\t\t\t\t\tconst struct odb_optimize_options *opts)\n+{\n+\treturn source->optimize_required(source, opts);\n+}\n+\n #endif\n\n-- \n2.55.0.141.g00534a21ce.dirty\n\n"},{"id":"547393","messageId":"xmqqo6gi1sng.fsf@gitster.g","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-1-aae607667be4@pks.im","subject":"Re: [PATCH 01/11] odb: run \"pre-auto-gc\" hook for all maintenance tasks","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-07T19:55:31Z","receivedAt":"2026-07-07T19:55:33Z","isPatch":true,"body":"Patrick Steinhardt writes:\n\n> While the former makes sense, the latter is somewhat off. While the hook\n> is indeed strongly tied to gc'ing a repository, the original intent of\n> the hook is rather to inhibit any kind of automated garbage collection.\n> That noticeably also includes all the other maintenance tasks that our\n> new infrastructure may run, but those aren't getting intercepted at all.\n\nIf we want to halt object collection right now for some reason, it\nis likely that for the same reason we may want automated pruning of\nold reflog entries, for example. So I can buy the above reasoning.\n\n> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> index d7f82e1bec..1212b306b6 100755\n> --- a/t/t7900-maintenance.sh\n> +++ b/t/t7900-maintenance.sh\n> @@ -740,6 +740,127 @@ test_expect_success 'geometric repacking honors configured split factor' '\n> \t)\n> '\n> \n> +test_expect_success 'pre-auto-gc hook runs exactly once' '\n> +\ttest_when_finished \"rm -rf repo\" &&\n> +\tgit init repo &&\n> +\t(\n> +\t\tcd repo &&\n> +\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n> +\t\techo hook >>hook.log\n> +\t\tEOF\n> +\n> +\t\t# Satisfy the auto condition for multiple tasks, both in the\n> +\t\t# foreground and in the background phase.\n> +\t\tgit config set maintenance.reflog-expire.auto -1 &&\n> +\t\tgit config set maintenance.geometric-repack.auto -1 &&\n> +\t\tgit config set maintenance.rerere-gc.auto -1 &&\n> +\n> +\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n> +\t\t\tgit maintenance run --auto 2>/dev/null &&\n> +\n> +\t\t# The successful hook does not inhibit any of the tasks...\n> +\t\ttest_subcommand git reflog expire --all +\t\ttest_subcommand_flex git repack +\t\ttest_subcommand git rerere gc +\t\t# ... but it must only have been executed a single time.\n> +\t\ttest_line_count = 1 hook.log\n> +\t)\n> +'\n\nSomehow I'd feel better if the hook used a full path to the append\nonly log file, but it is reasonably clear that these three commands\nare unlikely to chdir around, so it may be OK.\n\nObviously not in scope of this topic, but I wonder if we have a\nbetter way to test these three \"housekeeping tasks\" have run, than\ncasting in stone the current implementation that spawns these three\nexternal command as subprocesses.\n\n"},{"id":"547401","messageId":"xmqqbjci1r6z.fsf@gitster.g","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-2-aae607667be4@pks.im","subject":"Re: [PATCH 02/11] builtin/gc: move worktree and rerere tasks before object optimizations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-07T20:27:00Z","receivedAt":"2026-07-07T20:27:02Z","isPatch":true,"body":"Patrick Steinhardt writes:\n\n> In subsequent patches we'll consolidate all tasks that relate to\n> maintenance of the object database and move it into the \"files\" backend.\n> The relevant code is somewhat scattered though, as several other tasks\n> are interspersed between.\n>\n> Refactor the code so that all object database optimizations are grouped\n> together, which requires us to move worktree pruning and rerere garbage\n> collection around. In theory, rearranging this code can have an effect\n> on the object database optimizations:\n>\n> - Rerere entries really shouldn't impact garbage collection at all, as\n> these entries are not stored in the object database.\n>\n> - The index and HEAD reference of pruned worktrees may reference\n> objects that become unreachable.\n\nOver time \"gc\" (and more prominently, \"maintenance\") ceased to be\nabout object database optimization but about general housekeeping\noperations to keep your repository healthy. rerere database,\nreflog, packed-refs and reftable compaction are all outside the\nscope of object database optimization. Grouping these inside the\numbrella \"gc/maintenance\" framework would be a good first step to\nmake parts of them pluggable.\n\n> That being said, the impact should be overall rather negligible. If the\n> user was asking us to prune objects with immediate expiration time then\n> we might now prune objects that were previously still kept alive by the\n> worktree. But besides being a very specific edge case, it's arguably not\n> even the wrong thing to also prune any potentially-unreachable objects\n> immediately.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/gc.c | 14 +++++++-------\n> 1 file changed, 7 insertions(+), 7 deletions(-)\n>\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> index 77d0a5c948..8f568003ee 100644\n> --- a/builtin/gc.c\n> +++ b/builtin/gc.c\n> @@ -1011,6 +1011,13 @@ int cmd_gc(int argc,\n> \tif (opts.detach <= 0 && !skip_foreground_tasks)\n> \t\tgc_foreground_tasks(&opts, &cfg);\n> \n> +\tif (cfg.prune_worktrees_expire &&\n> +\t maintenance_task_worktree_prune(&opts, &cfg))\n> +\t\tdie(FAILED_RUN, \"worktree\");\n> +\n> +\tif (maintenance_task_rerere_gc(&opts, &cfg))\n> +\t\tdie(FAILED_RUN, \"rerere\");\n> +\n> \tif (!the_repository->repository_format_precious_objects) {\n> \t\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n> \n> @@ -1038,13 +1045,6 @@ int cmd_gc(int argc,\n> \t\t}\n> \t}\n> \n> -\tif (cfg.prune_worktrees_expire &&\n> -\t maintenance_task_worktree_prune(&opts, &cfg))\n> -\t\tdie(FAILED_RUN, \"worktree\");\n> -\n> -\tif (maintenance_task_rerere_gc(&opts, &cfg))\n> -\t\tdie(FAILED_RUN, \"rerere\");\n> -\n> \treport_garbage = report_pack_garbage;\n> \todb_reprepare(the_repository->objects);\n> \tif (pack_garbage.nr > 0) {\n"},{"id":"547402","messageId":"xmqq7bn61r1u.fsf@gitster.g","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-3-aae607667be4@pks.im","subject":"Re: [PATCH 03/11] builtin/gc: extract object database optimizations into separate function","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-07T20:30:05Z","receivedAt":"2026-07-07T20:30:09Z","isPatch":true,"body":"Patrick Steinhardt writes:\n\n> Extract the object database optimization logic from `cmd_gc()` into a\n> new `maintenance_task_odb()` helper function. This is a pure refactoring\n> with no intended functional change.\n>\n> Note that the message that notifies the user about too many loose\n> objects is moved into the new function, as well. It is inherently an\n> implementation detail of how the \"files\" source works, and as a\n> consequence we'll move it around in a later commit, as well. This\n> reordering means that the warning may now be printed at a different\n> point in time, but it's not expected that this will have any practical\n> implications.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/gc.c | 79 +++++++++++++++++++++++++++++++++++++-----------------------\n> 1 file changed, 49 insertions(+), 30 deletions(-)\n>\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> index 8f568003ee..2ff98fa727 100644\n> --- a/builtin/gc.c\n> +++ b/builtin/gc.c\n> @@ -839,6 +839,53 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n> \treturn 0;\n> }\n> \n> +static int maintenance_task_odb(struct maintenance_run_opts *opts,\n> +\t\t\t\tstruct gc_config *cfg,\n> +\t\t\t\tstruct strvec *repack_args)\n> +{\n> +\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n> +\tint ret;\n> +\n> +\tif (the_repository->repository_format_precious_objects)\n> +\t\treturn 0;\n> +\n> +\trepack_cmd.git_cmd = 1;\n> +\trepack_cmd.odb_to_close = the_repository->objects;\n> +\tstrvec_pushv(&repack_cmd.args, repack_args->v);\n> +\tif (run_command(&repack_cmd)) {\n> +\t\tret = error(FAILED_RUN, repack_args->v[0]);\n> +\t\tgoto out;\n> +\t}\n> +\n> +\tif (cfg->prune_expire) {\n> +\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n> +\n> +\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n> +\t\t/* run `git prune` even if using cruft packs */\n> +\t\tstrvec_push(&prune_cmd.args, cfg->prune_expire);\n> +\t\tif (opts->quiet)\n> +\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n> +\t\tif (repo_has_promisor_remote(the_repository))\n> +\t\t\tstrvec_push(&prune_cmd.args,\n> +\t\t\t\t \"--exclude-promisor-objects\");\n> +\t\tprune_cmd.git_cmd = 1;\n> +\n> +\t\tif (run_command(&prune_cmd)) {\n> +\t\t\tret = error(FAILED_RUN, prune_cmd.args.v[0]);\n> +\t\t\tgoto out;\n> +\t\t}\n> +\t}\n> +\n> +\tif (opts->auto_flag && too_many_loose_objects(cfg->gc_auto_threshold))\n> +\t\twarning(_(\"There are too many unreachable loose objects; \"\n> +\t\t\t\"run 'git prune' to remove them.\"));\n> +\n> +\tret = 0;\n> +\n> +out:\n> +\treturn ret;\n> +}\n> +\n> int cmd_gc(int argc,\n> \t const char **argv,\n> \t const char *prefix,\n> @@ -1018,32 +1065,8 @@ int cmd_gc(int argc,\n> \tif (maintenance_task_rerere_gc(&opts, &cfg))\n> \t\tdie(FAILED_RUN, \"rerere\");\n> \n> -\tif (!the_repository->repository_format_precious_objects) {\n> -\t\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n> -\n> -\t\trepack_cmd.git_cmd = 1;\n> -\t\trepack_cmd.odb_to_close = the_repository->objects;\n> -\t\tstrvec_pushv(&repack_cmd.args, repack_args.v);\n> -\t\tif (run_command(&repack_cmd))\n> -\t\t\tdie(FAILED_RUN, repack_args.v[0]);\n> -\n> -\t\tif (cfg.prune_expire) {\n> -\t\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n> -\n> -\t\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n> -\t\t\t/* run `git prune` even if using cruft packs */\n> -\t\t\tstrvec_push(&prune_cmd.args, cfg.prune_expire);\n> -\t\t\tif (opts.quiet)\n> -\t\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n> -\t\t\tif (repo_has_promisor_remote(the_repository))\n> -\t\t\t\tstrvec_push(&prune_cmd.args,\n> -\t\t\t\t\t \"--exclude-promisor-objects\");\n> -\t\t\tprune_cmd.git_cmd = 1;\n> -\n> -\t\t\tif (run_command(&prune_cmd))\n> -\t\t\t\tdie(FAILED_RUN, prune_cmd.args.v[0]);\n> -\t\t}\n> -\t}\n> +\tif (maintenance_task_odb(&opts, &cfg, &repack_args))\n> +\t\tdie(NULL);\n\nInstead of giving the \"fatal:\" message here from this function, the\nnew code lets the helper function issue an \"error:\", so we do not\nwant to say an extra \"fatal:\" from die(), so this die(NULL) may be a\ngood thing to do.\n"},{"id":"547476","messageId":"ak4CHGpIhVIT9sd2@pks.im","threadId":"65942","inReplyTo":"xmqqo6gi1sng.fsf@gitster.g","subject":"Re: [PATCH 01/11] odb: run \"pre-auto-gc\" hook for all maintenance tasks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-08T07:55:12Z","receivedAt":"2026-07-08T07:55:24Z","isPatch":true,"body":"On Tue, Jul 07, 2026 at 12:55:31PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> > diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\n> > index d7f82e1bec..1212b306b6 100755\n> > --- a/t/t7900-maintenance.sh\n> > +++ b/t/t7900-maintenance.sh\n> > @@ -740,6 +740,127 @@ test_expect_success 'geometric repacking honors configured split factor' '\n> > \t)\n> > '\n> > \n> > +test_expect_success 'pre-auto-gc hook runs exactly once' '\n> > +\ttest_when_finished \"rm -rf repo\" &&\n> > +\tgit init repo &&\n> > +\t(\n> > +\t\tcd repo &&\n> > +\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n> > +\t\techo hook >>hook.log\n> > +\t\tEOF\n> > +\n> > +\t\t# Satisfy the auto condition for multiple tasks, both in the\n> > +\t\t# foreground and in the background phase.\n> > +\t\tgit config set maintenance.reflog-expire.auto -1 &&\n> > +\t\tgit config set maintenance.geometric-repack.auto -1 &&\n> > +\t\tgit config set maintenance.rerere-gc.auto -1 &&\n> > +\n> > +\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n> > +\t\t\tgit maintenance run --auto 2>/dev/null &&\n> > +\n> > +\t\t# The successful hook does not inhibit any of the tasks...\n> > +\t\ttest_subcommand git reflog expire --all > +\t\ttest_subcommand_flex git repack > +\t\ttest_subcommand git rerere gc > +\t\t# ... but it must only have been executed a single time.\n> > +\t\ttest_line_count = 1 hook.log\n> > +\t)\n> > +'\n> \n> Somehow I'd feel better if the hook used a full path to the append\n> only log file, but it is reasonably clear that these three commands\n> are unlikely to chdir around, so it may be OK.\n> \n> Obviously not in scope of this topic, but I wonder if we have a\n> better way to test these three \"housekeeping tasks\" have run, than\n> casting in stone the current implementation that spawns these three\n> external command as subprocesses.\n\nYeah, this is awfully fragile indeed. Since 25914c4fde (maintenance: add\ntrace2 regions for task execution, 2020-09-17) we already have trace2\nmarkers for each task that's running, so that may indeed be a much\nbetter way to figure out whether the task is running or not.\n\nI've made some local changes, but will hold back with sending a v2 until\nthere's more feedback.\n\nThanks!\n\nPatrick\n"},{"id":"547945","messageId":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im","subject":"[PATCH v2 00/12] odb: make optimizations pluggable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:03Z","receivedAt":"2026-07-13T05:52:17Z","isPatch":true,"body":"Hi,\n\nthis patch series converts object housekeeping to become pluggable.\nThere isn't really anything else to say about this.\n\nThe series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n2026-07-06).\n\nChanges in v2:\n - Make tests in t7900 a bit more robust by not checking for exact\n commands, but instead by checking for executed tasks.\n - Link to v1: https://patch.msgid.link/20260707-b4-pks-odb-optimize-v1-0-aae607667be4@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (12):\n t7900: simplify how we check for maintenance tasks\n odb: run \"pre-auto-gc\" hook for all maintenance tasks\n builtin/gc: move worktree and rerere tasks before object optimizations\n builtin/gc: extract object database optimizations into separate function\n builtin/gc: make repack arguments self-contained\n builtin/gc: inline config values specific to the \"files\" backend\n builtin/gc: introduce object database optimization options\n builtin/gc: move geometric repacking into `odb_optimize()`\n builtin/gc: introduce `odb_optimize_required()`\n builtin/gc: refactor ODB optimizations to operate on \"files\" source\n builtin/gc: fix signedness issues in ODB-related functionality\n odb: make optimizations pluggable\n\n builtin/gc.c | 534 ++++++++-----------------------------------------\n odb.c | 12 ++\n odb.h | 45 +++++\n odb/source-files.c | 470 +++++++++++++++++++++++++++++++++++++++++++\n odb/source-files.h | 15 ++\n odb/source.h | 36 ++++\n t/t7900-maintenance.sh | 338 +++++++++++++++++++++----------\n 7 files changed, 894 insertions(+), 556 deletions(-)\n\nRange-diff versus v1:\n\n -: ---------- > 1: b05988c150 t7900: simplify how we check for maintenance tasks\n 1: df9ed71c10 ! 2: 734d3c6d17 odb: run \"pre-auto-gc\" hook for all maintenance tasks\n @@ t/t7900-maintenance.sh: test_expect_success 'geometric repacking honors configur\n +\t\t\tgit maintenance run --auto 2>/dev/null &&\n +\n +\t\t# The successful hook does not inhibit any of the tasks...\n -+\t\ttest_subcommand git reflog expire --all \n---\n t/t7900-maintenance.sh | 194 ++++++++++++++++++++++++++-----------------------\n 1 file changed, 102 insertions(+), 92 deletions(-)\n\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex d7f82e1bec..129829f1f4 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -23,6 +23,12 @@ test_xmllint () {\n \tfi\n }\n \n+test_maintenance_tasks () {\n+\tcat >expect &&\n+\tsed -ne \"s/.*\\\"region_enter\\\".*\\\"category\\\":\\\"maintenance\\([^\\\"]*\\)\\\".*\\\"label\\\":\\\"\\([^\\\"][^\\\"]*\\)\\\".*/\\2\\1/p\" \"$1\" >actual &&\n+\ttest_cmp expect actual\n+}\n+\n test_lazy_prereq SYSTEMD_ANALYZE '\n \tsystemd-analyze verify /lib/systemd/system/basic.target\n '\n@@ -180,8 +186,9 @@ test_expect_success 'maintenance..enabled' '\n \tgit config maintenance.gc.enabled false &&\n \tgit config maintenance.commit-graph.enabled true &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/run-config.txt\" git maintenance run 2>err &&\n-\ttest_subcommand ! git gc --quiet ' '\n@@ -189,16 +196,20 @@ test_expect_success 'run --task=' '\n \t\tgit maintenance run --task=commit-graph 2>/dev/null &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/run-gc.txt\" \\\n \t\tgit maintenance run --task=gc 2>/dev/null &&\n-\tGIT_TRACE2_EVENT=\"$(pwd)/run-commit-graph.txt\" \\\n-\t\tgit maintenance run --task=commit-graph 2>/dev/null &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/run-both.txt\" \\\n \t\tgit maintenance run --task=commit-graph --task=gc 2>/dev/null &&\n-\ttest_subcommand ! git gc --quiet --no-detach --skip-foreground-tasks daily -> hourly' '\n \n \tGIT_TRACE2_EVENT=\"$(pwd)/hourly.txt\" \\\n \t\tgit maintenance run --schedule=hourly 2>/dev/null &&\n-\ttest_subcommand git prune-packed --quiet /dev/null &&\n-\ttest_subcommand git prune-packed --quiet /dev/null &&\n-\ttest_subcommand git prune-packed --quiet expect &&\n \trm -f trace2.txt &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n \t\tgit -c maintenance.strategy=$STRATEGY maintenance run --quiet \"$@\" &&\n-\tsed -n 's/{\"event\":\"child_start\",\"sid\":\"[^/\"]*\",.*,\"argv\":\\[\"\\(.*\\)\\\"]}/\\1/p' actual\n-\ttest_cmp expect actual\n+\ttest_maintenance_tasks trace2.txt\n }\n \n test_expect_success 'maintenance.strategy is respected' '\n@@ -1017,48 +1031,44 @@ test_expect_success 'maintenance.strategy is respected' '\n \t\ttest_grep \"unknown maintenance strategy: .unknown.\" err &&\n \n \t\ttest_strategy incremental <<-\\EOF &&\n-\t\tgit pack-refs --all --prune\n-\t\tgit reflog expire --all\n-\t\tgit gc --quiet --no-detach --skip-foreground-tasks\n+\t\tgc foreground\n+\t\tgc\n \t\tEOF\n \n \t\ttest_strategy incremental --schedule=weekly <<-\\EOF &&\n-\t\tgit pack-refs --all --prune\n-\t\tgit prune-packed --quiet\n-\t\tgit multi-pack-index write --no-progress\n-\t\tgit multi-pack-index expire --no-progress\n-\t\tgit multi-pack-index repack --no-progress --batch-size=1\n-\t\tgit commit-graph write --split --reachable --no-progress\n+\t\tpack-refs foreground\n+\t\tprefetch\n+\t\tloose-objects\n+\t\tincremental-repack\n+\t\tcommit-graph\n \t\tEOF\n \n \t\ttest_strategy gc <<-\\EOF &&\n-\t\tgit pack-refs --all --prune\n-\t\tgit reflog expire --all\n-\t\tgit gc --quiet --no-detach --skip-foreground-tasks\n+\t\tgc foreground\n+\t\tgc\n \t\tEOF\n \n \t\ttest_strategy gc --schedule=weekly <<-\\EOF &&\n-\t\tgit pack-refs --all --prune\n-\t\tgit reflog expire --all\n-\t\tgit gc --quiet --no-detach --skip-foreground-tasks\n+\t\tgc foreground\n+\t\tgc\n \t\tEOF\n \n \t\ttest_strategy geometric <<-\\EOF &&\n-\t\tgit pack-refs --all --prune\n-\t\tgit reflog expire --all\n-\t\tgit repack -d -l --geometric=2 --quiet --write-midx\n-\t\tgit commit-graph write --split --reachable --no-progress\n-\t\tgit worktree prune --expire 3.months.ago\n-\t\tgit rerere gc\n+\t\tpack-refs foreground\n+\t\treflog-expire foreground\n+\t\tgeometric-repack\n+\t\tcommit-graph\n+\t\tworktree-prune\n+\t\trerere-gc\n \t\tEOF\n \n \t\ttest_strategy geometric --schedule=weekly <<-\\EOF\n-\t\tgit pack-refs --all --prune\n-\t\tgit reflog expire --all\n-\t\tgit repack -d -l --geometric=2 --quiet --write-midx\n-\t\tgit commit-graph write --split --reachable --no-progress\n-\t\tgit worktree prune --expire 3.months.ago\n-\t\tgit rerere gc\n+\t\tpack-refs foreground\n+\t\treflog-expire foreground\n+\t\tgeometric-repack\n+\t\tcommit-graph\n+\t\tworktree-prune\n+\t\trerere-gc\n \t\tEOF\n \t)\n '\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547947","messageId":"20260713-b4-pks-odb-optimize-v2-2-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 02/12] odb: run \"pre-auto-gc\" hook for all maintenance tasks","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:05Z","receivedAt":"2026-07-13T05:52:21Z","isPatch":true,"body":"The \"pre-auto-gc\" hook is supposed to run before auto-maintenance\nstarts. The intent of this is to give users the ability to intercept\nrunning maintenance in case there's for example an event that is not\nsupposed to run in parallel with repository maintenance.\n\nThis hook runs via `need_to_gc()`, which is invoked via two paths:\n\n - It is called directly by git-gc(1).\n\n - It is called indirectly by git-maintenance(1) via the \"gc\" task.\n\nWhile the former makes sense, the latter is somewhat off. While the hook\nis indeed strongly tied to gc'ing a repository, the original intent of\nthe hook is rather to inhibit any kind of automated garbage collection.\nThat noticeably also includes all the other maintenance tasks that our\nnew infrastructure may run, but those aren't getting intercepted at all.\nThe move towards our new maintenance strategy has thus somewhat neutered\nthe effectiveness of the hook.\n\nFix this issue by running the hook before the first auto-maintenance\ntask that would run as determined by the tasks's auto condition. Note\nthat this requires us to lift the call to `run_hooks()` out of\n`needs_to_gc()`, as the hook would otherwise potentially run multiple\ntimes.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 35 ++++++++++----\n t/t7900-maintenance.sh | 126 +++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 152 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex d32af422af..77d0a5c948 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -709,8 +709,6 @@ static int need_to_gc(struct gc_config *cfg, struct strvec *repack_args)\n \telse\n \t\treturn 0;\n \n-\tif (run_hooks(the_repository, \"pre-auto-gc\"))\n-\t\treturn 0;\n \treturn 1;\n }\n \n@@ -933,7 +931,8 @@ int cmd_gc(int argc,\n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(&cfg, &repack_args)) {\n+\t\tif (!need_to_gc(&cfg, &repack_args) ||\n+\t\t run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -1755,11 +1754,18 @@ enum task_phase {\n \tTASK_PHASE_BACKGROUND,\n };\n \n+enum auto_gc_hook_result {\n+\tAUTO_GC_HOOK_UNDECIDED = 0,\n+\tAUTO_GC_HOOK_RUN = 1,\n+\tAUTO_GC_HOOK_SKIP = 2,\n+};\n+\n static int maybe_run_task(const struct maintenance_task *task,\n \t\t\t struct repository *repo,\n \t\t\t struct maintenance_run_opts *opts,\n \t\t\t struct gc_config *cfg,\n-\t\t\t enum task_phase phase)\n+\t\t\t enum task_phase phase,\n+\t\t\t enum auto_gc_hook_result *auto_gc_hook_result)\n {\n \tint foreground = (phase == TASK_PHASE_FOREGROUND);\n \tmaintenance_task_fn fn = foreground ? task->foreground : task->background;\n@@ -1768,9 +1774,19 @@ static int maybe_run_task(const struct maintenance_task *task,\n \n \tif (!fn)\n \t\treturn 0;\n-\tif (opts->auto_flag &&\n-\t (!task->auto_condition || !task->auto_condition(cfg)))\n-\t\treturn 0;\n+\tif (opts->auto_flag) {\n+\t\tif (*auto_gc_hook_result == AUTO_GC_HOOK_SKIP)\n+\t\t\treturn 0;\n+\n+\t\tif (!task->auto_condition || !task->auto_condition(cfg))\n+\t\t\treturn 0;\n+\n+\t\tif (*auto_gc_hook_result == AUTO_GC_HOOK_UNDECIDED)\n+\t\t\t*auto_gc_hook_result = run_hooks(repo, \"pre-auto-gc\") ?\n+\t\t\t\tAUTO_GC_HOOK_SKIP : AUTO_GC_HOOK_RUN;\n+\t\tif (*auto_gc_hook_result == AUTO_GC_HOOK_SKIP)\n+\t\t\treturn 0;\n+\t}\n \n \ttrace2_region_enter(region, task->name, repo);\n \tif (fn(opts, cfg)) {\n@@ -1789,6 +1805,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,\n \tstruct lock_file lk;\n \tstruct repository *r = the_repository;\n \tchar *lock_path = xstrfmt(\"%s/maintenance\", r->objects->sources->path);\n+\tenum auto_gc_hook_result auto_gc_hook_result = AUTO_GC_HOOK_UNDECIDED;\n \n \tif (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {\n \t\t/*\n@@ -1808,7 +1825,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,\n \n \tfor (size_t i = 0; i < opts->tasks_nr; i++)\n \t\tif (maybe_run_task(&tasks[opts->tasks[i]], r, opts, cfg,\n-\t\t\t\t TASK_PHASE_FOREGROUND))\n+\t\t\t\t TASK_PHASE_FOREGROUND, &auto_gc_hook_result))\n \t\t\tresult = 1;\n \n \t/* Failure to daemonize is ok, we'll continue in foreground. */\n@@ -1820,7 +1837,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,\n \n \tfor (size_t i = 0; i < opts->tasks_nr; i++)\n \t\tif (maybe_run_task(&tasks[opts->tasks[i]], r, opts, cfg,\n-\t\t\t\t TASK_PHASE_BACKGROUND))\n+\t\t\t\t TASK_PHASE_BACKGROUND, &auto_gc_hook_result))\n \t\t\tresult = 1;\n \n \trollback_lock_file(&lk);\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 129829f1f4..2d52e7918a 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -758,6 +758,132 @@ test_expect_success 'geometric repacking honors configured split factor' '\n \t)\n '\n \n+test_expect_success 'pre-auto-gc hook runs exactly once' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\tEOF\n+\n+\t\t# Satisfy the auto condition for multiple tasks, both in the\n+\t\t# foreground and in the background phase.\n+\t\tgit config set maintenance.reflog-expire.auto -1 &&\n+\t\tgit config set maintenance.geometric-repack.auto -1 &&\n+\t\tgit config set maintenance.rerere-gc.auto -1 &&\n+\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run --auto 2>/dev/null &&\n+\n+\t\t# The successful hook does not inhibit any of the tasks...\n+\t\ttest_maintenance_tasks trace2.txt <<-\\EOF &&\n+\t\treflog-expire foreground\n+\t\tgeometric-repack\n+\t\trerere-gc\n+\t\tEOF\n+\t\t# ... but it must only have been executed a single time.\n+\t\ttest_line_count = 1 hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook can inhibit geometric strategy' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\texit 1\n+\t\tEOF\n+\n+\t\tgit config set maintenance.reflog-expire.auto -1 &&\n+\t\tgit config set maintenance.geometric-repack.auto -1 &&\n+\t\tgit config set maintenance.rerere-gc.auto -1 &&\n+\n+\t\t# Maintenance would be required...\n+\t\tgit maintenance is-needed --auto &&\n+\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run --auto 2>/dev/null &&\n+\n+\t\t# ... but the failing hook inhibits all tasks. The hook itself\n+\t\t# is expected to be the only child process being spawned, and\n+\t\t# it must only run a single time.\n+\t\ttest_grep \"child_start.*pre-auto-gc\" trace2.txt &&\n+\t\ttest_maintenance_tasks trace2.txt <<-\\EOF &&\n+\t\tEOF\n+\t\ttest_line_count = 1 hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook can inhibit gc strategy' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\texit 1\n+\t\tEOF\n+\n+\t\tgit config set maintenance.strategy gc &&\n+\t\tgit config set maintenance.auto false &&\n+\t\tgit config set gc.auto 3 &&\n+\n+\t\ttest_oid_init &&\n+\n+\t\t# We need to create two objects whose hashes start with 17\n+\t\t# since this is what the gc task counts.\n+\t\ttest_commit \"$(test_oid blob17_1)\" &&\n+\t\ttest_commit \"$(test_oid blob17_2)\" &&\n+\n+\t\t# Maintenance would be required...\n+\t\tgit maintenance is-needed --auto &&\n+\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run --auto 2>/dev/null &&\n+\n+\t\t# ... but the failing hook inhibits all tasks. The hook itself\n+\t\t# is expected to be the only child process being spawned, and\n+\t\t# it must only run a single time.\n+\t\ttest_grep \"child_start.*pre-auto-gc\" trace2.txt &&\n+\t\ttest_maintenance_tasks trace2.txt <<-\\EOF &&\n+\t\tEOF\n+\t\ttest_subcommand_flex ! git trace2 &&\n+\t\ttest_line_count = 1 hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook does not run when no maintenance is needed' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\t(\n+\t\tcd repo &&\n+\t\twrite_script .git/hooks/pre-auto-gc <<-\\EOF &&\n+\t\techo hook >>hook.log\n+\t\tEOF\n+\t\ttest_must_fail git maintenance is-needed --auto &&\n+\t\tgit maintenance run --auto 2>/dev/null &&\n+\t\ttest_path_is_missing hook.log\n+\t)\n+'\n+\n+test_expect_success 'pre-auto-gc hook does not run without --auto' '\n+\ttest_when_finished \"rm -rf repo\" &&\n+\tgit init repo &&\n+\ttest_hook -C repo pre-auto-gc <<-\\EOF &&\n+\techo hook >>hook.log\n+\tEOF\n+\t(\n+\t\tcd repo &&\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n+\t\t\tgit maintenance run 2>/dev/null &&\n+\t\ttest_grep \"\\[\\\"git\\\",\\\"repack\\\",\" trace2.txt &&\n+\t\ttest_path_is_missing hook.log\n+\t)\n+'\n+\n test_expect_success 'pack-refs task' '\n \tfor n in $(test_seq 1 5)\n \tdo\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547948","messageId":"20260713-b4-pks-odb-optimize-v2-3-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 03/12] builtin/gc: move worktree and rerere tasks before object optimizations","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:06Z","receivedAt":"2026-07-13T05:52:24Z","isPatch":true,"body":"In subsequent patches we'll consolidate all tasks that relate to\nmaintenance of the object database and move it into the \"files\" backend.\nThe relevant code is somewhat scattered though, as several other tasks\nare interspersed between.\n\nRefactor the code so that all object database optimizations are grouped\ntogether, which requires us to move worktree pruning and rerere garbage\ncollection around. In theory, rearranging this code can have an effect\non the object database optimizations:\n\n - Rerere entries really shouldn't impact garbage collection at all, as\n these entries are not stored in the object database.\n\n - The index and HEAD reference of pruned worktrees may reference\n objects that become unreachable.\n\nThat being said, the impact should be overall rather negligible. If the\nuser was asking us to prune objects with immediate expiration time then\nwe might now prune objects that were previously still kept alive by the\nworktree. But besides being a very specific edge case, it's arguably not\neven the wrong thing to also prune any potentially-unreachable objects\nimmediately.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 77d0a5c948..8f568003ee 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -1011,6 +1011,13 @@ int cmd_gc(int argc,\n \tif (opts.detach <= 0 && !skip_foreground_tasks)\n \t\tgc_foreground_tasks(&opts, &cfg);\n \n+\tif (cfg.prune_worktrees_expire &&\n+\t maintenance_task_worktree_prune(&opts, &cfg))\n+\t\tdie(FAILED_RUN, \"worktree\");\n+\n+\tif (maintenance_task_rerere_gc(&opts, &cfg))\n+\t\tdie(FAILED_RUN, \"rerere\");\n+\n \tif (!the_repository->repository_format_precious_objects) {\n \t\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \n@@ -1038,13 +1045,6 @@ int cmd_gc(int argc,\n \t\t}\n \t}\n \n-\tif (cfg.prune_worktrees_expire &&\n-\t maintenance_task_worktree_prune(&opts, &cfg))\n-\t\tdie(FAILED_RUN, \"worktree\");\n-\n-\tif (maintenance_task_rerere_gc(&opts, &cfg))\n-\t\tdie(FAILED_RUN, \"rerere\");\n-\n \treport_garbage = report_pack_garbage;\n \todb_reprepare(the_repository->objects);\n \tif (pack_garbage.nr > 0) {\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547950","messageId":"20260713-b4-pks-odb-optimize-v2-4-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 04/12] builtin/gc: extract object database optimizations into separate function","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:07Z","receivedAt":"2026-07-13T05:52:27Z","isPatch":true,"body":"Extract the object database optimization logic from `cmd_gc()` into a\nnew `maintenance_task_odb()` helper function. This is a pure refactoring\nwith no intended functional change.\n\nNote that the message that notifies the user about too many loose\nobjects is moved into the new function, as well. It is inherently an\nimplementation detail of how the \"files\" source works, and as a\nconsequence we'll move it around in a later commit, as well. This\nreordering means that the warning may now be printed at a different\npoint in time, but it's not expected that this will have any practical\nimplications.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 79 +++++++++++++++++++++++++++++++++++++-----------------------\n 1 file changed, 49 insertions(+), 30 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 8f568003ee..2ff98fa727 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -839,6 +839,53 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \treturn 0;\n }\n \n+static int maintenance_task_odb(struct maintenance_run_opts *opts,\n+\t\t\t\tstruct gc_config *cfg,\n+\t\t\t\tstruct strvec *repack_args)\n+{\n+\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n+\tint ret;\n+\n+\tif (the_repository->repository_format_precious_objects)\n+\t\treturn 0;\n+\n+\trepack_cmd.git_cmd = 1;\n+\trepack_cmd.odb_to_close = the_repository->objects;\n+\tstrvec_pushv(&repack_cmd.args, repack_args->v);\n+\tif (run_command(&repack_cmd)) {\n+\t\tret = error(FAILED_RUN, repack_args->v[0]);\n+\t\tgoto out;\n+\t}\n+\n+\tif (cfg->prune_expire) {\n+\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n+\n+\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n+\t\t/* run `git prune` even if using cruft packs */\n+\t\tstrvec_push(&prune_cmd.args, cfg->prune_expire);\n+\t\tif (opts->quiet)\n+\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n+\t\tif (repo_has_promisor_remote(the_repository))\n+\t\t\tstrvec_push(&prune_cmd.args,\n+\t\t\t\t \"--exclude-promisor-objects\");\n+\t\tprune_cmd.git_cmd = 1;\n+\n+\t\tif (run_command(&prune_cmd)) {\n+\t\t\tret = error(FAILED_RUN, prune_cmd.args.v[0]);\n+\t\t\tgoto out;\n+\t\t}\n+\t}\n+\n+\tif (opts->auto_flag && too_many_loose_objects(cfg->gc_auto_threshold))\n+\t\twarning(_(\"There are too many unreachable loose objects; \"\n+\t\t\t\"run 'git prune' to remove them.\"));\n+\n+\tret = 0;\n+\n+out:\n+\treturn ret;\n+}\n+\n int cmd_gc(int argc,\n \t const char **argv,\n \t const char *prefix,\n@@ -1018,32 +1065,8 @@ int cmd_gc(int argc,\n \tif (maintenance_task_rerere_gc(&opts, &cfg))\n \t\tdie(FAILED_RUN, \"rerere\");\n \n-\tif (!the_repository->repository_format_precious_objects) {\n-\t\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n-\n-\t\trepack_cmd.git_cmd = 1;\n-\t\trepack_cmd.odb_to_close = the_repository->objects;\n-\t\tstrvec_pushv(&repack_cmd.args, repack_args.v);\n-\t\tif (run_command(&repack_cmd))\n-\t\t\tdie(FAILED_RUN, repack_args.v[0]);\n-\n-\t\tif (cfg.prune_expire) {\n-\t\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n-\n-\t\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n-\t\t\t/* run `git prune` even if using cruft packs */\n-\t\t\tstrvec_push(&prune_cmd.args, cfg.prune_expire);\n-\t\t\tif (opts.quiet)\n-\t\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n-\t\t\tif (repo_has_promisor_remote(the_repository))\n-\t\t\t\tstrvec_push(&prune_cmd.args,\n-\t\t\t\t\t \"--exclude-promisor-objects\");\n-\t\t\tprune_cmd.git_cmd = 1;\n-\n-\t\t\tif (run_command(&prune_cmd))\n-\t\t\t\tdie(FAILED_RUN, prune_cmd.args.v[0]);\n-\t\t}\n-\t}\n+\tif (maintenance_task_odb(&opts, &cfg, &repack_args))\n+\t\tdie(NULL);\n \n \treport_garbage = report_pack_garbage;\n \todb_reprepare(the_repository->objects);\n@@ -1057,10 +1080,6 @@ int cmd_gc(int argc,\n \t\t\t\t\t !opts.quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0,\n \t\t\t\t\t NULL);\n \n-\tif (opts.auto_flag && too_many_loose_objects(cfg.gc_auto_threshold))\n-\t\twarning(_(\"There are too many unreachable loose objects; \"\n-\t\t\t\"run 'git prune' to remove them.\"));\n-\n \tif (!daemonized) {\n \t\tchar *path = repo_git_path(the_repository, \"gc.log\");\n \t\tunlink(path);\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547951","messageId":"20260713-b4-pks-odb-optimize-v2-5-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 05/12] builtin/gc: make repack arguments self-contained","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:08Z","receivedAt":"2026-07-13T05:52:29Z","isPatch":true,"body":"When optimizing the object database most of the heavy-lifting is done by\ngit-repack(1). The arguments we pass to this function are assembled in\nglobal scope, which is hard to follow.\n\nRefactor the logic by moving the vector into `maintenance_task_odb()`.\nWhile that means we have to pass more arguments to this function, it has\nthe upside that the logic becomes self-contained without any kind of\nglobal interdependencies.\n\nThis is a pure refactoring with no intended functional change.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 156 ++++++++++++++++++++++++++++-------------------------------\n 1 file changed, 75 insertions(+), 81 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 2ff98fa727..25a59caea6 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -661,7 +661,7 @@ static void add_repack_incremental_option(struct strvec *args)\n \tstrvec_push(args, \"--no-write-bitmap-index\");\n }\n \n-static int need_to_gc(struct gc_config *cfg, struct strvec *repack_args)\n+static int need_to_gc(struct gc_config *cfg)\n {\n \t/*\n \t * Setting gc.auto to 0 or negative can disable the\n@@ -669,46 +669,8 @@ static int need_to_gc(struct gc_config *cfg, struct strvec *repack_args)\n \t */\n \tif (cfg->gc_auto_threshold <= 0)\n \t\treturn 0;\n-\n-\t/*\n-\t * If there are too many loose objects, but not too many\n-\t * packs, we run \"repack -d -l\". If there are too many packs,\n-\t * we run \"repack -A -d -l\". Otherwise we tell the caller\n-\t * there is no need.\n-\t */\n-\tif (too_many_packs(cfg)) {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (cfg->big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n-\t\t\tif (keep_pack.nr >= cfg->gc_auto_pack_limit) {\n-\t\t\t\tcfg->big_pack_threshold = 0;\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t\t}\n-\t\t} else {\n-\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n-\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\tmem_have = total_ram();\n-\t\t\tmem_want = estimate_repack_memory(cfg, p);\n-\n-\t\t\t/*\n-\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t * the rest for the OS and other processes in the\n-\t\t\t * system.\n-\t\t\t */\n-\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t}\n-\n-\t\tadd_repack_all_option(cfg, &keep_pack, repack_args);\n-\t\tstring_list_clear(&keep_pack, 0);\n-\t} else if (too_many_loose_objects(cfg->gc_auto_threshold))\n-\t\tadd_repack_incremental_option(repack_args);\n-\telse\n+\tif (!too_many_packs(cfg) && !too_many_loose_objects(cfg->gc_auto_threshold))\n \t\treturn 0;\n-\n \treturn 1;\n }\n \n@@ -841,7 +803,8 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \n static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tstruct gc_config *cfg,\n-\t\t\t\tstruct strvec *repack_args)\n+\t\t\t\tint keep_largest_pack,\n+\t\t\t\tint aggressive)\n {\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \tint ret;\n@@ -851,9 +814,75 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \n \trepack_cmd.git_cmd = 1;\n \trepack_cmd.odb_to_close = the_repository->objects;\n-\tstrvec_pushv(&repack_cmd.args, repack_args->v);\n+\n+\tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n+\tif (aggressive) {\n+\t\tstrvec_push(&repack_cmd.args, \"-f\");\n+\t\tif (cfg->aggressive_depth > 0)\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", cfg->aggressive_depth);\n+\t\tif (cfg->aggressive_window > 0)\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", cfg->aggressive_window);\n+\t}\n+\tif (opts->quiet)\n+\t\tstrvec_push(&repack_cmd.args, \"-q\");\n+\n+\t/*\n+\t * There's three cases we need to consider:\n+\t *\n+\t * - If we're invoked without `--auto` we'll need to perform a full\n+\t * repack.\n+\t *\n+\t * - If we're invoked with `--auto` and there's too many packs, then\n+\t * we perform a full repack, as well.\n+\t *\n+\t * - Otherwise we perform an incremental repack.\n+\t */\n+\tif (!opts->auto_flag) {\n+\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\tif (keep_largest_pack != -1) {\n+\t\t\tif (keep_largest_pack)\n+\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t} else if (cfg->big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n+\t\t}\n+\n+\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n+\t\tstring_list_clear(&keep_pack, 0);\n+\t} else if (too_many_packs(cfg)) {\n+\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\tif (cfg->big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n+\t\t\tif (keep_pack.nr >= cfg->gc_auto_pack_limit) {\n+\t\t\t\tcfg->big_pack_threshold = 0;\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t}\n+\t\t} else {\n+\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\tmem_have = total_ram();\n+\t\t\tmem_want = estimate_repack_memory(cfg, p);\n+\n+\t\t\t/*\n+\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t * the rest for the OS and other processes in the\n+\t\t\t * system.\n+\t\t\t */\n+\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t}\n+\n+\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n+\t\tstring_list_clear(&keep_pack, 0);\n+\t} else {\n+\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t}\n+\n \tif (run_command(&repack_cmd)) {\n-\t\tret = error(FAILED_RUN, repack_args->v[0]);\n+\t\tret = error(FAILED_RUN, repack_cmd.args.v[0]);\n \t\tgoto out;\n \t}\n \n@@ -899,7 +928,6 @@ int cmd_gc(int argc,\n \tint keep_largest_pack = -1;\n \tint skip_foreground_tasks = 0;\n \ttimestamp_t dummy;\n-\tstruct strvec repack_args = STRVEC_INIT;\n \tstruct maintenance_run_opts opts = MAINTENANCE_RUN_OPTS_INIT;\n \tstruct gc_config cfg = GC_CONFIG_INIT;\n \tconst char *prune_expire_sentinel = \"sentinel\";\n@@ -939,8 +967,6 @@ int cmd_gc(int argc,\n \tshow_usage_with_options_if_asked(argc, argv,\n \t\t\t\t\t builtin_gc_usage, builtin_gc_options);\n \n-\tstrvec_pushl(&repack_args, \"repack\", \"-d\", \"-l\", NULL);\n-\n \tgc_config(&cfg);\n \n \tif (parse_expiry_date(cfg.gc_log_expire, &gc_log_expire_time))\n@@ -961,16 +987,6 @@ int cmd_gc(int argc,\n \tif (cfg.prune_expire && parse_expiry_date(cfg.prune_expire, &dummy))\n \t\tdie(_(\"failed to parse prune expiry value %s\"), cfg.prune_expire);\n \n-\tif (aggressive) {\n-\t\tstrvec_push(&repack_args, \"-f\");\n-\t\tif (cfg.aggressive_depth > 0)\n-\t\t\tstrvec_pushf(&repack_args, \"--depth=%d\", cfg.aggressive_depth);\n-\t\tif (cfg.aggressive_window > 0)\n-\t\t\tstrvec_pushf(&repack_args, \"--window=%d\", cfg.aggressive_window);\n-\t}\n-\tif (opts.quiet)\n-\t\tstrvec_push(&repack_args, \"-q\");\n-\n \tif (opts.auto_flag) {\n \t\tif (cfg.detach_auto && opts.detach < 0)\n \t\t\topts.detach = 1;\n@@ -978,8 +994,7 @@ int cmd_gc(int argc,\n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(&cfg, &repack_args) ||\n-\t\t run_hooks(the_repository, \"pre-auto-gc\")) {\n+\t\tif (!need_to_gc(&cfg) || run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -991,18 +1006,6 @@ int cmd_gc(int argc,\n \t\t\t\tfprintf(stderr, _(\"Auto packing the repository for optimum performance.\\n\"));\n \t\t\tfprintf(stderr, _(\"See \\\"git help gc\\\" for manual housekeeping.\\n\"));\n \t\t}\n-\t} else {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (keep_largest_pack != -1) {\n-\t\t\tif (keep_largest_pack)\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t} else if (cfg.big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg.big_pack_threshold);\n-\t\t}\n-\n-\t\tadd_repack_all_option(&cfg, &keep_pack, &repack_args);\n-\t\tstring_list_clear(&keep_pack, 0);\n \t}\n \n \tif (opts.detach > 0) {\n@@ -1065,7 +1068,7 @@ int cmd_gc(int argc,\n \tif (maintenance_task_rerere_gc(&opts, &cfg))\n \t\tdie(FAILED_RUN, \"rerere\");\n \n-\tif (maintenance_task_odb(&opts, &cfg, &repack_args))\n+\tif (maintenance_task_odb(&opts, &cfg, keep_largest_pack, aggressive))\n \t\tdie(NULL);\n \n \treport_garbage = report_pack_garbage;\n@@ -1088,7 +1091,6 @@ int cmd_gc(int argc,\n \n out:\n \tmaintenance_run_opts_release(&opts);\n-\tstrvec_clear(&repack_args);\n \tgc_config_release(&cfg);\n \treturn 0;\n }\n@@ -1291,15 +1293,7 @@ static int maintenance_task_gc_background(struct maintenance_run_opts *opts,\n \n static int gc_condition(struct gc_config *cfg)\n {\n-\t/*\n-\t * Note that it's fine to drop the repack arguments here, as we execute\n-\t * git-gc(1) as a separate child process anyway. So it knows to compute\n-\t * these arguments again.\n-\t */\n-\tstruct strvec repack_args = STRVEC_INIT;\n-\tint ret = need_to_gc(cfg, &repack_args);\n-\tstrvec_clear(&repack_args);\n-\treturn ret;\n+\treturn need_to_gc(cfg);\n }\n \n static int prune_packed(struct maintenance_run_opts *opts)\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547952","messageId":"20260713-b4-pks-odb-optimize-v2-6-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 06/12] builtin/gc: inline config values specific to the \"files\" backend","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:09Z","receivedAt":"2026-07-13T05:52:31Z","isPatch":true,"body":"The `struct gc_config` contains a set of values that we read via the Git\nrepository's configuration. Several of those values that are consumed by\nthe object database optimization logic are inherently specific to the\n\"files\" config.\n\nIn a later commit we'll make the logic to optimize object databases\npluggable. So by carrying these \"files\"-backend specific values in the\ngeneric config struct means that other backends would have to worry\nabout these values, too. This feels somewhat dirty, as implementation-\nspecific details should live with the backends themselves.\n\nInline these values directly at the call sites that need them instead.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 115 +++++++++++++++++++++++++++--------------------------------\n 1 file changed, 53 insertions(+), 62 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 25a59caea6..5d445edaa0 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -130,22 +130,11 @@ struct gc_config {\n \tunsigned long max_cruft_size;\n \tint aggressive_depth;\n \tint aggressive_window;\n-\tint gc_auto_threshold;\n-\tint gc_auto_pack_limit;\n \tint detach_auto;\n \tchar *gc_log_expire;\n \tchar *prune_expire;\n \tchar *prune_worktrees_expire;\n-\tchar *repack_filter;\n-\tchar *repack_filter_to;\n \tchar *repack_expire_to;\n-\tunsigned long big_pack_threshold;\n-\tunsigned long max_delta_cache_size;\n-\t/*\n-\t * Remove this member from gc_config once repo_settings is passed\n-\t * through the callchain.\n-\t */\n-\tsize_t delta_base_cache_limit;\n };\n \n #define GC_CONFIG_INIT { \\\n@@ -154,14 +143,10 @@ struct gc_config {\n \t.cruft_packs = 1, \\\n \t.aggressive_depth = 50, \\\n \t.aggressive_window = 250, \\\n-\t.gc_auto_threshold = 6700, \\\n-\t.gc_auto_pack_limit = 50, \\\n \t.detach_auto = 1, \\\n \t.gc_log_expire = xstrdup(\"1.day.ago\"), \\\n \t.prune_expire = xstrdup(\"2.weeks.ago\"), \\\n \t.prune_worktrees_expire = xstrdup(\"3.months.ago\"), \\\n-\t.max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE, \\\n-\t.delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT, \\\n }\n \n static void gc_config_release(struct gc_config *cfg)\n@@ -169,15 +154,12 @@ static void gc_config_release(struct gc_config *cfg)\n \tfree(cfg->gc_log_expire);\n \tfree(cfg->prune_expire);\n \tfree(cfg->prune_worktrees_expire);\n-\tfree(cfg->repack_filter);\n-\tfree(cfg->repack_filter_to);\n }\n \n static void gc_config(struct gc_config *cfg)\n {\n \tconst char *value;\n \tchar *owned = NULL;\n-\tunsigned long ulongval;\n \n \tif (!repo_config_get_value(the_repository, \"gc.packrefs\", &value)) {\n \t\tif (value && !strcmp(value, \"notbare\"))\n@@ -192,8 +174,6 @@ static void gc_config(struct gc_config *cfg)\n \n \trepo_config_get_int(the_repository, \"gc.aggressivewindow\", &cfg->aggressive_window);\n \trepo_config_get_int(the_repository, \"gc.aggressivedepth\", &cfg->aggressive_depth);\n-\trepo_config_get_int(the_repository, \"gc.auto\", &cfg->gc_auto_threshold);\n-\trepo_config_get_int(the_repository, \"gc.autopacklimit\", &cfg->gc_auto_pack_limit);\n \trepo_config_get_bool(the_repository, \"gc.autodetach\", &cfg->detach_auto);\n \trepo_config_get_bool(the_repository, \"gc.cruftpacks\", &cfg->cruft_packs);\n \trepo_config_get_ulong(the_repository, \"gc.maxcruftsize\", &cfg->max_cruft_size);\n@@ -213,22 +193,6 @@ static void gc_config(struct gc_config *cfg)\n \t\tcfg->gc_log_expire = owned;\n \t}\n \n-\trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &cfg->big_pack_threshold);\n-\trepo_config_get_ulong(the_repository, \"pack.deltacachesize\", &cfg->max_delta_cache_size);\n-\n-\tif (!repo_config_get_ulong(the_repository, \"core.deltabasecachelimit\", &ulongval))\n-\t\tcfg->delta_base_cache_limit = ulongval;\n-\n-\tif (!repo_config_get_string(the_repository, \"gc.repackfilter\", &owned)) {\n-\t\tfree(cfg->repack_filter);\n-\t\tcfg->repack_filter = owned;\n-\t}\n-\n-\tif (!repo_config_get_string(the_repository, \"gc.repackfilterto\", &owned)) {\n-\t\tfree(cfg->repack_filter_to);\n-\t\tcfg->repack_filter_to = owned;\n-\t}\n-\n \trepo_config(the_repository, git_default_config, NULL);\n }\n \n@@ -504,12 +468,12 @@ static struct packed_git *find_base_packs(struct string_list *packs,\n \treturn base;\n }\n \n-static int too_many_packs(struct gc_config *cfg)\n+static int too_many_packs(int gc_auto_pack_limit)\n {\n \tstruct packed_git *p;\n \tint cnt = 0;\n \n-\tif (cfg->gc_auto_pack_limit <= 0)\n+\tif (gc_auto_pack_limit <= 0)\n \t\treturn 0;\n \n \trepo_for_each_pack(the_repository, p) {\n@@ -523,7 +487,7 @@ static int too_many_packs(struct gc_config *cfg)\n \t\t */\n \t\tcnt++;\n \t}\n-\treturn cfg->gc_auto_pack_limit < cnt;\n+\treturn gc_auto_pack_limit < cnt;\n }\n \n static uint64_t total_ram(void)\n@@ -571,9 +535,10 @@ static uint64_t total_ram(void)\n \treturn 0;\n }\n \n-static uint64_t estimate_repack_memory(struct gc_config *cfg,\n-\t\t\t\t struct packed_git *pack)\n+static uint64_t estimate_repack_memory(struct packed_git *pack)\n {\n+\tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n+\tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n \tunsigned long nr_objects;\n \tsize_t os_cache, heap;\n \n@@ -584,6 +549,9 @@ static uint64_t estimate_repack_memory(struct gc_config *cfg,\n \tif (!pack || !nr_objects)\n \t\treturn 0;\n \n+\trepo_config_get_ulong(the_repository, \"pack.deltacachesize\", &max_delta_cache_size);\n+\trepo_config_get_ulong(the_repository, \"core.deltabasecachelimit\", &delta_base_cache_limit);\n+\n \t/*\n \t * First we have to scan through at least one pack.\n \t * Assume enough room in OS file cache to keep the entire pack\n@@ -611,9 +579,9 @@ static uint64_t estimate_repack_memory(struct gc_config *cfg,\n \t * read_sha1_file() (either at delta calculation phase, or\n \t * writing phase) also fills up the delta base cache\n \t */\n-\theap += cfg->delta_base_cache_limit;\n+\theap += delta_base_cache_limit;\n \t/* and of course pack-objects has its own delta cache */\n-\theap += cfg->max_delta_cache_size;\n+\theap += max_delta_cache_size;\n \n \treturn os_cache + heap;\n }\n@@ -629,6 +597,12 @@ static void add_repack_all_option(struct gc_config *cfg,\n \t\t\t\t struct string_list *keep_pack,\n \t\t\t\t struct strvec *args)\n {\n+\tchar *repack_filter = NULL;\n+\tchar *repack_filter_to = NULL;\n+\n+\trepo_config_get_string(the_repository, \"gc.repackfilter\", &repack_filter);\n+\trepo_config_get_string(the_repository, \"gc.repackfilterto\", &repack_filter_to);\n+\n \tif (cfg->prune_expire && !strcmp(cfg->prune_expire, \"now\")\n \t\t&& !(cfg->cruft_packs && cfg->repack_expire_to))\n \t\tstrvec_push(args, \"-a\");\n@@ -650,10 +624,13 @@ static void add_repack_all_option(struct gc_config *cfg,\n \tif (keep_pack)\n \t\tfor_each_string_list(keep_pack, keep_one_pack, args);\n \n-\tif (cfg->repack_filter && *cfg->repack_filter)\n-\t\tstrvec_pushf(args, \"--filter=%s\", cfg->repack_filter);\n-\tif (cfg->repack_filter_to && *cfg->repack_filter_to)\n-\t\tstrvec_pushf(args, \"--filter-to=%s\", cfg->repack_filter_to);\n+\tif (repack_filter && *repack_filter)\n+\t\tstrvec_pushf(args, \"--filter=%s\", repack_filter);\n+\tif (repack_filter_to && *repack_filter_to)\n+\t\tstrvec_pushf(args, \"--filter-to=%s\", repack_filter_to);\n+\n+\tfree(repack_filter);\n+\tfree(repack_filter_to);\n }\n \n static void add_repack_incremental_option(struct strvec *args)\n@@ -661,16 +638,24 @@ static void add_repack_incremental_option(struct strvec *args)\n \tstrvec_push(args, \"--no-write-bitmap-index\");\n }\n \n-static int need_to_gc(struct gc_config *cfg)\n+static int need_to_gc(struct repository *repo)\n {\n+\tint gc_auto_threshold = 6700;\n+\tint gc_auto_pack_limit = 50;\n+\n+\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\n \t/*\n \t * Setting gc.auto to 0 or negative can disable the\n \t * automatic gc.\n \t */\n-\tif (cfg->gc_auto_threshold <= 0)\n+\tif (gc_auto_threshold <= 0)\n \t\treturn 0;\n-\tif (!too_many_packs(cfg) && !too_many_loose_objects(cfg->gc_auto_threshold))\n+\tif (!too_many_packs(gc_auto_pack_limit) &&\n+\t !too_many_loose_objects(gc_auto_threshold))\n \t\treturn 0;\n+\n \treturn 1;\n }\n \n@@ -807,8 +792,15 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tint aggressive)\n {\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n+\tunsigned long big_pack_threshold = 0;\n+\tint gc_auto_threshold = 6700;\n+\tint gc_auto_pack_limit = 50;\n \tint ret;\n \n+\trepo_config_get_int(the_repository, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(the_repository, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &big_pack_threshold);\n+\n \tif (the_repository->repository_format_precious_objects)\n \t\treturn 0;\n \n@@ -843,19 +835,18 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\tif (keep_largest_pack != -1) {\n \t\t\tif (keep_largest_pack)\n \t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t} else if (cfg->big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n+\t\t} else if (big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n \t\t}\n \n \t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n \t\tstring_list_clear(&keep_pack, 0);\n-\t} else if (too_many_packs(cfg)) {\n+\t} else if (too_many_packs(gc_auto_pack_limit)) {\n \t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n-\t\tif (cfg->big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, cfg->big_pack_threshold);\n-\t\t\tif (keep_pack.nr >= cfg->gc_auto_pack_limit) {\n-\t\t\t\tcfg->big_pack_threshold = 0;\n+\t\tif (big_pack_threshold) {\n+\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n \t\t\t\tstring_list_clear(&keep_pack, 0);\n \t\t\t\tfind_base_packs(&keep_pack, 0);\n \t\t\t}\n@@ -864,7 +855,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\tuint64_t mem_have, mem_want;\n \n \t\t\tmem_have = total_ram();\n-\t\t\tmem_want = estimate_repack_memory(cfg, p);\n+\t\t\tmem_want = estimate_repack_memory(p);\n \n \t\t\t/*\n \t\t\t * Only allow 1/2 of memory for pack-objects, leave\n@@ -905,7 +896,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t}\n \t}\n \n-\tif (opts->auto_flag && too_many_loose_objects(cfg->gc_auto_threshold))\n+\tif (opts->auto_flag && too_many_loose_objects(gc_auto_threshold))\n \t\twarning(_(\"There are too many unreachable loose objects; \"\n \t\t\t\"run 'git prune' to remove them.\"));\n \n@@ -994,7 +985,7 @@ int cmd_gc(int argc,\n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(&cfg) || run_hooks(the_repository, \"pre-auto-gc\")) {\n+\t\tif (!need_to_gc(the_repository) || run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -1291,9 +1282,9 @@ static int maintenance_task_gc_background(struct maintenance_run_opts *opts,\n \treturn run_command(&child);\n }\n \n-static int gc_condition(struct gc_config *cfg)\n+static int gc_condition(struct gc_config *cfg UNUSED)\n {\n-\treturn need_to_gc(cfg);\n+\treturn need_to_gc(the_repository);\n }\n \n static int prune_packed(struct maintenance_run_opts *opts)\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547953","messageId":"20260713-b4-pks-odb-optimize-v2-7-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 07/12] builtin/gc: introduce object database optimization options","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:10Z","receivedAt":"2026-07-13T05:52:34Z","isPatch":true,"body":"Introduce `struct odb_optimize_options` to decouple the options that are\nspecific to optimizing the object database from `struct gc_config`. This\nstructure will be moved into the object database layer in a subsequent\ncommit.\n\nNote that there are a small set of backend-specific options in this\nstructure. In an ideal world those of course wouldn't exist, but as\nwe're introducing the object database abstractions retroactively we are\nsomewhat forced to keep them.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 181 +++++++++++++++++++++++++++++++++++++++--------------------\n 1 file changed, 120 insertions(+), 61 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 5d445edaa0..17490106fc 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -593,7 +593,39 @@ static int keep_one_pack(struct string_list_item *item, void *data)\n \treturn 0;\n }\n \n-static void add_repack_all_option(struct gc_config *cfg,\n+enum odb_optimize_flags {\n+\t/* Enable verbose logging and progress reporting. */\n+\tODB_OPTIMIZE_VERBOSE = (1 << 0),\n+\n+\t/* Perform auto-maintenance, only optimizing objects as required. */\n+\tODB_OPTIMIZE_AUTO = (1 << 1),\n+\n+\t/* Recompute existing deltas. */\n+\tODB_OPTIMIZE_NO_REUSE_DELTAS = (1 << 2),\n+};\n+\n+struct odb_optimize_options {\n+\tenum odb_optimize_flags flags;\n+\tconst char *prune_expire;\n+\tconst char *expire_to;\n+\tint depth;\n+\tint window;\n+\n+\t/* Backend-specific options. */\n+\tint keep_largest_pack;\n+\tint cruft_packs;\n+\tunsigned long max_cruft_size;\n+};\n+\n+#define OPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive) \\\n+\t.prune_expire = (cfg)->prune_expire, \\\n+\t.expire_to = (cfg)->repack_expire_to, \\\n+\t.cruft_packs = (cfg)->cruft_packs, \\\n+\t.max_cruft_size = (cfg)->max_cruft_size, \\\n+\t.window = (aggressive) ? (cfg)->aggressive_window : 0, \\\n+\t.depth = (aggressive) ? (cfg)->aggressive_depth : 0\n+\n+static void add_repack_all_option(const struct odb_optimize_options *opts,\n \t\t\t\t struct string_list *keep_pack,\n \t\t\t\t struct strvec *args)\n {\n@@ -603,22 +635,22 @@ static void add_repack_all_option(struct gc_config *cfg,\n \trepo_config_get_string(the_repository, \"gc.repackfilter\", &repack_filter);\n \trepo_config_get_string(the_repository, \"gc.repackfilterto\", &repack_filter_to);\n \n-\tif (cfg->prune_expire && !strcmp(cfg->prune_expire, \"now\")\n-\t\t&& !(cfg->cruft_packs && cfg->repack_expire_to))\n+\tif (opts->prune_expire && !strcmp(opts->prune_expire, \"now\") &&\n+\t !(opts->cruft_packs && opts->expire_to))\n \t\tstrvec_push(args, \"-a\");\n-\telse if (cfg->cruft_packs) {\n+\telse if (opts->cruft_packs) {\n \t\tstrvec_push(args, \"--cruft\");\n-\t\tif (cfg->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", cfg->prune_expire);\n-\t\tif (cfg->max_cruft_size)\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", opts->prune_expire);\n+\t\tif (opts->max_cruft_size)\n \t\t\tstrvec_pushf(args, \"--max-cruft-size=%lu\",\n-\t\t\t\t cfg->max_cruft_size);\n-\t\tif (cfg->repack_expire_to)\n-\t\t\tstrvec_pushf(args, \"--expire-to=%s\", cfg->repack_expire_to);\n+\t\t\t\t opts->max_cruft_size);\n+\t\tif (opts->expire_to)\n+\t\t\tstrvec_pushf(args, \"--expire-to=%s\", opts->expire_to);\n \t} else {\n \t\tstrvec_push(args, \"-A\");\n-\t\tif (cfg->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", cfg->prune_expire);\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", opts->prune_expire);\n \t}\n \n \tif (keep_pack)\n@@ -786,10 +818,8 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \treturn 0;\n }\n \n-static int maintenance_task_odb(struct maintenance_run_opts *opts,\n-\t\t\t\tstruct gc_config *cfg,\n-\t\t\t\tint keep_largest_pack,\n-\t\t\t\tint aggressive)\n+static int odb_optimize(struct object_database *odb,\n+\t\t\tconst struct odb_optimize_options *opts)\n {\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \tunsigned long big_pack_threshold = 0;\n@@ -801,21 +831,20 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \trepo_config_get_int(the_repository, \"gc.autopacklimit\", &gc_auto_pack_limit);\n \trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &big_pack_threshold);\n \n-\tif (the_repository->repository_format_precious_objects)\n+\tif (odb->repo->repository_format_precious_objects)\n \t\treturn 0;\n \n \trepack_cmd.git_cmd = 1;\n \trepack_cmd.odb_to_close = the_repository->objects;\n \n \tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (aggressive) {\n+\tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n \t\tstrvec_push(&repack_cmd.args, \"-f\");\n-\t\tif (cfg->aggressive_depth > 0)\n-\t\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", cfg->aggressive_depth);\n-\t\tif (cfg->aggressive_window > 0)\n-\t\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", cfg->aggressive_window);\n-\t}\n-\tif (opts->quiet)\n+\tif (opts->depth > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", opts->depth);\n+\tif (opts->window > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", opts->window);\n+\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n \t\tstrvec_push(&repack_cmd.args, \"-q\");\n \n \t/*\n@@ -829,47 +858,49 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t *\n \t * - Otherwise we perform an incremental repack.\n \t */\n-\tif (!opts->auto_flag) {\n+\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n \t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n-\t\tif (keep_largest_pack != -1) {\n-\t\t\tif (keep_largest_pack)\n+\t\tif (opts->keep_largest_pack != -1) {\n+\t\t\tif (opts->keep_largest_pack)\n \t\t\t\tfind_base_packs(&keep_pack, 0);\n \t\t} else if (big_pack_threshold) {\n \t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n \t\t}\n \n-\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n+\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n \t\tstring_list_clear(&keep_pack, 0);\n-\t} else if (too_many_packs(gc_auto_pack_limit)) {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n-\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t} else {\n+\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\tif (big_pack_threshold) {\n+\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n+\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t}\n+\t\t\t} else {\n+\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\t\tmem_have = total_ram();\n+\t\t\t\tmem_want = estimate_repack_memory(p);\n+\n+\t\t\t\t/*\n+\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t\t * the rest for the OS and other processes in the\n+\t\t\t\t * system.\n+\t\t\t\t */\n+\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n \t\t\t}\n-\t\t} else {\n-\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n-\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\tmem_have = total_ram();\n-\t\t\tmem_want = estimate_repack_memory(p);\n \n-\t\t\t/*\n-\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t * the rest for the OS and other processes in the\n-\t\t\t * system.\n-\t\t\t */\n-\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n+\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t} else {\n+\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n \t\t}\n-\n-\t\tadd_repack_all_option(cfg, &keep_pack, &repack_cmd.args);\n-\t\tstring_list_clear(&keep_pack, 0);\n-\t} else {\n-\t\tadd_repack_incremental_option(&repack_cmd.args);\n \t}\n \n \tif (run_command(&repack_cmd)) {\n@@ -877,13 +908,13 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\tgoto out;\n \t}\n \n-\tif (cfg->prune_expire) {\n+\tif (opts->prune_expire) {\n \t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n \n \t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n \t\t/* run `git prune` even if using cruft packs */\n-\t\tstrvec_push(&prune_cmd.args, cfg->prune_expire);\n-\t\tif (opts->quiet)\n+\t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n \t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n \t\tif (repo_has_promisor_remote(the_repository))\n \t\t\tstrvec_push(&prune_cmd.args,\n@@ -896,7 +927,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t}\n \t}\n \n-\tif (opts->auto_flag && too_many_loose_objects(gc_auto_threshold))\n+\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(gc_auto_threshold))\n \t\twarning(_(\"There are too many unreachable loose objects; \"\n \t\t\t\"run 'git prune' to remove them.\"));\n \n@@ -906,6 +937,26 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \treturn ret;\n }\n \n+static int maintenance_task_odb(struct maintenance_run_opts *opts,\n+\t\t\t\tstruct gc_config *cfg,\n+\t\t\t\tint keep_largest_pack,\n+\t\t\t\tint aggressive)\n+{\n+\tstruct odb_optimize_options odb_opts = {\n+\t\t.keep_largest_pack = keep_largest_pack,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive),\n+\t};\n+\n+\tif (opts->auto_flag)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_AUTO;\n+\tif (!opts->quiet)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n+\tif (aggressive)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_NO_REUSE_DELTAS;\n+\n+\treturn odb_optimize(the_repository->objects, &odb_opts);\n+}\n+\n int cmd_gc(int argc,\n \t const char **argv,\n \t const char *prefix,\n@@ -1596,11 +1647,19 @@ static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,\n \tchild.odb_to_close = the_repository->objects;\n \n \tstrvec_pushl(&child.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (geometry.split < geometry.pack_nr)\n+\tif (geometry.split < geometry.pack_nr) {\n \t\tstrvec_pushf(&child.args, \"--geometric=%d\",\n \t\t\t geometry.split_factor);\n-\telse\n-\t\tadd_repack_all_option(cfg, NULL, &child.args);\n+\t} else {\n+\t\tstruct odb_optimize_options odb_opts = {\n+\t\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n+\t\t};\n+\n+\t\tif (!opts->quiet)\n+\t\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n+\n+\t\tadd_repack_all_option(&odb_opts, NULL, &child.args);\n+\t}\n \tif (opts->quiet)\n \t\tstrvec_push(&child.args, \"--quiet\");\n \tif (the_repository->settings.core_multi_pack_index)\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547954","messageId":"20260713-b4-pks-odb-optimize-v2-8-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 08/12] builtin/gc: move geometric repacking into `odb_optimize()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:11Z","receivedAt":"2026-07-13T05:52:37Z","isPatch":true,"body":"We have two major object database optimization strategies:\n\n - The legacy strategy used by git-gc(1), which absorbs loose objects\n into packfiles, and eventually merges all packfiles once we have too\n many of them.\n\n - The more recent \"geometric\" strategy used by git-maintenance(1),\n which merges packfiles using a geometric sequence.\n\nThese two strategies are still using completely separate code paths. In\na subsequent commit we'll want to make both strategies pluggable though.\n\nPrepare for this change by merging the \"geometric\" strategy into\n`odb_optimize()`. This also allows us to reuse some of the logic we have\nin that function.\n\nNote that this change requires us to adapt tests because we're now using\n\"-q\" instead of \"--quiet\". Naturally though, these invocations are of\ncourse equivalent to one another.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 171 +++++++++++++++++++++++++------------------------\n t/t7900-maintenance.sh | 18 +++---\n 2 files changed, 96 insertions(+), 93 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 17490106fc..c8504f4456 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -593,6 +593,11 @@ static int keep_one_pack(struct string_list_item *item, void *data)\n \treturn 0;\n }\n \n+enum odb_optimize_strategy {\n+\tODB_OPTIMIZE_INCREMENTAL,\n+\tODB_OPTIMIZE_GEOMETRIC,\n+};\n+\n enum odb_optimize_flags {\n \t/* Enable verbose logging and progress reporting. */\n \tODB_OPTIMIZE_VERBOSE = (1 << 0),\n@@ -605,6 +610,7 @@ enum odb_optimize_flags {\n };\n \n struct odb_optimize_options {\n+\tenum odb_optimize_strategy strategy;\n \tenum odb_optimize_flags flags;\n \tconst char *prune_expire;\n \tconst char *expire_to;\n@@ -858,49 +864,87 @@ static int odb_optimize(struct object_database *odb,\n \t *\n \t * - Otherwise we perform an incremental repack.\n \t */\n-\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n-\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\tif (opts->keep_largest_pack != -1) {\n-\t\t\tif (opts->keep_largest_pack)\n-\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t} else if (big_pack_threshold) {\n-\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n-\t\t}\n-\n-\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n-\t\tstring_list_clear(&keep_pack, 0);\n-\t} else {\n-\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL:\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n \t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n-\t\t\tif (big_pack_threshold) {\n-\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n-\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n-\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\tif (opts->keep_largest_pack != -1) {\n+\t\t\t\tif (opts->keep_largest_pack)\n \t\t\t\t\tfind_base_packs(&keep_pack, 0);\n-\t\t\t\t}\n-\t\t\t} else {\n-\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n-\t\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\t\tmem_have = total_ram();\n-\t\t\t\tmem_want = estimate_repack_memory(p);\n-\n-\t\t\t\t/*\n-\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t\t * the rest for the OS and other processes in the\n-\t\t\t\t * system.\n-\t\t\t\t */\n-\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t} else if (big_pack_threshold) {\n+\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n \t\t\t}\n \n \t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n \t\t\tstring_list_clear(&keep_pack, 0);\n \t\t} else {\n-\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\t\tif (big_pack_threshold) {\n+\t\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t\t}\n+\t\t\t\t} else {\n+\t\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\t\t\tmem_have = total_ram();\n+\t\t\t\t\tmem_want = estimate_repack_memory(p);\n+\n+\t\t\t\t\t/*\n+\t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t\t\t * the rest for the OS and other processes in the\n+\t\t\t\t\t * system.\n+\t\t\t\t\t */\n+\t\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t}\n+\n+\t\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t} else {\n+\t\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t\t\t}\n \t\t}\n+\n+\t\tbreak;\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\n+\t\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = the_repository;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\tif (geometry.split < geometry.pack_nr) {\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--geometric=%d\",\n+\t\t\t\t geometry.split_factor);\n+\t\t} else {\n+\t\t\tadd_repack_all_option(opts, NULL, &repack_cmd.args);\n+\t\t}\n+\t\tif (the_repository->settings.core_multi_pack_index)\n+\t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n+\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tdie(\"unknown maintenance strategy '%d'\", opts->strategy);\n \t}\n \n \tif (run_command(&repack_cmd)) {\n@@ -908,7 +952,8 @@ static int odb_optimize(struct object_database *odb,\n \t\tgoto out;\n \t}\n \n-\tif (opts->prune_expire) {\n+\t/* Geometric repacking uses cruft packs, so we don't have to prune separately. */\n+\tif (opts->strategy != ODB_OPTIMIZE_GEOMETRIC && opts->prune_expire) {\n \t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n \n \t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n@@ -943,6 +988,7 @@ static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tint aggressive)\n {\n \tstruct odb_optimize_options odb_opts = {\n+\t\t.strategy = ODB_OPTIMIZE_INCREMENTAL,\n \t\t.keep_largest_pack = keep_largest_pack,\n \t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive),\n \t};\n@@ -1624,58 +1670,15 @@ static int maintenance_task_incremental_repack(struct maintenance_run_opts *opts\n static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,\n \t\t\t\t\t struct gc_config *cfg)\n {\n-\tstruct pack_geometry geometry = {\n-\t\t.split_factor = 2,\n-\t};\n-\tstruct pack_objects_args po_args = {\n-\t\t.local = 1,\n+\tstruct odb_optimize_options odb_opts = {\n+\t\t.strategy = ODB_OPTIMIZE_GEOMETRIC,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n \t};\n-\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\tstruct child_process child = CHILD_PROCESS_INIT;\n-\tint ret;\n-\n-\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t &geometry.split_factor);\n-\n-\texisting_packs.repo = the_repository;\n-\texisting_packs_collect(&existing_packs, &kept_packs);\n-\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\tpack_geometry_split(&geometry);\n-\n-\tchild.git_cmd = 1;\n-\tchild.odb_to_close = the_repository->objects;\n-\n-\tstrvec_pushl(&child.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (geometry.split < geometry.pack_nr) {\n-\t\tstrvec_pushf(&child.args, \"--geometric=%d\",\n-\t\t\t geometry.split_factor);\n-\t} else {\n-\t\tstruct odb_optimize_options odb_opts = {\n-\t\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n-\t\t};\n \n-\t\tif (!opts->quiet)\n-\t\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n-\n-\t\tadd_repack_all_option(&odb_opts, NULL, &child.args);\n-\t}\n-\tif (opts->quiet)\n-\t\tstrvec_push(&child.args, \"--quiet\");\n-\tif (the_repository->settings.core_multi_pack_index)\n-\t\tstrvec_push(&child.args, \"--write-midx\");\n-\n-\tif (run_command(&child)) {\n-\t\tret = error(_(\"failed to perform geometric repack\"));\n-\t\tgoto out;\n-\t}\n-\n-\tret = 0;\n+\tif (!opts->quiet)\n+\t\todb_opts.flags |= ODB_OPTIMIZE_VERBOSE;\n \n-out:\n-\texisting_packs_release(&existing_packs);\n-\tpack_geometry_release(&geometry);\n-\treturn ret;\n+\treturn odb_optimize(the_repository->objects, &odb_opts);\n }\n \n static int geometric_repack_auto_condition(struct gc_config *cfg UNUSED)\ndiff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh\nindex 2d52e7918a..6d87da2ae4 100755\n--- a/t/t7900-maintenance.sh\n+++ b/t/t7900-maintenance.sh\n@@ -574,8 +574,8 @@ run_and_verify_geometric_pack () {\n \trm -f \"trace2.txt\" &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/trace2.txt\" \\\n \t\tgit maintenance run --task=geometric-repack 2>/dev/null &&\n-\ttest_subcommand git repack -d -l --geometric=2 \\\n-\t\t--quiet --write-midx packfiles &&\n@@ -606,8 +606,8 @@ test_expect_success 'geometric repacking task' '\n \t\t# The initial repack causes an all-into-one repack.\n \t\tGIT_TRACE2_EVENT=\"$(pwd)/initial-repack.txt\" \\\n \t\t\tgit maintenance run --task=geometric-repack 2>/dev/null &&\n-\t\ttest_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \\\n-\t\t\t--quiet --write-midx /dev/null &&\n-\t\ttest_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \\\n-\t\t\t--quiet --write-midx /dev/null &&\n-\t\ttest_subcommand git repack -d -l --cruft --cruft-expiration=2.weeks.ago \\\n-\t\t\t--quiet --write-midx packs &&\n \t\ttest_line_count = 2 packs &&\n \t\tls .git/objects/pack/*.mtimes >cruft &&\n@@ -754,7 +754,7 @@ test_expect_success 'geometric repacking honors configured split factor' '\n \n \t\ttest_geometric_repack_needed false splitFactor=2 &&\n \t\ttest_geometric_repack_needed true splitFactor=3 &&\n-\t\ttest_subcommand git repack -d -l --geometric=3 --quiet --write-midx \n---\n builtin/gc.c | 160 ++++++++++++++++++++++++++++++++++-------------------------\n 1 file changed, 92 insertions(+), 68 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex c8504f4456..e119930adc 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -676,25 +676,84 @@ static void add_repack_incremental_option(struct strvec *args)\n \tstrvec_push(args, \"--no-write-bitmap-index\");\n }\n \n-static int need_to_gc(struct repository *repo)\n+static bool odb_optimize_required(struct object_database *odb,\n+\t\t\t\t const struct odb_optimize_options *opts)\n {\n-\tint gc_auto_threshold = 6700;\n-\tint gc_auto_pack_limit = 50;\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL: {\n+\t\tint gc_auto_threshold = 6700;\n+\t\tint gc_auto_pack_limit = 50;\n \n-\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n-\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\t\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n+\t\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n \n-\t/*\n-\t * Setting gc.auto to 0 or negative can disable the\n-\t * automatic gc.\n-\t */\n-\tif (gc_auto_threshold <= 0)\n-\t\treturn 0;\n-\tif (!too_many_packs(gc_auto_pack_limit) &&\n-\t !too_many_loose_objects(gc_auto_threshold))\n-\t\treturn 0;\n+\t\t/*\n+\t\t * Setting gc.auto to 0 or negative can disable the\n+\t\t * automatic gc.\n+\t\t */\n+\t\tif (gc_auto_threshold <= 0)\n+\t\t\treturn false;\n+\t\tif (!too_many_packs(gc_auto_pack_limit) &&\n+\t\t !too_many_loose_objects(gc_auto_threshold))\n+\t\t\treturn false;\n \n-\treturn 1;\n+\t\treturn true;\n+\t}\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\t\tint auto_value = 100;\n+\t\tbool ret;\n+\n+\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.auto\",\n+\t\t\t\t &auto_value);\n+\t\tif (!auto_value)\n+\t\t\treturn false;\n+\t\tif (auto_value < 0)\n+\t\t\treturn true;\n+\n+\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = odb->repo;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\t/*\n+\t\t * When we'd merge at least two packs with one another we always\n+\t\t * perform the repack.\n+\t\t */\n+\t\tif (geometry.split) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\t/*\n+\t\t * Otherwise, we estimate the number of loose objects to determine\n+\t\t * whether we want to create a new packfile or not.\n+\t\t */\n+\t\tif (too_many_loose_objects(auto_value)) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tret = false;\n+\n+\tout:\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\treturn ret;\n+\t}\n+\tdefault:\n+\t\tBUG(\"unknown maintenance strategy '%d'\", opts->strategy);\n+\t}\n }\n \n /* return NULL on success, else hostname running the gc */\n@@ -1076,13 +1135,19 @@ int cmd_gc(int argc,\n \t\tdie(_(\"failed to parse prune expiry value %s\"), cfg.prune_expire);\n \n \tif (opts.auto_flag) {\n+\t\tstruct odb_optimize_options optimize_opts = {\n+\t\t\t.strategy = ODB_OPTIMIZE_INCREMENTAL,\n+\t\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(&cfg, 0),\n+\t\t};\n+\n \t\tif (cfg.detach_auto && opts.detach < 0)\n \t\t\topts.detach = 1;\n \n \t\t/*\n \t\t * Auto-gc should be least intrusive as possible.\n \t\t */\n-\t\tif (!need_to_gc(the_repository) || run_hooks(the_repository, \"pre-auto-gc\")) {\n+\t\tif (!odb_optimize_required(the_repository->objects, &optimize_opts) ||\n+\t\t run_hooks(the_repository, \"pre-auto-gc\")) {\n \t\t\tret = 0;\n \t\t\tgoto out;\n \t\t}\n@@ -1379,9 +1444,13 @@ static int maintenance_task_gc_background(struct maintenance_run_opts *opts,\n \treturn run_command(&child);\n }\n \n-static int gc_condition(struct gc_config *cfg UNUSED)\n+static int gc_condition(struct gc_config *cfg)\n {\n-\treturn need_to_gc(the_repository);\n+\tstruct odb_optimize_options opts = {\n+\t\t.strategy = ODB_OPTIMIZE_INCREMENTAL,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n+\t};\n+\treturn odb_optimize_required(the_repository->objects, &opts);\n }\n \n static int prune_packed(struct maintenance_run_opts *opts)\n@@ -1681,58 +1750,13 @@ static int maintenance_task_geometric_repack(struct maintenance_run_opts *opts,\n \treturn odb_optimize(the_repository->objects, &odb_opts);\n }\n \n-static int geometric_repack_auto_condition(struct gc_config *cfg UNUSED)\n+static int geometric_repack_auto_condition(struct gc_config *cfg)\n {\n-\tstruct pack_geometry geometry = {\n-\t\t.split_factor = 2,\n-\t};\n-\tstruct pack_objects_args po_args = {\n-\t\t.local = 1,\n+\tstruct odb_optimize_options opts = {\n+\t\t.strategy = ODB_OPTIMIZE_GEOMETRIC,\n+\t\tOPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, 0),\n \t};\n-\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\tint auto_value = 100;\n-\tint ret;\n-\n-\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.auto\",\n-\t\t\t &auto_value);\n-\tif (!auto_value)\n-\t\treturn 0;\n-\tif (auto_value < 0)\n-\t\treturn 1;\n-\n-\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t &geometry.split_factor);\n-\n-\texisting_packs.repo = the_repository;\n-\texisting_packs_collect(&existing_packs, &kept_packs);\n-\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\tpack_geometry_split(&geometry);\n-\n-\t/*\n-\t * When we'd merge at least two packs with one another we always\n-\t * perform the repack.\n-\t */\n-\tif (geometry.split) {\n-\t\tret = 1;\n-\t\tgoto out;\n-\t}\n-\n-\t/*\n-\t * Otherwise, we estimate the number of loose objects to determine\n-\t * whether we want to create a new packfile or not.\n-\t */\n-\tif (too_many_loose_objects(auto_value)) {\n-\t\tret = 1;\n-\t\tgoto out;\n-\t}\n-\n-\tret = 0;\n-\n-out:\n-\texisting_packs_release(&existing_packs);\n-\tpack_geometry_release(&geometry);\n-\treturn ret;\n+\treturn odb_optimize_required(the_repository->objects, &opts);\n }\n \n typedef int (*maintenance_task_fn)(struct maintenance_run_opts *opts,\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547956","messageId":"20260713-b4-pks-odb-optimize-v2-10-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 10/12] builtin/gc: refactor ODB optimizations to operate on \"files\" source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:13Z","receivedAt":"2026-07-13T05:52:42Z","isPatch":true,"body":"We have a couple of functions that are implementation details of how the\n\"files\" object database source performs optimizations. These functions\noften use global state like `the_repository` and implicitly derive the\nsource they are supposed to optimize.\n\nRefactor these interfaces to accept a \"files\" source directly. This will\nmake it easier to move around the whole logic into \"odb/source-files.c\"\nin a subsequent step.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 79 +++++++++++++++++++++++++++++++-----------------------------\n 1 file changed, 41 insertions(+), 38 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex e119930adc..3207182488 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -428,9 +428,8 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n \treturn should_gc;\n }\n \n-static int too_many_loose_objects(int limit)\n+static int too_many_loose_objects(struct odb_source_files *files, int limit)\n {\n-\tstruct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);\n \t/*\n \t * This is weird, but stems from legacy behaviour: the GC auto\n \t * threshold was always essentially interpreted as if it was rounded up\n@@ -446,19 +445,21 @@ static int too_many_loose_objects(int limit)\n \treturn loose_count > auto_threshold;\n }\n \n-static struct packed_git *find_base_packs(struct string_list *packs,\n+static struct packed_git *find_base_packs(struct odb_source_files *files,\n+\t\t\t\t\t struct string_list *packs,\n \t\t\t\t\t unsigned long limit)\n {\n-\tstruct packed_git *p, *base = NULL;\n+\tstruct packfile_list_entry *e;\n+\tstruct packed_git *base = NULL;\n \n-\trepo_for_each_pack(the_repository, p) {\n-\t\tif (!p->pack_local || p->is_cruft)\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->is_cruft)\n \t\t\tcontinue;\n \t\tif (limit) {\n-\t\t\tif (p->pack_size >= limit)\n-\t\t\t\tstring_list_append(packs, p->pack_name);\n-\t\t} else if (!base || base->pack_size < p->pack_size) {\n-\t\t\tbase = p;\n+\t\t\tif (e->pack->pack_size >= limit)\n+\t\t\t\tstring_list_append(packs, e->pack->pack_name);\n+\t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n+\t\t\tbase = e->pack;\n \t\t}\n \t}\n \n@@ -468,18 +469,16 @@ static struct packed_git *find_base_packs(struct string_list *packs,\n \treturn base;\n }\n \n-static int too_many_packs(int gc_auto_pack_limit)\n+static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit)\n {\n-\tstruct packed_git *p;\n+\tstruct packfile_list_entry *e;\n \tint cnt = 0;\n \n \tif (gc_auto_pack_limit <= 0)\n \t\treturn 0;\n \n-\trepo_for_each_pack(the_repository, p) {\n-\t\tif (!p->pack_local)\n-\t\t\tcontinue;\n-\t\tif (p->pack_keep)\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->pack_keep)\n \t\t\tcontinue;\n \t\t/*\n \t\t * Perhaps check the size of the pack and count only\n@@ -535,15 +534,16 @@ static uint64_t total_ram(void)\n \treturn 0;\n }\n \n-static uint64_t estimate_repack_memory(struct packed_git *pack)\n+static uint64_t estimate_repack_memory(struct odb_source_files *files,\n+\t\t\t\t struct packed_git *pack)\n {\n \tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n \tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n \tunsigned long nr_objects;\n \tsize_t os_cache, heap;\n \n-\tif (odb_count_objects(the_repository->objects,\n-\t\t\t ODB_COUNT_OBJECTS_APPROXIMATE, &nr_objects) < 0)\n+\tif (odb_source_count_objects(&files->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n+\t\t\t\t &nr_objects) < 0)\n \t\treturn 0;\n \n \tif (!pack || !nr_objects)\n@@ -679,6 +679,8 @@ static void add_repack_incremental_option(struct strvec *args)\n static bool odb_optimize_required(struct object_database *odb,\n \t\t\t\t const struct odb_optimize_options *opts)\n {\n+\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n+\n \tswitch (opts->strategy) {\n \tcase ODB_OPTIMIZE_INCREMENTAL: {\n \t\tint gc_auto_threshold = 6700;\n@@ -693,8 +695,8 @@ static bool odb_optimize_required(struct object_database *odb,\n \t\t */\n \t\tif (gc_auto_threshold <= 0)\n \t\t\treturn false;\n-\t\tif (!too_many_packs(gc_auto_pack_limit) &&\n-\t\t !too_many_loose_objects(gc_auto_threshold))\n+\t\tif (!too_many_packs(files, gc_auto_pack_limit) &&\n+\t\t !too_many_loose_objects(files, gc_auto_threshold))\n \t\t\treturn false;\n \n \t\treturn true;\n@@ -739,7 +741,7 @@ static bool odb_optimize_required(struct object_database *odb,\n \t\t * Otherwise, we estimate the number of loose objects to determine\n \t\t * whether we want to create a new packfile or not.\n \t\t */\n-\t\tif (too_many_loose_objects(auto_value)) {\n+\t\tif (too_many_loose_objects(files, auto_value)) {\n \t\t\tret = true;\n \t\t\tgoto out;\n \t\t}\n@@ -886,21 +888,22 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n static int odb_optimize(struct object_database *odb,\n \t\t\tconst struct odb_optimize_options *opts)\n {\n+\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n \tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n \tunsigned long big_pack_threshold = 0;\n \tint gc_auto_threshold = 6700;\n \tint gc_auto_pack_limit = 50;\n \tint ret;\n \n-\trepo_config_get_int(the_repository, \"gc.auto\", &gc_auto_threshold);\n-\trepo_config_get_int(the_repository, \"gc.autopacklimit\", &gc_auto_pack_limit);\n-\trepo_config_get_ulong(the_repository, \"gc.bigpackthreshold\", &big_pack_threshold);\n+\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\trepo_config_get_ulong(odb->repo, \"gc.bigpackthreshold\", &big_pack_threshold);\n \n \tif (odb->repo->repository_format_precious_objects)\n \t\treturn 0;\n \n \trepack_cmd.git_cmd = 1;\n-\trepack_cmd.odb_to_close = the_repository->objects;\n+\trepack_cmd.odb_to_close = odb->repo->objects;\n \n \tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n \tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n@@ -930,29 +933,29 @@ static int odb_optimize(struct object_database *odb,\n \n \t\t\tif (opts->keep_largest_pack != -1) {\n \t\t\t\tif (opts->keep_largest_pack)\n-\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n \t\t\t} else if (big_pack_threshold) {\n-\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n \t\t\t}\n \n \t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n \t\t\tstring_list_clear(&keep_pack, 0);\n \t\t} else {\n-\t\t\tif (too_many_packs(gc_auto_pack_limit)) {\n+\t\t\tif (too_many_packs(files, gc_auto_pack_limit)) {\n \t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n \n \t\t\t\tif (big_pack_threshold) {\n-\t\t\t\t\tfind_base_packs(&keep_pack, big_pack_threshold);\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n \t\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n \t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\t\t\tfind_base_packs(&keep_pack, 0);\n+\t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n \t\t\t\t\t}\n \t\t\t\t} else {\n-\t\t\t\t\tstruct packed_git *p = find_base_packs(&keep_pack, 0);\n+\t\t\t\t\tstruct packed_git *p = find_base_packs(files, &keep_pack, 0);\n \t\t\t\t\tuint64_t mem_have, mem_want;\n \n \t\t\t\t\tmem_have = total_ram();\n-\t\t\t\t\tmem_want = estimate_repack_memory(p);\n+\t\t\t\t\tmem_want = estimate_repack_memory(files, p);\n \n \t\t\t\t\t/*\n \t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n@@ -981,10 +984,10 @@ static int odb_optimize(struct object_database *odb,\n \t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n \t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n \n-\t\trepo_config_get_int(the_repository, \"maintenance.geometric-repack.splitFactor\",\n+\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n \t\t\t\t &geometry.split_factor);\n \n-\t\texisting_packs.repo = the_repository;\n+\t\texisting_packs.repo = odb->repo;\n \t\texisting_packs_collect(&existing_packs, &kept_packs);\n \t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n \t\tpack_geometry_split(&geometry);\n@@ -995,7 +998,7 @@ static int odb_optimize(struct object_database *odb,\n \t\t} else {\n \t\t\tadd_repack_all_option(opts, NULL, &repack_cmd.args);\n \t\t}\n-\t\tif (the_repository->settings.core_multi_pack_index)\n+\t\tif (odb->repo->settings.core_multi_pack_index)\n \t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n \n \t\texisting_packs_release(&existing_packs);\n@@ -1020,7 +1023,7 @@ static int odb_optimize(struct object_database *odb,\n \t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n \t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n \t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n-\t\tif (repo_has_promisor_remote(the_repository))\n+\t\tif (repo_has_promisor_remote(odb->repo))\n \t\t\tstrvec_push(&prune_cmd.args,\n \t\t\t\t \"--exclude-promisor-objects\");\n \t\tprune_cmd.git_cmd = 1;\n@@ -1031,7 +1034,7 @@ static int odb_optimize(struct object_database *odb,\n \t\t}\n \t}\n \n-\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(gc_auto_threshold))\n+\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(files, gc_auto_threshold))\n \t\twarning(_(\"There are too many unreachable loose objects; \"\n \t\t\t\"run 'git prune' to remove them.\"));\n \n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547957","messageId":"20260713-b4-pks-odb-optimize-v2-11-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 11/12] builtin/gc: fix signedness issues in ODB-related functionality","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:14Z","receivedAt":"2026-07-13T05:52:44Z","isPatch":true,"body":"There are a couple of signedness issues in ODB-related functionality.\nThese are not a problem because we disable -Wsign-compare in this file,\nbut once we move these functions into \"odb/source-files.c\" they will\nresult in warnings.\n\nFix those issues:\n\n - In `too_many_loose_objects()` we receive a signed limit, but compare\n it with the unsigned actual number of loose objects. This is fixed\n by bailing out immediately when the limit is smaller than or equal\n to zero, which we also do similarly in other places. The warning is\n then squelched via a cast.\n\n - In `find_base_packs()` we compare the signed size of the pack\n against the unsigned limit. As the pack size is always going to be a\n positive file size it's safe to cast it to an unsigned value.\n\n - In `odb_optimize()` we compare the unsigned `keep_pack.nr` value\n against the signed `gc_auto_pack_limit`. We only reach this code\n when `too_many_packs()` returns true-ish, and that can only happen\n when `gc_auto_pack_limit > 0`. Consequently, we can fix the warning\n by casting the limit to an unsigned value.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 20 +++++++++++---------\n 1 file changed, 11 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 3207182488..8cf3781313 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -430,19 +430,21 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n \n static int too_many_loose_objects(struct odb_source_files *files, int limit)\n {\n-\t/*\n-\t * This is weird, but stems from legacy behaviour: the GC auto\n-\t * threshold was always essentially interpreted as if it was rounded up\n-\t * to the next multiple 256 of, so we retain this behaviour for now.\n-\t */\n-\tint auto_threshold = DIV_ROUND_UP(limit, 256) * 256;\n \tunsigned long loose_count;\n \n+\tif (limit <= 0)\n+\t\treturn 0;\n+\n \tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n \t\t\t\t &loose_count) < 0)\n \t\treturn 0;\n \n-\treturn loose_count > auto_threshold;\n+\t/*\n+\t * This is weird, but stems from legacy behaviour: the GC auto\n+\t * threshold was always essentially interpreted as if it was rounded up\n+\t * to the next multiple 256 of, so we retain this behaviour for now.\n+\t */\n+\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n }\n \n static struct packed_git *find_base_packs(struct odb_source_files *files,\n@@ -456,7 +458,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,\n \t\tif (e->pack->is_cruft)\n \t\t\tcontinue;\n \t\tif (limit) {\n-\t\t\tif (e->pack->pack_size >= limit)\n+\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n \t\t\t\tstring_list_append(packs, e->pack->pack_name);\n \t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n \t\t\tbase = e->pack;\n@@ -946,7 +948,7 @@ static int odb_optimize(struct object_database *odb,\n \n \t\t\t\tif (big_pack_threshold) {\n \t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n-\t\t\t\t\tif (keep_pack.nr >= gc_auto_pack_limit) {\n+\t\t\t\t\tif (keep_pack.nr >= (unsigned long) gc_auto_pack_limit) {\n \t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n \t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n \t\t\t\t\t}\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"547958","messageId":"20260713-b4-pks-odb-optimize-v2-12-9c2c3ee94b38@pks.im","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-0-9c2c3ee94b38@pks.im","subject":"[PATCH v2 12/12] odb: make optimizations pluggable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T05:52:15Z","receivedAt":"2026-07-13T05:52:47Z","isPatch":true,"body":"Move `odb_optimize()` and `odb_optimize_required()` from \"builtin/gc.c\"\ninto the \"files\" source and wire them up via newly introduced vtable\npointers for the object database sources. This makes the logic pluggable\nand thus allows other backends to have their own, custom implementation.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/gc.c | 490 +----------------------------------------------------\n odb.c | 12 ++\n odb.h | 45 +++++\n odb/source-files.c | 470 ++++++++++++++++++++++++++++++++++++++++++++++++++\n odb/source-files.h | 15 ++\n odb/source.h | 36 ++++\n 6 files changed, 579 insertions(+), 489 deletions(-)\n\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 8cf3781313..ac1a21e912 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -30,16 +30,11 @@\n #include \"commit-graph.h\"\n #include \"packfile.h\"\n #include \"object-file.h\"\n-#include \"pack.h\"\n-#include \"pack-objects.h\"\n+#include \"odb.h\"\n #include \"path.h\"\n #include \"reflog.h\"\n-#include \"repack.h\"\n #include \"rerere.h\"\n #include \"revision.h\"\n-#include \"blob.h\"\n-#include \"tree.h\"\n-#include \"promisor-remote.h\"\n #include \"refs.h\"\n #include \"remote.h\"\n #include \"exec-cmd.h\"\n@@ -428,203 +423,6 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n \treturn should_gc;\n }\n \n-static int too_many_loose_objects(struct odb_source_files *files, int limit)\n-{\n-\tunsigned long loose_count;\n-\n-\tif (limit <= 0)\n-\t\treturn 0;\n-\n-\tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n-\t\t\t\t &loose_count) < 0)\n-\t\treturn 0;\n-\n-\t/*\n-\t * This is weird, but stems from legacy behaviour: the GC auto\n-\t * threshold was always essentially interpreted as if it was rounded up\n-\t * to the next multiple 256 of, so we retain this behaviour for now.\n-\t */\n-\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n-}\n-\n-static struct packed_git *find_base_packs(struct odb_source_files *files,\n-\t\t\t\t\t struct string_list *packs,\n-\t\t\t\t\t unsigned long limit)\n-{\n-\tstruct packfile_list_entry *e;\n-\tstruct packed_git *base = NULL;\n-\n-\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n-\t\tif (e->pack->is_cruft)\n-\t\t\tcontinue;\n-\t\tif (limit) {\n-\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n-\t\t\t\tstring_list_append(packs, e->pack->pack_name);\n-\t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n-\t\t\tbase = e->pack;\n-\t\t}\n-\t}\n-\n-\tif (base)\n-\t\tstring_list_append(packs, base->pack_name);\n-\n-\treturn base;\n-}\n-\n-static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit)\n-{\n-\tstruct packfile_list_entry *e;\n-\tint cnt = 0;\n-\n-\tif (gc_auto_pack_limit <= 0)\n-\t\treturn 0;\n-\n-\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n-\t\tif (e->pack->pack_keep)\n-\t\t\tcontinue;\n-\t\t/*\n-\t\t * Perhaps check the size of the pack and count only\n-\t\t * very small ones here?\n-\t\t */\n-\t\tcnt++;\n-\t}\n-\treturn gc_auto_pack_limit < cnt;\n-}\n-\n-static uint64_t total_ram(void)\n-{\n-#if defined(HAVE_SYSINFO)\n-\tstruct sysinfo si;\n-\n-\tif (!sysinfo(&si)) {\n-\t\tuint64_t total = si.totalram;\n-\n-\t\tif (si.mem_unit > 1)\n-\t\t\ttotal *= (uint64_t)si.mem_unit;\n-\t\treturn total;\n-\t}\n-#elif defined(HAVE_BSD_SYSCTL) && (defined(HW_MEMSIZE) || defined(HW_PHYSMEM) || defined(HW_PHYSMEM64))\n-\tuint64_t physical_memory;\n-\tint mib[2];\n-\tsize_t length;\n-\n-\tmib[0] = CTL_HW;\n-# if defined(HW_MEMSIZE)\n-\tmib[1] = HW_MEMSIZE;\n-# elif defined(HW_PHYSMEM64)\n-\tmib[1] = HW_PHYSMEM64;\n-# else\n-\tmib[1] = HW_PHYSMEM;\n-# endif\n-\tlength = sizeof(physical_memory);\n-\tif (!sysctl(mib, 2, &physical_memory, &length, NULL, 0)) {\n-\t\tif (length == 4) {\n-\t\t\tuint32_t mem;\n-\n-\t\t\tif (!sysctl(mib, 2, &mem, &length, NULL, 0))\n-\t\t\t\tphysical_memory = mem;\n-\t\t}\n-\t\treturn physical_memory;\n-\t}\n-#elif defined(GIT_WINDOWS_NATIVE)\n-\tMEMORYSTATUSEX memInfo;\n-\n-\tmemInfo.dwLength = sizeof(MEMORYSTATUSEX);\n-\tif (GlobalMemoryStatusEx(&memInfo))\n-\t\treturn memInfo.ullTotalPhys;\n-#endif\n-\treturn 0;\n-}\n-\n-static uint64_t estimate_repack_memory(struct odb_source_files *files,\n-\t\t\t\t struct packed_git *pack)\n-{\n-\tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n-\tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n-\tunsigned long nr_objects;\n-\tsize_t os_cache, heap;\n-\n-\tif (odb_source_count_objects(&files->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n-\t\t\t\t &nr_objects) < 0)\n-\t\treturn 0;\n-\n-\tif (!pack || !nr_objects)\n-\t\treturn 0;\n-\n-\trepo_config_get_ulong(the_repository, \"pack.deltacachesize\", &max_delta_cache_size);\n-\trepo_config_get_ulong(the_repository, \"core.deltabasecachelimit\", &delta_base_cache_limit);\n-\n-\t/*\n-\t * First we have to scan through at least one pack.\n-\t * Assume enough room in OS file cache to keep the entire pack\n-\t * or we may accidentally evict data of other processes from\n-\t * the cache.\n-\t */\n-\tos_cache = pack->pack_size + pack->index_size;\n-\t/* then pack-objects needs lots more for book keeping */\n-\theap = sizeof(struct object_entry) * nr_objects;\n-\t/*\n-\t * internal rev-list --all --objects takes up some memory too,\n-\t * let's say half of it is for blobs\n-\t */\n-\theap += sizeof(struct blob) * nr_objects / 2;\n-\t/*\n-\t * and the other half is for trees (commits and tags are\n-\t * usually insignificant)\n-\t */\n-\theap += sizeof(struct tree) * nr_objects / 2;\n-\t/* and then obj_hash[], underestimated in fact */\n-\theap += sizeof(struct object *) * nr_objects;\n-\t/* revindex is used also */\n-\theap += (sizeof(off_t) + sizeof(uint32_t)) * nr_objects;\n-\t/*\n-\t * read_sha1_file() (either at delta calculation phase, or\n-\t * writing phase) also fills up the delta base cache\n-\t */\n-\theap += delta_base_cache_limit;\n-\t/* and of course pack-objects has its own delta cache */\n-\theap += max_delta_cache_size;\n-\n-\treturn os_cache + heap;\n-}\n-\n-static int keep_one_pack(struct string_list_item *item, void *data)\n-{\n-\tstruct strvec *args = data;\n-\tstrvec_pushf(args, \"--keep-pack=%s\", basename(item->string));\n-\treturn 0;\n-}\n-\n-enum odb_optimize_strategy {\n-\tODB_OPTIMIZE_INCREMENTAL,\n-\tODB_OPTIMIZE_GEOMETRIC,\n-};\n-\n-enum odb_optimize_flags {\n-\t/* Enable verbose logging and progress reporting. */\n-\tODB_OPTIMIZE_VERBOSE = (1 << 0),\n-\n-\t/* Perform auto-maintenance, only optimizing objects as required. */\n-\tODB_OPTIMIZE_AUTO = (1 << 1),\n-\n-\t/* Recompute existing deltas. */\n-\tODB_OPTIMIZE_NO_REUSE_DELTAS = (1 << 2),\n-};\n-\n-struct odb_optimize_options {\n-\tenum odb_optimize_strategy strategy;\n-\tenum odb_optimize_flags flags;\n-\tconst char *prune_expire;\n-\tconst char *expire_to;\n-\tint depth;\n-\tint window;\n-\n-\t/* Backend-specific options. */\n-\tint keep_largest_pack;\n-\tint cruft_packs;\n-\tunsigned long max_cruft_size;\n-};\n-\n #define OPTIMIZE_FIELDS_FROM_GC_CONFIG(cfg, aggressive) \\\n \t.prune_expire = (cfg)->prune_expire, \\\n \t.expire_to = (cfg)->repack_expire_to, \\\n@@ -633,133 +431,6 @@ struct odb_optimize_options {\n \t.window = (aggressive) ? (cfg)->aggressive_window : 0, \\\n \t.depth = (aggressive) ? (cfg)->aggressive_depth : 0\n \n-static void add_repack_all_option(const struct odb_optimize_options *opts,\n-\t\t\t\t struct string_list *keep_pack,\n-\t\t\t\t struct strvec *args)\n-{\n-\tchar *repack_filter = NULL;\n-\tchar *repack_filter_to = NULL;\n-\n-\trepo_config_get_string(the_repository, \"gc.repackfilter\", &repack_filter);\n-\trepo_config_get_string(the_repository, \"gc.repackfilterto\", &repack_filter_to);\n-\n-\tif (opts->prune_expire && !strcmp(opts->prune_expire, \"now\") &&\n-\t !(opts->cruft_packs && opts->expire_to))\n-\t\tstrvec_push(args, \"-a\");\n-\telse if (opts->cruft_packs) {\n-\t\tstrvec_push(args, \"--cruft\");\n-\t\tif (opts->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", opts->prune_expire);\n-\t\tif (opts->max_cruft_size)\n-\t\t\tstrvec_pushf(args, \"--max-cruft-size=%lu\",\n-\t\t\t\t opts->max_cruft_size);\n-\t\tif (opts->expire_to)\n-\t\t\tstrvec_pushf(args, \"--expire-to=%s\", opts->expire_to);\n-\t} else {\n-\t\tstrvec_push(args, \"-A\");\n-\t\tif (opts->prune_expire)\n-\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", opts->prune_expire);\n-\t}\n-\n-\tif (keep_pack)\n-\t\tfor_each_string_list(keep_pack, keep_one_pack, args);\n-\n-\tif (repack_filter && *repack_filter)\n-\t\tstrvec_pushf(args, \"--filter=%s\", repack_filter);\n-\tif (repack_filter_to && *repack_filter_to)\n-\t\tstrvec_pushf(args, \"--filter-to=%s\", repack_filter_to);\n-\n-\tfree(repack_filter);\n-\tfree(repack_filter_to);\n-}\n-\n-static void add_repack_incremental_option(struct strvec *args)\n-{\n-\tstrvec_push(args, \"--no-write-bitmap-index\");\n-}\n-\n-static bool odb_optimize_required(struct object_database *odb,\n-\t\t\t\t const struct odb_optimize_options *opts)\n-{\n-\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n-\n-\tswitch (opts->strategy) {\n-\tcase ODB_OPTIMIZE_INCREMENTAL: {\n-\t\tint gc_auto_threshold = 6700;\n-\t\tint gc_auto_pack_limit = 50;\n-\n-\t\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n-\t\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n-\n-\t\t/*\n-\t\t * Setting gc.auto to 0 or negative can disable the\n-\t\t * automatic gc.\n-\t\t */\n-\t\tif (gc_auto_threshold <= 0)\n-\t\t\treturn false;\n-\t\tif (!too_many_packs(files, gc_auto_pack_limit) &&\n-\t\t !too_many_loose_objects(files, gc_auto_threshold))\n-\t\t\treturn false;\n-\n-\t\treturn true;\n-\t}\n-\tcase ODB_OPTIMIZE_GEOMETRIC: {\n-\t\tstruct pack_geometry geometry = {\n-\t\t\t.split_factor = 2,\n-\t\t};\n-\t\tstruct pack_objects_args po_args = {\n-\t\t\t.local = 1,\n-\t\t};\n-\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\t\tint auto_value = 100;\n-\t\tbool ret;\n-\n-\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.auto\",\n-\t\t\t\t &auto_value);\n-\t\tif (!auto_value)\n-\t\t\treturn false;\n-\t\tif (auto_value < 0)\n-\t\t\treturn true;\n-\n-\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t\t &geometry.split_factor);\n-\n-\t\texisting_packs.repo = odb->repo;\n-\t\texisting_packs_collect(&existing_packs, &kept_packs);\n-\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\t\tpack_geometry_split(&geometry);\n-\n-\t\t/*\n-\t\t * When we'd merge at least two packs with one another we always\n-\t\t * perform the repack.\n-\t\t */\n-\t\tif (geometry.split) {\n-\t\t\tret = true;\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\t/*\n-\t\t * Otherwise, we estimate the number of loose objects to determine\n-\t\t * whether we want to create a new packfile or not.\n-\t\t */\n-\t\tif (too_many_loose_objects(files, auto_value)) {\n-\t\t\tret = true;\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tret = false;\n-\n-\tout:\n-\t\texisting_packs_release(&existing_packs);\n-\t\tpack_geometry_release(&geometry);\n-\t\treturn ret;\n-\t}\n-\tdefault:\n-\t\tBUG(\"unknown maintenance strategy '%d'\", opts->strategy);\n-\t}\n-}\n-\n /* return NULL on success, else hostname running the gc */\n static const char *lock_repo_for_gc(int force, pid_t* ret_pid)\n {\n@@ -887,165 +558,6 @@ static int gc_foreground_tasks(struct maintenance_run_opts *opts,\n \treturn 0;\n }\n \n-static int odb_optimize(struct object_database *odb,\n-\t\t\tconst struct odb_optimize_options *opts)\n-{\n-\tstruct odb_source_files *files = odb_source_files_downcast(odb->sources);\n-\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n-\tunsigned long big_pack_threshold = 0;\n-\tint gc_auto_threshold = 6700;\n-\tint gc_auto_pack_limit = 50;\n-\tint ret;\n-\n-\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n-\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n-\trepo_config_get_ulong(odb->repo, \"gc.bigpackthreshold\", &big_pack_threshold);\n-\n-\tif (odb->repo->repository_format_precious_objects)\n-\t\treturn 0;\n-\n-\trepack_cmd.git_cmd = 1;\n-\trepack_cmd.odb_to_close = odb->repo->objects;\n-\n-\tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n-\tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n-\t\tstrvec_push(&repack_cmd.args, \"-f\");\n-\tif (opts->depth > 0)\n-\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", opts->depth);\n-\tif (opts->window > 0)\n-\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", opts->window);\n-\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n-\t\tstrvec_push(&repack_cmd.args, \"-q\");\n-\n-\t/*\n-\t * There's three cases we need to consider:\n-\t *\n-\t * - If we're invoked without `--auto` we'll need to perform a full\n-\t * repack.\n-\t *\n-\t * - If we're invoked with `--auto` and there's too many packs, then\n-\t * we perform a full repack, as well.\n-\t *\n-\t * - Otherwise we perform an incremental repack.\n-\t */\n-\tswitch (opts->strategy) {\n-\tcase ODB_OPTIMIZE_INCREMENTAL:\n-\t\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n-\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\t\tif (opts->keep_largest_pack != -1) {\n-\t\t\t\tif (opts->keep_largest_pack)\n-\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n-\t\t\t} else if (big_pack_threshold) {\n-\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n-\t\t\t}\n-\n-\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n-\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t} else {\n-\t\t\tif (too_many_packs(files, gc_auto_pack_limit)) {\n-\t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n-\n-\t\t\t\tif (big_pack_threshold) {\n-\t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n-\t\t\t\t\tif (keep_pack.nr >= (unsigned long) gc_auto_pack_limit) {\n-\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n-\t\t\t\t\t}\n-\t\t\t\t} else {\n-\t\t\t\t\tstruct packed_git *p = find_base_packs(files, &keep_pack, 0);\n-\t\t\t\t\tuint64_t mem_have, mem_want;\n-\n-\t\t\t\t\tmem_have = total_ram();\n-\t\t\t\t\tmem_want = estimate_repack_memory(files, p);\n-\n-\t\t\t\t\t/*\n-\t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n-\t\t\t\t\t * the rest for the OS and other processes in the\n-\t\t\t\t\t * system.\n-\t\t\t\t\t */\n-\t\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n-\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t\t}\n-\n-\t\t\t\tadd_repack_all_option(opts, &keep_pack, &repack_cmd.args);\n-\t\t\t\tstring_list_clear(&keep_pack, 0);\n-\t\t\t} else {\n-\t\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n-\t\t\t}\n-\t\t}\n-\n-\t\tbreak;\n-\tcase ODB_OPTIMIZE_GEOMETRIC: {\n-\t\tstruct pack_geometry geometry = {\n-\t\t\t.split_factor = 2,\n-\t\t};\n-\t\tstruct pack_objects_args po_args = {\n-\t\t\t.local = 1,\n-\t\t};\n-\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n-\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n-\n-\t\trepo_config_get_int(odb->repo, \"maintenance.geometric-repack.splitFactor\",\n-\t\t\t\t &geometry.split_factor);\n-\n-\t\texisting_packs.repo = odb->repo;\n-\t\texisting_packs_collect(&existing_packs, &kept_packs);\n-\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n-\t\tpack_geometry_split(&geometry);\n-\n-\t\tif (geometry.split < geometry.pack_nr) {\n-\t\t\tstrvec_pushf(&repack_cmd.args, \"--geometric=%d\",\n-\t\t\t\t geometry.split_factor);\n-\t\t} else {\n-\t\t\tadd_repack_all_option(opts, NULL, &repack_cmd.args);\n-\t\t}\n-\t\tif (odb->repo->settings.core_multi_pack_index)\n-\t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n-\n-\t\texisting_packs_release(&existing_packs);\n-\t\tpack_geometry_release(&geometry);\n-\t\tbreak;\n-\t}\n-\tdefault:\n-\t\tdie(\"unknown maintenance strategy '%d'\", opts->strategy);\n-\t}\n-\n-\tif (run_command(&repack_cmd)) {\n-\t\tret = error(FAILED_RUN, repack_cmd.args.v[0]);\n-\t\tgoto out;\n-\t}\n-\n-\t/* Geometric repacking uses cruft packs, so we don't have to prune separately. */\n-\tif (opts->strategy != ODB_OPTIMIZE_GEOMETRIC && opts->prune_expire) {\n-\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n-\n-\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n-\t\t/* run `git prune` even if using cruft packs */\n-\t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n-\t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n-\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n-\t\tif (repo_has_promisor_remote(odb->repo))\n-\t\t\tstrvec_push(&prune_cmd.args,\n-\t\t\t\t \"--exclude-promisor-objects\");\n-\t\tprune_cmd.git_cmd = 1;\n-\n-\t\tif (run_command(&prune_cmd)) {\n-\t\t\tret = error(FAILED_RUN, prune_cmd.args.v[0]);\n-\t\t\tgoto out;\n-\t\t}\n-\t}\n-\n-\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(files, gc_auto_threshold))\n-\t\twarning(_(\"There are too many unreachable loose objects; \"\n-\t\t\t\"run 'git prune' to remove them.\"));\n-\n-\tret = 0;\n-\n-out:\n-\treturn ret;\n-}\n-\n static int maintenance_task_odb(struct maintenance_run_opts *opts,\n \t\t\t\tstruct gc_config *cfg,\n \t\t\t\tint keep_largest_pack,\ndiff --git a/odb.c b/odb.c\nindex 7d555be09f..89660981fe 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -1003,6 +1003,18 @@ int odb_write_object_stream(struct object_database *odb,\n \treturn odb_source_write_object_stream(odb->sources, stream, len, oid);\n }\n \n+int odb_optimize(struct object_database *odb,\n+\t\t const struct odb_optimize_options *opts)\n+{\n+\treturn odb_source_optimize(odb->sources, opts);\n+}\n+\n+bool odb_optimize_required(struct object_database *odb,\n+\t\t\t const struct odb_optimize_options *opts)\n+{\n+\treturn odb_source_optimize_required(odb->sources, opts);\n+}\n+\n struct object_database *odb_new(struct repository *repo,\n \t\t\t\tconst char *primary_source,\n \t\t\t\tconst char *secondary_sources)\ndiff --git a/odb.h b/odb.h\nindex 3834a0dcbf..7e1c85c22e 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -117,6 +117,51 @@ struct object_database *odb_new(struct repository *repo,\n /* Free the object database and release all resources. */\n void odb_free(struct object_database *o);\n \n+enum odb_optimize_strategy {\n+\tODB_OPTIMIZE_INCREMENTAL,\n+\tODB_OPTIMIZE_GEOMETRIC,\n+};\n+\n+enum odb_optimize_flags {\n+\t/* Enable verbose logging and progress reporting. */\n+\tODB_OPTIMIZE_VERBOSE = (1 << 0),\n+\n+\t/* Perform auto-maintenance, only optimizing objects as required. */\n+\tODB_OPTIMIZE_AUTO = (1 << 1),\n+\n+\t/* Recompute existing deltas. */\n+\tODB_OPTIMIZE_NO_REUSE_DELTAS = (1 << 2),\n+};\n+\n+struct odb_optimize_options {\n+\tenum odb_optimize_strategy strategy;\n+\tenum odb_optimize_flags flags;\n+\tconst char *prune_expire;\n+\tconst char *expire_to;\n+\tint depth;\n+\tint window;\n+\n+\t/* Backend-specific options. */\n+\tint keep_largest_pack;\n+\tint cruft_packs;\n+\tunsigned long max_cruft_size;\n+};\n+\n+/*\n+ * Optimize the object database. Returns 0 on success, a negative error code\n+ * otherwise.\n+ */\n+int odb_optimize(struct object_database *odb,\n+\t\t const struct odb_optimize_options *opts);\n+\n+/*\n+ * Check whether optimization of the object database is required given the\n+ * provided options. Returns true if optimization should be performed, false\n+ * otherwise.\n+ */\n+bool odb_optimize_required(struct object_database *odb,\n+\t\t\t const struct odb_optimize_options *opts);\n+\n /*\n * Close the object database and all of its sources so that any held resources\n * will be released. The database can still be used after closing it, in which\ndiff --git a/odb/source-files.c b/odb/source-files.c\nindex bbd1784b33..82cf61da4a 100644\n--- a/odb/source-files.c\n+++ b/odb/source-files.c\n@@ -1,6 +1,8 @@\n #include \"git-compat-util.h\"\n #include \"abspath.h\"\n+#include \"blob.h\"\n #include \"chdir-notify.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"lockfile.h\"\n #include \"object-file.h\"\n@@ -8,8 +10,16 @@\n #include \"odb/source.h\"\n #include \"odb/source-files.h\"\n #include \"odb/source-loose.h\"\n+#include \"pack-objects.h\"\n #include \"packfile.h\"\n+#include \"path.h\"\n+#include \"promisor-remote.h\"\n+#include \"repack.h\"\n+#include \"run-command.h\"\n #include \"strbuf.h\"\n+#include \"string-list.h\"\n+#include \"strvec.h\"\n+#include \"tree.h\"\n #include \"write-or-die.h\"\n \n static void odb_source_files_reparent(const char *name UNUSED,\n@@ -260,6 +270,464 @@ static int odb_source_files_write_alternate(struct odb_source *source,\n \treturn ret;\n }\n \n+static int too_many_loose_objects(struct odb_source_files *files, int limit)\n+{\n+\tunsigned long loose_count;\n+\n+\tif (limit <= 0)\n+\t\treturn 0;\n+\n+\tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n+\t\t\t\t &loose_count) < 0)\n+\t\treturn 0;\n+\n+\t/*\n+\t * This is weird, but stems from legacy behaviour: the GC auto\n+\t * threshold was always essentially interpreted as if it was rounded up\n+\t * to the next multiple 256 of, so we retain this behaviour for now.\n+\t */\n+\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n+}\n+\n+static struct packed_git *find_base_packs(struct odb_source_files *files,\n+\t\t\t\t\t struct string_list *packs,\n+\t\t\t\t\t unsigned long limit)\n+{\n+\tstruct packfile_list_entry *e;\n+\tstruct packed_git *base = NULL;\n+\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->is_cruft)\n+\t\t\tcontinue;\n+\t\tif (limit) {\n+\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n+\t\t\t\tstring_list_append(packs, e->pack->pack_name);\n+\t\t} else if (!base || base->pack_size < e->pack->pack_size) {\n+\t\t\tbase = e->pack;\n+\t\t}\n+\t}\n+\n+\tif (base)\n+\t\tstring_list_append(packs, base->pack_name);\n+\n+\treturn base;\n+}\n+\n+static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit)\n+{\n+\tstruct packfile_list_entry *e;\n+\tint cnt = 0;\n+\n+\tif (gc_auto_pack_limit <= 0)\n+\t\treturn 0;\n+\n+\tfor (e = packfile_store_get_packs(files->packed); e; e = e->next) {\n+\t\tif (e->pack->pack_keep)\n+\t\t\tcontinue;\n+\t\t/*\n+\t\t * Perhaps check the size of the pack and count only\n+\t\t * very small ones here?\n+\t\t */\n+\t\tcnt++;\n+\t}\n+\treturn gc_auto_pack_limit < cnt;\n+}\n+\n+static uint64_t total_ram(void)\n+{\n+#if defined(HAVE_SYSINFO)\n+\tstruct sysinfo si;\n+\n+\tif (!sysinfo(&si)) {\n+\t\tuint64_t total = si.totalram;\n+\n+\t\tif (si.mem_unit > 1)\n+\t\t\ttotal *= (uint64_t)si.mem_unit;\n+\t\treturn total;\n+\t}\n+#elif defined(HAVE_BSD_SYSCTL) && (defined(HW_MEMSIZE) || defined(HW_PHYSMEM) || defined(HW_PHYSMEM64))\n+\tuint64_t physical_memory;\n+\tint mib[2];\n+\tsize_t length;\n+\n+\tmib[0] = CTL_HW;\n+# if defined(HW_MEMSIZE)\n+\tmib[1] = HW_MEMSIZE;\n+# elif defined(HW_PHYSMEM64)\n+\tmib[1] = HW_PHYSMEM64;\n+# else\n+\tmib[1] = HW_PHYSMEM;\n+# endif\n+\tlength = sizeof(physical_memory);\n+\tif (!sysctl(mib, 2, &physical_memory, &length, NULL, 0)) {\n+\t\tif (length == 4) {\n+\t\t\tuint32_t mem;\n+\n+\t\t\tif (!sysctl(mib, 2, &mem, &length, NULL, 0))\n+\t\t\t\tphysical_memory = mem;\n+\t\t}\n+\t\treturn physical_memory;\n+\t}\n+#elif defined(GIT_WINDOWS_NATIVE)\n+\tMEMORYSTATUSEX memInfo;\n+\n+\tmemInfo.dwLength = sizeof(MEMORYSTATUSEX);\n+\tif (GlobalMemoryStatusEx(&memInfo))\n+\t\treturn memInfo.ullTotalPhys;\n+#endif\n+\treturn 0;\n+}\n+\n+static uint64_t estimate_repack_memory(struct odb_source_files *files,\n+\t\t\t\t struct packed_git *pack)\n+{\n+\tunsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;\n+\tunsigned long delta_base_cache_limit = DEFAULT_DELTA_BASE_CACHE_LIMIT;\n+\tunsigned long nr_objects;\n+\tsize_t os_cache, heap;\n+\n+\tif (odb_source_count_objects(&files->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n+\t\t\t\t &nr_objects) < 0)\n+\t\treturn 0;\n+\n+\tif (!pack || !nr_objects)\n+\t\treturn 0;\n+\n+\trepo_config_get_ulong(files->base.odb->repo, \"pack.deltacachesize\",\n+\t\t\t &max_delta_cache_size);\n+\trepo_config_get_ulong(files->base.odb->repo, \"core.deltabasecachelimit\",\n+\t\t\t &delta_base_cache_limit);\n+\n+\t/*\n+\t * First we have to scan through at least one pack.\n+\t * Assume enough room in OS file cache to keep the entire pack\n+\t * or we may accidentally evict data of other processes from\n+\t * the cache.\n+\t */\n+\tos_cache = pack->pack_size + pack->index_size;\n+\t/* then pack-objects needs lots more for book keeping */\n+\theap = sizeof(struct object_entry) * nr_objects;\n+\t/*\n+\t * internal rev-list --all --objects takes up some memory too,\n+\t * let's say half of it is for blobs\n+\t */\n+\theap += sizeof(struct blob) * nr_objects / 2;\n+\t/*\n+\t * and the other half is for trees (commits and tags are\n+\t * usually insignificant)\n+\t */\n+\theap += sizeof(struct tree) * nr_objects / 2;\n+\t/* and then obj_hash[], underestimated in fact */\n+\theap += sizeof(struct object *) * nr_objects;\n+\t/* revindex is used also */\n+\theap += (sizeof(off_t) + sizeof(uint32_t)) * nr_objects;\n+\t/*\n+\t * read_sha1_file() (either at delta calculation phase, or\n+\t * writing phase) also fills up the delta base cache\n+\t */\n+\theap += delta_base_cache_limit;\n+\t/* and of course pack-objects has its own delta cache */\n+\theap += max_delta_cache_size;\n+\n+\treturn os_cache + heap;\n+}\n+\n+static int keep_one_pack(struct string_list_item *item, void *data)\n+{\n+\tstruct strvec *args = data;\n+\tstrvec_pushf(args, \"--keep-pack=%s\", basename(item->string));\n+\treturn 0;\n+}\n+\n+static void add_repack_all_option(struct repository *repo,\n+\t\t\t\t const struct odb_optimize_options *opts,\n+\t\t\t\t struct string_list *keep_pack,\n+\t\t\t\t struct strvec *args)\n+{\n+\tchar *repack_filter = NULL;\n+\tchar *repack_filter_to = NULL;\n+\n+\trepo_config_get_string(repo, \"gc.repackfilter\", &repack_filter);\n+\trepo_config_get_string(repo, \"gc.repackfilterto\", &repack_filter_to);\n+\n+\tif (opts->prune_expire && !strcmp(opts->prune_expire, \"now\") &&\n+\t !(opts->cruft_packs && opts->expire_to))\n+\t\tstrvec_push(args, \"-a\");\n+\telse if (opts->cruft_packs) {\n+\t\tstrvec_push(args, \"--cruft\");\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--cruft-expiration=%s\", opts->prune_expire);\n+\t\tif (opts->max_cruft_size)\n+\t\t\tstrvec_pushf(args, \"--max-cruft-size=%lu\",\n+\t\t\t\t opts->max_cruft_size);\n+\t\tif (opts->expire_to)\n+\t\t\tstrvec_pushf(args, \"--expire-to=%s\", opts->expire_to);\n+\t} else {\n+\t\tstrvec_push(args, \"-A\");\n+\t\tif (opts->prune_expire)\n+\t\t\tstrvec_pushf(args, \"--unpack-unreachable=%s\", opts->prune_expire);\n+\t}\n+\n+\tif (keep_pack)\n+\t\tfor_each_string_list(keep_pack, keep_one_pack, args);\n+\n+\tif (repack_filter && *repack_filter)\n+\t\tstrvec_pushf(args, \"--filter=%s\", repack_filter);\n+\tif (repack_filter_to && *repack_filter_to)\n+\t\tstrvec_pushf(args, \"--filter-to=%s\", repack_filter_to);\n+\n+\tfree(repack_filter);\n+\tfree(repack_filter_to);\n+}\n+\n+static void add_repack_incremental_option(struct strvec *args)\n+{\n+\tstrvec_push(args, \"--no-write-bitmap-index\");\n+}\n+\n+bool odb_source_files_optimize_required(struct odb_source *source,\n+\t\t\t\t\tconst struct odb_optimize_options *opts)\n+{\n+\tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\tstruct repository *repo = source->odb->repo;\n+\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL: {\n+\t\tint gc_auto_threshold = 6700;\n+\t\tint gc_auto_pack_limit = 50;\n+\n+\t\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n+\t\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\n+\t\t/*\n+\t\t * Setting gc.auto to 0 or negative can disable the\n+\t\t * automatic gc.\n+\t\t */\n+\t\tif (gc_auto_threshold <= 0)\n+\t\t\treturn false;\n+\t\tif (!too_many_packs(files, gc_auto_pack_limit) &&\n+\t\t !too_many_loose_objects(files, gc_auto_threshold))\n+\t\t\treturn false;\n+\n+\t\treturn true;\n+\t}\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\t\tint auto_value = 100;\n+\t\tbool ret;\n+\n+\t\trepo_config_get_int(repo, \"maintenance.geometric-repack.auto\",\n+\t\t\t\t &auto_value);\n+\t\tif (!auto_value)\n+\t\t\treturn false;\n+\t\tif (auto_value < 0)\n+\t\t\treturn true;\n+\n+\t\trepo_config_get_int(repo, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = repo;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\t/*\n+\t\t * When we'd merge at least two packs with one another we always\n+\t\t * perform the repack.\n+\t\t */\n+\t\tif (geometry.split) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\t/*\n+\t\t * Otherwise, we estimate the number of loose objects to determine\n+\t\t * whether we want to create a new packfile or not.\n+\t\t */\n+\t\tif (too_many_loose_objects(files, auto_value)) {\n+\t\t\tret = true;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tret = false;\n+\n+\tout:\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\treturn ret;\n+\t}\n+\tdefault:\n+\t\tBUG(\"unknown maintenance strategy '%d'\", opts->strategy);\n+\t}\n+}\n+\n+int odb_source_files_optimize(struct odb_source *source,\n+\t\t\t const struct odb_optimize_options *opts)\n+{\n+\tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\tstruct repository *repo = source->odb->repo;\n+\tstruct child_process repack_cmd = CHILD_PROCESS_INIT;\n+\tunsigned long big_pack_threshold = 0;\n+\tint gc_auto_threshold = 6700;\n+\tint gc_auto_pack_limit = 50;\n+\tint ret;\n+\n+\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n+\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n+\trepo_config_get_ulong(repo, \"gc.bigpackthreshold\", &big_pack_threshold);\n+\n+\tif (repo->repository_format_precious_objects)\n+\t\treturn 0;\n+\n+\trepack_cmd.git_cmd = 1;\n+\trepack_cmd.odb_to_close = repo->objects;\n+\n+\tstrvec_pushl(&repack_cmd.args, \"repack\", \"-d\", \"-l\", NULL);\n+\tif (opts->flags & ODB_OPTIMIZE_NO_REUSE_DELTAS)\n+\t\tstrvec_push(&repack_cmd.args, \"-f\");\n+\tif (opts->depth > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--depth=%d\", opts->depth);\n+\tif (opts->window > 0)\n+\t\tstrvec_pushf(&repack_cmd.args, \"--window=%d\", opts->window);\n+\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n+\t\tstrvec_push(&repack_cmd.args, \"-q\");\n+\n+\t/*\n+\t * There's three cases we need to consider:\n+\t *\n+\t * - If we're invoked without `--auto` we'll need to perform a full\n+\t * repack.\n+\t *\n+\t * - If we're invoked with `--auto` and there's too many packs, then\n+\t * we perform a full repack, as well.\n+\t *\n+\t * - Otherwise we perform an incremental repack.\n+\t */\n+\tswitch (opts->strategy) {\n+\tcase ODB_OPTIMIZE_INCREMENTAL:\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_AUTO)) {\n+\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\tif (opts->keep_largest_pack != -1) {\n+\t\t\t\tif (opts->keep_largest_pack)\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n+\t\t\t} else if (big_pack_threshold) {\n+\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n+\t\t\t}\n+\n+\t\t\tadd_repack_all_option(repo, opts, &keep_pack, &repack_cmd.args);\n+\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t} else {\n+\t\t\tif (too_many_packs(files, gc_auto_pack_limit)) {\n+\t\t\t\tstruct string_list keep_pack = STRING_LIST_INIT_NODUP;\n+\n+\t\t\t\tif (big_pack_threshold) {\n+\t\t\t\t\tfind_base_packs(files, &keep_pack, big_pack_threshold);\n+\t\t\t\t\tif (keep_pack.nr >= (unsigned long) gc_auto_pack_limit) {\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t\t\tfind_base_packs(files, &keep_pack, 0);\n+\t\t\t\t\t}\n+\t\t\t\t} else {\n+\t\t\t\t\tstruct packed_git *p = find_base_packs(files, &keep_pack, 0);\n+\t\t\t\t\tuint64_t mem_have, mem_want;\n+\n+\t\t\t\t\tmem_have = total_ram();\n+\t\t\t\t\tmem_want = estimate_repack_memory(files, p);\n+\n+\t\t\t\t\t/*\n+\t\t\t\t\t * Only allow 1/2 of memory for pack-objects, leave\n+\t\t\t\t\t * the rest for the OS and other processes in the\n+\t\t\t\t\t * system.\n+\t\t\t\t\t */\n+\t\t\t\t\tif (!mem_have || mem_want < mem_have / 2)\n+\t\t\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t\t}\n+\n+\t\t\t\tadd_repack_all_option(repo, opts, &keep_pack, &repack_cmd.args);\n+\t\t\t\tstring_list_clear(&keep_pack, 0);\n+\t\t\t} else {\n+\t\t\t\tadd_repack_incremental_option(&repack_cmd.args);\n+\t\t\t}\n+\t\t}\n+\n+\t\tbreak;\n+\tcase ODB_OPTIMIZE_GEOMETRIC: {\n+\t\tstruct pack_geometry geometry = {\n+\t\t\t.split_factor = 2,\n+\t\t};\n+\t\tstruct pack_objects_args po_args = {\n+\t\t\t.local = 1,\n+\t\t};\n+\t\tstruct existing_packs existing_packs = EXISTING_PACKS_INIT;\n+\t\tstruct string_list kept_packs = STRING_LIST_INIT_DUP;\n+\n+\t\trepo_config_get_int(repo, \"maintenance.geometric-repack.splitFactor\",\n+\t\t\t\t &geometry.split_factor);\n+\n+\t\texisting_packs.repo = repo;\n+\t\texisting_packs_collect(&existing_packs, &kept_packs);\n+\t\tpack_geometry_init(&geometry, &existing_packs, &po_args);\n+\t\tpack_geometry_split(&geometry);\n+\n+\t\tif (geometry.split < geometry.pack_nr) {\n+\t\t\tstrvec_pushf(&repack_cmd.args, \"--geometric=%d\",\n+\t\t\t\t geometry.split_factor);\n+\t\t} else {\n+\t\t\tadd_repack_all_option(repo, opts, NULL, &repack_cmd.args);\n+\t\t}\n+\t\tif (repo->settings.core_multi_pack_index)\n+\t\t\tstrvec_push(&repack_cmd.args, \"--write-midx\");\n+\n+\t\texisting_packs_release(&existing_packs);\n+\t\tpack_geometry_release(&geometry);\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tdie(\"unknown maintenance strategy '%d'\", opts->strategy);\n+\t}\n+\n+\tif (run_command(&repack_cmd)) {\n+\t\tret = error(\"failed to run %s\", repack_cmd.args.v[0]);\n+\t\tgoto out;\n+\t}\n+\n+\t/* Geometric repacking uses cruft packs, so we don't have to prune separately. */\n+\tif (opts->strategy != ODB_OPTIMIZE_GEOMETRIC && opts->prune_expire) {\n+\t\tstruct child_process prune_cmd = CHILD_PROCESS_INIT;\n+\n+\t\tstrvec_pushl(&prune_cmd.args, \"prune\", \"--expire\", NULL);\n+\t\t/* run `git prune` even if using cruft packs */\n+\t\tstrvec_push(&prune_cmd.args, opts->prune_expire);\n+\t\tif (!(opts->flags & ODB_OPTIMIZE_VERBOSE))\n+\t\t\tstrvec_push(&prune_cmd.args, \"--no-progress\");\n+\t\tif (repo_has_promisor_remote(repo))\n+\t\t\tstrvec_push(&prune_cmd.args,\n+\t\t\t\t \"--exclude-promisor-objects\");\n+\t\tprune_cmd.git_cmd = 1;\n+\n+\t\tif (run_command(&prune_cmd)) {\n+\t\t\tret = error(\"failed to run %s\", prune_cmd.args.v[0]);\n+\t\t\tgoto out;\n+\t\t}\n+\t}\n+\n+\tif (opts->flags & ODB_OPTIMIZE_AUTO && too_many_loose_objects(files, gc_auto_threshold))\n+\t\twarning(_(\"There are too many unreachable loose objects; \"\n+\t\t\t\"run 'git prune' to remove them.\"));\n+\n+\tret = 0;\n+\n+out:\n+\treturn ret;\n+}\n+\n struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \t\t\t\t\t const char *path,\n \t\t\t\t\t bool local)\n@@ -285,6 +753,8 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \tfiles->base.begin_transaction = odb_source_files_begin_transaction;\n \tfiles->base.read_alternates = odb_source_files_read_alternates;\n \tfiles->base.write_alternate = odb_source_files_write_alternate;\n+\tfiles->base.optimize = odb_source_files_optimize;\n+\tfiles->base.optimize_required = odb_source_files_optimize_required;\n \n \t/*\n \t * Ideally, we would only ever store absolute paths in the source. This\ndiff --git a/odb/source-files.h b/odb/source-files.h\nindex d7ac3c1c81..044242bc36 100644\n--- a/odb/source-files.h\n+++ b/odb/source-files.h\n@@ -21,6 +21,21 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \t\t\t\t\t const char *path,\n \t\t\t\t\t bool local);\n \n+/*\n+ * Optimize the files object database source by repacking loose objects and\n+ * packfiles as needed. Returns 0 on success, a negative error code otherwise.\n+ */\n+int odb_source_files_optimize(struct odb_source *source,\n+\t\t\t const struct odb_optimize_options *opts);\n+\n+/*\n+ * Check whether optimization of the files object database source is required\n+ * given the provided options. Returns true if optimization should be\n+ * performed, false otherwise.\n+ */\n+bool odb_source_files_optimize_required(struct odb_source *source,\n+\t\t\t\t\tconst struct odb_optimize_options *opts);\n+\n /*\n * Cast the given object database source to the files backend. This will cause\n * a BUG in case the source doesn't use this backend.\ndiff --git a/odb/source.h b/odb/source.h\nindex 8767708c9c..88a48ba3c3 100644\n--- a/odb/source.h\n+++ b/odb/source.h\n@@ -258,6 +258,21 @@ struct odb_source {\n \t */\n \tint (*write_alternate)(struct odb_source *source,\n \t\t\t const char *alternate);\n+\n+\t/*\n+\t * This callback is expected to optimize the object database source.\n+\t * Returns 0 on success, a negative error code otherwise.\n+\t */\n+\tint (*optimize)(struct odb_source *source,\n+\t\t\tconst struct odb_optimize_options *opts);\n+\n+\t/*\n+\t * This callback is expected to check whether optimization of the\n+\t * object database source is required given the provided options.\n+\t * Returns true if optimization should be performed, false otherwise.\n+\t */\n+\tbool (*optimize_required)(struct odb_source *source,\n+\t\t\t\t const struct odb_optimize_options *opts);\n };\n \n /*\n@@ -475,4 +490,25 @@ static inline int odb_source_begin_transaction(struct odb_source *source,\n \treturn source->begin_transaction(source, out);\n }\n \n+/*\n+ * Optimize the object database source. Returns 0 on success, a negative error\n+ * code otherwise.\n+ */\n+static inline int odb_source_optimize(struct odb_source *source,\n+\t\t\t\t const struct odb_optimize_options *opts)\n+{\n+\treturn source->optimize(source, opts);\n+}\n+\n+/*\n+ * Check whether optimization of the object database source is required given\n+ * the provided options. Returns true if optimization should be performed,\n+ * false otherwise.\n+ */\n+static inline bool odb_source_optimize_required(struct odb_source *source,\n+\t\t\t\t\t\tconst struct odb_optimize_options *opts)\n+{\n+\treturn source->optimize_required(source, opts);\n+}\n+\n #endif\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548018","messageId":"xmqqwluyyhv1.fsf@gitster.g","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-11-9c2c3ee94b38@pks.im","subject":"Re: [PATCH v2 11/12] builtin/gc: fix signedness issues in ODB-related functionality","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-13T16:28:02Z","receivedAt":"2026-07-13T16:28:04Z","isPatch":true,"body":"Patrick Steinhardt writes:\n\n> There are a couple of signedness issues in ODB-related functionality.\n> These are not a problem because we disable -Wsign-compare in this file,\n> but once we move these functions into \"odb/source-files.c\" they will\n> result in warnings.\n>\n> Fix those issues:\n>\n> - In `too_many_loose_objects()` we receive a signed limit, but compare\n> it with the unsigned actual number of loose objects. This is fixed\n> by bailing out immediately when the limit is smaller than or equal\n> to zero, which we also do similarly in other places. The warning is\n> then squelched via a cast.\n>\n> - In `find_base_packs()` we compare the signed size of the pack\n> against the unsigned limit. As the pack size is always going to be a\n> positive file size it's safe to cast it to an unsigned value.\n>\n> - In `odb_optimize()` we compare the unsigned `keep_pack.nr` value\n> against the signed `gc_auto_pack_limit`. We only reach this code\n> when `too_many_packs()` returns true-ish, and that can only happen\n> when `gc_auto_pack_limit > 0`. Consequently, we can fix the warning\n> by casting the limit to an unsigned value.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/gc.c | 20 +++++++++++---------\n> 1 file changed, 11 insertions(+), 9 deletions(-)\n\nYuck. The -Wsign-compare strikes again X-<.\n\n> diff --git a/builtin/gc.c b/builtin/gc.c\n> index 3207182488..8cf3781313 100644\n> --- a/builtin/gc.c\n> +++ b/builtin/gc.c\n> @@ -430,19 +430,21 @@ static int rerere_gc_condition(struct gc_config *cfg UNUSED)\n> \n> static int too_many_loose_objects(struct odb_source_files *files, int limit)\n> {\n> -\t/*\n> -\t * This is weird, but stems from legacy behaviour: the GC auto\n> -\t * threshold was always essentially interpreted as if it was rounded up\n> -\t * to the next multiple 256 of, so we retain this behaviour for now.\n> -\t */\n> -\tint auto_threshold = DIV_ROUND_UP(limit, 256) * 256;\n> \tunsigned long loose_count;\n> \n> +\tif (limit <= 0)\n> +\t\treturn 0;\n> +\n> \tif (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,\n> \t\t\t\t &loose_count) < 0)\n> \t\treturn 0;\n> \n> -\treturn loose_count > auto_threshold;\n> +\t/*\n> +\t * This is weird, but stems from legacy behaviour: the GC auto\n> +\t * threshold was always essentially interpreted as if it was rounded up\n> +\t * to the next multiple 256 of, so we retain this behaviour for now.\n> +\t */\n> +\treturn loose_count > (DIV_ROUND_UP(((unsigned long) limit), 256) * 256);\n> }\n\nOK. It is trivially correct (if rounding up is correct, that is).\n\n> @@ -456,7 +458,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,\n> \t\tif (e->pack->is_cruft)\n> \t\t\tcontinue;\n> \t\tif (limit) {\n> -\t\t\tif (e->pack->pack_size >= limit)\n> +\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n\nHere, just like in too_many_loose_objects(), 'limit' is of type\n'unsigned long'. While it makes sense to convert both sides of\nthe comparison to an unsigned type, casting only the left side\nto a type that differs from the right side puzzles me.\n\nPresumably, the other side is of type 'off_t', which is signed,\nexplaining the desire to cast it to an unsigned type. But I am\nnot sure what happens if 'off_t' is wider than 'unsigned long'.\n\nOtherwise looking good.\n"},{"id":"548022","messageId":"xmqqh5m2yhkm.fsf@gitster.g","threadId":"65942","inReplyTo":"20260713-b4-pks-odb-optimize-v2-12-9c2c3ee94b38@pks.im","subject":"Re: [PATCH v2 12/12] odb: make optimizations pluggable","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-13T16:34:17Z","receivedAt":"2026-07-13T16:34:19Z","isPatch":true,"body":"Patrick Steinhardt writes:\n\n> Move `odb_optimize()` and `odb_optimize_required()` from \"builtin/gc.c\"\n> into the \"files\" source and wire them up via newly introduced vtable\n> pointers for the object database sources. This makes the logic pluggable\n> and thus allows other backends to have their own, custom implementation.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/gc.c | 490 +----------------------------------------------------\n> odb.c | 12 ++\n> odb.h | 45 +++++\n> odb/source-files.c | 470 ++++++++++++++++++++++++++++++++++++++++++++++++++\n\nReviewing this commit with \"git show --color-moved\" is an excellent\nway to verify that this is 90% pure code motion. Only minimal\nadjustments were required, such as replacing a direct reference to a\nstruct member in the original with a variable that was previously\nassigned the equivalent value.\n\nFor example, this original\n\n> -\tswitch (opts->strategy) {\n> -\tcase ODB_OPTIMIZE_INCREMENTAL: {\n> -\t\tint gc_auto_threshold = 6700;\n> -\t\tint gc_auto_pack_limit = 50;\n> -\n> -\t\trepo_config_get_int(odb->repo, \"gc.auto\", &gc_auto_threshold);\n> -\t\trepo_config_get_int(odb->repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n> -\n> -\t\t/*\n> -\t\t * Setting gc.auto to 0 or negative can disable the\n> -\t\t * automatic gc.\n> -\t\t */\n\nturns into\n\n> +\tswitch (opts->strategy) {\n> +\tcase ODB_OPTIMIZE_INCREMENTAL: {\n> +\t\tint gc_auto_threshold = 6700;\n> +\t\tint gc_auto_pack_limit = 50;\n> +\n> +\t\trepo_config_get_int(repo, \"gc.auto\", &gc_auto_threshold);\n> +\t\trepo_config_get_int(repo, \"gc.autopacklimit\", &gc_auto_pack_limit);\n> +\n> +\t\t/*\n> +\t\t * Setting gc.auto to 0 or negative can disable the\n> +\t\t * automatic gc.\n> +\t\t */\n\nAnd the change from odb->repo vs repo is such an adjustment.\n\nLooking good.\n"},{"id":"548772","messageId":"amCmwKjbq2aNt8mZ@pks.im","threadId":"65942","inReplyTo":"xmqqwluyyhv1.fsf@gitster.g","subject":"Re: [PATCH v2 11/12] builtin/gc: fix signedness issues in ODB-related functionality","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-22T11:17:20Z","receivedAt":"2026-07-22T11:17:30Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 09:28:02AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> > diff --git a/builtin/gc.c b/builtin/gc.c\n> > index 3207182488..8cf3781313 100644\n> > --- a/builtin/gc.c\n> > @@ -456,7 +458,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,\n> > \t\tif (e->pack->is_cruft)\n> > \t\t\tcontinue;\n> > \t\tif (limit) {\n> > -\t\t\tif (e->pack->pack_size >= limit)\n> > +\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n> \n> Here, just like in too_many_loose_objects(), 'limit' is of type\n> 'unsigned long'. While it makes sense to convert both sides of\n> the comparison to an unsigned type, casting only the left side\n> to a type that differs from the right side puzzles me.\n> \n> Presumably, the other side is of type 'off_t', which is signed,\n> explaining the desire to cast it to an unsigned type. But I am\n> not sure what happens if 'off_t' is wider than 'unsigned long'.\n\nYeah, `pack_size` is an `off_t`, which is signed. But we never populate\nit with a negative value, so casting it to `uintmax_t` in unnecessary.\nThe right-hand side is already unsigned, so due to the usual arithmetic\nconversion rules it would be automatically promoted to `uintmax_t`, as\nwell.\n\nPatrick\n"},{"id":"549024","messageId":"xmqqwluhvhtu.fsf@gitster.g","threadId":"65942","inReplyTo":"amCmwKjbq2aNt8mZ@pks.im","subject":"Re: [PATCH v2 11/12] builtin/gc: fix signedness issues in ODB-related functionality","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-26T16:25:33Z","receivedAt":"2026-07-26T16:25:36Z","isPatch":true,"body":"Patrick Steinhardt writes:\n\n> On Mon, Jul 13, 2026 at 09:28:02AM -0700, Junio C Hamano wrote:\n>> Patrick Steinhardt writes:\n>> > diff --git a/builtin/gc.c b/builtin/gc.c\n>> > index 3207182488..8cf3781313 100644\n>> > --- a/builtin/gc.c\n>> > @@ -456,7 +458,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,\n>> > \t\tif (e->pack->is_cruft)\n>> > \t\t\tcontinue;\n>> > \t\tif (limit) {\n>> > -\t\t\tif (e->pack->pack_size >= limit)\n>> > +\t\t\tif ((uintmax_t) e->pack->pack_size >= limit)\n>> \n>> Here, just like in too_many_loose_objects(), 'limit' is of type\n>> 'unsigned long'. While it makes sense to convert both sides of\n>> the comparison to an unsigned type, casting only the left side\n>> to a type that differs from the right side puzzles me.\n>> \n>> Presumably, the other side is of type 'off_t', which is signed,\n>> explaining the desire to cast it to an unsigned type. But I am\n>> not sure what happens if 'off_t' is wider than 'unsigned long'.\n>\n> Yeah, `pack_size` is an `off_t`, which is signed. But we never populate\n> it with a negative value, so casting it to `uintmax_t` in unnecessary.\n> The right-hand side is already unsigned, so due to the usual arithmetic\n> conversion rules it would be automatically promoted to `uintmax_t`, as\n> well.\n\nSo we are in agreement that this hunk, unlike others, have little to\ndo with \"fix signedness issues\"?\n\nI personally feel that the true fix in signedness issues is not to\nuse -Wsign-compare or teach compilers to be intelligent about when\nto issue the warning when given that flag, but that is a separate\ntopic.\n\nThanks.\n"}]}