git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 15/15] builtin/gc: pack refs when using `git maintenance run --auto`

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 18, 2024, 10:53 UTC
Message-ID
<71f4800d36bcc77d8f36c5fc7b48eccbb90e6a93.1710706119.git.ps@pks.im>
In-Reply-To
<cover.1710706118.git.ps@pks.im>

When running `git maintenance run --auto`, then the various subtasks will only run as needed. Thus, we for example end up only packing loose objects if we hit a certain threshold.

Interestingly enough, the "pack-refs" task is actually _never_ executed when the auto-flag is set because it does not have a condition at all. As 41abfe15d9 (maintenance: add pack-refs task, 2021-02-09) mentions:

    The 'auto_condition' function pointer is left NULL for now. We could
    extend this in the future to have a condition check if pack-refs
    should be run during 'git maintenance run --auto'.

It is not quite clear from that quote whether it is actually intended that the task doesn't run at all in this mode. Also, no test was added to verify this behaviour. Ultimately though, it feels quite surprising that `git maintenance run --auto --task=pack-refs` would quietly never do anything at all.

In any case, now that we do have the logic in place to let ref backends decide whether or not to repack refs, it does make sense to wire it up accordingly. With the "reftable" backend we will thus now perform auto-compaction, which optimizes the refdb as needed.

But for the "files" backend we now unconditionally pack refs as it does not yet know to handle the "auto" flag. Arguably, this can be seen as a bug fix given that previously the task never did anything at all. Eventually though we should amend the "files" backend to use some heuristics for auto compaction, as well.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/gc.c                  | 12 +++++++++++-
 t/t0601-reffiles-pack-refs.sh | 10 ++++++++++
 t/t0610-reftable-basics.sh    |  2 +-
 3 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/builtin/gc.c b/builtin/gc.c
index bf1f2a621a..3c874b248b 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -206,6 +206,16 @@ struct maintenance_run_opts {
 	enum schedule_priority schedule;
 };
 
+static int pack_refs_condition(void)
+{
+	/*
+	 * The auto-repacking logic for refs is handled by the ref backends and
+	 * exposed via `git pack-refs --auto`. We thus always return truish
+	 * here and let the backend decide for us.
+	 */
+	return 1;
+}
+
 static int maintenance_task_pack_refs(MAYBE_UNUSED struct maintenance_run_opts *opts)
 {
 	struct child_process cmd = CHILD_PROCESS_INIT;
@@ -1298,7 +1308,7 @@ static struct maintenance_task tasks[] = {
 	[TASK_PACK_REFS] = {
 		"pack-refs",
 		maintenance_task_pack_refs,
-		NULL,
+		pack_refs_condition,
 	},
 };
 
diff --git a/t/t0601-reffiles-pack-refs.sh b/t/t0601-reffiles-pack-refs.sh
index 219a495451..7d4ab0b91a 100755
--- a/t/t0601-reffiles-pack-refs.sh
+++ b/t/t0601-reffiles-pack-refs.sh
@@ -370,4 +370,14 @@ test_expect_success 'pack-refs does not drop broken refs during deletion' '
 	test_cmp expect actual
 '
 
+test_expect_success 'maintenance --auto unconditionally packs loose refs' '
+	git update-ref refs/heads/something HEAD &&
+	test_path_is_file .git/refs/heads/something &&
+	git rev-parse refs/heads/something >expect &&
+	git maintenance run --task=pack-refs --auto &&
+	test_path_is_missing .git/refs/heads/something &&
+	git rev-parse refs/heads/something >actual &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index b37d8bf3b1..931d888bbb 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -387,7 +387,7 @@ test_expect_success 'pack-refs: compaction raises locking errors' '
 	test_cmp expect err
 '
 
-for command in pack-refs gc
+for command in pack-refs gc "maintenance run --task=pack-refs"
 do
 test_expect_success "$command: auto compaction" '
 	test_when_finished "rm -rf repo" &&
-- 
2.44.0
Previous: Patrick SteinhardtNext: Karthik Nayak
Message 26 of 49 in “refs: introduce `--auto` to pack refs as needed”
  1. 00/15 refs: introduce `--auto` to pack refs as neededPatrick Steinhardt, Mar 18, 2024
  2. 01/15 reftable/stack: fix error handling in `reftable_stack_init_addition()`Patrick Steinhardt, Mar 18, 2024
  3. Karthik NayakMar 20, 2024
  4. Patrick SteinhardtMar 25, 2024
  5. 02/15 reftable/error: discern locked/outdated errorsPatrick Steinhardt, Mar 18, 2024
  6. 03/15 reftable/stack: use error codes when locking fails during compactionPatrick Steinhardt, Mar 18, 2024
  7. Karthik NayakMar 20, 2024
  8. Patrick SteinhardtMar 25, 2024
  9. 04/15 reftable/stack: gracefully handle failed auto-compaction due to locksPatrick Steinhardt, Mar 18, 2024
  10. Karthik NayakMar 20, 2024
  11. Patrick SteinhardtMar 25, 2024
  12. 05/15 refs/reftable: print errors on compaction failurePatrick Steinhardt, Mar 18, 2024
  13. 06/15 t/helper: drop pack-refs wrapperPatrick Steinhardt, Mar 18, 2024
  14. 07/15 refs: move `struct pack_refs_opts` to where it's usedPatrick Steinhardt, Mar 18, 2024
  15. 08/15 refs: remove `PACK_REFS_ALL` flagPatrick Steinhardt, Mar 18, 2024
  16. 09/15 refs/reftable: expose auto compaction via new flagPatrick Steinhardt, Mar 18, 2024
  17. 10/15 builtin/pack-refs: release allocated memoryPatrick Steinhardt, Mar 18, 2024
  18. Karthik NayakMar 20, 2024
  19. Patrick SteinhardtMar 25, 2024
  20. 11/15 builtin/pack-refs: introduce new "--auto" flagPatrick Steinhardt, Mar 18, 2024
  21. 12/15 builtin/gc: move `struct maintenance_run_opts`Patrick Steinhardt, Mar 18, 2024
  22. 13/15 t6500: extract objects with "17" prefixPatrick Steinhardt, Mar 18, 2024
  23. 14/15 builtin/gc: forward git-gc(1)'s `--auto` flag when packing refsPatrick Steinhardt, Mar 18, 2024
  24. Karthik NayakMar 20, 2024
  25. Patrick SteinhardtMar 25, 2024
  26. 15/15 builtin/gc: pack refs when using `git maintenance run --auto`Patrick Steinhardt, Mar 18, 2024
  27. Karthik NayakMar 20, 2024
  28. Patrick SteinhardtMar 25, 2024
  29. Karthik NayakMar 20, 2024
  30. Patrick SteinhardtMar 25, 2024
  31. 00/15 refs: introduce `--auto` to pack refs as neededPatrick Steinhardt, Mar 25, 2024
  32. 01/15 reftable/stack: fix error handling in `reftable_stack_init_addition()`Patrick Steinhardt, Mar 25, 2024
  33. 02/15 reftable/error: discern locked/outdated errorsPatrick Steinhardt, Mar 25, 2024
  34. 03/15 reftable/stack: use error codes when locking fails during compactionPatrick Steinhardt, Mar 25, 2024
  35. 04/15 reftable/stack: gracefully handle failed auto-compaction due to locksPatrick Steinhardt, Mar 25, 2024
  36. Patrick SteinhardtMar 25, 2024
  37. 05/15 refs/reftable: print errors on compaction failurePatrick Steinhardt, Mar 25, 2024
  38. 06/15 t/helper: drop pack-refs wrapperPatrick Steinhardt, Mar 25, 2024
  39. 07/15 refs: move `struct pack_refs_opts` to where it's usedPatrick Steinhardt, Mar 25, 2024
  40. 08/15 refs: remove `PACK_REFS_ALL` flagPatrick Steinhardt, Mar 25, 2024
  41. 09/15 refs/reftable: expose auto compaction via new flagPatrick Steinhardt, Mar 25, 2024
  42. 10/15 builtin/pack-refs: release allocated memoryPatrick Steinhardt, Mar 25, 2024
  43. 11/15 builtin/pack-refs: introduce new "--auto" flagPatrick Steinhardt, Mar 25, 2024
  44. 12/15 builtin/gc: move `struct maintenance_run_opts`Patrick Steinhardt, Mar 25, 2024
  45. 13/15 t6500: extract objects with "17" prefixPatrick Steinhardt, Mar 25, 2024
  46. 14/15 builtin/gc: forward git-gc(1)'s `--auto` flag when packing refsPatrick Steinhardt, Mar 25, 2024
  47. 15/15 builtin/gc: pack refs when using `git maintenance run --auto`Patrick Steinhardt, Mar 25, 2024
  48. Karthik NayakMar 25, 2024
  49. Junio C HamanoMar 25, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.