[PATCH 09/15] refs/reftable: expose auto compaction via new flag

From: Patrick Steinhardt <ps@pks.im>
Date: Mar 18, 2024, 10:53 UTC
Message-ID: <61ebcb2d52e829cf8d8a5b24f16a79b8ff4d0f3b.1710706118.git.ps@pks.im>
In-Reply-To: <cover.1710706118.git.ps@pks.im>

Under normal circumstances, the "reftable" backend will automatically perform compaction after appending to the stack. It is thus not necessary and may even be considered wasteful to run git-pack-refs(1) in "reftable"-backed repositories as it will cause the backend to compact all tables into a single one. We do exactly that though when running `git maintenance run --auto` or `git gc --auto`, which gets spawned by Git after running some specific commands.

The `--auto` mode is typically only executing optimizations as needed. To do so, we already use several heuristics for the various different data structures in Git to determine whether to optimize them or not. We do not use any heuristics for refs though and instead always optimize them.

Introduce a new `PACK_REFS_AUTO` flag that can be passed to the backend. When not handled by the backend we will continue to behave the exact same as we do right now, that is we optimize refs unconditionally. This is done for the "files" backend for now to retain current behaviour, even though we may eventually also want to introduce heuristics here. For the "reftable" backend though we already do have auto-compaction, so we can easily reuse that logic to implement the new auto-packing flag.

Note that under normal circumstances, this should always end up being a no-op. After all, we already invoke the code for every single addition to the stack. But there are special cases where it can still be helpful to execute the auto-compaction code explicitly:

  - Concurrent writers may cause compaction to not run due to locks.

  - Callers may decide to disable compaction altogether and then pack
    refs at a later point due to various reasons.

  - Other implementations of the reftable format may do compaction
    differently or even not at all.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 refs.h                  | 6 +++++-
 refs/reftable-backend.c | 5 ++++-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/refs.h b/refs.h
index 8c8994cb29..d278775e08 100644
--- a/refs.h
+++ b/refs.h
@@ -422,8 +422,12 @@ void warn_dangling_symrefs(FILE *fp, const char *msg_fmt,
 /*
  * Flags for controlling behaviour of pack_refs()
  * PACK_REFS_PRUNE: Prune loose refs after packing
+ * PACK_REFS_AUTO: Pack refs on a best effort basis. The heuristics and end
+ *                 result are decided by the ref backend. Backends may ignore
+ *                 this flag and fall back to a normal repack.
  */
-#define PACK_REFS_PRUNE 0x0001
+#define PACK_REFS_PRUNE (1 << 0)
+#define PACK_REFS_AUTO  (1 << 1)
 
 struct pack_refs_opts {
 	unsigned int flags;
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index 66cdbbdb24..135bd4e268 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -1220,7 +1220,10 @@ static int reftable_be_pack_refs(struct ref_store *ref_store,
 	if (!stack)
 		stack = refs->main_stack;
 
-	ret = reftable_stack_compact_all(stack, NULL);
+	if (opts->flags & PACK_REFS_AUTO)
+		ret = reftable_stack_auto_compact(stack);
+	else
+		ret = reftable_stack_compact_all(stack, NULL);
 	if (ret < 0) {
 		ret = error(_("unable to compact stack: %s"),
 			    reftable_error_str(ret));

-- 
2.44.0