git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 4/4] reftable/stack: avoid reloading the stack when already locked

From
Karthik Nayak <karthik.188@gmail.com>
Date
Aug 24, 2026, 09:31 UTC
Message-ID
<20260824-740-optimize-reloading-the-reftable-stack-v2-4-9c9de2eb0af7@gmail.com>
In-Reply-To
<20260824-740-optimize-reloading-the-reftable-stack-v2-0-9c9de2eb0af7@gmail.com>

When making modifications to the reftable stack, the stack obtains a lock to the list file and removes the lock after the commit phase. Since most operations reload the stack to ensure we have the latest state, any branched operation during the locked phase could trigger a state reload.

To prevent data loss due to concurrent writes, state reload is necessary right after obtaining the lock. But any reloads after that are just a no-op. Now that the struct has access to the lock file status, simply skip reloading if the lock is present.

Benchmarking with a fixed, non-symbolic target OID in the 'refs/tags/' namespace (since it triggers a stack reload when checking if reflog exists for the given tag name), shows a consistent 15-20% improvement with these patches:

  refcount   master     patch     speedup
  --------   -------    -------   -------
  2,000       18.5 ms    16.6 ms   1.11x
  20,000     120.7 ms   102.8 ms   1.17x
  50,000     296.5 ms   247.1 ms   1.20x

We can also see the improvements in the number of syscall counts. On master, the number of calls to `newfstatat()` grows linearly with the number of refs created. With this patch, the number is now a constant:

  refcount   master   patch
  --------   ------   ------
  1,000      1,059       55
  5,000      5,059       55
  10,000     10,059      55
  20,000     20,059      55
Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
---
 reftable/stack.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/reftable/stack.c b/reftable/stack.c
index c3d4deff29..47a60db079 100644
--- a/reftable/stack.c
+++ b/reftable/stack.c
@@ -553,14 +553,21 @@ int reftable_new_stack(struct reftable_stack **dest, const char *dir,
 
 /*
  * Check whether the given stack is up-to-date with what we have in memory.
+ * If skip_if_locked is set skip stack reloading if the stack is currently
+ * locked. Stack reloading must _not_ be skipped right after obtaining the
+ * lock, to check for concurrent updates which may have happened.
+ *
  * Returns 0 if so, 1 if the stack is out-of-date or a negative error code
  * otherwise.
  */
-static int stack_uptodate(struct reftable_stack *st)
+static int stack_uptodate(struct reftable_stack *st, int skip_if_locked)
 {
 	char **names = NULL;
 	int err;
 
+	if (skip_if_locked && st->list_lock.fd != -1)
+		return 0;
+
 	/*
 	 * When we have cached stat information available then we use it to
 	 * verify whether the file has been rewritten.
@@ -623,7 +630,7 @@ static int stack_uptodate(struct reftable_stack *st)
 
 int reftable_stack_reload(struct reftable_stack *st)
 {
-	int err = stack_uptodate(st);
+	int err = stack_uptodate(st, 1);
 	if (err > 0)
 		return reftable_stack_reload_maybe_reuse(st, 1);
 	return err;
@@ -693,7 +700,7 @@ static int reftable_stack_init_addition(struct reftable_addition *add,
 		}
 	}
 
-	err = stack_uptodate(st);
+	err = stack_uptodate(st, 0);
 	if (err < 0)
 		goto done;
 	if (err > 0) {
@@ -1200,7 +1207,7 @@ static int stack_compact_range(struct reftable_stack *st,
 	 * we could check that relevant tables still exist. But for now it's
 	 * good enough to just abort.
 	 */
-	err = stack_uptodate(st);
+	err = stack_uptodate(st, 0);
 	if (err < 0)
 		goto done;
 	if (err > 0) {
@@ -1319,7 +1326,7 @@ static int stack_compact_range(struct reftable_stack *st,
 	 * tables with our compacted version. If they don't, then we need to
 	 * abort.
 	 */
-	err = stack_uptodate(st);
+	err = stack_uptodate(st, 0);
 	if (err < 0)
 		goto done;
 	if (err > 0) {
-- 
2.55.GIT
Previous: Karthik NayakNext: Jeff King
Message 7 of 9 in “reftable/stack: avoid reloading the stack when locked”
  1. 0/4 reftable/stack: avoid reloading the stack when lockedKarthik Nayak, Aug 24, 2026
  2. 1/4 reftable/stack: remove `REFTABLE_STACK_NEW_ADDITION_RELOAD`Karthik Nayak, Aug 24, 2026
  3. 2/4 reftable/stack: rename reftable_stack_new_addition()Karthik Nayak, Aug 24, 2026
  4. 3/4 reftable/stack: move list lock to `struct reftable_stack`Karthik Nayak, Aug 24, 2026
  5. Patrick SteinhardtAug 25, 2026
  6. Karthik NayakAug 26, 2026
  7. 4/4 reftable/stack: avoid reloading the stack when already lockedKarthik Nayak, Aug 24, 2026
  8. Jeff KingAug 24, 2026
  9. Karthik NayakAug 26, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.