git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 04/15] reftable/stack: gracefully handle failed auto-compaction due to locks

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 18, 2024, 10:52 UTC
Message-ID
<37a18b91ca998ca5ad27d17e86d286040b6e6ee1.1710706118.git.ps@pks.im>
In-Reply-To
<cover.1710706118.git.ps@pks.im>

Whenever we commit a new table to the reftable stack we will end up invoking auto-compaction of the stack to keep the total number of tables at bay. This auto-compaction may fail though in case at least one of the tables which we are about to compact is locked. This is indicated by the compaction function returning a positive value. We do not handle this case though, and thus bubble that return value up the calling chain, which will ultimately cause a failure.

Fix this bug by handling positive return values.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/stack.c           | 13 ++++++++++++-
 t/t0610-reftable-basics.sh | 20 ++++++++++++++++++++
 2 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/reftable/stack.c b/reftable/stack.c
index ba15c48ddd..dd5160d751 100644
--- a/reftable/stack.c
+++ b/reftable/stack.c
@@ -680,8 +680,19 @@ int reftable_addition_commit(struct reftable_addition *add)
 	if (err)
 		goto done;
 
-	if (!add->stack->disable_auto_compact)
+	if (!add->stack->disable_auto_compact) {
+		/*
+		 * Auto-compact the stack to keep the number of tables in
+		 * control. Note that we explicitly ignore locking issues which
+		 * may indicate that a concurrent process is already trying to
+		 * compact tables. This is fine, so we simply ignore that error
+		 * condition.
+		 */
 		err = reftable_stack_auto_compact(add->stack);
+		if (err < 0 && err != REFTABLE_LOCK_ERROR)
+			goto done;
+		err = 0;
+	}
 
 done:
 	reftable_addition_close(add);
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index 686781192e..5f2f9baa9b 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -340,6 +340,26 @@ test_expect_success 'ref transaction: empty transaction in empty repo' '
 	EOF
 '
 
+test_expect_success 'ref transaction: fails gracefully when auto compaction fails' '
+	test_when_finished "rm -rf repo" &&
+	git init repo &&
+	(
+		cd repo &&
+
+		test_commit A &&
+		for i in $(test_seq 10)
+		do
+			git branch branch-$i &&
+			for table in .git/reftable/*.ref
+			do
+				touch "$table.lock" || exit 1
+			done ||
+			exit 1
+		done &&
+		test_line_count = 13 .git/reftable/tables.list
+	)
+'
+
 test_expect_success 'pack-refs: compacts tables' '
 	test_when_finished "rm -rf repo" &&
 	git init repo &&
-- 
2.44.0
Previous: Patrick SteinhardtNext: Karthik Nayak
Message 9 of 49 in “refs: introduce `--auto` to pack refs as needed”
  1. 00/15 refs: introduce `--auto` to pack refs as neededPatrick Steinhardt, Mar 18, 2024
  2. 01/15 reftable/stack: fix error handling in `reftable_stack_init_addition()`Patrick Steinhardt, Mar 18, 2024
  3. Karthik NayakMar 20, 2024
  4. Patrick SteinhardtMar 25, 2024
  5. 02/15 reftable/error: discern locked/outdated errorsPatrick Steinhardt, Mar 18, 2024
  6. 03/15 reftable/stack: use error codes when locking fails during compactionPatrick Steinhardt, Mar 18, 2024
  7. Karthik NayakMar 20, 2024
  8. Patrick SteinhardtMar 25, 2024
  9. 04/15 reftable/stack: gracefully handle failed auto-compaction due to locksPatrick Steinhardt, Mar 18, 2024
  10. Karthik NayakMar 20, 2024
  11. Patrick SteinhardtMar 25, 2024
  12. 05/15 refs/reftable: print errors on compaction failurePatrick Steinhardt, Mar 18, 2024
  13. 06/15 t/helper: drop pack-refs wrapperPatrick Steinhardt, Mar 18, 2024
  14. 07/15 refs: move `struct pack_refs_opts` to where it's usedPatrick Steinhardt, Mar 18, 2024
  15. 08/15 refs: remove `PACK_REFS_ALL` flagPatrick Steinhardt, Mar 18, 2024
  16. 09/15 refs/reftable: expose auto compaction via new flagPatrick Steinhardt, Mar 18, 2024
  17. 10/15 builtin/pack-refs: release allocated memoryPatrick Steinhardt, Mar 18, 2024
  18. Karthik NayakMar 20, 2024
  19. Patrick SteinhardtMar 25, 2024
  20. 11/15 builtin/pack-refs: introduce new "--auto" flagPatrick Steinhardt, Mar 18, 2024
  21. 12/15 builtin/gc: move `struct maintenance_run_opts`Patrick Steinhardt, Mar 18, 2024
  22. 13/15 t6500: extract objects with "17" prefixPatrick Steinhardt, Mar 18, 2024
  23. 14/15 builtin/gc: forward git-gc(1)'s `--auto` flag when packing refsPatrick Steinhardt, Mar 18, 2024
  24. Karthik NayakMar 20, 2024
  25. Patrick SteinhardtMar 25, 2024
  26. 15/15 builtin/gc: pack refs when using `git maintenance run --auto`Patrick Steinhardt, Mar 18, 2024
  27. Karthik NayakMar 20, 2024
  28. Patrick SteinhardtMar 25, 2024
  29. Karthik NayakMar 20, 2024
  30. Patrick SteinhardtMar 25, 2024
  31. 00/15 refs: introduce `--auto` to pack refs as neededPatrick Steinhardt, Mar 25, 2024
  32. 01/15 reftable/stack: fix error handling in `reftable_stack_init_addition()`Patrick Steinhardt, Mar 25, 2024
  33. 02/15 reftable/error: discern locked/outdated errorsPatrick Steinhardt, Mar 25, 2024
  34. 03/15 reftable/stack: use error codes when locking fails during compactionPatrick Steinhardt, Mar 25, 2024
  35. 04/15 reftable/stack: gracefully handle failed auto-compaction due to locksPatrick Steinhardt, Mar 25, 2024
  36. Patrick SteinhardtMar 25, 2024
  37. 05/15 refs/reftable: print errors on compaction failurePatrick Steinhardt, Mar 25, 2024
  38. 06/15 t/helper: drop pack-refs wrapperPatrick Steinhardt, Mar 25, 2024
  39. 07/15 refs: move `struct pack_refs_opts` to where it's usedPatrick Steinhardt, Mar 25, 2024
  40. 08/15 refs: remove `PACK_REFS_ALL` flagPatrick Steinhardt, Mar 25, 2024
  41. 09/15 refs/reftable: expose auto compaction via new flagPatrick Steinhardt, Mar 25, 2024
  42. 10/15 builtin/pack-refs: release allocated memoryPatrick Steinhardt, Mar 25, 2024
  43. 11/15 builtin/pack-refs: introduce new "--auto" flagPatrick Steinhardt, Mar 25, 2024
  44. 12/15 builtin/gc: move `struct maintenance_run_opts`Patrick Steinhardt, Mar 25, 2024
  45. 13/15 t6500: extract objects with "17" prefixPatrick Steinhardt, Mar 25, 2024
  46. 14/15 builtin/gc: forward git-gc(1)'s `--auto` flag when packing refsPatrick Steinhardt, Mar 25, 2024
  47. 15/15 builtin/gc: pack refs when using `git maintenance run --auto`Patrick Steinhardt, Mar 25, 2024
  48. Karthik NayakMar 25, 2024
  49. Junio C HamanoMar 25, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.