git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/4] reftable/stack: don't perform auto-compaction with less than two tables

From
Patrick Steinhardt <ps@pks.im>
Date
Dec 22, 2024, 07:24 UTC
Message-ID
<20241222-b4-pks-reftable-oom-fix-without-readers-v2-1-19550090d15a@pks.im>
In-Reply-To
<20241222-b4-pks-reftable-oom-fix-without-readers-v2-0-19550090d15a@pks.im>

In order to compact tables we need at least two tables. Bail out early from `reftable_stack_auto_compact()` in case we have less than two tables.

In the original, `stack_table_sizes_for_compaction()` yields an array that has the same length as the number of tables. This array is then passed on to `suggest_compaction_segment()`, which returns an empty segment in case we have less than two tables. The segment is then passed to `segment_size()`, which will return `0` because both start and end of the segment are `0`. And because we only call `stack_compact_range()` in case we have a positive segment size we don't perform auto-compaction at all. Consequently, this change does not result in a user-visible change in behaviour when called with a single table.

But when called with no tables this protects us against a potential out-of-memory error: `stack_table_sizes_for_compaction()` would try to allocate a zero-byte object when there aren't any tables, and that may lead to a `NULL` pointer on some platforms like NonStop which causes us to bail out with an out-of-memory error.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/stack.c | 3 +++
 1 file changed, 3 insertions(+)
diff --git a/reftable/stack.c b/reftable/stack.c
index 59fd695a12c2033ed589a21ef1c9155eeecc4641..6ca21965d8e1135d986043113d465abd14cd532c 100644
--- a/reftable/stack.c
+++ b/reftable/stack.c
@@ -1627,6 +1627,9 @@ int reftable_stack_auto_compact(struct reftable_stack *st)
 	struct segment seg;
 	uint64_t *sizes;
 
+	if (st->merged->readers_len < 2)
+		return 0;
+
 	sizes = stack_table_sizes_for_compaction(st);
 	if (!sizes)
 		return REFTABLE_OUT_OF_MEMORY_ERROR;
-- 
2.48.0.rc0.184.g0fc57dec57.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 26 of 30 in “reftable: fix out-of-memory errors on NonStop”
  1. 0/4 reftable: fix out-of-memory errors on NonStopPatrick Steinhardt, Dec 21, 2024
  2. 1/4 reftable/stack: don't perform auto-compaction with less than two tablesPatrick Steinhardt, Dec 21, 2024
  3. Junio C HamanoDec 21, 2024
  4. Junio C HamanoDec 21, 2024
  5. Patrick SteinhardtDec 22, 2024
  6. 2/4 reftable/merged: fix zero-sized allocation when there are no readersPatrick Steinhardt, Dec 21, 2024
  7. Kristoffer HaugsbakkDec 21, 2024
  8. Patrick SteinhardtDec 21, 2024
  9. Junio C HamanoDec 21, 2024
  10. 3/4 reftable/stack: fix zero-sized allocation when there are no readersPatrick Steinhardt, Dec 21, 2024
  11. Junio C HamanoDec 21, 2024
  12. Junio C HamanoDec 21, 2024
  13. rsbecker@nexbridge.comDec 21, 2024
  14. Junio C HamanoDec 21, 2024
  15. rsbecker@nexbridge.comDec 22, 2024
  16. rsbecker@nexbridge.comDec 22, 2024
  17. Patrick SteinhardtDec 22, 2024
  18. Junio C HamanoDec 23, 2024
  19. Patrick SteinhardtDec 22, 2024
  20. 4/4 reftable/basics: return NULL on zero-sized allocationsPatrick Steinhardt, Dec 21, 2024
  21. Kristoffer HaugsbakkDec 21, 2024
  22. Randall BeckerDec 21, 2024
  23. Junio C HamanoDec 21, 2024
  24. Junio C HamanoDec 21, 2024
  25. 0/4 reftable: fix out-of-memory errors on NonStopPatrick Steinhardt, Dec 22, 2024
  26. 1/4 reftable/stack: don't perform auto-compaction with less than two tablesPatrick Steinhardt, Dec 22, 2024
  27. 2/4 reftable/merged: fix zero-sized allocation when there are no readersPatrick Steinhardt, Dec 22, 2024
  28. 3/4 reftable/stack: fix zero-sized allocation when there are no readersPatrick Steinhardt, Dec 22, 2024
  29. 4/4 reftable/basics: return NULL on zero-sized allocationsPatrick Steinhardt, Dec 22, 2024
  30. Junio C HamanoDec 22, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.