git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 7/7] reftable/block: avoid decoding keys when searching restart points

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 25, 2024, 10:11 UTC
Message-ID
<e751b3c536ace78f975b7d2553c22dbf6845a8d4.1711361340.git.ps@pks.im>
In-Reply-To
<cover.1711361340.git.ps@pks.im>

When searching over restart points in a block we decode the key of each of the records, which results in a memory allocation. This is quite pointless though given that records it restart points will never use prefix compression and thus store their keys verbatim in the block.

Refactor the code so that we can avoid decoding the keys, which saves us some allocations.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/block.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)
diff --git a/reftable/block.c b/reftable/block.c
index ca80a05e21..8bb4e43cec 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -287,23 +287,32 @@ static int restart_needle_less(size_t idx, void *_args)
 		.buf = args->reader->block.data + off,
 		.len = args->reader->block_len - off,
 	};
-	struct strbuf kth_restart_key = STRBUF_INIT;
-	uint8_t unused_extra;
-	int result, n;
+	uint64_t prefix_len, suffix_len;
+	uint8_t extra;
+	int n;
 
 	/*
-	 * TODO: The restart key is verbatim in the block, so we can in theory
-	 * avoid decoding the key and thus save some allocations.
+	 * Records at restart points are stored without prefix compression, so
+	 * there is no need to fully decode the record key here. This removes
+	 * the need for allocating memory.
 	 */
-	n = reftable_decode_key(&kth_restart_key, &unused_extra, in);
-	if (n < 0) {
+	n = reftable_decode_keylen(in, &prefix_len, &suffix_len, &extra);
+	if (n < 0 || prefix_len) {
 		args->error = 1;
 		return -1;
 	}
 
-	result = strbuf_cmp(&args->needle, &kth_restart_key);
-	strbuf_release(&kth_restart_key);
-	return result < 0;
+	string_view_consume(&in, n);
+	if (suffix_len > in.len) {
+		args->error = 1;
+		return -1;
+	}
+
+	n = memcmp(args->needle.buf, in.buf,
+		   args->needle.len < suffix_len ? args->needle.len : suffix_len);
+	if (n)
+		return n < 0;
+	return args->needle.len < suffix_len;
 }
 
 void block_iter_copy_from(struct block_iter *dest, struct block_iter *src)
-- 
2.44.GIT
Previous: Patrick SteinhardtNext: Justin Tobler
Message 26 of 45 in “reftable: improvements for the `binsearch()` mechanism”
  1. 0/7 reftable: improvements for the `binsearch()` mechanismPatrick Steinhardt, Mar 22, 2024
  2. 1/7 reftable/basics: fix return type of `binsearch()` to be `size_t`Patrick Steinhardt, Mar 22, 2024
  3. 2/7 reftable/basics: improve `binsearch()` testPatrick Steinhardt, Mar 22, 2024
  4. Justin ToblerMar 22, 2024
  5. Patrick SteinhardtMar 25, 2024
  6. 3/7 reftable/refname: refactor binary search over refnamesPatrick Steinhardt, Mar 22, 2024
  7. Justin ToblerMar 22, 2024
  8. Patrick SteinhardtMar 25, 2024
  9. 4/7 reftable/block: refactor binary search over restart pointsPatrick Steinhardt, Mar 22, 2024
  10. 5/7 reftable/block: fix error handling when searching restart pointsPatrick Steinhardt, Mar 22, 2024
  11. 6/7 reftable/record: extract function to decode key lengthsPatrick Steinhardt, Mar 22, 2024
  12. 7/7 reftable/block: avoid decoding keys when searching restart pointsPatrick Steinhardt, Mar 22, 2024
  13. 0/7 reftable: improvements for the `binsearch()` mechanismPatrick Steinhardt, Mar 25, 2024
  14. 1/7 reftable/basics: fix return type of `binsearch()` to be `size_t`Patrick Steinhardt, Mar 25, 2024
  15. 2/7 reftable/basics: improve `binsearch()` testPatrick Steinhardt, Mar 25, 2024
  16. 3/7 reftable/refname: refactor binary search over refnamesPatrick Steinhardt, Mar 25, 2024
  17. Justin ToblerApr 2, 2024
  18. Patrick SteinhardtApr 2, 2024
  19. 4/7 reftable/block: refactor binary search over restart pointsPatrick Steinhardt, Mar 25, 2024
  20. Justin ToblerApr 2, 2024
  21. Patrick SteinhardtApr 2, 2024
  22. Justin ToblerApr 2, 2024
  23. Patrick SteinhardtApr 3, 2024
  24. 5/7 reftable/block: fix error handling when searching restart pointsPatrick Steinhardt, Mar 25, 2024
  25. 6/7 reftable/record: extract function to decode key lengthsPatrick Steinhardt, Mar 25, 2024
  26. 7/7 reftable/block: avoid decoding keys when searching restart pointsPatrick Steinhardt, Mar 25, 2024
  27. Justin ToblerApr 2, 2024
  28. Patrick SteinhardtApr 2, 2024
  29. 0/7 reftable: improvements for the `binsearch()` mechanismPatrick Steinhardt, Apr 2, 2024
  30. 1/7 reftable/basics: fix return type of `binsearch()` to be `size_t`Patrick Steinhardt, Apr 2, 2024
  31. 2/7 reftable/basics: improve `binsearch()` testPatrick Steinhardt, Apr 2, 2024
  32. 3/7 reftable/refname: refactor binary search over refnamesPatrick Steinhardt, Apr 2, 2024
  33. 4/7 reftable/block: refactor binary search over restart pointsPatrick Steinhardt, Apr 2, 2024
  34. 5/7 reftable/block: fix error handling when searching restart pointsPatrick Steinhardt, Apr 2, 2024
  35. 6/7 reftable/record: extract function to decode key lengthsPatrick Steinhardt, Apr 2, 2024
  36. 7/7 reftable/block: avoid decoding keys when searching restart pointsPatrick Steinhardt, Apr 2, 2024
  37. Justin ToblerApr 2, 2024
  38. 0/7 reftable: improvements for the `binsearch()` mechanismPatrick Steinhardt, Apr 3, 2024
  39. 1/7 reftable/basics: fix return type of `binsearch()` to be `size_t`Patrick Steinhardt, Apr 3, 2024
  40. 2/7 reftable/basics: improve `binsearch()` testPatrick Steinhardt, Apr 3, 2024
  41. 3/7 reftable/refname: refactor binary search over refnamesPatrick Steinhardt, Apr 3, 2024
  42. 4/7 reftable/block: refactor binary search over restart pointsPatrick Steinhardt, Apr 3, 2024
  43. 5/7 reftable/block: fix error handling when searching restart pointsPatrick Steinhardt, Apr 3, 2024
  44. 6/7 reftable/record: extract function to decode key lengthsPatrick Steinhardt, Apr 3, 2024
  45. 7/7 reftable/block: avoid decoding keys when searching restart pointsPatrick Steinhardt, Apr 3, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.