git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 11/11] reftable/block: reuse buffer to compute record keys

From
Patrick Steinhardt <ps@pks.im>
Date
Dec 8, 2023, 14:53 UTC
Message-ID
<02b11f3a80608ba8748a0d0e2294f432e02464e5.1702047081.git.ps@pks.im>
In-Reply-To
<cover.1702047081.git.ps@pks.im>

When iterating over entries in the block iterator we compute the key of each of the entries and write it into a buffer. We do not reuse the buffer though and thus re-allocate it on every iteration, which is wasteful.

Refactor the code to reuse the buffer.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/block.c | 19 ++++++++-----------
 reftable/block.h |  2 ++
 2 files changed, 10 insertions(+), 11 deletions(-)
diff --git a/reftable/block.c b/reftable/block.c
index 8c6a8c77fc..1df3d8a0f0 100644
--- a/reftable/block.c
+++ b/reftable/block.c
@@ -323,30 +323,28 @@ int block_iter_next(struct block_iter *it, struct reftable_record *rec)
 		.len = it->br->block_len - it->next_off,
 	};
 	struct string_view start = in;
-	struct strbuf key = STRBUF_INIT;
 	uint8_t extra = 0;
 	int n = 0;
 
 	if (it->next_off >= it->br->block_len)
 		return 1;
 
-	n = reftable_decode_key(&key, &extra, it->last_key, in);
+	n = reftable_decode_key(&it->key, &extra, it->last_key, in);
 	if (n < 0)
 		return -1;
 
-	if (!key.len)
+	if (!it->key.len)
 		return REFTABLE_FORMAT_ERROR;
 
 	string_view_consume(&in, n);
-	n = reftable_record_decode(rec, key, extra, in, it->br->hash_size);
+	n = reftable_record_decode(rec, it->key, extra, in, it->br->hash_size);
 	if (n < 0)
 		return -1;
 	string_view_consume(&in, n);
 
 	strbuf_reset(&it->last_key);
-	strbuf_addbuf(&it->last_key, &key);
+	strbuf_addbuf(&it->last_key, &it->key);
 	it->next_off += start.len - in.len;
-	strbuf_release(&key);
 	return 0;
 }
 
@@ -377,6 +375,7 @@ int block_iter_seek(struct block_iter *it, struct strbuf *want)
 void block_iter_close(struct block_iter *it)
 {
 	strbuf_release(&it->last_key);
+	strbuf_release(&it->key);
 }
 
 int block_reader_seek(struct block_reader *br, struct block_iter *it,
@@ -387,7 +386,6 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,
 		.r = br,
 	};
 	struct reftable_record rec = reftable_new_record(block_reader_type(br));
-	struct strbuf key = STRBUF_INIT;
 	int err = 0;
 	struct block_iter next = BLOCK_ITER_INIT;
 
@@ -414,8 +412,8 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,
 		if (err < 0)
 			goto done;
 
-		reftable_record_key(&rec, &key);
-		if (err > 0 || strbuf_cmp(&key, want) >= 0) {
+		reftable_record_key(&rec, &it->key);
+		if (err > 0 || strbuf_cmp(&it->key, want) >= 0) {
 			err = 0;
 			goto done;
 		}
@@ -424,8 +422,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,
 	}
 
 done:
-	strbuf_release(&key);
-	strbuf_release(&next.last_key);
+	block_iter_close(&next);
 	reftable_record_release(&rec);
 
 	return err;
diff --git a/reftable/block.h b/reftable/block.h
index 51699af233..17481e6331 100644
--- a/reftable/block.h
+++ b/reftable/block.h
@@ -84,10 +84,12 @@ struct block_iter {
 
 	/* key for last entry we read. */
 	struct strbuf last_key;
+	struct strbuf key;
 };
 
 #define BLOCK_ITER_INIT { \
 	.last_key = STRBUF_INIT, \
+	.key = STRBUF_INIT, \
 }
 
 /* initializes a block reader. */
-- 
2.43.0
Previous: Patrick SteinhardtNext: Han-Wen Nienhuys
Message 38 of 58 in “reftable: small set of fixes”
  1. 0/8 reftable: small set of fixesPatrick Steinhardt, Nov 21, 2023
  2. 1/8 reftable: wrap EXPECT macros in do/whilePatrick Steinhardt, Nov 21, 2023
  3. 2/8 reftable: handle interrupted readsPatrick Steinhardt, Nov 21, 2023
  4. 3/8 reftable: handle interrupted writesPatrick Steinhardt, Nov 21, 2023
  5. 4/8 reftable/stack: verify that `reftable_stack_add()` uses auto-compactionPatrick Steinhardt, Nov 21, 2023
  6. 5/8 reftable/stack: perform auto-compaction with transactional interfacePatrick Steinhardt, Nov 21, 2023
  7. Han-Wen NienhuysDec 21, 2023
  8. Patrick SteinhardtDec 21, 2023
  9. 6/8 reftable/stack: reuse buffers when reloading stackPatrick Steinhardt, Nov 21, 2023
  10. 7/8 reftable/merged: reuse buffer to compute record keysPatrick Steinhardt, Nov 21, 2023
  11. Han-Wen NienhuysDec 21, 2023
  12. 8/8 reftable/stack: fix stale lock when dyingPatrick Steinhardt, Nov 21, 2023
  13. 00/11 reftable: small set of fixesPatrick Steinhardt, Dec 8, 2023
  14. 01/11 reftable: wrap EXPECT macros in do/whilePatrick Steinhardt, Dec 8, 2023
  15. 02/11 reftable: handle interrupted readsPatrick Steinhardt, Dec 8, 2023
  16. Taylor BlauDec 8, 2023
  17. Patrick SteinhardtDec 11, 2023
  18. 03/11 reftable: handle interrupted writesPatrick Steinhardt, Dec 8, 2023
  19. 04/11 reftable/stack: verify that `reftable_stack_add()` uses auto-compactionPatrick Steinhardt, Dec 8, 2023
  20. Taylor BlauDec 8, 2023
  21. Eric SunshineDec 8, 2023
  22. Patrick SteinhardtDec 11, 2023
  23. Eric SunshineDec 11, 2023
  24. 05/11 reftable/stack: perform auto-compaction with transactional interfacePatrick Steinhardt, Dec 8, 2023
  25. Taylor BlauDec 8, 2023
  26. Patrick SteinhardtDec 11, 2023
  27. 06/11 reftable/stack: reuse buffers when reloading stackPatrick Steinhardt, Dec 8, 2023
  28. Taylor BlauDec 8, 2023
  29. Patrick SteinhardtDec 11, 2023
  30. Han-Wen NienhuysDec 21, 2023
  31. 07/11 reftable/stack: fix stale lock when dyingPatrick Steinhardt, Dec 8, 2023
  32. Taylor BlauDec 8, 2023
  33. Patrick SteinhardtDec 11, 2023
  34. 08/11 reftable/stack: fix use of unseeded randomnessPatrick Steinhardt, Dec 8, 2023
  35. Han-Wen NienhuysDec 21, 2023
  36. 09/11 reftable/merged: reuse buffer to compute record keysPatrick Steinhardt, Dec 8, 2023
  37. 10/11 reftable/block: introduce macro to initialize `struct block_iter`Patrick Steinhardt, Dec 8, 2023
  38. 11/11 reftable/block: reuse buffer to compute record keysPatrick Steinhardt, Dec 8, 2023
  39. Han-Wen NienhuysDec 21, 2023
  40. Patrick SteinhardtDec 28, 2023
  41. Taylor BlauDec 8, 2023
  42. 00/11 reftable: small set of fixesPatrick Steinhardt, Dec 11, 2023
  43. 01/11 reftable: wrap EXPECT macros in do/whilePatrick Steinhardt, Dec 11, 2023
  44. 02/11 reftable: handle interrupted readsPatrick Steinhardt, Dec 11, 2023
  45. 03/11 reftable: handle interrupted writesPatrick Steinhardt, Dec 11, 2023
  46. 04/11 reftable/stack: verify that `reftable_stack_add()` uses auto-compactionPatrick Steinhardt, Dec 11, 2023
  47. Taylor BlauDec 11, 2023
  48. Patrick SteinhardtDec 12, 2023
  49. 05/11 reftable/stack: perform auto-compaction with transactional interfacePatrick Steinhardt, Dec 11, 2023
  50. 06/11 reftable/stack: reuse buffers when reloading stackPatrick Steinhardt, Dec 11, 2023
  51. 07/11 reftable/stack: fix stale lock when dyingPatrick Steinhardt, Dec 11, 2023
  52. 08/11 reftable/stack: fix use of unseeded randomnessPatrick Steinhardt, Dec 11, 2023
  53. 09/11 reftable/merged: reuse buffer to compute record keysPatrick Steinhardt, Dec 11, 2023
  54. 10/11 reftable/block: introduce macro to initialize `struct block_iter`Patrick Steinhardt, Dec 11, 2023
  55. 11/11 reftable/block: reuse buffer to compute record keysPatrick Steinhardt, Dec 11, 2023
  56. Taylor BlauDec 11, 2023
  57. Patrick SteinhardtDec 12, 2023
  58. Han-Wen NienhuysDec 21, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.