git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 4/7] reftable/record: reuse refnames when decoding log records

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 5, 2024, 12:11 UTC
Message-ID
<f2b59e58a4e54e2a3d1df2c73c4f7acd92a57604.1709640322.git.ps@pks.im>
In-Reply-To
<cover.1709640322.git.ps@pks.im>

When decoding a log record we always reallocate their refname arrays. This results in quite a lot of needless allocation churn.

Refactor the code to grow the array as required only. Like this, we should usually only end up reallocating the array a small handful of times when iterating over many refs. Before:

    HEAP SUMMARY:
        in use at exit: 13,473 bytes in 122 blocks
      total heap usage: 4,068,487 allocs, 4,068,365 frees, 332,011,793 bytes allocated
After:
    HEAP SUMMARY:
        in use at exit: 13,473 bytes in 122 blocks
      total heap usage: 3,068,488 allocs, 3,068,366 frees, 307,122,961 bytes allocated
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/record.c          | 2 +-
 reftable/reftable-record.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/reftable/record.c b/reftable/record.c
index d816de6d93..82780b2d06 100644
--- a/reftable/record.c
+++ b/reftable/record.c
@@ -861,7 +861,7 @@ static int reftable_log_record_decode(void *rec, struct strbuf key,
 	if (key.len <= 9 || key.buf[key.len - 9] != 0)
 		return REFTABLE_FORMAT_ERROR;
 
-	r->refname = reftable_realloc(r->refname, key.len - 8);
+	REFTABLE_ALLOC_GROW(r->refname, key.len - 8, r->refname_cap);
 	memcpy(r->refname, key.buf, key.len - 8);
 	ts = get_be64(key.buf + key.len - 8);
 
diff --git a/reftable/reftable-record.h b/reftable/reftable-record.h
index 2659ea008c..d28f38175c 100644
--- a/reftable/reftable-record.h
+++ b/reftable/reftable-record.h
@@ -74,6 +74,7 @@ int reftable_ref_record_equal(const struct reftable_ref_record *a,
 /* reftable_log_record holds a reflog entry */
 struct reftable_log_record {
 	char *refname;
+	size_t refname_cap;
 	uint64_t update_index; /* logical timestamp of a transactional update.
 				*/
 
-- 
2.44.0
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 12 of 20 in “reftable: memory optimizations for reflog iteration”
  1. 0/7 reftable: memory optimizations for reflog iterationPatrick Steinhardt, Mar 5, 2024
  2. 1/7 refs/reftable: reload correct stack when creating reflog iterPatrick Steinhardt, Mar 5, 2024
  3. Karthik NayakMar 6, 2024
  4. Junio C HamanoMar 6, 2024
  5. Patrick SteinhardtMar 7, 2024
  6. Josh SteadmonMar 11, 2024
  7. Patrick SteinhardtMar 11, 2024
  8. 2/7 reftable/record: convert old and new object IDs to arraysPatrick Steinhardt, Mar 5, 2024
  9. 3/7 reftable/record: avoid copying author infoPatrick Steinhardt, Mar 5, 2024
  10. James LiuMar 13, 2024
  11. Patrick SteinhardtMar 21, 2024
  12. 4/7 reftable/record: reuse refnames when decoding log recordsPatrick Steinhardt, Mar 5, 2024
  13. 5/7 reftable/record: reuse message when decoding log recordsPatrick Steinhardt, Mar 5, 2024
  14. 6/7 reftable/record: use scratch buffer when decoding recordsPatrick Steinhardt, Mar 5, 2024
  15. Josh SteadmonMar 11, 2024
  16. Patrick SteinhardtMar 11, 2024
  17. Josh SteadmonMar 11, 2024
  18. 7/7 refs/reftable: track last log record name via strbufPatrick Steinhardt, Mar 5, 2024
  19. Josh SteadmonMar 11, 2024
  20. Patrick SteinhardtMar 11, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.