git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 13/13] refs/reftable: precompute prefix length

From
Patrick Steinhardt <ps@pks.im>
Date
Feb 27, 2024, 15:07 UTC
Message-ID
<627bd1f5f784a2a34d7dabf95c3adf279c3a81a9.1709045927.git.ps@pks.im>
In-Reply-To
<cover.1709045927.git.ps@pks.im>

We're recomputing the prefix length on every iteration of the ref iterator. Precompute it for another speedup when iterating over 1 million refs:

    Benchmark 1: show-ref: single matching ref (revision = HEAD~)
      Time (mean ± σ):     100.3 ms ±   3.7 ms    [User: 97.3 ms, System: 2.8 ms]
      Range (min … max):    97.5 ms … 139.7 ms    1000 runs
    Benchmark 2: show-ref: single matching ref (revision = HEAD)
      Time (mean ± σ):      95.8 ms ±   3.4 ms    [User: 92.9 ms, System: 2.8 ms]
      Range (min … max):    93.0 ms … 121.9 ms    1000 runs
    Summary
      show-ref: single matching ref (revision = HEAD) ran
        1.05 ± 0.05 times faster than show-ref: single matching ref (revision = HEAD~)
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 refs/reftable-backend.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index a14f2ad7f4..4d27fdde54 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -346,6 +346,7 @@ struct reftable_ref_iterator {
 	struct object_id oid;
 
 	const char *prefix;
+	size_t prefix_len;
 	unsigned int flags;
 	int err;
 };
@@ -371,8 +372,8 @@ static int reftable_ref_iterator_advance(struct ref_iterator *ref_iterator)
 		if (!starts_with(iter->ref.refname, "refs/"))
 			continue;
 
-		if (iter->prefix &&
-		    strncmp(iter->prefix, iter->ref.refname, strlen(iter->prefix))) {
+		if (iter->prefix_len &&
+		    strncmp(iter->prefix, iter->ref.refname, iter->prefix_len)) {
 			iter->err = 1;
 			break;
 		}
@@ -481,6 +482,7 @@ static struct reftable_ref_iterator *ref_iterator_for_stack(struct reftable_ref_
 	iter = xcalloc(1, sizeof(*iter));
 	base_ref_iterator_init(&iter->base, &reftable_ref_iterator_vtable, 1);
 	iter->prefix = prefix;
+	iter->prefix_len = prefix ? strlen(prefix) : 0;
 	iter->base.oid = &iter->oid;
 	iter->flags = flags;
 	iter->refs = refs;
-- 
2.44.0
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 37 of 51 in “reftable: improve ref iteration performance (pt.2)”
  1. 00/12 reftable: improve ref iteration performance (pt.2)Patrick Steinhardt, Feb 14, 2024
  2. 01/12 reftable/pq: use `size_t` to track iterator indexPatrick Steinhardt, Feb 14, 2024
  3. 02/12 reftable/merged: make `merged_iter` structure privatePatrick Steinhardt, Feb 14, 2024
  4. Justin ToblerFeb 20, 2024
  5. Patrick SteinhardtFeb 27, 2024
  6. 03/12 reftable/merged: advance subiter on subsequent iterationPatrick Steinhardt, Feb 14, 2024
  7. Justin ToblerFeb 20, 2024
  8. Patrick SteinhardtFeb 27, 2024
  9. 04/12 reftable/merged: make subiters own their recordsPatrick Steinhardt, Feb 14, 2024
  10. 05/12 reftable/merged: remove unnecessary null check for subitersPatrick Steinhardt, Feb 14, 2024
  11. 06/12 reftable/merged: handle subiter cleanup on close onlyPatrick Steinhardt, Feb 14, 2024
  12. 07/12 reftable/merged: circumvent pqueue with single subiterPatrick Steinhardt, Feb 14, 2024
  13. 08/12 reftable/merged: avoid duplicate pqueue emptiness checkPatrick Steinhardt, Feb 14, 2024
  14. James LiuFeb 27, 2024
  15. 09/12 reftable/record: reuse refname when decodingPatrick Steinhardt, Feb 14, 2024
  16. James LiuFeb 28, 2024
  17. Patrick SteinhardtMar 4, 2024
  18. 10/12 reftable/record: reuse refname when copyingPatrick Steinhardt, Feb 14, 2024
  19. James LiuFeb 28, 2024
  20. 11/12 reftable/record: decode keys in placePatrick Steinhardt, Feb 14, 2024
  21. James LiuFeb 28, 2024
  22. Patrick SteinhardtMar 4, 2024
  23. 12/12 reftable: allow inlining of a few functionsPatrick Steinhardt, Feb 14, 2024
  24. 00/13 reftable: improve ref iteration performance (pt.2)Patrick Steinhardt, Feb 27, 2024
  25. 01/13 reftable/pq: use `size_t` to track iterator indexPatrick Steinhardt, Feb 27, 2024
  26. 02/13 reftable/merged: make `merged_iter` structure privatePatrick Steinhardt, Feb 27, 2024
  27. 03/13 reftable/merged: advance subiter on subsequent iterationPatrick Steinhardt, Feb 27, 2024
  28. 04/13 reftable/merged: make subiters own their recordsPatrick Steinhardt, Feb 27, 2024
  29. 05/13 reftable/merged: remove unnecessary null check for subitersPatrick Steinhardt, Feb 27, 2024
  30. 06/13 reftable/merged: handle subiter cleanup on close onlyPatrick Steinhardt, Feb 27, 2024
  31. 07/13 reftable/merged: circumvent pqueue with single subiterPatrick Steinhardt, Feb 27, 2024
  32. 08/13 reftable/merged: avoid duplicate pqueue emptiness checkPatrick Steinhardt, Feb 27, 2024
  33. 09/13 reftable/record: reuse refname when decodingPatrick Steinhardt, Feb 27, 2024
  34. 10/13 reftable/record: reuse refname when copyingPatrick Steinhardt, Feb 27, 2024
  35. 11/13 reftable/record: decode keys in placePatrick Steinhardt, Feb 27, 2024
  36. 12/13 reftable: allow inlining of a few functionsPatrick Steinhardt, Feb 27, 2024
  37. 13/13 refs/reftable: precompute prefix lengthPatrick Steinhardt, Feb 27, 2024
  38. 00/13 reftable: improve ref iteration performance (pt.2)Patrick Steinhardt, Mar 4, 2024
  39. 01/13 reftable/pq: use `size_t` to track iterator indexPatrick Steinhardt, Mar 4, 2024
  40. 02/13 reftable/merged: make `merged_iter` structure privatePatrick Steinhardt, Mar 4, 2024
  41. 03/13 reftable/merged: advance subiter on subsequent iterationPatrick Steinhardt, Mar 4, 2024
  42. 04/13 reftable/merged: make subiters own their recordsPatrick Steinhardt, Mar 4, 2024
  43. 05/13 reftable/merged: remove unnecessary null check for subitersPatrick Steinhardt, Mar 4, 2024
  44. 06/13 reftable/merged: handle subiter cleanup on close onlyPatrick Steinhardt, Mar 4, 2024
  45. 07/13 reftable/merged: circumvent pqueue with single subiterPatrick Steinhardt, Mar 4, 2024
  46. 08/13 reftable/merged: avoid duplicate pqueue emptiness checkPatrick Steinhardt, Mar 4, 2024
  47. 09/13 reftable/record: reuse refname when decodingPatrick Steinhardt, Mar 4, 2024
  48. 10/13 reftable/record: reuse refname when copyingPatrick Steinhardt, Mar 4, 2024
  49. 11/13 reftable/record: decode keys in placePatrick Steinhardt, Mar 4, 2024
  50. 12/13 reftable: allow inlining of a few functionsPatrick Steinhardt, Mar 4, 2024
  51. 13/13 refs/reftable: precompute prefix lengthPatrick Steinhardt, Mar 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.