git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 08/13] strmap: enable faster clearing and reusing of strmaps

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Nov 2, 2020, 18:55 UTC
Message-ID
<ea942eb8035686af4b284e66affa3a0a1d2a5b0c.1604343314.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.835.v3.git.git.1604343313.gitgitgadget@gmail.com>
From: Elijah Newren <newren@gmail.com>

When strmaps are used heavily, such as is done by my new merge-ort algorithm, and strmaps need to be cleared but then re-used (because of e.g. picking multiple commits to cherry-pick, or due to a recursive merge having several different merges while recursing), free-ing and reallocating map->table repeatedly can add up in time, especially since it will likely be reallocated to a much smaller size but the previous merge provides a good guide to the right size to use for the next merge.

Introduce strmap_partial_clear() to take advantage of this type of situation; it will act similar to strmap_clear() except that map->table's entries are zeroed instead of map->table being free'd. Making use of this function reduced the cost of clear_or_reinit_internal_opts() by about 20% in mert-ort, and dropped the overall runtime of my rebase testcase by just under 2%.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 strmap.c | 6 ++++++
 strmap.h | 6 ++++++
 2 files changed, 12 insertions(+)
diff --git a/strmap.c b/strmap.c
index 829f1bc095..c410c5241a 100644
--- a/strmap.c
+++ b/strmap.c
@@ -64,6 +64,12 @@ void strmap_clear(struct strmap *map, int free_values)
 	hashmap_clear(&map->map);
 }
 
+void strmap_partial_clear(struct strmap *map, int free_values)
+{
+	strmap_free_entries_(map, free_values);
+	hashmap_partial_clear(&map->map);
+}
+
 void *strmap_put(struct strmap *map, const char *str, void *data)
 {
 	struct strmap_entry *entry = find_strmap_entry(map, str);
diff --git a/strmap.h b/strmap.h
index ee4307cca5..10b4642860 100644
--- a/strmap.h
+++ b/strmap.h
@@ -42,6 +42,12 @@ void strmap_init_with_options(struct strmap *map,
  */
 void strmap_clear(struct strmap *map, int free_values);
 
+/*
+ * Similar to strmap_clear() but leaves map->map->table allocated and
+ * pre-sized so that subsequent uses won't need as many rehashings.
+ */
+void strmap_partial_clear(struct strmap *map, int free_values);
+
 /*
  * Insert "str" into the map, pointing to "data".
  *
-- 
gitgitgadget
Previous: Elijah NewrenNext: Elijah Newren via GitGitGadget
Message 79 of 144 in “Add struct strmap and associated utility functions”
  1. 0/5 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Aug 21, 2020
  2. 3/5 strmap: add more utility functionsElijah Newren via GitGitGadget, Aug 21, 2020
  3. Jeff KingAug 21, 2020
  4. 1/5 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Aug 21, 2020
  5. Jeff KingAug 21, 2020
  6. 5/5 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Aug 21, 2020
  7. Jeff KingAug 21, 2020
  8. Elijah NewrenAug 21, 2020
  9. Jeff KingAug 21, 2020
  10. 2/5 strmap: new utility functionsElijah Newren via GitGitGadget, Aug 21, 2020
  11. Jeff KingAug 21, 2020
  12. 4/5 strmap: add strdup_strings optionElijah Newren via GitGitGadget, Aug 21, 2020
  13. Jeff KingAug 21, 2020
  14. Elijah NewrenAug 21, 2020
  15. Jeff KingAug 21, 2020
  16. Elijah NewrenAug 21, 2020
  17. Jeff KingAug 28, 2020
  18. Elijah NewrenAug 28, 2020
  19. Jeff KingAug 21, 2020
  20. Elijah NewrenAug 21, 2020
  21. Elijah NewrenAug 21, 2020
  22. Jeff KingAug 28, 2020
  23. Elijah NewrenAug 28, 2020
  24. Jeff KingSep 1, 2020
  25. 00/10 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Oct 13, 2020
  26. 01/10 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Oct 13, 2020
  27. Jeff KingOct 30, 2020
  28. Elijah NewrenOct 30, 2020
  29. Jeff KingNov 3, 2020
  30. Elijah NewrenNov 3, 2020
  31. 05/10 strmap: new utility functionsElijah Newren via GitGitGadget, Oct 13, 2020
  32. Jeff KingOct 30, 2020
  33. Elijah NewrenOct 30, 2020
  34. 04/10 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Oct 13, 2020
  35. Jeff KingOct 30, 2020
  36. Elijah NewrenOct 30, 2020
  37. Jeff KingNov 3, 2020
  38. 03/10 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Oct 13, 2020
  39. Jeff KingOct 30, 2020
  40. Elijah NewrenOct 30, 2020
  41. Jeff KingNov 3, 2020
  42. Elijah NewrenNov 3, 2020
  43. 02/10 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Oct 13, 2020
  44. Jeff KingOct 30, 2020
  45. 07/10 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Oct 13, 2020
  46. Jeff KingOct 30, 2020
  47. 10/10 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Oct 13, 2020
  48. Jeff KingOct 30, 2020
  49. Elijah NewrenOct 30, 2020
  50. Jeff KingNov 3, 2020
  51. 06/10 strmap: add more utility functionsElijah Newren via GitGitGadget, Oct 13, 2020
  52. Jeff KingOct 30, 2020
  53. Elijah NewrenOct 30, 2020
  54. Jeff KingNov 3, 2020
  55. 08/10 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Oct 13, 2020
  56. Jeff KingOct 30, 2020
  57. Elijah NewrenOct 30, 2020
  58. Jeff KingNov 3, 2020
  59. Elijah NewrenNov 3, 2020
  60. 09/10 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Oct 13, 2020
  61. Jeff KingOct 30, 2020
  62. Elijah NewrenOct 30, 2020
  63. 00/13 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 2, 2020
  64. 01/13 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 2, 2020
  65. 02/13 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 2, 2020
  66. 04/13 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 2, 2020
  67. 03/13 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 2, 2020
  68. 05/13 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 2, 2020
  69. 09/13 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 2, 2020
  70. Jeff KingNov 4, 2020
  71. 11/13 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 2, 2020
  72. 10/13 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 2, 2020
  73. Jeff KingNov 4, 2020
  74. 13/13 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 2, 2020
  75. Jeff KingNov 4, 2020
  76. 07/13 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 2, 2020
  77. Jeff KingNov 4, 2020
  78. Elijah NewrenNov 4, 2020
  79. 08/13 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 2, 2020
  80. 06/13 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 2, 2020
  81. 12/13 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 2, 2020
  82. Jeff KingNov 4, 2020
  83. Jeff KingNov 4, 2020
  84. Elijah NewrenNov 4, 2020
  85. 00/13 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 5, 2020
  86. 13/13 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 5, 2020
  87. 11/13 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 5, 2020
  88. 10/13 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 5, 2020
  89. 12/13 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 5, 2020
  90. 07/13 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 5, 2020
  91. 06/13 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 5, 2020
  92. 09/13 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 5, 2020
  93. 08/13 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 5, 2020
  94. 04/13 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 5, 2020
  95. 05/13 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 5, 2020
  96. 02/13 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 5, 2020
  97. 03/13 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 5, 2020
  98. 01/13 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 5, 2020
  99. Jeff KingNov 5, 2020
  100. Junio C HamanoNov 5, 2020
  101. Jeff KingNov 5, 2020
  102. Elijah NewrenNov 5, 2020
  103. Junio C HamanoNov 5, 2020
  104. 00/15 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 6, 2020
  105. 01/15 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 6, 2020
  106. 02/15 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 6, 2020
  107. 04/15 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 6, 2020
  108. 03/15 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 6, 2020
  109. 10/15 strmap: split create_entry() out of strmap_put()Elijah Newren via GitGitGadget, Nov 6, 2020
  110. 11/15 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 6, 2020
  111. 05/15 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 6, 2020
  112. 13/15 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 6, 2020
  113. 06/15 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 6, 2020
  114. 09/15 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 6, 2020
  115. 08/15 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 6, 2020
  116. 12/15 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 6, 2020
  117. Phillip WoodNov 11, 2020
  118. Elijah NewrenNov 11, 2020
  119. Jeff KingNov 11, 2020
  120. Chris TorekNov 11, 2020
  121. 07/15 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 6, 2020
  122. 15/15 shortlog: use strset from strmap.hElijah Newren via GitGitGadget, Nov 6, 2020
  123. 14/15 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 6, 2020
  124. Junio C HamanoNov 6, 2020
  125. Elijah NewrenNov 6, 2020
  126. Jeff KingNov 6, 2020
  127. Junio C HamanoNov 6, 2020
  128. 00/15 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 11, 2020
  129. 02/15 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 11, 2020
  130. 01/15 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 11, 2020
  131. 03/15 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 11, 2020
  132. 06/15 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 11, 2020
  133. 05/15 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 11, 2020
  134. 09/15 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 11, 2020
  135. 08/15 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 11, 2020
  136. 10/15 strmap: split create_entry() out of strmap_put()Elijah Newren via GitGitGadget, Nov 11, 2020
  137. 11/15 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 11, 2020
  138. 12/15 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 11, 2020
  139. 04/15 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 11, 2020
  140. 14/15 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 11, 2020
  141. 15/15 shortlog: use strset from strmap.hElijah Newren via GitGitGadget, Nov 11, 2020
  142. 13/15 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 11, 2020
  143. 07/15 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 11, 2020
  144. Jeff KingNov 11, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.