git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 05/10] strmap: new utility functions

From
Elijah Newren <newren@gmail.com>
Date
Oct 30, 2020, 16:26 UTC
Message-ID
<CABPp-BGTjUdgmiEneVNsLWyODZtH-d+x6uTFfwbOtyba3K76HQ@mail.gmail.com>
In-Reply-To
<20201030141233.GE3277724@coredump.intra.peff.net>
On Fri, Oct 30, 2020 at 7:12 AM Jeff King <peff@peff.net> wrote:
Show 21 quoted lines
>
> On Tue, Oct 13, 2020 at 12:40:45AM +0000, Elijah Newren via GitGitGadget wrote:
>
> > Add strmap as a new struct and associated utility functions,
> > specifically for hashmaps that map strings to some value.  The API is
> > taken directly from Peff's proposal at
> > https://lore.kernel.org/git/20180906191203.GA26184@sigill.intra.peff.net/
>
> This looks overall sane to me. I mentioned elsewhere that we could be
> using FLEXPTR_ALLOC to save an extra allocation. I think it's easy and
> worth doing here, as the logic would be completely contained within
> strmap_put():
>
>   if (strdup_strings)
>         FLEXPTR_ALLOC_STR(entry, key, str);
>   else {
>         entry = xmalloc(sizeof(*entry));
>         entry->key = str;
>   }
>
> And free_entries() then doesn't even have to care about strdup_strings.

Yeah, as you noted in your review of 10/10 this idea wouldn't play well with the later mem_pool changes.

Show 10 quoted lines
> > A couple of items of note:
> >
> >   * Similar to string-list, I have a strdup_strings setting.  However,
> >     unlike string-list, strmap_init() does not take a parameter for this
> >     setting and instead automatically sets it to 1; callers who want to
> >     control this detail need to instead call strmap_ocd_init().
>
> That seems reasonable. It could just be a parameter, but I like that you
> push people in the direction of doing the simple and safe thing, rather
> than having them wonder whether they ought to set strdup_strings or not.

Well, in my first round of the series where I did make it a parameter you balked pretty loudly. ;-)

Show 18 quoted lines
> >   * I do not have a STRMAP_INIT macro.  I could possibly add one, but
> >       #define STRMAP_INIT { { NULL, cmp_str_entry, NULL, 0, 0, 0, 0, 0 }, 1 }
> >     feels a bit unwieldy and possibly error-prone in terms of future
> >     expansion of the hashmap struct.  The fact that cmp_str_entry needs to
> >     be in there prevents us from passing all zeros for the hashmap, and makes
> >     me worry that STRMAP_INIT would just be more trouble than it is worth.
>
> You can actually omit everything after cmp_str_entry, and those fields
> would all get zero-initialized. But we also allow C99 designed
> initializers these days. Coupled with the HASHMAP_INIT() I mentioned in
> the earlier email, you'd have:
>
>   #define STRMAP_INIT { \
>                 .map = HASHMAP_INIT(cmp_strmap_entry, NULL), \
>                 .strdup_strings = 1, \
>           }
>
> which seems pretty maintainable.
Makes sense; will add.
Show 21 quoted lines
> > +static int cmp_strmap_entry(const void *hashmap_cmp_fn_data,
> > +                         const struct hashmap_entry *entry1,
> > +                         const struct hashmap_entry *entry2,
> > +                         const void *keydata)
> > +{
> > +     const struct strmap_entry *e1, *e2;
> > +
> > +     e1 = container_of(entry1, const struct strmap_entry, ent);
> > +     e2 = container_of(entry2, const struct strmap_entry, ent);
> > +     return strcmp(e1->key, e2->key);
> > +}
>
> I expected to use keydata here, but it's pretty easy to make a fake
> strmap_entry because of the use of the "key" pointer. So that makes
> sense.
>
> > +static void strmap_free_entries_(struct strmap *map, int free_util)
>
> You use the term "value" for the mapped-to value in this iteration. So
> perhaps free_values here (and in other functions) would be a better
> name?
Oops, yes, definitely.
Show 55 quoted lines
> > +     /*
> > +      * We need to iterate over the hashmap entries and free
> > +      * e->key and e->value ourselves; hashmap has no API to
> > +      * take care of that for us.  Since we're already iterating over
> > +      * the hashmap, though, might as well free e too and avoid the need
> > +      * to make some call into the hashmap API to do that.
> > +      */
> > +     hashmap_for_each_entry(&map->map, &iter, e, ent) {
> > +             if (free_util)
> > +                     free(e->value);
> > +             if (map->strdup_strings)
> > +                     free((char*)e->key);
> > +             free(e);
> > +     }
> > +}
>
> Yep, makes sense.
>
> > +void strmap_clear(struct strmap *map, int free_util)
> > +{
> > +     strmap_free_entries_(map, free_util);
> > +     hashmap_free(&map->map);
> > +}
>
> This made me wonder about a partial_clear(), but it looks like that
> comes later.
>
> > +void *strmap_put(struct strmap *map, const char *str, void *data)
> > +{
> > +     struct strmap_entry *entry = find_strmap_entry(map, str);
> > +     void *old = NULL;
> > +
> > +     if (entry) {
> > +             old = entry->value;
> > +             entry->value = data;
>
> Here's a weird hypothetical. If strdup_strings is not set and I do:
>
>   const char *one = xstrdup("foo");
>   const char *two = xstrdup("foo");
>
>   hashmap_put(map, one, x);
>   hashmap_put(map, two, y);
>
> it's clear that the value should be pointing to "y" afterwards (and you
> return "x" so the caller can free it or whatever, good).
>
> But which key should the entry be pointing to? The old one or the new
> one? I'm trying and failing to think of a case where it would matter.
> Certainly I could add a free() to the toy above where it would, but it
> feels like a real caller would have to have pretty convoluted memory
> lifetime semantics for it to make a difference.
>
> So I'm not really asking for a particular behavior, but just bringing it
> up in case you can think of something relevant.

I'll keep mulling it over, but I likewise can't currently think of a case where it'd matter.

Show 18 quoted lines
>
> > +     } else {
> > +             /*
> > +              * We won't modify entry->key so it really should be const.
> > +              */
> > +             const char *key = str;
>
> The "should be" here confused me. It _is_ const. I'd probably just
> delete the comment entirely, but perhaps:
>
>   /*
>    * We'll store a const pointer. For non-duplicated strings, they belong
>    * to the caller and we received them as const in the first place. For
>    * our duplicated ones, they do point to memory we own, but they're
>    * still conceptually constant within the lifetime of an entry.
>    */
>
> Though it might make more sense in the struct definition, not here.

Either I was (mistakenly) worried about "I'm going to allocate and copy, but during the copy it isn't actually const", or this is a leftover artifact from some of the other iterations I tried. Anyway, I think this comment isn't useful; I'll just strike it.

Show 14 quoted lines
> > +void *strmap_get(struct strmap *map, const char *str)
> > +{
> > +     struct strmap_entry *entry = find_strmap_entry(map, str);
> > +     return entry ? entry->value : NULL;
> > +}
>
> Just noting that the caller can't tell the difference between "no such
> entry" and "the entry is storing NULL". I think the simplicity offered
> by this interface makes it worth having (and being the primary one). If
> some caller really needs to tell the difference between the two, we can
> add another function later.
>
> Obviously they could use strmap_contains(), but that would mean two hash
> lookups.

Yep, addressed later by strmap_get_entry() in another patch, as you noticed in your later review.

Show 14 quoted lines
> > +/*
> > + * Same as strmap_init, but for those who want to control the memory management
> > + * carefully instead of using the default of strdup_strings=1.
> > + * (OCD = Obsessive Compulsive Disorder, a joke that those who use this function
> > + * are obsessing over minor details.)
> > + */
> > +void strmap_ocd_init(struct strmap *map,
> > +                  int strdup_strings);
>
> I'm not personally bothered by this name, but I wonder if some people
> may be (because they have or know somebody who actually has OCD).
>
> Perhaps strmap_init_with_options() would be a better name? It likewise
> would extend well if we want to add other non-default options later.
Doh!  That's going to push a bunch of lines past 80 characters.  Sigh...
It's probably a better name though; I'll change it.
Previous: Jeff KingNext: Elijah Newren via GitGitGadget
Message 33 of 144 in “Add struct strmap and associated utility functions”
  1. 0/5 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Aug 21, 2020
  2. 3/5 strmap: add more utility functionsElijah Newren via GitGitGadget, Aug 21, 2020
  3. Jeff KingAug 21, 2020
  4. 1/5 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Aug 21, 2020
  5. Jeff KingAug 21, 2020
  6. 5/5 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Aug 21, 2020
  7. Jeff KingAug 21, 2020
  8. Elijah NewrenAug 21, 2020
  9. Jeff KingAug 21, 2020
  10. 2/5 strmap: new utility functionsElijah Newren via GitGitGadget, Aug 21, 2020
  11. Jeff KingAug 21, 2020
  12. 4/5 strmap: add strdup_strings optionElijah Newren via GitGitGadget, Aug 21, 2020
  13. Jeff KingAug 21, 2020
  14. Elijah NewrenAug 21, 2020
  15. Jeff KingAug 21, 2020
  16. Elijah NewrenAug 21, 2020
  17. Jeff KingAug 28, 2020
  18. Elijah NewrenAug 28, 2020
  19. Jeff KingAug 21, 2020
  20. Elijah NewrenAug 21, 2020
  21. Elijah NewrenAug 21, 2020
  22. Jeff KingAug 28, 2020
  23. Elijah NewrenAug 28, 2020
  24. Jeff KingSep 1, 2020
  25. 00/10 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Oct 13, 2020
  26. 01/10 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Oct 13, 2020
  27. Jeff KingOct 30, 2020
  28. Elijah NewrenOct 30, 2020
  29. Jeff KingNov 3, 2020
  30. Elijah NewrenNov 3, 2020
  31. 05/10 strmap: new utility functionsElijah Newren via GitGitGadget, Oct 13, 2020
  32. Jeff KingOct 30, 2020
  33. Elijah NewrenOct 30, 2020
  34. 04/10 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Oct 13, 2020
  35. Jeff KingOct 30, 2020
  36. Elijah NewrenOct 30, 2020
  37. Jeff KingNov 3, 2020
  38. 03/10 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Oct 13, 2020
  39. Jeff KingOct 30, 2020
  40. Elijah NewrenOct 30, 2020
  41. Jeff KingNov 3, 2020
  42. Elijah NewrenNov 3, 2020
  43. 02/10 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Oct 13, 2020
  44. Jeff KingOct 30, 2020
  45. 07/10 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Oct 13, 2020
  46. Jeff KingOct 30, 2020
  47. 10/10 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Oct 13, 2020
  48. Jeff KingOct 30, 2020
  49. Elijah NewrenOct 30, 2020
  50. Jeff KingNov 3, 2020
  51. 06/10 strmap: add more utility functionsElijah Newren via GitGitGadget, Oct 13, 2020
  52. Jeff KingOct 30, 2020
  53. Elijah NewrenOct 30, 2020
  54. Jeff KingNov 3, 2020
  55. 08/10 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Oct 13, 2020
  56. Jeff KingOct 30, 2020
  57. Elijah NewrenOct 30, 2020
  58. Jeff KingNov 3, 2020
  59. Elijah NewrenNov 3, 2020
  60. 09/10 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Oct 13, 2020
  61. Jeff KingOct 30, 2020
  62. Elijah NewrenOct 30, 2020
  63. 00/13 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 2, 2020
  64. 01/13 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 2, 2020
  65. 02/13 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 2, 2020
  66. 04/13 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 2, 2020
  67. 03/13 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 2, 2020
  68. 05/13 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 2, 2020
  69. 09/13 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 2, 2020
  70. Jeff KingNov 4, 2020
  71. 11/13 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 2, 2020
  72. 10/13 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 2, 2020
  73. Jeff KingNov 4, 2020
  74. 13/13 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 2, 2020
  75. Jeff KingNov 4, 2020
  76. 07/13 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 2, 2020
  77. Jeff KingNov 4, 2020
  78. Elijah NewrenNov 4, 2020
  79. 08/13 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 2, 2020
  80. 06/13 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 2, 2020
  81. 12/13 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 2, 2020
  82. Jeff KingNov 4, 2020
  83. Jeff KingNov 4, 2020
  84. Elijah NewrenNov 4, 2020
  85. 00/13 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 5, 2020
  86. 13/13 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 5, 2020
  87. 11/13 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 5, 2020
  88. 10/13 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 5, 2020
  89. 12/13 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 5, 2020
  90. 07/13 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 5, 2020
  91. 06/13 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 5, 2020
  92. 09/13 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 5, 2020
  93. 08/13 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 5, 2020
  94. 04/13 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 5, 2020
  95. 05/13 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 5, 2020
  96. 02/13 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 5, 2020
  97. 03/13 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 5, 2020
  98. 01/13 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 5, 2020
  99. Jeff KingNov 5, 2020
  100. Junio C HamanoNov 5, 2020
  101. Jeff KingNov 5, 2020
  102. Elijah NewrenNov 5, 2020
  103. Junio C HamanoNov 5, 2020
  104. 00/15 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 6, 2020
  105. 01/15 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 6, 2020
  106. 02/15 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 6, 2020
  107. 04/15 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 6, 2020
  108. 03/15 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 6, 2020
  109. 10/15 strmap: split create_entry() out of strmap_put()Elijah Newren via GitGitGadget, Nov 6, 2020
  110. 11/15 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 6, 2020
  111. 05/15 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 6, 2020
  112. 13/15 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 6, 2020
  113. 06/15 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 6, 2020
  114. 09/15 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 6, 2020
  115. 08/15 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 6, 2020
  116. 12/15 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 6, 2020
  117. Phillip WoodNov 11, 2020
  118. Elijah NewrenNov 11, 2020
  119. Jeff KingNov 11, 2020
  120. Chris TorekNov 11, 2020
  121. 07/15 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 6, 2020
  122. 15/15 shortlog: use strset from strmap.hElijah Newren via GitGitGadget, Nov 6, 2020
  123. 14/15 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 6, 2020
  124. Junio C HamanoNov 6, 2020
  125. Elijah NewrenNov 6, 2020
  126. Jeff KingNov 6, 2020
  127. Junio C HamanoNov 6, 2020
  128. 00/15 Add struct strmap and associated utility functionsElijah Newren via GitGitGadget, Nov 11, 2020
  129. 02/15 hashmap: adjust spacing to fix argument alignmentElijah Newren via GitGitGadget, Nov 11, 2020
  130. 01/15 hashmap: add usage documentation explaining hashmap_free[_entries]()Elijah Newren via GitGitGadget, Nov 11, 2020
  131. 03/15 hashmap: allow re-use after hashmap_free()Elijah Newren via GitGitGadget, Nov 11, 2020
  132. 06/15 strmap: new utility functionsElijah Newren via GitGitGadget, Nov 11, 2020
  133. 05/15 hashmap: provide deallocation function namesElijah Newren via GitGitGadget, Nov 11, 2020
  134. 09/15 strmap: add functions facilitating use as a string->int mapElijah Newren via GitGitGadget, Nov 11, 2020
  135. 08/15 strmap: enable faster clearing and reusing of strmapsElijah Newren via GitGitGadget, Nov 11, 2020
  136. 10/15 strmap: split create_entry() out of strmap_put()Elijah Newren via GitGitGadget, Nov 11, 2020
  137. 11/15 strmap: add a strset sub-typeElijah Newren via GitGitGadget, Nov 11, 2020
  138. 12/15 strmap: enable allocations to come from a mem_poolElijah Newren via GitGitGadget, Nov 11, 2020
  139. 04/15 hashmap: introduce a new hashmap_partial_clear()Elijah Newren via GitGitGadget, Nov 11, 2020
  140. 14/15 Use new HASHMAP_INIT macro to simplify hashmap initializationElijah Newren via GitGitGadget, Nov 11, 2020
  141. 15/15 shortlog: use strset from strmap.hElijah Newren via GitGitGadget, Nov 11, 2020
  142. 13/15 strmap: take advantage of FLEXPTR_ALLOC_STR when relevantElijah Newren via GitGitGadget, Nov 11, 2020
  143. 07/15 strmap: add more utility functionsElijah Newren via GitGitGadget, Nov 11, 2020
  144. Jeff KingNov 11, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.