git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] hashmap: add API to disable item counting when threaded

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Sep 4, 2017, 15:59 UTC
Message-ID
<alpine.DEB.2.21.1.1709041758350.4132@virtualbox>
In-Reply-To
<20170902081747.lca2kkzpniykdxy2@sigill.intra.peff.net>
Hi Peff,
On Sat, 2 Sep 2017, Jeff King wrote:
Show 34 quoted lines
> On Sat, Sep 02, 2017 at 01:31:19AM +0200, Johannes Schindelin wrote:
> 
> > > https://public-inbox.org/git/adb37b70139fd1e2bac18bfd22c8b96683ae18eb.1502780344.git.martin.agren@gmail.com/
> > [...]
> > > +static inline void hashmap_enable_item_counting(struct hashmap *map)
> > > +{
> > > +	void *item;
> > > +	unsigned int n = 0;
> > > +	struct hashmap_iter iter;
> > > +
> > > +	hashmap_iter_init(map, &iter);
> > > +	while ((item = hashmap_iter_next(&iter)))
> > > +		n++;
> > > +
> > > +	map->do_count_items = 1;
> > > +	map->private_size = n;
> > > +}
> > 
> > BTW this made me think that we may have a problem in our code since
> > switching from my original hashmap implementation to the bucket one
> > added in 6a364ced497 (add a hashtable implementation that supports
> > O(1) removal, 2013-11-14): while it is not expected that there are
> > many collisions, the "grow_at" logic still essentially assumes the
> > number of buckets to be equal to the number of hashmap entries.
> 
> I'm confused about what the problem is. If I am reading the code
> correctly, "size" is always the number of elements and "grow_at" is the
> table size times a load factor. Those are the same numbers you'd use to
> decide to grow in an open-address table.
> 
> It's true that this does not take into account the actual number of
> collisions we see (or the average per bucket, or however you want to
> count it). But generally nor do open-address schemes (and certainly our
> other hash tables just use load factor to decide when to grow).

In the worst case, there is only one bucket when the table is grown already, is all I tried to point out.

Ciao, Dscho

Previous: Jeff KingNext: Jeff Hostetler
Message 23 of 64 in “Some ThreadSanitizer-results”
  1. 0/5 Some ThreadSanitizer-resultsMartin Ågren, Aug 15, 2017
  2. 1/5 convert: initialize attr_action in convert_attrsMartin Ågren, Aug 15, 2017
  3. Torsten BögershausenAug 15, 2017
  4. Torsten BögershausenAug 15, 2017
  5. Martin ÅgrenAug 15, 2017
  6. 2/5 pack-objects: take lock before accessing `remaining`Martin Ågren, Aug 15, 2017
  7. Johannes SixtAug 15, 2017
  8. 5/5 ThreadSanitizer: add suppressionsMartin Ågren, Aug 15, 2017
  9. tsan: t3008: hashmap_add touches size from multiple threadsMartin Ågren, Aug 15, 2017
  10. Jeff HostetlerAug 15, 2017
  11. Stefan BellerAug 15, 2017
  12. Martin ÅgrenAug 15, 2017
  13. Stefan BellerAug 15, 2017
  14. Martin ÅgrenAug 15, 2017
  15. Jeff HostetlerAug 15, 2017
  16. hashmap: address ThreadSanitizer concernsJeff Hostetler, Aug 30, 2017
  17. hashmap: add API to disable item counting when threadedJeff Hostetler, Aug 30, 2017
  18. Johannes SchindelinSep 1, 2017
  19. Jonathan NiederSep 1, 2017
  20. Jeff HostetlerSep 5, 2017
  21. Martin ÅgrenSep 5, 2017
  22. Jeff KingSep 2, 2017
  23. Johannes SchindelinSep 4, 2017
  24. Jeff HostetlerSep 5, 2017
  25. Junio C HamanoSep 6, 2017
  26. Jeff HostetlerSep 5, 2017
  27. Jeff KingSep 2, 2017
  28. Jeff HostetlerSep 5, 2017
  29. Simon RuderichSep 2, 2017
  30. Junio C HamanoSep 6, 2017
  31. Jeff HostetlerSep 6, 2017
  32. hashmap: address ThreadSanitizer concernsJeff Hostetler, Sep 6, 2017
  33. hashmap: add API to disable item counting when threadedJeff Hostetler, Sep 6, 2017
  34. tsan: t5400: set_try_to_free_routineMartin Ågren, Aug 15, 2017
  35. Stefan BellerAug 15, 2017
  36. Martin ÅgrenAug 15, 2017
  37. Jeff KingAug 17, 2017
  38. 4/5 strbuf_reset: don't write to slopbuf with ThreadSanitizerMartin Ågren, Aug 15, 2017
  39. Junio C HamanoAug 15, 2017
  40. Martin ÅgrenAug 15, 2017
  41. Junio C HamanoAug 15, 2017
  42. 3/5 Makefile: define GIT_THREAD_SANITIZERMartin Ågren, Aug 15, 2017
  43. Jeff KingAug 20, 2017
  44. Martin ÅgrenAug 20, 2017
  45. 0/4 Some ThreadSanitizer-resultsMartin Ågren, Aug 21, 2017
  46. 1/4 convert: always initialize attr_action in convert_attrsMartin Ågren, Aug 21, 2017
  47. 2/4 pack-objects: take lock before accessing `remaining`Martin Ågren, Aug 21, 2017
  48. 3/4 strbuf_setlen: don't write to strbuf_slopbufMartin Ågren, Aug 21, 2017
  49. Junio C HamanoAug 23, 2017
  50. Martin ÅgrenAug 23, 2017
  51. Junio C HamanoAug 23, 2017
  52. Brandon CaseyAug 23, 2017
  53. Junio C HamanoAug 23, 2017
  54. Brandon CaseyAug 23, 2017
  55. Brandon CaseyAug 23, 2017
  56. Brandon CaseyAug 23, 2017
  57. Junio C HamanoAug 24, 2017
  58. Brandon CaseyAug 24, 2017
  59. Martin ÅgrenAug 24, 2017
  60. Junio C HamanoAug 23, 2017
  61. Brandon CaseyAug 23, 2017
  62. 4/4 ThreadSanitizer: add suppressionsMartin Ågren, Aug 21, 2017
  63. Jeff KingAug 25, 2017
  64. Jeff HostetlerAug 28, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.