git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 3/5] make object_directory.loose_objects_subdir_seen a bitmap

From
EWEric Wong <e@80x24.org>
Date
Jun 29, 2021, 20:53 UTC
Message-ID
<20210629205305.7100-4-e@80x24.org>
In-Reply-To
<20210627024718.25383-1-e@80x24.org>

There's no point in using 8 bits per-directory when 1 bit will do. This saves us 224 bytes per object directory, which ends up being 22MB when dealing with 100K alternates.

v2: use bitsizeof() macro and better variable names
Signed-off-by: Eric Wong <e@80x24.org>
---
 object-file.c  | 11 ++++++++---
 object-store.h |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/object-file.c b/object-file.c
index 6be43c2b60..91183d1297 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2463,12 +2463,17 @@ struct oid_array *odb_loose_cache(struct object_directory *odb,
 {
 	int subdir_nr = oid->hash[0];
 	struct strbuf buf = STRBUF_INIT;
+	size_t word_bits = bitsizeof(odb->loose_objects_subdir_seen[0]);
+	size_t word_index = subdir_nr / word_bits;
+	size_t mask = 1 << (subdir_nr % word_bits);
+	uint32_t *bitmap;
 
 	if (subdir_nr < 0 ||
-	    subdir_nr >= ARRAY_SIZE(odb->loose_objects_subdir_seen))
+	    subdir_nr >= bitsizeof(odb->loose_objects_subdir_seen))
 		BUG("subdir_nr out of range");
 
-	if (odb->loose_objects_subdir_seen[subdir_nr])
+	bitmap = &odb->loose_objects_subdir_seen[word_index];
+	if (*bitmap & mask)
 		return &odb->loose_objects_cache[subdir_nr];
 
 	strbuf_addstr(&buf, odb->path);
@@ -2476,7 +2481,7 @@ struct oid_array *odb_loose_cache(struct object_directory *odb,
 				    append_loose_object,
 				    NULL, NULL,
 				    &odb->loose_objects_cache[subdir_nr]);
-	odb->loose_objects_subdir_seen[subdir_nr] = 1;
+	*bitmap |= mask;
 	strbuf_release(&buf);
 	return &odb->loose_objects_cache[subdir_nr];
 }
diff --git a/object-store.h b/object-store.h
index 20c1cedb75..8fcddf3e65 100644
--- a/object-store.h
+++ b/object-store.h
@@ -22,7 +22,7 @@ struct object_directory {
 	 *
 	 * Be sure to call odb_load_loose_cache() before using.
 	 */
-	char loose_objects_subdir_seen[256];
+	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
 	struct oid_array loose_objects_cache[256];
 
 	/*
Previous: Eric WongNext: Eric Wong
Message 28 of 99 in “speed up alt_odb_usable() with many alternates”
  1. speed up alt_odb_usable() with many alternatesEric Wong, Jun 24, 2021
  2. 0/5 optimizations for many odb alternatesEric Wong, Jun 27, 2021
  3. 2/5 avoid strlen via strbuf_addstr in link_alt_odb_entryEric Wong, Jun 27, 2021
  4. 1/5 speed up alt_odb_usable() with many alternatesEric Wong, Jun 27, 2021
  5. 3/5 make object_directory.loose_objects_subdir_seen a bitmapEric Wong, Jun 27, 2021
  6. René ScharfeJun 27, 2021
  7. Eric WongJun 28, 2021
  8. 4/5 oidcpy_with_padding: constify `src' argEric Wong, Jun 27, 2021
  9. 5/5 oidtree: a crit-bit tree for odb_loose_cacheEric Wong, Jun 27, 2021
  10. Junio C HamanoJun 29, 2021
  11. Eric WongJun 29, 2021
  12. 0/5 optimizations for many alternatesEric Wong, Jun 29, 2021
  13. 0/5 optimizations for many alternatesEric Wong, Jul 7, 2021
  14. 1/5 speed up alt_odb_usable() with many alternatesEric Wong, Jul 7, 2021
  15. Junio C HamanoJul 8, 2021
  16. Eric WongJul 8, 2021
  17. Junio C HamanoJul 8, 2021
  18. 2/5 avoid strlen via strbuf_addstr in link_alt_odb_entryEric Wong, Jul 7, 2021
  19. Junio C HamanoJul 8, 2021
  20. 3/5 make object_directory.loose_objects_subdir_seen a bitmapEric Wong, Jul 7, 2021
  21. 4/5 oidcpy_with_padding: constify `src' argEric Wong, Jul 7, 2021
  22. 5/5 oidtree: a crit-bit tree for odb_loose_cacheEric Wong, Jul 7, 2021
  23. 1/5 speed up alt_odb_usable() with many alternatesEric Wong, Jun 29, 2021
  24. René ScharfeJul 3, 2021
  25. René ScharfeJul 4, 2021
  26. Eric WongJul 6, 2021
  27. 2/5 avoid strlen via strbuf_addstr in link_alt_odb_entryEric Wong, Jun 29, 2021
  28. 3/5 make object_directory.loose_objects_subdir_seen a bitmapEric Wong, Jun 29, 2021
  29. 4/5 oidcpy_with_padding: constify `src' argEric Wong, Jun 29, 2021
  30. 5/5 oidtree: a crit-bit tree for odb_loose_cacheEric Wong, Jun 29, 2021
  31. René ScharfeJul 4, 2021
  32. Eric WongJul 6, 2021
  33. Ævar Arnfjörð BjarmasonJul 4, 2021
  34. Eric WongJul 7, 2021
  35. Andrzej HuntAug 6, 2021
  36. René ScharfeAug 6, 2021
  37. Eric WongAug 7, 2021
  38. Carlo ArenasAug 9, 2021
  39. 0/3 pedantic errors in nextCarlo Marcelo Arenas Belón, Aug 9, 2021
  40. 2/3 object-store: avoid extra ';' from KHASH_INITCarlo Marcelo Arenas Belón, Aug 9, 2021
  41. Junio C HamanoAug 9, 2021
  42. 1/3 oidtree: avoid nested struct oidtree_nodeCarlo Marcelo Arenas Belón, Aug 9, 2021
  43. 3/3 ci: run a pedantic build as part of the GitHub workflowCarlo Marcelo Arenas Belón, Aug 9, 2021
  44. Bagas SanjayaAug 9, 2021
  45. Carlo ArenasAug 9, 2021
  46. Phillip WoodAug 9, 2021
  47. Carlo ArenasAug 9, 2021
  48. Phillip WoodAug 10, 2021
  49. Junio C HamanoAug 10, 2021
  50. Ævar Arnfjörð BjarmasonAug 30, 2021
  51. Carlo ArenasAug 31, 2021
  52. Ævar Arnfjörð BjarmasonAug 31, 2021
  53. Carlo ArenasAug 31, 2021
  54. Jeff KingSep 1, 2021
  55. Junio C HamanoSep 1, 2021
  56. Ævar Arnfjörð BjarmasonAug 30, 2021
  57. 0/4 developer: support pedanticCarlo Marcelo Arenas Belón, Sep 1, 2021
  58. 1/4 developer: retire USE_PARENS_AROUND_GETTEXT_N supportCarlo Marcelo Arenas Belón, Sep 1, 2021
  59. 2/4 developer: enable pedantic by defaultCarlo Marcelo Arenas Belón, Sep 1, 2021
  60. 3/4 developer: add an alternative script for detecting broken N_()Carlo Marcelo Arenas Belón, Sep 1, 2021
  61. 4/4 developer: move detect-compiler out of the main directoryCarlo Marcelo Arenas Belón, Sep 1, 2021
  62. Jeff KingSep 1, 2021
  63. gettext: remove optional non-standard parens in N_() definitionÆvar Arnfjörð Bjarmason, Sep 1, 2021
  64. Eric SunshineSep 1, 2021
  65. Jeff KingSep 2, 2021
  66. Junio C HamanoSep 2, 2021
  67. Ævar Arnfjörð BjarmasonSep 1, 2021
  68. Carlo ArenasSep 1, 2021
  69. 0/3 support pedantic in developer modeCarlo Marcelo Arenas Belón, Sep 3, 2021
  70. 1/3 gettext: remove optional non-standard parens in N_() definitionCarlo Marcelo Arenas Belón, Sep 3, 2021
  71. Ævar Arnfjörð BjarmasonSep 10, 2021
  72. 2/3 win32: allow building with pedantic mode enabledCarlo Marcelo Arenas Belón, Sep 3, 2021
  73. René ScharfeSep 3, 2021
  74. Carlo Marcelo Arenas BelónSep 3, 2021
  75. Junio C HamanoSep 3, 2021
  76. René ScharfeSep 3, 2021
  77. René ScharfeSep 4, 2021
  78. Carlo ArenasSep 4, 2021
  79. Jonathan TanSep 27, 2021
  80. Carlo ArenasSep 28, 2021
  81. Jonathan TanSep 28, 2021
  82. Junio C HamanoSep 28, 2021
  83. Jonathan TanSep 28, 2021
  84. Carlo ArenasSep 29, 2021
  85. Junio C HamanoSep 29, 2021
  86. 3/3 developer: enable pedantic by defaultCarlo Marcelo Arenas Belón, Sep 3, 2021
  87. Ævar Arnfjörð BjarmasonSep 5, 2021
  88. Junio C HamanoAug 9, 2021
  89. Eric WongAug 9, 2021
  90. Carlo Marcelo Arenas BelónAug 10, 2021
  91. René ScharfeAug 10, 2021
  92. Carlo ArenasAug 10, 2021
  93. Carlo ArenasAug 11, 2021
  94. René ScharfeAug 11, 2021
  95. Junio C HamanoAug 11, 2021
  96. René ScharfeAug 10, 2021
  97. René ScharfeAug 10, 2021
  98. oidtree: avoid unaligned access to crit-bit treeRené Scharfe, Aug 14, 2021
  99. Junio C HamanoAug 16, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.