git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 01/17] t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`

From
Taylor Blau <me@ttaylorr.com>
Date
Oct 18, 2023, 18:32 UTC
Message-ID
<e0fc51c3fb345c7e9ee3a64dca94e87ba2378382.1697653929.git.me@ttaylorr.com>
In-Reply-To
<cover.1697653929.git.me@ttaylorr.com>

The existing implementation of test_bloom_filters_not_used() asserts that the Bloom filter sub-system has not been initialized at all, by checking for the absence of any data from it from trace2.

In the following commit, it will become possible to load Bloom filters without using them (e.g., because `commitGraph.changedPathVersion` is incompatible with the hash version with which the commit-graph's Bloom filters were written).

When this is the case, it's possible to initialize the Bloom filter sub-system, while still not using any Bloom filters. When this is the case, check that the data dump from the Bloom sub-system is all zeros, indicating that no filters were used.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 t/t4216-log-bloom.sh | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/t/t4216-log-bloom.sh b/t/t4216-log-bloom.sh
index fa9d32facf..487fc3d6b9 100755
--- a/t/t4216-log-bloom.sh
+++ b/t/t4216-log-bloom.sh
@@ -81,7 +81,19 @@ test_bloom_filters_used () {
 test_bloom_filters_not_used () {
 	log_args=$1
 	setup "$log_args" &&
-	! grep -q "statistics:{\"filter_not_present\":" "$TRASH_DIRECTORY/trace.perf" &&
+
+	if grep -q "statistics:{\"filter_not_present\":" "$TRASH_DIRECTORY/trace.perf"
+	then
+		# if the Bloom filter system is initialized, ensure that no
+		# filters were used
+		data="statistics:{"
+		data="$data\"filter_not_present\":0,"
+		data="$data\"maybe\":0,"
+		data="$data\"definitely_not\":0,"
+		data="$data\"false_positive\":0}"
+
+		grep -q "$data" "$TRASH_DIRECTORY/trace.perf"
+	fi &&
 	test_cmp log_wo_bloom log_w_bloom
 }
 
-- 
2.42.0.415.g8942f205c8
Previous: Taylor BlauNext: Taylor Blau
Message 49 of 89 in “merge-ort: implement support for packing objects together”
  1. 0/7 merge-ort: implement support for packing objects togetherTaylor Blau, Oct 6, 2023
  2. 1/7 bulk-checkin: factor out `format_object_header_hash()`Taylor Blau, Oct 6, 2023
  3. 2/7 bulk-checkin: factor out `prepare_checkpoint()`Taylor Blau, Oct 6, 2023
  4. 3/7 bulk-checkin: factor out `truncate_checkpoint()`Taylor Blau, Oct 6, 2023
  5. 4/7 bulk-checkin: factor our `finalize_checkpoint()`Taylor Blau, Oct 6, 2023
  6. 5/7 bulk-checkin: introduce `index_blob_bulk_checkin_incore()`Taylor Blau, Oct 6, 2023
  7. 6/7 bulk-checkin: introduce `index_tree_bulk_checkin_incore()`Taylor Blau, Oct 6, 2023
  8. Eric BiedermanOct 7, 2023
  9. Taylor BlauOct 9, 2023
  10. 7/7 builtin/merge-tree.c: implement support for `--write-pack`Taylor Blau, Oct 6, 2023
  11. Junio C HamanoOct 6, 2023
  12. Taylor BlauOct 6, 2023
  13. Elijah NewrenOct 8, 2023
  14. Taylor BlauOct 8, 2023
  15. Jeff KingOct 8, 2023
  16. Taylor BlauOct 9, 2023
  17. Jeff KingOct 9, 2023
  18. Junio C HamanoOct 9, 2023
  19. Patrick SteinhardtOct 9, 2023
  20. Taylor BlauOct 9, 2023
  21. Patrick SteinhardtOct 10, 2023
  22. 0/7 merge-ort: implement support for packing objects togetherTaylor Blau, Oct 17, 2023
  23. 1/7 bulk-checkin: factor out `format_object_header_hash()`Taylor Blau, Oct 17, 2023
  24. 2/7 bulk-checkin: factor out `prepare_checkpoint()`Taylor Blau, Oct 17, 2023
  25. 3/7 bulk-checkin: factor out `truncate_checkpoint()`Taylor Blau, Oct 17, 2023
  26. 4/7 bulk-checkin: factor our `finalize_checkpoint()`Taylor Blau, Oct 17, 2023
  27. 5/7 bulk-checkin: introduce `index_blob_bulk_checkin_incore()`Taylor Blau, Oct 17, 2023
  28. Junio C HamanoOct 18, 2023
  29. Taylor BlauOct 18, 2023
  30. 6/7 bulk-checkin: introduce `index_tree_bulk_checkin_incore()`Taylor Blau, Oct 17, 2023
  31. 7/7 builtin/merge-tree.c: implement support for `--write-pack`Taylor Blau, Oct 17, 2023
  32. 00/10 merge-ort: implement support for packing objects togetherTaylor Blau, Oct 18, 2023
  33. 03/10 bulk-checkin: factor out `truncate_checkpoint()`Taylor Blau, Oct 18, 2023
  34. 10/10 builtin/merge-tree.c: implement support for `--write-pack`Taylor Blau, Oct 18, 2023
  35. 07/10 bulk-checkin: generify `stream_blob_to_pack()` for arbitrary typesTaylor Blau, Oct 18, 2023
  36. 08/10 bulk-checkin: introduce `index_blob_bulk_checkin_incore()`Taylor Blau, Oct 18, 2023
  37. Junio C HamanoOct 18, 2023
  38. Taylor BlauOct 19, 2023
  39. 09/10 bulk-checkin: introduce `index_tree_bulk_checkin_incore()`Taylor Blau, Oct 18, 2023
  40. 01/10 bulk-checkin: factor out `format_object_header_hash()`Taylor Blau, Oct 18, 2023
  41. 02/10 bulk-checkin: factor out `prepare_checkpoint()`Taylor Blau, Oct 18, 2023
  42. 04/10 bulk-checkin: factor out `finalize_checkpoint()`Taylor Blau, Oct 18, 2023
  43. 05/10 bulk-checkin: extract abstract `bulk_checkin_source`Taylor Blau, Oct 18, 2023
  44. Junio C HamanoOct 18, 2023
  45. Taylor BlauOct 19, 2023
  46. Junio C HamanoOct 19, 2023
  47. 06/10 bulk-checkin: implement `SOURCE_INCORE` mode for `bulk_checkin_source`Taylor Blau, Oct 18, 2023
  48. 00/17 bloom: changed-path Bloom filters v2 (& sundries)Taylor Blau, Oct 18, 2023
  49. 01/17 t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`Taylor Blau, Oct 18, 2023
  50. 02/17 revision.c: consult Bloom filters for root commitsTaylor Blau, Oct 18, 2023
  51. 03/17 commit-graph: ensure Bloom filters are read with consistent settingsTaylor Blau, Oct 18, 2023
  52. 04/17 gitformat-commit-graph: describe version 2 of BDATTaylor Blau, Oct 18, 2023
  53. 05/17 t/helper/test-read-graph.c: extract `dump_graph_info()`Taylor Blau, Oct 18, 2023
  54. 06/17 bloom.h: make `load_bloom_filter_from_graph()` publicTaylor Blau, Oct 18, 2023
  55. 07/17 t/helper/test-read-graph: implement `bloom-filters` modeTaylor Blau, Oct 18, 2023
  56. 08/17 t4216: test changed path filters with high bit pathsTaylor Blau, Oct 18, 2023
  57. 09/17 repo-settings: introduce commitgraph.changedPathsVersionTaylor Blau, Oct 18, 2023
  58. 10/17 commit-graph: new filter ver. that fixes murmur3Taylor Blau, Oct 18, 2023
  59. 11/17 bloom: annotate filters with hash versionTaylor Blau, Oct 18, 2023
  60. 12/17 bloom: prepare to discard incompatible Bloom filtersTaylor Blau, Oct 18, 2023
  61. 13/17 commit-graph.c: unconditionally load Bloom filtersTaylor Blau, Oct 18, 2023
  62. 14/17 commit-graph: drop unnecessary `graph_read_bloom_data_context`Taylor Blau, Oct 18, 2023
  63. 15/17 object.h: fix mis-aligned flag bits tableTaylor Blau, Oct 18, 2023
  64. 16/17 commit-graph: reuse existing Bloom filters where possibleTaylor Blau, Oct 18, 2023
  65. 17/17 bloom: introduce `deinit_bloom_filters()`Taylor Blau, Oct 18, 2023
  66. Junio C HamanoOct 18, 2023
  67. Taylor BlauOct 20, 2023
  68. SZEDER GáborOct 23, 2023
  69. Taylor BlauOct 30, 2023
  70. 00/17 bloom: changed-path Bloom filters v2 (& sundries)Taylor Blau, Jan 16, 2024
  71. 01/17 t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`Taylor Blau, Jan 16, 2024
  72. 02/17 revision.c: consult Bloom filters for root commitsTaylor Blau, Jan 16, 2024
  73. 03/17 commit-graph: ensure Bloom filters are read with consistent settingsTaylor Blau, Jan 16, 2024
  74. 04/17 gitformat-commit-graph: describe version 2 of BDATTaylor Blau, Jan 16, 2024
  75. 05/17 t/helper/test-read-graph.c: extract `dump_graph_info()`Taylor Blau, Jan 16, 2024
  76. 06/17 bloom.h: make `load_bloom_filter_from_graph()` publicTaylor Blau, Jan 16, 2024
  77. 07/17 t/helper/test-read-graph: implement `bloom-filters` modeTaylor Blau, Jan 16, 2024
  78. 08/17 t4216: test changed path filters with high bit pathsTaylor Blau, Jan 16, 2024
  79. 09/17 repo-settings: introduce commitgraph.changedPathsVersionTaylor Blau, Jan 16, 2024
  80. SZEDER GáborJan 29, 2024
  81. Taylor BlauJan 29, 2024
  82. 10/17 commit-graph: new Bloom filter version that fixes murmur3Taylor Blau, Jan 16, 2024
  83. 11/17 bloom: annotate filters with hash versionTaylor Blau, Jan 16, 2024
  84. 12/17 bloom: prepare to discard incompatible Bloom filtersTaylor Blau, Jan 16, 2024
  85. 13/17 commit-graph.c: unconditionally load Bloom filtersTaylor Blau, Jan 16, 2024
  86. 14/17 commit-graph: drop unnecessary `graph_read_bloom_data_context`Taylor Blau, Jan 16, 2024
  87. 15/17 object.h: fix mis-aligned flag bits tableTaylor Blau, Jan 16, 2024
  88. 16/17 commit-graph: reuse existing Bloom filters where possibleTaylor Blau, Jan 16, 2024
  89. 17/17 bloom: introduce `deinit_bloom_filters()`Taylor Blau, Jan 16, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.