git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 2/6] bloom: prepare to discard incompatible Bloom filters

From
Taylor Blau <me@ttaylorr.com>
Date
Aug 25, 2023, 19:00 UTC
Message-ID
<ZOj6PazHzDeQrY88@nand.local>
In-Reply-To
<20230824230527.2332958-1-jonathantanmy@google.com>
On Thu, Aug 24, 2023 at 04:05:27PM -0700, Jonathan Tan wrote:
Show 9 quoted lines
> Up to here is fine.
>
> > +	hash_version = r->settings.commit_graph_changed_paths_version;
>
> Instead of doing this, do this (well, move the struct declaration to
> the top):
>
>   struct bloom_filter_settings *s = get_bloom_filter_settings(r);
>   hash_version = s->hash_version == 2 ? 2 : 1;

Do we need this normalization? We assign s->hash_version in commit-graph.c::graph_read_bloom_data() by reading it from the start of the BDAT chunk, so this should only ever be 1 or 2.

> > +	if (!(hash_version == -1 || hash_version == filter->version))
>
> No need for the comparison to -1 here.

I'm not sure I understand your suggestion. When we fetch the filter from get_or_compute_bloom_filter(), we have filter->version set to the hash_version from the containing graph's Bloom filter settings.

So (besides the normalization), I would think that:
    struct bloom_filter_settings *s = get_bloom_filter_settings(r);
    struct bloom_filter *f = get_bloom_filter(...);
    assert(s->hash_version == f->version);
would hold.

I think the check that we want to make is instead: is this Bloom filter's version (or equivalently, the hash version indicated by that graph's BDAT chunk) something that we can read? And I think "what we can read" here is dictated by the commit_graph_changed_paths_version member of our repository_settings, no?

Thanks, Taylor

Previous: Jonathan TanNext: Jonathan Tan
Message 12 of 24 in “bloom: reuse existing Bloom filters when possible during upgrade”
  1. 0/6 bloom: reuse existing Bloom filters when possible during upgradeTaylor Blau, Aug 7, 2023
  2. 3/6 t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`Taylor Blau, Aug 7, 2023
  3. 4/6 commit-graph.c: unconditionally load Bloom filtersTaylor Blau, Aug 7, 2023
  4. Jonathan TanAug 11, 2023
  5. Taylor BlauAug 21, 2023
  6. 2/6 bloom: prepare to discard incompatible Bloom filtersTaylor Blau, Aug 7, 2023
  7. Jonathan TanAug 11, 2023
  8. Taylor BlauAug 21, 2023
  9. Jonathan TanAug 24, 2023
  10. Taylor BlauAug 24, 2023
  11. Jonathan TanAug 24, 2023
  12. Taylor BlauAug 25, 2023
  13. Jonathan TanAug 29, 2023
  14. Taylor BlauAug 29, 2023
  15. Jonathan TanAug 29, 2023
  16. 5/6 object.h: fix mis-aligned flag bits tableTaylor Blau, Aug 7, 2023
  17. 1/6 bloom: annotate filters with hash versionTaylor Blau, Aug 7, 2023
  18. Jonathan TanAug 11, 2023
  19. Taylor BlauAug 17, 2023
  20. Taylor BlauAug 21, 2023
  21. 6/6 commit-graph: reuse existing Bloom filters where possibleTaylor Blau, Aug 7, 2023
  22. Jonathan TanAug 11, 2023
  23. Jonathan TanAug 11, 2023
  24. Taylor BlauAug 21, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.