git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 2/6] bloom: prepare to discard incompatible Bloom filters

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Aug 24, 2023, 23:05 UTC
Message-ID
<20230824230527.2332958-1-jonathantanmy@google.com>
In-Reply-To
<ZOfeE1K8aRIECVm4@nand.local>
Taylor Blau <me@ttaylorr.com> writes:
Show 36 quoted lines
> I'm not sure I'm following. Are you saying that we should use
> get_bloom_filter_settings() instead of reading the value from
> r->settings directly?
> 
> > then compare filter->version to version 2 if hash_version is 2, and to
> > version 1 otherwise.
> 
> Hmm. I think we're already doing the right thing here, no? IIUC, you're
> saying to do something like:
> 
>     struct bloom_filter_settings *s = get_bloom_filter_settings(r);
>     struct bloom_filter *f = get_or_compute_bloom_filter(r, c, ...);
>     int hash_version;
> 
>     if (!f)
>       return NULL;
> 
>     prepare_repo_settings(r);
>     hash_version = r->settings.commit_graph_changed_paths_version;
> 
>     if (!(hash_version == -1 || hash_version == s->hash_version))
>       return NULL; /* incompatible */
>     return f;
> 
> ?
> 
> If so, I think that we're already OK here, since s->hash_version is
> always going to take the same value as f->version, since f->version is
> assigned in bloom.c::load_bloom_filter_from_graph(), which does:
> 
>     filter->version = g->bloom_filter_settings->hash_version;
> 
> Or are we talking about two different things entirely? ;-)
> 
> Thanks,
> Taylor

Ah, sorry for not being clear. What I meant is in how you compute hash_version after we have found that filter is not NULL.

Show 10 quoted lines
> +struct bloom_filter *get_bloom_filter(struct repository *r, struct commit *c)
> +{
> +	struct bloom_filter *filter;
> +	int hash_version;
> +
> +	filter = get_or_compute_bloom_filter(r, c, 0, NULL, NULL);
> +	if (!filter)
> +		return NULL;
> +
> +	prepare_repo_settings(r);
Up to here is fine.
> +	hash_version = r->settings.commit_graph_changed_paths_version;

Instead of doing this, do this (well, move the struct declaration to the top):

  struct bloom_filter_settings *s = get_bloom_filter_settings(r);
  hash_version = s->hash_version == 2 ? 2 : 1;
  
> +	if (!(hash_version == -1 || hash_version == filter->version))
No need for the comparison to -1 here.
> +		return NULL; /* unusable filter */
> +	return filter;
> +}
This is fine.
Previous: Taylor BlauNext: Taylor Blau
Message 11 of 24 in “bloom: reuse existing Bloom filters when possible during upgrade”
  1. 0/6 bloom: reuse existing Bloom filters when possible during upgradeTaylor Blau, Aug 7, 2023
  2. 3/6 t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`Taylor Blau, Aug 7, 2023
  3. 4/6 commit-graph.c: unconditionally load Bloom filtersTaylor Blau, Aug 7, 2023
  4. Jonathan TanAug 11, 2023
  5. Taylor BlauAug 21, 2023
  6. 2/6 bloom: prepare to discard incompatible Bloom filtersTaylor Blau, Aug 7, 2023
  7. Jonathan TanAug 11, 2023
  8. Taylor BlauAug 21, 2023
  9. Jonathan TanAug 24, 2023
  10. Taylor BlauAug 24, 2023
  11. Jonathan TanAug 24, 2023
  12. Taylor BlauAug 25, 2023
  13. Jonathan TanAug 29, 2023
  14. Taylor BlauAug 29, 2023
  15. Jonathan TanAug 29, 2023
  16. 5/6 object.h: fix mis-aligned flag bits tableTaylor Blau, Aug 7, 2023
  17. 1/6 bloom: annotate filters with hash versionTaylor Blau, Aug 7, 2023
  18. Jonathan TanAug 11, 2023
  19. Taylor BlauAug 17, 2023
  20. Taylor BlauAug 21, 2023
  21. 6/6 commit-graph: reuse existing Bloom filters where possibleTaylor Blau, Aug 7, 2023
  22. Jonathan TanAug 11, 2023
  23. Jonathan TanAug 11, 2023
  24. Taylor BlauAug 21, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.