git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 6/6] commit-graph: reuse existing Bloom filters where possible

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Aug 11, 2023, 22:06 UTC
Message-ID
<20230811220637.3330277-1-jonathantanmy@google.com>
In-Reply-To
<93f830ca61d17bb20f63c6a4254fe95816ae1cbe.1691426160.git.me@ttaylorr.com>
Taylor Blau <me@ttaylorr.com> writes:
Show 26 quoted lines
> @@ -292,9 +362,23 @@ struct bloom_filter *get_or_compute_bloom_filter(struct repository *r,
>  						     filter, graph_pos);
>  	}
>  
> -	if ((filter->data && filter->len) &&
> -	    (!settings || settings->hash_version == filter->version))
> -		return filter;
> +	if (filter->data && filter->len) {
> +		struct bloom_filter *upgrade;
> +		if (!settings || settings->hash_version == filter->version)
> +			return filter;
> +
> +		/* version mismatch, see if we can upgrade */
> +		if (compute_if_not_present &&
> +		    git_env_bool("GIT_TEST_UPGRADE_BLOOM_FILTERS", 1)) {
> +			upgrade = upgrade_filter(r, c, filter,
> +						 settings->hash_version);
> +			if (upgrade) {
> +				if (computed)
> +					*computed |= BLOOM_UPGRADED;
> +				return upgrade;
> +			}
> +		}
> +	}
>  	if (!compute_if_not_present)
>  		return NULL;

There's a part in this function that's untouched by the diff that computes and stores a new Bloom filter if it was not possible to upgrade the filter (look at the part where filter->data is calloc-ed). That part unconditionally overrides the existing filter->data, which may at that point contain genuine data (a Bloom filter that does not match the version we want). Right now this is fine because all overrides that we do involve overriding references to existing buffers, so we do not need to free anything.

Previous: Taylor BlauNext: Jonathan Tan
Message 22 of 24 in “bloom: reuse existing Bloom filters when possible during upgrade”
  1. 0/6 bloom: reuse existing Bloom filters when possible during upgradeTaylor Blau, Aug 7, 2023
  2. 3/6 t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`Taylor Blau, Aug 7, 2023
  3. 4/6 commit-graph.c: unconditionally load Bloom filtersTaylor Blau, Aug 7, 2023
  4. Jonathan TanAug 11, 2023
  5. Taylor BlauAug 21, 2023
  6. 2/6 bloom: prepare to discard incompatible Bloom filtersTaylor Blau, Aug 7, 2023
  7. Jonathan TanAug 11, 2023
  8. Taylor BlauAug 21, 2023
  9. Jonathan TanAug 24, 2023
  10. Taylor BlauAug 24, 2023
  11. Jonathan TanAug 24, 2023
  12. Taylor BlauAug 25, 2023
  13. Jonathan TanAug 29, 2023
  14. Taylor BlauAug 29, 2023
  15. Jonathan TanAug 29, 2023
  16. 5/6 object.h: fix mis-aligned flag bits tableTaylor Blau, Aug 7, 2023
  17. 1/6 bloom: annotate filters with hash versionTaylor Blau, Aug 7, 2023
  18. Jonathan TanAug 11, 2023
  19. Taylor BlauAug 17, 2023
  20. Taylor BlauAug 21, 2023
  21. 6/6 commit-graph: reuse existing Bloom filters where possibleTaylor Blau, Aug 7, 2023
  22. Jonathan TanAug 11, 2023
  23. Jonathan TanAug 11, 2023
  24. Taylor BlauAug 21, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.