git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v5.1 3.5/4] revision: make helper for pathspec to bloom key

From
Lidong Yan <502024330056@smail.nju.edu.cn>
Date
Jul 10, 2025, 15:42 UTC
Message-ID
<7885EBB2-0D99-4456-A704-86362219AC17@smail.nju.edu.cn>
In-Reply-To
<3c59af48-23fe-4cc7-87e9-1de94f509a2b@gmail.com>
Derrick Stolee <stolee@gmail.com> wrote:
Show 64 quoted lines
> 
> This diff is still bigger than I was hoping, so I'm sending a couple
> of patches that simplify this code movement. Feel free to ignore
> them as being too nit-picky.
> 
> --- >8 ---
> 
> From 69fa36dc615e140ae842b536f7da792beaebb272 Mon Sep 17 00:00:00 2001
> From: Derrick Stolee <stolee@gmail.com>
> Date: Thu, 10 Jul 2025 08:06:29 -0400
> Subject: [PATCH v5.1 3.5/4] revision: make helper for pathspec to bloom key
> 
> When preparing to use bloom filters in a revision walk, Git populates a
> boom_keyvec with an array of bloom keys for the components of a path.
> Before we create the ability to map multiple pathspecs to multiple
> bloom_keyvecs, extract the conversion from a pathspec to a bloom_keyvec
> into its own helper method. This simplifies the state that persists in
> prepare_to_use_bloom_filter() as well as makes the next change much
> simpler.
> 
> Signed-off-by: Derrick Stolee <stolee@gmail.com>
> ---
> revision.c | 50 +++++++++++++++++++++++++++++++-------------------
> 1 file changed, 31 insertions(+), 19 deletions(-)
> 
> diff --git a/revision.c b/revision.c
> index 22bcfab7f93..4c09b594c55 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -687,14 +687,37 @@ static int forbid_bloom_filters(struct pathspec *spec)
> 
> static void release_revisions_bloom_keyvecs(struct rev_info *revs);
> 
> -static void prepare_to_use_bloom_filter(struct rev_info *revs)
> +static int convert_pathspec_to_filter(const struct pathspec_item *pi,
> +      struct bloom_keyvec **bloom_keyvec,
> +      const struct bloom_filter_settings *settings)
> {
> - struct pathspec_item *pi;
> - struct bloom_keyvec *bloom_keyvec;
> - char *path_alloc = NULL;
> - const char *path, *p;
> size_t len;
> + const char *path;
> + char *path_alloc = NULL;
> + int res = 0;
> +
> + /* remove single trailing slash from path, if needed */
> + if (pi->len > 0 && pi->match[pi->len - 1] == '/') {
> + path_alloc = xmemdupz(pi->match, pi->len - 1);
> + path = path_alloc;
> + } else
> + path = pi->match;
> +
> + len = strlen(path);
> + if (!len) {
> + res = -1;
> + goto cleanup;
> + }
> +
> + *bloom_keyvec = bloom_keyvec_new(path, len, settings);
> 
> +cleanup:
> + FREE_AND_NULL(path_alloc);
I think we don’t need to NULL path_alloc here, but it doesn’t hurt.
Show 44 quoted lines
> + return res;
> +}
> +
> +static void prepare_to_use_bloom_filter(struct rev_info *revs)
> +{
> if (!revs->commits)
> return;
> 
> @@ -712,22 +735,12 @@ static void prepare_to_use_bloom_filter(struct rev_info *revs)
> 
> revs->bloom_keyvecs_nr = 1;
> CALLOC_ARRAY(revs->bloom_keyvecs, 1);
> - pi = &revs->pruning.pathspec.items[0];
> 
> - /* remove single trailing slash from path, if needed */
> - if (pi->len > 0 && pi->match[pi->len - 1] == '/') {
> - path_alloc = xmemdupz(pi->match, pi->len - 1);
> - path = path_alloc;
> - } else
> - path = pi->match;
> -
> - len = strlen(path);
> - if (!len)
> + if (convert_pathspec_to_filter(&revs->pruning.pathspec.items[0],
> +       &revs->bloom_keyvecs[0],
> +       revs->bloom_filter_settings))
> goto fail;
> 
> - revs->bloom_keyvecs[0] =
> - bloom_keyvec_new(path, len, revs->bloom_filter_settings);
> -
> if (trace2_is_enabled() && !bloom_filter_atexit_registered) {
> atexit(trace2_bloom_filter_statistics_atexit);
> bloom_filter_atexit_registered = 1;
> @@ -737,7 +750,6 @@ static void prepare_to_use_bloom_filter(struct rev_info *revs)
> 
> fail:
> revs->bloom_filter_settings = NULL;
> - free(path_alloc);
> release_revisions_bloom_keyvecs(revs);
> }
> 
> -- 
> 2.47.2.vfs.0.2
This looks perfect to me. I would squash patch 3.5 to patch 3 in v6.

Thanks for your patch, Lidong

Previous: Derrick StoleeNext: Derrick Stolee
Message 44 of 72 in “bloom: use bloom filter given multiple pathspec”
  1. 0/2 bloom: use bloom filter given multiple pathspecLidong Yan, Jun 25, 2025
  2. 1/2 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jun 25, 2025
  3. Junio C HamanoJun 25, 2025
  4. Lidong YanJun 26, 2025
  5. 2/2 bloom: enable multiple pathspec bloom keysLidong Yan, Jun 25, 2025
  6. Junio C HamanoJun 27, 2025
  7. Lidong YanJun 27, 2025
  8. Junio C HamanoJun 27, 2025
  9. Lidong YanJul 1, 2025
  10. Junio C HamanoJul 1, 2025
  11. Lidong YanJul 2, 2025
  12. Junio C HamanoJul 2, 2025
  13. Lidong YanJul 3, 2025
  14. Lidong YanJul 4, 2025
  15. SZEDER GáborJul 1, 2025
  16. Lidong YanJul 1, 2025
  17. Junio C HamanoJul 1, 2025
  18. Junio C HamanoJun 27, 2025
  19. Lidong YanJun 28, 2025
  20. Junio C HamanoJun 25, 2025
  21. Lidong YanJun 26, 2025
  22. Junio C HamanoJun 26, 2025
  23. 0/2 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jun 27, 2025
  24. 0/2 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jun 28, 2025
  25. 0/4 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jul 4, 2025
  26. 1/4 bloom: add test helper to return murmur3 hashLidong Yan, Jul 4, 2025
  27. 2/4 bloom: rename function operates on bloom_keyLidong Yan, Jul 4, 2025
  28. 3/4 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jul 4, 2025
  29. Derrick StoleeJul 7, 2025
  30. Lidong YanJul 7, 2025
  31. 4/4 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jul 4, 2025
  32. Derrick StoleeJul 7, 2025
  33. Lidong YanJul 7, 2025
  34. Junio C HamanoJul 7, 2025
  35. 0/4 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jul 10, 2025
  36. 1/4 bloom: add test helper to return murmur3 hashLidong Yan, Jul 10, 2025
  37. 2/4 bloom: rename function operates on bloom_keyLidong Yan, Jul 10, 2025
  38. 3/4 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jul 10, 2025
  39. Junio C HamanoJul 10, 2025
  40. Lidong YanJul 11, 2025
  41. Junio C HamanoJul 11, 2025
  42. 4/4 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jul 10, 2025
  43. 5/4 revision: make helper for pathspec to bloom keyDerrick Stolee, Jul 10, 2025
  44. Lidong YanJul 10, 2025
  45. 4/4 bloom: optimize multiple pathspec items in revisionDerrick Stolee, Jul 10, 2025
  46. Lidong YanJul 10, 2025
  47. Derrick StoleeJul 10, 2025
  48. 0/5 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jul 12, 2025
  49. 1/5 bloom: add test helper to return murmur3 hashLidong Yan, Jul 12, 2025
  50. 2/5 bloom: rename function operates on bloom_keyLidong Yan, Jul 12, 2025
  51. 3/5 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jul 12, 2025
  52. 4/5 revision: make helper for pathspec to bloom keyvecLidong Yan, Jul 12, 2025
  53. 5/5 To enable optimize multiple pathspec items in revision traversal, return 0 if all pathspec item is literal in forbid_bloom_filters(). Add for loops to initialize and check each pathspec item's bloom_keyvec when optimization is possible.Lidong Yan, Jul 12, 2025
  54. Lidong YanJul 12, 2025
  55. 5/5 bloom: optimize multiple pathspec items in revisionLidong Yan, Jul 12, 2025
  56. Derrick StoleeJul 14, 2025
  57. Junio C HamanoJul 14, 2025
  58. Lidong YanJul 15, 2025
  59. [RESEND][PATCH v6 5/5] bloom: optimize multiple pathspec items in revisionLidong Yan, Jul 15, 2025
  60. Derrick StoleeJul 14, 2025
  61. Junio C HamanoJul 14, 2025
  62. Lidong YanJul 15, 2025
  63. Derrick StoleeJul 15, 2025
  64. Junio C HamanoJul 15, 2025
  65. 1/2 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jun 28, 2025
  66. Patrick SteinhardtJul 2, 2025
  67. Lidong YanJul 2, 2025
  68. Junio C HamanoJul 2, 2025
  69. Lidong YanJul 3, 2025
  70. 2/2 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jun 28, 2025
  71. 1/2 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jun 27, 2025
  72. 2/2 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jun 27, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.