git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 1/2] bloom: replace struct bloom_key * with struct bloom_keyvec

From
Lidong Yan <502024330056@smail.nju.edu.cn>
Date
Jul 2, 2025, 15:49 UTC
Message-ID
<9B2AC9DD-1462-4B6D-B2D5-B2FEEA70B4C3@smail.nju.edu.cn>
In-Reply-To
<aGVLZ9VUf2M1sWhL@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 18 quoted lines
> 
> On Sat, Jun 28, 2025 at 12:21:39PM +0800, Lidong Yan wrote:
>> diff --git a/bloom.h b/bloom.h
>> index 6e46489a20..9e4e832c8c 100644
>> --- a/bloom.h
>> +++ b/bloom.h
>> @@ -74,6 +74,11 @@ struct bloom_key {
>> uint32_t *hashes;
>> };
>> 
>> +struct bloom_keyvec {
>> + size_t count;
>> + struct bloom_key key[FLEX_ARRAY];
>> +};
>> +
> 
> A short comment would help readers understand what the intent of this
> data structure is.
Understood, will add comment for struct bloom_keyvec in v4.
Show 22 quoted lines
> 
>> int load_bloom_filter_from_graph(struct commit_graph *g,
>> struct bloom_filter *filter,
>> uint32_t graph_pos);
>> @@ -100,6 +105,17 @@ void add_key_to_filter(const struct bloom_key *key,
>> void init_bloom_filters(void);
>> void deinit_bloom_filters(void);
>> 
>> +struct bloom_keyvec *create_bloom_keyvec(size_t count);
>> +void destroy_bloom_keyvec(struct bloom_keyvec *vec);
> 
> These functions are named very unusually for us -- the first version of
> this patch series was following our coding guidelines, but this version
> here isn't anymore.
> 
> - The primary data structure that a subsystem 'S' deals with is called
>   `struct S`. Functions that operate on `struct S` are named
>   `S_<verb>()` and should generally receive a pointer to `struct S` as
>   first parameter. E.g.
> 
> Second, the functions should probably be called `*_new()` and `*_free()`
> instead of `create_*()` and `destroy_*()`.

Though I think create and destroy doesn’t match the 'operate on struct S’ definition, I will rename these function to *_verb in v4.

Show 11 quoted lines
> 
>> +static inline void fill_bloom_keyvec_key(const char *data, size_t len,
>> + struct bloom_keyvec *vec, size_t nr,
>> + const struct bloom_filter_settings *settings)
>> +{
>> + assert(nr < vec->count);
>> + fill_bloom_key(data, len, &vec->key[nr], settings);
>> +}
>> +
> 
> Similarly, this should probably be called `bloom_keyvec_fill_key()`.

I initially wanted the new function to have a name similar to fill_bloom_key, but perhaps I should consider renaming it.

Show 38 quoted lines
> 
>> enum bloom_filter_computed {
>> BLOOM_NOT_COMPUTED = (1 << 0),
>> BLOOM_COMPUTED     = (1 << 1),
>> @@ -137,4 +153,8 @@ int bloom_filter_contains(const struct bloom_filter *filter,
>>  const struct bloom_key *key,
>>  const struct bloom_filter_settings *settings);
>> 
>> +int bloom_filter_contains_vec(const struct bloom_filter *filter,
>> +      const struct bloom_keyvec *v,
>> +      const struct bloom_filter_settings *settings);
>> +
>> #endif
> 
> This one looks alright though.
> 
>> diff --git a/revision.c b/revision.c
>> index afee111196..3aa544c137 100644
>> --- a/revision.c
>> +++ b/revision.c
>> @@ -779,11 +782,8 @@ static int check_maybe_different_in_bloom_filter(struct rev_info *revs,
>> return -1;
>> }
>> 
>> - for (j = 0; result && j < revs->bloom_keys_nr; j++) {
>> - result = bloom_filter_contains(filter,
>> -       &revs->bloom_keys[j],
>> -       revs->bloom_filter_settings);
>> - }
>> + result = bloom_filter_contains_vec(filter, revs->bloom_keyvecs[0],
>> +   revs->bloom_filter_settings);
>> 
>> if (result)
>> count_bloom_filter_maybe++;
> 
> This conversion feels wrong to me. Why don't we end up iterating through
> `revs->bloom_keyvecs_nr` here?  We do indeed change it back in the next
> patch to use a for loop.

My original intention was to include all the loop-related logic in [PATCH 2/2]. However, the lack of a loop here does make the code look error-prone. I will add the loop in v4.

Show 13 quoted lines
> 
>> @@ -3230,10 +3230,10 @@ void release_revisions(struct rev_info *revs)
>> line_log_free(revs);
>> oidset_clear(&revs->missing_commits);
>> 
>> - for (int i = 0; i < revs->bloom_keys_nr; i++)
>> - clear_bloom_key(&revs->bloom_keys[i]);
>> - FREE_AND_NULL(revs->bloom_keys);
>> - revs->bloom_keys_nr = 0;
>> + for (int i = 0; i < revs->bloom_keyvecs_nr; i++)
> 
> It's puzzling that the number of keys is declared as `int`. It's not an
> issue introduced by you, but can we maybe fix it while at it?
Yes, of course.

Thanks for your review, Lidong

Previous: Patrick SteinhardtNext: Junio C Hamano
Message 67 of 72 in “bloom: use bloom filter given multiple pathspec”
  1. 0/2 bloom: use bloom filter given multiple pathspecLidong Yan, Jun 25, 2025
  2. 1/2 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jun 25, 2025
  3. Junio C HamanoJun 25, 2025
  4. Lidong YanJun 26, 2025
  5. 2/2 bloom: enable multiple pathspec bloom keysLidong Yan, Jun 25, 2025
  6. Junio C HamanoJun 27, 2025
  7. Lidong YanJun 27, 2025
  8. Junio C HamanoJun 27, 2025
  9. Lidong YanJul 1, 2025
  10. Junio C HamanoJul 1, 2025
  11. Lidong YanJul 2, 2025
  12. Junio C HamanoJul 2, 2025
  13. Lidong YanJul 3, 2025
  14. Lidong YanJul 4, 2025
  15. SZEDER GáborJul 1, 2025
  16. Lidong YanJul 1, 2025
  17. Junio C HamanoJul 1, 2025
  18. Junio C HamanoJun 27, 2025
  19. Lidong YanJun 28, 2025
  20. Junio C HamanoJun 25, 2025
  21. Lidong YanJun 26, 2025
  22. Junio C HamanoJun 26, 2025
  23. 0/2 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jun 27, 2025
  24. 0/2 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jun 28, 2025
  25. 0/4 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jul 4, 2025
  26. 1/4 bloom: add test helper to return murmur3 hashLidong Yan, Jul 4, 2025
  27. 2/4 bloom: rename function operates on bloom_keyLidong Yan, Jul 4, 2025
  28. 3/4 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jul 4, 2025
  29. Derrick StoleeJul 7, 2025
  30. Lidong YanJul 7, 2025
  31. 4/4 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jul 4, 2025
  32. Derrick StoleeJul 7, 2025
  33. Lidong YanJul 7, 2025
  34. Junio C HamanoJul 7, 2025
  35. 0/4 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jul 10, 2025
  36. 1/4 bloom: add test helper to return murmur3 hashLidong Yan, Jul 10, 2025
  37. 2/4 bloom: rename function operates on bloom_keyLidong Yan, Jul 10, 2025
  38. 3/4 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jul 10, 2025
  39. Junio C HamanoJul 10, 2025
  40. Lidong YanJul 11, 2025
  41. Junio C HamanoJul 11, 2025
  42. 4/4 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jul 10, 2025
  43. 5/4 revision: make helper for pathspec to bloom keyDerrick Stolee, Jul 10, 2025
  44. Lidong YanJul 10, 2025
  45. 4/4 bloom: optimize multiple pathspec items in revisionDerrick Stolee, Jul 10, 2025
  46. Lidong YanJul 10, 2025
  47. Derrick StoleeJul 10, 2025
  48. 0/5 bloom: enable bloom filter optimization for multiple pathspec elements in revision traversalLidong Yan, Jul 12, 2025
  49. 1/5 bloom: add test helper to return murmur3 hashLidong Yan, Jul 12, 2025
  50. 2/5 bloom: rename function operates on bloom_keyLidong Yan, Jul 12, 2025
  51. 3/5 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jul 12, 2025
  52. 4/5 revision: make helper for pathspec to bloom keyvecLidong Yan, Jul 12, 2025
  53. 5/5 To enable optimize multiple pathspec items in revision traversal, return 0 if all pathspec item is literal in forbid_bloom_filters(). Add for loops to initialize and check each pathspec item's bloom_keyvec when optimization is possible.Lidong Yan, Jul 12, 2025
  54. Lidong YanJul 12, 2025
  55. 5/5 bloom: optimize multiple pathspec items in revisionLidong Yan, Jul 12, 2025
  56. Derrick StoleeJul 14, 2025
  57. Junio C HamanoJul 14, 2025
  58. Lidong YanJul 15, 2025
  59. [RESEND][PATCH v6 5/5] bloom: optimize multiple pathspec items in revisionLidong Yan, Jul 15, 2025
  60. Derrick StoleeJul 14, 2025
  61. Junio C HamanoJul 14, 2025
  62. Lidong YanJul 15, 2025
  63. Derrick StoleeJul 15, 2025
  64. Junio C HamanoJul 15, 2025
  65. 1/2 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jun 28, 2025
  66. Patrick SteinhardtJul 2, 2025
  67. Lidong YanJul 2, 2025
  68. Junio C HamanoJul 2, 2025
  69. Lidong YanJul 3, 2025
  70. 2/2 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jun 28, 2025
  71. 1/2 bloom: replace struct bloom_key * with struct bloom_keyvecLidong Yan, Jun 27, 2025
  72. 2/2 bloom: optimize multiple pathspec items in revision traversalLidong Yan, Jun 27, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.