git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [GSoC][PATCH v2] Optimize ewah_bitmap.c for efficiency using trailing zeros for set bit iteration

From
Junio C Hamano <gitster@pobox.com>
Date
Mar 18, 2024, 15:42 UTC
Message-ID
<xmqqr0g7k0lj.fsf@gitster.g>
In-Reply-To
<CAOLa=ZSijjnfa8UEEWPi8Cb7Q1hLbgrez7VX-LxshmUhNM29DQ@mail.gmail.com>
Karthik Nayak <karthik.188@gmail.com> writes:
Show 19 quoted lines
> Aryan Gupta <garyan447@gmail.com> writes:
>
> Hello,
>
>> Signed-off-by: Aryan Gupta <garyan447@gmail.com>
>> ---
>>
>> Thank you Vicent for the guidance. I am still not sure how
>> to do the performance measurement for this improvement. Any
>> guidance would be appreciated.
>>
>
> I guess there is some off-list discussion here. That along with the fact
> that the commit message is missing makes it really hard to understand
> how this is better than what was here already.
>
> The guidelines ('Documentation/SubmittingPatches') also state how to
> draft the commit message. This patch only seems to have a title, it is
> recommend to add a description as to why this change is being made.
Yes.
Show 29 quoted lines
>> diff --git a/ewah/ewah_bitmap.c b/ewah/ewah_bitmap.c
>> index 8785cbc54a..1a75f50682 100644
>> --- a/ewah/ewah_bitmap.c
>> +++ b/ewah/ewah_bitmap.c
>> @@ -257,12 +257,15 @@ void ewah_each_bit(struct ewah_bitmap *self, void (*callback)(size_t, void*), vo
>>  		for (k = 0; k < rlw_get_literal_words(word); ++k) {
>>  			int c;
>>
>> -			/* todo: zero count optimization */
>> -			for (c = 0; c < BITS_IN_EWORD; ++c, ++pos) {
>> -				if ((self->buffer[pointer] & ((eword_t)1 << c)) != 0)
>> -					callback(pos, payload);
>> +			eword_t bitset = self->buffer[pointer];
>> +			while(bitset != 0) {
>> +				eword_t t = bitset & -bitset;
>> +				int r = __builtin_ctzl(bitset);
>> +				bitset ^= t;
>> +				callback(pos+r, payload);
>>  			}
>> -
>> +			
>> +			pos += BITS_IN_EWORD;
>>  			++pointer;
>>  		}
>>  	}
>
> The bit manipulation done here is slightly hard to comprehend, it would
> be nice if you could also add some comments as to what is being done
> here and why.

In addition, this patch assumes that __builtin_ctzl() function is always available no matter what environment the code is built on, which I am not sure is a safe. Quite honestory, I suspect that the whole of "todo" is to seamlessly detect the presense of the builtin support to count the top zero bit, use it only when it is there, and giving a fallback implementation when it does not exist. The code itself to use the builtin is only 20% of that effort ;-)

And of course, there is benchmark. To show how much better performance gets for people with that function, and more importantly to show that the performance does not degrade for those who are without.

Thanks.
Previous: Karthik Nayak
Message 8 of 8 in “add zero count optimization in ewah_bitmap.c”
  1. Aryan GuptaMar 10, 2024
  2. [GSoC][PATCH 1/1] add zero count optimization in ewah_bitmap.cAryan Gupta, Mar 10, 2024
  3. Junio C HamanoMar 11, 2024
  4. Junio C HamanoMar 11, 2024
  5. Aryan GuptaMar 11, 2024
  6. [GSoC][PATCH v2] Optimize ewah_bitmap.c for efficiency using trailing zeros for set bit iterationAryan Gupta, Mar 13, 2024
  7. Karthik NayakMar 18, 2024
  8. Junio C HamanoMar 18, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.