git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] pack-write: use hashwrite_be32() instead of double-buffering array

From
Junio C Hamano <gitster@pobox.com>
Date
Nov 4, 2020, 16:23 UTC
Message-ID
<xmqqmtzxnlad.fsf@gitster.c.googlers.com>
In-Reply-To
<20201104133653.GC3030480@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
Show 27 quoted lines
> On Sun, Nov 01, 2020 at 09:52:12AM +0100, René Scharfe wrote:
>
>> hashwrite() already buffers writes, so pass the fanout table entries
>> individually via hashwrite_be32(), which also does the endianess
>> conversion for us.  This avoids a memory copy, shortens the code and
>> reduces the number of magic numbers.
>
> Yep, this seems trivially correct. The key observation is that we are
> filling the array in order:
>
>> @@ -101,20 +100,19 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
>>  	for (i = 0; i < 256; i++) {
>>  		struct pack_idx_entry **next = list;
>>  		while (next < last) {
>>  			struct pack_idx_entry *obj = *next;
>>  			if (obj->oid.hash[0] != i)
>>  				break;
>>  			next++;
>>  		}
>> -		array[i] = htonl(next - sorted_by_sha);
>> +		hashwrite_be32(f, next - sorted_by_sha);
>>  		list = next;
>>  	}
>> -	hashwrite(f, array, 256 * 4);
>
> Perhaps obvious, but I got bit trying to do another similar conversion
> recently that was filling in the array out-of-order...

Yeah, filling an array out of order and then writing the result in order would obviously different from writing out individual pieces in the order the original codeflow used to fill the array. Worse, the order the data items are fed to hashwrite() obviously affects the hash computed at the end. An example of too much abstraction biting us? ;-)

Previous: Jeff King
Message 3 of 3 in “pack-write: use hashwrite_be32() instead of double-buffering array”
  1. pack-write: use hashwrite_be32() instead of double-buffering arrayRené Scharfe, Nov 1, 2020
  2. Jeff KingNov 4, 2020
  3. Junio C HamanoNov 4, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.