git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 0/7] thread lazy_init_name_hash

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Apr 6, 2017, 13:53 UTC
Message-ID
<c89de108-916e-49a1-934c-0fb5d8901f3a@jeffhostetler.com>
In-Reply-To
<CACsJy8CCb+i3xjtEY=6_dWCGw8VJHT1Mo5973YmQ-6xpTd=z=g@mail.gmail.com>
On 4/5/2017 10:22 PM, Duy Nguyen wrote:
Show 17 quoted lines
> On Thu, Mar 23, 2017 at 8:46 PM,  <git@jeffhostetler.com> wrote:
>> This patch series is a performance optimization for
>> lazy_init_name_hash() in name-hash.c on very large
>> repositories.
>>
>> This change allows lazy_init_name_hash() to optionally
>> use multiple threads when building the the_index.dir_hash
>> and the_index.name_hash hashmaps.  The original code path
>> has been preserved and is used when the repo is small or
>> the system does not have sufficient CPUs.
>
> If sha1 verification in the index file can now be optionally skipped,
> I wonder if you would have faster startup time by storing hashes in
> the index as an extension. I have never tried it (though I planned to
> have some sort of caching for this) but I would guess loading hashes
> would cost less than 0.27 seconds, hopefully closer to 0.05 seconds.
>

I've thought about doing that, but haven't gotten around to actually trying it yet. It's on my TODO list though.

Thanks Jeff

Previous: Duy Nguyen
Message 20 of 20 in “thread lazy_init_name_hash”
  1. 0/7 thread lazy_init_name_hashgit@jeffhostetler.com, Mar 23, 2017
  2. 2/7 hashmap: allow memihash computation to be continuedgit@jeffhostetler.com, Mar 23, 2017
  3. 6/7 name-hash: add test-lazy-init-name-hashgit@jeffhostetler.com, Mar 23, 2017
  4. Ramsay JonesMar 23, 2017
  5. 1/7 name-hash: specify initial size for istate.dir_hash tablegit@jeffhostetler.com, Mar 23, 2017
  6. 7/7 name-hash: add perf test for lazy_init_name_hashgit@jeffhostetler.com, Mar 23, 2017
  7. 5/7 name-hash: perf improvement for lazy_init_name_hashgit@jeffhostetler.com, Mar 23, 2017
  8. Ramsay JonesMar 23, 2017
  9. Stefan BellerMar 23, 2017
  10. Jeff HostetlerMar 24, 2017
  11. Junio C HamanoMar 27, 2017
  12. Jeff HostetlerMar 27, 2017
  13. 3/7 hashmap: Add disallow_rehash settinggit@jeffhostetler.com, Mar 23, 2017
  14. 4/7 hashmap: document memihash_cont, hashmap_disallow_rehash apigit@jeffhostetler.com, Mar 23, 2017
  15. Junio C HamanoMar 23, 2017
  16. Jeff HostetlerMar 24, 2017
  17. Stefan BellerMar 24, 2017
  18. Junio C HamanoMar 24, 2017
  19. Duy NguyenApr 6, 2017
  20. Jeff HostetlerApr 6, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.