git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v10 3/3] read-cache: speed up add_index_entry during checkout

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Apr 18, 2017, 16:19 UTC
Message-ID
<42739319-599f-b761-bfaa-9563e427ae8f@jeffhostetler.com>
In-Reply-To
<28ef82b9-ab7a-ccaf-7bae-4eaa332725f8@jeffhostetler.com>
On 4/17/2017 10:53 AM, Jeff Hostetler wrote:
Show 13 quoted lines
>
>
> On 4/15/2017 1:55 PM, René Scharfe wrote:
>> Am 14.04.2017 um 21:12 schrieb git@jeffhostetler.com:
>>> From: Jeff Hostetler <jeffhost@microsoft.com>
>>
>> Very nice, especially the perf test!  But can we unbundle the different
>> optimizations into separate patches with their own performance numbers?
>> Candidates IMHO: The change in add_index_entry_with_check(), the first
>> hunk in has_dir_name(), the loop shortcuts.  That might also help find
>> the reason for the slight slowdown of 0006.3 for the kernel repository.
>
> Let me take a look at this and see if it helps.

Last night I pushed up version 11 which has the 3 parts of read-cache.c in 3 commits (but still in the same patch series). This should allow for more experimentation.

The add_index_entry_with_check() shows a gain. For the operations in p0006 on linux.git, the short-cut was being taken 57993 of 57994 times.

The top of has_dir_name() -- by itself -- does not, but the short-cut only triggers when the paths have no prefix in common -- which only happens when the top-level directory changes. On linux.git, this was 19 of 57993. However, it does set us up for the next part.

The 3 loop body short-cuts hit 54372, 3509, and 86 (sum
57967) times.  So in p0006, the search was only attempted
7 times (57993 - 19 - 57967) most of the time.

WRT the slowdown of 0006.3 on linux.git, I suspect this is I/O noise. In the commit message for part 2 in V11, I show 2 runs on linux.git that show wide variance in the 0006.3 times. And given the nature of that test, the speed up in the lookups is completely hidden by the I/O of the full checkouts. When I step up to a repo with 4M files, the results are very clear.

https://public-inbox.org/git/20170417213734.55373-6-git@jeffhostetler.com/
Previous: Jeff Hostetler
Message 7 of 7 in “read-cache: speed up add_index_entry”
  1. 0/3 read-cache: speed up add_index_entrygit@jeffhostetler.com, Apr 14, 2017
  2. 2/3 p0006-read-tree-checkout: perf test to time read-treegit@jeffhostetler.com, Apr 14, 2017
  3. 1/3 read-cache: add strcmp_offset functiongit@jeffhostetler.com, Apr 14, 2017
  4. 3/3 read-cache: speed up add_index_entry during checkoutgit@jeffhostetler.com, Apr 14, 2017
  5. René ScharfeApr 15, 2017
  6. Jeff HostetlerApr 17, 2017
  7. Jeff HostetlerApr 18, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.