Re: [PATCH 0/5] sparse-index: improve clear_skip_worktree_from_present_files()

From: Junio C Hamano <gitster@pobox.com>
Date: Jun 20, 2024, 19:16 UTC
Message-ID: <xmqqiky3tofw.fsf@gitster.g>
In-Reply-To: <pull.1754.git.1718899877.gitgitgadget@gmail.com>

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

Show 13 quoted lines

> While doing some investigation in a private monorepo with sparse-checkout
> and a sparse index, I accidentally left a modified file outside of my
> sparse-checkout cone. This caused my Git commands to slow to a crawl, so I
> reran with GIT_TRACE2_PERF=1.
>
> While I was able to identify clear_skip_worktree_from_present_files() as the
> culprit, it took longer than desired to figure out what was going on. This
> series intends to both fix the performance issue (as much as possible) and
> do some refactoring to make it easier to understand what is happening.
>
> In the end, I was able to reduce the number of lstat() calls in my case from
> over 170,000 to about 6,500, improving the time from 2.5s to 71ms on a warm
> disk cache.   Thanks, Stolee

That's impressive but I cannot offhand tell how big 170k (or 6.5k for that matter) is relative to the size of the tree. How many paths are there in the entire tree (i.e. "git ls-tree -r HEAD | wc -l") vs the number of the in-cone paths in the working tree?

If 6.5k is in the same ballpark as the latter, it would be really good.

Thanks.