[PATCH] preload-index: avoid lstat for skip-worktree items

Johannes Schindelin· Feb 10, 2017, 15:10 UTC · lore

From: Jeff Hostetler <jeffhost@microsoft.com>

Teach preload-index to avoid lstat() calls for index-entries with skip-worktree bit set. This is a performance optimization.

During a sparse-checkout, the skip-worktree bit is set on items that were not populated and therefore are not present in the worktree. The per-thread preload-index loop performs a series of tests on each index-entry as it attempts to compare the worktree version with the index and mark them up-to-date. This patch short-cuts that work.

On a Windows 10 system with a very large repo (450MB index) and various levels of sparseness, performance was improved in the {preloadindex=true, fscache=false} case by 80% and in the {preloadindex=true, fscache=true} case by 20% for various commands.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Published-As: https://github.com/dscho/git/releases/tag/preload-index-sparse-v1
Fetch-It-Via: git fetch https://github.com/dscho/git preload-index-sparse-v1

 preload-index.c | 2 ++
 1 file changed, 2 insertions(+)

Show changes to preload-index.c +2 −0

diff --git a/preload-index.c b/preload-index.c
index c1fe3a3ef9c..70a4c808783 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -53,6 +53,8 @@ static void *preload_thread(void *_data)
 			continue;
 		if (ce_uptodate(ce))
 			continue;
+		if (ce_skip_worktree(ce))
+			continue;
 		if (!ce_path_match(ce, &p->pathspec, NULL))
 			continue;
 		if (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))

base-commit: 6e3a7b3398559305c7a239a42e447c21a8f39ff8

-- 
2.11.1.windows.1

Junio C Hamano· Feb 10, 2017, 22:43 UTC · re: Johannes Schindelin · lore

Re: [PATCH] preload-index: avoid lstat for skip-worktree items

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

Show 16 quoted lines

> Teach preload-index to avoid lstat() calls for index-entries
> with skip-worktree bit set.  This is a performance optimization.
> ...
> diff --git a/preload-index.c b/preload-index.c
> index c1fe3a3ef9c..70a4c808783 100644
> --- a/preload-index.c
> +++ b/preload-index.c
> @@ -53,6 +53,8 @@ static void *preload_thread(void *_data)
>  			continue;
>  		if (ce_uptodate(ce))
>  			continue;
> +		if (ce_skip_worktree(ce))
> +			continue;
>  		if (!ce_path_match(ce, &p->pathspec, NULL))
>  			continue;
>  		if (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))

Because we are only interested in marking the ones that match between the index and the working tree as "up-to-date", and we are not doing the opposite (i.e. toggle "up-to-date" bit off by noticing that things are now different) in this codepath, this change does make sense. The ones marked as "skip", even if there were an unrelated file or directory at the path where the index expects a regular file, can be safely ignored.

Thanks.

patchpreload-index: avoid lstat for skip-worktree items