Re: [PATCH v5 2/2] diff-files: integrate with sparse index

From: Victoria Dye <vdye@github.com>
Date: Mar 10, 2023, 18:23 UTC
Message-ID: <c5a32703-29a7-4e9f-f669-6a017ffdce60@github.com>
In-Reply-To: <20230310050021.123769-3-cheskaqiqi@gmail.com>

Shuqi Liang wrote:

Show 17 quoted lines

> Remove full index requirement for `git diff-files`
> and test to ensure the index is not expanded in `git diff-files`.
> 
> The `p2000` tests demonstrate a ~96% execution time reduction for 'git
> diff-files' and a ~97% execution time reduction for 'git diff-files'
> for a file using a sparse index:
> 
> Test                                           before  after
> -----------------------------------------------------------------
> 2000.78: git diff-files (full-v3)              0.09    0.08 -11.1%
> 2000.79: git diff-files (full-v4)              0.09    0.09 +0.0%
> 2000.80: git diff-files (sparse-v3)            0.52    0.02 -96.2%
> 2000.81: git diff-files (sparse-v4)            0.51    0.02 -96.1%
> 2000.82: git diff-files f2/f4/a (full-v3)      0.06    0.07 +16.7%
> 2000.83: git diff-files f2/f4/a (full-v4)      0.08    0.08 +0.0%
> 2000.84: git diff-files f2/f4/a (sparse-v3)    0.46    0.01 -97.8%
> 2000.85: git diff-files f2/f4/a (sparse-v4)    0.51    0.02 -96.1%

These are great performance results!

Show 19 quoted lines

> 
> Signed-off-by: Shuqi Liang <cheskaqiqi@gmail.com>
> ---
>  builtin/diff-files.c                     |  4 ++++
>  t/perf/p2000-sparse-operations.sh        |  2 ++
>  t/t1092-sparse-checkout-compatibility.sh | 13 +++++++++++++
>  3 files changed, 19 insertions(+)
> 
> diff --git a/builtin/diff-files.c b/builtin/diff-files.c
> index dc991f753b..360464e6ef 100644
> --- a/builtin/diff-files.c
> +++ b/builtin/diff-files.c
> @@ -27,6 +27,10 @@ int cmd_diff_files(int argc, const char **argv, const char *prefix)
>  		usage(diff_files_usage);
>  
>  	git_config(git_diff_basic_config, NULL); /* no "diff" UI options */
> +
> +	prepare_repo_settings(the_repository);
> +	the_repository->settings.command_requires_full_index = 0;

Looks good.

Show 35 quoted lines

> +
>  	repo_init_revisions(the_repository, &rev, prefix);
>  	rev.abbrev = 0;
>  
> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh
> index 3242cfe91a..82751f2ca3 100755
> --- a/t/perf/p2000-sparse-operations.sh
> +++ b/t/perf/p2000-sparse-operations.sh
> @@ -125,5 +125,7 @@ test_perf_on_all git checkout-index -f --all
>  test_perf_on_all git update-index --add --remove $SPARSE_CONE/a
>  test_perf_on_all "git rm -f $SPARSE_CONE/a && git checkout HEAD -- $SPARSE_CONE/a"
>  test_perf_on_all git grep --cached --sparse bogus -- "f2/f1/f1/*"
> +test_perf_on_all git diff-files
> +test_perf_on_all git diff-files $SPARSE_CONE/a
>  
>  test_done
> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
> index 9b71d7f5f9..4f582164a3 100755
> --- a/t/t1092-sparse-checkout-compatibility.sh
> +++ b/t/t1092-sparse-checkout-compatibility.sh
> @@ -2103,4 +2103,17 @@ test_expect_success 'diff-files with pathspec outside sparse definition' '
>  	test_cmp expect sparse-checkout-out
>  '
>  
> +test_expect_success 'sparse index is not expanded: diff-files' '
> +	init_repos &&
> +
> +	write_script edit-contents <<-\EOF &&
> +	echo text >>"$1"
> +	EOF
> +
> +	run_on_all ../edit-contents deep/a &&
> +
> +	ensure_not_expanded diff-files  &&
> +	ensure_not_expanded diff-files deep/a

IIRC, in many cases, the internal diff machinery won't expand a sparse index even if the pathspec matches files outside the sparse-checkout definition. Does 'ensure_not_expanded diff-files folder1/a' work? What about 'ensure_not_expanded diff-files "*a"'?

> +'
> +
>  test_done