git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 3/4] diff: enable and test the sparse index

From
Elijah Newren <newren@gmail.com>
Date
Nov 23, 2021, 07:47 UTC
Message-ID
<CABPp-BEurZH8RaLLB2vvw3aPCWy4Aq1Zs+PkTY9WFboq8OuZ2w@mail.gmail.com>
In-Reply-To
<273ee16b74ed4b36ffba5762fa892410317ea02b.1637620958.git.gitgitgadget@gmail.com>

On Mon, Nov 22, 2021 at 2:42 PM Lessley Dennington via GitGitGadget <gitgitgadget@gmail.com> wrote:

Show 10 quoted lines
>
> From: Lessley Dennington <lessleydennington@gmail.com>
>
> Enable the sparse index within the 'git diff' command. Its implementation
> already safely integrates with the sparse index because it shares code
> with the 'git status' and 'git checkout' commands that were already
> integrated.  For more details see:
>
> d76723e (status: use sparse-index throughout, 2021-07-14)
> 1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29)
I preferred the references in your v3:

d76723ee53 (status: use sparse-index throughout, 2021-07-14) 1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)

because 7-character abbreviations aren't very future proof; 10-character seems better to me.

(Very micro nit.)
Show 117 quoted lines
>
> The most interesting thing to do is to add tests that verify that 'git
> diff' behaves correctly when the sparse index is enabled. These cases are:
>
> 1. The index is not expanded for 'diff' and 'diff --staged'
> 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse
> checkout, and sparse index repositories in the following partially-staged
> scenarios (i.e. the index, HEAD, and working directory differ at a given
> path):
>     1. Path is within sparse-checkout cone
>     2. Path is outside sparse-checkout cone
>     3. A merge conflict exists for paths outside sparse-checkout cone
>
> The `p2000` tests demonstrate a ~44% execution time reduction for 'git
> diff' and a ~86% execution time reduction for 'git diff --staged' using a
> sparse index:
>
> Test                                      before  after
> -------------------------------------------------------------
> 2000.30: git diff (full-v3)               0.33    0.34 +3.0%
> 2000.31: git diff (full-v4)               0.33    0.35 +6.1%
> 2000.32: git diff (sparse-v3)             0.53    0.31 -41.5%
> 2000.33: git diff (sparse-v4)             0.54    0.29 -46.3%
> 2000.34: git diff --cached (full-v3)      0.07    0.07 +0.0%
> 2000.35: git diff --cached (full-v4)      0.07    0.08 +14.3%
> 2000.36: git diff --cached (sparse-v3)    0.28    0.04 -85.7%
> 2000.37: git diff --cached (sparse-v4)    0.23    0.03 -87.0%
>
> Co-authored-by: Derrick Stolee <dstolee@microsoft.com>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> Signed-off-by: Lessley Dennington <lessleydennington@gmail.com>
> ---
>  builtin/diff.c                           |  5 +++
>  t/perf/p2000-sparse-operations.sh        |  2 ++
>  t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++
>  3 files changed, 53 insertions(+)
>
> diff --git a/builtin/diff.c b/builtin/diff.c
> index dd8ce688ba7..fa4683377eb 100644
> --- a/builtin/diff.c
> +++ b/builtin/diff.c
> @@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
>
>         prefix = setup_git_directory_gently(&nongit);
>
> +       if (!nongit) {
> +               prepare_repo_settings(the_repository);
> +               the_repository->settings.command_requires_full_index = 0;
> +       }
> +
>         if (!no_index) {
>                 /*
>                  * Treat git diff with at least one path outside of the
> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh
> index bfd332120c8..5cf94627383 100755
> --- a/t/perf/p2000-sparse-operations.sh
> +++ b/t/perf/p2000-sparse-operations.sh
> @@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -
>  test_perf_on_all git reset
>  test_perf_on_all git reset --hard
>  test_perf_on_all git reset -- does-not-exist
> +test_perf_on_all git diff
> +test_perf_on_all git diff --cached
>
>  test_done
> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
> index 44d5e11c762..53524660759 100755
> --- a/t/t1092-sparse-checkout-compatibility.sh
> +++ b/t/t1092-sparse-checkout-compatibility.sh
> @@ -832,6 +832,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '
>         )
>  '
>
> +test_expect_success 'sparse index is not expanded: diff' '
> +       init_repos &&
> +
> +       write_script edit-contents <<-\EOF &&
> +       echo text >>$1
> +       EOF
> +
> +       # Add file within cone
> +       test_sparse_match git sparse-checkout set deep &&
> +       run_on_all ../edit-contents deep/testfile &&
> +       test_all_match git add deep/testfile &&
> +       run_on_all ../edit-contents deep/testfile &&
> +
> +       test_all_match git diff &&
> +       test_all_match git diff --staged &&
> +       ensure_not_expanded diff &&
> +       ensure_not_expanded diff --staged &&
> +
> +       # Add file outside cone
> +       test_all_match git reset --hard &&
> +       run_on_all mkdir newdirectory &&
> +       run_on_all ../edit-contents newdirectory/testfile &&
> +       test_sparse_match git sparse-checkout set newdirectory &&
> +       test_all_match git add newdirectory/testfile &&
> +       run_on_all ../edit-contents newdirectory/testfile &&
> +       test_sparse_match git sparse-checkout set &&
> +
> +       test_all_match git diff &&
> +       test_all_match git diff --staged &&
> +       ensure_not_expanded diff &&
> +       ensure_not_expanded diff --staged &&
> +
> +       # Merge conflict outside cone
> +       # The sparse checkout will report a warning that is not in the
> +       # full checkout, so we use `run_on_all` instead of
> +       # `test_all_match`
> +       run_on_all git reset --hard &&
> +       test_all_match git checkout merge-left &&
> +       test_all_match test_must_fail git merge merge-right &&
> +
> +       test_all_match git diff &&
> +       test_all_match git diff --staged &&
> +       ensure_not_expanded diff &&
> +       ensure_not_expanded diff --staged

You've changed some of the --staged to --cached, but based on Junio's comments on the previous round, you probably want to convert the others too.

Show 8 quoted lines
> +'
> +
>  # NEEDSWORK: a sparse-checkout behaves differently from a full checkout
>  # in this scenario, but it shouldn't.
>  test_expect_success 'reset mixed and checkout orphan' '
> --
> gitgitgadget
>
Previous: Lessley Dennington via GitGitGadgetNext: Lessley Dennington
Message 40 of 66 in “Sparse Index: diff and blame builtins”
  1. 0/2 Sparse Index: diff and blame builtinsLessley Dennington via GitGitGadget, Oct 14, 2021
  2. 1/2 diff: enable and test the sparse indexLessley Dennington via GitGitGadget, Oct 14, 2021
  3. Derrick StoleeOct 15, 2021
  4. 2/2 blame: enable and test the sparse indexLessley Dennington via GitGitGadget, Oct 14, 2021
  5. Elijah NewrenNov 23, 2021
  6. Lessley DenningtonNov 23, 2021
  7. 0/2 Sparse Index: diff and blame builtinsLessley Dennington via GitGitGadget, Oct 15, 2021
  8. 1/2 diff: enable and test the sparse indexLessley Dennington via GitGitGadget, Oct 15, 2021
  9. Taylor BlauOct 25, 2021
  10. Lessley DenningtonOct 26, 2021
  11. Taylor BlauOct 26, 2021
  12. 2/2 blame: enable and test the sparse indexLessley Dennington via GitGitGadget, Oct 15, 2021
  13. Taylor BlauOct 25, 2021
  14. Lessley DenningtonOct 26, 2021
  15. Elijah NewrenNov 21, 2021
  16. 0/2 Sparse Index: diff and blame builtinsLessley Dennington via GitGitGadget, Nov 1, 2021
  17. 1/2 diff: enable and test the sparse indexLessley Dennington via GitGitGadget, Nov 1, 2021
  18. Junio C HamanoNov 3, 2021
  19. Lessley DenningtonNov 4, 2021
  20. 2/2 blame: enable and test the sparse indexLessley Dennington via GitGitGadget, Nov 1, 2021
  21. Junio C HamanoNov 3, 2021
  22. Lessley DenningtonNov 5, 2021
  23. Elijah NewrenNov 21, 2021
  24. 0/4 Sparse Index: diff and blame builtinsLessley Dennington via GitGitGadget, Nov 22, 2021
  25. 1/4 sparse index: enable only for git reposLessley Dennington via GitGitGadget, Nov 22, 2021
  26. Elijah NewrenNov 23, 2021
  27. Lessley DenningtonNov 23, 2021
  28. Junio C HamanoNov 23, 2021
  29. Lessley DenningtonNov 24, 2021
  30. Junio C HamanoNov 24, 2021
  31. Lessley DenningtonNov 29, 2021
  32. Junio C HamanoNov 30, 2021
  33. Lessley DenningtonNov 30, 2021
  34. 2/4 test-read-cache: set up repo after git directoryLessley Dennington via GitGitGadget, Nov 22, 2021
  35. Junio C HamanoNov 23, 2021
  36. Lessley DenningtonNov 24, 2021
  37. Junio C HamanoNov 24, 2021
  38. Lessley DenningtonNov 29, 2021
  39. 3/4 diff: enable and test the sparse indexLessley Dennington via GitGitGadget, Nov 22, 2021
  40. Elijah NewrenNov 23, 2021
  41. Lessley DenningtonNov 23, 2021
  42. Junio C HamanoNov 23, 2021
  43. 4/4 blame: enable and test the sparse indexLessley Dennington via GitGitGadget, Nov 22, 2021
  44. Junio C HamanoNov 23, 2021
  45. Lessley DenningtonNov 24, 2021
  46. 0/7 Sparse Index: diff and blame builtinsLessley Dennington via GitGitGadget, Dec 3, 2021
  47. 1/7 git: esnure correct git directory setup with -hLessley Dennington via GitGitGadget, Dec 3, 2021
  48. Elijah NewrenDec 4, 2021
  49. Junio C HamanoDec 4, 2021
  50. 2/7 commit-graph: return if there is no git directoryLessley Dennington via GitGitGadget, Dec 3, 2021
  51. 4/7 repo-settings: prepare_repo_settings only in git reposLessley Dennington via GitGitGadget, Dec 3, 2021
  52. Ævar Arnfjörð BjarmasonDec 7, 2021
  53. Lessley DenningtonDec 8, 2021
  54. 3/7 test-read-cache: set up repo after git directoryLessley Dennington via GitGitGadget, Dec 3, 2021
  55. 5/7 diff: replace --staged with --cached in t1092 testsLessley Dennington via GitGitGadget, Dec 3, 2021
  56. 6/7 diff: enable and test the sparse indexLessley Dennington via GitGitGadget, Dec 3, 2021
  57. 7/7 blame: enable and test the sparse indexLessley Dennington via GitGitGadget, Dec 3, 2021
  58. Elijah NewrenDec 4, 2021
  59. 0/7 Sparse Index: diff and blame builtinsLessley Dennington via GitGitGadget, Dec 6, 2021
  60. 1/7 git: ensure correct git directory setup with -hLessley Dennington via GitGitGadget, Dec 6, 2021
  61. 2/7 commit-graph: return if there is no git directoryLessley Dennington via GitGitGadget, Dec 6, 2021
  62. 3/7 test-read-cache: set up repo after git directoryLessley Dennington via GitGitGadget, Dec 6, 2021
  63. 4/7 repo-settings: prepare_repo_settings only in git reposLessley Dennington via GitGitGadget, Dec 6, 2021
  64. 5/7 diff: replace --staged with --cached in t1092 testsLessley Dennington via GitGitGadget, Dec 6, 2021
  65. 6/7 diff: enable and test the sparse indexLessley Dennington via GitGitGadget, Dec 6, 2021
  66. 7/7 blame: enable and test the sparse indexLessley Dennington via GitGitGadget, Dec 6, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.