{"thread":{"id":"56710","subject":"[PATCH 0/2] Sparse Index: diff and blame builtins","startedAt":"2021-10-14T17:26:00Z","lastAt":"2021-12-08T15:46:26Z","messageCount":66,"participants":["Lessley Dennington via GitGitGadget","Derrick Stolee","Taylor Blau","Lessley Dennington","Junio C Hamano","Elijah Newren","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"438770","messageId":"pull.1050.git.1634232352.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":null,"subject":"[PATCH 0/2] Sparse Index: diff and blame builtins","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-10-14T17:25:50Z","receivedAt":"2021-10-14T17:26:00Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"This series is based on vd/sparse-reset. It integrates the sparse index with\ngit diff and git blame and includes:\n\n 1. tests added to t1092 and p2000 to establish the baseline functionality\n of the commands\n 2. repository settings to enable the sparse index\n\nThe p2000 tests demonstrate a ~30% execution time reduction for 'git diff'\nand a ~75% execution time reduction for 'git diff --staged' using a sparse\nindex. For 'git blame', the reduction time was ~60% for a file two levels\ndeep and ~30% for a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\n\nThanks, Lessley\n\nLessley Dennington (2):\n diff: enable and test the sparse index\n blame: enable and test the sparse index\n\n builtin/blame.c | 3 ++\n builtin/diff.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 4 ++\n t/t1092-sparse-checkout-compatibility.sh | 66 +++++++++++++++++++++---\n 4 files changed, 69 insertions(+), 7 deletions(-)\n\n\nbase-commit: 57049e844c80d5fe1394e6d65b28705eabfd6585\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1050%2Fldennington%2Fdiff-blame-sparse-index-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1050/ldennington/diff-blame-sparse-index-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1050\n-- \ngitgitgadget\n"},{"id":"438771","messageId":"9a597233cf4127e97ac9f680f540c7c9a9304211.1634232353.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.git.1634232352.gitgitgadget@gmail.com","subject":"[PATCH 1/2] diff: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-10-14T17:25:51Z","receivedAt":"2021-10-14T17:26:01Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index within the 'git diff' command. Its implementation\nalready safely integrates with the sparse index because it shares code with\nthe 'git status' and 'git checkout' commands that were already integrated.\nThe most interesting thing to do is to add tests that verify that 'git diff'\nbehaves correctly when the sparse index is enabled. These cases are:\n\n1. The index is not expanded for 'diff' and 'diff --staged'\n2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\ncheckout, and sparse index repositories in the following partially-staged\nscenarios (i.e. the index, HEAD, and working directory differ at a given\npath):\n 1. Path is within sparse-checkout cone\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n\nThe `p2000` tests demonstrate a ~30% execution time reduction for 'git\ndiff' and a ~75% execution time reduction for 'git diff --staged' using a\nsparse index:\n\nTest before after\n-------------------------------------------------------------\n2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n\nCo-authored-by: Derrick Stolee \nSigned-off-by: Derrick Stolee \nSigned-off-by: Lessley Dennington \n---\n builtin/diff.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 42 ++++++++++++++++++++++++\n 3 files changed, 47 insertions(+)\n\ndiff --git a/builtin/diff.c b/builtin/diff.c\nindex dd8ce688ba7..cbf7b51c7c0 100644\n--- a/builtin/diff.c\n+++ b/builtin/diff.c\n@@ -437,6 +437,9 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tif (!no_index) {\n \t\t/*\n \t\t * Treat git diff with at least one path outside of the\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bfd332120c8..bff93f16e93 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n+test_perf_on_all git diff\n+test_perf_on_all git diff --staged\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex f19c1b3e2eb..1070bff1a83 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -386,6 +386,43 @@ test_expect_success 'diff --staged' '\n \ttest_all_match git diff --staged\n '\n \n+test_expect_success 'diff partially-staged' '\n+\tinit_repos &&\n+\n+\twrite_script edit-contents <<-\\EOF &&\n+\techo text >>$1\n+\tEOF\n+\n+\t# Add file within cone\n+\ttest_all_match git sparse-checkout set deep &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\ttest_all_match git add deep/testfile &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\n+\t# Add file outside cone\n+\ttest_all_match git reset --hard &&\n+\trun_on_all mkdir newdirectory &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_all_match git sparse-checkout set newdirectory &&\n+\ttest_all_match git add newdirectory/testfile &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_all_match git sparse-checkout set &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\n+\t# Merge conflict outside cone\n+\ttest_all_match git reset --hard &&\n+\ttest_all_match git checkout merge-left &&\n+\ttest_all_match test_must_fail git merge merge-right &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged\n+'\n+\n # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n # running this test with 'df-conflict-2' after 'df-conflict-1'.\n test_expect_success 'diff with renames and conflicts' '\n@@ -800,6 +837,11 @@ test_expect_success 'sparse-index is not expanded' '\n \t# Wildcard identifies only full sparse directories, no index expansion\n \tensure_not_expanded reset deepest -- folder\\* &&\n \n+\techo a test change >>sparse-index/README.md &&\n+\tensure_not_expanded diff &&\n+\tgit -C sparse-index add README.md &&\n+\tensure_not_expanded diff --staged &&\n+\n \tensure_not_expanded checkout -f update-deep &&\n \ttest_config -C sparse-index pull.twohead ort &&\n \t(\n-- \ngitgitgadget\n\n"},{"id":"438772","messageId":"ddcee003c9292d1b4cdd64a97f9e1731a72faee8.1634232353.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.git.1634232352.gitgitgadget@gmail.com","subject":"[PATCH 2/2] blame: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-10-14T17:25:52Z","receivedAt":"2021-10-14T17:26:03Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index for the 'git blame' command. The index was already\nnot expanded with this command, so the most interesting thing to do is to\nadd tests that verify that 'git blame' behaves correctly when the sparse\nindex is enabled and that its performance improves. More specifically, these\ncases are:\n\n1. The index is not expanded for 'blame' when given paths in the sparse\ncheckout cone at multiple levels.\n\n2. Performance measurably improves for 'blame' with sparse index when given\npaths in the sparse checkout cone at multiple levels.\n\nThe `p2000` tests demonstrate a ~60% execution time reduction when running\n'blame' for a file two levels deep and and a ~30% execution time reduction\nfor a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nWe do not include paths outside the sparse checkout cone because blame\ncurrently does not support blaming files outside of the sparse definition.\nAttempting to do so fails with the following error:\n\nfatal: no such path '' in HEAD\n\nSigned-off-by: Lessley Dennington \n---\n builtin/blame.c | 3 +++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 24 +++++++++++++++++-------\n 3 files changed, 22 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 641523ff9af..af3d81e2bd4 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \tlong anchor;\n \tconst int hexsz = the_hash_algo->hexsz;\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tsetup_default_color_by_age();\n \tgit_config(git_blame_config, &output_option);\n \trepo_init_revisions(the_repository, &revs, NULL);\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bff93f16e93..9ac76a049b8 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n test_perf_on_all git diff --staged\n+test_perf_on_all git blame $SPARSE_CONE/a\n+test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 1070bff1a83..54826e858a9 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -485,15 +485,16 @@ test_expect_success 'blame with pathspec inside sparse definition' '\n \ttest_all_match git blame deep/deeper1/deepest/a\n '\n \n-# TODO: blame currently does not support blaming files outside of the\n-# sparse definition. It complains that the file doesn't exist locally.\n-test_expect_failure 'blame with pathspec outside sparse definition' '\n+# Blame does not support blaming files outside of the sparse\n+# definition, so we verify this scenario.\n+test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n \n-\ttest_all_match git blame folder1/a &&\n-\ttest_all_match git blame folder2/a &&\n-\ttest_all_match git blame deep/deeper2/a &&\n-\ttest_all_match git blame deep/deeper2/deepest/a\n+\ttest_sparse_match git sparse-checkout set &&\n+\ttest_sparse_match test_must_fail git blame folder1/a &&\n+\ttest_sparse_match test_must_fail git blame folder2/a &&\n+\ttest_sparse_match test_must_fail git blame deep/deeper2/a &&\n+\ttest_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n@@ -871,6 +872,15 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n \t)\n '\n \n+test_expect_success 'sparse index is not expanded: blame' '\n+\tinit_repos &&\n+\n+\tensure_not_expanded blame a &&\n+\tensure_not_expanded blame deep/a &&\n+\tensure_not_expanded blame deep/deeper1/a &&\n+\tensure_not_expanded blame deep/deeper1/deepest/a\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n"},{"id":"438875","messageId":"a8e7c755-6ee0-cec7-9721-0e244e561542@gmail.com","threadId":"56710","inReplyTo":"9a597233cf4127e97ac9f680f540c7c9a9304211.1634232353.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/2] diff: enable and test the sparse index","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2021-10-15T16:46:01Z","receivedAt":"2021-10-15T16:46:08Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 10/14/2021 1:25 PM, Lessley Dennington via GitGitGadget wrote:\n\nThere is a failure in 'seen', and it's due to a subtle reason\nthat we didn't catch in gitgitgadget PR builds. It's because\nds/add-rm-with-sparse-index wasn't in your history until it was\nmerged into 'seen'.\n\n> +test_expect_success 'diff partially-staged' '\n> +\tinit_repos &&\n> +\n> +\twrite_script edit-contents <<-\\EOF &&\n> +\techo text >>$1\n> +\tEOF\n> +\n> +\t# Add file within cone\n> +\ttest_all_match git sparse-checkout set deep &&\n\nThe root cause is that you should use \"test_sparse_match\" when\nadjusting the sparse-checkout definition. The full-checkout repo\nis getting the sparse-checkout set to a single pattern \"deep\",\nbut without cone mode.\n\n> +\trun_on_all ../edit-contents deep/testfile &&\n> +\ttest_all_match git add deep/testfile &&\n\nBut the test fails here because \"deep/testfile\" doesn't match\nthe sparse-checkout definition in the full-checkout repo.\n\n> +\trun_on_all ../edit-contents deep/testfile &&\n> +\n> +\ttest_all_match git diff &&\n> +\ttest_all_match git diff --staged &&\n> +\n> +\t# Add file outside cone\n> +\ttest_all_match git reset --hard &&\n> +\trun_on_all mkdir newdirectory &&\n> +\trun_on_all ../edit-contents newdirectory/testfile &&\n> +\ttest_all_match git sparse-checkout set newdirectory &&\n\nYou'll want to change this line, too.\n\nThanks,\n-Stolee\n"},{"id":"438911","messageId":"pull.1050.v2.git.1634332835.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.git.1634232352.gitgitgadget@gmail.com","subject":"[PATCH v2 0/2] Sparse Index: diff and blame builtins","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-10-15T21:20:33Z","receivedAt":"2021-10-15T21:20:41Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"This series is based on vd/sparse-reset. It integrates the sparse index with\ngit diff and git blame and includes:\n\n 1. tests added to t1092 and p2000 to establish the baseline functionality\n of the commands\n 2. repository settings to enable the sparse index\n\nThe p2000 tests demonstrate a ~30% execution time reduction for 'git diff'\nand a ~75% execution time reduction for 'git diff --staged' using a sparse\nindex. For 'git blame', the reduction time was ~60% for a file two levels\ndeep and ~30% for a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\n\n\nChanges since V1\n================\n\n * Fix failing diff partially-staged test in\n t1092-sparse-checkout-compatibility.sh, which was breaking in seen.\n\nThanks, Lessley\n\nLessley Dennington (2):\n diff: enable and test the sparse index\n blame: enable and test the sparse index\n\n builtin/blame.c | 3 ++\n builtin/diff.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 4 ++\n t/t1092-sparse-checkout-compatibility.sh | 69 +++++++++++++++++++++---\n 4 files changed, 72 insertions(+), 7 deletions(-)\n\n\nbase-commit: 57049e844c80d5fe1394e6d65b28705eabfd6585\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1050%2Fldennington%2Fdiff-blame-sparse-index-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1050/ldennington/diff-blame-sparse-index-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1050\n\nRange-diff vs v1:\n\n 1: 9a597233cf4 ! 1: ac33159d020 diff: enable and test the sparse index\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'diff --staged' '\n +\tEOF\n +\n +\t# Add file within cone\n -+\ttest_all_match git sparse-checkout set deep &&\n ++\ttest_sparse_match git sparse-checkout set deep &&\n +\trun_on_all ../edit-contents deep/testfile &&\n +\ttest_all_match git add deep/testfile &&\n +\trun_on_all ../edit-contents deep/testfile &&\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'diff --staged' '\n +\ttest_all_match git reset --hard &&\n +\trun_on_all mkdir newdirectory &&\n +\trun_on_all ../edit-contents newdirectory/testfile &&\n -+\ttest_all_match git sparse-checkout set newdirectory &&\n ++\ttest_sparse_match git sparse-checkout set newdirectory &&\n +\ttest_all_match git add newdirectory/testfile &&\n +\trun_on_all ../edit-contents newdirectory/testfile &&\n -+\ttest_all_match git sparse-checkout set &&\n ++\ttest_sparse_match git sparse-checkout set &&\n +\n +\ttest_all_match git diff &&\n +\ttest_all_match git diff --staged &&\n +\n +\t# Merge conflict outside cone\n -+\ttest_all_match git reset --hard &&\n ++\t# The sparse checkout will report a warning that is not in the\n ++\t# full checkout, so we use `run_on_all` instead of\n ++\t# `test_all_match`\n ++\trun_on_all git reset --hard &&\n +\ttest_all_match git checkout merge-left &&\n +\ttest_all_match test_must_fail git merge merge-right &&\n +\n 2: ddcee003c92 = 2: a0b6a152c75 blame: enable and test the sparse index\n\n-- \ngitgitgadget\n"},{"id":"438912","messageId":"ac33159d020cc0c0f6fbee36eb74fff773cb8f9f.1634332836.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v2.git.1634332835.gitgitgadget@gmail.com","subject":"[PATCH v2 1/2] diff: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-10-15T21:20:34Z","receivedAt":"2021-10-15T21:20:42Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index within the 'git diff' command. Its implementation\nalready safely integrates with the sparse index because it shares code with\nthe 'git status' and 'git checkout' commands that were already integrated.\nThe most interesting thing to do is to add tests that verify that 'git diff'\nbehaves correctly when the sparse index is enabled. These cases are:\n\n1. The index is not expanded for 'diff' and 'diff --staged'\n2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\ncheckout, and sparse index repositories in the following partially-staged\nscenarios (i.e. the index, HEAD, and working directory differ at a given\npath):\n 1. Path is within sparse-checkout cone\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n\nThe `p2000` tests demonstrate a ~30% execution time reduction for 'git\ndiff' and a ~75% execution time reduction for 'git diff --staged' using a\nsparse index:\n\nTest before after\n-------------------------------------------------------------\n2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n\nCo-authored-by: Derrick Stolee \nSigned-off-by: Derrick Stolee \nSigned-off-by: Lessley Dennington \n---\n builtin/diff.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 45 ++++++++++++++++++++++++\n 3 files changed, 50 insertions(+)\n\ndiff --git a/builtin/diff.c b/builtin/diff.c\nindex dd8ce688ba7..cbf7b51c7c0 100644\n--- a/builtin/diff.c\n+++ b/builtin/diff.c\n@@ -437,6 +437,9 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tif (!no_index) {\n \t\t/*\n \t\t * Treat git diff with at least one path outside of the\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bfd332120c8..bff93f16e93 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n+test_perf_on_all git diff\n+test_perf_on_all git diff --staged\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex f19c1b3e2eb..e5d15be9d45 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -386,6 +386,46 @@ test_expect_success 'diff --staged' '\n \ttest_all_match git diff --staged\n '\n \n+test_expect_success 'diff partially-staged' '\n+\tinit_repos &&\n+\n+\twrite_script edit-contents <<-\\EOF &&\n+\techo text >>$1\n+\tEOF\n+\n+\t# Add file within cone\n+\ttest_sparse_match git sparse-checkout set deep &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\ttest_all_match git add deep/testfile &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\n+\t# Add file outside cone\n+\ttest_all_match git reset --hard &&\n+\trun_on_all mkdir newdirectory &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set newdirectory &&\n+\ttest_all_match git add newdirectory/testfile &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\n+\t# Merge conflict outside cone\n+\t# The sparse checkout will report a warning that is not in the\n+\t# full checkout, so we use `run_on_all` instead of\n+\t# `test_all_match`\n+\trun_on_all git reset --hard &&\n+\ttest_all_match git checkout merge-left &&\n+\ttest_all_match test_must_fail git merge merge-right &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged\n+'\n+\n # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n # running this test with 'df-conflict-2' after 'df-conflict-1'.\n test_expect_success 'diff with renames and conflicts' '\n@@ -800,6 +840,11 @@ test_expect_success 'sparse-index is not expanded' '\n \t# Wildcard identifies only full sparse directories, no index expansion\n \tensure_not_expanded reset deepest -- folder\\* &&\n \n+\techo a test change >>sparse-index/README.md &&\n+\tensure_not_expanded diff &&\n+\tgit -C sparse-index add README.md &&\n+\tensure_not_expanded diff --staged &&\n+\n \tensure_not_expanded checkout -f update-deep &&\n \ttest_config -C sparse-index pull.twohead ort &&\n \t(\n-- \ngitgitgadget\n\n"},{"id":"438913","messageId":"a0b6a152c754862323e9a5b89ad43ab34b6548f7.1634332836.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v2.git.1634332835.gitgitgadget@gmail.com","subject":"[PATCH v2 2/2] blame: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-10-15T21:20:35Z","receivedAt":"2021-10-15T21:20:44Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index for the 'git blame' command. The index was already\nnot expanded with this command, so the most interesting thing to do is to\nadd tests that verify that 'git blame' behaves correctly when the sparse\nindex is enabled and that its performance improves. More specifically, these\ncases are:\n\n1. The index is not expanded for 'blame' when given paths in the sparse\ncheckout cone at multiple levels.\n\n2. Performance measurably improves for 'blame' with sparse index when given\npaths in the sparse checkout cone at multiple levels.\n\nThe `p2000` tests demonstrate a ~60% execution time reduction when running\n'blame' for a file two levels deep and and a ~30% execution time reduction\nfor a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nWe do not include paths outside the sparse checkout cone because blame\ncurrently does not support blaming files outside of the sparse definition.\nAttempting to do so fails with the following error:\n\nfatal: no such path '' in HEAD\n\nSigned-off-by: Lessley Dennington \n---\n builtin/blame.c | 3 +++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 24 +++++++++++++++++-------\n 3 files changed, 22 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 641523ff9af..af3d81e2bd4 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \tlong anchor;\n \tconst int hexsz = the_hash_algo->hexsz;\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tsetup_default_color_by_age();\n \tgit_config(git_blame_config, &output_option);\n \trepo_init_revisions(the_repository, &revs, NULL);\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bff93f16e93..9ac76a049b8 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n test_perf_on_all git diff --staged\n+test_perf_on_all git blame $SPARSE_CONE/a\n+test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex e5d15be9d45..960ccf2d150 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -488,15 +488,16 @@ test_expect_success 'blame with pathspec inside sparse definition' '\n \ttest_all_match git blame deep/deeper1/deepest/a\n '\n \n-# TODO: blame currently does not support blaming files outside of the\n-# sparse definition. It complains that the file doesn't exist locally.\n-test_expect_failure 'blame with pathspec outside sparse definition' '\n+# Blame does not support blaming files outside of the sparse\n+# definition, so we verify this scenario.\n+test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n \n-\ttest_all_match git blame folder1/a &&\n-\ttest_all_match git blame folder2/a &&\n-\ttest_all_match git blame deep/deeper2/a &&\n-\ttest_all_match git blame deep/deeper2/deepest/a\n+\ttest_sparse_match git sparse-checkout set &&\n+\ttest_sparse_match test_must_fail git blame folder1/a &&\n+\ttest_sparse_match test_must_fail git blame folder2/a &&\n+\ttest_sparse_match test_must_fail git blame deep/deeper2/a &&\n+\ttest_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n@@ -874,6 +875,15 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n \t)\n '\n \n+test_expect_success 'sparse index is not expanded: blame' '\n+\tinit_repos &&\n+\n+\tensure_not_expanded blame a &&\n+\tensure_not_expanded blame deep/a &&\n+\tensure_not_expanded blame deep/deeper1/a &&\n+\tensure_not_expanded blame deep/deeper1/deepest/a\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n"},{"id":"439587","messageId":"YXcX5QWFQFIFNXo0@nand.local","threadId":"56710","inReplyTo":"ac33159d020cc0c0f6fbee36eb74fff773cb8f9f.1634332836.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/2] diff: enable and test the sparse index","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-10-25T20:47:33Z","receivedAt":"2021-10-25T20:47:42Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Oct 15, 2021 at 09:20:34PM +0000, Lessley Dennington via GitGitGadget wrote:\n> From: Lessley Dennington \n>\n> Enable the sparse index within the 'git diff' command. Its implementation\n> already safely integrates with the sparse index because it shares code with\n> the 'git status' and 'git checkout' commands that were already integrated.\n\nGood, it looks like most of the heavy-lifting to make `git diff` work\nwith the sparse index was already done elsewhere.\n\nIt may be helpful here to include either one of two things to help\nreaders and reviewers understand what's going on:\n\n - A summary of what `git status` and/or `git checkout` does to work\n with the sparse index.\n - Or the patches which make those commands work with the sparse index\n so that readers can refer back to them.\n\nHaving either of those would help readers who are unfamiliar with\nbuiltin/diff.c convince themselves more easily that setting\n'command_requires_full_index = 0' is all that's needed here.\n\n> The most interesting thing to do is to add tests that verify that 'git diff'\n> behaves correctly when the sparse index is enabled. These cases are:\n>\n> 1. The index is not expanded for 'diff' and 'diff --staged'\n> 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\n> checkout, and sparse index repositories in the following partially-staged\n> scenarios (i.e. the index, HEAD, and working directory differ at a given\n> path):\n> 1. Path is within sparse-checkout cone\n> 2. Path is outside sparse-checkout cone\n> 3. A merge conflict exists for paths outside sparse-checkout cone\n\nNice, these are all of the test cases that I would expect to demonstrate\ninteresting behavior.\n\n> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n> index f19c1b3e2eb..e5d15be9d45 100755\n> --- a/t/t1092-sparse-checkout-compatibility.sh\n> +++ b/t/t1092-sparse-checkout-compatibility.sh\n> @@ -386,6 +386,46 @@ test_expect_success 'diff --staged' '\n> \ttest_all_match git diff --staged\n> '\n>\n> +test_expect_success 'diff partially-staged' '\n> +\tinit_repos &&\n> +\n> +\twrite_script edit-contents <<-\\EOF &&\n> +\techo text >>$1\n> +\tEOF\n> +\n> +\t# Add file within cone\n> +\ttest_sparse_match git sparse-checkout set deep &&\n> +\trun_on_all ../edit-contents deep/testfile &&\n> +\ttest_all_match git add deep/testfile &&\n> +\trun_on_all ../edit-contents deep/testfile &&\n> +\n> +\ttest_all_match git diff &&\n> +\ttest_all_match git diff --staged &&\n> +\n> +\t# Add file outside cone\n> +\ttest_all_match git reset --hard &&\n> +\trun_on_all mkdir newdirectory &&\n> +\trun_on_all ../edit-contents newdirectory/testfile &&\n> +\ttest_sparse_match git sparse-checkout set newdirectory &&\n> +\ttest_all_match git add newdirectory/testfile &&\n> +\trun_on_all ../edit-contents newdirectory/testfile &&\n> +\ttest_sparse_match git sparse-checkout set &&\n> +\n> +\ttest_all_match git diff &&\n> +\ttest_all_match git diff --staged &&\n> +\n> +\t# Merge conflict outside cone\n> +\t# The sparse checkout will report a warning that is not in the\n> +\t# full checkout, so we use `run_on_all` instead of\n> +\t# `test_all_match`\n> +\trun_on_all git reset --hard &&\n> +\ttest_all_match git checkout merge-left &&\n> +\ttest_all_match test_must_fail git merge merge-right &&\n> +\n> +\ttest_all_match git diff &&\n> +\ttest_all_match git diff --staged\n> +'\n> +\n> # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n> # running this test with 'df-conflict-2' after 'df-conflict-1'.\n> test_expect_success 'diff with renames and conflicts' '\n> @@ -800,6 +840,11 @@ test_expect_success 'sparse-index is not expanded' '\n> \t# Wildcard identifies only full sparse directories, no index expansion\n> \tensure_not_expanded reset deepest -- folder\\* &&\n>\n> +\techo a test change >>sparse-index/README.md &&\n> +\tensure_not_expanded diff &&\n\nThinking aloud here as somebody who is unfamiliar with the sparse-index\ntests. ensure_not_expanded relies on the existence of the \"sparse-index\"\nrepository, and its top-level README.md is outside of the\nsparse-checkout cone.\n\nThat makes sense, and when I create a repository with a file outside of\nthe sparse-checkout cone and then run `git diff`, I see no changes as\nexpected.\n\nBut isn't the top-level directory always part of the cone? If so, I\nthink that what this (and the below test) is demonstrating is that we\ncan show changes inside of the cone without expanding the sparse-index.\n\nHaving that test makes absolute sense to me. But I think it might also\nmake sense to have a test that creates some directory structure outside\nof the cone, modifies it, and then ensures that both (a) those changes\naren't visible to `git diff` when the sparse-checkout is active and (b)\nthat running `git diff` doesn't cause the sparse-index to be expanded.\n\nThanks,\nTaylor\n"},{"id":"439588","messageId":"YXcZPxYlRLEEwU16@nand.local","threadId":"56710","inReplyTo":"a0b6a152c754862323e9a5b89ad43ab34b6548f7.1634332836.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/2] blame: enable and test the sparse index","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-10-25T20:53:19Z","receivedAt":"2021-10-25T20:53:23Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Oct 15, 2021 at 09:20:35PM +0000, Lessley Dennington via GitGitGadget wrote:\n> From: Lessley Dennington \n>\n> Enable the sparse index for the 'git blame' command. The index was already\n> not expanded with this command, so the most interesting thing to do is to\n> add tests that verify that 'git blame' behaves correctly when the sparse\n> index is enabled and that its performance improves. More specifically, these\n> cases are:\n>\n> 1. The index is not expanded for 'blame' when given paths in the sparse\n> checkout cone at multiple levels.\n>\n> 2. Performance measurably improves for 'blame' with sparse index when given\n> paths in the sparse checkout cone at multiple levels.\n>\n> The `p2000` tests demonstrate a ~60% execution time reduction when running\n> 'blame' for a file two levels deep and and a ~30% execution time reduction\n> for a file three levels deep.\n\nEek. What's eating up the other 30% when we have to open up another\nlayer of trees?\n\n>\n> Test before after\n> ----------------------------------------------------------------\n> 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n> 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n> 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n> 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n> 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n> 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n> 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n> 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n>\n> We do not include paths outside the sparse checkout cone because blame\n> currently does not support blaming files outside of the sparse definition.\n> Attempting to do so fails with the following error:\n>\n> fatal: no such path '' in HEAD.\n\nSmall nit; this error message should be indented with a couple of space\ncharacters to indicate that it's the output of running Git instead of\npart of your patch message. Not worth a reroll on its own, but something\nto keep in mind for your many future patches :).\n\n>\n> Signed-off-by: Lessley Dennington \n> ---\n> builtin/blame.c | 3 +++\n> t/perf/p2000-sparse-operations.sh | 2 ++\n> t/t1092-sparse-checkout-compatibility.sh | 24 +++++++++++++++++-------\n> 3 files changed, 22 insertions(+), 7 deletions(-)\n>\n> diff --git a/builtin/blame.c b/builtin/blame.c\n> index 641523ff9af..af3d81e2bd4 100644\n> --- a/builtin/blame.c\n> +++ b/builtin/blame.c\n> @@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n> \tlong anchor;\n> \tconst int hexsz = the_hash_algo->hexsz;\n>\n> +\tprepare_repo_settings(the_repository);\n> +\tthe_repository->settings.command_requires_full_index = 0;\n> +\n\nBy now we're quite used to seeing this ;). Makes sense to me.\n\n> \tsetup_default_color_by_age();\n> \tgit_config(git_blame_config, &output_option);\n> \trepo_init_revisions(the_repository, &revs, NULL);\n> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n> index bff93f16e93..9ac76a049b8 100755\n> --- a/t/perf/p2000-sparse-operations.sh\n> +++ b/t/perf/p2000-sparse-operations.sh\n> @@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n> test_perf_on_all git reset -- does-not-exist\n> test_perf_on_all git diff\n> test_perf_on_all git diff --staged\n> +test_perf_on_all git blame $SPARSE_CONE/a\n> +test_perf_on_all git blame $SPARSE_CONE/f3/a\n\nGood.\n\n> test_done\n> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n> index e5d15be9d45..960ccf2d150 100755\n> --- a/t/t1092-sparse-checkout-compatibility.sh\n> +++ b/t/t1092-sparse-checkout-compatibility.sh\n> @@ -488,15 +488,16 @@ test_expect_success 'blame with pathspec inside sparse definition' '\n> \ttest_all_match git blame deep/deeper1/deepest/a\n> '\n>\n> -# TODO: blame currently does not support blaming files outside of the\n> -# sparse definition. It complains that the file doesn't exist locally.\n> -test_expect_failure 'blame with pathspec outside sparse definition' '\n> +# Blame does not support blaming files outside of the sparse\n> +# definition, so we verify this scenario.\n> +test_expect_success 'blame with pathspec outside sparse definition' '\n> \tinit_repos &&\n>\n> -\ttest_all_match git blame folder1/a &&\n> -\ttest_all_match git blame folder2/a &&\n> -\ttest_all_match git blame deep/deeper2/a &&\n> -\ttest_all_match git blame deep/deeper2/deepest/a\n> +\ttest_sparse_match git sparse-checkout set &&\n> +\ttest_sparse_match test_must_fail git blame folder1/a &&\n> +\ttest_sparse_match test_must_fail git blame folder2/a &&\n> +\ttest_sparse_match test_must_fail git blame deep/deeper2/a &&\n> +\ttest_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n> '\n\ntest_must_fail used to allow for segfaults, but doesn't these days. So\nthis is a good test of \"it should fail in sparse checkouts but not\ncrash\", although I think it would be good to ensure that it's failing in\nthe way you expect (i.e., by checking that stderr contains \"no such path\n in HEAD\").\n>\n> test_expect_success 'checkout and reset (mixed)' '\n> @@ -874,6 +875,15 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n> \t)\n> '\n>\n> +test_expect_success 'sparse index is not expanded: blame' '\n> +\tinit_repos &&\n> +\n> +\tensure_not_expanded blame a &&\n> +\tensure_not_expanded blame deep/a &&\n> +\tensure_not_expanded blame deep/deeper1/a &&\n> +\tensure_not_expanded blame deep/deeper1/deepest/a\n> +'\n\nMakes sense. Probably just one of these is necessary, but I haven't\nlooked into init_repos (or the \"setup\" test) enough to know for sure.\nEither way, not worth changing.\n\nThanks,\nTaylor\n"},{"id":"439655","messageId":"6e2c07e7-c66f-01c0-a64e-2e8b5b7c1f8e@gmail.com","threadId":"56710","inReplyTo":"YXcX5QWFQFIFNXo0@nand.local","subject":"Re: [PATCH v2 1/2] diff: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-10-26T16:10:20Z","receivedAt":"2021-10-26T16:10:24Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 10/25/21 1:47 PM, Taylor Blau wrote:\n> On Fri, Oct 15, 2021 at 09:20:34PM +0000, Lessley Dennington via GitGitGadget wrote:\n>> From: Lessley Dennington \n>>\n>> Enable the sparse index within the 'git diff' command. Its implementation\n>> already safely integrates with the sparse index because it shares code with\n>> the 'git status' and 'git checkout' commands that were already integrated.\n> \n> Good, it looks like most of the heavy-lifting to make `git diff` work\n> with the sparse index was already done elsewhere.\n> \n> It may be helpful here to include either one of two things to help\n> readers and reviewers understand what's going on:\n> \n> - A summary of what `git status` and/or `git checkout` does to work\n> with the sparse index.\n> - Or the patches which make those commands work with the sparse index\n> so that readers can refer back to them.\n> \n> Having either of those would help readers who are unfamiliar with\n> builtin/diff.c convince themselves more easily that setting\n> 'command_requires_full_index = 0' is all that's needed here.\n> \nGreat suggestion, thank you!\n>> The most interesting thing to do is to add tests that verify that 'git diff'\n>> behaves correctly when the sparse index is enabled. These cases are:\n>>\n>> 1. The index is not expanded for 'diff' and 'diff --staged'\n>> 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\n>> checkout, and sparse index repositories in the following partially-staged\n>> scenarios (i.e. the index, HEAD, and working directory differ at a given\n>> path):\n>> 1. Path is within sparse-checkout cone\n>> 2. Path is outside sparse-checkout cone\n>> 3. A merge conflict exists for paths outside sparse-checkout cone\n> \n> Nice, these are all of the test cases that I would expect to demonstrate\n> interesting behavior.\n> \n>> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n>> index f19c1b3e2eb..e5d15be9d45 100755\n>> --- a/t/t1092-sparse-checkout-compatibility.sh\n>> +++ b/t/t1092-sparse-checkout-compatibility.sh\n>> @@ -386,6 +386,46 @@ test_expect_success 'diff --staged' '\n>> \ttest_all_match git diff --staged\n>> '\n>>\n>> +test_expect_success 'diff partially-staged' '\n>> +\tinit_repos &&\n>> +\n>> +\twrite_script edit-contents <<-\\EOF &&\n>> +\techo text >>$1\n>> +\tEOF\n>> +\n>> +\t# Add file within cone\n>> +\ttest_sparse_match git sparse-checkout set deep &&\n>> +\trun_on_all ../edit-contents deep/testfile &&\n>> +\ttest_all_match git add deep/testfile &&\n>> +\trun_on_all ../edit-contents deep/testfile &&\n>> +\n>> +\ttest_all_match git diff &&\n>> +\ttest_all_match git diff --staged &&\n>> +\n>> +\t# Add file outside cone\n>> +\ttest_all_match git reset --hard &&\n>> +\trun_on_all mkdir newdirectory &&\n>> +\trun_on_all ../edit-contents newdirectory/testfile &&\n>> +\ttest_sparse_match git sparse-checkout set newdirectory &&\n>> +\ttest_all_match git add newdirectory/testfile &&\n>> +\trun_on_all ../edit-contents newdirectory/testfile &&\n>> +\ttest_sparse_match git sparse-checkout set &&\n>> +\n>> +\ttest_all_match git diff &&\n>> +\ttest_all_match git diff --staged &&\n>> +\n>> +\t# Merge conflict outside cone\n>> +\t# The sparse checkout will report a warning that is not in the\n>> +\t# full checkout, so we use `run_on_all` instead of\n>> +\t# `test_all_match`\n>> +\trun_on_all git reset --hard &&\n>> +\ttest_all_match git checkout merge-left &&\n>> +\ttest_all_match test_must_fail git merge merge-right &&\n>> +\n>> +\ttest_all_match git diff &&\n>> +\ttest_all_match git diff --staged\n>> +'\n>> +\n>> # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n>> # running this test with 'df-conflict-2' after 'df-conflict-1'.\n>> test_expect_success 'diff with renames and conflicts' '\n>> @@ -800,6 +840,11 @@ test_expect_success 'sparse-index is not expanded' '\n>> \t# Wildcard identifies only full sparse directories, no index expansion\n>> \tensure_not_expanded reset deepest -- folder\\* &&\n>>\n>> +\techo a test change >>sparse-index/README.md &&\n>> +\tensure_not_expanded diff &&\n> \n> Thinking aloud here as somebody who is unfamiliar with the sparse-index\n> tests. ensure_not_expanded relies on the existence of the \"sparse-index\"\n> repository, and its top-level README.md is outside of the\n> sparse-checkout cone.\n> \n> That makes sense, and when I create a repository with a file outside of\n> the sparse-checkout cone and then run `git diff`, I see no changes as\n> expected.\n> \n> But isn't the top-level directory always part of the cone? If so, I\n> think that what this (and the below test) is demonstrating is that we\n> can show changes inside of the cone without expanding the sparse-index.\n> \n> Having that test makes absolute sense to me. But I think it might also\n> make sense to have a test that creates some directory structure outside\n> of the cone, modifies it, and then ensures that both (a) those changes\n> aren't visible to `git diff` when the sparse-checkout is active and (b)\n> that running `git diff` doesn't cause the sparse-index to be expanded.\n> \nREADME.md is actually within the sparse checkout cone - all files at \nroot are included by default. So your understanding is correct - we are \nensuring that making a change to a file in the cone and running both \ndiff and diff --staged once the file is in the index doesn't expand the \nsparse index.\n\nI like your idea of verifying that running diff against files outside \nthe sparse checkout cone won't expand the index. I've updated the diff \ntests in v3 (which I will send out shortly) to do so.\n\n> Thanks,\n> Taylor\n> \nBest,\nLessley\n"},{"id":"439656","messageId":"YXgpqJIS2OgOgS+k@nand.local","threadId":"56710","inReplyTo":"6e2c07e7-c66f-01c0-a64e-2e8b5b7c1f8e@gmail.com","subject":"Re: [PATCH v2 1/2] diff: enable and test the sparse index","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-10-26T16:15:36Z","receivedAt":"2021-10-26T16:16:06Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Oct 26, 2021 at 09:10:20AM -0700, Lessley Dennington wrote:\n> > But isn't the top-level directory always part of the cone? If so, I\n> > think that what this (and the below test) is demonstrating is that we\n> > can show changes inside of the cone without expanding the sparse-index.\n> >\n> > Having that test makes absolute sense to me. But I think it might also\n> > make sense to have a test that creates some directory structure outside\n> > of the cone, modifies it, and then ensures that both (a) those changes\n> > aren't visible to `git diff` when the sparse-checkout is active and (b)\n> > that running `git diff` doesn't cause the sparse-index to be expanded.\n> >\n> README.md is actually within the sparse checkout cone - all files at root\n> are included by default. So your understanding is correct - we are ensuring\n> that making a change to a file in the cone and running both diff and diff\n> --staged once the file is in the index doesn't expand the sparse index.\n>\n> I like your idea of verifying that running diff against files outside the\n> sparse checkout cone won't expand the index. I've updated the diff tests in\n> v3 (which I will send out shortly) to do so.\n\nGreat, thank you! There is no hurry to send out an updated revision from\nme, either. It may be good to wait a day or two and see if any other\nreview trickles in before sending another revision to the list that way\nyou can batch together updates from multiple reviewers.\n\nThanks,\nTaylor\n"},{"id":"439657","messageId":"086ad119-0739-5ce0-af99-0b1ab1c3d484@gmail.com","threadId":"56710","inReplyTo":"YXcZPxYlRLEEwU16@nand.local","subject":"Re: [PATCH v2 2/2] blame: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-10-26T16:17:03Z","receivedAt":"2021-10-26T16:18:02Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"On 10/25/21 1:53 PM, Taylor Blau wrote:\n> On Fri, Oct 15, 2021 at 09:20:35PM +0000, Lessley Dennington via GitGitGadget wrote:\n>> From: Lessley Dennington \n>>\n>> Enable the sparse index for the 'git blame' command. The index was already\n>> not expanded with this command, so the most interesting thing to do is to\n>> add tests that verify that 'git blame' behaves correctly when the sparse\n>> index is enabled and that its performance improves. More specifically, these\n>> cases are:\n>>\n>> 1. The index is not expanded for 'blame' when given paths in the sparse\n>> checkout cone at multiple levels.\n>>\n>> 2. Performance measurably improves for 'blame' with sparse index when given\n>> paths in the sparse checkout cone at multiple levels.\n>>\n>> The `p2000` tests demonstrate a ~60% execution time reduction when running\n>> 'blame' for a file two levels deep and and a ~30% execution time reduction\n>> for a file three levels deep.\n> \n> Eek. What's eating up the other 30% when we have to open up another\n> layer of trees?\n> \nI'm not sure to be totally honest. However, given these are both pretty \ngood time reductions I don't think we should be terribly concerned.\n>>\n>> Test before after\n>> ----------------------------------------------------------------\n>> 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n>> 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n>> 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n>> 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n>> 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n>> 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n>> 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n>> 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n>>\n>> We do not include paths outside the sparse checkout cone because blame\n>> currently does not support blaming files outside of the sparse definition.\n>> Attempting to do so fails with the following error:\n>>\n>> fatal: no such path '' in HEAD.\n> \n> Small nit; this error message should be indented with a couple of space\n> characters to indicate that it's the output of running Git instead of\n> part of your patch message. Not worth a reroll on its own, but something\n> to keep in mind for your many future patches :).\n> \nEh, I'm making some changes based on your suggestions anyway, so I'm \nincluding this in v3. Thanks for letting me know!\n>>\n>> Signed-off-by: Lessley Dennington \n>> ---\n>> builtin/blame.c | 3 +++\n>> t/perf/p2000-sparse-operations.sh | 2 ++\n>> t/t1092-sparse-checkout-compatibility.sh | 24 +++++++++++++++++-------\n>> 3 files changed, 22 insertions(+), 7 deletions(-)\n>>\n>> diff --git a/builtin/blame.c b/builtin/blame.c\n>> index 641523ff9af..af3d81e2bd4 100644\n>> --- a/builtin/blame.c\n>> +++ b/builtin/blame.c\n>> @@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n>> \tlong anchor;\n>> \tconst int hexsz = the_hash_algo->hexsz;\n>>\n>> +\tprepare_repo_settings(the_repository);\n>> +\tthe_repository->settings.command_requires_full_index = 0;\n>> +\n> \n> By now we're quite used to seeing this ;). Makes sense to me.\n> \n>> \tsetup_default_color_by_age();\n>> \tgit_config(git_blame_config, &output_option);\n>> \trepo_init_revisions(the_repository, &revs, NULL);\n>> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n>> index bff93f16e93..9ac76a049b8 100755\n>> --- a/t/perf/p2000-sparse-operations.sh\n>> +++ b/t/perf/p2000-sparse-operations.sh\n>> @@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n>> test_perf_on_all git reset -- does-not-exist\n>> test_perf_on_all git diff\n>> test_perf_on_all git diff --staged\n>> +test_perf_on_all git blame $SPARSE_CONE/a\n>> +test_perf_on_all git blame $SPARSE_CONE/f3/a\n> \n> Good.\n> \n>> test_done\n>> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n>> index e5d15be9d45..960ccf2d150 100755\n>> --- a/t/t1092-sparse-checkout-compatibility.sh\n>> +++ b/t/t1092-sparse-checkout-compatibility.sh\n>> @@ -488,15 +488,16 @@ test_expect_success 'blame with pathspec inside sparse definition' '\n>> \ttest_all_match git blame deep/deeper1/deepest/a\n>> '\n>>\n>> -# TODO: blame currently does not support blaming files outside of the\n>> -# sparse definition. It complains that the file doesn't exist locally.\n>> -test_expect_failure 'blame with pathspec outside sparse definition' '\n>> +# Blame does not support blaming files outside of the sparse\n>> +# definition, so we verify this scenario.\n>> +test_expect_success 'blame with pathspec outside sparse definition' '\n>> \tinit_repos &&\n>>\n>> -\ttest_all_match git blame folder1/a &&\n>> -\ttest_all_match git blame folder2/a &&\n>> -\ttest_all_match git blame deep/deeper2/a &&\n>> -\ttest_all_match git blame deep/deeper2/deepest/a\n>> +\ttest_sparse_match git sparse-checkout set &&\n>> +\ttest_sparse_match test_must_fail git blame folder1/a &&\n>> +\ttest_sparse_match test_must_fail git blame folder2/a &&\n>> +\ttest_sparse_match test_must_fail git blame deep/deeper2/a &&\n>> +\ttest_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n>> '\n> \n> test_must_fail used to allow for segfaults, but doesn't these days. So\n> this is a good test of \"it should fail in sparse checkouts but not\n> crash\", although I think it would be good to ensure that it's failing in\n> the way you expect (i.e., by checking that stderr contains \"no such path\n> in HEAD\").\nGood suggestion, coming in v3!\n>>\n>> test_expect_success 'checkout and reset (mixed)' '\n>> @@ -874,6 +875,15 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n>> \t)\n>> '\n>>\n>> +test_expect_success 'sparse index is not expanded: blame' '\n>> +\tinit_repos &&\n>> +\n>> +\tensure_not_expanded blame a &&\n>> +\tensure_not_expanded blame deep/a &&\n>> +\tensure_not_expanded blame deep/deeper1/a &&\n>> +\tensure_not_expanded blame deep/deeper1/deepest/a\n>> +'\n> \n> Makes sense. Probably just one of these is necessary, but I haven't\n> looked into init_repos (or the \"setup\" test) enough to know for sure.\n> Either way, not worth changing.\n> \n> Thanks,\n> Taylor\n> \n"},{"id":"440274","messageId":"pull.1050.v3.git.1635802069.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v2.git.1634332835.gitgitgadget@gmail.com","subject":"[PATCH v3 0/2] Sparse Index: diff and blame builtins","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-01T21:27:47Z","receivedAt":"2021-11-01T21:27:55Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"This series is based on vd/sparse-reset. It integrates the sparse index with\ngit diff and git blame and includes:\n\n 1. tests added to t1092 and p2000 to establish the baseline functionality\n of the commands\n 2. repository settings to enable the sparse index\n\nThe p2000 tests demonstrate a ~30% execution time reduction for 'git diff'\nand a ~75% execution time reduction for 'git diff --staged' using a sparse\nindex. For 'git blame', the reduction time was ~60% for a file two levels\ndeep and ~30% for a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\n\n\nChanges since V1\n================\n\n * Fix failing diff partially-staged test in\n t1092-sparse-checkout-compatibility.sh, which was breaking in seen.\n\n\nChanges since V2\n================\n\n * Update diff commit description to include patches that make the checkout\n and status commands work with the sparse index for readers to reference.\n * Add new test case to verify diff behaves as expected when run against\n files outside the sparse checkout cone.\n * Indent error message in blame commit\n * Check error message in blame with pathspec outside sparse definition test\n matches expectations.\n * Loop blame tests (instead of running the same command multiple time\n against different files).\n\nThanks, Lessley\n\nLessley Dennington (2):\n diff: enable and test the sparse index\n blame: enable and test the sparse index\n\n builtin/blame.c | 3 +\n builtin/diff.c | 3 +\n t/perf/p2000-sparse-operations.sh | 4 +\n t/t1092-sparse-checkout-compatibility.sh | 94 +++++++++++++++++++++---\n 4 files changed, 93 insertions(+), 11 deletions(-)\n\n\nbase-commit: 7159bf518eed5c997cf4ff0f17d9cb69192a091c\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1050%2Fldennington%2Fdiff-blame-sparse-index-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1050/ldennington/diff-blame-sparse-index-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/1050\n\nRange-diff vs v2:\n\n 1: ac33159d020 ! 1: 991aaad37b4 diff: enable and test the sparse index\n @@ Commit message\n Enable the sparse index within the 'git diff' command. Its implementation\n already safely integrates with the sparse index because it shares code with\n the 'git status' and 'git checkout' commands that were already integrated.\n + For more details see:\n +\n + d76723ee53 (status: use sparse-index throughout, 2021-07-14)\n + 1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n +\n The most interesting thing to do is to add tests that verify that 'git diff'\n behaves correctly when the sparse index is enabled. These cases are:\n \n @@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git checkout -f -\n test_done\n \n ## t/t1092-sparse-checkout-compatibility.sh ##\n -@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'diff --staged' '\n - \ttest_all_match git diff --staged\n +@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n + \t)\n '\n \n -+test_expect_success 'diff partially-staged' '\n ++test_expect_success 'sparse index is not expanded: diff' '\n +\tinit_repos &&\n +\n +\twrite_script edit-contents <<-\\EOF &&\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'diff --staged' '\n +\n +\ttest_all_match git diff &&\n +\ttest_all_match git diff --staged &&\n ++\tensure_not_expanded diff &&\n ++\tensure_not_expanded diff --staged &&\n +\n +\t# Add file outside cone\n +\ttest_all_match git reset --hard &&\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'diff --staged' '\n +\n +\ttest_all_match git diff &&\n +\ttest_all_match git diff --staged &&\n ++\tensure_not_expanded diff &&\n ++\tensure_not_expanded diff --staged &&\n +\n +\t# Merge conflict outside cone\n +\t# The sparse checkout will report a warning that is not in the\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'diff --staged' '\n +\ttest_all_match test_must_fail git merge merge-right &&\n +\n +\ttest_all_match git diff &&\n -+\ttest_all_match git diff --staged\n -+'\n -+\n - # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n - # running this test with 'df-conflict-2' after 'df-conflict-1'.\n - test_expect_success 'diff with renames and conflicts' '\n -@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is not expanded' '\n - \t# Wildcard identifies only full sparse directories, no index expansion\n - \tensure_not_expanded reset deepest -- folder\\* &&\n - \n -+\techo a test change >>sparse-index/README.md &&\n ++\ttest_all_match git diff --staged &&\n +\tensure_not_expanded diff &&\n -+\tgit -C sparse-index add README.md &&\n -+\tensure_not_expanded diff --staged &&\n ++\tensure_not_expanded diff --staged\n ++'\n +\n - \tensure_not_expanded checkout -f update-deep &&\n - \ttest_config -C sparse-index pull.twohead ort &&\n - \t(\n + # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n + # in this scenario, but it shouldn't.\n + test_expect_success 'reset mixed and checkout orphan' '\n 2: a0b6a152c75 ! 2: cfdd33129ec blame: enable and test the sparse index\n @@ Commit message\n currently does not support blaming files outside of the sparse definition.\n Attempting to do so fails with the following error:\n \n - fatal: no such path '' in HEAD\n + fatal: no such path '' in HEAD\n \n Signed-off-by: Lessley Dennington \n \n @@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git reset --hard\n test_done\n \n ## t/t1092-sparse-checkout-compatibility.sh ##\n -@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'blame with pathspec inside sparse definition' '\n - \ttest_all_match git blame deep/deeper1/deepest/a\n +@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'log with pathspec outside sparse definition' '\n + test_expect_success 'blame with pathspec inside sparse definition' '\n + \tinit_repos &&\n + \n +-\ttest_all_match git blame a &&\n +-\ttest_all_match git blame deep/a &&\n +-\ttest_all_match git blame deep/deeper1/a &&\n +-\ttest_all_match git blame deep/deeper1/deepest/a\n ++\tfor file in a \\\n ++\t\t\tdeep/a \\\n ++\t\t\tdeep/deeper1/a \\\n ++\t\t\tdeep/deeper1/deepest/a\n ++\tdo\n ++\t\ttest_all_match git blame $file\n ++\tdone\n '\n \n -# TODO: blame currently does not support blaming files outside of the\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'blame with pathsp\n +# definition, so we verify this scenario.\n +test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n ++\ttest_sparse_match git sparse-checkout set &&\n \n -\ttest_all_match git blame folder1/a &&\n -\ttest_all_match git blame folder2/a &&\n -\ttest_all_match git blame deep/deeper2/a &&\n -\ttest_all_match git blame deep/deeper2/deepest/a\n -+\ttest_sparse_match git sparse-checkout set &&\n -+\ttest_sparse_match test_must_fail git blame folder1/a &&\n -+\ttest_sparse_match test_must_fail git blame folder2/a &&\n -+\ttest_sparse_match test_must_fail git blame deep/deeper2/a &&\n -+\ttest_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n ++\tfor file in a \\\n ++\t\t\tdeep/a \\\n ++\t\t\tdeep/deeper1/a \\\n ++\t\t\tdeep/deeper1/deepest/a\n ++\tdo\n ++\t\ttest_sparse_match test_must_fail git blame $file &&\n ++\t\tcat >expect <<-EOF &&\n ++\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n ++\t\tEOF\n ++\t\t# We compare sparse-checkout-err and sparse-index-err in\n ++\t\t# `test_sparse_match`. Given we know they are the same, we\n ++\t\t# only check the content of sparse-index-err here.\n ++\t\ttest_cmp expect sparse-index-err\n ++\tdone\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n -@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n - \t)\n +@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse index is not expanded: diff' '\n + \tensure_not_expanded diff --staged\n '\n \n +test_expect_success 'sparse index is not expanded: blame' '\n +\tinit_repos &&\n +\n -+\tensure_not_expanded blame a &&\n -+\tensure_not_expanded blame deep/a &&\n -+\tensure_not_expanded blame deep/deeper1/a &&\n -+\tensure_not_expanded blame deep/deeper1/deepest/a\n ++\tfor file in a \\\n ++\t\t\tdeep/a \\\n ++\t\t\tdeep/deeper1/a \\\n ++\t\t\tdeep/deeper1/deepest/a\n ++\tdo\n ++\t\tensure_not_expanded blame $file\n ++\tdone\n +'\n +\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n\n-- \ngitgitgadget\n"},{"id":"440275","messageId":"991aaad37b41f71faa19fdef4373ccc115edcc40.1635802069.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v3.git.1635802069.gitgitgadget@gmail.com","subject":"[PATCH v3 1/2] diff: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-01T21:27:48Z","receivedAt":"2021-11-01T21:27:56Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index within the 'git diff' command. Its implementation\nalready safely integrates with the sparse index because it shares code with\nthe 'git status' and 'git checkout' commands that were already integrated.\nFor more details see:\n\nd76723ee53 (status: use sparse-index throughout, 2021-07-14)\n1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n\nThe most interesting thing to do is to add tests that verify that 'git diff'\nbehaves correctly when the sparse index is enabled. These cases are:\n\n1. The index is not expanded for 'diff' and 'diff --staged'\n2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\ncheckout, and sparse index repositories in the following partially-staged\nscenarios (i.e. the index, HEAD, and working directory differ at a given\npath):\n 1. Path is within sparse-checkout cone\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n\nThe `p2000` tests demonstrate a ~30% execution time reduction for 'git\ndiff' and a ~75% execution time reduction for 'git diff --staged' using a\nsparse index:\n\nTest before after\n-------------------------------------------------------------\n2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n\nCo-authored-by: Derrick Stolee \nSigned-off-by: Derrick Stolee \nSigned-off-by: Lessley Dennington \n---\n builtin/diff.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++\n 3 files changed, 51 insertions(+)\n\ndiff --git a/builtin/diff.c b/builtin/diff.c\nindex dd8ce688ba7..cbf7b51c7c0 100644\n--- a/builtin/diff.c\n+++ b/builtin/diff.c\n@@ -437,6 +437,9 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tif (!no_index) {\n \t\t/*\n \t\t * Treat git diff with at least one path outside of the\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bfd332120c8..bff93f16e93 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n+test_perf_on_all git diff\n+test_perf_on_all git diff --staged\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 44d5e11c762..53524660759 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -832,6 +832,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n \t)\n '\n \n+test_expect_success 'sparse index is not expanded: diff' '\n+\tinit_repos &&\n+\n+\twrite_script edit-contents <<-\\EOF &&\n+\techo text >>$1\n+\tEOF\n+\n+\t# Add file within cone\n+\ttest_sparse_match git sparse-checkout set deep &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\ttest_all_match git add deep/testfile &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --staged &&\n+\n+\t# Add file outside cone\n+\ttest_all_match git reset --hard &&\n+\trun_on_all mkdir newdirectory &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set newdirectory &&\n+\ttest_all_match git add newdirectory/testfile &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --staged &&\n+\n+\t# Merge conflict outside cone\n+\t# The sparse checkout will report a warning that is not in the\n+\t# full checkout, so we use `run_on_all` instead of\n+\t# `test_all_match`\n+\trun_on_all git reset --hard &&\n+\ttest_all_match git checkout merge-left &&\n+\ttest_all_match test_must_fail git merge merge-right &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --staged\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n\n"},{"id":"440276","messageId":"cfdd33129ec6860cbec0cb20302598429db1115e.1635802069.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v3.git.1635802069.gitgitgadget@gmail.com","subject":"[PATCH v3 2/2] blame: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-01T21:27:49Z","receivedAt":"2021-11-01T21:28:02Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index for the 'git blame' command. The index was already\nnot expanded with this command, so the most interesting thing to do is to\nadd tests that verify that 'git blame' behaves correctly when the sparse\nindex is enabled and that its performance improves. More specifically, these\ncases are:\n\n1. The index is not expanded for 'blame' when given paths in the sparse\ncheckout cone at multiple levels.\n\n2. Performance measurably improves for 'blame' with sparse index when given\npaths in the sparse checkout cone at multiple levels.\n\nThe `p2000` tests demonstrate a ~60% execution time reduction when running\n'blame' for a file two levels deep and and a ~30% execution time reduction\nfor a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nWe do not include paths outside the sparse checkout cone because blame\ncurrently does not support blaming files outside of the sparse definition.\nAttempting to do so fails with the following error:\n\n fatal: no such path '' in HEAD\n\nSigned-off-by: Lessley Dennington \n---\n builtin/blame.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 2 +\n t/t1092-sparse-checkout-compatibility.sh | 48 ++++++++++++++++++------\n 3 files changed, 42 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 641523ff9af..af3d81e2bd4 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \tlong anchor;\n \tconst int hexsz = the_hash_algo->hexsz;\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tsetup_default_color_by_age();\n \tgit_config(git_blame_config, &output_option);\n \trepo_init_revisions(the_repository, &revs, NULL);\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bff93f16e93..9ac76a049b8 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n test_perf_on_all git diff --staged\n+test_perf_on_all git blame $SPARSE_CONE/a\n+test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 53524660759..da442f3dcff 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -442,21 +442,35 @@ test_expect_success 'log with pathspec outside sparse definition' '\n test_expect_success 'blame with pathspec inside sparse definition' '\n \tinit_repos &&\n \n-\ttest_all_match git blame a &&\n-\ttest_all_match git blame deep/a &&\n-\ttest_all_match git blame deep/deeper1/a &&\n-\ttest_all_match git blame deep/deeper1/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_all_match git blame $file\n+\tdone\n '\n \n-# TODO: blame currently does not support blaming files outside of the\n-# sparse definition. It complains that the file doesn't exist locally.\n-test_expect_failure 'blame with pathspec outside sparse definition' '\n+# Blame does not support blaming files outside of the sparse\n+# definition, so we verify this scenario.\n+test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n+\ttest_sparse_match git sparse-checkout set &&\n \n-\ttest_all_match git blame folder1/a &&\n-\ttest_all_match git blame folder2/a &&\n-\ttest_all_match git blame deep/deeper2/a &&\n-\ttest_all_match git blame deep/deeper2/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_sparse_match test_must_fail git blame $file &&\n+\t\tcat >expect <<-EOF &&\n+\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n+\t\tEOF\n+\t\t# We compare sparse-checkout-err and sparse-index-err in\n+\t\t# `test_sparse_match`. Given we know they are the same, we\n+\t\t# only check the content of sparse-index-err here.\n+\t\ttest_cmp expect sparse-index-err\n+\tdone\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n@@ -878,6 +892,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n \tensure_not_expanded diff --staged\n '\n \n+test_expect_success 'sparse index is not expanded: blame' '\n+\tinit_repos &&\n+\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\tensure_not_expanded blame $file\n+\tdone\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n"},{"id":"440382","messageId":"xmqqh7ct89us.fsf@gitster.g","threadId":"56710","inReplyTo":"cfdd33129ec6860cbec0cb20302598429db1115e.1635802069.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 2/2] blame: enable and test the sparse index","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-03T16:47:23Z","receivedAt":"2021-11-03T16:47:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> We do not include paths outside the sparse checkout cone because blame\n> currently does not support blaming files outside of the sparse definition.\n> Attempting to do so fails with the following error:\n>\n> fatal: no such path '' in HEAD\n\nDoes this indicate that we need to update how the command line\nsafety in verify_working_tree_path() works in a sparsely checked out\nworking tree? If foo/bar is outside the sparse definition,\n\n git blame HEAD foo/bar\n\nmay get such a message, but shouldn't\n\n git blame HEAD -- foo/bar\n\nmake it work?\n\n> -# TODO: blame currently does not support blaming files outside of the\n> -# sparse definition. It complains that the file doesn't exist locally.\n> -test_expect_failure 'blame with pathspec outside sparse definition' '\n> +# Blame does not support blaming files outside of the sparse\n> +# definition, so we verify this scenario.\n\nIOW, why is it a good idea to drop the \"TODO\" and \"currently\" and pretend\nas if the current behaviour is the desirable one?\n\n> +test_expect_success 'blame with pathspec outside sparse definition' '\n> \tinit_repos &&\n> +\ttest_sparse_match git sparse-checkout set &&\n> \n> -\ttest_all_match git blame folder1/a &&\n> -\ttest_all_match git blame folder2/a &&\n> -\ttest_all_match git blame deep/deeper2/a &&\n> -\ttest_all_match git blame deep/deeper2/deepest/a\n> +\tfor file in a \\\n> +\t\t\tdeep/a \\\n> +\t\t\tdeep/deeper1/a \\\n> +\t\t\tdeep/deeper1/deepest/a\n> +\tdo\n> +\t\ttest_sparse_match test_must_fail git blame $file &&\n> +\t\tcat >expect <<-EOF &&\n> +\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n> +\t\tEOF\n> +\t\t# We compare sparse-checkout-err and sparse-index-err in\n> +\t\t# `test_sparse_match`. Given we know they are the same, we\n> +\t\t# only check the content of sparse-index-err here.\n> +\t\ttest_cmp expect sparse-index-err\n> +\tdone\n> '\n> \n> test_expect_success 'checkout and reset (mixed)' '\n> @@ -878,6 +892,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n> \tensure_not_expanded diff --staged\n> '\n> \n> +test_expect_success 'sparse index is not expanded: blame' '\n> +\tinit_repos &&\n> +\n> +\tfor file in a \\\n> +\t\t\tdeep/a \\\n> +\t\t\tdeep/deeper1/a \\\n> +\t\t\tdeep/deeper1/deepest/a\n> +\tdo\n> +\t\tensure_not_expanded blame $file\n> +\tdone\n> +'\n> +\n> # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n> # in this scenario, but it shouldn't.\n> test_expect_success 'reset mixed and checkout orphan' '\n"},{"id":"440384","messageId":"xmqqcznh8913.fsf@gitster.g","threadId":"56710","inReplyTo":"991aaad37b41f71faa19fdef4373ccc115edcc40.1635802069.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 1/2] diff: enable and test the sparse index","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-03T17:05:12Z","receivedAt":"2021-11-03T17:05:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> 2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n> 2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n> 2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n> 2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n\nPlease do not add more use of the synonym to the test suite, other\nthan the one that makes sure the synonym works the same way as the\nreal option, which is \"--cached\".\n\n> diff --git a/builtin/diff.c b/builtin/diff.c\n> index dd8ce688ba7..cbf7b51c7c0 100644\n> --- a/builtin/diff.c\n> +++ b/builtin/diff.c\n> @@ -437,6 +437,9 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n> \n> \tprefix = setup_git_directory_gently(&nongit);\n> \n> +\tprepare_repo_settings(the_repository);\n> +\tthe_repository->settings.command_requires_full_index = 0;\n> +\n\nDoesn't the code need to be protected with\n\n\tif (!nongit) {\n\t\tprepare_repo_settings(the_repository);\n\t\tthe_repository->settings.command_requires_full_index = 0;\n\t}\n\nat the very least? It may be that the code is getting lucky because\nthe_repository may be initialized with a random value (after all,\nwhen we are not in a repository, there is nowhere to read the\non-disk settings from) and we may even be able to set a bit in the\nsettings structure without crashing, but conceptually, doing the\nabove when we _know_ we are not in any repository is simply wrong.\n\nI wonder if prepare_repo_settings() needs be more strict. For\nexample, shouldn't it check if we have a repository to begin with\nand BUG() if it was called when there is not a repository? After\nall, it tries to read from the repository configuration file, so any\nnecessary set-up to discover where the gitdir is must have been done\nalready before it can be called.\n\nWith such a safety feature to catch a programmer errors, perhaps the\nabove could have been caught before the patch hit the list.\n\nThoughts? Am I missing some chicken-and-egg situation where\nprepare_repo_settings() must be callable before we know where the\nrepository is, or something, which justifies why the function is so\nloose in its sanity checks in the current form?\n\n\n"},{"id":"440496","messageId":"1a269d05-49f6-6c49-860a-044070f14ffc@gmail.com","threadId":"56710","inReplyTo":"xmqqcznh8913.fsf@gitster.g","subject":"Re: [PATCH v3 1/2] diff: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-04T23:55:06Z","receivedAt":"2021-11-04T23:55:14Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"On 11/3/21 10:05 AM, Junio C Hamano wrote:\n> \"Lessley Dennington via GitGitGadget\" \n> writes:\n> \n>> 2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n>> 2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n>> 2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n>> 2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n> \n> Please do not add more use of the synonym to the test suite, other\n> than the one that makes sure the synonym works the same way as the\n> real option, which is \"--cached\".\n>\n\nThank you, changed for v4.\n\n>> diff --git a/builtin/diff.c b/builtin/diff.c\n>> index dd8ce688ba7..cbf7b51c7c0 100644\n>> --- a/builtin/diff.c\n>> +++ b/builtin/diff.c\n>> @@ -437,6 +437,9 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n>> \n>> \tprefix = setup_git_directory_gently(&nongit);\n>> \n>> +\tprepare_repo_settings(the_repository);\n>> +\tthe_repository->settings.command_requires_full_index = 0;\n>> +\n> \n> Doesn't the code need to be protected with\n> \n> \tif (!nongit) {\n> \t\tprepare_repo_settings(the_repository);\n> \t\tthe_repository->settings.command_requires_full_index = 0;\n> \t}\n> \n> at the very least? It may be that the code is getting lucky because\n> the_repository may be initialized with a random value (after all,\n> when we are not in a repository, there is nowhere to read the\n> on-disk settings from) and we may even be able to set a bit in the\n> settings structure without crashing, but conceptually, doing the\n> above when we _know_ we are not in any repository is simply wrong.\n> \n> I wonder if prepare_repo_settings() needs be more strict. For\n> example, shouldn't it check if we have a repository to begin with\n> and BUG() if it was called when there is not a repository? After\n> all, it tries to read from the repository configuration file, so any\n> necessary set-up to discover where the gitdir is must have been done\n> already before it can be called.\n> \n> With such a safety feature to catch a programmer errors, perhaps the\n> above could have been caught before the patch hit the list.\n> \n> Thoughts? Am I missing some chicken-and-egg situation where\n> prepare_repo_settings() must be callable before we know where the\n> repository is, or something, which justifies why the function is so\n> loose in its sanity checks in the current form?\n> \n> \n\nThis seems like a good idea. I've added both the nongit check and the \nprepare_repo_settings() updates you've suggested for v4, pending review \nby my team.\n\nBest,\nLessley\n"},{"id":"440497","messageId":"49ada9fe-3cd5-622d-fdc7-92f95b040b47@gmail.com","threadId":"56710","inReplyTo":"xmqqh7ct89us.fsf@gitster.g","subject":"Re: [PATCH v3 2/2] blame: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-05T00:04:49Z","receivedAt":"2021-11-05T00:04:53Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"On 11/3/21 9:47 AM, Junio C Hamano wrote:\n> \"Lessley Dennington via GitGitGadget\" \n> writes:\n> \n>> We do not include paths outside the sparse checkout cone because blame\n>> currently does not support blaming files outside of the sparse definition.\n>> Attempting to do so fails with the following error:\n>>\n>> fatal: no such path '' in HEAD\n> \n> Does this indicate that we need to update how the command line\n> safety in verify_working_tree_path() works in a sparsely checked out\n> working tree? If foo/bar is outside the sparse definition,\n> \n> git blame HEAD foo/bar\n> \n> may get such a message, but shouldn't\n> \n> git blame HEAD -- foo/bar\n> \n> make it work?\n> \nThis is something we could consider for the future, and I've updated the \ncomment you called out below to better reflect that in the next version. \nHowever, I don't know that this change belongs in this series, which \naims to enable the sparse index. Perhaps even updating the 'blame with \npathspec outside sparse definition' test was a bit too far outside the \ntarget scope of this change - I can remove my updates if that is the case.\n\n>> -# TODO: blame currently does not support blaming files outside of the\n>> -# sparse definition. It complains that the file doesn't exist locally.\n>> -test_expect_failure 'blame with pathspec outside sparse definition' '\n>> +# Blame does not support blaming files outside of the sparse\n>> +# definition, so we verify this scenario.\n> \n> IOW, why is it a good idea to drop the \"TODO\" and \"currently\" and pretend\n> as if the current behaviour is the desirable one?\n> \nThank you, updated for v4.\n\n>> +test_expect_success 'blame with pathspec outside sparse definition' '\n>> \tinit_repos &&\n>> +\ttest_sparse_match git sparse-checkout set &&\n>> \n>> -\ttest_all_match git blame folder1/a &&\n>> -\ttest_all_match git blame folder2/a &&\n>> -\ttest_all_match git blame deep/deeper2/a &&\n>> -\ttest_all_match git blame deep/deeper2/deepest/a\n>> +\tfor file in a \\\n>> +\t\t\tdeep/a \\\n>> +\t\t\tdeep/deeper1/a \\\n>> +\t\t\tdeep/deeper1/deepest/a\n>> +\tdo\n>> +\t\ttest_sparse_match test_must_fail git blame $file &&\n>> +\t\tcat >expect <<-EOF &&\n>> +\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n>> +\t\tEOF\n>> +\t\t# We compare sparse-checkout-err and sparse-index-err in\n>> +\t\t# `test_sparse_match`. Given we know they are the same, we\n>> +\t\t# only check the content of sparse-index-err here.\n>> +\t\ttest_cmp expect sparse-index-err\n>> +\tdone\n>> '\n>> \n>> test_expect_success 'checkout and reset (mixed)' '\n>> @@ -878,6 +892,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n>> \tensure_not_expanded diff --staged\n>> '\n>> \n>> +test_expect_success 'sparse index is not expanded: blame' '\n>> +\tinit_repos &&\n>> +\n>> +\tfor file in a \\\n>> +\t\t\tdeep/a \\\n>> +\t\t\tdeep/deeper1/a \\\n>> +\t\t\tdeep/deeper1/deepest/a\n>> +\tdo\n>> +\t\tensure_not_expanded blame $file\n>> +\tdone\n>> +'\n>> +\n>> # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n>> # in this scenario, but it shouldn't.\n>> test_expect_success 'reset mixed and checkout orphan' '\n\nBest,\n\nLessley\n"},{"id":"441877","messageId":"CABPp-BFSOyGZom-Cf_7++8Yecw76e+E6O7iYA7okkfCwwXXpCQ@mail.gmail.com","threadId":"56710","inReplyTo":"086ad119-0739-5ce0-af99-0b1ab1c3d484@gmail.com","subject":"Re: [PATCH v2 2/2] blame: enable and test the sparse index","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-11-21T01:32:36Z","receivedAt":"2021-11-21T01:32:52Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Oct 26, 2021 at 9:17 AM Lessley Dennington\n wrote:\n>\n> On 10/25/21 1:53 PM, Taylor Blau wrote:\n> > On Fri, Oct 15, 2021 at 09:20:35PM +0000, Lessley Dennington via GitGitGadget wrote:\n> >> From: Lessley Dennington \n> >>\n> >> Enable the sparse index for the 'git blame' command. The index was already\n> >> not expanded with this command, so the most interesting thing to do is to\n> >> add tests that verify that 'git blame' behaves correctly when the sparse\n> >> index is enabled and that its performance improves. More specifically, these\n> >> cases are:\n> >>\n> >> 1. The index is not expanded for 'blame' when given paths in the sparse\n> >> checkout cone at multiple levels.\n> >>\n> >> 2. Performance measurably improves for 'blame' with sparse index when given\n> >> paths in the sparse checkout cone at multiple levels.\n> >>\n> >> The `p2000` tests demonstrate a ~60% execution time reduction when running\n> >> 'blame' for a file two levels deep and and a ~30% execution time reduction\n> >> for a file three levels deep.\n> >\n> > Eek. What's eating up the other 30% when we have to open up another\n> > layer of trees?\n> >\n> I'm not sure to be totally honest. However, given these are both pretty\n> good time reductions I don't think we should be terribly concerned.\n\nIt's not something eating up more time in the sparse-index code; let's\nlook a bit closer...\n\n> >>\n> >> Test before after\n> >> ----------------------------------------------------------------\n> >> 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n> >> 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n> >> 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n> >> 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n> >> 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n> >> 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n> >> 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n> >> 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nTime was ~0.55s for the full at two levels deep, and dropped by just\nover 0.3s in sparse-index.\nTime was ~1.05s for the full at three levels deep, and dropped by just\nover 0.3s in sparse-index.\n\nSo, the sparse-index enabling saves us the same amount of time, it's\njust that the overall execution time for the non-sparse-index\ncomparison point goes up. Saving the same amount of time for the two\ncases seems intuitive to me; both cases get to avoid looking at the\nsame number of index entries outside the sparsity paths.\n"},{"id":"441878","messageId":"CABPp-BH2O+fezCui0utYbpShp6NaMwWapyr8i+mDDjjhzec4LQ@mail.gmail.com","threadId":"56710","inReplyTo":"xmqqh7ct89us.fsf@gitster.g","subject":"Re: [PATCH v3 2/2] blame: enable and test the sparse index","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-11-21T01:46:52Z","receivedAt":"2021-11-21T01:47:07Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Nov 3, 2021 at 9:47 AM Junio C Hamano wrote:\n>\n> \"Lessley Dennington via GitGitGadget\" \n> writes:\n>\n> > We do not include paths outside the sparse checkout cone because blame\n> > currently does not support blaming files outside of the sparse definition.\n> > Attempting to do so fails with the following error:\n> >\n> > fatal: no such path '' in HEAD\n>\n> Does this indicate that we need to update how the command line\n> safety in verify_working_tree_path() works in a sparsely checked out\n> working tree?\n\nI wondered the same thing, but no I don't think we need any extra\ncommand line safety here. The behavior Lessley is reporting here is\nequivalent to the following in a regular (non-sparse) checkout:\n\n$ rm t/test-lib.sh\n$ git blame t/test-lib.sh\nfatal: Cannot lstat 't/test-lib.sh': No such file or directory\n$ git blame -- t/test-lib.sh\nfatal: Cannot lstat 't/test-lib.sh': No such file or directory\n\nblame without a revision has always failed for files not in the\nworking tree, regardless of whether those files are found in the index\nor HEAD.\n\n> If foo/bar is outside the sparse definition,\n>\n> git blame HEAD foo/bar\n\nActually, that works; there's no error and the code with Lessley's\npatch will show the blame info for foo/bar (assuming foo/bar was a\npath in HEAD, of course).\n\n> may get such a message, but shouldn't\n>\n> git blame HEAD -- foo/bar\n>\n> make it work?\n\nThis also works. But both of these things are kind of testing\nsomething different; when given a revision, the checkout is irrelevant\nto git blame: git blame with a revision will work regardless of\nwhether the checkout is full, sparse, completely empty, or\nnon-existent (i.e. a bare clone).\n\n> > -# TODO: blame currently does not support blaming files outside of the\n> > -# sparse definition. It complains that the file doesn't exist locally.\n> > -test_expect_failure 'blame with pathspec outside sparse definition' '\n> > +# Blame does not support blaming files outside of the sparse\n> > +# definition, so we verify this scenario.\n>\n> IOW, why is it a good idea to drop the \"TODO\" and \"currently\" and pretend\n> as if the current behaviour is the desirable one?\n\nI think dropping the TODO is correct, but the wording is confusing --\nit has nothing to do with sparse checkouts. I'd rather say, \"Without\na specified revision, blame will only handle files present in the\ncurrent working directory and error on any other paths\"\n\n> > +test_expect_success 'blame with pathspec outside sparse definition' '\n> > init_repos &&\n> > + test_sparse_match git sparse-checkout set &&\n> >\n> > - test_all_match git blame folder1/a &&\n> > - test_all_match git blame folder2/a &&\n> > - test_all_match git blame deep/deeper2/a &&\n> > - test_all_match git blame deep/deeper2/deepest/a\n> > + for file in a \\\n> > + deep/a \\\n> > + deep/deeper1/a \\\n> > + deep/deeper1/deepest/a\n> > + do\n> > + test_sparse_match test_must_fail git blame $file &&\n> > + cat >expect <<-EOF &&\n> > + fatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n> > + EOF\n> > + # We compare sparse-checkout-err and sparse-index-err in\n> > + # `test_sparse_match`. Given we know they are the same, we\n> > + # only check the content of sparse-index-err here.\n> > + test_cmp expect sparse-index-err\n> > + done\n> > '\n> >\n> > test_expect_success 'checkout and reset (mixed)' '\n> > @@ -878,6 +892,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n> > ensure_not_expanded diff --staged\n> > '\n> >\n> > +test_expect_success 'sparse index is not expanded: blame' '\n> > + init_repos &&\n> > +\n> > + for file in a \\\n> > + deep/a \\\n> > + deep/deeper1/a \\\n> > + deep/deeper1/deepest/a\n> > + do\n> > + ensure_not_expanded blame $file\n> > + done\n> > +'\n> > +\n> > # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n> > # in this scenario, but it shouldn't.\n> > test_expect_success 'reset mixed and checkout orphan' '\n"},{"id":"442051","messageId":"pull.1050.v4.git.1637620958.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v3.git.1635802069.gitgitgadget@gmail.com","subject":"[PATCH v4 0/4] Sparse Index: diff and blame builtins","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-22T22:42:34Z","receivedAt":"2021-11-22T22:42:44Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"This series is based on vd/sparse-reset. It integrates the sparse index with\ngit diff and git blame and includes:\n\n 1. tests added to t1092 and p2000 to establish the baseline functionality\n of the commands\n 2. repository settings to enable the sparse index\n\nThe p2000 tests demonstrate a ~44% execution time reduction for 'git diff'\nand a ~86% execution time reduction for 'git diff --staged' using a sparse\nindex. For 'git blame', the reduction time was ~60% for a file two levels\ndeep and ~30% for a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\n\n\nChanges since V1\n================\n\n * Fix failing diff partially-staged test in\n t1092-sparse-checkout-compatibility.sh, which was breaking in seen.\n\n\nChanges since V2\n================\n\n * Update diff commit description to include patches that make the checkout\n and status commands work with the sparse index for readers to reference.\n * Add new test case to verify diff behaves as expected when run against\n files outside the sparse checkout cone.\n * Indent error message in blame commit\n * Check error message in blame with pathspec outside sparse definition test\n matches expectations.\n * Loop blame tests (instead of running the same command multiple time\n against different files).\n\n\nChanges since V3\n================\n\n * Update diff p2000 tests to use --cached instead of --staged. Execute new\n run and update results in commit description and cover letter.\n * Update comment on blame with pathspec outside sparse definition test in\n t1092-sparse-checkout-compatibility.sh to clarify that it tests the\n current state and could be improved in the future.\n * Ensure sparse index is only activated when diff is running against files\n in a Git repo.\n * BUG if prepare_repo_settings() is called outside a repository.\n * Ensure sparse index is not activated for calls to blame, checkout, or\n pack-object with -h.\n * Ensure commit-graph is only loaded if a git directory exists.\n\nThanks, Lessley\n\nLessley Dennington (4):\n sparse index: enable only for git repos\n test-read-cache: set up repo after git directory\n diff: enable and test the sparse index\n blame: enable and test the sparse index\n\n builtin/blame.c | 5 ++\n builtin/checkout.c | 6 +-\n builtin/diff.c | 5 ++\n builtin/pack-objects.c | 9 ++-\n commit-graph.c | 5 +-\n repo-settings.c | 3 +\n t/helper/test-read-cache.c | 5 +-\n t/perf/p2000-sparse-operations.sh | 4 +\n t/t1092-sparse-checkout-compatibility.sh | 95 +++++++++++++++++++++---\n 9 files changed, 118 insertions(+), 19 deletions(-)\n\n\nbase-commit: 7159bf518eed5c997cf4ff0f17d9cb69192a091c\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1050%2Fldennington%2Fdiff-blame-sparse-index-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1050/ldennington/diff-blame-sparse-index-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/1050\n\nRange-diff vs v3:\n\n -: ----------- > 1: 81e208cf454 sparse index: enable only for git repos\n -: ----------- > 2: 5bc5e8465ab test-read-cache: set up repo after git directory\n 1: 991aaad37b4 ! 3: 273ee16b74e diff: enable and test the sparse index\n @@ Commit message\n diff: enable and test the sparse index\n \n Enable the sparse index within the 'git diff' command. Its implementation\n - already safely integrates with the sparse index because it shares code with\n - the 'git status' and 'git checkout' commands that were already integrated.\n - For more details see:\n + already safely integrates with the sparse index because it shares code\n + with the 'git status' and 'git checkout' commands that were already\n + integrated. For more details see:\n \n - d76723ee53 (status: use sparse-index throughout, 2021-07-14)\n - 1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n + d76723e (status: use sparse-index throughout, 2021-07-14)\n + 1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29)\n \n - The most interesting thing to do is to add tests that verify that 'git diff'\n - behaves correctly when the sparse index is enabled. These cases are:\n + The most interesting thing to do is to add tests that verify that 'git\n + diff' behaves correctly when the sparse index is enabled. These cases are:\n \n 1. The index is not expanded for 'diff' and 'diff --staged'\n 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\n @@ Commit message\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n \n - The `p2000` tests demonstrate a ~30% execution time reduction for 'git\n - diff' and a ~75% execution time reduction for 'git diff --staged' using a\n + The `p2000` tests demonstrate a ~44% execution time reduction for 'git\n + diff' and a ~86% execution time reduction for 'git diff --staged' using a\n sparse index:\n \n Test before after\n -------------------------------------------------------------\n - 2000.30: git diff (full-v3) 0.37 0.36 -2.7%\n - 2000.31: git diff (full-v4) 0.36 0.35 -2.8%\n - 2000.32: git diff (sparse-v3) 0.46 0.30 -34.8%\n - 2000.33: git diff (sparse-v4) 0.43 0.31 -27.9%\n - 2000.34: git diff --staged (full-v3) 0.08 0.08 +0.0%\n - 2000.35: git diff --staged (full-v4) 0.08 0.08 +0.0%\n - 2000.36: git diff --staged (sparse-v3) 0.17 0.04 -76.5%\n - 2000.37: git diff --staged (sparse-v4) 0.16 0.04 -75.0%\n + 2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n + 2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n + 2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n + 2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n + 2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n + 2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n + 2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n + 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n \n Co-authored-by: Derrick Stolee \n Signed-off-by: Derrick Stolee \n @@ builtin/diff.c: int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n -+\tprepare_repo_settings(the_repository);\n -+\tthe_repository->settings.command_requires_full_index = 0;\n ++\tif (!nongit) {\n ++\t\tprepare_repo_settings(the_repository);\n ++\t\tthe_repository->settings.command_requires_full_index = 0;\n ++\t}\n +\n \tif (!no_index) {\n \t\t/*\n @@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git checkout -f -\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n +test_perf_on_all git diff\n -+test_perf_on_all git diff --staged\n ++test_perf_on_all git diff --cached\n \n test_done\n \n 2: cfdd33129ec ! 4: 7acf5118bf5 blame: enable and test the sparse index\n @@ builtin/blame.c: int cmd_blame(int argc, const char **argv, const char *prefix)\n \tlong anchor;\n \tconst int hexsz = the_hash_algo->hexsz;\n \n -+\tprepare_repo_settings(the_repository);\n -+\tthe_repository->settings.command_requires_full_index = 0;\n ++\tif (startup_info->have_repository) {\n ++\t\tprepare_repo_settings(the_repository);\n ++\t\tthe_repository->settings.command_requires_full_index = 0;\n ++\t}\n +\n \tsetup_default_color_by_age();\n \tgit_config(git_blame_config, &output_option);\n \trepo_init_revisions(the_repository, &revs, NULL);\n \n ## t/perf/p2000-sparse-operations.sh ##\n -@@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git reset --hard\n +@@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git reset\n + test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n - test_perf_on_all git diff --staged\n +-test_perf_on_all git diff --cached\n ++test_perf_on_all git diff --staged\n +test_perf_on_all git blame $SPARSE_CONE/a\n +test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'log with pathspec\n -# TODO: blame currently does not support blaming files outside of the\n -# sparse definition. It complains that the file doesn't exist locally.\n -test_expect_failure 'blame with pathspec outside sparse definition' '\n -+# Blame does not support blaming files outside of the sparse\n -+# definition, so we verify this scenario.\n ++# NEEDSWORK: This test documents the current behavior, but this could\n ++# change in the future if we decide to support blaming files outside\n ++# the sparse definition.\n +test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n +\ttest_sparse_match git sparse-checkout set &&\n\n-- \ngitgitgadget\n"},{"id":"442052","messageId":"81e208cf454b61c761fa66e4f43a464ed439ba59.1637620958.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v4.git.1637620958.gitgitgadget@gmail.com","subject":"[PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-22T22:42:35Z","receivedAt":"2021-11-22T22:42:48Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nCheck whether git dir exists before adding any repo settings. If it\ndoes not exist, BUG with the message that one cannot add settings for an\nuninitialized repository. If it does exist, proceed with adding repo\nsettings.\n\nAdditionally, ensure the above BUG is not triggered when users pass the -h\nflag by adding a check for the repository to the checkout and pack-objects\nbuiltins.\n\nFinally, ensure the above BUG is not triggered for commit-graph by\nreturning early if the git directory does not exist.\n\nSigned-off-by: Lessley Dennington \n---\n builtin/checkout.c | 6 ++++--\n builtin/pack-objects.c | 9 ++++++---\n commit-graph.c | 5 ++++-\n repo-settings.c | 3 +++\n 4 files changed, 17 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin/checkout.c b/builtin/checkout.c\nindex 8c69dcdf72a..31632b07888 100644\n--- a/builtin/checkout.c\n+++ b/builtin/checkout.c\n@@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n \n \tgit_config(git_checkout_config, opts);\n \n-\tprepare_repo_settings(the_repository);\n-\tthe_repository->settings.command_requires_full_index = 0;\n+\tif (startup_info->have_repository) {\n+\t\tprepare_repo_settings(the_repository);\n+\t\tthe_repository->settings.command_requires_full_index = 0;\n+\t}\n \n \topts->track = BRANCH_TRACK_UNSPECIFIED;\n \ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 1a3dd445f83..45dc2258dc7 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -3976,9 +3976,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tread_replace_refs = 0;\n \n \tsparse = git_env_bool(\"GIT_TEST_PACK_SPARSE\", -1);\n-\tprepare_repo_settings(the_repository);\n-\tif (sparse < 0)\n-\t\tsparse = the_repository->settings.pack_use_sparse;\n+\n+\tif (startup_info->have_repository) {\n+\t\tprepare_repo_settings(the_repository);\n+\t\tif (sparse < 0)\n+\t\t\tsparse = the_repository->settings.pack_use_sparse;\n+\t}\n \n \treset_pack_idx_option(&pack_idx_opts);\n \tgit_config(git_pack_config, NULL);\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 2706683acfe..265c010122e 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n \tstruct object_directory *odb;\n \n \t/*\n+\t * Early return if there is no git dir or if the commit graph is\n+\t * disabled.\n+\t *\n \t * This must come before the \"already attempted?\" check below, because\n \t * we want to disable even an already-loaded graph file.\n \t */\n-\tif (r->commit_graph_disabled)\n+\tif (!r->gitdir || r->commit_graph_disabled)\n \t\treturn 0;\n \n \tif (r->objects->commit_graph_attempted)\ndiff --git a/repo-settings.c b/repo-settings.c\nindex b93e91a212e..00ca5571a1a 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n \tchar *strval;\n \tint manyfiles;\n \n+\tif (!r->gitdir)\n+\t\tBUG(\"Cannot add settings for uninitialized repository\");\n+\n \tif (r->settings.initialized++)\n \t\treturn;\n \n-- \ngitgitgadget\n\n"},{"id":"442053","messageId":"5bc5e8465ab5fe871965e6c6d578efc51e55b505.1637620958.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v4.git.1637620958.gitgitgadget@gmail.com","subject":"[PATCH v4 2/4] test-read-cache: set up repo after git directory","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-22T22:42:36Z","receivedAt":"2021-11-22T22:42:50Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nMove repo setup to occur after git directory is set up. This will ensure\nenabling the sparse index for `diff` (and guarding against the nongit\nscenario) will not cause tests to start failing, since that change will include\nadding a check to prepare_repo_settings() with the new BUG.\n\nSigned-off-by: Lessley Dennington \n---\n t/helper/test-read-cache.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c\nindex b52c174acc7..0d9f08931a1 100644\n--- a/t/helper/test-read-cache.c\n+++ b/t/helper/test-read-cache.c\n@@ -39,8 +39,6 @@ int cmd__read_cache(int argc, const char **argv)\n \tint table = 0, expand = 0;\n \n \tinitialize_the_repository();\n-\tprepare_repo_settings(r);\n-\tr->settings.command_requires_full_index = 0;\n \n \tfor (++argv, --argc; *argv && starts_with(*argv, \"--\"); ++argv, --argc) {\n \t\tif (skip_prefix(*argv, \"--print-and-refresh=\", &name))\n@@ -56,6 +54,9 @@ int cmd__read_cache(int argc, const char **argv)\n \tsetup_git_directory();\n \tgit_config(git_default_config, NULL);\n \n+\tprepare_repo_settings(r);\n+\tr->settings.command_requires_full_index = 0;\n+\n \tfor (i = 0; i < cnt; i++) {\n \t\trepo_read_index(r);\n \n-- \ngitgitgadget\n\n"},{"id":"442054","messageId":"273ee16b74ed4b36ffba5762fa892410317ea02b.1637620958.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v4.git.1637620958.gitgitgadget@gmail.com","subject":"[PATCH v4 3/4] diff: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-22T22:42:37Z","receivedAt":"2021-11-22T22:42:51Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index within the 'git diff' command. Its implementation\nalready safely integrates with the sparse index because it shares code\nwith the 'git status' and 'git checkout' commands that were already\nintegrated. For more details see:\n\nd76723e (status: use sparse-index throughout, 2021-07-14)\n1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29)\n\nThe most interesting thing to do is to add tests that verify that 'git\ndiff' behaves correctly when the sparse index is enabled. These cases are:\n\n1. The index is not expanded for 'diff' and 'diff --staged'\n2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\ncheckout, and sparse index repositories in the following partially-staged\nscenarios (i.e. the index, HEAD, and working directory differ at a given\npath):\n 1. Path is within sparse-checkout cone\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n\nThe `p2000` tests demonstrate a ~44% execution time reduction for 'git\ndiff' and a ~86% execution time reduction for 'git diff --staged' using a\nsparse index:\n\nTest before after\n-------------------------------------------------------------\n2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n\nCo-authored-by: Derrick Stolee \nSigned-off-by: Derrick Stolee \nSigned-off-by: Lessley Dennington \n---\n builtin/diff.c | 5 +++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++\n 3 files changed, 53 insertions(+)\n\ndiff --git a/builtin/diff.c b/builtin/diff.c\nindex dd8ce688ba7..fa4683377eb 100644\n--- a/builtin/diff.c\n+++ b/builtin/diff.c\n@@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n+\tif (!nongit) {\n+\t\tprepare_repo_settings(the_repository);\n+\t\tthe_repository->settings.command_requires_full_index = 0;\n+\t}\n+\n \tif (!no_index) {\n \t\t/*\n \t\t * Treat git diff with at least one path outside of the\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bfd332120c8..5cf94627383 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n+test_perf_on_all git diff\n+test_perf_on_all git diff --cached\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 44d5e11c762..53524660759 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -832,6 +832,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n \t)\n '\n \n+test_expect_success 'sparse index is not expanded: diff' '\n+\tinit_repos &&\n+\n+\twrite_script edit-contents <<-\\EOF &&\n+\techo text >>$1\n+\tEOF\n+\n+\t# Add file within cone\n+\ttest_sparse_match git sparse-checkout set deep &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\ttest_all_match git add deep/testfile &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --staged &&\n+\n+\t# Add file outside cone\n+\ttest_all_match git reset --hard &&\n+\trun_on_all mkdir newdirectory &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set newdirectory &&\n+\ttest_all_match git add newdirectory/testfile &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --staged &&\n+\n+\t# Merge conflict outside cone\n+\t# The sparse checkout will report a warning that is not in the\n+\t# full checkout, so we use `run_on_all` instead of\n+\t# `test_all_match`\n+\trun_on_all git reset --hard &&\n+\ttest_all_match git checkout merge-left &&\n+\ttest_all_match test_must_fail git merge merge-right &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --staged &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --staged\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n\n"},{"id":"442055","messageId":"7acf5118bf5602fbafca2d42c4f363b5adbcbd54.1637620958.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v4.git.1637620958.gitgitgadget@gmail.com","subject":"[PATCH v4 4/4] blame: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-11-22T22:42:38Z","receivedAt":"2021-11-22T22:42:53Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index for the 'git blame' command. The index was already\nnot expanded with this command, so the most interesting thing to do is to\nadd tests that verify that 'git blame' behaves correctly when the sparse\nindex is enabled and that its performance improves. More specifically, these\ncases are:\n\n1. The index is not expanded for 'blame' when given paths in the sparse\ncheckout cone at multiple levels.\n\n2. Performance measurably improves for 'blame' with sparse index when given\npaths in the sparse checkout cone at multiple levels.\n\nThe `p2000` tests demonstrate a ~60% execution time reduction when running\n'blame' for a file two levels deep and and a ~30% execution time reduction\nfor a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nWe do not include paths outside the sparse checkout cone because blame\ncurrently does not support blaming files outside of the sparse definition.\nAttempting to do so fails with the following error:\n\n fatal: no such path '' in HEAD\n\nSigned-off-by: Lessley Dennington \n---\n builtin/blame.c | 5 +++\n t/perf/p2000-sparse-operations.sh | 4 +-\n t/t1092-sparse-checkout-compatibility.sh | 49 ++++++++++++++++++------\n 3 files changed, 46 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 641523ff9af..247b9eaf88f 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -902,6 +902,11 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n \tlong anchor;\n \tconst int hexsz = the_hash_algo->hexsz;\n \n+\tif (startup_info->have_repository) {\n+\t\tprepare_repo_settings(the_repository);\n+\t\tthe_repository->settings.command_requires_full_index = 0;\n+\t}\n+\n \tsetup_default_color_by_age();\n \tgit_config(git_blame_config, &output_option);\n \trepo_init_revisions(the_repository, &revs, NULL);\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex 5cf94627383..9ac76a049b8 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -114,6 +114,8 @@ test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n-test_perf_on_all git diff --cached\n+test_perf_on_all git diff --staged\n+test_perf_on_all git blame $SPARSE_CONE/a\n+test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 53524660759..d680dbd8867 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -442,21 +442,36 @@ test_expect_success 'log with pathspec outside sparse definition' '\n test_expect_success 'blame with pathspec inside sparse definition' '\n \tinit_repos &&\n \n-\ttest_all_match git blame a &&\n-\ttest_all_match git blame deep/a &&\n-\ttest_all_match git blame deep/deeper1/a &&\n-\ttest_all_match git blame deep/deeper1/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_all_match git blame $file\n+\tdone\n '\n \n-# TODO: blame currently does not support blaming files outside of the\n-# sparse definition. It complains that the file doesn't exist locally.\n-test_expect_failure 'blame with pathspec outside sparse definition' '\n+# NEEDSWORK: This test documents the current behavior, but this could\n+# change in the future if we decide to support blaming files outside\n+# the sparse definition.\n+test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n+\ttest_sparse_match git sparse-checkout set &&\n \n-\ttest_all_match git blame folder1/a &&\n-\ttest_all_match git blame folder2/a &&\n-\ttest_all_match git blame deep/deeper2/a &&\n-\ttest_all_match git blame deep/deeper2/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_sparse_match test_must_fail git blame $file &&\n+\t\tcat >expect <<-EOF &&\n+\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n+\t\tEOF\n+\t\t# We compare sparse-checkout-err and sparse-index-err in\n+\t\t# `test_sparse_match`. Given we know they are the same, we\n+\t\t# only check the content of sparse-index-err here.\n+\t\ttest_cmp expect sparse-index-err\n+\tdone\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n@@ -878,6 +893,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n \tensure_not_expanded diff --staged\n '\n \n+test_expect_success 'sparse index is not expanded: blame' '\n+\tinit_repos &&\n+\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\tensure_not_expanded blame $file\n+\tdone\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n"},{"id":"442096","messageId":"CABPp-BEM+FpdQ4yJxDcqvdz3LNmFV+5CBMAQdAnEfc0ytbZ-dA@mail.gmail.com","threadId":"56710","inReplyTo":"81e208cf454b61c761fa66e4f43a464ed439ba59.1637620958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-11-23T07:41:37Z","receivedAt":"2021-11-23T07:41:52Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Nov 22, 2021 at 2:42 PM Lessley Dennington via GitGitGadget\n wrote:\n>\n> From: Lessley Dennington \n>\n> Check whether git dir exists before adding any repo settings. If it\n> does not exist, BUG with the message that one cannot add settings for an\n> uninitialized repository. If it does exist, proceed with adding repo\n> settings.\n>\n> Additionally, ensure the above BUG is not triggered when users pass the -h\n> flag by adding a check for the repository to the checkout and pack-objects\n> builtins.\n\nWhy only checkout and pack-objects? Why don't the -h flags to all of\nthe following need it as well?:\n\n$ git grep -l prepare_repo_settings | grep builtin/\nbuiltin/add.c\nbuiltin/blame.c\nbuiltin/checkout.c\nbuiltin/commit.c\nbuiltin/diff.c\nbuiltin/fetch.c\nbuiltin/gc.c\nbuiltin/merge.c\nbuiltin/pack-objects.c\nbuiltin/rebase.c\nbuiltin/reset.c\nbuiltin/revert.c\nbuiltin/sparse-checkout.c\nbuiltin/update-index.c\n\nIf none of these need it, was it because they put\nprepare_repo_settings() calls after some other basic checks had been\ndone so more do not have to be added? If so, is there a similar place\nin checkout and pack-objects where their calls to\nprepare_repo_settings() can be moved? (Looking ahead, it appears you\nmoved some code in patch 2 to do something like this. Are the similar\nmoves that could be done here?)\n\n> Finally, ensure the above BUG is not triggered for commit-graph by\n> returning early if the git directory does not exist.\n\nIf commit-graph needs a special case to avoid triggering the BUG,\nwouldn't several of these need it too?:\n\n$ git grep -l prepare_repo_settings | grep -v builtin/\ncommit-graph.c\nfetch-negotiator.c\nmerge-recursive.c\nmidx.c\nread-cache.c\nrepo-settings.c\nrepository.c\nrepository.h\nsparse-index.c\nt/helper/test-read-cache.c\nt/helper/test-read-graph.c\nunpack-trees.c\n\nor are their calls to prepare_repo_settings() only done after gitdir\nsetup? If the latter, perhaps the commit-graph function calls could\nbe moved after gitdir setup too to avoid the need to do extra checks\nin it?\n\n> Signed-off-by: Lessley Dennington \n> ---\n> builtin/checkout.c | 6 ++++--\n> builtin/pack-objects.c | 9 ++++++---\n> commit-graph.c | 5 ++++-\n> repo-settings.c | 3 +++\n> 4 files changed, 17 insertions(+), 6 deletions(-)\n>\n> diff --git a/builtin/checkout.c b/builtin/checkout.c\n> index 8c69dcdf72a..31632b07888 100644\n> --- a/builtin/checkout.c\n> +++ b/builtin/checkout.c\n> @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n>\n> git_config(git_checkout_config, opts);\n>\n> - prepare_repo_settings(the_repository);\n> - the_repository->settings.command_requires_full_index = 0;\n> + if (startup_info->have_repository) {\n> + prepare_repo_settings(the_repository);\n> + the_repository->settings.command_requires_full_index = 0;\n> + }\n>\n> opts->track = BRANCH_TRACK_UNSPECIFIED;\n>\n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index 1a3dd445f83..45dc2258dc7 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -3976,9 +3976,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n> read_replace_refs = 0;\n>\n> sparse = git_env_bool(\"GIT_TEST_PACK_SPARSE\", -1);\n> - prepare_repo_settings(the_repository);\n> - if (sparse < 0)\n> - sparse = the_repository->settings.pack_use_sparse;\n> +\n> + if (startup_info->have_repository) {\n> + prepare_repo_settings(the_repository);\n> + if (sparse < 0)\n> + sparse = the_repository->settings.pack_use_sparse;\n> + }\n>\n> reset_pack_idx_option(&pack_idx_opts);\n> git_config(git_pack_config, NULL);\n> diff --git a/commit-graph.c b/commit-graph.c\n> index 2706683acfe..265c010122e 100644\n> --- a/commit-graph.c\n> +++ b/commit-graph.c\n> @@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n> struct object_directory *odb;\n>\n> /*\n> + * Early return if there is no git dir or if the commit graph is\n> + * disabled.\n> + *\n> * This must come before the \"already attempted?\" check below, because\n> * we want to disable even an already-loaded graph file.\n> */\n> - if (r->commit_graph_disabled)\n> + if (!r->gitdir || r->commit_graph_disabled)\n> return 0;\n>\n> if (r->objects->commit_graph_attempted)\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b93e91a212e..00ca5571a1a 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n> char *strval;\n> int manyfiles;\n>\n> + if (!r->gitdir)\n> + BUG(\"Cannot add settings for uninitialized repository\");\n> +\n> if (r->settings.initialized++)\n> return;\n\nI'm not what the BUG() is trying to help us catch, but I'm worried\nthat there are many additional places that now need workarounds to\navoid triggering bugs.\n"},{"id":"442097","messageId":"CABPp-BEurZH8RaLLB2vvw3aPCWy4Aq1Zs+PkTY9WFboq8OuZ2w@mail.gmail.com","threadId":"56710","inReplyTo":"273ee16b74ed4b36ffba5762fa892410317ea02b.1637620958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 3/4] diff: enable and test the sparse index","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-11-23T07:47:25Z","receivedAt":"2021-11-23T07:47:43Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Nov 22, 2021 at 2:42 PM Lessley Dennington via GitGitGadget\n wrote:\n>\n> From: Lessley Dennington \n>\n> Enable the sparse index within the 'git diff' command. Its implementation\n> already safely integrates with the sparse index because it shares code\n> with the 'git status' and 'git checkout' commands that were already\n> integrated. For more details see:\n>\n> d76723e (status: use sparse-index throughout, 2021-07-14)\n> 1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29)\n\nI preferred the references in your v3:\n\nd76723ee53 (status: use sparse-index throughout, 2021-07-14)\n1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n\nbecause 7-character abbreviations aren't very future proof;\n10-character seems better to me.\n\n(Very micro nit.)\n\n>\n> The most interesting thing to do is to add tests that verify that 'git\n> diff' behaves correctly when the sparse index is enabled. These cases are:\n>\n> 1. The index is not expanded for 'diff' and 'diff --staged'\n> 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\n> checkout, and sparse index repositories in the following partially-staged\n> scenarios (i.e. the index, HEAD, and working directory differ at a given\n> path):\n> 1. Path is within sparse-checkout cone\n> 2. Path is outside sparse-checkout cone\n> 3. A merge conflict exists for paths outside sparse-checkout cone\n>\n> The `p2000` tests demonstrate a ~44% execution time reduction for 'git\n> diff' and a ~86% execution time reduction for 'git diff --staged' using a\n> sparse index:\n>\n> Test before after\n> -------------------------------------------------------------\n> 2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n> 2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n> 2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n> 2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n> 2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n> 2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n> 2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n> 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n>\n> Co-authored-by: Derrick Stolee \n> Signed-off-by: Derrick Stolee \n> Signed-off-by: Lessley Dennington \n> ---\n> builtin/diff.c | 5 +++\n> t/perf/p2000-sparse-operations.sh | 2 ++\n> t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++\n> 3 files changed, 53 insertions(+)\n>\n> diff --git a/builtin/diff.c b/builtin/diff.c\n> index dd8ce688ba7..fa4683377eb 100644\n> --- a/builtin/diff.c\n> +++ b/builtin/diff.c\n> @@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n>\n> prefix = setup_git_directory_gently(&nongit);\n>\n> + if (!nongit) {\n> + prepare_repo_settings(the_repository);\n> + the_repository->settings.command_requires_full_index = 0;\n> + }\n> +\n> if (!no_index) {\n> /*\n> * Treat git diff with at least one path outside of the\n> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n> index bfd332120c8..5cf94627383 100755\n> --- a/t/perf/p2000-sparse-operations.sh\n> +++ b/t/perf/p2000-sparse-operations.sh\n> @@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n> test_perf_on_all git reset\n> test_perf_on_all git reset --hard\n> test_perf_on_all git reset -- does-not-exist\n> +test_perf_on_all git diff\n> +test_perf_on_all git diff --cached\n>\n> test_done\n> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n> index 44d5e11c762..53524660759 100755\n> --- a/t/t1092-sparse-checkout-compatibility.sh\n> +++ b/t/t1092-sparse-checkout-compatibility.sh\n> @@ -832,6 +832,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n> )\n> '\n>\n> +test_expect_success 'sparse index is not expanded: diff' '\n> + init_repos &&\n> +\n> + write_script edit-contents <<-\\EOF &&\n> + echo text >>$1\n> + EOF\n> +\n> + # Add file within cone\n> + test_sparse_match git sparse-checkout set deep &&\n> + run_on_all ../edit-contents deep/testfile &&\n> + test_all_match git add deep/testfile &&\n> + run_on_all ../edit-contents deep/testfile &&\n> +\n> + test_all_match git diff &&\n> + test_all_match git diff --staged &&\n> + ensure_not_expanded diff &&\n> + ensure_not_expanded diff --staged &&\n> +\n> + # Add file outside cone\n> + test_all_match git reset --hard &&\n> + run_on_all mkdir newdirectory &&\n> + run_on_all ../edit-contents newdirectory/testfile &&\n> + test_sparse_match git sparse-checkout set newdirectory &&\n> + test_all_match git add newdirectory/testfile &&\n> + run_on_all ../edit-contents newdirectory/testfile &&\n> + test_sparse_match git sparse-checkout set &&\n> +\n> + test_all_match git diff &&\n> + test_all_match git diff --staged &&\n> + ensure_not_expanded diff &&\n> + ensure_not_expanded diff --staged &&\n> +\n> + # Merge conflict outside cone\n> + # The sparse checkout will report a warning that is not in the\n> + # full checkout, so we use `run_on_all` instead of\n> + # `test_all_match`\n> + run_on_all git reset --hard &&\n> + test_all_match git checkout merge-left &&\n> + test_all_match test_must_fail git merge merge-right &&\n> +\n> + test_all_match git diff &&\n> + test_all_match git diff --staged &&\n> + ensure_not_expanded diff &&\n> + ensure_not_expanded diff --staged\n\nYou've changed some of the --staged to --cached, but based on Junio's\ncomments on the previous round, you probably want to convert the\nothers too.\n\n> +'\n> +\n> # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n> # in this scenario, but it shouldn't.\n> test_expect_success 'reset mixed and checkout orphan' '\n> --\n> gitgitgadget\n>\n"},{"id":"442098","messageId":"CABPp-BGM-57Xxxb-MRvkkgGFbVn4z6PEZFKryuBCwj7Ymr_oSA@mail.gmail.com","threadId":"56710","inReplyTo":"ddcee003c9292d1b4cdd64a97f9e1731a72faee8.1634232353.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/2] blame: enable and test the sparse index","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-11-23T07:57:55Z","receivedAt":"2021-11-23T07:58:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Oct 14, 2021 at 10:25 AM Lessley Dennington via GitGitGadget\n wrote:\n>\n> From: Lessley Dennington \n>\n> Enable the sparse index for the 'git blame' command. The index was already\n> not expanded with this command, so the most interesting thing to do is to\n> add tests that verify that 'git blame' behaves correctly when the sparse\n> index is enabled and that its performance improves. More specifically, these\n> cases are:\n>\n> 1. The index is not expanded for 'blame' when given paths in the sparse\n> checkout cone at multiple levels.\n>\n> 2. Performance measurably improves for 'blame' with sparse index when given\n> paths in the sparse checkout cone at multiple levels.\n>\n> The `p2000` tests demonstrate a ~60% execution time reduction when running\n> 'blame' for a file two levels deep and and a ~30% execution time reduction\n> for a file three levels deep.\n>\n> Test before after\n> ----------------------------------------------------------------\n> 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n> 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n> 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n> 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n> 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n> 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n> 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n> 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nLooks good.\n\n> We do not include paths outside the sparse checkout cone because blame\n> currently does not support blaming files outside of the sparse definition.\n> Attempting to do so fails with the following error:\n>\n> fatal: no such path '' in HEAD\n\nWhile technically accurate, this wording is misleading; it implies\nthat there is something unique to sparse checkouts, and perhaps even\nto cone mode, affecting how blame handles files not in the working\ndirectory. That's not true, though; git blame without a revision has\nalways reported an error when given a file that does not exist in the\nworking tree. Try this in git.git:\n\n$ rm t/README\n$ git blame t/README\nfatal: Cannot lstat 't/README': No such file or directory\n\nThe reason is that with no revisions, calling git blame with a\nfilename means asking the question \"Which commit did each line in that\nfile come from?\" If there's no file, the question just doesn't make\nsense. You could make sense of it by thinking in terms of some\nrevision of the file, but then you're passing a revision along --\nwhich works just fine in a sparse checkout too.\n\n>\n> Signed-off-by: Lessley Dennington \n> ---\n> builtin/blame.c | 3 +++\n> t/perf/p2000-sparse-operations.sh | 2 ++\n> t/t1092-sparse-checkout-compatibility.sh | 24 +++++++++++++++++-------\n> 3 files changed, 22 insertions(+), 7 deletions(-)\n>\n> diff --git a/builtin/blame.c b/builtin/blame.c\n> index 641523ff9af..af3d81e2bd4 100644\n> --- a/builtin/blame.c\n> +++ b/builtin/blame.c\n> @@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n> long anchor;\n> const int hexsz = the_hash_algo->hexsz;\n>\n> + prepare_repo_settings(the_repository);\n> + the_repository->settings.command_requires_full_index = 0;\n> +\n> setup_default_color_by_age();\n> git_config(git_blame_config, &output_option);\n> repo_init_revisions(the_repository, &revs, NULL);\n> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n> index bff93f16e93..9ac76a049b8 100755\n> --- a/t/perf/p2000-sparse-operations.sh\n> +++ b/t/perf/p2000-sparse-operations.sh\n> @@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n> test_perf_on_all git reset -- does-not-exist\n> test_perf_on_all git diff\n> test_perf_on_all git diff --staged\n> +test_perf_on_all git blame $SPARSE_CONE/a\n> +test_perf_on_all git blame $SPARSE_CONE/f3/a\n>\n> test_done\n> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n> index 1070bff1a83..54826e858a9 100755\n> --- a/t/t1092-sparse-checkout-compatibility.sh\n> +++ b/t/t1092-sparse-checkout-compatibility.sh\n> @@ -485,15 +485,16 @@ test_expect_success 'blame with pathspec inside sparse definition' '\n> test_all_match git blame deep/deeper1/deepest/a\n> '\n>\n> -# TODO: blame currently does not support blaming files outside of the\n> -# sparse definition. It complains that the file doesn't exist locally.\n> -test_expect_failure 'blame with pathspec outside sparse definition' '\n> +# Blame does not support blaming files outside of the sparse\n> +# definition, so we verify this scenario.\n\nAs above, this is misleading. It'd be better to word it something like:\n\n# Without a revision specified, blame will error if passed any file that\n# is not present in the working directory (even if the file is tracked).\n# Here we just verify that this is also true with sparse checkouts.\n\n> +test_expect_success 'blame with pathspec outside sparse definition' '\n> init_repos &&\n>\n> - test_all_match git blame folder1/a &&\n> - test_all_match git blame folder2/a &&\n> - test_all_match git blame deep/deeper2/a &&\n> - test_all_match git blame deep/deeper2/deepest/a\n> + test_sparse_match git sparse-checkout set &&\n> + test_sparse_match test_must_fail git blame folder1/a &&\n> + test_sparse_match test_must_fail git blame folder2/a &&\n> + test_sparse_match test_must_fail git blame deep/deeper2/a &&\n> + test_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n> '\n>\n> test_expect_success 'checkout and reset (mixed)' '\n> @@ -871,6 +872,15 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n> )\n> '\n>\n> +test_expect_success 'sparse index is not expanded: blame' '\n> + init_repos &&\n> +\n> + ensure_not_expanded blame a &&\n> + ensure_not_expanded blame deep/a &&\n> + ensure_not_expanded blame deep/deeper1/a &&\n> + ensure_not_expanded blame deep/deeper1/deepest/a\n> +'\n> +\n> # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n> # in this scenario, but it shouldn't.\n> test_expect_success 'reset mixed and checkout orphan' '\n> --\n> gitgitgadget\n"},{"id":"442152","messageId":"cd9f20aa-0fff-6749-eaec-0527cf4cf4cc@gmail.com","threadId":"56710","inReplyTo":"CABPp-BEM+FpdQ4yJxDcqvdz3LNmFV+5CBMAQdAnEfc0ytbZ-dA@mail.gmail.com","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-23T14:52:49Z","receivedAt":"2021-11-23T14:52:54Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/22/21 11:41 PM, Elijah Newren wrote:\n> On Mon, Nov 22, 2021 at 2:42 PM Lessley Dennington via GitGitGadget\n> wrote:\n>>\n>> From: Lessley Dennington \n>>\n>> Check whether git dir exists before adding any repo settings. If it\n>> does not exist, BUG with the message that one cannot add settings for an\n>> uninitialized repository. If it does exist, proceed with adding repo\n>> settings.\n>>\n>> Additionally, ensure the above BUG is not triggered when users pass the -h\n>> flag by adding a check for the repository to the checkout and pack-objects\n>> builtins.\n> \n> Why only checkout and pack-objects? Why don't the -h flags to all of\n> the following need it as well?:\n> \n> $ git grep -l prepare_repo_settings | grep builtin/\n> builtin/add.c\n> builtin/blame.c\n> builtin/checkout.c\n> builtin/commit.c\n> builtin/diff.c\n> builtin/fetch.c\n> builtin/gc.c\n> builtin/merge.c\n> builtin/pack-objects.c\n> builtin/rebase.c\n> builtin/reset.c\n> builtin/revert.c\n> builtin/sparse-checkout.c\n> builtin/update-index.c\n> \n> If none of these need it, was it because they put\n> prepare_repo_settings() calls after some other basic checks had been\n> done so more do not have to be added? If so, is there a similar place\n> in checkout and pack-objects where their calls to\n> prepare_repo_settings() can be moved? (Looking ahead, it appears you\n> moved some code in patch 2 to do something like this. Are the similar\n> moves that could be done here?)\n> \nThank you for the quick feedback. Yes, I believe there are similar moves \nthat can be done here. I was attempting to be explicit about the case I'm \nguarding against, but you're right - it shouldn't be done one way for some \nbuiltins and another way for others.\n>> Finally, ensure the above BUG is not triggered for commit-graph by\n>> returning early if the git directory does not exist.\n> \n> If commit-graph needs a special case to avoid triggering the BUG,\n> wouldn't several of these need it too?:\n> \n> $ git grep -l prepare_repo_settings | grep -v builtin/\n> commit-graph.c\n> fetch-negotiator.c\n> merge-recursive.c\n> midx.c\n> read-cache.c\n> repo-settings.c\n> repository.c\n> repository.h\n> sparse-index.c\n> t/helper/test-read-cache.c\n> t/helper/test-read-graph.c\n> unpack-trees.c\n> \n> or are their calls to prepare_repo_settings() only done after gitdir\n> setup? If the latter, perhaps the commit-graph function calls could\n> be moved after gitdir setup too to avoid the need to do extra checks\n> in it?\n> \nAgreed, I can move this as well.\n>> Signed-off-by: Lessley Dennington \n>> ---\n>> builtin/checkout.c | 6 ++++--\n>> builtin/pack-objects.c | 9 ++++++---\n>> commit-graph.c | 5 ++++-\n>> repo-settings.c | 3 +++\n>> 4 files changed, 17 insertions(+), 6 deletions(-)\n>>\n>> diff --git a/builtin/checkout.c b/builtin/checkout.c\n>> index 8c69dcdf72a..31632b07888 100644\n>> --- a/builtin/checkout.c\n>> +++ b/builtin/checkout.c\n>> @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n>>\n>> git_config(git_checkout_config, opts);\n>>\n>> - prepare_repo_settings(the_repository);\n>> - the_repository->settings.command_requires_full_index = 0;\n>> + if (startup_info->have_repository) {\n>> + prepare_repo_settings(the_repository);\n>> + the_repository->settings.command_requires_full_index = 0;\n>> + }\n>>\n>> opts->track = BRANCH_TRACK_UNSPECIFIED;\n>>\n>> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n>> index 1a3dd445f83..45dc2258dc7 100644\n>> --- a/builtin/pack-objects.c\n>> +++ b/builtin/pack-objects.c\n>> @@ -3976,9 +3976,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n>> read_replace_refs = 0;\n>>\n>> sparse = git_env_bool(\"GIT_TEST_PACK_SPARSE\", -1);\n>> - prepare_repo_settings(the_repository);\n>> - if (sparse < 0)\n>> - sparse = the_repository->settings.pack_use_sparse;\n>> +\n>> + if (startup_info->have_repository) {\n>> + prepare_repo_settings(the_repository);\n>> + if (sparse < 0)\n>> + sparse = the_repository->settings.pack_use_sparse;\n>> + }\n>>\n>> reset_pack_idx_option(&pack_idx_opts);\n>> git_config(git_pack_config, NULL);\n>> diff --git a/commit-graph.c b/commit-graph.c\n>> index 2706683acfe..265c010122e 100644\n>> --- a/commit-graph.c\n>> +++ b/commit-graph.c\n>> @@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n>> struct object_directory *odb;\n>>\n>> /*\n>> + * Early return if there is no git dir or if the commit graph is\n>> + * disabled.\n>> + *\n>> * This must come before the \"already attempted?\" check below, because\n>> * we want to disable even an already-loaded graph file.\n>> */\n>> - if (r->commit_graph_disabled)\n>> + if (!r->gitdir || r->commit_graph_disabled)\n>> return 0;\n>>\n>> if (r->objects->commit_graph_attempted)\n>> diff --git a/repo-settings.c b/repo-settings.c\n>> index b93e91a212e..00ca5571a1a 100644\n>> --- a/repo-settings.c\n>> +++ b/repo-settings.c\n>> @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n>> char *strval;\n>> int manyfiles;\n>>\n>> + if (!r->gitdir)\n>> + BUG(\"Cannot add settings for uninitialized repository\");\n>> +\n>> if (r->settings.initialized++)\n>> return;\n> \n> I'm not what the BUG() is trying to help us catch, but I'm worried\n> that there are many additional places that now need workarounds to\n> avoid triggering bugs.\n> \nI see your point. We're trying to make sure we catch issues like the \nnongit scenario I overlooked in diff in earlier versions of this series. \nBut if we feel this change is too disruptive and will likely cause issues \nbeyond those I've fixed to ensure our tests pass, I can remove.\n"},{"id":"442153","messageId":"f86c920b-6b56-27ca-fba5-a54c5c15c765@gmail.com","threadId":"56710","inReplyTo":"CABPp-BEurZH8RaLLB2vvw3aPCWy4Aq1Zs+PkTY9WFboq8OuZ2w@mail.gmail.com","subject":"Re: [PATCH v4 3/4] diff: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-23T14:53:55Z","receivedAt":"2021-11-23T14:54:00Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/22/21 11:47 PM, Elijah Newren wrote:\n> On Mon, Nov 22, 2021 at 2:42 PM Lessley Dennington via GitGitGadget\n> wrote:\n>>\n>> From: Lessley Dennington \n>>\n>> Enable the sparse index within the 'git diff' command. Its implementation\n>> already safely integrates with the sparse index because it shares code\n>> with the 'git status' and 'git checkout' commands that were already\n>> integrated. For more details see:\n>>\n>> d76723e (status: use sparse-index throughout, 2021-07-14)\n>> 1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29)\n> \n> I preferred the references in your v3:\n> \n> d76723ee53 (status: use sparse-index throughout, 2021-07-14)\n> 1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n> \n> because 7-character abbreviations aren't very future proof;\n> 10-character seems better to me.\n> \n> (Very micro nit.)\n> \nAppreciate the pointer - I'll return to the old formatting for v5.\n>>\n>> The most interesting thing to do is to add tests that verify that 'git\n>> diff' behaves correctly when the sparse index is enabled. These cases are:\n>>\n>> 1. The index is not expanded for 'diff' and 'diff --staged'\n>> 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\n>> checkout, and sparse index repositories in the following partially-staged\n>> scenarios (i.e. the index, HEAD, and working directory differ at a given\n>> path):\n>> 1. Path is within sparse-checkout cone\n>> 2. Path is outside sparse-checkout cone\n>> 3. A merge conflict exists for paths outside sparse-checkout cone\n>>\n>> The `p2000` tests demonstrate a ~44% execution time reduction for 'git\n>> diff' and a ~86% execution time reduction for 'git diff --staged' using a\n>> sparse index:\n>>\n>> Test before after\n>> -------------------------------------------------------------\n>> 2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n>> 2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n>> 2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n>> 2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n>> 2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n>> 2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n>> 2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n>> 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n>>\n>> Co-authored-by: Derrick Stolee \n>> Signed-off-by: Derrick Stolee \n>> Signed-off-by: Lessley Dennington \n>> ---\n>> builtin/diff.c | 5 +++\n>> t/perf/p2000-sparse-operations.sh | 2 ++\n>> t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++\n>> 3 files changed, 53 insertions(+)\n>>\n>> diff --git a/builtin/diff.c b/builtin/diff.c\n>> index dd8ce688ba7..fa4683377eb 100644\n>> --- a/builtin/diff.c\n>> +++ b/builtin/diff.c\n>> @@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n>>\n>> prefix = setup_git_directory_gently(&nongit);\n>>\n>> + if (!nongit) {\n>> + prepare_repo_settings(the_repository);\n>> + the_repository->settings.command_requires_full_index = 0;\n>> + }\n>> +\n>> if (!no_index) {\n>> /*\n>> * Treat git diff with at least one path outside of the\n>> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n>> index bfd332120c8..5cf94627383 100755\n>> --- a/t/perf/p2000-sparse-operations.sh\n>> +++ b/t/perf/p2000-sparse-operations.sh\n>> @@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n>> test_perf_on_all git reset\n>> test_perf_on_all git reset --hard\n>> test_perf_on_all git reset -- does-not-exist\n>> +test_perf_on_all git diff\n>> +test_perf_on_all git diff --cached\n>>\n>> test_done\n>> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n>> index 44d5e11c762..53524660759 100755\n>> --- a/t/t1092-sparse-checkout-compatibility.sh\n>> +++ b/t/t1092-sparse-checkout-compatibility.sh\n>> @@ -832,6 +832,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n>> )\n>> '\n>>\n>> +test_expect_success 'sparse index is not expanded: diff' '\n>> + init_repos &&\n>> +\n>> + write_script edit-contents <<-\\EOF &&\n>> + echo text >>$1\n>> + EOF\n>> +\n>> + # Add file within cone\n>> + test_sparse_match git sparse-checkout set deep &&\n>> + run_on_all ../edit-contents deep/testfile &&\n>> + test_all_match git add deep/testfile &&\n>> + run_on_all ../edit-contents deep/testfile &&\n>> +\n>> + test_all_match git diff &&\n>> + test_all_match git diff --staged &&\n>> + ensure_not_expanded diff &&\n>> + ensure_not_expanded diff --staged &&\n>> +\n>> + # Add file outside cone\n>> + test_all_match git reset --hard &&\n>> + run_on_all mkdir newdirectory &&\n>> + run_on_all ../edit-contents newdirectory/testfile &&\n>> + test_sparse_match git sparse-checkout set newdirectory &&\n>> + test_all_match git add newdirectory/testfile &&\n>> + run_on_all ../edit-contents newdirectory/testfile &&\n>> + test_sparse_match git sparse-checkout set &&\n>> +\n>> + test_all_match git diff &&\n>> + test_all_match git diff --staged &&\n>> + ensure_not_expanded diff &&\n>> + ensure_not_expanded diff --staged &&\n>> +\n>> + # Merge conflict outside cone\n>> + # The sparse checkout will report a warning that is not in the\n>> + # full checkout, so we use `run_on_all` instead of\n>> + # `test_all_match`\n>> + run_on_all git reset --hard &&\n>> + test_all_match git checkout merge-left &&\n>> + test_all_match test_must_fail git merge merge-right &&\n>> +\n>> + test_all_match git diff &&\n>> + test_all_match git diff --staged &&\n>> + ensure_not_expanded diff &&\n>> + ensure_not_expanded diff --staged\n> \n> You've changed some of the --staged to --cached, but based on Junio's\n> comments on the previous round, you probably want to convert the\n> others too.\n> \nWill do for v5.\n>> +'\n>> +\n>> # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n>> # in this scenario, but it shouldn't.\n>> test_expect_success 'reset mixed and checkout orphan' '\n>> --\n>> gitgitgadget\n>>\n"},{"id":"442154","messageId":"2ac61ff8-816d-99b4-333b-381d9a1dbdeb@gmail.com","threadId":"56710","inReplyTo":"CABPp-BGM-57Xxxb-MRvkkgGFbVn4z6PEZFKryuBCwj7Ymr_oSA@mail.gmail.com","subject":"Re: [PATCH 2/2] blame: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-23T14:57:59Z","receivedAt":"2021-11-23T14:58:03Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/22/21 11:57 PM, Elijah Newren wrote:\n> On Thu, Oct 14, 2021 at 10:25 AM Lessley Dennington via GitGitGadget\n> wrote:\n>>\n>> From: Lessley Dennington \n>>\n>> Enable the sparse index for the 'git blame' command. The index was already\n>> not expanded with this command, so the most interesting thing to do is to\n>> add tests that verify that 'git blame' behaves correctly when the sparse\n>> index is enabled and that its performance improves. More specifically, these\n>> cases are:\n>>\n>> 1. The index is not expanded for 'blame' when given paths in the sparse\n>> checkout cone at multiple levels.\n>>\n>> 2. Performance measurably improves for 'blame' with sparse index when given\n>> paths in the sparse checkout cone at multiple levels.\n>>\n>> The `p2000` tests demonstrate a ~60% execution time reduction when running\n>> 'blame' for a file two levels deep and and a ~30% execution time reduction\n>> for a file three levels deep.\n>>\n>> Test before after\n>> ----------------------------------------------------------------\n>> 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n>> 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n>> 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n>> 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n>> 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n>> 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n>> 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n>> 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n> \n> Looks good.\n> \n>> We do not include paths outside the sparse checkout cone because blame\n>> currently does not support blaming files outside of the sparse definition.\n>> Attempting to do so fails with the following error:\n>>\n>> fatal: no such path '' in HEAD\n> \n> While technically accurate, this wording is misleading; it implies\n> that there is something unique to sparse checkouts, and perhaps even\n> to cone mode, affecting how blame handles files not in the working\n> directory. That's not true, though; git blame without a revision has\n> always reported an error when given a file that does not exist in the\n> working tree. Try this in git.git:\n> \n> $ rm t/README\n> $ git blame t/README\n> fatal: Cannot lstat 't/README': No such file or directory\n> \n> The reason is that with no revisions, calling git blame with a\n> filename means asking the question \"Which commit did each line in that\n> file come from?\" If there's no file, the question just doesn't make\n> sense. You could make sense of it by thinking in terms of some\n> revision of the file, but then you're passing a revision along --\n> which works just fine in a sparse checkout too.\n> \nThank you for clarifying that this is actually the expected behavior and \nisn't something we need to \"fix\" for sparse-checkout. I will update \naccordingly for v5.\n>>\n>> Signed-off-by: Lessley Dennington \n>> ---\n>> builtin/blame.c | 3 +++\n>> t/perf/p2000-sparse-operations.sh | 2 ++\n>> t/t1092-sparse-checkout-compatibility.sh | 24 +++++++++++++++++-------\n>> 3 files changed, 22 insertions(+), 7 deletions(-)\n>>\n>> diff --git a/builtin/blame.c b/builtin/blame.c\n>> index 641523ff9af..af3d81e2bd4 100644\n>> --- a/builtin/blame.c\n>> +++ b/builtin/blame.c\n>> @@ -902,6 +902,9 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n>> long anchor;\n>> const int hexsz = the_hash_algo->hexsz;\n>>\n>> + prepare_repo_settings(the_repository);\n>> + the_repository->settings.command_requires_full_index = 0;\n>> +\n>> setup_default_color_by_age();\n>> git_config(git_blame_config, &output_option);\n>> repo_init_revisions(the_repository, &revs, NULL);\n>> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n>> index bff93f16e93..9ac76a049b8 100755\n>> --- a/t/perf/p2000-sparse-operations.sh\n>> +++ b/t/perf/p2000-sparse-operations.sh\n>> @@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n>> test_perf_on_all git reset -- does-not-exist\n>> test_perf_on_all git diff\n>> test_perf_on_all git diff --staged\n>> +test_perf_on_all git blame $SPARSE_CONE/a\n>> +test_perf_on_all git blame $SPARSE_CONE/f3/a\n>>\n>> test_done\n>> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\n>> index 1070bff1a83..54826e858a9 100755\n>> --- a/t/t1092-sparse-checkout-compatibility.sh\n>> +++ b/t/t1092-sparse-checkout-compatibility.sh\n>> @@ -485,15 +485,16 @@ test_expect_success 'blame with pathspec inside sparse definition' '\n>> test_all_match git blame deep/deeper1/deepest/a\n>> '\n>>\n>> -# TODO: blame currently does not support blaming files outside of the\n>> -# sparse definition. It complains that the file doesn't exist locally.\n>> -test_expect_failure 'blame with pathspec outside sparse definition' '\n>> +# Blame does not support blaming files outside of the sparse\n>> +# definition, so we verify this scenario.\n> \n> As above, this is misleading. It'd be better to word it something like:\n> \n> # Without a revision specified, blame will error if passed any file that\n> # is not present in the working directory (even if the file is tracked).\n> # Here we just verify that this is also true with sparse checkouts.\n> \nThank you. Will update for v5.\n>> +test_expect_success 'blame with pathspec outside sparse definition' '\n>> init_repos &&\n>>\n>> - test_all_match git blame folder1/a &&\n>> - test_all_match git blame folder2/a &&\n>> - test_all_match git blame deep/deeper2/a &&\n>> - test_all_match git blame deep/deeper2/deepest/a\n>> + test_sparse_match git sparse-checkout set &&\n>> + test_sparse_match test_must_fail git blame folder1/a &&\n>> + test_sparse_match test_must_fail git blame folder2/a &&\n>> + test_sparse_match test_must_fail git blame deep/deeper2/a &&\n>> + test_sparse_match test_must_fail git blame deep/deeper2/deepest/a\n>> '\n>>\n>> test_expect_success 'checkout and reset (mixed)' '\n>> @@ -871,6 +872,15 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n>> )\n>> '\n>>\n>> +test_expect_success 'sparse index is not expanded: blame' '\n>> + init_repos &&\n>> +\n>> + ensure_not_expanded blame a &&\n>> + ensure_not_expanded blame deep/a &&\n>> + ensure_not_expanded blame deep/deeper1/a &&\n>> + ensure_not_expanded blame deep/deeper1/deepest/a\n>> +'\n>> +\n>> # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n>> # in this scenario, but it shouldn't.\n>> test_expect_success 'reset mixed and checkout orphan' '\n>> --\n>> gitgitgadget\n"},{"id":"442210","messageId":"xmqq8rxe787g.fsf@gitster.g","threadId":"56710","inReplyTo":"81e208cf454b61c761fa66e4f43a464ed439ba59.1637620958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-23T23:39:31Z","receivedAt":"2021-11-23T23:39:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> From: Lessley Dennington \n>\n> Check whether git dir exists before adding any repo settings. If it\n> does not exist, BUG with the message that one cannot add settings for an\n> uninitialized repository. If it does exist, proceed with adding repo\n> settings.\n>\n> Additionally, ensure the above BUG is not triggered when users pass the -h\n> flag by adding a check for the repository to the checkout and pack-objects\n> builtins.\n>\n> Finally, ensure the above BUG is not triggered for commit-graph by\n> returning early if the git directory does not exist.\n>\n> Signed-off-by: Lessley Dennington \n> ---\n> builtin/checkout.c | 6 ++++--\n> builtin/pack-objects.c | 9 ++++++---\n> commit-graph.c | 5 ++++-\n> repo-settings.c | 3 +++\n> 4 files changed, 17 insertions(+), 6 deletions(-)\n>\n> diff --git a/builtin/checkout.c b/builtin/checkout.c\n> index 8c69dcdf72a..31632b07888 100644\n> --- a/builtin/checkout.c\n> +++ b/builtin/checkout.c\n> @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n> \n> \tgit_config(git_checkout_config, opts);\n> \n> -\tprepare_repo_settings(the_repository);\n> -\tthe_repository->settings.command_requires_full_index = 0;\n> +\tif (startup_info->have_repository) {\n> +\t\tprepare_repo_settings(the_repository);\n> +\t\tthe_repository->settings.command_requires_full_index = 0;\n> +\t}\n\nI am kind-a surprised if the control reaches this deep if you are\nnot in a repository. In git.c::commands[] list, all three primary\nentry points that call checkout_main(), namely, cmd_checkout(),\ncmd_restore(), and cmd_switch(), are marked with RUN_SETUP bit,\nwhich makes us call setup_git_directory() even before we call the\ncmd_X() function. And setup_git_directory() dies with \"not a git\nrepository (or any of the parent directories)\" outside a repository.\n\nSo, how can startup_info->have_repository bit be false if the\ncontrol flow reaches here? Or am I grossly misunderstanding what\nyou are trying to do?\n\n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index 1a3dd445f83..45dc2258dc7 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -3976,9 +3976,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n> \tread_replace_refs = 0;\n\nDitto wrt RUN_SETUP.\n\n> diff --git a/commit-graph.c b/commit-graph.c\n> index 2706683acfe..265c010122e 100644\n> --- a/commit-graph.c\n> +++ b/commit-graph.c\n> @@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n> \tstruct object_directory *odb;\n> \t/*\n> +\t * Early return if there is no git dir or if the commit graph is\n> +\t * disabled.\n> +\t *\n> \t * This must come before the \"already attempted?\" check below, because\n> \t * we want to disable even an already-loaded graph file.\n> \t */\n> -\tif (r->commit_graph_disabled)\n> +\tif (!r->gitdir || r->commit_graph_disabled)\n> \t\treturn 0;\n\nI haven't followed the control flow, but this one probably is a good\naddition (in other words, unlike cmd_pack_objects(), I cannot convince\nmyself that r->gitdir will never be NULL here).\n\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b93e91a212e..00ca5571a1a 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n> \tchar *strval;\n> \tint manyfiles;\n> \n> +\tif (!r->gitdir)\n> +\t\tBUG(\"Cannot add settings for uninitialized repository\");\n> +\n\nThis is a very good idea. If I recall correctly, I think I reviewed\na bugfix patch that would have been simplified if we had this check\nhere.\n"},{"id":"442211","messageId":"xmqq4k82781w.fsf@gitster.g","threadId":"56710","inReplyTo":"5bc5e8465ab5fe871965e6c6d578efc51e55b505.1637620958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 2/4] test-read-cache: set up repo after git directory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-23T23:42:51Z","receivedAt":"2021-11-23T23:42:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> From: Lessley Dennington \n>\n> Move repo setup to occur after git directory is set up. This will ensure\n> enabling the sparse index for `diff` (and guarding against the nongit\n> scenario) will not cause tests to start failing, since that change will include\n> adding a check to prepare_repo_settings() with the new BUG.\n\nThis looks obviously the right thing to do. Would anything break\nbecause of the \"wrong\" ordering of events in the original code?\n\nIOW, can this \"bugfix\" be protected with a new test against\nregression?\n\n> Signed-off-by: Lessley Dennington \n> ---\n> t/helper/test-read-cache.c | 5 +++--\n> 1 file changed, 3 insertions(+), 2 deletions(-)\n>\n> diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c\n> index b52c174acc7..0d9f08931a1 100644\n> --- a/t/helper/test-read-cache.c\n> +++ b/t/helper/test-read-cache.c\n> @@ -39,8 +39,6 @@ int cmd__read_cache(int argc, const char **argv)\n> \tint table = 0, expand = 0;\n> \n> \tinitialize_the_repository();\n> -\tprepare_repo_settings(r);\n> -\tr->settings.command_requires_full_index = 0;\n> \n> \tfor (++argv, --argc; *argv && starts_with(*argv, \"--\"); ++argv, --argc) {\n> \t\tif (skip_prefix(*argv, \"--print-and-refresh=\", &name))\n> @@ -56,6 +54,9 @@ int cmd__read_cache(int argc, const char **argv)\n> \tsetup_git_directory();\n> \tgit_config(git_default_config, NULL);\n> \n> +\tprepare_repo_settings(r);\n> +\tr->settings.command_requires_full_index = 0;\n> +\n> \tfor (i = 0; i < cnt; i++) {\n> \t\trepo_read_index(r);\n"},{"id":"442213","messageId":"xmqqzgpu5t83.fsf@gitster.g","threadId":"56710","inReplyTo":"273ee16b74ed4b36ffba5762fa892410317ea02b.1637620958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 3/4] diff: enable and test the sparse index","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-23T23:48:28Z","receivedAt":"2021-11-23T23:48:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> diff --git a/builtin/diff.c b/builtin/diff.c\n> index dd8ce688ba7..fa4683377eb 100644\n> --- a/builtin/diff.c\n> +++ b/builtin/diff.c\n> @@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n> \n> \tprefix = setup_git_directory_gently(&nongit);\n> \n> +\tif (!nongit) {\n> +\t\tprepare_repo_settings(the_repository);\n> +\t\tthe_repository->settings.command_requires_full_index = 0;\n> +\t}\n> +\n\nIt is very pleasing to see that with all the preparations, it is\njsut a very simple matter of adding four lines to enable the big\nfeature.\n\nAt this point immediately after asking setup_git_directory_gently(),\nwe positively know if we are in a repository, so the placement of\nthe new code makes perfect sense.\n\nNice.\n\n\n"},{"id":"442214","messageId":"xmqqr1b65sz1.fsf@gitster.g","threadId":"56710","inReplyTo":"7acf5118bf5602fbafca2d42c4f363b5adbcbd54.1637620958.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 4/4] blame: enable and test the sparse index","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-23T23:53:54Z","receivedAt":"2021-11-23T23:54:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> diff --git a/builtin/blame.c b/builtin/blame.c\n> index 641523ff9af..247b9eaf88f 100644\n> --- a/builtin/blame.c\n> +++ b/builtin/blame.c\n> @@ -902,6 +902,11 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n> \tlong anchor;\n> \tconst int hexsz = the_hash_algo->hexsz;\n> \n> +\tif (startup_info->have_repository) {\n\nThe command is marked with RUN_SETUP bit in git.c::commands[] list,\nso I would think we wouldn't even get called if we are not in a\nrepository here.\n\nUnder what condition can startup_info->have_repository be false at\nthis point in the control flow? If there is such a case, it would\nmean that startup_info->have_repository bit can be false even if we\nare in a repository. That sounds like a bug in some code (I do not\nknow where offhand) that is supposed to prepare the startup_info\nbefore cmd_X() functions are called.\n\n> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n> index 5cf94627383..9ac76a049b8 100755\n> --- a/t/perf/p2000-sparse-operations.sh\n> +++ b/t/perf/p2000-sparse-operations.sh\n> @@ -114,6 +114,8 @@ test_perf_on_all git reset\n> test_perf_on_all git reset --hard\n> test_perf_on_all git reset -- does-not-exist\n> test_perf_on_all git diff\n> -test_perf_on_all git diff --cached\n> +test_perf_on_all git diff --staged\n\nThat's a funny revert of what the previous step did; I thought this\nstep was about \"blame\" and not \"diff\".\n\n> +test_perf_on_all git blame $SPARSE_CONE/a\n> +test_perf_on_all git blame $SPARSE_CONE/f3/a\n> \n> test_done\n> -# TODO: blame currently does not support blaming files outside of the\n> -# sparse definition. It complains that the file doesn't exist locally.\n> -test_expect_failure 'blame with pathspec outside sparse definition' '\n> +# NEEDSWORK: This test documents the current behavior, but this could\n> +# change in the future if we decide to support blaming files outside\n> +# the sparse definition.\n\nOK. From the description it is clear that we do not support it\nright now, which is OK by me.\n"},{"id":"442244","messageId":"724abbd4-b9ee-3b3d-226c-b7999f138152@gmail.com","threadId":"56710","inReplyTo":"xmqq8rxe787g.fsf@gitster.g","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-24T14:41:26Z","receivedAt":"2021-11-24T14:49:41Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/23/21 3:39 PM, Junio C Hamano wrote:\n> \"Lessley Dennington via GitGitGadget\" \n> writes:\n> \n>> From: Lessley Dennington \n>>\n>> Check whether git dir exists before adding any repo settings. If it\n>> does not exist, BUG with the message that one cannot add settings for an\n>> uninitialized repository. If it does exist, proceed with adding repo\n>> settings.\n>>\n>> Additionally, ensure the above BUG is not triggered when users pass the -h\n>> flag by adding a check for the repository to the checkout and pack-objects\n>> builtins.\n>>\n>> Finally, ensure the above BUG is not triggered for commit-graph by\n>> returning early if the git directory does not exist.\n>>\n>> Signed-off-by: Lessley Dennington \n>> ---\n>> builtin/checkout.c | 6 ++++--\n>> builtin/pack-objects.c | 9 ++++++---\n>> commit-graph.c | 5 ++++-\n>> repo-settings.c | 3 +++\n>> 4 files changed, 17 insertions(+), 6 deletions(-)\n>>\n>> diff --git a/builtin/checkout.c b/builtin/checkout.c\n>> index 8c69dcdf72a..31632b07888 100644\n>> --- a/builtin/checkout.c\n>> +++ b/builtin/checkout.c\n>> @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n>> \n>> \tgit_config(git_checkout_config, opts);\n>> \n>> -\tprepare_repo_settings(the_repository);\n>> -\tthe_repository->settings.command_requires_full_index = 0;\n>> +\tif (startup_info->have_repository) {\n>> +\t\tprepare_repo_settings(the_repository);\n>> +\t\tthe_repository->settings.command_requires_full_index = 0;\n>> +\t}\n> \n> I am kind-a surprised if the control reaches this deep if you are\n> not in a repository. In git.c::commands[] list, all three primary\n> entry points that call checkout_main(), namely, cmd_checkout(),\n> cmd_restore(), and cmd_switch(), are marked with RUN_SETUP bit,\n> which makes us call setup_git_directory() even before we call the\n> cmd_X() function. And setup_git_directory() dies with \"not a git\n> repository (or any of the parent directories)\" outside a repository.\n> \n> So, how can startup_info->have_repository bit be false if the\n> control flow reaches here? Or am I grossly misunderstanding what\n> you are trying to do?\n> \nThis was in reaction to the t0012-help.sh tests failing with the\nnew BUG in prepare_repo_settings. However, Elijah pointed out\nthat it's a better idea to move prepare_repo_settings farther\ndown (after parse_options) instead. So I will be issuing that\nchange as part of v5.\n>> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n>> index 1a3dd445f83..45dc2258dc7 100644\n>> --- a/builtin/pack-objects.c\n>> +++ b/builtin/pack-objects.c\n>> @@ -3976,9 +3976,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n>> \tread_replace_refs = 0;\n> \n> Ditto wrt RUN_SETUP.\n> \nUpdating with the same change as checkout for v5.\n>> diff --git a/commit-graph.c b/commit-graph.c\n>> index 2706683acfe..265c010122e 100644\n>> --- a/commit-graph.c\n>> +++ b/commit-graph.c\n>> @@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n>> \tstruct object_directory *odb;\n>> \t/*\n>> +\t * Early return if there is no git dir or if the commit graph is\n>> +\t * disabled.\n>> +\t *\n>> \t * This must come before the \"already attempted?\" check below, because\n>> \t * we want to disable even an already-loaded graph file.\n>> \t */\n>> -\tif (r->commit_graph_disabled)\n>> +\tif (!r->gitdir || r->commit_graph_disabled)\n>> \t\treturn 0;\n> \n> I haven't followed the control flow, but this one probably is a good\n> addition (in other words, unlike cmd_pack_objects(), I cannot convince\n> myself that r->gitdir will never be NULL here).\n> \n>> diff --git a/repo-settings.c b/repo-settings.c\n>> index b93e91a212e..00ca5571a1a 100644\n>> --- a/repo-settings.c\n>> +++ b/repo-settings.c\n>> @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n>> \tchar *strval;\n>> \tint manyfiles;\n>> \n>> +\tif (!r->gitdir)\n>> +\t\tBUG(\"Cannot add settings for uninitialized repository\");\n>> +\n> \n> This is a very good idea. If I recall correctly, I think I reviewed\n> a bugfix patch that would have been simplified if we had this check\n> here.\n>Lessley\n"},{"id":"442245","messageId":"c8e86ac1-a391-8957-9301-8a9004d46191@gmail.com","threadId":"56710","inReplyTo":"xmqqr1b65sz1.fsf@gitster.g","subject":"Re: [PATCH v4 4/4] blame: enable and test the sparse index","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-24T14:52:09Z","receivedAt":"2021-11-24T14:52:21Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/23/21 3:53 PM, Junio C Hamano wrote:\n> \"Lessley Dennington via GitGitGadget\" \n> writes:\n> \n>> diff --git a/builtin/blame.c b/builtin/blame.c\n>> index 641523ff9af..247b9eaf88f 100644\n>> --- a/builtin/blame.c\n>> +++ b/builtin/blame.c\n>> @@ -902,6 +902,11 @@ int cmd_blame(int argc, const char **argv, const char *prefix)\n>> \tlong anchor;\n>> \tconst int hexsz = the_hash_algo->hexsz;\n>> \n>> +\tif (startup_info->have_repository) {\n> \n> The command is marked with RUN_SETUP bit in git.c::commands[] list,\n> so I would think we wouldn't even get called if we are not in a\n> repository here.\n> \n> Under what condition can startup_info->have_repository be false at\n> this point in the control flow? If there is such a case, it would\n> mean that startup_info->have_repository bit can be false even if we\n> are in a repository. That sounds like a bug in some code (I do not\n> know where offhand) that is supposed to prepare the startup_info\n> before cmd_X() functions are called.\n> \nAs with checkout and pack-objects, this was added to protect against\nthe new BUG() in prepare_repo_settings being triggered with the help\ncommand (which was surfaced with failures in t0012-help.sh). As with\nthose builtins, I will be removing the conditional and moving\nprepare_repo_settings after parse_options in v5.\n>> diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\n>> index 5cf94627383..9ac76a049b8 100755\n>> --- a/t/perf/p2000-sparse-operations.sh\n>> +++ b/t/perf/p2000-sparse-operations.sh\n>> @@ -114,6 +114,8 @@ test_perf_on_all git reset\n>> test_perf_on_all git reset --hard\n>> test_perf_on_all git reset -- does-not-exist\n>> test_perf_on_all git diff\n>> -test_perf_on_all git diff --cached\n>> +test_perf_on_all git diff --staged\n> \n> That's a funny revert of what the previous step did; I thought this\n> step was about \"blame\" and not \"diff\".\n> \n>> +test_perf_on_all git blame $SPARSE_CONE/a\n>> +test_perf_on_all git blame $SPARSE_CONE/f3/a\n>> \n>> test_done\n>> -# TODO: blame currently does not support blaming files outside of the\n>> -# sparse definition. It complains that the file doesn't exist locally.\n>> -test_expect_failure 'blame with pathspec outside sparse definition' '\n>> +# NEEDSWORK: This test documents the current behavior, but this could\n>> +# change in the future if we decide to support blaming files outside\n>> +# the sparse definition.\n> \n> OK. From the description it is clear that we do not support it\n> right now, which is OK by me.\n> \nElijah made the point that blaming files outside the directory isn't\nsupported in full checkouts either. So I've updated the comment and\ncommit description a bit for v5.\n\nLessley\n"},{"id":"442247","messageId":"0372ee0b-8dda-c505-2b83-30da74e4fb36@gmail.com","threadId":"56710","inReplyTo":"xmqq4k82781w.fsf@gitster.g","subject":"Re: [PATCH v4 2/4] test-read-cache: set up repo after git directory","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-24T15:10:33Z","receivedAt":"2021-11-24T15:10:38Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/23/21 3:42 PM, Junio C Hamano wrote:\n> \"Lessley Dennington via GitGitGadget\" \n> writes:\n> \n>> From: Lessley Dennington \n>>\n>> Move repo setup to occur after git directory is set up. This will ensure\n>> enabling the sparse index for `diff` (and guarding against the nongit\n>> scenario) will not cause tests to start failing, since that change will include\n>> adding a check to prepare_repo_settings() with the new BUG.\n> \n> This looks obviously the right thing to do. Would anything break\n> because of the \"wrong\" ordering of events in the original code?\n> \n> IOW, can this \"bugfix\" be protected with a new test against\n> regression?\n> \nYep! Tests 2, 3, 28, and 34 in t1092-sparse-checkout-compatibility.sh\nwill fail without this change.\n>> Signed-off-by: Lessley Dennington \n>> ---\n>> t/helper/test-read-cache.c | 5 +++--\n>> 1 file changed, 3 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c\n>> index b52c174acc7..0d9f08931a1 100644\n>> --- a/t/helper/test-read-cache.c\n>> +++ b/t/helper/test-read-cache.c\n>> @@ -39,8 +39,6 @@ int cmd__read_cache(int argc, const char **argv)\n>> \tint table = 0, expand = 0;\n>> \n>> \tinitialize_the_repository();\n>> -\tprepare_repo_settings(r);\n>> -\tr->settings.command_requires_full_index = 0;\n>> \n>> \tfor (++argv, --argc; *argv && starts_with(*argv, \"--\"); ++argv, --argc) {\n>> \t\tif (skip_prefix(*argv, \"--print-and-refresh=\", &name))\n>> @@ -56,6 +54,9 @@ int cmd__read_cache(int argc, const char **argv)\n>> \tsetup_git_directory();\n>> \tgit_config(git_default_config, NULL);\n>> \n>> +\tprepare_repo_settings(r);\n>> +\tr->settings.command_requires_full_index = 0;\n>> +\n>> \tfor (i = 0; i < cnt; i++) {\n>> \t\trepo_read_index(r);\n"},{"id":"442255","messageId":"xmqqzgpt2z0h.fsf@gitster.g","threadId":"56710","inReplyTo":"724abbd4-b9ee-3b3d-226c-b7999f138152@gmail.com","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-24T18:23:58Z","receivedAt":"2021-11-24T18:24:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lessley Dennington writes:\n\n>>> @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n>>> \tgit_config(git_checkout_config, opts);\n>>> -\tprepare_repo_settings(the_repository);\n>>> -\tthe_repository->settings.command_requires_full_index = 0;\n>>> +\tif (startup_info->have_repository) {\n>>> +\t\tprepare_repo_settings(the_repository);\n>>> +\t\tthe_repository->settings.command_requires_full_index = 0;\n>>> +\t}\n>> I am kind-a surprised if the control reaches this deep if you are\n>> not in a repository. In git.c::commands[] list, all three primary\n>> entry points that call checkout_main(), namely, cmd_checkout(),\n>> cmd_restore(), and cmd_switch(), are marked with RUN_SETUP bit,\n>> which makes us call setup_git_directory() even before we call the\n>> cmd_X() function. And setup_git_directory() dies with \"not a git\n>> repository (or any of the parent directories)\" outside a repository.\n>> So, how can startup_info->have_repository bit be false if the\n>> control flow reaches here? Or am I grossly misunderstanding what\n>> you are trying to do?\n>> \n> This was in reaction to the t0012-help.sh tests failing with the\n> new BUG in prepare_repo_settings. However, Elijah pointed out that\n> it's a better idea to move prepare_repo_settings farther down\n> (after parse_options) instead. So I will be issuing that change as\n> part of v5.\n\nI forgot that \"git foo -h\" for a builtin command 'foo' calls\nrun_builtin() but bypasses the RUN_SETUP and NEED_WORK_TREE handling\nbefore it in turn calls cmd_foo(). We expect a call in cmd_foo() to\nparse_options() emit a short-help and exit.\n\nSo, yes, there is a way to reach this point in the codeflow without\nbeing in a repository (or even when in a repository, we may have\nchosen not to realize it). Feels ugly.\n\nNow a bit of tangent.\n\nI wonder if it is a problem to completely bypass RUN_SETUP in such a\ncase. In general, we read the configuration to tweak the hardcoded\ndefault behaviour, and then further override it by parsing command\nline options. In order to read configuration fully, we'd need to\nknow where the repository is. So the start-up sequence must be in\nthis order:\n\n - discover where the repository is (either gently or with a hard failure)\n - read the configuration files\n - call parse_options()\n\nAnd by completly bypassing RUN_SETUP, we are not reading per-repo\nsettings from the configuration files.\n\nSomething along this line (note: there is an always-taken if block\nto reduce the patch noise for this illustration), perhaps.\n\n\n\n\n git.c | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git i/git.c w/git.c\nindex 5ff21be21f..50e258508e 100644\n--- i/git.c\n+++ w/git.c\n@@ -421,25 +421,30 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \tint status, help;\n \tstruct stat st;\n \tconst char *prefix;\n+\tint run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n \n \tprefix = NULL;\n \thelp = argc == 2 && !strcmp(argv[1], \"-h\");\n-\tif (!help) {\n-\t\tif (p->option & RUN_SETUP)\n+\tif (help && (run_setup & RUN_SETUP))\n+\t\t/* demote to GENTLY to allow 'git cmd -h' outside repo */\n+\t\trun_setup = RUN_SETUP_GENTLY;\n+\n+\tif (1) {\n+\t\tif (run_setup & RUN_SETUP)\n \t\t\tprefix = setup_git_directory();\n-\t\telse if (p->option & RUN_SETUP_GENTLY) {\n+\t\telse if (run_setup & RUN_SETUP_GENTLY) {\n \t\t\tint nongit_ok;\n \t\t\tprefix = setup_git_directory_gently(&nongit_ok);\n \t\t}\n \t\tprecompose_argv_prefix(argc, argv, NULL);\n-\t\tif (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY) &&\n+\t\tif (use_pager == -1 && run_setup &&\n \t\t !(p->option & DELAY_PAGER_CONFIG))\n \t\t\tuse_pager = check_pager_config(p->cmd);\n \t\tif (use_pager == -1 && p->option & USE_PAGER)\n \t\t\tuse_pager = 1;\n \n-\t\tif ((p->option & (RUN_SETUP | RUN_SETUP_GENTLY)) &&\n-\t\t startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */\n+\t\tif (run_setup && startup_info->have_repository)\n+\t\t\t/* get_git_dir() may set up repo, avoid that */\n \t\t\ttrace_repo_setup(prefix);\n \t}\n \tcommit_pager_choice();\n"},{"id":"442257","messageId":"xmqqtug12yga.fsf@gitster.g","threadId":"56710","inReplyTo":"0372ee0b-8dda-c505-2b83-30da74e4fb36@gmail.com","subject":"Re: [PATCH v4 2/4] test-read-cache: set up repo after git directory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-24T18:36:05Z","receivedAt":"2021-11-24T18:36:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lessley Dennington writes:\n\n> On 11/23/21 3:42 PM, Junio C Hamano wrote:\n>> \"Lessley Dennington via GitGitGadget\" \n>> writes:\n>> \n>>> From: Lessley Dennington \n>>>\n>>> Move repo setup to occur after git directory is set up. This will ensure\n>>> enabling the sparse index for `diff` (and guarding against the nongit\n>>> scenario) will not cause tests to start failing, since that change will include\n>>> adding a check to prepare_repo_settings() with the new BUG.\n\n>> This looks obviously the right thing to do. Would anything break\n>> because of the \"wrong\" ordering of events in the original code?\n>> IOW, can this \"bugfix\" be protected with a new test against\n>> regression?\n>> \n> Yep! Tests 2, 3, 28, and 34 in t1092-sparse-checkout-compatibility.sh\n> will fail without this change.\n\nI do not understand. When 1/4 and 2/4 are applied, no tests in\nt1092 fail for me.\n\nI think the presentation order of this series is not reviewer\nfriendly; \"the new BUG\" is introduced in a separate step and\nobscures the reason why this step is needed. It is better than\nadding \"the new BUG\" first and let some tests fail and then fix the\nbreakage the series caused in later steps, though.\n\n\n\n\n"},{"id":"442574","messageId":"eef31466-e574-1f0e-91c5-efaa78ad8880@gmail.com","threadId":"56710","inReplyTo":"xmqqtug12yga.fsf@gitster.g","subject":"Re: [PATCH v4 2/4] test-read-cache: set up repo after git directory","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-29T23:01:55Z","receivedAt":"2021-11-29T23:14:12Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/24/21 10:36 AM, Junio C Hamano wrote:\n> Lessley Dennington writes:\n> \n>> On 11/23/21 3:42 PM, Junio C Hamano wrote:\n>>> \"Lessley Dennington via GitGitGadget\" \n>>> writes:\n>>>\n>>>> From: Lessley Dennington \n>>>>\n>>>> Move repo setup to occur after git directory is set up. This will ensure\n>>>> enabling the sparse index for `diff` (and guarding against the nongit\n>>>> scenario) will not cause tests to start failing, since that change will include\n>>>> adding a check to prepare_repo_settings() with the new BUG.\n> \n>>> This looks obviously the right thing to do. Would anything break\n>>> because of the \"wrong\" ordering of events in the original code?\n>>> IOW, can this \"bugfix\" be protected with a new test against\n>>> regression?\n>>>\n>> Yep! Tests 2, 3, 28, and 34 in t1092-sparse-checkout-compatibility.sh\n>> will fail without this change.\n> \n> I do not understand. When 1/4 and 2/4 are applied, no tests in\n> t1092 fail for me.\n> \n> I think the presentation order of this series is not reviewer\n> friendly; \"the new BUG\" is introduced in a separate step and\n> obscures the reason why this step is needed. It is better than\n> adding \"the new BUG\" first and let some tests fail and then fix the\n> breakage the series caused in later steps, though.\n> \n> \n> \n> \nWhat I meant was that those tests fail with the first patch in the\nseries. Once this patch is applied they no longer fail.\n\nThat being said, I agree with this feedback - I have reorganized the\nchanges for v5 in a way in which I hope will make more sense and will\nimprove the reviewer experience.\n\nLessley\n"},{"id":"442581","messageId":"6db27c92-6ea6-fa5f-8aa7-e1fef7d5ff5a@gmail.com","threadId":"56710","inReplyTo":"xmqqzgpt2z0h.fsf@gitster.g","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-29T23:38:03Z","receivedAt":"2021-11-29T23:38:07Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/24/21 10:23 AM, Junio C Hamano wrote:\n> Lessley Dennington writes:\n> \n>>>> @@ -1588,8 +1588,10 @@ static int checkout_main(int argc, const char **argv, const char *prefix,\n>>>> \tgit_config(git_checkout_config, opts);\n>>>> -\tprepare_repo_settings(the_repository);\n>>>> -\tthe_repository->settings.command_requires_full_index = 0;\n>>>> +\tif (startup_info->have_repository) {\n>>>> +\t\tprepare_repo_settings(the_repository);\n>>>> +\t\tthe_repository->settings.command_requires_full_index = 0;\n>>>> +\t}\n>>> I am kind-a surprised if the control reaches this deep if you are\n>>> not in a repository. In git.c::commands[] list, all three primary\n>>> entry points that call checkout_main(), namely, cmd_checkout(),\n>>> cmd_restore(), and cmd_switch(), are marked with RUN_SETUP bit,\n>>> which makes us call setup_git_directory() even before we call the\n>>> cmd_X() function. And setup_git_directory() dies with \"not a git\n>>> repository (or any of the parent directories)\" outside a repository.\n>>> So, how can startup_info->have_repository bit be false if the\n>>> control flow reaches here? Or am I grossly misunderstanding what\n>>> you are trying to do?\n>>>\n>> This was in reaction to the t0012-help.sh tests failing with the\n>> new BUG in prepare_repo_settings. However, Elijah pointed out that\n>> it's a better idea to move prepare_repo_settings farther down\n>> (after parse_options) instead. So I will be issuing that change as\n>> part of v5.\n> \n> I forgot that \"git foo -h\" for a builtin command 'foo' calls\n> run_builtin() but bypasses the RUN_SETUP and NEED_WORK_TREE handling\n> before it in turn calls cmd_foo(). We expect a call in cmd_foo() to\n> parse_options() emit a short-help and exit.\n> \n> So, yes, there is a way to reach this point in the codeflow without\n> being in a repository (or even when in a repository, we may have\n> chosen not to realize it). Feels ugly.\n> \n> Now a bit of tangent.\n> \n> I wonder if it is a problem to completely bypass RUN_SETUP in such a\n> case. In general, we read the configuration to tweak the hardcoded\n> default behaviour, and then further override it by parsing command\n> line options. In order to read configuration fully, we'd need to\n> know where the repository is. So the start-up sequence must be in\n> this order:\n> \n> - discover where the repository is (either gently or with a hard failure)\n> - read the configuration files\n> - call parse_options()\n> \n> And by completly bypassing RUN_SETUP, we are not reading per-repo\n> settings from the configuration files.\n> \n> Something along this line (note: there is an always-taken if block\n> to reduce the patch noise for this illustration), perhaps.\n> \n> \n> \n> \n> git.c | 17 +++++++++++------\n> 1 file changed, 11 insertions(+), 6 deletions(-)\n> \n> diff --git i/git.c w/git.c\n> index 5ff21be21f..50e258508e 100644\n> --- i/git.c\n> +++ w/git.c\n> @@ -421,25 +421,30 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n> \tint status, help;\n> \tstruct stat st;\n> \tconst char *prefix;\n> +\tint run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n> \n> \tprefix = NULL;\n> \thelp = argc == 2 && !strcmp(argv[1], \"-h\");\n> -\tif (!help) {\n> -\t\tif (p->option & RUN_SETUP)\n> +\tif (help && (run_setup & RUN_SETUP))\n> +\t\t/* demote to GENTLY to allow 'git cmd -h' outside repo */\n> +\t\trun_setup = RUN_SETUP_GENTLY;\n> +\n> +\tif (1) {\n> +\t\tif (run_setup & RUN_SETUP)\n> \t\t\tprefix = setup_git_directory();\n> -\t\telse if (p->option & RUN_SETUP_GENTLY) {\n> +\t\telse if (run_setup & RUN_SETUP_GENTLY) {\n> \t\t\tint nongit_ok;\n> \t\t\tprefix = setup_git_directory_gently(&nongit_ok);\n> \t\t}\n> \t\tprecompose_argv_prefix(argc, argv, NULL);\n> -\t\tif (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY) &&\n> +\t\tif (use_pager == -1 && run_setup &&\n> \t\t !(p->option & DELAY_PAGER_CONFIG))\n> \t\t\tuse_pager = check_pager_config(p->cmd);\n> \t\tif (use_pager == -1 && p->option & USE_PAGER)\n> \t\t\tuse_pager = 1;\n> \n> -\t\tif ((p->option & (RUN_SETUP | RUN_SETUP_GENTLY)) &&\n> -\t\t startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */\n> +\t\tif (run_setup && startup_info->have_repository)\n> +\t\t\t/* get_git_dir() may set up repo, avoid that */\n> \t\t\ttrace_repo_setup(prefix);\n> \t}\n> \tcommit_pager_choice();\n> \nThis is cool! I applied it locally, and it seems to be working well. I\nwill plan to replace my changes to checkout and pack-objects with this\nfor v5.\n\nLessley\n"},{"id":"442608","messageId":"xmqqmtlm2lx1.fsf@gitster.g","threadId":"56710","inReplyTo":"6db27c92-6ea6-fa5f-8aa7-e1fef7d5ff5a@gmail.com","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-11-30T06:32:42Z","receivedAt":"2021-11-30T06:32:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lessley Dennington writes:\n\n> This is cool! I applied it locally, and it seems to be working well. I\n> will plan to replace my changes to checkout and pack-objects with this\n> for v5.\n\nI didn't write it to replace the changes you were preparing, though.\n\nWhat the patch is meant to solve is that \"git checkout -h\" in a\nrepository, whose .git/config has some custom configuration\nvariables that affect how the short help text is shown, were\nignoring that per-repository settings by not even checking if we are\nin a repository at all.\n\nWhen \"-h\" is in effect, and if you are outside a repository, this\nwould cause the setup_git_directory_gently() function to be called,\nand we'd reach checkout_main() without having discovered repository.\nI do not think it removes the need for your \"only when startup-info\nsays we have repository, do these\" safety. At least, I didn't write\nthe patch to remove that need.\n"},{"id":"442718","messageId":"07c87109-48ea-074e-8090-7e3db0ec9ad5@gmail.com","threadId":"56710","inReplyTo":"xmqqmtlm2lx1.fsf@gitster.g","subject":"Re: [PATCH v4 1/4] sparse index: enable only for git repos","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-11-30T23:25:16Z","receivedAt":"2021-11-30T23:27:55Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 11/29/21 10:32 PM, Junio C Hamano wrote:\n> Lessley Dennington writes:\n> \n>> This is cool! I applied it locally, and it seems to be working well. I\n>> will plan to replace my changes to checkout and pack-objects with this\n>> for v5.\n> \n> I didn't write it to replace the changes you were preparing, though.\n> \n> What the patch is meant to solve is that \"git checkout -h\" in a\n> repository, whose .git/config has some custom configuration\n> variables that affect how the short help text is shown, were\n> ignoring that per-repository settings by not even checking if we are\n> in a repository at all.\n> \n> When \"-h\" is in effect, and if you are outside a repository, this\n> would cause the setup_git_directory_gently() function to be called,\n> and we'd reach checkout_main() without having discovered repository.\n> I do not think it removes the need for your \"only when startup-info\n> says we have repository, do these\" safety. At least, I didn't write\n> the patch to remove that need.\n> \nTo be clear, I am not replacing the change in repo-settings.c to BUG\nif we're trying to prepare settings for an uninitialized repository.\nI just removed the conditional I had added around the\nprepare_repo_settings method in checkout and pack-objects, since the\nfailing tests for those builtins started passing with the application\nof this patch.\n"},{"id":"443031","messageId":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v4.git.1637620958.gitgitgadget@gmail.com","subject":"[PATCH v5 0/7] Sparse Index: diff and blame builtins","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:15:58Z","receivedAt":"2021-12-03T21:16:11Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"This series is based on vd/sparse-reset. It integrates the sparse index with\ngit diff and git blame and includes:\n\n 1. tests added to t1092 and p2000 to establish the baseline functionality\n of the commands\n 2. repository settings to enable the sparse index\n\nThe p2000 tests demonstrate a ~44% execution time reduction for 'git diff'\nand a ~86% execution time reduction for 'git diff --staged' using a sparse\nindex. For 'git blame', the reduction time was ~60% for a file two levels\ndeep and ~30% for a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\n\n\nChanges since V1\n================\n\n * Fix failing diff partially-staged test in\n t1092-sparse-checkout-compatibility.sh, which was breaking in seen.\n\n\nChanges since V2\n================\n\n * Update diff commit description to include patches that make the checkout\n and status commands work with the sparse index for readers to reference.\n * Add new test case to verify diff behaves as expected when run against\n files outside the sparse checkout cone.\n * Indent error message in blame commit\n * Check error message in blame with pathspec outside sparse definition test\n matches expectations.\n * Loop blame tests (instead of running the same command multiple time\n against different files).\n\n\nChanges since V3\n================\n\n * Update diff p2000 tests to use --cached instead of --staged. Execute new\n run and update results in commit description and cover letter.\n * Update comment on blame with pathspec outside sparse definition test in\n t1092-sparse-checkout-compatibility.sh to clarify that it tests the\n current state and could be improved in the future.\n * Ensure sparse index is only activated when diff is running against files\n in a Git repo.\n * BUG if prepare_repo_settings() is called outside a repository.\n * Ensure sparse index is not activated for calls to blame, checkout, or\n pack-object with -h.\n * Ensure commit-graph is only loaded if a git directory exists.\n\n\nChanges since V4\n================\n\n * Remove startup_info->have_repository check from checkout, pack-objects,\n and blame. Update git.c to no longer bypass setup when -h is passed\n instead.\n * Move commit-graph, test-read-cache, and repo-settings changes into their\n own patches with details in commit description of why the changes are\n being made.\n * Update t1092-sparse-checkout-compatibility.sh tests to use --cached\n instead of --staged.\n * Use 10-character hash abbreviations for commits referenced in diff commit\n message.\n * Clarify that being unable to blame files outside the working directory is\n not supported in either sparse or non-sparse checkouts both in comment on\n blame with pathspec outside sparse definition test in\n t1092-sparse-checkout-compatibility.sh and blame commit message.\n\nThanks, Lessley\n\nLessley Dennington (7):\n git: esnure correct git directory setup with -h\n commit-graph: return if there is no git directory\n test-read-cache: set up repo after git directory\n repo-settings: prepare_repo_settings only in git repos\n diff: replace --staged with --cached in t1092 tests\n diff: enable and test the sparse index\n blame: enable and test the sparse index\n\n builtin/blame.c | 3 +\n builtin/diff.c | 5 ++\n commit-graph.c | 5 +-\n git.c | 37 ++++----\n repo-settings.c | 3 +\n t/helper/test-read-cache.c | 5 +-\n t/perf/p2000-sparse-operations.sh | 4 +\n t/t1092-sparse-checkout-compatibility.sh | 109 +++++++++++++++++++----\n 8 files changed, 132 insertions(+), 39 deletions(-)\n\n\nbase-commit: f2a454e0a5e26c0f7b840970f69d195c37b16565\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1050%2Fldennington%2Fdiff-blame-sparse-index-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1050/ldennington/diff-blame-sparse-index-v5\nPull-Request: https://github.com/gitgitgadget/git/pull/1050\n\nRange-diff vs v4:\n\n -: ----------- > 1: 09c2ff9f898 git: esnure correct git directory setup with -h\n 1: 81e208cf454 ! 2: 9e53a6435e4 sparse index: enable only for git repos\n @@ Metadata\n Author: Lessley Dennington \n \n ## Commit message ##\n - sparse index: enable only for git repos\n + commit-graph: return if there is no git directory\n \n - Check whether git dir exists before adding any repo settings. If it\n - does not exist, BUG with the message that one cannot add settings for an\n - uninitialized repository. If it does exist, proceed with adding repo\n - settings.\n -\n - Additionally, ensure the above BUG is not triggered when users pass the -h\n - flag by adding a check for the repository to the checkout and pack-objects\n - builtins.\n -\n - Finally, ensure the above BUG is not triggered for commit-graph by\n - returning early if the git directory does not exist.\n + Return early if git directory does not exist. This will protect against\n + test failures in the upcoming change to BUG in prepare_repo_settings if no\n + git directory exists.\n \n Signed-off-by: Lessley Dennington \n \n - ## builtin/checkout.c ##\n -@@ builtin/checkout.c: static int checkout_main(int argc, const char **argv, const char *prefix,\n - \n - \tgit_config(git_checkout_config, opts);\n - \n --\tprepare_repo_settings(the_repository);\n --\tthe_repository->settings.command_requires_full_index = 0;\n -+\tif (startup_info->have_repository) {\n -+\t\tprepare_repo_settings(the_repository);\n -+\t\tthe_repository->settings.command_requires_full_index = 0;\n -+\t}\n - \n - \topts->track = BRANCH_TRACK_UNSPECIFIED;\n - \n -\n - ## builtin/pack-objects.c ##\n -@@ builtin/pack-objects.c: int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n - \tread_replace_refs = 0;\n - \n - \tsparse = git_env_bool(\"GIT_TEST_PACK_SPARSE\", -1);\n --\tprepare_repo_settings(the_repository);\n --\tif (sparse < 0)\n --\t\tsparse = the_repository->settings.pack_use_sparse;\n -+\n -+\tif (startup_info->have_repository) {\n -+\t\tprepare_repo_settings(the_repository);\n -+\t\tif (sparse < 0)\n -+\t\t\tsparse = the_repository->settings.pack_use_sparse;\n -+\t}\n - \n - \treset_pack_idx_option(&pack_idx_opts);\n - \tgit_config(git_pack_config, NULL);\n -\n ## commit-graph.c ##\n @@ commit-graph.c: static int prepare_commit_graph(struct repository *r)\n \tstruct object_directory *odb;\n @@ commit-graph.c: static int prepare_commit_graph(struct repository *r)\n \t\treturn 0;\n \n \tif (r->objects->commit_graph_attempted)\n -\n - ## repo-settings.c ##\n -@@ repo-settings.c: void prepare_repo_settings(struct repository *r)\n - \tchar *strval;\n - \tint manyfiles;\n - \n -+\tif (!r->gitdir)\n -+\t\tBUG(\"Cannot add settings for uninitialized repository\");\n -+\n - \tif (r->settings.initialized++)\n - \t\treturn;\n - \n 2: 5bc5e8465ab ! 3: 219a4158b6a test-read-cache: set up repo after git directory\n @@ Metadata\n ## Commit message ##\n test-read-cache: set up repo after git directory\n \n - Move repo setup to occur after git directory is set up. This will ensure\n - enabling the sparse index for `diff` (and guarding against the nongit\n - scenario) will not cause tests to start failing, since that change will include\n - adding a check to prepare_repo_settings() with the new BUG.\n + Move repo setup to occur after git directory is set up. This will protect\n + against test failures in the upcoming change to BUG in\n + prepare_repo_settings if no git directory exists.\n \n Signed-off-by: Lessley Dennington \n \n -: ----------- > 4: 4d8d58c473b repo-settings: prepare_repo_settings only in git repos\n -: ----------- > 5: 85e3e5c78e7 diff: replace --staged with --cached in t1092 tests\n 3: 273ee16b74e ! 6: 4f16366e5ad diff: enable and test the sparse index\n @@ Commit message\n with the 'git status' and 'git checkout' commands that were already\n integrated. For more details see:\n \n - d76723e (status: use sparse-index throughout, 2021-07-14)\n - 1ba5f45 (checkout: stop expanding sparse indexes, 2021-06-29)\n + d76723ee53 (status: use sparse-index throughout, 2021-07-14)\n + 1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n \n The most interesting thing to do is to add tests that verify that 'git\n diff' behaves correctly when the sparse index is enabled. These cases are:\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is n\n +\trun_on_all ../edit-contents deep/testfile &&\n +\n +\ttest_all_match git diff &&\n -+\ttest_all_match git diff --staged &&\n ++\ttest_all_match git diff --cached &&\n +\tensure_not_expanded diff &&\n -+\tensure_not_expanded diff --staged &&\n ++\tensure_not_expanded diff --cached &&\n +\n +\t# Add file outside cone\n +\ttest_all_match git reset --hard &&\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is n\n +\ttest_sparse_match git sparse-checkout set &&\n +\n +\ttest_all_match git diff &&\n -+\ttest_all_match git diff --staged &&\n ++\ttest_all_match git diff --cached &&\n +\tensure_not_expanded diff &&\n -+\tensure_not_expanded diff --staged &&\n ++\tensure_not_expanded diff --cached &&\n +\n +\t# Merge conflict outside cone\n +\t# The sparse checkout will report a warning that is not in the\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse-index is n\n +\ttest_all_match test_must_fail git merge merge-right &&\n +\n +\ttest_all_match git diff &&\n -+\ttest_all_match git diff --staged &&\n ++\ttest_all_match git diff --cached &&\n +\tensure_not_expanded diff &&\n -+\tensure_not_expanded diff --staged\n ++\tensure_not_expanded diff --cached\n +'\n +\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n 4: 7acf5118bf5 ! 7: 04532378734 blame: enable and test the sparse index\n @@ Commit message\n 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n \n We do not include paths outside the sparse checkout cone because blame\n - currently does not support blaming files outside of the sparse definition.\n - Attempting to do so fails with the following error:\n -\n - fatal: no such path '' in HEAD\n + does not support blaming files that are not present in the working\n + directory. This is true in both sparse and full checkouts.\n \n Signed-off-by: Lessley Dennington \n \n ## builtin/blame.c ##\n -@@ builtin/blame.c: int cmd_blame(int argc, const char **argv, const char *prefix)\n - \tlong anchor;\n - \tconst int hexsz = the_hash_algo->hexsz;\n +@@ builtin/blame.c: parse_done:\n + \trevs.diffopt.flags.follow_renames = 0;\n + \targc = parse_options_end(&ctx);\n \n -+\tif (startup_info->have_repository) {\n -+\t\tprepare_repo_settings(the_repository);\n -+\t\tthe_repository->settings.command_requires_full_index = 0;\n -+\t}\n ++\tprepare_repo_settings(the_repository);\n ++\tthe_repository->settings.command_requires_full_index = 0;\n +\n - \tsetup_default_color_by_age();\n - \tgit_config(git_blame_config, &output_option);\n - \trepo_init_revisions(the_repository, &revs, NULL);\n + \tif (incremental || (output_option & OUTPUT_PORCELAIN)) {\n + \t\tif (show_progress > 0)\n + \t\t\tdie(_(\"--progress can't be used with --incremental or porcelain formats\"));\n \n ## t/perf/p2000-sparse-operations.sh ##\n -@@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git reset\n - test_perf_on_all git reset --hard\n +@@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n --test_perf_on_all git diff --cached\n -+test_perf_on_all git diff --staged\n + test_perf_on_all git diff --cached\n +test_perf_on_all git blame $SPARSE_CONE/a\n +test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'log with pathspec\n -# TODO: blame currently does not support blaming files outside of the\n -# sparse definition. It complains that the file doesn't exist locally.\n -test_expect_failure 'blame with pathspec outside sparse definition' '\n -+# NEEDSWORK: This test documents the current behavior, but this could\n -+# change in the future if we decide to support blaming files outside\n -+# the sparse definition.\n ++# Without a revision specified, blame will error if passed any file that\n ++# is not present in the working directory (even if the file is tracked).\n ++# Here we just verify that this is also true with sparse checkouts.\n +test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n +\ttest_sparse_match git sparse-checkout set &&\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'log with pathspec\n \n test_expect_success 'checkout and reset (mixed)' '\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'sparse index is not expanded: diff' '\n - \tensure_not_expanded diff --staged\n + \tensure_not_expanded diff --cached\n '\n \n +test_expect_success 'sparse index is not expanded: blame' '\n\n-- \ngitgitgadget\n"},{"id":"443032","messageId":"09c2ff9f89833b3ac918a399e10d1b6abe71b339.1638566165.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 1/7] git: esnure correct git directory setup with -h","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:15:59Z","receivedAt":"2021-12-03T21:16:15Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnsure correct git directory setup when -h is passed with commands. This\nspecifically applies to repos with special help text configuration\nvariables and to commands run with -h outside a repository. This\nwill also protect against test failures in the upcoming change to BUG in\nprepare_repo_settings if no git directory exists.\n\nNote: this diff is better seen when ignoring whitespace changes.\n\nCo-authored-by: Junio C Hamano \nSigned-off-by: Lessley Dennington \n---\n git.c | 37 +++++++++++++++++++------------------\n 1 file changed, 19 insertions(+), 18 deletions(-)\n\ndiff --git a/git.c b/git.c\nindex 60c2784be45..03a2dfa5174 100644\n--- a/git.c\n+++ b/git.c\n@@ -421,27 +421,28 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \tint status, help;\n \tstruct stat st;\n \tconst char *prefix;\n-\n+\tint run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n \tprefix = NULL;\n \thelp = argc == 2 && !strcmp(argv[1], \"-h\");\n-\tif (!help) {\n-\t\tif (p->option & RUN_SETUP)\n-\t\t\tprefix = setup_git_directory();\n-\t\telse if (p->option & RUN_SETUP_GENTLY) {\n-\t\t\tint nongit_ok;\n-\t\t\tprefix = setup_git_directory_gently(&nongit_ok);\n-\t\t}\n-\t\tprecompose_argv_prefix(argc, argv, NULL);\n-\t\tif (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY) &&\n-\t\t !(p->option & DELAY_PAGER_CONFIG))\n-\t\t\tuse_pager = check_pager_config(p->cmd);\n-\t\tif (use_pager == -1 && p->option & USE_PAGER)\n-\t\t\tuse_pager = 1;\n-\n-\t\tif ((p->option & (RUN_SETUP | RUN_SETUP_GENTLY)) &&\n-\t\t startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */\n-\t\t\ttrace_repo_setup(prefix);\n+\tif (help && (run_setup & RUN_SETUP))\n+\t\t/* demote to GENTLY to allow 'git cmd -h' outside repo */\n+\t\trun_setup = RUN_SETUP_GENTLY;\n+\n+\tif (run_setup & RUN_SETUP)\n+\t\tprefix = setup_git_directory();\n+\telse if (run_setup & RUN_SETUP_GENTLY) {\n+\t\tint nongit_ok;\n+\t\tprefix = setup_git_directory_gently(&nongit_ok);\n \t}\n+\tprecompose_argv_prefix(argc, argv, NULL);\n+\tif (use_pager == -1 && run_setup &&\n+\t\t!(p->option & DELAY_PAGER_CONFIG))\n+\t\tuse_pager = check_pager_config(p->cmd);\n+\tif (use_pager == -1 && p->option & USE_PAGER)\n+\t\tuse_pager = 1;\n+\tif (run_setup && startup_info->have_repository)\n+\t\t/* get_git_dir() may set up repo, avoid that */\n+\t\ttrace_repo_setup(prefix);\n \tcommit_pager_choice();\n \n \tif (!help && get_super_prefix()) {\n-- \ngitgitgadget\n\n"},{"id":"443033","messageId":"9e53a6435e4e7bdde941875fac6742288947b5e0.1638566165.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 2/7] commit-graph: return if there is no git directory","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:16:00Z","receivedAt":"2021-12-03T21:16:16Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nReturn early if git directory does not exist. This will protect against\ntest failures in the upcoming change to BUG in prepare_repo_settings if no\ngit directory exists.\n\nSigned-off-by: Lessley Dennington \n---\n commit-graph.c | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 2706683acfe..265c010122e 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n \tstruct object_directory *odb;\n \n \t/*\n+\t * Early return if there is no git dir or if the commit graph is\n+\t * disabled.\n+\t *\n \t * This must come before the \"already attempted?\" check below, because\n \t * we want to disable even an already-loaded graph file.\n \t */\n-\tif (r->commit_graph_disabled)\n+\tif (!r->gitdir || r->commit_graph_disabled)\n \t\treturn 0;\n \n \tif (r->objects->commit_graph_attempted)\n-- \ngitgitgadget\n\n"},{"id":"443034","messageId":"4d8d58c473b51cac6645a7d71336e995c1f3d28b.1638566165.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 4/7] repo-settings: prepare_repo_settings only in git repos","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:16:02Z","receivedAt":"2021-12-03T21:16:18Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nCheck whether git directory exists before adding any repo settings. If it\ndoes not exist, BUG with the message that one cannot add settings for an\nuninitialized repository. If it does exist, proceed with adding repo\nsettings.\n\nSigned-off-by: Lessley Dennington \n---\n repo-settings.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/repo-settings.c b/repo-settings.c\nindex b93e91a212e..00ca5571a1a 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n \tchar *strval;\n \tint manyfiles;\n \n+\tif (!r->gitdir)\n+\t\tBUG(\"Cannot add settings for uninitialized repository\");\n+\n \tif (r->settings.initialized++)\n \t\treturn;\n \n-- \ngitgitgadget\n\n"},{"id":"443035","messageId":"219a4158b6a0f14e8cfc5b74108282968a2d8ed6.1638566165.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 3/7] test-read-cache: set up repo after git directory","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:16:01Z","receivedAt":"2021-12-03T21:16:19Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nMove repo setup to occur after git directory is set up. This will protect\nagainst test failures in the upcoming change to BUG in\nprepare_repo_settings if no git directory exists.\n\nSigned-off-by: Lessley Dennington \n---\n t/helper/test-read-cache.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c\nindex b52c174acc7..0d9f08931a1 100644\n--- a/t/helper/test-read-cache.c\n+++ b/t/helper/test-read-cache.c\n@@ -39,8 +39,6 @@ int cmd__read_cache(int argc, const char **argv)\n \tint table = 0, expand = 0;\n \n \tinitialize_the_repository();\n-\tprepare_repo_settings(r);\n-\tr->settings.command_requires_full_index = 0;\n \n \tfor (++argv, --argc; *argv && starts_with(*argv, \"--\"); ++argv, --argc) {\n \t\tif (skip_prefix(*argv, \"--print-and-refresh=\", &name))\n@@ -56,6 +54,9 @@ int cmd__read_cache(int argc, const char **argv)\n \tsetup_git_directory();\n \tgit_config(git_default_config, NULL);\n \n+\tprepare_repo_settings(r);\n+\tr->settings.command_requires_full_index = 0;\n+\n \tfor (i = 0; i < cnt; i++) {\n \t\trepo_read_index(r);\n \n-- \ngitgitgadget\n\n"},{"id":"443036","messageId":"85e3e5c78e741496a1c99ccc84e59b16fa0a7158.1638566165.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 5/7] diff: replace --staged with --cached in t1092 tests","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:16:03Z","receivedAt":"2021-12-03T21:16:20Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nReplace uses of the synonym --staged in t1092 tests with --cached (which\nis the real and preferred option). This will allow consistency in the new\ntests to be added with the upcoming change to enable the sparse index for\ndiff.\n\nSigned-off-by: Lessley Dennington \n---\n t/t1092-sparse-checkout-compatibility.sh | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 77e302a0ef3..203a594fa45 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -371,7 +371,7 @@ test_expect_success 'checkout and reset --hard' '\n \ttest_all_match git reset --hard update-folder2\n '\n \n-test_expect_success 'diff --staged' '\n+test_expect_success 'diff --cached' '\n \tinit_repos &&\n \n \twrite_script edit-contents <<-\\EOF &&\n@@ -380,10 +380,10 @@ test_expect_success 'diff --staged' '\n \trun_on_all ../edit-contents &&\n \n \ttest_all_match git diff &&\n-\ttest_all_match git diff --staged &&\n+\ttest_all_match git diff --cached &&\n \ttest_all_match git add README.md &&\n \ttest_all_match git diff &&\n-\ttest_all_match git diff --staged\n+\ttest_all_match git diff --cached\n '\n \n # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n@@ -400,8 +400,8 @@ test_expect_success 'diff with renames and conflicts' '\n \t\ttest_all_match git checkout rename-base &&\n \t\ttest_all_match git checkout $branch -- . &&\n \t\ttest_all_match git status --porcelain=v2 &&\n-\t\ttest_all_match git diff --staged --no-renames &&\n-\t\ttest_all_match git diff --staged --find-renames || return 1\n+\t\ttest_all_match git diff --cached --no-renames &&\n+\t\ttest_all_match git diff --cached --find-renames || return 1\n \tdone\n '\n \n@@ -420,8 +420,8 @@ test_expect_success 'diff with directory/file conflicts' '\n \t\ttest_all_match git checkout $branch &&\n \t\ttest_all_match git checkout rename-base -- . &&\n \t\ttest_all_match git status --porcelain=v2 &&\n-\t\ttest_all_match git diff --staged --no-renames &&\n-\t\ttest_all_match git diff --staged --find-renames || return 1\n+\t\ttest_all_match git diff --cached --no-renames &&\n+\t\ttest_all_match git diff --cached --find-renames || return 1\n \tdone\n '\n \n-- \ngitgitgadget\n\n"},{"id":"443037","messageId":"4f16366e5adfd47094f971f8611533a85955aa54.1638566165.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 6/7] diff: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:16:04Z","receivedAt":"2021-12-03T21:16:21Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index within the 'git diff' command. Its implementation\nalready safely integrates with the sparse index because it shares code\nwith the 'git status' and 'git checkout' commands that were already\nintegrated. For more details see:\n\nd76723ee53 (status: use sparse-index throughout, 2021-07-14)\n1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n\nThe most interesting thing to do is to add tests that verify that 'git\ndiff' behaves correctly when the sparse index is enabled. These cases are:\n\n1. The index is not expanded for 'diff' and 'diff --staged'\n2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\ncheckout, and sparse index repositories in the following partially-staged\nscenarios (i.e. the index, HEAD, and working directory differ at a given\npath):\n 1. Path is within sparse-checkout cone\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n\nThe `p2000` tests demonstrate a ~44% execution time reduction for 'git\ndiff' and a ~86% execution time reduction for 'git diff --staged' using a\nsparse index:\n\nTest before after\n-------------------------------------------------------------\n2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n\nCo-authored-by: Derrick Stolee \nSigned-off-by: Derrick Stolee \nSigned-off-by: Lessley Dennington \n---\n builtin/diff.c | 5 +++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++\n 3 files changed, 53 insertions(+)\n\ndiff --git a/builtin/diff.c b/builtin/diff.c\nindex dd8ce688ba7..fa4683377eb 100644\n--- a/builtin/diff.c\n+++ b/builtin/diff.c\n@@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n+\tif (!nongit) {\n+\t\tprepare_repo_settings(the_repository);\n+\t\tthe_repository->settings.command_requires_full_index = 0;\n+\t}\n+\n \tif (!no_index) {\n \t\t/*\n \t\t * Treat git diff with at least one path outside of the\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bfd332120c8..5cf94627383 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n+test_perf_on_all git diff\n+test_perf_on_all git diff --cached\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 203a594fa45..abfb4994bb9 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -846,6 +846,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n \t)\n '\n \n+test_expect_success 'sparse index is not expanded: diff' '\n+\tinit_repos &&\n+\n+\twrite_script edit-contents <<-\\EOF &&\n+\techo text >>$1\n+\tEOF\n+\n+\t# Add file within cone\n+\ttest_sparse_match git sparse-checkout set deep &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\ttest_all_match git add deep/testfile &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --cached &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --cached &&\n+\n+\t# Add file outside cone\n+\ttest_all_match git reset --hard &&\n+\trun_on_all mkdir newdirectory &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set newdirectory &&\n+\ttest_all_match git add newdirectory/testfile &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --cached &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --cached &&\n+\n+\t# Merge conflict outside cone\n+\t# The sparse checkout will report a warning that is not in the\n+\t# full checkout, so we use `run_on_all` instead of\n+\t# `test_all_match`\n+\trun_on_all git reset --hard &&\n+\ttest_all_match git checkout merge-left &&\n+\ttest_all_match test_must_fail git merge merge-right &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --cached &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --cached\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n\n"},{"id":"443038","messageId":"0453237873422a3919200b3f641b7847d44f0efa.1638566166.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v5 7/7] blame: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-03T21:16:05Z","receivedAt":"2021-12-03T21:16:22Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index for the 'git blame' command. The index was already\nnot expanded with this command, so the most interesting thing to do is to\nadd tests that verify that 'git blame' behaves correctly when the sparse\nindex is enabled and that its performance improves. More specifically, these\ncases are:\n\n1. The index is not expanded for 'blame' when given paths in the sparse\ncheckout cone at multiple levels.\n\n2. Performance measurably improves for 'blame' with sparse index when given\npaths in the sparse checkout cone at multiple levels.\n\nThe `p2000` tests demonstrate a ~60% execution time reduction when running\n'blame' for a file two levels deep and and a ~30% execution time reduction\nfor a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nWe do not include paths outside the sparse checkout cone because blame\ndoes not support blaming files that are not present in the working\ndirectory. This is true in both sparse and full checkouts.\n\nSigned-off-by: Lessley Dennington \n---\n builtin/blame.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 2 +\n t/t1092-sparse-checkout-compatibility.sh | 49 ++++++++++++++++++------\n 3 files changed, 43 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 641523ff9af..33e411d2203 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -940,6 +940,9 @@ parse_done:\n \trevs.diffopt.flags.follow_renames = 0;\n \targc = parse_options_end(&ctx);\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tif (incremental || (output_option & OUTPUT_PORCELAIN)) {\n \t\tif (show_progress > 0)\n \t\t\tdie(_(\"--progress can't be used with --incremental or porcelain formats\"));\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex 5cf94627383..cb777c74a24 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n test_perf_on_all git diff --cached\n+test_perf_on_all git blame $SPARSE_CONE/a\n+test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex abfb4994bb9..6187a997b7d 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -442,21 +442,36 @@ test_expect_success 'log with pathspec outside sparse definition' '\n test_expect_success 'blame with pathspec inside sparse definition' '\n \tinit_repos &&\n \n-\ttest_all_match git blame a &&\n-\ttest_all_match git blame deep/a &&\n-\ttest_all_match git blame deep/deeper1/a &&\n-\ttest_all_match git blame deep/deeper1/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_all_match git blame $file\n+\tdone\n '\n \n-# TODO: blame currently does not support blaming files outside of the\n-# sparse definition. It complains that the file doesn't exist locally.\n-test_expect_failure 'blame with pathspec outside sparse definition' '\n+# Without a revision specified, blame will error if passed any file that\n+# is not present in the working directory (even if the file is tracked).\n+# Here we just verify that this is also true with sparse checkouts.\n+test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n+\ttest_sparse_match git sparse-checkout set &&\n \n-\ttest_all_match git blame folder1/a &&\n-\ttest_all_match git blame folder2/a &&\n-\ttest_all_match git blame deep/deeper2/a &&\n-\ttest_all_match git blame deep/deeper2/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_sparse_match test_must_fail git blame $file &&\n+\t\tcat >expect <<-EOF &&\n+\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n+\t\tEOF\n+\t\t# We compare sparse-checkout-err and sparse-index-err in\n+\t\t# `test_sparse_match`. Given we know they are the same, we\n+\t\t# only check the content of sparse-index-err here.\n+\t\ttest_cmp expect sparse-index-err\n+\tdone\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n@@ -892,6 +907,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n \tensure_not_expanded diff --cached\n '\n \n+test_expect_success 'sparse index is not expanded: blame' '\n+\tinit_repos &&\n+\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\tensure_not_expanded blame $file\n+\tdone\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n"},{"id":"443080","messageId":"CABPp-BEN2m9bqhQ+bDOa75-OWaZjXs7dv-=VBQTY9_O2LYGkJA@mail.gmail.com","threadId":"56710","inReplyTo":"09c2ff9f89833b3ac918a399e10d1b6abe71b339.1638566165.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 1/7] git: esnure correct git directory setup with -h","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-12-04T18:41:53Z","receivedAt":"2021-12-04T18:43:36Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Dec 3, 2021 at 1:16 PM Lessley Dennington via GitGitGadget\n wrote:\n\nSimple typo in the subject of the commit message: \"esnure\" -> \"ensure\"\n\n> From: Lessley Dennington \n>\n> Ensure correct git directory setup when -h is passed with commands. This\n> specifically applies to repos with special help text configuration\n> variables and to commands run with -h outside a repository. This\n> will also protect against test failures in the upcoming change to BUG in\n> prepare_repo_settings if no git directory exists.\n>\n> Note: this diff is better seen when ignoring whitespace changes.\n>\n> Co-authored-by: Junio C Hamano \n> Signed-off-by: Lessley Dennington \n> ---\n> git.c | 37 +++++++++++++++++++------------------\n> 1 file changed, 19 insertions(+), 18 deletions(-)\n>\n> diff --git a/git.c b/git.c\n> index 60c2784be45..03a2dfa5174 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -421,27 +421,28 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n> int status, help;\n> struct stat st;\n> const char *prefix;\n> -\n> + int run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n> prefix = NULL;\n> help = argc == 2 && !strcmp(argv[1], \"-h\");\n> - if (!help) {\n> - if (p->option & RUN_SETUP)\n> - prefix = setup_git_directory();\n> - else if (p->option & RUN_SETUP_GENTLY) {\n> - int nongit_ok;\n> - prefix = setup_git_directory_gently(&nongit_ok);\n> - }\n> - precompose_argv_prefix(argc, argv, NULL);\n> - if (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY) &&\n> - !(p->option & DELAY_PAGER_CONFIG))\n> - use_pager = check_pager_config(p->cmd);\n> - if (use_pager == -1 && p->option & USE_PAGER)\n> - use_pager = 1;\n> -\n> - if ((p->option & (RUN_SETUP | RUN_SETUP_GENTLY)) &&\n> - startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */\n> - trace_repo_setup(prefix);\n> + if (help && (run_setup & RUN_SETUP))\n> + /* demote to GENTLY to allow 'git cmd -h' outside repo */\n> + run_setup = RUN_SETUP_GENTLY;\n> +\n> + if (run_setup & RUN_SETUP)\n> + prefix = setup_git_directory();\n> + else if (run_setup & RUN_SETUP_GENTLY) {\n> + int nongit_ok;\n> + prefix = setup_git_directory_gently(&nongit_ok);\n> }\n> + precompose_argv_prefix(argc, argv, NULL);\n> + if (use_pager == -1 && run_setup &&\n> + !(p->option & DELAY_PAGER_CONFIG))\n> + use_pager = check_pager_config(p->cmd);\n> + if (use_pager == -1 && p->option & USE_PAGER)\n> + use_pager = 1;\n> + if (run_setup && startup_info->have_repository)\n> + /* get_git_dir() may set up repo, avoid that */\n> + trace_repo_setup(prefix);\n> commit_pager_choice();\n>\n> if (!help && get_super_prefix()) {\n> --\n> gitgitgadget\n"},{"id":"443081","messageId":"CABPp-BFLxHb7XQuJ+cPxm6N-TAJGZ-jeTbfGAemr91q7JJutbw@mail.gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 0/7] Sparse Index: diff and blame builtins","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-12-04T19:43:05Z","receivedAt":"2021-12-04T19:43:22Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Dec 3, 2021 at 1:16 PM Lessley Dennington via GitGitGadget\n wrote:\n>\n> This series is based on vd/sparse-reset. It integrates the sparse index with\n> git diff and git blame and includes:\n>\n> 1. tests added to t1092 and p2000 to establish the baseline functionality\n> of the commands\n> 2. repository settings to enable the sparse index\n>\n> The p2000 tests demonstrate a ~44% execution time reduction for 'git diff'\n> and a ~86% execution time reduction for 'git diff --staged' using a sparse\n> index. For 'git blame', the reduction time was ~60% for a file two levels\n> deep and ~30% for a file three levels deep.\n>\n> Test before after\n> ----------------------------------------------------------------\n> 2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n> 2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n> 2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n> 2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n> 2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n> 2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n> 2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n> 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n> 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n> 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n> 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n> 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n> 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n> 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n> 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n> 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n>\n>\n>\n> Changes since V1\n> ================\n>\n> * Fix failing diff partially-staged test in\n> t1092-sparse-checkout-compatibility.sh, which was breaking in seen.\n>\n>\n> Changes since V2\n> ================\n>\n> * Update diff commit description to include patches that make the checkout\n> and status commands work with the sparse index for readers to reference.\n> * Add new test case to verify diff behaves as expected when run against\n> files outside the sparse checkout cone.\n> * Indent error message in blame commit\n> * Check error message in blame with pathspec outside sparse definition test\n> matches expectations.\n> * Loop blame tests (instead of running the same command multiple time\n> against different files).\n>\n>\n> Changes since V3\n> ================\n>\n> * Update diff p2000 tests to use --cached instead of --staged. Execute new\n> run and update results in commit description and cover letter.\n> * Update comment on blame with pathspec outside sparse definition test in\n> t1092-sparse-checkout-compatibility.sh to clarify that it tests the\n> current state and could be improved in the future.\n> * Ensure sparse index is only activated when diff is running against files\n> in a Git repo.\n> * BUG if prepare_repo_settings() is called outside a repository.\n> * Ensure sparse index is not activated for calls to blame, checkout, or\n> pack-object with -h.\n> * Ensure commit-graph is only loaded if a git directory exists.\n>\n>\n> Changes since V4\n> ================\n>\n> * Remove startup_info->have_repository check from checkout, pack-objects,\n> and blame. Update git.c to no longer bypass setup when -h is passed\n> instead.\n> * Move commit-graph, test-read-cache, and repo-settings changes into their\n> own patches with details in commit description of why the changes are\n> being made.\n> * Update t1092-sparse-checkout-compatibility.sh tests to use --cached\n> instead of --staged.\n> * Use 10-character hash abbreviations for commits referenced in diff commit\n> message.\n> * Clarify that being unable to blame files outside the working directory is\n> not supported in either sparse or non-sparse checkouts both in comment on\n> blame with pathspec outside sparse definition test in\n> t1092-sparse-checkout-compatibility.sh and blame commit message.\n\nThis round addresses all my concerns from previous rounds. There's a\ntrivial typo in the subject of the new patch 1, but feel free to add\nmy\n\nReviewed-by: Elijah Newren \n\nwhen you resubmit with that fix. Nice work!\n"},{"id":"443082","messageId":"xmqq1r2sp2g4.fsf@gitster.g","threadId":"56710","inReplyTo":"09c2ff9f89833b3ac918a399e10d1b6abe71b339.1638566165.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 1/7] git: esnure correct git directory setup with -h","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-12-04T19:58:19Z","receivedAt":"2021-12-04T19:58:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Lessley Dennington via GitGitGadget\" \nwrites:\n\n> @@ -421,27 +421,28 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n> \tint status, help;\n> \tstruct stat st;\n> \tconst char *prefix;\n> -\n> +\tint run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n> \tprefix = NULL;\n\nDon't lose the blank line that separates the variable declarations\nand the first statement. Also, it probably makes sense to remove\nthe assignment of NULL to prefix from here (see below).\n\n> \thelp = argc == 2 && !strcmp(argv[1], \"-h\");\n> -...\n> +\tif (help && (run_setup & RUN_SETUP))\n> +\t\t/* demote to GENTLY to allow 'git cmd -h' outside repo */\n> +\t\trun_setup = RUN_SETUP_GENTLY;\n\nOK.\n\n> +\tif (run_setup & RUN_SETUP)\n> +\t\tprefix = setup_git_directory();\n> +\telse if (run_setup & RUN_SETUP_GENTLY) {\n> +\t\tint nongit_ok;\n> +\t\tprefix = setup_git_directory_gently(&nongit_ok);\n> \t}\n\nHere, we say \"depending on how run_setup is specified, we compute\nthe prefix in different ways\". So the \"we do not run setup, so set\nprefix to NULL\" naturally belongs here. Perhaps something like...\n\n\tif (run_setup & RUN_SETUP) {\n\t\tprefix = setup_git_directory();\n\t} else if (run_setup & RUN_SETUP_GENTLY) {\n\t\tint nongit_ok;\n\t\tprefix = setup_git_directory_gently(&nongit_ok);\n\t} else {\n\t\tprefix = NULL;\n\t}\n\n> +\tprecompose_argv_prefix(argc, argv, NULL);\n> +\tif (use_pager == -1 && run_setup &&\n> +\t\t!(p->option & DELAY_PAGER_CONFIG))\n> +\t\tuse_pager = check_pager_config(p->cmd);\n> +\tif (use_pager == -1 && p->option & USE_PAGER)\n> +\t\tuse_pager = 1;\n> +\tif (run_setup && startup_info->have_repository)\n> +\t\t/* get_git_dir() may set up repo, avoid that */\n> +\t\ttrace_repo_setup(prefix);\n> \tcommit_pager_choice();\n> \n> \tif (!help && get_super_prefix()) {\n"},{"id":"443156","messageId":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v5.git.1638566165.gitgitgadget@gmail.com","subject":"[PATCH v6 0/7] Sparse Index: diff and blame builtins","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:55:54Z","receivedAt":"2021-12-06T16:10:39Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"This series is based on vd/sparse-reset. It integrates the sparse index with\ngit diff and git blame and includes:\n\n 1. tests added to t1092 and p2000 to establish the baseline functionality\n of the commands\n 2. repository settings to enable the sparse index\n\nThe p2000 tests demonstrate a ~44% execution time reduction for 'git diff'\nand a ~86% execution time reduction for 'git diff --staged' using a sparse\nindex. For 'git blame', the reduction time was ~60% for a file two levels\ndeep and ~30% for a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\n\n\nChanges since V1\n================\n\n * Fix failing diff partially-staged test in\n t1092-sparse-checkout-compatibility.sh, which was breaking in seen.\n\n\nChanges since V2\n================\n\n * Update diff commit description to include patches that make the checkout\n and status commands work with the sparse index for readers to reference.\n * Add new test case to verify diff behaves as expected when run against\n files outside the sparse checkout cone.\n * Indent error message in blame commit\n * Check error message in blame with pathspec outside sparse definition test\n matches expectations.\n * Loop blame tests (instead of running the same command multiple time\n against different files).\n\n\nChanges since V3\n================\n\n * Update diff p2000 tests to use --cached instead of --staged. Execute new\n run and update results in commit description and cover letter.\n * Update comment on blame with pathspec outside sparse definition test in\n t1092-sparse-checkout-compatibility.sh to clarify that it tests the\n current state and could be improved in the future.\n * Ensure sparse index is only activated when diff is running against files\n in a Git repo.\n * BUG if prepare_repo_settings() is called outside a repository.\n * Ensure sparse index is not activated for calls to blame, checkout, or\n pack-object with -h.\n * Ensure commit-graph is only loaded if a git directory exists.\n\n\nChanges since V4\n================\n\n * Remove startup_info->have_repository check from checkout, pack-objects,\n and blame. Update git.c to no longer bypass setup when -h is passed\n instead.\n * Move commit-graph, test-read-cache, and repo-settings changes into their\n own patches with details in commit description of why the changes are\n being made.\n * Update t1092-sparse-checkout-compatibility.sh tests to use --cached\n instead of --staged.\n * Use 10-character hash abbreviations for commits referenced in diff commit\n message.\n * Clarify that being unable to blame files outside the working directory is\n not supported in either sparse or non-sparse checkouts both in comment on\n blame with pathspec outside sparse definition test in\n t1092-sparse-checkout-compatibility.sh and blame commit message.\n\n\nChanges since V5\n================\n\n * Fix commit message typo.\n * Re-add blank line to separate variable declarations from statements in\n run_builtin.\n * Refactor prefix NULL assignment in run_builtin.\n\nThanks, Lessley\n\nLessley Dennington (7):\n git: ensure correct git directory setup with -h\n commit-graph: return if there is no git directory\n test-read-cache: set up repo after git directory\n repo-settings: prepare_repo_settings only in git repos\n diff: replace --staged with --cached in t1092 tests\n diff: enable and test the sparse index\n blame: enable and test the sparse index\n\n builtin/blame.c | 3 +\n builtin/diff.c | 5 ++\n commit-graph.c | 5 +-\n git.c | 39 ++++----\n repo-settings.c | 3 +\n t/helper/test-read-cache.c | 5 +-\n t/perf/p2000-sparse-operations.sh | 4 +\n t/t1092-sparse-checkout-compatibility.sh | 109 +++++++++++++++++++----\n 8 files changed, 134 insertions(+), 39 deletions(-)\n\n\nbase-commit: f2a454e0a5e26c0f7b840970f69d195c37b16565\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1050%2Fldennington%2Fdiff-blame-sparse-index-v6\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1050/ldennington/diff-blame-sparse-index-v6\nPull-Request: https://github.com/gitgitgadget/git/pull/1050\n\nRange-diff vs v5:\n\n 1: 09c2ff9f898 ! 1: efdd55c126d git: esnure correct git directory setup with -h\n @@ Metadata\n Author: Lessley Dennington \n \n ## Commit message ##\n - git: esnure correct git directory setup with -h\n + git: ensure correct git directory setup with -h\n \n Ensure correct git directory setup when -h is passed with commands. This\n specifically applies to repos with special help text configuration\n @@ Commit message\n \n Co-authored-by: Junio C Hamano \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## git.c ##\n @@ git.c: static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \tint status, help;\n \tstruct stat st;\n \tconst char *prefix;\n --\n +\tint run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n - \tprefix = NULL;\n + \n +-\tprefix = NULL;\n \thelp = argc == 2 && !strcmp(argv[1], \"-h\");\n -\tif (!help) {\n -\t\tif (p->option & RUN_SETUP)\n @@ git.c: static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n +\t\t/* demote to GENTLY to allow 'git cmd -h' outside repo */\n +\t\trun_setup = RUN_SETUP_GENTLY;\n +\n -+\tif (run_setup & RUN_SETUP)\n ++\tif (run_setup & RUN_SETUP) {\n +\t\tprefix = setup_git_directory();\n -+\telse if (run_setup & RUN_SETUP_GENTLY) {\n ++\t} else if (run_setup & RUN_SETUP_GENTLY) {\n +\t\tint nongit_ok;\n +\t\tprefix = setup_git_directory_gently(&nongit_ok);\n ++\t} else {\n ++\t\tprefix = NULL;\n \t}\n +\tprecompose_argv_prefix(argc, argv, NULL);\n +\tif (use_pager == -1 && run_setup &&\n 2: 9e53a6435e4 ! 2: f676f03ccb0 commit-graph: return if there is no git directory\n @@ Commit message\n git directory exists.\n \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## commit-graph.c ##\n @@ commit-graph.c: static int prepare_commit_graph(struct repository *r)\n 3: 219a4158b6a ! 3: 7b1fab86a4a test-read-cache: set up repo after git directory\n @@ Commit message\n prepare_repo_settings if no git directory exists.\n \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## t/helper/test-read-cache.c ##\n @@ t/helper/test-read-cache.c: int cmd__read_cache(int argc, const char **argv)\n 4: 4d8d58c473b ! 4: fd28be71ca4 repo-settings: prepare_repo_settings only in git repos\n @@ Commit message\n settings.\n \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## repo-settings.c ##\n @@ repo-settings.c: void prepare_repo_settings(struct repository *r)\n 5: 85e3e5c78e7 ! 5: 2a1524a7e9a diff: replace --staged with --cached in t1092 tests\n @@ Commit message\n diff.\n \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## t/t1092-sparse-checkout-compatibility.sh ##\n @@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'checkout and reset --hard' '\n 6: 4f16366e5ad ! 6: 897611682af diff: enable and test the sparse index\n @@ Commit message\n 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n \n Co-authored-by: Derrick Stolee \n - Signed-off-by: Derrick Stolee \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## builtin/diff.c ##\n @@ builtin/diff.c: int cmd_diff(int argc, const char **argv, const char *prefix)\n 7: 04532378734 ! 7: 85bcbaa1771 blame: enable and test the sparse index\n @@ Commit message\n directory. This is true in both sparse and full checkouts.\n \n Signed-off-by: Lessley Dennington \n + Reviewed-by: Elijah Newren \n \n ## builtin/blame.c ##\n @@ builtin/blame.c: parse_done:\n\n-- \ngitgitgadget\n"},{"id":"443157","messageId":"efdd55c126d9704e750f00d9fc3f8e99a7c7343a.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 1/7] git: ensure correct git directory setup with -h","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:55:55Z","receivedAt":"2021-12-06T16:10:41Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnsure correct git directory setup when -h is passed with commands. This\nspecifically applies to repos with special help text configuration\nvariables and to commands run with -h outside a repository. This\nwill also protect against test failures in the upcoming change to BUG in\nprepare_repo_settings if no git directory exists.\n\nNote: this diff is better seen when ignoring whitespace changes.\n\nCo-authored-by: Junio C Hamano \nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n git.c | 39 +++++++++++++++++++++------------------\n 1 file changed, 21 insertions(+), 18 deletions(-)\n\ndiff --git a/git.c b/git.c\nindex 60c2784be45..eb6890087c3 100644\n--- a/git.c\n+++ b/git.c\n@@ -421,27 +421,30 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \tint status, help;\n \tstruct stat st;\n \tconst char *prefix;\n+\tint run_setup = (p->option & (RUN_SETUP | RUN_SETUP_GENTLY));\n \n-\tprefix = NULL;\n \thelp = argc == 2 && !strcmp(argv[1], \"-h\");\n-\tif (!help) {\n-\t\tif (p->option & RUN_SETUP)\n-\t\t\tprefix = setup_git_directory();\n-\t\telse if (p->option & RUN_SETUP_GENTLY) {\n-\t\t\tint nongit_ok;\n-\t\t\tprefix = setup_git_directory_gently(&nongit_ok);\n-\t\t}\n-\t\tprecompose_argv_prefix(argc, argv, NULL);\n-\t\tif (use_pager == -1 && p->option & (RUN_SETUP | RUN_SETUP_GENTLY) &&\n-\t\t !(p->option & DELAY_PAGER_CONFIG))\n-\t\t\tuse_pager = check_pager_config(p->cmd);\n-\t\tif (use_pager == -1 && p->option & USE_PAGER)\n-\t\t\tuse_pager = 1;\n-\n-\t\tif ((p->option & (RUN_SETUP | RUN_SETUP_GENTLY)) &&\n-\t\t startup_info->have_repository) /* get_git_dir() may set up repo, avoid that */\n-\t\t\ttrace_repo_setup(prefix);\n+\tif (help && (run_setup & RUN_SETUP))\n+\t\t/* demote to GENTLY to allow 'git cmd -h' outside repo */\n+\t\trun_setup = RUN_SETUP_GENTLY;\n+\n+\tif (run_setup & RUN_SETUP) {\n+\t\tprefix = setup_git_directory();\n+\t} else if (run_setup & RUN_SETUP_GENTLY) {\n+\t\tint nongit_ok;\n+\t\tprefix = setup_git_directory_gently(&nongit_ok);\n+\t} else {\n+\t\tprefix = NULL;\n \t}\n+\tprecompose_argv_prefix(argc, argv, NULL);\n+\tif (use_pager == -1 && run_setup &&\n+\t\t!(p->option & DELAY_PAGER_CONFIG))\n+\t\tuse_pager = check_pager_config(p->cmd);\n+\tif (use_pager == -1 && p->option & USE_PAGER)\n+\t\tuse_pager = 1;\n+\tif (run_setup && startup_info->have_repository)\n+\t\t/* get_git_dir() may set up repo, avoid that */\n+\t\ttrace_repo_setup(prefix);\n \tcommit_pager_choice();\n \n \tif (!help && get_super_prefix()) {\n-- \ngitgitgadget\n\n"},{"id":"443158","messageId":"f676f03ccb024b9011eff999b342f96cfb76b892.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 2/7] commit-graph: return if there is no git directory","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:55:56Z","receivedAt":"2021-12-06T16:10:42Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nReturn early if git directory does not exist. This will protect against\ntest failures in the upcoming change to BUG in prepare_repo_settings if no\ngit directory exists.\n\nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n commit-graph.c | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 2706683acfe..265c010122e 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -632,10 +632,13 @@ static int prepare_commit_graph(struct repository *r)\n \tstruct object_directory *odb;\n \n \t/*\n+\t * Early return if there is no git dir or if the commit graph is\n+\t * disabled.\n+\t *\n \t * This must come before the \"already attempted?\" check below, because\n \t * we want to disable even an already-loaded graph file.\n \t */\n-\tif (r->commit_graph_disabled)\n+\tif (!r->gitdir || r->commit_graph_disabled)\n \t\treturn 0;\n \n \tif (r->objects->commit_graph_attempted)\n-- \ngitgitgadget\n\n"},{"id":"443159","messageId":"7b1fab86a4a15f33b37ac3364a599c3f5237bf21.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 3/7] test-read-cache: set up repo after git directory","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:55:57Z","receivedAt":"2021-12-06T16:10:46Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nMove repo setup to occur after git directory is set up. This will protect\nagainst test failures in the upcoming change to BUG in\nprepare_repo_settings if no git directory exists.\n\nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n t/helper/test-read-cache.c | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/t/helper/test-read-cache.c b/t/helper/test-read-cache.c\nindex b52c174acc7..0d9f08931a1 100644\n--- a/t/helper/test-read-cache.c\n+++ b/t/helper/test-read-cache.c\n@@ -39,8 +39,6 @@ int cmd__read_cache(int argc, const char **argv)\n \tint table = 0, expand = 0;\n \n \tinitialize_the_repository();\n-\tprepare_repo_settings(r);\n-\tr->settings.command_requires_full_index = 0;\n \n \tfor (++argv, --argc; *argv && starts_with(*argv, \"--\"); ++argv, --argc) {\n \t\tif (skip_prefix(*argv, \"--print-and-refresh=\", &name))\n@@ -56,6 +54,9 @@ int cmd__read_cache(int argc, const char **argv)\n \tsetup_git_directory();\n \tgit_config(git_default_config, NULL);\n \n+\tprepare_repo_settings(r);\n+\tr->settings.command_requires_full_index = 0;\n+\n \tfor (i = 0; i < cnt; i++) {\n \t\trepo_read_index(r);\n \n-- \ngitgitgadget\n\n"},{"id":"443160","messageId":"fd28be71ca47a71b224a5b08cb81f903ada6a2bd.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 4/7] repo-settings: prepare_repo_settings only in git repos","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:55:58Z","receivedAt":"2021-12-06T16:10:47Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nCheck whether git directory exists before adding any repo settings. If it\ndoes not exist, BUG with the message that one cannot add settings for an\nuninitialized repository. If it does exist, proceed with adding repo\nsettings.\n\nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n repo-settings.c | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/repo-settings.c b/repo-settings.c\nindex b93e91a212e..00ca5571a1a 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n \tchar *strval;\n \tint manyfiles;\n \n+\tif (!r->gitdir)\n+\t\tBUG(\"Cannot add settings for uninitialized repository\");\n+\n \tif (r->settings.initialized++)\n \t\treturn;\n \n-- \ngitgitgadget\n\n"},{"id":"443161","messageId":"2a1524a7e9ac267a550d5b88c02f853fe36fe08c.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 5/7] diff: replace --staged with --cached in t1092 tests","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:55:59Z","receivedAt":"2021-12-06T16:11:01Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nReplace uses of the synonym --staged in t1092 tests with --cached (which\nis the real and preferred option). This will allow consistency in the new\ntests to be added with the upcoming change to enable the sparse index for\ndiff.\n\nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n t/t1092-sparse-checkout-compatibility.sh | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 77e302a0ef3..203a594fa45 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -371,7 +371,7 @@ test_expect_success 'checkout and reset --hard' '\n \ttest_all_match git reset --hard update-folder2\n '\n \n-test_expect_success 'diff --staged' '\n+test_expect_success 'diff --cached' '\n \tinit_repos &&\n \n \twrite_script edit-contents <<-\\EOF &&\n@@ -380,10 +380,10 @@ test_expect_success 'diff --staged' '\n \trun_on_all ../edit-contents &&\n \n \ttest_all_match git diff &&\n-\ttest_all_match git diff --staged &&\n+\ttest_all_match git diff --cached &&\n \ttest_all_match git add README.md &&\n \ttest_all_match git diff &&\n-\ttest_all_match git diff --staged\n+\ttest_all_match git diff --cached\n '\n \n # NEEDSWORK: sparse-checkout behaves differently from full-checkout when\n@@ -400,8 +400,8 @@ test_expect_success 'diff with renames and conflicts' '\n \t\ttest_all_match git checkout rename-base &&\n \t\ttest_all_match git checkout $branch -- . &&\n \t\ttest_all_match git status --porcelain=v2 &&\n-\t\ttest_all_match git diff --staged --no-renames &&\n-\t\ttest_all_match git diff --staged --find-renames || return 1\n+\t\ttest_all_match git diff --cached --no-renames &&\n+\t\ttest_all_match git diff --cached --find-renames || return 1\n \tdone\n '\n \n@@ -420,8 +420,8 @@ test_expect_success 'diff with directory/file conflicts' '\n \t\ttest_all_match git checkout $branch &&\n \t\ttest_all_match git checkout rename-base -- . &&\n \t\ttest_all_match git status --porcelain=v2 &&\n-\t\ttest_all_match git diff --staged --no-renames &&\n-\t\ttest_all_match git diff --staged --find-renames || return 1\n+\t\ttest_all_match git diff --cached --no-renames &&\n+\t\ttest_all_match git diff --cached --find-renames || return 1\n \tdone\n '\n \n-- \ngitgitgadget\n\n"},{"id":"443162","messageId":"897611682af64ba6bd0d2dfcfeae56cfe953c45e.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 6/7] diff: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:56:00Z","receivedAt":"2021-12-06T16:11:04Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index within the 'git diff' command. Its implementation\nalready safely integrates with the sparse index because it shares code\nwith the 'git status' and 'git checkout' commands that were already\nintegrated. For more details see:\n\nd76723ee53 (status: use sparse-index throughout, 2021-07-14)\n1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29)\n\nThe most interesting thing to do is to add tests that verify that 'git\ndiff' behaves correctly when the sparse index is enabled. These cases are:\n\n1. The index is not expanded for 'diff' and 'diff --staged'\n2. 'diff' and 'diff --staged' behave the same in full checkout, sparse\ncheckout, and sparse index repositories in the following partially-staged\nscenarios (i.e. the index, HEAD, and working directory differ at a given\npath):\n 1. Path is within sparse-checkout cone\n 2. Path is outside sparse-checkout cone\n 3. A merge conflict exists for paths outside sparse-checkout cone\n\nThe `p2000` tests demonstrate a ~44% execution time reduction for 'git\ndiff' and a ~86% execution time reduction for 'git diff --staged' using a\nsparse index:\n\nTest before after\n-------------------------------------------------------------\n2000.30: git diff (full-v3) 0.33 0.34 +3.0%\n2000.31: git diff (full-v4) 0.33 0.35 +6.1%\n2000.32: git diff (sparse-v3) 0.53 0.31 -41.5%\n2000.33: git diff (sparse-v4) 0.54 0.29 -46.3%\n2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0%\n2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3%\n2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7%\n2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0%\n\nCo-authored-by: Derrick Stolee \nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n builtin/diff.c | 5 +++\n t/perf/p2000-sparse-operations.sh | 2 ++\n t/t1092-sparse-checkout-compatibility.sh | 46 ++++++++++++++++++++++++\n 3 files changed, 53 insertions(+)\n\ndiff --git a/builtin/diff.c b/builtin/diff.c\nindex dd8ce688ba7..fa4683377eb 100644\n--- a/builtin/diff.c\n+++ b/builtin/diff.c\n@@ -437,6 +437,11 @@ int cmd_diff(int argc, const char **argv, const char *prefix)\n \n \tprefix = setup_git_directory_gently(&nongit);\n \n+\tif (!nongit) {\n+\t\tprepare_repo_settings(the_repository);\n+\t\tthe_repository->settings.command_requires_full_index = 0;\n+\t}\n+\n \tif (!no_index) {\n \t\t/*\n \t\t * Treat git diff with at least one path outside of the\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex bfd332120c8..5cf94627383 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -113,5 +113,7 @@ test_perf_on_all git checkout -f -\n test_perf_on_all git reset\n test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n+test_perf_on_all git diff\n+test_perf_on_all git diff --cached\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex 203a594fa45..abfb4994bb9 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -846,6 +846,52 @@ test_expect_success 'sparse-index is not expanded: merge conflict in cone' '\n \t)\n '\n \n+test_expect_success 'sparse index is not expanded: diff' '\n+\tinit_repos &&\n+\n+\twrite_script edit-contents <<-\\EOF &&\n+\techo text >>$1\n+\tEOF\n+\n+\t# Add file within cone\n+\ttest_sparse_match git sparse-checkout set deep &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\ttest_all_match git add deep/testfile &&\n+\trun_on_all ../edit-contents deep/testfile &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --cached &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --cached &&\n+\n+\t# Add file outside cone\n+\ttest_all_match git reset --hard &&\n+\trun_on_all mkdir newdirectory &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set newdirectory &&\n+\ttest_all_match git add newdirectory/testfile &&\n+\trun_on_all ../edit-contents newdirectory/testfile &&\n+\ttest_sparse_match git sparse-checkout set &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --cached &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --cached &&\n+\n+\t# Merge conflict outside cone\n+\t# The sparse checkout will report a warning that is not in the\n+\t# full checkout, so we use `run_on_all` instead of\n+\t# `test_all_match`\n+\trun_on_all git reset --hard &&\n+\ttest_all_match git checkout merge-left &&\n+\ttest_all_match test_must_fail git merge merge-right &&\n+\n+\ttest_all_match git diff &&\n+\ttest_all_match git diff --cached &&\n+\tensure_not_expanded diff &&\n+\tensure_not_expanded diff --cached\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n\n"},{"id":"443163","messageId":"85bcbaa1771c97a0f6ca3cc03c80f02ee2f84061.1638806161.git.gitgitgadget@gmail.com","threadId":"56710","inReplyTo":"pull.1050.v6.git.1638806161.gitgitgadget@gmail.com","subject":"[PATCH v6 7/7] blame: enable and test the sparse index","fromName":"Lessley Dennington via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-12-06T15:56:01Z","receivedAt":"2021-12-06T16:11:05Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"From: Lessley Dennington \n\nEnable the sparse index for the 'git blame' command. The index was already\nnot expanded with this command, so the most interesting thing to do is to\nadd tests that verify that 'git blame' behaves correctly when the sparse\nindex is enabled and that its performance improves. More specifically, these\ncases are:\n\n1. The index is not expanded for 'blame' when given paths in the sparse\ncheckout cone at multiple levels.\n\n2. Performance measurably improves for 'blame' with sparse index when given\npaths in the sparse checkout cone at multiple levels.\n\nThe `p2000` tests demonstrate a ~60% execution time reduction when running\n'blame' for a file two levels deep and and a ~30% execution time reduction\nfor a file three levels deep.\n\nTest before after\n----------------------------------------------------------------\n2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2%\n2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9%\n2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2%\n2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6%\n2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4%\n2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8%\n2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7%\n2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5%\n\nWe do not include paths outside the sparse checkout cone because blame\ndoes not support blaming files that are not present in the working\ndirectory. This is true in both sparse and full checkouts.\n\nSigned-off-by: Lessley Dennington \nReviewed-by: Elijah Newren \n---\n builtin/blame.c | 3 ++\n t/perf/p2000-sparse-operations.sh | 2 +\n t/t1092-sparse-checkout-compatibility.sh | 49 ++++++++++++++++++------\n 3 files changed, 43 insertions(+), 11 deletions(-)\n\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 641523ff9af..33e411d2203 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -940,6 +940,9 @@ parse_done:\n \trevs.diffopt.flags.follow_renames = 0;\n \targc = parse_options_end(&ctx);\n \n+\tprepare_repo_settings(the_repository);\n+\tthe_repository->settings.command_requires_full_index = 0;\n+\n \tif (incremental || (output_option & OUTPUT_PORCELAIN)) {\n \t\tif (show_progress > 0)\n \t\t\tdie(_(\"--progress can't be used with --incremental or porcelain formats\"));\ndiff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh\nindex 5cf94627383..cb777c74a24 100755\n--- a/t/perf/p2000-sparse-operations.sh\n+++ b/t/perf/p2000-sparse-operations.sh\n@@ -115,5 +115,7 @@ test_perf_on_all git reset --hard\n test_perf_on_all git reset -- does-not-exist\n test_perf_on_all git diff\n test_perf_on_all git diff --cached\n+test_perf_on_all git blame $SPARSE_CONE/a\n+test_perf_on_all git blame $SPARSE_CONE/f3/a\n \n test_done\ndiff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh\nindex abfb4994bb9..6187a997b7d 100755\n--- a/t/t1092-sparse-checkout-compatibility.sh\n+++ b/t/t1092-sparse-checkout-compatibility.sh\n@@ -442,21 +442,36 @@ test_expect_success 'log with pathspec outside sparse definition' '\n test_expect_success 'blame with pathspec inside sparse definition' '\n \tinit_repos &&\n \n-\ttest_all_match git blame a &&\n-\ttest_all_match git blame deep/a &&\n-\ttest_all_match git blame deep/deeper1/a &&\n-\ttest_all_match git blame deep/deeper1/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_all_match git blame $file\n+\tdone\n '\n \n-# TODO: blame currently does not support blaming files outside of the\n-# sparse definition. It complains that the file doesn't exist locally.\n-test_expect_failure 'blame with pathspec outside sparse definition' '\n+# Without a revision specified, blame will error if passed any file that\n+# is not present in the working directory (even if the file is tracked).\n+# Here we just verify that this is also true with sparse checkouts.\n+test_expect_success 'blame with pathspec outside sparse definition' '\n \tinit_repos &&\n+\ttest_sparse_match git sparse-checkout set &&\n \n-\ttest_all_match git blame folder1/a &&\n-\ttest_all_match git blame folder2/a &&\n-\ttest_all_match git blame deep/deeper2/a &&\n-\ttest_all_match git blame deep/deeper2/deepest/a\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\ttest_sparse_match test_must_fail git blame $file &&\n+\t\tcat >expect <<-EOF &&\n+\t\tfatal: Cannot lstat '\"'\"'$file'\"'\"': No such file or directory\n+\t\tEOF\n+\t\t# We compare sparse-checkout-err and sparse-index-err in\n+\t\t# `test_sparse_match`. Given we know they are the same, we\n+\t\t# only check the content of sparse-index-err here.\n+\t\ttest_cmp expect sparse-index-err\n+\tdone\n '\n \n test_expect_success 'checkout and reset (mixed)' '\n@@ -892,6 +907,18 @@ test_expect_success 'sparse index is not expanded: diff' '\n \tensure_not_expanded diff --cached\n '\n \n+test_expect_success 'sparse index is not expanded: blame' '\n+\tinit_repos &&\n+\n+\tfor file in a \\\n+\t\t\tdeep/a \\\n+\t\t\tdeep/deeper1/a \\\n+\t\t\tdeep/deeper1/deepest/a\n+\tdo\n+\t\tensure_not_expanded blame $file\n+\tdone\n+'\n+\n # NEEDSWORK: a sparse-checkout behaves differently from a full checkout\n # in this scenario, but it shouldn't.\n test_expect_success 'reset mixed and checkout orphan' '\n-- \ngitgitgadget\n"},{"id":"443229","messageId":"211207.86r1apow9f.gmgdl@evledraar.gmail.com","threadId":"56710","inReplyTo":"4d8d58c473b51cac6645a7d71336e995c1f3d28b.1638566165.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 4/7] repo-settings: prepare_repo_settings only in git repos","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-12-07T04:43:32Z","receivedAt":"2021-12-07T04:48:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Dec 03 2021, Lessley Dennington via GitGitGadget wrote:\n\n> From: Lessley Dennington \n>\n> Check whether git directory exists before adding any repo settings. If it\n> does not exist, BUG with the message that one cannot add settings for an\n> uninitialized repository. If it does exist, proceed with adding repo\n> settings.\n>\n> Signed-off-by: Lessley Dennington \n> ---\n> repo-settings.c | 3 +++\n> 1 file changed, 3 insertions(+)\n>\n> diff --git a/repo-settings.c b/repo-settings.c\n> index b93e91a212e..00ca5571a1a 100644\n> --- a/repo-settings.c\n> +++ b/repo-settings.c\n> @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n> \tchar *strval;\n> \tint manyfiles;\n> \n> +\tif (!r->gitdir)\n> +\t\tBUG(\"Cannot add settings for uninitialized repository\");\n\nnit: start BUG(), error() etc. messages with lower-case.\n\n> +\n> \tif (r->settings.initialized++)\n> \t\treturn;\n\nOur config doesn't require us to have a repo, and most of what\nprepare_repo_settings() is doing is reading global config.\n\nI think that *currently* this won't break things, but e.g. if we ever\nwant to have \"feature.experimental\" or whatever change the behavior of a\na command that doesn't require a repository we'd need to untangle this\n(currently everything it changes requires a repo AFAICT).\n\nPerhaps this is fine, and if we ever need such a \"global config\" point\nwe should stick it closer to common-main.c...\n"},{"id":"443468","messageId":"53695770-68b8-ab92-1c35-268953160076@gmail.com","threadId":"56710","inReplyTo":"211207.86r1apow9f.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v5 4/7] repo-settings: prepare_repo_settings only in git repos","fromName":"Lessley Dennington","fromEmail":"lessleydennington@gmail.com","sentAt":"2021-12-08T15:46:22Z","receivedAt":"2021-12-08T15:46:26Z","isPatch":true,"sender":{"key":"lessleydennington@gmail.com","avatar":"https://avatars.githubusercontent.com/u/11321782?v=4"},"body":"\n\nOn 12/6/21 8:43 PM, Ævar Arnfjörð Bjarmason wrote:\n> \n> On Fri, Dec 03 2021, Lessley Dennington via GitGitGadget wrote:\n> \n>> From: Lessley Dennington \n>>\n>> Check whether git directory exists before adding any repo settings. If it\n>> does not exist, BUG with the message that one cannot add settings for an\n>> uninitialized repository. If it does exist, proceed with adding repo\n>> settings.\n>>\n>> Signed-off-by: Lessley Dennington \n>> ---\n>> repo-settings.c | 3 +++\n>> 1 file changed, 3 insertions(+)\n>>\n>> diff --git a/repo-settings.c b/repo-settings.c\n>> index b93e91a212e..00ca5571a1a 100644\n>> --- a/repo-settings.c\n>> +++ b/repo-settings.c\n>> @@ -17,6 +17,9 @@ void prepare_repo_settings(struct repository *r)\n>> \tchar *strval;\n>> \tint manyfiles;\n>> \n>> +\tif (!r->gitdir)\n>> +\t\tBUG(\"Cannot add settings for uninitialized repository\");\n> \n> nit: start BUG(), error() etc. messages with lower-case.\n> \nThanks for catching this. Instead of re-sending the whole series for a\none-letter change, I've included a patch with the fix at the end of this\nmessage.\n>> +\n>> \tif (r->settings.initialized++)\n>> \t\treturn;\n> \n> Our config doesn't require us to have a repo, and most of what\n> prepare_repo_settings() is doing is reading global config.\n> \n> I think that *currently* this won't break things, but e.g. if we ever\n> want to have \"feature.experimental\" or whatever change the behavior of a\n> a command that doesn't require a repository we'd need to untangle this\n> (currently everything it changes requires a repo AFAICT).\n> \n> Perhaps this is fine, and if we ever need such a \"global config\" point\n> we should stick it closer to common-main.c...\n> \nAgreed, let's keep the change for now and address updates in the future if\nthe need arises.\n-----------\n From 2764ca684320f488006093505deb074217ac7b31 Mon Sep 17 00:00:00 2001\nFrom: Lessley Dennington \nDate: Wed, 8 Dec 2021 09:29:56 -0600\nSubject: [PATCH] fixup! repo-settings: prepare_repo_settings only in git repos\n\nUpdate the BUG() message in prepare_repo_settings to begin with a\nlower-case letter.\n\nReported-by: Ævar Arnfjörð Bjarmason \nSigned-off-by: Lessley Dennington \n---\n repo-settings.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/repo-settings.c b/repo-settings.c\nindex 00ca5571a1..2644ac3353 100644\n--- a/repo-settings.c\n+++ b/repo-settings.c\n@@ -18,7 +18,7 @@ void prepare_repo_settings(struct repository *r)\n int manyfiles;\n\n if (!r->gitdir)\n- BUG(\"Cannot add settings for uninitialized repository\");\n+ BUG(\"cannot add settings for uninitialized repository\");\n\n if (r->settings.initialized++)\n return;\n--\n2.34.0.vfs.0.0\n"}]}