[PATCH v3] write-tree: integrate with sparse index
- From
Shuqi Liang <cheskaqiqi@gmail.com>
- Date
- Apr 19, 2023, 07:21 UTC
- Message-ID
- <20230419072148.4297-1-cheskaqiqi@gmail.com>
- In-Reply-To
- <20230404003539.1578245-1-cheskaqiqi@gmail.com>
Update 'git write-tree' to allow using the sparse-index in memory without expanding to a full one.
The recursive algorithm for update_one() was already updated in 2de37c5 (cache-tree: integrate with sparse directory entries, 2021-03-03) to handle sparse directory entries in the index. Hence we can just set the requires-full-index to false for "write-tree".
The `p2000` tests demonstrate a ~96% execution time reduction for 'git write-tree' using a sparse index:
Test before after ----------------------------------------------------------------- 2000.78: git write-tree (full-v3) 0.34 0.33 -2.9% 2000.79: git write-tree (full-v4) 0.32 0.30 -6.3% 2000.80: git write-tree (sparse-v3) 0.47 0.02 -95.8% 2000.81: git write-tree (sparse-v4) 0.45 0.02 -95.6%
Signed-off-by: Shuqi Liang <cheskaqiqi@gmail.com> ---
* Modified the code to ensure prepare_repo_settings() is called only when inside a repository.
* Change 'write-tree on all' to just 'write-tree'.
* Have a baseline 'test_all_match git write-tree' before making any changes to the index.
* Add 'git status --porcelain=v2'.
* Ensuring that SKIP_WORKTREE files weren't materialized on disk by using "test_path_is_missing".
* Use 'test_all_match' on the 'git update-index'.
Range-diff against v2:
1: 8873c79759 ! 1: cfa43c6cc7 write-tree: integrate with sparse index
@@ Commit message
## builtin/write-tree.c ##
@@ builtin/write-tree.c: int cmd_write_tree(int argc, const char **argv, const char *cmd_prefix)
- argc = parse_options(argc, argv, cmd_prefix, write_tree_options,
- write_tree_usage, 0);
+ };
+ git_config(git_default_config, NULL);
++
++ if (the_repository->gitdir) {
+ prepare_repo_settings(the_repository);
+ the_repository->settings.command_requires_full_index = 0;
-+
- ret = write_cache_as_tree(&oid, flags, tree_prefix);
- switch (ret) {
- case 0:
++ }
++
+ argc = parse_options(argc, argv, cmd_prefix, write_tree_options,
+ write_tree_usage, 0);
+
## t/perf/p2000-sparse-operations.sh ##
@@ t/perf/p2000-sparse-operations.sh: test_perf_on_all git checkout-index -f --all
@@ t/t1092-sparse-checkout-compatibility.sh: test_expect_success 'grep sparse direc
test_cmp actual expect
'
-+test_expect_success 'write-tree on all' '
++test_expect_success 'write-tree' '
+ init_repos &&
+
++ test_all_match git write-tree &&
++
+ write_script edit-contents <<-\EOF &&
+ echo text >>"$1"
+ EOF
+
++ # make a change inside the sparse cone
+ run_on_all ../edit-contents deep/a &&
-+ run_on_all git update-index deep/a &&
++ test_all_match git update-index deep/a &&
+ test_all_match git write-tree &&
++ test_all_match git status --porcelain=v2 &&
+
++ # make a change outside the sparse cone
+ run_on_all mkdir -p folder1 &&
+ run_on_all cp a folder1/a &&
+ run_on_all ../edit-contents folder1/a &&
-+ run_on_all git update-index folder1/a &&
-+ test_all_match git write-tree
++ test_all_match git update-index folder1/a &&
++ test_all_match git write-tree &&
++ test_all_match git status --porcelain=v2 &&
++
++ # check that SKIP_WORKTREE files are not materialized
++ test_path_is_missing sparse-checkout/folder2/a &&
++ test_path_is_missing sparse-index/folder2/a
+'
+
+test_expect_success 'sparse-index is not expanded: write-tree' 'builtin/write-tree.c | 6 ++++ t/perf/p2000-sparse-operations.sh | 1 + t/t1092-sparse-checkout-compatibility.sh | 38 ++++++++++++++++++++++++ 3 files changed, 45 insertions(+)
diff --git a/builtin/write-tree.c b/builtin/write-tree.c index 45d61707e7..23d63844de 100644 --- a/builtin/write-tree.c +++ b/builtin/write-tree.c @@ -35,6 +35,12 @@ int cmd_write_tree(int argc, const char **argv, const char *cmd_prefix) }; git_config(git_default_config, NULL); + + if (the_repository->gitdir) { + prepare_repo_settings(the_repository); + the_repository->settings.command_requires_full_index = 0; + } + argc = parse_options(argc, argv, cmd_prefix, write_tree_options, write_tree_usage, 0); diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 3242cfe91a..9924adfc26 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -125,5 +125,6 @@ test_perf_on_all git checkout-index -f --all test_perf_on_all git update-index --add --remove $SPARSE_CONE/a test_perf_on_all "git rm -f $SPARSE_CONE/a && git checkout HEAD -- $SPARSE_CONE/a" test_perf_on_all git grep --cached --sparse bogus -- "f2/f1/f1/*" +test_perf_on_all git write-tree test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 801919009e..d3eb31326b 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -2055,4 +2055,42 @@ test_expect_success 'grep sparse directory within submodules' ' test_cmp actual expect ' +test_expect_success 'write-tree' ' + init_repos && + + test_all_match git write-tree && + + write_script edit-contents <<-\EOF && + echo text >>"$1" + EOF + + # make a change inside the sparse cone + run_on_all ../edit-contents deep/a && + test_all_match git update-index deep/a && + test_all_match git write-tree && + test_all_match git status --porcelain=v2 && + + # make a change outside the sparse cone + run_on_all mkdir -p folder1 && + run_on_all cp a folder1/a && + run_on_all ../edit-contents folder1/a && + test_all_match git update-index folder1/a && + test_all_match git write-tree && + test_all_match git status --porcelain=v2 && + + # check that SKIP_WORKTREE files are not materialized + test_path_is_missing sparse-checkout/folder2/a && + test_path_is_missing sparse-index/folder2/a +' + +test_expect_success 'sparse-index is not expanded: write-tree' ' + init_repos && + + ensure_not_expanded write-tree && + + echo "test1" >>sparse-index/a && + git -C sparse-index update-index a && + ensure_not_expanded write-tree +' + test_done
-- 2.39.0