git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4] write-tree: integrate with sparse index

From
Shuqi Liang <cheskaqiqi@gmail.com>
Date
Apr 21, 2023, 00:41 UTC
Message-ID
<20230421004108.32554-1-cheskaqiqi@gmail.com>
In-Reply-To
<20230419072148.4297-1-cheskaqiqi@gmail.com>

Update 'git write-tree' to allow using the sparse-index in memory without expanding to a full one.

The recursive algorithm for update_one() was already updated in 2de37c5 (cache-tree: integrate with sparse directory entries, 2021-03-03) to handle sparse directory entries in the index. Hence we can just set the requires-full-index to false for "write-tree".

The `p2000` tests demonstrate a ~96% execution time reduction for 'git write-tree' using a sparse index:

Test before after ----------------------------------------------------------------- 2000.78: git write-tree (full-v3) 0.34 0.33 -2.9% 2000.79: git write-tree (full-v4) 0.32 0.30 -6.3% 2000.80: git write-tree (sparse-v3) 0.47 0.02 -95.8% 2000.81: git write-tree (sparse-v4) 0.45 0.02 -95.6%

Signed-off-by: Shuqi Liang <cheskaqiqi@gmail.com>
---
* Modified the code to ensure prepare_repo_settings() is called only 
when inside a repository.
* Change 'write-tree on all' to just 'write-tree'.
* Have a baseline 'test_all_match git write-tree' before making any 
changes to the index.
* Add 'git status --porcelain=v2'.
* Ensuring that SKIP_WORKTREE files weren't materialized on disk by
using "test_path_is_missing".
* Use 'test_all_match' on the 'git update-index'.
 builtin/write-tree.c                     |  9 ++++++---
 t/t1092-sparse-checkout-compatibility.sh | 20 +++++++++++++++-----
 2 files changed, 21 insertions(+), 8 deletions(-)
diff --git a/builtin/write-tree.c b/builtin/write-tree.c
index 32e302a813..a9d5c20cde 100644
--- a/builtin/write-tree.c
+++ b/builtin/write-tree.c
@@ -38,12 +38,15 @@ int cmd_write_tree(int argc, const char **argv, const char *cmd_prefix)
 	};
 
 	git_config(git_default_config, NULL);
+	
+	if (the_repository->gitdir) {
+		prepare_repo_settings(the_repository);
+		the_repository->settings.command_requires_full_index = 0;
+	}
+
 	argc = parse_options(argc, argv, cmd_prefix, write_tree_options,
 			     write_tree_usage, 0);
 
-	prepare_repo_settings(the_repository);
-	the_repository->settings.command_requires_full_index = 0;
-
 	ret = write_index_as_tree(&oid, &the_index, get_index_file(), flags,
 				  tree_prefix);
 	switch (ret) {
diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
index 9bbc0d646b..d3eb31326b 100755
--- a/t/t1092-sparse-checkout-compatibility.sh
+++ b/t/t1092-sparse-checkout-compatibility.sh
@@ -2055,22 +2055,32 @@ test_expect_success 'grep sparse directory within submodules' '
 	test_cmp actual expect
 '
 
-test_expect_success 'write-tree on all' '
+test_expect_success 'write-tree' '
 	init_repos &&
 
+	test_all_match git write-tree &&
+
 	write_script edit-contents <<-\EOF &&
 	echo text >>"$1"
 	EOF
 
+	# make a change inside the sparse cone
 	run_on_all ../edit-contents deep/a &&
-	run_on_all git update-index deep/a &&
+	test_all_match git update-index deep/a &&
 	test_all_match git write-tree &&
+	test_all_match git status --porcelain=v2 &&
 
+	# make a change outside the sparse cone
 	run_on_all mkdir -p folder1 &&
 	run_on_all cp a folder1/a &&
 	run_on_all ../edit-contents folder1/a &&
-	run_on_all git update-index folder1/a &&
-	test_all_match git write-tree
+	test_all_match git update-index folder1/a &&
+	test_all_match git write-tree &&
+	test_all_match git status --porcelain=v2 &&
+	
+	# check that SKIP_WORKTREE files are not materialized
+	test_path_is_missing sparse-checkout/folder2/a &&
+	test_path_is_missing sparse-index/folder2/a
 '
 
 test_expect_success 'sparse-index is not expanded: write-tree' '
@@ -2080,7 +2090,7 @@ test_expect_success 'sparse-index is not expanded: write-tree' '
 
 	echo "test1" >>sparse-index/a &&
 	git -C sparse-index update-index a &&
-	ensure_not_expanded write-tree
+	ensure_not_expanded write-tree 
 '
 
 test_done
-- 
2.39.0
Previous: Junio C HamanoNext: Victoria Dye
Message 12 of 20 in “write-tree: integrate with sparse index”
  1. Shuqi LiangApr 2, 2023
  2. Junio C HamanoApr 3, 2023
  3. Shuqi LiangApr 3, 2023
  4. Junio C HamanoApr 3, 2023
  5. write-tree: integrate with sparse indexShuqi Liang, Apr 4, 2023
  6. Victoria DyeApr 5, 2023
  7. Junio C HamanoApr 5, 2023
  8. write-tree: integrate with sparse indexShuqi Liang, Apr 19, 2023
  9. Junio C HamanoApr 19, 2023
  10. Shuqi LiangApr 20, 2023
  11. Junio C HamanoApr 20, 2023
  12. write-tree: integrate with sparse indexShuqi Liang, Apr 21, 2023
  13. Victoria DyeApr 21, 2023
  14. Junio C HamanoApr 24, 2023
  15. write-tree: optimize sparse integrationShuqi Liang, Apr 23, 2023
  16. Junio C HamanoApr 24, 2023
  17. write-tree: optimize sparse integrationShuqi Liang, May 8, 2023
  18. write-tree: optimize sparse integrationShuqi Liang, May 8, 2023
  19. Junio C HamanoMay 8, 2023
  20. Shuqi LiangMay 8, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.