git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 1/2] index: add trace2 region for clear skip worktree

From
Anh Le via GitGitGadget <gitgitgadget@gmail.com>
Date
Nov 3, 2022, 23:05 UTC
Message-ID
<33e9b2afd91f4376ef9a64fd267fa61f1a8de07f.1667516701.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1368.v4.git.git.1667516701.gitgitgadget@gmail.com>
From: Anh Le <anh@canva.com>

When using sparse checkout, clear_skip_worktree_from_present_files() must enumerate index entries to find ones with the SKIP_WORKTREE bit to determine whether those index entries exist on disk (in which case their SKIP_WORKTREE bit should be removed).

In a large repository, this may take considerable time depending on the size of the index.

Add a trace2 region to surface this information, keeping a count of how many paths have been checked. Separately, keep counts after a full index is materialized.

Signed-off-by: Anh Le <anh@canva.com>
---
 sparse-index.c | 28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)
diff --git a/sparse-index.c b/sparse-index.c
index e4a54ce1943..8713a15611d 100644
--- a/sparse-index.c
+++ b/sparse-index.c
@@ -493,24 +493,40 @@ void clear_skip_worktree_from_present_files(struct index_state *istate)
 	int dir_found = 1;
 
 	int i;
+	int path_count[2] = {0, 0};
+	int restarted = 0;
 
 	if (!core_apply_sparse_checkout ||
 	    sparse_expect_files_outside_of_patterns)
 		return;
 
+	trace2_region_enter("index", "clear_skip_worktree_from_present_files",
+			    istate->repo);
 restart:
 	for (i = 0; i < istate->cache_nr; i++) {
 		struct cache_entry *ce = istate->cache[i];
 
-		if (ce_skip_worktree(ce) &&
-		    path_found(ce->name, &last_dirname, &dir_len, &dir_found)) {
-			if (S_ISSPARSEDIR(ce->ce_mode)) {
-				ensure_full_index(istate);
-				goto restart;
+		if (ce_skip_worktree(ce)) {
+			path_count[restarted]++;
+			if (path_found(ce->name, &last_dirname, &dir_len, &dir_found)) {
+				if (S_ISSPARSEDIR(ce->ce_mode)) {
+					ensure_full_index(istate);
+					restarted = 1;
+					goto restart;
+				}
+				ce->ce_flags &= ~CE_SKIP_WORKTREE;
 			}
-			ce->ce_flags &= ~CE_SKIP_WORKTREE;
 		}
 	}
+
+	if (path_count[0])
+		trace2_data_intmax("index", istate->repo,
+				   "sparse_path_count", path_count[0]);
+	if (restarted)
+		trace2_data_intmax("index", istate->repo,
+				   "sparse_path_count_full", path_count[1]);
+	trace2_region_leave("index", "clear_skip_worktree_from_present_files",
+			    istate->repo);
 }
 
 /*
-- 
gitgitgadget
Previous: Anh Le via GitGitGadgetNext: Taylor Blau
Message 16 of 18 in “index: add trace2 region for clear skip worktree”
  1. index: add trace2 region for clear skip worktreeAnh Le via GitGitGadget, Oct 26, 2022
  2. Junio C HamanoOct 26, 2022
  3. Jeff HostetlerOct 26, 2022
  4. Junio C HamanoOct 26, 2022
  5. Jeff HostetlerOct 26, 2022
  6. Anh LeOct 27, 2022
  7. index: add trace2 region for clear skip worktreeAnh Le via GitGitGadget, Oct 28, 2022
  8. Derrick StoleeOct 28, 2022
  9. Junio C HamanoOct 28, 2022
  10. Anh LeOct 30, 2022
  11. Jeff HostetlerOct 28, 2022
  12. index: add trace2 region for clear skip worktreeAnh Le via GitGitGadget, Oct 31, 2022
  13. Taylor BlauOct 31, 2022
  14. 0/2 index: add trace2 region for clear skip worktreeAnh Le via GitGitGadget, Nov 3, 2022
  15. 2/2 index: raise a bug if the index is materialised more than onceAnh Le via GitGitGadget, Nov 3, 2022
  16. 1/2 index: add trace2 region for clear skip worktreeAnh Le via GitGitGadget, Nov 3, 2022
  17. Taylor BlauNov 5, 2022
  18. Derrick StoleeNov 7, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.