git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 07/24] untracked cache: mark what dirs should be recursed/saved

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Feb 8, 2015, 08:55 UTC
Message-ID
<1423385748-19825-8-git-send-email-pclouds@gmail.com>
In-Reply-To
<1423385748-19825-1-git-send-email-pclouds@gmail.com>

If we redo this thing in a functional style, we would have one struct untracked_dir as input tree and another as output. The input is used for verification. The output is a brand new tree, reflecting current worktree.

But that means recreate a lot of dir nodes even if a lot could be shared between input and output trees in good cases. So we go with the messy but efficient way, combining both input and output trees into one. We need a way to know which node in this combined tree belongs to the output. This is the purpose of this "recurse" flag.

"valid" bit can't be used for this because it's about data of the node except the subdirs. When we invalidate a directory, we want to keep cached data of the subdirs intact even though we don't really know what subdir still exists (yet). Then we check worktree to see what actual subdir remains on disk. Those will have 'recurse' bit set again. If cached data for those are still valid, we may be able to avoid computing exclude files for them. Those subdirs that are deleted will have 'recurse' remained clear and their 'valid' bits do not matter.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 dir.c | 14 +++++++++++++-
 dir.h |  3 ++-
 2 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/dir.c b/dir.c
index 8c989e3..02cdc26 100644
--- a/dir.c
+++ b/dir.c
@@ -615,9 +615,12 @@ static void invalidate_gitignore(struct untracked_cache *uc,
 static void invalidate_directory(struct untracked_cache *uc,
 				 struct untracked_cache_dir *dir)
 {
+	int i;
 	uc->dir_invalidated++;
 	dir->valid = 0;
 	dir->untracked_nr = 0;
+	for (i = 0; i < dir->dirs_nr; i++)
+		dir->dirs[i]->recurse = 0;
 }
 
 /*
@@ -1577,6 +1580,10 @@ static int read_cached_dir(struct cached_dir *cdir)
 	}
 	while (cdir->nr_dirs < cdir->untracked->dirs_nr) {
 		struct untracked_cache_dir *d = cdir->untracked->dirs[cdir->nr_dirs];
+		if (!d->recurse) {
+			cdir->nr_dirs++;
+			continue;
+		}
 		cdir->ucd = d;
 		cdir->nr_dirs++;
 		return 0;
@@ -1598,8 +1605,10 @@ static void close_cached_dir(struct cached_dir *cdir)
 	 * We have gone through this directory and found no untracked
 	 * entries. Mark it valid.
 	 */
-	if (cdir->untracked)
+	if (cdir->untracked) {
 		cdir->untracked->valid = 1;
+		cdir->untracked->recurse = 1;
+	}
 }
 
 /*
@@ -1842,6 +1851,9 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d
 		invalidate_gitignore(dir->untracked, root);
 		dir->untracked->ss_excludes_file = dir->ss_excludes_file;
 	}
+
+	/* Make sure this directory is not dropped out at saving phase */
+	root->recurse = 1;
 	return root;
 }
 
diff --git a/dir.h b/dir.h
index ff3d99b..95baf01 100644
--- a/dir.h
+++ b/dir.h
@@ -115,8 +115,9 @@ struct untracked_cache_dir {
 	unsigned int untracked_alloc, dirs_nr, dirs_alloc;
 	unsigned int untracked_nr;
 	unsigned int check_only : 1;
-	/* all data in this struct are good */
+	/* all data except 'dirs' in this struct are good */
 	unsigned int valid : 1;
+	unsigned int recurse : 1;
 	/* null SHA-1 means this directory does not have .gitignore */
 	unsigned char exclude_sha1[20];
 	char name[FLEX_ARRAY];
-- 
2.3.0.rc1.137.g477eb31
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 11 of 30 in “nd/untracked-cache updates”
  1. 00/24 nd/untracked-cache updatesNguyễn Thái Ngọc Duy, Feb 8, 2015
  2. 01/24 dir.c: optionally compute sha-1 of a .gitignore fileNguyễn Thái Ngọc Duy, Feb 8, 2015
  3. Junio C HamanoFeb 11, 2015
  4. Duy NguyenFeb 16, 2015
  5. Junio C HamanoFeb 16, 2015
  6. 02/24 untracked cache: record .gitignore information and dir hierarchyNguyễn Thái Ngọc Duy, Feb 8, 2015
  7. 03/24 untracked cache: initial untracked cache validationNguyễn Thái Ngọc Duy, Feb 8, 2015
  8. 04/24 untracked cache: invalidate dirs recursively if .gitignore changesNguyễn Thái Ngọc Duy, Feb 8, 2015
  9. 05/24 untracked cache: make a wrapper around {open,read,close}dir()Nguyễn Thái Ngọc Duy, Feb 8, 2015
  10. 06/24 untracked cache: record/validate dir mtime and reuse cached outputNguyễn Thái Ngọc Duy, Feb 8, 2015
  11. 07/24 untracked cache: mark what dirs should be recursed/savedNguyễn Thái Ngọc Duy, Feb 8, 2015
  12. 08/24 untracked cache: don't open non-existent .gitignoreNguyễn Thái Ngọc Duy, Feb 8, 2015
  13. 09/24 ewah: add convenient wrapper ewah_serialize_strbuf()Nguyễn Thái Ngọc Duy, Feb 8, 2015
  14. 10/24 untracked cache: save to an index extensionNguyễn Thái Ngọc Duy, Feb 8, 2015
  15. Stefan BellerMar 7, 2015
  16. 11/24 untracked cache: load from UNTR index extensionNguyễn Thái Ngọc Duy, Feb 8, 2015
  17. Junio C HamanoFeb 9, 2015
  18. 12/24 untracked cache: invalidate at index addition or removalNguyễn Thái Ngọc Duy, Feb 8, 2015
  19. 13/24 read-cache.c: split racy stat test to a separate functionNguyễn Thái Ngọc Duy, Feb 8, 2015
  20. 14/24 untracked cache: avoid racy timestampsNguyễn Thái Ngọc Duy, Feb 8, 2015
  21. 15/24 untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATSNguyễn Thái Ngọc Duy, Feb 8, 2015
  22. 16/24 untracked cache: mark index dirty if untracked cache is updatedNguyễn Thái Ngọc Duy, Feb 8, 2015
  23. 17/24 untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHENguyễn Thái Ngọc Duy, Feb 8, 2015
  24. 18/24 status: enable untracked cacheNguyễn Thái Ngọc Duy, Feb 8, 2015
  25. 19/24 update-index: manually enable or disable untracked cacheNguyễn Thái Ngọc Duy, Feb 8, 2015
  26. 20/24 update-index: test the system before enabling untracked cacheNguyễn Thái Ngọc Duy, Feb 8, 2015
  27. 21/24 t7063: tests for untracked cacheNguyễn Thái Ngọc Duy, Feb 8, 2015
  28. 22/24 mingw32: add uname()Nguyễn Thái Ngọc Duy, Feb 8, 2015
  29. 23/24 untracked cache: guard and disable on system changesNguyễn Thái Ngọc Duy, Feb 8, 2015
  30. 24/24 git-status.txt: advertisement for untracked cacheNguyễn Thái Ngọc Duy, Feb 8, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.