git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 4/4] tree_entry_interesting: do basedir compare on wildcard patterns when possible

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Nov 24, 2012, 04:33 UTC
Message-ID
<1353731631-20593-5-git-send-email-pclouds@gmail.com>
In-Reply-To
<1353731631-20593-1-git-send-email-pclouds@gmail.com>

Currently we treat "*.c" and "path/to/*.c" the same way. Which means we check all possible paths in repo against "path/to/*.c". One could see that "path/elsewhere/foo.c" obviously cannot match "path/to/*.c" and we only need to check all paths _inside_ "path/to/" against that pattern.

This patch checks the leading fixed part of a pathspec against base directory and exit early if possible. We could even optimize further in "path/to/something*.c" case (i.e. check the fixed part against name_entry as well) but that's more complicated and probably does not gain us much.

-O2 build on linux-2.6, without and with this patch respectively:
$ time git rev-list --quiet HEAD -- 'drivers/*.c'

real 1m9.484s user 1m9.128s sys 0m0.181s

$ time ~/w/git/git rev-list --quiet HEAD -- 'drivers/*.c'

real 0m15.710s user 0m15.564s sys 0m0.107s

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 tree-walk.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 62 insertions(+), 1 deletion(-)
diff --git a/tree-walk.c b/tree-walk.c
index 585899e..9d6e59b 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -573,6 +573,52 @@ static int match_dir_prefix(const char *base,
 }
 
 /*
+ * Perform matching on the leading non-wildcard part of
+ * pathspec. item->nowildcard_len must be greater than zero. Return
+ * non-zero if base is matched.
+ */
+static int match_wildcard_base(const struct pathspec_item *item,
+			       const char *base, int baselen,
+			       int *matched)
+{
+	const char *match = item->match;
+	/* the wildcard part is not considered in this function */
+	int matchlen = item->nowildcard_len;
+
+	if (baselen) {
+		int dirlen;
+		/*
+		 * Return early if base is longer than the
+		 * non-wildcard part but it does not match.
+		 */
+		if (baselen >= matchlen) {
+			*matched = matchlen;
+			return !strncmp(base, match, matchlen);
+		}
+
+		dirlen = matchlen;
+		while (dirlen && match[dirlen - 1] != '/')
+			dirlen--;
+
+		/* Return early if base is shorter than the
+		   non-wildcard part but it does not match. Note that
+		   base ends with '/' so we are sure it really matches
+		   directory */
+		if (strncmp(base, match, baselen))
+			return 0;
+		*matched = baselen;
+	} else
+		*matched = 0;
+	/*
+	 * we could have checked entry against the non-wildcard part
+	 * that is not in base and does similar never_interesting
+	 * optimization as in match_entry. For now just be happy with
+	 * base comparison.
+	 */
+	return entry_interesting;
+}
+
+/*
  * Is a tree entry interesting given the pathspec we have?
  *
  * Pre-condition: either baselen == base_offset (i.e. empty path)
@@ -602,7 +648,7 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 		const struct pathspec_item *item = ps->items+i;
 		const char *match = item->match;
 		const char *base_str = base->buf + base_offset;
-		int matchlen = item->len;
+		int matchlen = item->len, matched = 0;
 
 		if (baselen >= matchlen) {
 			/* If it doesn't match, move along... */
@@ -647,9 +693,24 @@ match_wildcards:
 		if (item->nowildcard_len == item->len)
 			continue;
 
+		if (item->nowildcard_len &&
+		    !match_wildcard_base(item, base_str, baselen, &matched))
+			return entry_not_interesting;
+
 		/*
 		 * Concatenate base and entry->path into one and do
 		 * fnmatch() on it.
+		 *
+		 * While we could avoid concatenation in certain cases
+		 * [1], which saves a memcpy and potentially a
+		 * realloc, it turns out not worth it. Measurement on
+		 * linux-2.6 does not show any clear improvements,
+		 * partly because of the nowildcard_len optimization
+		 * in git_fnmatch(). Avoid micro-optimizations here.
+		 *
+		 * [1] if match_wildcard_base() says the base
+		 * directory is already matched, we only need to match
+		 * the rest, which is shorter so _in theory_ faster.
 		 */
 
 		strbuf_add(base, entry->path, pathlen);
-- 
1.8.0.rc2.23.g1fb49df
Previous: Nguyễn Thái Ngọc Duy
Message 5 of 5 in “nd/pathspec-wildcard”
  1. 0/4 nd/pathspec-wildcardNguyễn Thái Ngọc Duy, Nov 24, 2012
  2. 1/4 pathspec: save the non-wildcard length partNguyễn Thái Ngọc Duy, Nov 24, 2012
  3. 2/4 pathspec: do exact comparison on the leading non-wildcard partNguyễn Thái Ngọc Duy, Nov 24, 2012
  4. 3/4 pathspec: apply "*.c" optimization from excludeNguyễn Thái Ngọc Duy, Nov 24, 2012
  5. 4/4 tree_entry_interesting: do basedir compare on wildcard patterns when possibleNguyễn Thái Ngọc Duy, Nov 24, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.