git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 24/25] ref-filter: limit traversal to prefix

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
May 22, 2017, 14:17 UTC
Message-ID
<e6eab70183ce8016efd9763d6dfe0892b91dc53b.1495460199.git.mhagger@alum.mit.edu>
In-Reply-To
<cover.1495460199.git.mhagger@alum.mit.edu>
From: Jeff King <peff@peff.net>

When we are matching refnames against a pattern, then we know that the beginning of any refname that can match the pattern has to match the part of the pattern up to the first glob character. For example, if the pattern is `refs/heads/foo*bar`, then it can only match a reference that has the prefix `refs/heads/foo`.

So pass that prefix to `for_each_fullref_in()`. This lets the ref code avoid passing us the full set of refs, and in some cases avoid reading them in the first place.

Note that this applies only when the `match_as_path` flag is set (i.e., when `for-each-ref` is the caller), as the matching rules for git-branch and git-tag are subtly different.

This could be generalized to the case of multiple patterns, but (a) it probably doesn't come up that often, and (b) it is more awkward to deal with multiple patterns (e.g., the patterns might not be disjoint). So, since this is just an optimization, punt on the case of multiple patterns.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 ref-filter.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 63 insertions(+), 1 deletion(-)
diff --git a/ref-filter.c b/ref-filter.c
index 6cc93dcd9f..25ca56d62f 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -1665,6 +1665,68 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname)
 	return match_pattern(filter, refname);
 }
 
+/*
+ * Find the longest prefix of pattern we can pass to
+ * `for_each_fullref_in()`, namely the part of pattern preceding the
+ * first glob character. (Note that `for_each_fullref_in()` is
+ * perfectly happy working with a prefix that doesn't end at a
+ * pathname component boundary.)
+ */
+static void find_longest_prefix(struct strbuf *out, const char *pattern)
+{
+	const char *p;
+
+	for (p = pattern; *p && !is_glob_special(*p); p++)
+		;
+
+	strbuf_add(out, pattern, p - pattern);
+}
+
+/*
+ * This is the same as for_each_fullref_in(), but it tries to iterate
+ * only over the patterns we'll care about. Note that it _doesn't_ do a full
+ * pattern match, so the callback still has to match each ref individually.
+ */
+static int for_each_fullref_in_pattern(struct ref_filter *filter,
+				       each_ref_fn cb,
+				       void *cb_data,
+				       int broken)
+{
+	struct strbuf prefix = STRBUF_INIT;
+	int ret;
+
+	if (!filter->match_as_path) {
+		/*
+		 * in this case, the patterns are applied after
+		 * prefixes like "refs/heads/" etc. are stripped off,
+		 * so we have to look at everything:
+		 */
+		return for_each_fullref_in("", cb, cb_data, broken);
+	}
+
+	if (!filter->name_patterns[0]) {
+		/* no patterns; we have to look at everything */
+		return for_each_fullref_in("", cb, cb_data, broken);
+	}
+
+	if (filter->name_patterns[1]) {
+		/*
+		 * multiple patterns; in theory this could still work as long
+		 * as the patterns are disjoint. We'd just make multiple calls
+		 * to for_each_ref(). But if they're not disjoint, we'd end up
+		 * reporting the same ref multiple times. So let's punt on that
+		 * for now.
+		 */
+		return for_each_fullref_in("", cb, cb_data, broken);
+	}
+
+	find_longest_prefix(&prefix, filter->name_patterns[0]);
+
+	ret = for_each_fullref_in(prefix.buf, cb, cb_data, broken);
+	strbuf_release(&prefix);
+	return ret;
+}
+
 /*
  * Given a ref (sha1, refname), check if the ref belongs to the array
  * of sha1s. If the given ref is a tag, check if the given tag points
@@ -1911,7 +1973,7 @@ int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int
 		else if (filter->kind == FILTER_REFS_TAGS)
 			ret = for_each_fullref_in("refs/tags/", ref_filter_handler, &ref_cbdata, broken);
 		else if (filter->kind & FILTER_REFS_ALL)
-			ret = for_each_fullref_in("", ref_filter_handler, &ref_cbdata, broken);
+			ret = for_each_fullref_in_pattern(filter, ref_filter_handler, &ref_cbdata, broken);
 		if (!ret && (filter->kind & FILTER_REFS_DETACHED_HEAD))
 			head_ref(ref_filter_handler, &ref_cbdata);
 	}
-- 
2.11.0
Previous: Michael HaggertyNext: Michael Haggerty
Message 22 of 31 in “Prepare to separate out a packed_ref_store”
  1. 00/25 Prepare to separate out a packed_ref_storeMichael Haggerty, May 22, 2017
  2. 01/25 t3600: clean up permissions test properlyMichael Haggerty, May 22, 2017
  3. 02/25 refs.h: clarify docstring for the ref_transaction_update()-related fnsMichael Haggerty, May 22, 2017
  4. 03/25 ref_iterator_begin_fn(): fix docstringMichael Haggerty, May 22, 2017
  5. 04/25 files-backend: use `die("BUG: ...")`, not `die("internal error: ...")`Michael Haggerty, May 22, 2017
  6. 05/25 prefix_ref_iterator: don't trim too muchMichael Haggerty, May 22, 2017
  7. 06/25 refs_ref_iterator_begin(): don't check prefixes redundantlyMichael Haggerty, May 22, 2017
  8. 07/25 refs: use `size_t` indexes when iterating over ref transaction updatesMichael Haggerty, May 22, 2017
  9. 08/25 ref_store: take a `msg` parameter when deleting referencesMichael Haggerty, May 22, 2017
  10. 10/25 files-backend: move `lock` member to `files_ref_store`Michael Haggerty, May 22, 2017
  11. 12/25 files_transaction_cleanup(): new helper functionMichael Haggerty, May 22, 2017
  12. 11/25 files_ref_store: put the packed files lock directly in this structMichael Haggerty, May 22, 2017
  13. 09/25 lockfile: add a new method, is_lock_file_locked()Michael Haggerty, May 22, 2017
  14. 14/25 ref_transaction_prepare(): new optional step for reference updatesMichael Haggerty, May 22, 2017
  15. 16/25 ref_update_reject_duplicates(): use `size_t` rather than `int`Michael Haggerty, May 22, 2017
  16. 15/25 ref_update_reject_duplicates(): expose function to whole refs moduleMichael Haggerty, May 22, 2017
  17. 17/25 ref_update_reject_duplicates(): add a sanity checkMichael Haggerty, May 22, 2017
  18. 19/25 get_packed_ref_cache(): assume "packed-refs" won't change while lockedMichael Haggerty, May 22, 2017
  19. 20/25 read_packed_refs(): do more of the work of reading packed refsMichael Haggerty, May 22, 2017
  20. 22/25 refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`Michael Haggerty, May 22, 2017
  21. 23/25 create_ref_entry(): remove `check_name` optionMichael Haggerty, May 22, 2017
  22. 24/25 ref-filter: limit traversal to prefixMichael Haggerty, May 22, 2017
  23. 25/25 cache_ref_iterator_begin(): avoid priming unneeded directoriesMichael Haggerty, May 22, 2017
  24. Jeff KingMay 23, 2017
  25. Michael HaggertyMay 24, 2017
  26. Junio C HamanoMay 24, 2017
  27. 21/25 read_packed_refs(): report unexpected fopen() failuresMichael Haggerty, May 22, 2017
  28. 13/25 ref_transaction_commit(): check for valid `transaction->state`Michael Haggerty, May 22, 2017
  29. 18/25 should_pack_ref(): new function, extracted from `files_pack_refs()`Michael Haggerty, May 22, 2017
  30. Stefan BellerMay 22, 2017
  31. Junio C HamanoMay 24, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.