git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v12 4/5] read-cache: speed up has_dir_name (part 1)

From
Ggit@jeffhostetler.com <git@jeffhostetler.com>
Date
Apr 19, 2017, 17:06 UTC
Message-ID
<20170419170618.16535-5-git@jeffhostetler.com>
In-Reply-To
<20170419170618.16535-1-git@jeffhostetler.com>
From: Jeff Hostetler <jeffhost@microsoft.com>

Teach has_dir_name() to see if the path of the new item is greater than the last path in the index array before attempting to search for it.

has_dir_name() is looking for file/directory collisions in the index and has to consider each sub-directory prefix in turn. This can cause multiple binary searches for each path.

During operations like checkout, merge_working_tree() populates the new index in sorted order, so we expect to be able to append in many cases.

This commit is part 1 of 2. This commit handles the top of has_dir_name() and the trivial optimization.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 read-cache.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 6a27688..9af0bd4 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -910,6 +910,9 @@ int strcmp_offset(const char *s1, const char *s2, size_t *first_change)
 /*
  * Do we have another file with a pathname that is a proper
  * subset of the name we're trying to add?
+ *
+ * That is, is there another file in the index with a path
+ * that matches a sub-directory in the given entry?
  */
 static int has_dir_name(struct index_state *istate,
 			const struct cache_entry *ce, int pos, int ok_to_replace)
@@ -918,6 +921,48 @@ static int has_dir_name(struct index_state *istate,
 	int stage = ce_stage(ce);
 	const char *name = ce->name;
 	const char *slash = name + ce_namelen(ce);
+	size_t len_eq_last;
+	int cmp_last = 0;
+
+	/*
+	 * We are frequently called during an iteration on a sorted
+	 * list of pathnames and while building a new index.  Therefore,
+	 * there is a high probability that this entry will eventually
+	 * be appended to the index, rather than inserted in the middle.
+	 * If we can confirm that, we can avoid binary searches on the
+	 * components of the pathname.
+	 *
+	 * Compare the entry's full path with the last path in the index.
+	 */
+	if (istate->cache_nr > 0) {
+		cmp_last = strcmp_offset(name,
+			istate->cache[istate->cache_nr - 1]->name,
+			&len_eq_last);
+		if (cmp_last > 0) {
+			if (len_eq_last == 0) {
+				/*
+				 * The entry sorts AFTER the last one in the
+				 * index and their paths have no common prefix,
+				 * so there cannot be a F/D conflict.
+				 */
+				return retval;
+			} else {
+				/*
+				 * The entry sorts AFTER the last one in the
+				 * index, but has a common prefix.  Fall through
+				 * to the loop below to disect the entry's path
+				 * and see where the difference is.
+				 */
+			}
+		} else if (cmp_last == 0) {
+			/*
+			 * The entry exactly matches the last one in the
+			 * index, but because of multiple stage and CE_REMOVE
+			 * items, we fall through and let the regular search
+			 * code handle it.
+			 */
+		}
+	}
 
 	for (;;) {
 		int len;
-- 
2.9.3
Previous: René Scharfe
Message 11 of 11 in “read-cache: speed up add_index_entry”
  1. 0/5 read-cache: speed up add_index_entrygit@jeffhostetler.com, Apr 19, 2017
  2. 2/5 p0006-read-tree-checkout: perf test to time read-treegit@jeffhostetler.com, Apr 19, 2017
  3. 1/5 read-cache: add strcmp_offset functiongit@jeffhostetler.com, Apr 19, 2017
  4. 3/5 read-cache: speed up add_index_entry during checkoutgit@jeffhostetler.com, Apr 19, 2017
  5. 5/5 read-cache: speed up has_dir_name (part 2)git@jeffhostetler.com, Apr 19, 2017
  6. SZEDER GáborJul 4, 2020
  7. René ScharfeJul 5, 2020
  8. Junio C HamanoJul 6, 2020
  9. Jeff HostetlerJul 8, 2020
  10. René ScharfeJul 16, 2020
  11. 4/5 read-cache: speed up has_dir_name (part 1)git@jeffhostetler.com, Apr 19, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.