git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v12 5/5] read-cache: speed up has_dir_name (part 2)

From
René Scharfe <l.s.r@web.de>
Date
Jul 16, 2020, 17:11 UTC
Message-ID
<6d769a78-72aa-5b5d-9cc7-41b8c0ddf9d8@web.de>
In-Reply-To
<be8dec3e-087e-b973-280d-3d0d2e481de6@jeffhostetler.com>
Am 08.07.20 um 15:49 schrieb Jeff Hostetler:
Show 19 quoted lines
>
>
> On 7/6/20 2:39 AM, Junio C Hamano wrote:
>> SZEDER Gábor <szeder.dev@gmail.com> writes:
>>
>>>> +                 * last: xxx/yy-file (because '-' sorts before '/')
>>>> +                 * this: xxx/yy/abc
>>>
>>> This is problematic, because the index can already contain 'xxx/yy' as
>>> a file, when adding 'xxx/yy/abc', but since 'xxx/yy' as a file sorts
>>> before 'xxx/yy-file', the short-circuiting here doesn't see it and
>>> thus leaves the d-f collision undetected.  Consequently, even Git
>>> porcelain commands can create tree objects with duplicate entries, as
>>> demonstrated in the tests below.
>>
>> Yeah, the "optimization" is quite bogus.  Thanks for catching it.
>>
>
> yes, thanks!
OK, so now we just need to fix it.  Like this perhaps?
-- >8 --
From f4b2c70a34bb612e2ccc23a31e2ba8e01dedc373 Mon Sep 17 00:00:00 2001
Subject: [PATCH] read-cache: remove bogus shortcut

has_dir_name() has some optimizations for the case where entries are added to an index in the correct order. They kick in if the new entry sorts after the last one. One of them exits early if the last entry has a longer name than the directory of the new entry. Here's its comment:

/*
 * The directory prefix lines up with part of
 * a longer file or directory name, but sorts
 * after it, so this sub-directory cannot
 * collide with a file.
 *
 * last: xxx/yy-file (because '-' sorts before '/')
 * this: xxx/yy/abc
 */

However, a file named xxx/yy would be sorted before xxx/yy-file because '-' sorts after NUL, so the length check against the last entry is not sufficient to rule out a collision. Remove it.

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Suggested-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 read-cache.c | 14 --------------
 1 file changed, 14 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index aa427c5c17..8ed1c29b54 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1171,20 +1171,6 @@ static int has_dir_name(struct index_state *istate,
 				return retval;
 			}

-			if (istate->cache_nr > 0 &&
-				ce_namelen(istate->cache[istate->cache_nr - 1]) > len) {
-				/*
-				 * The directory prefix lines up with part of
-				 * a longer file or directory name, but sorts
-				 * after it, so this sub-directory cannot
-				 * collide with a file.
-				 *
-				 * last: xxx/yy-file (because '-' sorts before '/')
-				 * this: xxx/yy/abc
-				 */
-				return retval;
-			}
-
 			/*
 			 * This is a possible collision. Fall through and
 			 * let the regular search code handle it.
--
2.27.0
Previous: Jeff HostetlerNext: git@jeffhostetler.com
Message 10 of 11 in “read-cache: speed up add_index_entry”
  1. 0/5 read-cache: speed up add_index_entrygit@jeffhostetler.com, Apr 19, 2017
  2. 2/5 p0006-read-tree-checkout: perf test to time read-treegit@jeffhostetler.com, Apr 19, 2017
  3. 1/5 read-cache: add strcmp_offset functiongit@jeffhostetler.com, Apr 19, 2017
  4. 3/5 read-cache: speed up add_index_entry during checkoutgit@jeffhostetler.com, Apr 19, 2017
  5. 5/5 read-cache: speed up has_dir_name (part 2)git@jeffhostetler.com, Apr 19, 2017
  6. SZEDER GáborJul 4, 2020
  7. René ScharfeJul 5, 2020
  8. Junio C HamanoJul 6, 2020
  9. Jeff HostetlerJul 8, 2020
  10. René ScharfeJul 16, 2020
  11. 4/5 read-cache: speed up has_dir_name (part 1)git@jeffhostetler.com, Apr 19, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.