git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] unpack-trees: fix accidentally quadratic behavior

From
Junio C Hamano <gitster@pobox.com>
Date
Jan 21, 2016, 19:51 UTC
Message-ID
<xmqqpowuyall.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<1453403398.16226.40.camel@twopensource.com>
David Turner <dturner@twopensource.com> writes:
Show 30 quoted lines
> On Wed, 2016-01-20 at 20:58 -0800, Junio C Hamano wrote:
>> David Turner <dturner@twopensource.com> writes:
>> 
>> > While unpacking trees (e.g. during git checkout), when we hit a
>> > cache
>> > entry that's past and outside our path, we cut off iteration.
>> > 
>> > This provides about a 45% speedup on git checkout between master
>> > and
>> > master^20000 on Twitter's monorepo.  Speedup in general will depend
>> > on
>> > repostitory structure, number of changes, and packfile packing
>> > decisions.
>> > 
>> > Signed-off-by: David Turner <dturner@twopensource.com>
>> > ---
>> 
>> I haven't thought things through, but does this get fooled by the
>> somewhat strange ordering rules of tree entries (i.e. a subtree
>> sorts as if its name is suffixed with a '/' in a tree object)?
>> 
>> Other than that, I like this.  "We know the list is sorted, and
>> after seeing this entry we know there is nothing that will match" is
>> an obvious optimization that we already use elsewhere.
>> 
>> Thanks.
>
> I think this is correct, because we first do the more complicated check
> (ce_in_traverse_path), and only check the ordering once that has
> failed.  
But the patch does this:
Show 12 quoted lines
> +			if (info->prev && info->traverse_path) {
> +				int prefix_cmp = strncmp(ce->name, info->traverse_path, info->pathlen);
> +				if (prefix_cmp > 0)
> +					break;
> +				else if (prefix_cmp == 0 &&
> +					 ce_namelen(ce) >= info->pathlen &&
> +					 strcmp(ce->name + info->pathlen,
> +						 info->name.path) > 0) {
> +					break;
> +				}
> +			}
>  			continue;

The first break is correct, but I am not sure about the "else if" part. Shouldn't it be doing something similar to the logic to "keep looking" that talks about "t-i", "t" and "t/a" at the end of the loop?

> The tests all pass, so this should be good.

Please don't ever say that again. The existing tests do not cover all corner cases, and they fundamentally cannot cover the corner cases future changes may introduce.

Saying "Even test X breaks, so it is clearly broken. Why did you send it out without testing?" is OK, though.

Previous: David TurnerNext: David Turner
Message 4 of 6 in “unpack-trees: fix accidentally quadratic behavior”
  1. unpack-trees: fix accidentally quadratic behaviorDavid Turner, Jan 21, 2016
  2. Junio C HamanoJan 21, 2016
  3. David TurnerJan 21, 2016
  4. Junio C HamanoJan 21, 2016
  5. David TurnerJan 21, 2016
  6. Junio C HamanoJan 21, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.