git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] fast-import: use binary search in tree_content_remove

From
Shawn O. Pearce <spearce@spearce.org>
Date
Mar 11, 2007, 03:38 UTC
Message-ID
<20070311033833.GB10781@spearce.org>
In-Reply-To
<20070310194012.GA5126@coredump.intra.peff.net>
Jeff King <peff@peff.net> wrote:
> And here it is. However, I should note that this patch is _not_
> necessary.

Well, its not necessary for you, as you aren't trying to remove something from your huge tree. ;-)

> I had originally thought that removal might destroy the
> sorting that I added in the last patch, but it looks like the entry
> isn't actually removed. Shawn, can you sanity check this?

Your patch is fine. fast-import takes an "optimization" here and does not bother to actually delete entries from a tree until *after* we have written the tree data out to the packfile. The reason is we need to retain the version 0 tree entry objects to recreate the delta base. Since these are combined with the version 1 data (the "new" data) we have to delay the actual deletes until after we store the tree and have its SHA-1.

It turns out however that your entire series was broken. I had to commit the following on top of it to fix it:

-->8--
From 0af148553a94f7e856089fa68395524932240145 Mon Sep 17 00:00:00 2001
From: Shawn O. Pearce <spearce@spearce.org>
Date: Sat, 10 Mar 2007 22:34:12 -0500
Subject: [PATCH] fast-import: Brown paper bag fix tree sorting

Jeff King's recent changes to sort trees by strictly name (and binary search to locate an entry) works OK up until we have to write a tree out that uses the funny name/mode sorting that native Git uses:

  b.
  b/
  ba

Here the subtree "b" must sort between files "b." and "ba", but Jeff's changes have it sorting before "b.". This means we would fail to find entries during future modifications to that tree as Jeff's binary search algorithm won't find subtree "b" between b. and ba.

I'm plastering over the problem by resorting a tree strictly by name after it has been written out and the deleted entries have been filtered out.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 fast-import.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
diff --git a/fast-import.c b/fast-import.c
index 716819f..fa3b766 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1136,6 +1136,13 @@ static int tecmp1 (const void *_a, const void *_b)
 		b->name->str_dat, b->name->str_len, b->versions[1].mode);
 }
 
+static int nmcmp (const void *_a, const void *_b)
+{
+	struct tree_entry *a = *((struct tree_entry**)_a);
+	struct tree_entry *b = *((struct tree_entry**)_b);
+	return strcmp(a->name->str_dat, b->name->str_dat);
+}
+
 static void mktree(struct tree_content *t,
 	int v,
 	unsigned long *szp,
@@ -1218,6 +1225,7 @@ static void store_tree(struct tree_entry *root)
 		}
 	}
 	t->entry_count -= del;
+	qsort(t->entries, t->entry_count, sizeof(t->entries[0]), nmcmp);
 }
 
 static int tree_content_set(
-- 
1.5.0.3.942.g299f
Previous: Jeff KingNext: Jeff King
Message 11 of 15 in “fast-import slow on large directories”
  1. 0/3 fast-import slow on large directoriesJeff King, Mar 10, 2007
  2. 1/3 fast-import: grow tree storage more aggressivelyJeff King, Mar 10, 2007
  3. 2/3 fast-import: tree allocation cleanupsJeff King, Mar 10, 2007
  4. Shawn O. PearceMar 11, 2007
  5. Jeff KingMar 11, 2007
  6. Jeff KingMar 11, 2007
  7. Shawn O. PearceMar 12, 2007
  8. 3/3 fast-import: improve efficiency of tree_content_setJeff King, Mar 10, 2007
  9. Jeff KingMar 10, 2007
  10. fast-import: use binary search in tree_content_removeJeff King, Mar 10, 2007
  11. Shawn O. PearceMar 11, 2007
  12. Jeff KingMar 11, 2007
  13. Jeff KingMar 11, 2007
  14. Shawn O. PearceMar 11, 2007
  15. Shawn O. PearceMar 12, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.