git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/2] fetch: Speed up fetch of large numbers of refs

From
Julian Phillips <julian@quantumfyre.co.uk>
Date
Oct 25, 2009, 21:28 UTC
Message-ID
<20091025212813.48498.87676.julian@quantumfyre.co.uk>
In-Reply-To
<20091025212449.48498.23208.julian@quantumfyre.co.uk>

When there are large numbers of refs, calling read_ref for each ref is inefficent (and infact downright slow) - so instead use for_each_ref to build up a string list of all the refs that we currently have, which significantly improves the volume.

Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
---
 builtin-fetch.c |   17 ++++++++++++++---
 1 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/builtin-fetch.c b/builtin-fetch.c
index acb08e4..0f53cbd 100644
--- a/builtin-fetch.c
+++ b/builtin-fetch.c
@@ -489,7 +489,8 @@ static int add_existing(const char *refname, const unsigned char *sha1,
 			int flag, void *cbdata)
 {
 	struct string_list *list = (struct string_list *)cbdata;
-	string_list_insert(refname, list);
+	struct string_list_item *item = string_list_insert(refname, list);
+	item->util = (void *)sha1;
 	return 0;
 }
 
@@ -606,9 +607,14 @@ static void check_not_current_branch(struct ref *ref_map)
 static int do_fetch(struct transport *transport,
 		    struct refspec *refs, int ref_count)
 {
+	struct string_list existing_refs = { NULL, 0, 0, 0 };
+	struct string_list_item *peer_item = NULL;
 	struct ref *ref_map;
 	struct ref *rm;
 	int autotags = (transport->remote->fetch_tags == 1);
+
+	for_each_ref(add_existing, &existing_refs);
+
 	if (transport->remote->fetch_tags == 2 && tags != TAGS_UNSET)
 		tags = TAGS_SET;
 	if (transport->remote->fetch_tags == -1)
@@ -631,8 +637,13 @@ static int do_fetch(struct transport *transport,
 		check_not_current_branch(ref_map);
 
 	for (rm = ref_map; rm; rm = rm->next) {
-		if (rm->peer_ref)
-			read_ref(rm->peer_ref->name, rm->peer_ref->old_sha1);
+		if (rm->peer_ref) {
+			peer_item = string_list_lookup(rm->peer_ref->name,
+						       &existing_refs);
+			if (peer_item)
+				hashcpy(rm->peer_ref->old_sha1,
+					peer_item->util);
+		}
 	}
 
 	if (tags == TAGS_DEFAULT && autotags)
-- 
1.6.5.rc2
Previous: Julian Phillips
Message 4 of 4 in “Speedup fetch with large numbers of refs”
  1. 0/2 Speedup fetch with large numbers of refsJulian Phillips, Oct 25, 2009
  2. 1/2 remote: Make ref_remove_duplicates faster for large numbers of refsJulian Phillips, Oct 25, 2009
  3. 3/2 remote: fix poential ref_map list corruption in ref_remove_duplicatesJulian Phillips, Oct 26, 2009
  4. 2/2 fetch: Speed up fetch of large numbers of refsJulian Phillips, Oct 25, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.