git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 08/19] prune_refs(): use delete_refs()

From
Michael Haggerty <mhagger@alum.mit.edu>
Date
Jun 22, 2015, 14:02 UTC
Message-ID
<a338a63f4c3225c7faa001a5bdde92c7690eba79.1434980615.git.mhagger@alum.mit.edu>
In-Reply-To
<cover.1434980615.git.mhagger@alum.mit.edu>

The old version just looped over the references to delete, calling delete_ref() on each one. But that has quadratic behavior, because each call to delete_ref() might have to rewrite the packed-refs file. This can be very expensive in a repository with a large number of references. In some (admittedly extreme) repositories, we've seen cases where the ref-pruning part of fetch takes multiple tens of minutes.

Instead call delete_refs(), which (aside from being less code) has the optimization that it only rewrites the packed-refs file a single time.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/fetch.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 8d5b2db..34b6f5e 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -790,20 +790,29 @@ static int prune_refs(struct refspec *refs, int ref_count, struct ref *ref_map,
 	if (4 < i && !strncmp(".git", url + i - 3, 4))
 		url_len = i - 3;
 
-	for (ref = stale_refs; ref; ref = ref->next) {
-		if (!dry_run)
-			result |= delete_ref(ref->name, NULL, 0);
-		if (verbosity >= 0 && !shown_url) {
-			fprintf(stderr, _("From %.*s\n"), url_len, url);
-			shown_url = 1;
-		}
-		if (verbosity >= 0) {
+	if (!dry_run) {
+		struct string_list refnames = STRING_LIST_INIT_NODUP;
+
+		for (ref = stale_refs; ref; ref = ref->next)
+			string_list_append(&refnames, ref->name);
+
+		result = delete_refs(&refnames);
+		string_list_clear(&refnames, 0);
+	}
+
+	if (verbosity >= 0) {
+		for (ref = stale_refs; ref; ref = ref->next) {
+			if (!shown_url) {
+				fprintf(stderr, _("From %.*s\n"), url_len, url);
+				shown_url = 1;
+			}
 			fprintf(stderr, " x %-*s %-*s -> %s\n",
 				TRANSPORT_SUMMARY(_("[deleted]")),
 				REFCOL_WIDTH, _("(none)"), prettify_refname(ref->name));
 			warn_dangling_symref(stderr, dangling_msg, ref->name);
 		}
 	}
+
 	free(url);
 	free_refs(stale_refs);
 	return result;

-- 
2.1.4
Previous: Michael HaggertyNext: Michael Haggerty
Message 9 of 24 in “Improve "refs" encapsulation and speed up deletes”
  1. 00/19 Improve "refs" encapsulation and speed up deletesMichael Haggerty, Jun 22, 2015
  2. 01/19 delete_ref(): move declaration to refs.hMichael Haggerty, Jun 22, 2015
  3. 02/19 remove_branches(): remove temporaryMichael Haggerty, Jun 22, 2015
  4. 03/19 delete_ref(): handle special case more explicitlyMichael Haggerty, Jun 22, 2015
  5. 04/19 delete_refs(): new function for the refs APIMichael Haggerty, Jun 22, 2015
  6. 05/19 delete_refs(): make error message more genericMichael Haggerty, Jun 22, 2015
  7. 06/19 delete_refs(): bail early if the packed-refs file cannot be rewrittenMichael Haggerty, Jun 22, 2015
  8. 07/19 prune_remote(): use delete_refs()Michael Haggerty, Jun 22, 2015
  9. 08/19 prune_refs(): use delete_refs()Michael Haggerty, Jun 22, 2015
  10. 09/19 repack_without_refs(): make function privateMichael Haggerty, Jun 22, 2015
  11. 10/19 initial_ref_transaction_commit(): function for initial ref creationMichael Haggerty, Jun 22, 2015
  12. 11/19 refs: remove some functions from the module's public interfaceMichael Haggerty, Jun 22, 2015
  13. 12/19 initial_ref_transaction_commit(): check for duplicate refsMichael Haggerty, Jun 22, 2015
  14. Junio C HamanoJun 22, 2015
  15. Michael HaggertyJun 23, 2015
  16. Junio C HamanoJun 23, 2015
  17. 13/19 initial_ref_transaction_commit(): check for ref D/F conflictsMichael Haggerty, Jun 22, 2015
  18. 14/19 refs: move the remaining ref module declarations to refs.hMichael Haggerty, Jun 22, 2015
  19. 15/19 refs.h: add some parameter names to function declarationsMichael Haggerty, Jun 22, 2015
  20. 16/19 check_branch_commit(): make first parameter constMichael Haggerty, Jun 22, 2015
  21. 17/19 update_ref(): don't read old reference value before deleteMichael Haggerty, Jun 22, 2015
  22. 18/19 cmd_update_ref(): make logic more straightforwardMichael Haggerty, Jun 22, 2015
  23. 19/19 delete_ref(): use the usual convention for old_sha1Michael Haggerty, Jun 22, 2015
  24. Junio C HamanoJun 22, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.