git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH v2 1/9] diffcore-rename: no point trying to find a match better than exact

From
Elijah Newren <newren@gmail.com>
Date
Nov 20, 2017, 22:19 UTC
Message-ID
<20171120221944.15431-2-newren@gmail.com>
In-Reply-To
<20171120221944.15431-1-newren@gmail.com>

diffcore_rename() had some code to avoid having destination paths that already had an exact rename detected from being re-checked for other renames. Source paths, however, were re-checked because we wanted to allow the possibility of detecting copies. But if copy detection isn't turned on, then this merely amounts to attempting to find a better-than-exact match, which naturally ends up being an expensive no-op. In particular, copy detection is never turned on by the merge recursive machinery.

In a large repository (~50k files, about 60% of which was java) that had a number of high level directories involved in renames, this cut the time necessary for a cherry-pick down by about 50% (from around 9 minutes to 4.5 minutes).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diffcore-rename.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 9ca0eaec70..fe37e3b861 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -377,11 +377,10 @@ static void record_if_better(struct diff_score m[], struct diff_score *o)
  * 1 if we need to disable inexact rename detection;
  * 2 if we would be under the limit if we were given -C instead of -C -C.
  */
-static int too_many_rename_candidates(int num_create,
+static int too_many_rename_candidates(int num_create, int num_src,
 				      struct diff_options *options)
 {
 	int rename_limit = options->rename_limit;
-	int num_src = rename_src_nr;
 	int i;
 
 	options->needed_rename_limit = 0;
@@ -446,7 +445,7 @@ void diffcore_rename(struct diff_options *options)
 	struct diff_queue_struct outq;
 	struct diff_score *mx;
 	int i, j, rename_count, skip_unmodified = 0;
-	int num_create, dst_cnt;
+	int num_create, dst_cnt, num_src;
 	struct progress *progress = NULL;
 
 	if (!minimum_score)
@@ -512,12 +511,14 @@ void diffcore_rename(struct diff_options *options)
 	 * files still remain as options for rename/copies!)
 	 */
 	num_create = (rename_dst_nr - rename_count);
+	num_src = (detect_rename == DIFF_DETECT_COPY ?
+		   rename_src_nr : rename_src_nr - rename_count);
 
 	/* All done? */
 	if (!num_create)
 		goto cleanup;
 
-	switch (too_many_rename_candidates(num_create, options)) {
+	switch (too_many_rename_candidates(num_create, num_src, options)) {
 	case 1:
 		goto cleanup;
 	case 2:
@@ -531,7 +532,7 @@ void diffcore_rename(struct diff_options *options)
 	if (options->show_rename_progress) {
 		progress = start_delayed_progress(
 				_("Performing inexact rename detection"),
-				(uint64_t)rename_dst_nr * (uint64_t)rename_src_nr);
+				(uint64_t)num_create * (uint64_t)num_src);
 	}
 
 	mx = xcalloc(st_mult(NUM_CANDIDATE_PER_DST, num_create), sizeof(*mx));
@@ -550,6 +551,10 @@ void diffcore_rename(struct diff_options *options)
 			struct diff_filespec *one = rename_src[j].p->one;
 			struct diff_score this_src;
 
+			if (one->rename_used &&
+			    detect_rename != DIFF_DETECT_COPY)
+				continue;
+
 			if (skip_unmodified &&
 			    diff_unmodified_pair(rename_src[j].p))
 				continue;
@@ -568,7 +573,7 @@ void diffcore_rename(struct diff_options *options)
 			diff_free_filespec_blob(two);
 		}
 		dst_cnt++;
-		display_progress(progress, (uint64_t)(i+1)*(uint64_t)rename_src_nr);
+		display_progress(progress, (uint64_t)dst_cnt*(uint64_t)num_src);
 	}
 	stop_progress(&progress);
 
-- 
2.15.0.323.g31fe956618
Previous: Elijah NewrenNext: Elijah Newren
Message 4 of 10 in “Improve merge recursive performance”
  1. 0/9 Improve merge recursive performanceElijah Newren, Nov 20, 2017
  2. 2/9 merge-recursive: avoid unnecessary string list lookupsElijah Newren, Nov 20, 2017
  3. 3/9 merge-recursive: new function for better colliding conflict resolutionsElijah Newren, Nov 20, 2017
  4. 1/9 diffcore-rename: no point trying to find a match better than exactElijah Newren, Nov 20, 2017
  5. 7/9 merge-recursive: improve handling for add/add conflictsElijah Newren, Nov 20, 2017
  6. 6/9 merge-recursive: improve handling for rename/rename(2to1) conflictsElijah Newren, Nov 20, 2017
  7. 4/9 Add testcases for improved file collision conflict handlingElijah Newren, Nov 20, 2017
  8. 9/9 diffcore-rename: filter rename_src list when possibleElijah Newren, Nov 20, 2017
  9. 5/9 merge-recursive: fix rename/add conflict handlingElijah Newren, Nov 20, 2017
  10. 8/9 merge-recursive: accelerate rename detectionElijah Newren, Nov 20, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.