git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 7/8] merge-ort: skip rename detection entirely if possible

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 11, 2021, 00:38 UTC
Message-ID
<80a0c27a74ad0314a84e956acf233f6de3b16252.1615423112.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.845.v3.git.1615423111.gitgitgadget@gmail.com>
From: Elijah Newren <newren@gmail.com>
diffcore_rename_extended() will do a bunch of setup, then check for
exact renames, then abort before inexact rename detection if there are
no more sources or destinations that need to be matched.  It will
sometimes be the case, however, that either
  * we start with neither any sources or destinations
  * we start with no *relevant* sources
In the first of these two cases, the setup and exact rename detection
will be very cheap since there are 0 files to operate on.  In the second
case, it is quite possible to have thousands of files with none of the
source ones being relevant.  Avoid calling diffcore_rename_extended() or
even some of the setup before diffcore_rename_extended() when we can
determine that rename detection is unnecessary.

For the testcases mentioned in commit 557ac0350d ("merge-ort: begin performance work; instrument with trace2_region_* calls", 2020-10-28), this change improves the performance as follows:

                            Before                  After
    no-renames:        6.003 s ±  0.048 s     5.708 s ±  0.111 s
    mega-renames:    114.009 s ±  0.236 s   102.171 s ±  0.440 s
    just-one-mega:     3.489 s ±  0.017 s     3.471 s ±  0.015 s
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
diff --git a/merge-ort.c b/merge-ort.c
index eea14024c657..bd089cb9a76f 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -2157,6 +2157,19 @@ static int process_renames(struct merge_options *opt,
 	return clean_merge;
 }
 
+static inline int possible_side_renames(struct rename_info *renames,
+					unsigned side_index)
+{
+	return renames->pairs[side_index].nr > 0 &&
+	       !strset_empty(&renames->relevant_sources[side_index]);
+}
+
+static inline int possible_renames(struct rename_info *renames)
+{
+	return possible_side_renames(renames, 1) ||
+	       possible_side_renames(renames, 2);
+}
+
 static void resolve_diffpair_statuses(struct diff_queue_struct *q)
 {
 	/*
@@ -2193,6 +2206,16 @@ static void detect_regular_renames(struct merge_options *opt,
 	struct diff_options diff_opts;
 	struct rename_info *renames = &opt->priv->renames;
 
+	if (!possible_side_renames(renames, side_index)) {
+		/*
+		 * No rename detection needed for this side, but we still need
+		 * to make sure 'adds' are marked correctly in case the other
+		 * side had directory renames.
+		 */
+		resolve_diffpair_statuses(&renames->pairs[side_index]);
+		return;
+	}
+
 	repo_diff_setup(opt->repo, &diff_opts);
 	diff_opts.flags.recursive = 1;
 	diff_opts.flags.rename_empty = 0;
@@ -2310,6 +2333,8 @@ static int detect_and_process_renames(struct merge_options *opt,
 	int need_dir_renames, s, clean = 1;
 
 	memset(&combined, 0, sizeof(combined));
+	if (!possible_renames(renames))
+		goto cleanup;
 
 	trace2_region_enter("merge", "regular renames", opt->repo);
 	detect_regular_renames(opt, MERGE_SIDE1);
@@ -2344,6 +2369,25 @@ static int detect_and_process_renames(struct merge_options *opt,
 	clean &= process_renames(opt, &combined);
 	trace2_region_leave("merge", "process renames", opt->repo);
 
+	goto simple_cleanup; /* collect_renames() handles some of cleanup */
+
+cleanup:
+	/*
+	 * Free now unneeded filepairs, which would have been handled
+	 * in collect_renames() normally but we skipped that code.
+	 */
+	for (s = MERGE_SIDE1; s <= MERGE_SIDE2; s++) {
+		struct diff_queue_struct *side_pairs;
+		int i;
+
+		side_pairs = &renames->pairs[s];
+		for (i = 0; i < side_pairs->nr; ++i) {
+			struct diff_filepair *p = side_pairs->queue[i];
+			diff_free_filepair(p);
+		}
+	}
+
+simple_cleanup:
 	/* Free memory for renames->pairs[] and combined */
 	for (s = MERGE_SIDE1; s <= MERGE_SIDE2; s++) {
 		free(renames->pairs[s].queue);
-- 
gitgitgadget
Previous: Elijah Newren via GitGitGadgetNext: Derrick Stolee
Message 37 of 39 in “Optimization batch 9: avoid detecting irrelevant renames”
  1. 0/8 Optimization batch 9: avoid detecting irrelevant renamesElijah Newren via GitGitGadget, Feb 28, 2021
  2. 1/8 diffcore-rename: enable filtering possible rename sourcesElijah Newren via GitGitGadget, Feb 28, 2021
  3. 3/8 merge-ort: add data structures for an alternate tree traversalElijah Newren via GitGitGadget, Feb 28, 2021
  4. 2/8 merge-ort: precompute subset of sources for which we need rename detectionElijah Newren via GitGitGadget, Feb 28, 2021
  5. 4/8 merge-ort: introduce wrappers for alternate tree traversalElijah Newren via GitGitGadget, Feb 28, 2021
  6. 5/8 merge-ort: precompute whether directory rename detection is neededElijah Newren via GitGitGadget, Feb 28, 2021
  7. 6/8 merge-ort: use relevant_sources to filter possible rename sourcesElijah Newren via GitGitGadget, Feb 28, 2021
  8. 7/8 merge-ort: skip rename detection entirely if possibleElijah Newren via GitGitGadget, Feb 28, 2021
  9. 8/8 diffcore-rename: avoid doing basename comparisons for irrelevant sourcesElijah Newren via GitGitGadget, Feb 28, 2021
  10. Elijah NewrenMar 1, 2021
  11. Elijah NewrenMar 4, 2021
  12. 0/8 Optimization batch 9: avoid detecting irrelevant renamesElijah Newren via GitGitGadget, Mar 9, 2021
  13. 2/8 merge-ort: precompute subset of sources for which we need rename detectionElijah Newren via GitGitGadget, Mar 9, 2021
  14. 1/8 diffcore-rename: enable filtering possible rename sourcesElijah Newren via GitGitGadget, Mar 9, 2021
  15. Derrick StoleeMar 9, 2021
  16. Elijah NewrenMar 9, 2021
  17. Derrick StoleeMar 9, 2021
  18. 6/8 merge-ort: use relevant_sources to filter possible rename sourcesElijah Newren via GitGitGadget, Mar 9, 2021
  19. 3/8 merge-ort: add data structures for an alternate tree traversalElijah Newren via GitGitGadget, Mar 9, 2021
  20. 4/8 merge-ort: introduce wrappers for alternate tree traversalElijah Newren via GitGitGadget, Mar 9, 2021
  21. Derrick StoleeMar 9, 2021
  22. 5/8 merge-ort: precompute whether directory rename detection is neededElijah Newren via GitGitGadget, Mar 9, 2021
  23. 8/8 diffcore-rename: avoid doing basename comparisons for irrelevant sourcesElijah Newren via GitGitGadget, Mar 9, 2021
  24. 7/8 merge-ort: skip rename detection entirely if possibleElijah Newren via GitGitGadget, Mar 9, 2021
  25. Derrick StoleeMar 9, 2021
  26. Elijah NewrenMar 9, 2021
  27. Derrick StoleeMar 9, 2021
  28. Derrick StoleeMar 10, 2021
  29. 0/8 Optimization batch 9: avoid detecting irrelevant renamesElijah Newren via GitGitGadget, Mar 11, 2021
  30. 1/8 diffcore-rename: enable filtering possible rename sourcesElijah Newren via GitGitGadget, Mar 11, 2021
  31. 3/8 merge-ort: add data structures for an alternate tree traversalElijah Newren via GitGitGadget, Mar 11, 2021
  32. 5/8 merge-ort: precompute whether directory rename detection is neededElijah Newren via GitGitGadget, Mar 11, 2021
  33. 2/8 merge-ort: precompute subset of sources for which we need rename detectionElijah Newren via GitGitGadget, Mar 11, 2021
  34. 4/8 merge-ort: introduce wrappers for alternate tree traversalElijah Newren via GitGitGadget, Mar 11, 2021
  35. 8/8 diffcore-rename: avoid doing basename comparisons for irrelevant sourcesElijah Newren via GitGitGadget, Mar 11, 2021
  36. 6/8 merge-ort: use relevant_sources to filter possible rename sourcesElijah Newren via GitGitGadget, Mar 11, 2021
  37. 7/8 merge-ort: skip rename detection entirely if possibleElijah Newren via GitGitGadget, Mar 11, 2021
  38. Derrick StoleeMar 15, 2021
  39. Elijah NewrenMar 15, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.