git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 10/10] diffcore-rename: compute dir_rename_guess from dir_rename_counts

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 27, 2021, 00:30 UTC
Message-ID
<4be565c472088d4144063b736308bf2a57331f45.1614385849.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.844.v4.git.1614385849.gitgitgadget@gmail.com>
From: Elijah Newren <newren@gmail.com>
dir_rename_counts has a mapping of a mapping, in particular, it has
   old_dir => { new_dir => count }
We want a simple mapping of
   old_dir => new_dir
based on which new_dir had the highest count for a given old_dir.
Compute this and store it in dir_rename_guess.

This is the final piece of the puzzle needed to make our guesses at which directory files have been moved to when basenames aren't unique.

For the testcases mentioned in commit 557ac0350d ("merge-ort: begin performance work; instrument with trace2_region_* calls", 2020-10-28), this change improves the performance as follows:

                            Before                  After
    no-renames:       12.775 s ±  0.062 s    12.596 s ±  0.061 s
    mega-renames:    188.754 s ±  0.284 s   130.465 s ±  0.259 s
    just-one-mega:     5.599 s ±  0.019 s     3.958 s ±  0.010 s
Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diffcore-rename.c | 45 +++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 41 insertions(+), 4 deletions(-)
diff --git a/diffcore-rename.c b/diffcore-rename.c
index e5fa0cb555dd..1fe902ed2af0 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -389,6 +389,24 @@ static void dirname_munge(char *filename)
 	*slash = '\0';
 }
 
+static const char *get_highest_rename_path(struct strintmap *counts)
+{
+	int highest_count = 0;
+	const char *highest_destination_dir = NULL;
+	struct hashmap_iter iter;
+	struct strmap_entry *entry;
+
+	strintmap_for_each_entry(counts, &iter, entry) {
+		const char *destination_dir = entry->key;
+		intptr_t count = (intptr_t)entry->value;
+		if (count > highest_count) {
+			highest_count = count;
+			highest_destination_dir = destination_dir;
+		}
+	}
+	return highest_destination_dir;
+}
+
 static void increment_count(struct dir_rename_info *info,
 			    char *old_dir,
 			    char *new_dir)
@@ -512,6 +530,8 @@ static void initialize_dir_rename_info(struct dir_rename_info *info,
 				       struct strset *dirs_removed,
 				       struct strmap *dir_rename_count)
 {
+	struct hashmap_iter iter;
+	struct strmap_entry *entry;
 	int i;
 
 	if (!dirs_removed) {
@@ -558,6 +578,23 @@ static void initialize_dir_rename_info(struct dir_rename_info *info,
 					 rename_dst[i].p->one->path,
 					 rename_dst[i].p->two->path);
 	}
+
+	/*
+	 * Now we collapse
+	 *    dir_rename_count: old_directory -> {new_directory -> count}
+	 * down to
+	 *    dir_rename_guess: old_directory -> best_new_directory
+	 * where best_new_directory is the one with the highest count.
+	 */
+	strmap_for_each_entry(info->dir_rename_count, &iter, entry) {
+		/* entry->key is source_dir */
+		struct strintmap *counts = entry->value;
+		char *best_newdir;
+
+		best_newdir = xstrdup(get_highest_rename_path(counts));
+		strmap_put(&info->dir_rename_guess, entry->key,
+			   best_newdir);
+	}
 }
 
 void partial_clear_dir_rename_count(struct strmap *dir_rename_count)
@@ -682,10 +719,10 @@ static int idx_possible_rename(char *filename, struct dir_rename_info *info)
 	 *       rename.
 	 *
 	 * This function, idx_possible_rename(), is only responsible for (4).
-	 * The conditions/steps in (1)-(3) will be handled via setting up
-	 * dir_rename_count and dir_rename_guess in a future
-	 * initialize_dir_rename_info() function.  Steps (0) and (5) are
-	 * handled by the caller of this function.
+	 * The conditions/steps in (1)-(3) are handled via setting up
+	 * dir_rename_count and dir_rename_guess in
+	 * initialize_dir_rename_info().  Steps (0) and (5) are handled by
+	 * the caller of this function.
 	 */
 	char *old_dir, *new_dir;
 	struct strbuf new_path = STRBUF_INIT;
-- 
gitgitgadget
Previous: Elijah Newren via GitGitGadgetNext: Derrick Stolee
Message 60 of 61 in “Optimization batch 8: use file basenames even more”
  1. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 14, 2021
  2. 02/10 diffcore-rename: add functions for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 14, 2021
  3. 01/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 14, 2021
  4. 03/10 diffcore-rename: move dir_rename_counts into a dir_rename_info structElijah Newren via GitGitGadget, Feb 14, 2021
  5. 04/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 14, 2021
  6. 05/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 14, 2021
  7. 06/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 14, 2021
  8. 08/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 14, 2021
  9. 07/10 diffcore-rename: add a dir_rename_guess field to dir_rename_infoElijah Newren via GitGitGadget, Feb 14, 2021
  10. 10/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 14, 2021
  11. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 14, 2021
  12. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 23, 2021
  13. 03/10 diffcore-rename: move dir_rename_counts into a dir_rename_info structElijah Newren via GitGitGadget, Feb 23, 2021
  14. 02/10 diffcore-rename: add functions for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 23, 2021
  15. 01/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 23, 2021
  16. Derrick StoleeFeb 24, 2021
  17. Elijah NewrenFeb 24, 2021
  18. 04/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 23, 2021
  19. Derrick StoleeFeb 24, 2021
  20. Ævar Arnfjörð BjarmasonFeb 25, 2021
  21. Ævar Arnfjörð BjarmasonFeb 25, 2021
  22. Junio C HamanoFeb 25, 2021
  23. 08/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 23, 2021
  24. Derrick StoleeFeb 24, 2021
  25. Elijah NewrenFeb 25, 2021
  26. 06/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 23, 2021
  27. 10/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 23, 2021
  28. Derrick StoleeFeb 24, 2021
  29. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 23, 2021
  30. 07/10 diffcore-rename: add a dir_rename_guess field to dir_rename_infoElijah Newren via GitGitGadget, Feb 23, 2021
  31. 05/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 23, 2021
  32. Derrick StoleeFeb 24, 2021
  33. Derrick StoleeFeb 24, 2021
  34. Elijah NewrenFeb 25, 2021
  35. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 26, 2021
  36. 01/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 26, 2021
  37. 02/10 diffcore-rename: add a new idx_possible_rename functionElijah Newren via GitGitGadget, Feb 26, 2021
  38. Derrick StoleeFeb 26, 2021
  39. 03/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 26, 2021
  40. 04/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 26, 2021
  41. Derrick StoleeFeb 26, 2021
  42. 05/10 diffcore-rename: add function for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 26, 2021
  43. 06/10 diffcore-rename: move dir_rename_counts into dir_rename_info structElijah Newren via GitGitGadget, Feb 26, 2021
  44. 07/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 26, 2021
  45. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 26, 2021
  46. 10/10 diffcore-rename: compute dir_rename_guess from dir_rename_countsElijah Newren via GitGitGadget, Feb 26, 2021
  47. 08/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 26, 2021
  48. Derrick StoleeFeb 26, 2021
  49. Elijah NewrenFeb 26, 2021
  50. 00/10 Optimization batch 8: use file basenames even moreElijah Newren via GitGitGadget, Feb 27, 2021
  51. 03/10 diffcore-rename: add a mapping of destination names to their indicesElijah Newren via GitGitGadget, Feb 27, 2021
  52. 02/10 diffcore-rename: provide basic implementation of idx_possible_rename()Elijah Newren via GitGitGadget, Feb 27, 2021
  53. 01/10 diffcore-rename: use directory rename guided basename comparisonsElijah Newren via GitGitGadget, Feb 27, 2021
  54. 04/10 Move computation of dir_rename_count from merge-ort to diffcore-renameElijah Newren via GitGitGadget, Feb 27, 2021
  55. 05/10 diffcore-rename: add function for clearing dir_rename_countElijah Newren via GitGitGadget, Feb 27, 2021
  56. 06/10 diffcore-rename: move dir_rename_counts into dir_rename_info structElijah Newren via GitGitGadget, Feb 27, 2021
  57. 07/10 diffcore-rename: extend cleanup_dir_rename_info()Elijah Newren via GitGitGadget, Feb 27, 2021
  58. 09/10 diffcore-rename: limit dir_rename_counts computation to relevant dirsElijah Newren via GitGitGadget, Feb 27, 2021
  59. 08/10 diffcore-rename: compute dir_rename_counts in stagesElijah Newren via GitGitGadget, Feb 27, 2021
  60. 10/10 diffcore-rename: compute dir_rename_guess from dir_rename_countsElijah Newren via GitGitGadget, Feb 27, 2021
  61. Derrick StoleeMar 9, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.