git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 8/8] diff: optionally use rename cache

From
Jeff King <peff@peff.net>
Date
Aug 4, 2012, 17:14 UTC
Message-ID
<20120804171421.GH19378@sigill.intra.peff.net>
In-Reply-To
<20120804170905.GA19267@sigill.intra.peff.net>

This speeds up estimate_similarity by caching the similarity score of pairs of blob sha1s.

Signed-off-by: Jeff King <peff@peff.net>
---
Some interesting things to time with this are:
  - "git log --raw -M" on a repo with a lot of paths or a lot of renames
    (I found on git.git, the speedup was not that impressive)
  - "git log --raw -C -C" on any repo (this speeds up a lot in git.git).
  - "git show -M" on commits with very large blobs
 cache.h           |  1 +
 diff.c            |  6 ++++++
 diffcore-rename.c | 11 ++++++++++-
 3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/cache.h b/cache.h
index 23a2f93..7ee1caf 100644
--- a/cache.h
+++ b/cache.h
@@ -1228,6 +1228,7 @@ int add_files_to_cache(const char *prefix, const char **pathspec, int flags);
 
 /* diff.c */
 extern int diff_auto_refresh_index;
+extern int diff_cache_renames;
 
 /* match-trees.c */
 void shift_tree(const unsigned char *, const unsigned char *, unsigned char *, int);
diff --git a/diff.c b/diff.c
index 95706a5..c84e043 100644
--- a/diff.c
+++ b/diff.c
@@ -34,6 +34,7 @@ static int diff_no_prefix;
 static int diff_stat_graph_width;
 static int diff_dirstat_permille_default = 30;
 static struct diff_options default_diff_options;
+int diff_cache_renames;
 
 static char diff_colors[][COLOR_MAXLEN] = {
 	GIT_COLOR_RESET,
@@ -214,6 +215,11 @@ int git_diff_basic_config(const char *var, const char *value, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(var, "diff.cacherenames")) {
+		diff_cache_renames = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!prefixcmp(var, "submodule."))
 		return parse_submodule_config_option(var, value);
 
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 216a7a4..611e1d3 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -6,6 +6,7 @@
 #include "diffcore.h"
 #include "hash.h"
 #include "progress.h"
+#include "metadata-cache.h"
 
 /* Table of rename/copy destinations */
 
@@ -137,7 +138,8 @@ static int estimate_similarity(struct diff_filespec *src,
 	 */
 	unsigned long max_size, delta_size, base_size, src_copied, literal_added;
 	unsigned long delta_limit;
-	int score;
+	uint32_t score;
+	struct sha1pair pair;
 
 	/* We deal only with regular files.  Symlink renames are handled
 	 * only when they are exact matches --- in other words, no edits
@@ -175,6 +177,11 @@ static int estimate_similarity(struct diff_filespec *src,
 	if (max_size * (MAX_SCORE-minimum_score) < delta_size * MAX_SCORE)
 		return 0;
 
+	hashcpy(pair.one, src->sha1);
+	hashcpy(pair.two, dst->sha1);
+	if (diff_cache_renames && rename_cache_get(&pair, &score))
+			return score;
+
 	if (!src->cnt_data && diff_populate_filespec(src, 0))
 		return 0;
 	if (!dst->cnt_data && diff_populate_filespec(dst, 0))
@@ -195,6 +202,8 @@ static int estimate_similarity(struct diff_filespec *src,
 		score = 0; /* should not happen */
 	else
 		score = (int)(src_copied * MAX_SCORE / max_size);
+	if (diff_cache_renames)
+		rename_cache_set(&pair, score);
 	return score;
 }
 
-- 
1.7.12.rc1.7.g7a223a6
Previous: Jeff King
Message 35 of 35 in “Manual rename correction”
  1. Manual rename correctionNguyen Thai Ngoc Duy, Jul 31, 2012
  2. Junio C HamanoJul 31, 2012
  3. Jeff KingJul 31, 2012
  4. Junio C HamanoJul 31, 2012
  5. Jeff KingAug 1, 2012
  6. Junio C HamanoAug 1, 2012
  7. Jeff KingAug 1, 2012
  8. Junio C HamanoAug 1, 2012
  9. Jeff KingAug 2, 2012
  10. Junio C HamanoAug 2, 2012
  11. Jeff KingAug 2, 2012
  12. Junio C HamanoAug 2, 2012
  13. Nguyen Thai Ngoc DuyAug 1, 2012
  14. Jeff KingAug 1, 2012
  15. Nguyen Thai Ngoc DuyAug 1, 2012
  16. Junio C HamanoAug 1, 2012
  17. Nguyen Thai Ngoc DuyAug 1, 2012
  18. Jeff KingAug 1, 2012
  19. Jeff KingAug 1, 2012
  20. Nguyen Thai Ngoc DuyAug 2, 2012
  21. Jeff KingAug 2, 2012
  22. 0/8 caching rename resultsJeff King, Aug 4, 2012
  23. 1/8 implement generic key/value mapJeff King, Aug 4, 2012
  24. Junio C HamanoAug 4, 2012
  25. Jeff KingAug 6, 2012
  26. 2/8 map: add helper functions for objects as keysJeff King, Aug 4, 2012
  27. 3/8 fast-export: use object to uint32 map instead of "decorate"Jeff King, Aug 4, 2012
  28. 4/8 decorate: use "map" for the underlying implementationJeff King, Aug 4, 2012
  29. 5/8 map: implement persistent mapsJeff King, Aug 4, 2012
  30. 6/8 implement metadata cache subsystemJeff King, Aug 4, 2012
  31. Junio C HamanoAug 4, 2012
  32. Jeff KingAug 6, 2012
  33. Jeff KingAug 6, 2012
  34. 7/8 implement rename cacheJeff King, Aug 4, 2012
  35. 8/8 diff: optionally use rename cacheJeff King, Aug 4, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.