git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 5/5] Use xdiff caching to improve git blame performance

From
Brian Downing <bdowning@lavos.net>
Date
Aug 21, 2008, 23:22 UTC
Message-ID
<1219360921-28529-6-git-send-email-bdowning@lavos.net>
In-Reply-To
<1219360921-28529-5-git-send-email-bdowning@lavos.net>

git blame -C -C diffs all entries against the same file. To avoid having to recompute the line hashes for this file over and over again, use the xddiff cache feature to store the hashes in struct origin.

Note that because this bypasses the xdi_diff tail trimming, it can return different (but still valid) blame results for certain cases. See http://article.gmane.org/gmane.comp.version-control.git/93112 for more details.

This yields another significant speed improvement for some cases:

:; time git-blame -M -C -C -p --incremental server.c >/dev/null Before: 48.66user 0.08system 0:48.75elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+36961minor)pagefaults 0swaps After: 29.68user 0.22system 0:29.98elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+37897minor)pagefaults 0swaps

Signed-off-by: Brian Downing <bdowning@lavos.net>
---
 builtin-blame.c |   14 ++++++++++----
 1 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/builtin-blame.c b/builtin-blame.c
index 66b7d15..3e90668 100644
--- a/builtin-blame.c
+++ b/builtin-blame.c
@@ -80,6 +80,7 @@ struct origin {
 	int refcnt;
 	struct commit *commit;
 	mmfile_t file;
+	xdcache_t xdcache;
 	unsigned char blob_sha1[20];
 	char path[FLEX_ARRAY];
 };
@@ -119,6 +120,7 @@ static inline struct origin *origin_incref(struct origin *o)
 static void origin_decref(struct origin *o)
 {
 	if (o && --o->refcnt <= 0) {
+	        xdl_cache_free(&o->xdcache);
 		free(o->file.ptr);
 		free(o);
 	}
@@ -494,7 +496,8 @@ static int process_diff(xdfenv_t *xe, xdchange_t *xscr, xdemitcb_t *ecb,
 	return 0;
 }
 
-static struct patch *compare_buffer(mmfile_t *file_p, mmfile_t *file_o,
+static struct patch *compare_buffer(mmfile_t *file_p, xdcache_t *cache_p,
+				    mmfile_t *file_o, xdcache_t *cache_o,
 				    int context)
 {
 	struct patch *patch;
@@ -504,6 +507,8 @@ static struct patch *compare_buffer(mmfile_t *file_p, mmfile_t *file_o,
 
 	memset(&xpp, 0, sizeof(xpp));
 	xpp.flags = xdl_opts;
+	xpp.mf1_cache = cache_p;
+	xpp.mf2_cache = cache_o;
 	memset(&xecfg, 0, sizeof(xecfg));
 	xecfg.ctxlen = context;
 	patch = xmalloc(sizeof(struct patch));
@@ -511,7 +516,7 @@ static struct patch *compare_buffer(mmfile_t *file_p, mmfile_t *file_o,
 	patch->num = 0;
 	xecfg.emit_func = (void (*)())process_diff;
 	ecb.priv = patch;
-	xdi_diff(file_p, file_o, &xpp, &xecfg, &ecb);
+	xdl_diff(file_p, file_o, &xpp, &xecfg, &ecb);
 
 	return patch;
 }
@@ -530,7 +535,8 @@ static struct patch *get_patch(struct origin *parent, struct origin *origin)
 	fill_origin_blob(origin, &file_o);
 	if (!file_p.ptr || !file_o.ptr)
 		return NULL;
-	patch = compare_buffer(&file_p, &file_o, 0);
+	patch = compare_buffer(&file_p, &parent->xdcache,
+			       &file_o, &origin->xdcache, 0);
 	num_get_patch++;
 	return patch;
 }
@@ -937,7 +943,7 @@ static void find_copy_in_blob(struct scoreboard *sb,
 	}
 	file_o.size = cp - file_o.ptr;
 
-	patch = compare_buffer(file_p, &file_o, 1);
+	patch = compare_buffer(file_p, &parent->xdcache, &file_o, NULL, 1);
 
 	/*
 	 * file_o is a part of final image we are annotating.
-- 
1.5.6.1
Previous: Brian DowningNext: René Scharfe
Message 6 of 17 in “More git blame speed improvements”
  1. 0/5 More git blame speed improvementsBrian Downing, Aug 21, 2008
  2. 1/5 Allow alternate "low-level" emit function from xdl_diffBrian Downing, Aug 21, 2008
  3. 2/5 Bypass textual patch generation and parsing in git blameBrian Downing, Aug 21, 2008
  4. 3/5 Always initialize xpparam_t to 0Brian Downing, Aug 21, 2008
  5. 4/5 Allow xdiff machinery to cache hash results for a fileBrian Downing, Aug 21, 2008
  6. 5/5 Use xdiff caching to improve git blame performanceBrian Downing, Aug 21, 2008
  7. René ScharfeAug 23, 2008
  8. Junio C HamanoAug 23, 2008
  9. Brian DowningAug 24, 2008
  10. René ScharfeSep 3, 2008
  11. 1/5 blame: inline get_patch()René Scharfe, Oct 25, 2008
  12. 2/5 Always initialize xpparam_t to 0René Scharfe, Oct 25, 2008
  13. 3/5 Allow alternate "low-level" emit function from xdl_diffRené Scharfe, Oct 25, 2008
  14. 4/5 add xdi_diff_hunks() for callers that only need hunk lengthsRené Scharfe, Oct 25, 2008
  15. 5/5 blame: use xdi_diff_hunks(), get rid of struct patchRené Scharfe, Oct 25, 2008
  16. Junio C HamanoOct 25, 2008
  17. René ScharfeOct 26, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.