git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC/PATCH 2/4] limit "contains" traversals based on commit timestamp

From
Jeff King <peff@peff.net>
Date
Jul 5, 2010, 12:34 UTC
Message-ID
<20100705123419.GB25699@sigill.intra.peff.net>
In-Reply-To
<20100705122723.GB21146@sigill.intra.peff.net>

When looking for commits that contain other commits (e.g., via "git tag --contains"), we can end up traversing useless portions of the graph. For example, if I am looking for a tag that contains a commit made last week, there is not much point in traversing portions of the history graph made five years ago.

This optimization can provide massive speedups. For example, doing "git tag --contains HEAD~200" in the linux-2.6 repository goes from:

  real    0m5.302s
  user    0m5.116s
  sys     0m0.184s
to:
  real    0m0.030s
  user    0m0.020s
  sys     0m0.008s

The downside is that we will no longer find some answers in the face of extreme clock skew, as we will stop the traversal early when seeing commits skewed too far into the past.

Name-rev already implements a similar optimization, using a "slop" of one day to allow for a certain amount of clock skew in commit timestamps. This patch introduces a "core.clockskew" variable, which allows specifying the allowable amount of clock skew in seconds. For safety, it defaults to "none", causing a full traversal (i.e., no change in behavior from previous versions).

Signed-off-by: Jeff King <peff@peff.net>
---
 cache.h  |    1 +
 commit.c |   27 ++++++++++++++++++++++++---
 config.c |    8 ++++++++
 3 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/cache.h b/cache.h
index c9fa3df..2dfb636 100644
--- a/cache.h
+++ b/cache.h
@@ -551,6 +551,7 @@ extern int read_replace_refs;
 extern int fsync_object_files;
 extern int core_preload_index;
 extern int core_apply_sparse_checkout;
+extern int core_clock_skew;
 
 enum safe_crlf {
 	SAFE_CRLF_FALSE = 0,
diff --git a/commit.c b/commit.c
index 20354c6..c849b50 100644
--- a/commit.c
+++ b/commit.c
@@ -7,6 +7,7 @@
 #include "revision.h"
 #include "notes.h"
 
+int core_clock_skew = -1;
 int save_commit_buffer = 1;
 
 const char *commit_type = "commit";
@@ -855,7 +856,8 @@ static int in_commit_list(const struct commit_list *want, struct commit *c)
 }
 
 static int contains_recurse(struct commit *candidate,
-			    const struct commit_list *want)
+			    const struct commit_list *want,
+			    unsigned long cutoff)
 {
 	struct commit_list *p;
 
@@ -872,9 +874,13 @@ static int contains_recurse(struct commit *candidate,
 	if (parse_commit(candidate) < 0)
 		return 0;
 
+	/* stop searching if we go too far back in time */
+	if (candidate->date < cutoff)
+		return 0;
+
 	/* Otherwise recurse and mark ourselves for future traversals. */
 	for (p = candidate->parents; p; p = p->next) {
-		if (contains_recurse(p->item, want)) {
+		if (contains_recurse(p->item, want, cutoff)) {
 			candidate->object.flags |= TMP_MARK;
 			return 1;
 		}
@@ -885,5 +891,20 @@ static int contains_recurse(struct commit *candidate,
 
 int contains(struct commit *candidate, const struct commit_list *want)
 {
-	return contains_recurse(candidate, want);
+	unsigned long cutoff = 0;
+
+	if (core_clock_skew >= 0) {
+		const struct commit_list *c;
+		unsigned long min_date = ULONG_MAX;
+		for (c = want; c; c = c->next) {
+			if (parse_commit(c->item) < 0)
+				continue;
+			if (c->item->date < min_date)
+				min_date = c->item->date;
+		}
+		if (min_date > core_clock_skew)
+			cutoff = min_date - core_clock_skew;
+	}
+
+	return contains_recurse(candidate, want, cutoff);
 }
diff --git a/config.c b/config.c
index cdcf583..7a18bc9 100644
--- a/config.c
+++ b/config.c
@@ -595,6 +595,14 @@ static int git_default_core_config(const char *var, const char *value)
 		return 0;
 	}
 
+	if (!strcmp(var, "core.clockskew")) {
+		if (!value || !strcmp(value, "none"))
+			core_clock_skew = -1;
+		else
+			core_clock_skew = git_config_int(var, value);
+		return 0;
+	}
+
 	/* Add other config variables here and to Documentation/config.txt. */
 	return 0;
 }
-- 
1.7.2.rc1.209.g2a36c
Previous: Jeff KingNext: Jonathan Nieder
Message 17 of 46 in “Why is "git tag --contains" so slow?”
  1. Theodore Ts'oJul 1, 2010
  2. Shawn O. PearceJul 1, 2010
  3. Sam VilainJul 3, 2010
  4. Avery PennarunJul 1, 2010
  5. tytso@mit.eduJul 1, 2010
  6. Jeff KingJul 1, 2010
  7. Jeff KingJul 1, 2010
  8. tytso@mit.eduJul 2, 2010
  9. Jeff KingJul 3, 2010
  10. tytso@mit.eduJul 4, 2010
  11. Jeff KingJul 5, 2010
  12. 1/4 tag: speed up --contains calculationJeff King, Jul 5, 2010
  13. Jonathan NiederOct 13, 2010
  14. Clemens BuchacherOct 13, 2010
  15. Ævar Arnfjörð BjarmasonFeb 23, 2011
  16. Jeff KingFeb 23, 2011
  17. 2/4 limit "contains" traversals based on commit timestampJeff King, Jul 5, 2010
  18. Jonathan NiederOct 13, 2010
  19. 3/4 default core.clockskew variable to one dayJeff King, Jul 5, 2010
  20. 4/4 name-rev: respect core.clockskewJeff King, Jul 5, 2010
  21. Jeff KingJul 5, 2010
  22. Jonathan NiederOct 14, 2010
  23. Clemens BuchacherOct 16, 2010
  24. Jeff KingOct 27, 2010
  25. Clemens BuchacherOct 28, 2010
  26. tytso@mit.eduJul 5, 2010
  27. Jeff KingJul 6, 2010
  28. Will PalmerJul 6, 2010
  29. tytso@mit.eduJul 6, 2010
  30. Jeff KingJul 8, 2010
  31. Will PalmerJul 8, 2010
  32. tytso@mit.eduJul 8, 2010
  33. Jeff KingJul 7, 2010
  34. Theodore TsoJul 8, 2010
  35. Jakub NarebskiJul 8, 2010
  36. Nicolas PitreJul 8, 2010
  37. Avery PennarunJul 8, 2010
  38. Nicolas PitreJul 8, 2010
  39. Jakub NarebskiJul 8, 2010
  40. Sverre RabbelierJul 8, 2010
  41. Nicolas PitreJul 8, 2010
  42. Nicolas PitreJul 8, 2010
  43. Jeff KingJul 8, 2010
  44. Johan HerlandJul 8, 2010
  45. Nicolas PitreJul 8, 2010
  46. Jeff KingJul 7, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.