git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 4/6] revision.c: begin refactoring --topo-order logic

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Sep 18, 2018, 04:08 UTC
Message-ID
<a74ae13d4e207acec20b53bf48f24a2de192ad97.1537243720.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.25.v2.git.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

When running 'git rev-list --topo-order' and its kin, the topo_order setting in struct rev_info implies the limited setting. This means that the following things happen during prepare_revision_walk():

* revs->limited implies we run limit_list() to walk the entire
  reachable set. There are some short-cuts here, such as if we
  perform a range query like 'git rev-list COMPARE..HEAD' and we
  can stop limit_list() when all queued commits are uninteresting.
* revs->topo_order implies we run sort_in_topological_order(). See
  the implementation of that method in commit.c. It implies that
  the full set of commits to order is in the given commit_list.

These two methods imply that a 'git rev-list --topo-order HEAD' command must walk the entire reachable set of commits _twice_ before returning a single result.

If we have a commit-graph file with generation numbers computed, then there is a better way. This patch introduces some necessary logic redirection when we are in this situation.

In v2.18.0, the commit-graph file contains zero-valued bytes in the positions where the generation number is stored in v2.19.0 and later. Thus, we use generation_numbers_enabled() to check if the commit-graph is available and has non-zero generation numbers.

When setting revs->limited only because revs->topo_order is true, only do so if generation numbers are not available. There is no reason to use the new logic as it will behave similarly when all generation numbers are INFINITY or ZERO.

In prepare_revision_walk(), if we have revs->topo_order but not revs->limited, then we trigger the new logic. It breaks the logic into three pieces, to fit with the existing framework:

1. init_topo_walk() fills a new struct topo_walk_info in the rev_info
   struct. We use the presence of this struct as a signal to use the
   new methods during our walk. In this patch, this method simply
   calls limit_list() and sort_in_topological_order(). In the future,
   this method will set up a new data structure to perform that logic
   in-line.
2. next_topo_commit() provides get_revision_1() with the next topo-
   ordered commit in the list. Currently, this simply pops the commit
   from revs->commits.
3. expand_topo_walk() provides get_revision_1() with a way to signal
   walking beyond the latest commit. Currently, this calls
   add_parents_to_list() exactly like the old logic.

While this commit presents method redirection for performing the exact same logic as before, it allows the next commit to focus only on the new logic.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 revision.c | 42 ++++++++++++++++++++++++++++++++++++++----
 revision.h |  4 ++++
 2 files changed, 42 insertions(+), 4 deletions(-)
diff --git a/revision.c b/revision.c
index e18bd530e4..2dcde8a8ac 100644
--- a/revision.c
+++ b/revision.c
@@ -25,6 +25,7 @@
 #include "worktree.h"
 #include "argv-array.h"
 #include "commit-reach.h"
+#include "commit-graph.h"
 
 volatile show_early_output_fn_t show_early_output;
 
@@ -2454,7 +2455,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 	if (revs->diffopt.objfind)
 		revs->simplify_history = 0;
 
-	if (revs->topo_order)
+	if (revs->topo_order && !generation_numbers_enabled(the_repository))
 		revs->limited = 1;
 
 	if (revs->prune_data.nr) {
@@ -2892,6 +2893,33 @@ static int mark_uninteresting(const struct object_id *oid,
 	return 0;
 }
 
+struct topo_walk_info {};
+
+static void init_topo_walk(struct rev_info *revs)
+{
+	struct topo_walk_info *info;
+	revs->topo_walk_info = xmalloc(sizeof(struct topo_walk_info));
+	info = revs->topo_walk_info;
+	memset(info, 0, sizeof(struct topo_walk_info));
+
+	limit_list(revs);
+	sort_in_topological_order(&revs->commits, revs->sort_order);
+}
+
+static struct commit *next_topo_commit(struct rev_info *revs)
+{
+	return pop_commit(&revs->commits);
+}
+
+static void expand_topo_walk(struct rev_info *revs, struct commit *commit)
+{
+	if (add_parents_to_list(revs, commit, &revs->commits, NULL) < 0) {
+		if (!revs->ignore_missing_links)
+			die("Failed to traverse parents of commit %s",
+			    oid_to_hex(&commit->object.oid));
+	}
+}
+
 int prepare_revision_walk(struct rev_info *revs)
 {
 	int i;
@@ -2928,11 +2956,13 @@ int prepare_revision_walk(struct rev_info *revs)
 		commit_list_sort_by_date(&revs->commits);
 	if (revs->no_walk)
 		return 0;
-	if (revs->limited)
+	if (revs->limited) {
 		if (limit_list(revs) < 0)
 			return -1;
-	if (revs->topo_order)
-		sort_in_topological_order(&revs->commits, revs->sort_order);
+		if (revs->topo_order)
+			sort_in_topological_order(&revs->commits, revs->sort_order);
+	} else if (revs->topo_order)
+		init_topo_walk(revs);
 	if (revs->line_level_traverse)
 		line_log_filter(revs);
 	if (revs->simplify_merges)
@@ -3257,6 +3287,8 @@ static struct commit *get_revision_1(struct rev_info *revs)
 
 		if (revs->reflog_info)
 			commit = next_reflog_entry(revs->reflog_info);
+		else if (revs->topo_walk_info)
+			commit = next_topo_commit(revs);
 		else
 			commit = pop_commit(&revs->commits);
 
@@ -3278,6 +3310,8 @@ static struct commit *get_revision_1(struct rev_info *revs)
 
 			if (revs->reflog_info)
 				try_to_simplify_commit(revs, commit);
+			else if (revs->topo_walk_info)
+				expand_topo_walk(revs, commit);
 			else if (add_parents_to_list(revs, commit, &revs->commits, NULL) < 0) {
 				if (!revs->ignore_missing_links)
 					die("Failed to traverse parents of commit %s",
diff --git a/revision.h b/revision.h
index 2b30ac270d..fd4154ff75 100644
--- a/revision.h
+++ b/revision.h
@@ -56,6 +56,8 @@ struct rev_cmdline_info {
 #define REVISION_WALK_NO_WALK_SORTED 1
 #define REVISION_WALK_NO_WALK_UNSORTED 2
 
+struct topo_walk_info;
+
 struct rev_info {
 	/* Starting list */
 	struct commit_list *commits;
@@ -245,6 +247,8 @@ struct rev_info {
 	const char *break_bar;
 
 	struct revision_sources *sources;
+
+	struct topo_walk_info *topo_walk_info;
 };
 
 int ref_excluded(struct string_list *, const char *path);
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 17 of 87 in “Use generation numbers for --topo-order”
  1. 0/6 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Aug 27, 2018
  2. 1/6 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Aug 27, 2018
  3. 2/6 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Aug 27, 2018
  4. 3/6 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Aug 27, 2018
  5. 4/6 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Aug 27, 2018
  6. 5/6 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Aug 27, 2018
  7. 6/6 revision.c: refactor basic topo-order logicDerrick Stolee via GitGitGadget, Aug 27, 2018
  8. Junio C HamanoAug 27, 2018
  9. 0/6 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Sep 18, 2018
  10. 1/6 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Sep 18, 2018
  11. 2/6 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Sep 18, 2018
  12. SZEDER GáborSep 18, 2018
  13. Junio C HamanoSep 19, 2018
  14. Junio C HamanoSep 19, 2018
  15. Junio C HamanoSep 20, 2018
  16. 3/6 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Sep 18, 2018
  17. 4/6 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Sep 18, 2018
  18. 5/6 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Sep 18, 2018
  19. 6/6 revision.c: refactor basic topo-order logicDerrick Stolee via GitGitGadget, Sep 18, 2018
  20. Ævar Arnfjörð BjarmasonSep 18, 2018
  21. Ævar Arnfjörð BjarmasonSep 18, 2018
  22. Derrick StoleeSep 21, 2018
  23. 0/7 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Sep 21, 2018
  24. 1/7 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Sep 21, 2018
  25. Derrick StoleeSep 26, 2018
  26. Jeff KingOct 11, 2018
  27. 2/7 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Sep 21, 2018
  28. Jeff KingOct 11, 2018
  29. 3/7 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Sep 21, 2018
  30. Jeff KingOct 11, 2018
  31. Junio C HamanoOct 12, 2018
  32. 4/7 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Sep 21, 2018
  33. Jeff KingOct 11, 2018
  34. Junio C HamanoOct 12, 2018
  35. Derrick StoleeOct 12, 2018
  36. Johannes SixtOct 12, 2018
  37. Junio C HamanoOct 13, 2018
  38. 5/7 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Sep 21, 2018
  39. Jeff KingOct 11, 2018
  40. 6/7 revision.h: add whitespace in flag definitionsDerrick Stolee via GitGitGadget, Sep 21, 2018
  41. 7/7 revision.c: refactor basic topo-order logicDerrick Stolee via GitGitGadget, Sep 21, 2018
  42. Derrick StoleeSep 27, 2018
  43. Jakub NarebskiOct 6, 2018
  44. Jeff KingOct 11, 2018
  45. Derrick StoleeOct 11, 2018
  46. Jeff KingOct 25, 2018
  47. Derrick StoleeOct 25, 2018
  48. Stefan BellerOct 11, 2018
  49. Junio C HamanoSep 21, 2018
  50. 0/7 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Oct 16, 2018
  51. 1/7 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Oct 16, 2018
  52. 3/7 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Oct 16, 2018
  53. Jakub NarebskiOct 21, 2018
  54. Derrick StoleeOct 21, 2018
  55. 4/7 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Oct 16, 2018
  56. Jakub NarebskiOct 21, 2018
  57. Junio C HamanoOct 22, 2018
  58. Derrick StoleeOct 22, 2018
  59. revision.c: use new algorithm in A..B caseDerrick Stolee, Oct 22, 2018
  60. Junio C HamanoOct 25, 2018
  61. Jakub NarebskiOct 26, 2018
  62. 2/7 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Oct 16, 2018
  63. 5/7 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Oct 16, 2018
  64. Jakub NarebskiOct 21, 2018
  65. 6/7 revision.c: generation-based topo-order algorithmDerrick Stolee via GitGitGadget, Oct 16, 2018
  66. Jakub NarebskiOct 22, 2018
  67. Derrick StoleeOct 23, 2018
  68. Jakub NarebskiOct 26, 2018
  69. 7/7 t6012: make rev-list tests more interestingDerrick Stolee via GitGitGadget, Oct 16, 2018
  70. Jakub NarebskiOct 23, 2018
  71. Jakub NarebskiOct 21, 2018
  72. Junio C HamanoNov 1, 2018
  73. Derrick StoleeNov 1, 2018
  74. Junio C HamanoNov 1, 2018
  75. 0/7 Use generation numbers for --topo-orderDerrick Stolee, Nov 1, 2018
  76. 1/7 prio-queue: add 'peek' operationDerrick Stolee, Nov 1, 2018
  77. 2/7 test-reach: add run_three_modes methodDerrick Stolee, Nov 1, 2018
  78. 3/7 test-reach: add rev-list testsDerrick Stolee, Nov 1, 2018
  79. 4/7 revision.c: begin refactoring --topo-order logicDerrick Stolee, Nov 1, 2018
  80. 5/7 commit/revisions: bookkeeping before refactoringDerrick Stolee, Nov 1, 2018
  81. 6/7 revision.c: generation-based topo-order algorithmDerrick Stolee, Nov 1, 2018
  82. SZEDER GáborNov 1, 2018
  83. Derrick StoleeNov 1, 2018
  84. Mike HommeyNov 8, 2019
  85. Derrick StoleeNov 11, 2019
  86. SZEDER GáborNov 18, 2019
  87. 7/7 t6012: make rev-list tests more interestingDerrick Stolee, Nov 1, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.