git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 7/7] revision.c: refactor basic topo-order logic

From
Derrick Stolee <stolee@gmail.com>
Date
Sep 27, 2018, 17:57 UTC
Message-ID
<7b1640a0-4002-9e5f-ea06-4504a5cc9f43@gmail.com>
In-Reply-To
<020b2f50c5703e8291577b008fdfa567093c6eab.1537551564.git.gitgitgadget@gmail.com>
On 9/21/2018 1:39 PM, Derrick Stolee via GitGitGadget wrote:
Show 31 quoted lines
> From: Derrick Stolee <dstolee@microsoft.com>
>
> When running a command like 'git rev-list --topo-order HEAD',
> Git performed the following steps:
>
> 1. Run limit_list(), which parses all reachable commits,
>     adds them to a linked list, and distributes UNINTERESTING
>     flags. If all unprocessed commits are UNINTERESTING, then
>     it may terminate without walking all reachable commits.
>     This does not occur if we do not specify UNINTERESTING
>     commits.
>
> 2. Run sort_in_topological_order(), which is an implementation
>     of Kahn's algorithm. It first iterates through the entire
>     set of important commits and computes the in-degree of each
>     (plus one, as we use 'zero' as a special value here). Then,
>     we walk the commits in priority order, adding them to the
>     priority queue if and only if their in-degree is one. As
>     we remove commits from this priority queue, we decrement the
>     in-degree of their parents.
>
> 3. While we are peeling commits for output, get_revision_1()
>     uses pop_commit on the full list of commits computed by
>     sort_in_topological_order().
>
> In the new algorithm, these three steps correspond to three
> different commit walks. We run these walks simultaneously,
> and advance each only as far as necessary to satisfy the
> requirements of the 'higher order' walk. We know when we can
> pause each walk by using generation numbers from the commit-
> graph feature.
Hello, Git contributors.

I understand that this commit message and patch are pretty daunting. There is a lot to read and digest. I would like to see if anyone is willing to put the work in to review this patch, as I quite like what it does, and the performance numbers below.

Show 16 quoted lines
> In my local testing, I used the following Git commands on the
> Linux repository in three modes: HEAD~1 with no commit-graph,
> HEAD~1 with a commit-graph, and HEAD with a commit-graph. This
> allows comparing the benefits we get from parsing commits from
> the commit-graph and then again the benefits we get by
> restricting the set of commits we walk.
>
> Test: git rev-list --topo-order -100 HEAD
> HEAD~1, no commit-graph: 6.80 s
> HEAD~1, w/ commit-graph: 0.77 s
>    HEAD, w/ commit-graph: 0.02 s
>
> Test: git rev-list --topo-order -100 HEAD -- tools
> HEAD~1, no commit-graph: 9.63 s
> HEAD~1, w/ commit-graph: 6.06 s
>    HEAD, w/ commit-graph: 0.06 s

If there is something I can do to make this easier to review, then please let me know.

Thanks, -Stolee

Previous: Derrick Stolee via GitGitGadgetNext: Jakub Narebski
Message 42 of 87 in “Use generation numbers for --topo-order”
  1. 0/6 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Aug 27, 2018
  2. 1/6 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Aug 27, 2018
  3. 2/6 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Aug 27, 2018
  4. 3/6 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Aug 27, 2018
  5. 4/6 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Aug 27, 2018
  6. 5/6 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Aug 27, 2018
  7. 6/6 revision.c: refactor basic topo-order logicDerrick Stolee via GitGitGadget, Aug 27, 2018
  8. Junio C HamanoAug 27, 2018
  9. 0/6 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Sep 18, 2018
  10. 1/6 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Sep 18, 2018
  11. 2/6 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Sep 18, 2018
  12. SZEDER GáborSep 18, 2018
  13. Junio C HamanoSep 19, 2018
  14. Junio C HamanoSep 19, 2018
  15. Junio C HamanoSep 20, 2018
  16. 3/6 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Sep 18, 2018
  17. 4/6 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Sep 18, 2018
  18. 5/6 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Sep 18, 2018
  19. 6/6 revision.c: refactor basic topo-order logicDerrick Stolee via GitGitGadget, Sep 18, 2018
  20. Ævar Arnfjörð BjarmasonSep 18, 2018
  21. Ævar Arnfjörð BjarmasonSep 18, 2018
  22. Derrick StoleeSep 21, 2018
  23. 0/7 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Sep 21, 2018
  24. 1/7 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Sep 21, 2018
  25. Derrick StoleeSep 26, 2018
  26. Jeff KingOct 11, 2018
  27. 2/7 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Sep 21, 2018
  28. Jeff KingOct 11, 2018
  29. 3/7 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Sep 21, 2018
  30. Jeff KingOct 11, 2018
  31. Junio C HamanoOct 12, 2018
  32. 4/7 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Sep 21, 2018
  33. Jeff KingOct 11, 2018
  34. Junio C HamanoOct 12, 2018
  35. Derrick StoleeOct 12, 2018
  36. Johannes SixtOct 12, 2018
  37. Junio C HamanoOct 13, 2018
  38. 5/7 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Sep 21, 2018
  39. Jeff KingOct 11, 2018
  40. 6/7 revision.h: add whitespace in flag definitionsDerrick Stolee via GitGitGadget, Sep 21, 2018
  41. 7/7 revision.c: refactor basic topo-order logicDerrick Stolee via GitGitGadget, Sep 21, 2018
  42. Derrick StoleeSep 27, 2018
  43. Jakub NarebskiOct 6, 2018
  44. Jeff KingOct 11, 2018
  45. Derrick StoleeOct 11, 2018
  46. Jeff KingOct 25, 2018
  47. Derrick StoleeOct 25, 2018
  48. Stefan BellerOct 11, 2018
  49. Junio C HamanoSep 21, 2018
  50. 0/7 Use generation numbers for --topo-orderDerrick Stolee via GitGitGadget, Oct 16, 2018
  51. 1/7 prio-queue: add 'peek' operationDerrick Stolee via GitGitGadget, Oct 16, 2018
  52. 3/7 test-reach: add rev-list testsDerrick Stolee via GitGitGadget, Oct 16, 2018
  53. Jakub NarebskiOct 21, 2018
  54. Derrick StoleeOct 21, 2018
  55. 4/7 revision.c: begin refactoring --topo-order logicDerrick Stolee via GitGitGadget, Oct 16, 2018
  56. Jakub NarebskiOct 21, 2018
  57. Junio C HamanoOct 22, 2018
  58. Derrick StoleeOct 22, 2018
  59. revision.c: use new algorithm in A..B caseDerrick Stolee, Oct 22, 2018
  60. Junio C HamanoOct 25, 2018
  61. Jakub NarebskiOct 26, 2018
  62. 2/7 test-reach: add run_three_modes methodDerrick Stolee via GitGitGadget, Oct 16, 2018
  63. 5/7 commit/revisions: bookkeeping before refactoringDerrick Stolee via GitGitGadget, Oct 16, 2018
  64. Jakub NarebskiOct 21, 2018
  65. 6/7 revision.c: generation-based topo-order algorithmDerrick Stolee via GitGitGadget, Oct 16, 2018
  66. Jakub NarebskiOct 22, 2018
  67. Derrick StoleeOct 23, 2018
  68. Jakub NarebskiOct 26, 2018
  69. 7/7 t6012: make rev-list tests more interestingDerrick Stolee via GitGitGadget, Oct 16, 2018
  70. Jakub NarebskiOct 23, 2018
  71. Jakub NarebskiOct 21, 2018
  72. Junio C HamanoNov 1, 2018
  73. Derrick StoleeNov 1, 2018
  74. Junio C HamanoNov 1, 2018
  75. 0/7 Use generation numbers for --topo-orderDerrick Stolee, Nov 1, 2018
  76. 1/7 prio-queue: add 'peek' operationDerrick Stolee, Nov 1, 2018
  77. 2/7 test-reach: add run_three_modes methodDerrick Stolee, Nov 1, 2018
  78. 3/7 test-reach: add rev-list testsDerrick Stolee, Nov 1, 2018
  79. 4/7 revision.c: begin refactoring --topo-order logicDerrick Stolee, Nov 1, 2018
  80. 5/7 commit/revisions: bookkeeping before refactoringDerrick Stolee, Nov 1, 2018
  81. 6/7 revision.c: generation-based topo-order algorithmDerrick Stolee, Nov 1, 2018
  82. SZEDER GáborNov 1, 2018
  83. Derrick StoleeNov 1, 2018
  84. Mike HommeyNov 8, 2019
  85. Derrick StoleeNov 11, 2019
  86. SZEDER GáborNov 18, 2019
  87. 7/7 t6012: make rev-list tests more interestingDerrick Stolee, Nov 1, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.