git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] commit-graph: split up close_reachable() progress output

From
SZEDER Gábor <szeder.dev@gmail.com>
Date
Nov 19, 2018, 22:57 UTC
Message-ID
<20181119225720.GV30222@szeder.dev>
In-Reply-To
<20181119202300.18670-1-avarab@gmail.com>
On Mon, Nov 19, 2018 at 08:23:00PM +0000, Ævar Arnfjörð Bjarmason wrote:
> Amend the progress output added in 7b0f229222 ("commit-graph write:
> add progress output", 2018-09-17) so that the total numbers it reports
> aren't higher than the total number of commits anymore. See [1] for a
> bug report pointing that out.

Please make the commit message more self-contained, i.e. describe the issue this patch fixes in more detail, so readers won't have to follow links to understand the problem.

Show 12 quoted lines
> When I added this I wasn't intending to provide an accurate count, but
> just have some progress output to show the user the command wasn't
> hanging[2]. But since we are showing numbers, let's make them
> accurate. The progress descriptions were suggested by Derrick Stolee
> in [3].
> 
> As noted in [2] we are unlikely to show anything except the "Expanding
> reachable..." message even on fairly large repositories such as
> linux.git. On a test repository I have with north of 7 million commits
> all of these are displayed. Two of them don't show up for long, but as
> noted in [5] future-proofing this for if the loops become more
> expensive in the future makes sense.

In my opinion this is rather one of those "we'll cross that bridge when (or if ever) we get there" situations.

Show 22 quoted lines
> 1. https://public-inbox.org/git/20181010203738.GE23446@szeder.dev/
> 2. https://public-inbox.org/git/87pnwhea8y.fsf@evledraar.gmail.com/
> 3. https://public-inbox.org/git/f7a0cbee-863c-61d3-4959-5cec8b43c705@gmail.com/
> 4. https://public-inbox.org/git/20181015160545.GG19800@szeder.dev/
> 5. https://public-inbox.org/git/87murle8da.fsf@evledraar.gmail.com/
> 
> Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
> Helped-by: Derrick Stolee <stolee@gmail.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> 
> On Mon, Nov 19 2018, SZEDER Gábor wrote:
> 
> > Ping?
> >
> > We are at -rc0, this progress output is a new feature since v2.19.0,
> > and the numbers shown are still way off.
> 
> I was under the impression after your
> https://public-inbox.org/git/20181015160545.GG19800@szeder.dev/ that
> you were going to do some more digging & report back, so I put it on
> my "waiting for feedback" list and then forgot about it.

No, after I managed to "get my numbers straight" I sent another bug report in

  https://public-inbox.org/git/20181015165447.GH19800@szeder.dev/
but as a reply to your original patch.  Sorry about the confusion.
> But here's a patch that should address the issue you pointed out, but
> I don't know if it fixes whatever you were alluding to in the linked
> E-Mail above.

I'm afraid this patch doesn't address that issue, as it's limited to close_reachable(), and that issue is related to the progress output in add_packed_commits().

Show 55 quoted lines
>  commit-graph.c | 13 ++++++++++---
>  1 file changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/commit-graph.c b/commit-graph.c
> index 40c855f185..9c0d6914be 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -641,26 +641,29 @@ static void add_missing_parents(struct packed_oid_list *oids, struct commit *com
>  
>  static void close_reachable(struct packed_oid_list *oids, int report_progress)
>  {
> -	int i;
> +	int i, j;
>  	struct commit *commit;
>  	struct progress *progress = NULL;
> -	int j = 0;
>  
>  	if (report_progress)
>  		progress = start_delayed_progress(
> -			_("Annotating commits in commit graph"), 0);
> +			_("Loading known commits in commit graph"), j = 0);
>  	for (i = 0; i < oids->nr; i++) {
>  		display_progress(progress, ++j);
>  		commit = lookup_commit(the_repository, &oids->list[i]);
>  		if (commit)
>  			commit->object.flags |= UNINTERESTING;
>  	}
> +	stop_progress(&progress);
>  
>  	/*
>  	 * As this loop runs, oids->nr may grow, but not more
>  	 * than the number of missing commits in the reachable
>  	 * closure.
>  	 */
> +	if (report_progress)
> +		progress = start_delayed_progress(
> +			_("Expanding reachable commits in commit graph"), j = 0);
>  	for (i = 0; i < oids->nr; i++) {
>  		display_progress(progress, ++j);
>  		commit = lookup_commit(the_repository, &oids->list[i]);
> @@ -668,7 +671,11 @@ static void close_reachable(struct packed_oid_list *oids, int report_progress)
>  		if (commit && !parse_commit(commit))
>  			add_missing_parents(oids, commit);
>  	}
> +	stop_progress(&progress);
>  
> +	if (report_progress)
> +		progress = start_delayed_progress(
> +			_("Clearing commit marks in commit graph"), j = 0);
>  	for (i = 0; i < oids->nr; i++) {
>  		display_progress(progress, ++j);
>  		commit = lookup_commit(the_repository, &oids->list[i]);
> -- 
> 2.19.1.1182.g4ecb1133ce
> 
Previous: Derrick StoleeNext: Ævar Arnfjörð Bjarmason
Message 42 of 133 in “commit-graph: add progress output”
  1. 0/2 commit-graph: add progress outputÆvar Arnfjörð Bjarmason, Sep 4, 2018
  2. 2/2 commit-graph verify: add progress outputÆvar Arnfjörð Bjarmason, Sep 4, 2018
  3. Junio C HamanoSep 4, 2018
  4. 1/2 commit-graph write: add progress outputÆvar Arnfjörð Bjarmason, Sep 4, 2018
  5. Eric SunshineSep 4, 2018
  6. Junio C HamanoSep 4, 2018
  7. Derrick StoleeSep 5, 2018
  8. Ævar Arnfjörð BjarmasonSep 5, 2018
  9. Junio C HamanoSep 5, 2018
  10. Derrick StoleeSep 5, 2018
  11. Ævar Arnfjörð BjarmasonSep 7, 2018
  12. Ævar Arnfjörð BjarmasonSep 7, 2018
  13. Jeff KingSep 7, 2018
  14. Derrick StoleeSep 7, 2018
  15. Derrick StoleeSep 5, 2018
  16. Ævar Arnfjörð BjarmasonSep 7, 2018
  17. Derrick StoleeSep 7, 2018
  18. Derrick StoleeSep 5, 2018
  19. 0/2 commit-graph: add progress outputÆvar Arnfjörð Bjarmason, Sep 7, 2018
  20. Junio C HamanoSep 11, 2018
  21. 1/2 commit-graph write: add progress outputÆvar Arnfjörð Bjarmason, Sep 7, 2018
  22. Derrick StoleeSep 21, 2018
  23. Junio C HamanoSep 21, 2018
  24. Junio C HamanoSep 21, 2018
  25. 2/2 commit-graph verify: add progress outputÆvar Arnfjörð Bjarmason, Sep 7, 2018
  26. Duy NguyenSep 16, 2018
  27. 0/2 commit-graph: add progress outputÆvar Arnfjörð Bjarmason, Sep 17, 2018
  28. 1/2 commit-graph write: add progress outputÆvar Arnfjörð Bjarmason, Sep 17, 2018
  29. SZEDER GáborOct 10, 2018
  30. Ævar Arnfjörð BjarmasonOct 10, 2018
  31. SZEDER GáborOct 10, 2018
  32. Ævar Arnfjörð BjarmasonOct 10, 2018
  33. Ævar Arnfjörð BjarmasonOct 11, 2018
  34. SZEDER GáborOct 15, 2018
  35. Junio C HamanoOct 12, 2018
  36. Ævar Arnfjörð BjarmasonOct 12, 2018
  37. Derrick StoleeOct 12, 2018
  38. SZEDER GáborOct 15, 2018
  39. SZEDER GáborNov 19, 2018
  40. commit-graph: split up close_reachable() progress outputÆvar Arnfjörð Bjarmason, Nov 19, 2018
  41. Derrick StoleeNov 19, 2018
  42. SZEDER GáborNov 19, 2018
  43. 0/6 commit-graph write: progress output improvementsÆvar Arnfjörð Bjarmason, Nov 20, 2018
  44. 1/6 commit-graph write: rephrase confusing progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  45. 2/6 commit-graph write: add more progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  46. SZEDER GáborNov 20, 2018
  47. 0/6 commit-graph write: progress output improvementsÆvar Arnfjörð Bjarmason, Nov 20, 2018
  48. 1/6 commit-graph write: rephrase confusing progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  49. 2/6 commit-graph write: add more progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  50. SZEDER GáborNov 20, 2018
  51. 3/6 commit-graph write: show progress for object searchÆvar Arnfjörð Bjarmason, Nov 20, 2018
  52. 4/6 commit-graph write: add more describing progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  53. 5/6 commit-graph write: remove empty line for readabilityÆvar Arnfjörð Bjarmason, Nov 20, 2018
  54. 6/6 commit-graph write: add even more progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  55. SZEDER Gábor, Nov 21, 2018
  56. 1/2 commit-graph: rename 'num_extra_edges' variable to 'num_large_edges'SZEDER Gábor, Nov 21, 2018
  57. Junio C HamanoNov 21, 2018
  58. Derrick StoleeNov 21, 2018
  59. 0/2 commit-graph: minor cleanup and optimizationSZEDER Gábor, Jan 18, 2019
  60. 1/2 commit-graph: rename "large edges" to "extra edges"SZEDER Gábor, Jan 18, 2019
  61. 2/2 commit-graph: don't call write_graph_chunk_large_edges() unnecessarilySZEDER Gábor, Jan 18, 2019
  62. Martin ÅgrenJan 19, 2019
  63. Junio C HamanoJan 18, 2019
  64. 2/2 commit-graph: don't call write_graph_chunk_large_edges() unnecessarilySZEDER Gábor, Nov 21, 2018
  65. Derrick StoleeNov 21, 2018
  66. 00/10 commit-graph write: progress output improvementsÆvar Arnfjörð Bjarmason, Nov 22, 2018
  67. 00/10 commit-graph write: progress output improvementsÆvar Arnfjörð Bjarmason, Nov 22, 2018
  68. 01/10 commit-graph: rename 'num_extra_edges' variable to 'num_large_edges'Ævar Arnfjörð Bjarmason, Nov 22, 2018
  69. 0/9 commit-graph write: progress output improvementsÆvar Arnfjörð Bjarmason, Jan 16, 2019
  70. Junio C HamanoJan 19, 2019
  71. 00/10 commit-graph write: progress output improvementsÆvar Arnfjörð Bjarmason, Jan 19, 2019
  72. Derrick StoleeJan 22, 2019
  73. Junio C HamanoJan 23, 2019
  74. Junio C HamanoJan 22, 2019
  75. 01/10 commit-graph write: use pack order when finding commitsÆvar Arnfjörð Bjarmason, Jan 19, 2019
  76. 02/10 commit-graph: rename "large edges" to "extra edges"Ævar Arnfjörð Bjarmason, Jan 19, 2019
  77. 03/10 commit-graph: don't call write_graph_chunk_large_edges() unnecessarilyÆvar Arnfjörð Bjarmason, Jan 19, 2019
  78. 03/10 commit-graph: don't call write_graph_chunk_extra_edges() unnecessarilySZEDER Gábor, Jan 23, 2019
  79. 04/10 commit-graph write: add "Writing out" progress outputÆvar Arnfjörð Bjarmason, Jan 19, 2019
  80. 05/10 commit-graph write: more descriptive "writing out" outputÆvar Arnfjörð Bjarmason, Jan 19, 2019
  81. 06/10 commit-graph write: show progress for object searchÆvar Arnfjörð Bjarmason, Jan 19, 2019
  82. 08/10 commit-graph write: remove empty line for readabilityÆvar Arnfjörð Bjarmason, Jan 19, 2019
  83. 07/10 commit-graph write: add more descriptive progress outputÆvar Arnfjörð Bjarmason, Jan 19, 2019
  84. 09/10 commit-graph write: add itermediate progressÆvar Arnfjörð Bjarmason, Jan 19, 2019
  85. 10/10 commit-graph write: emit a percentage for all progressÆvar Arnfjörð Bjarmason, Jan 19, 2019
  86. commit-graph: don't show progress percentages while expanding reachable commitsSZEDER Gábor, Mar 22, 2019
  87. Ævar Arnfjörð BjarmasonMar 22, 2019
  88. SZEDER GáborMar 22, 2019
  89. Ævar Arnfjörð BjarmasonMar 22, 2019
  90. Ævar Arnfjörð BjarmasonMar 22, 2019
  91. SZEDER GáborMar 22, 2019
  92. Ævar Arnfjörð BjarmasonMar 22, 2019
  93. SZEDER GáborMar 22, 2019
  94. SZEDER GáborMar 22, 2019
  95. Ævar Arnfjörð BjarmasonMar 22, 2019
  96. 1/9 commit-graph: rename 'num_extra_edges' variable to 'num_large_edges'Ævar Arnfjörð Bjarmason, Jan 16, 2019
  97. 2/9 commit-graph: don't call write_graph_chunk_large_edges() unnecessarilyÆvar Arnfjörð Bjarmason, Jan 16, 2019
  98. 3/9 commit-graph write: add "Writing out" progress outputÆvar Arnfjörð Bjarmason, Jan 16, 2019
  99. SZEDER GáborJan 18, 2019
  100. 4/9 commit-graph write: more descriptive "writing out" outputÆvar Arnfjörð Bjarmason, Jan 16, 2019
  101. 5/9 commit-graph write: show progress for object searchÆvar Arnfjörð Bjarmason, Jan 16, 2019
  102. 6/9 commit-graph write: add more descriptive progress outputÆvar Arnfjörð Bjarmason, Jan 16, 2019
  103. 7/9 commit-graph write: remove empty line for readabilityÆvar Arnfjörð Bjarmason, Jan 16, 2019
  104. 8/9 commit-graph write: add itermediate progressÆvar Arnfjörð Bjarmason, Jan 16, 2019
  105. 9/9 commit-graph write: emit a percentage for all progressÆvar Arnfjörð Bjarmason, Jan 16, 2019
  106. commit-graph write: use pack order when finding commitsÆvar Arnfjörð Bjarmason, Jan 17, 2019
  107. Derrick StoleeJan 17, 2019
  108. Derrick StoleeJan 17, 2019
  109. 02/10 commit-graph: don't call write_graph_chunk_large_edges() unnecessarilyÆvar Arnfjörð Bjarmason, Nov 22, 2018
  110. 03/10 commit-graph write: rephrase confusing progress outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  111. 05/10 commit-graph write: more descriptive "writing out" outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  112. 04/10 commit-graph write: add "Writing out" progress outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  113. 06/10 commit-graph write: show progress for object searchÆvar Arnfjörð Bjarmason, Nov 22, 2018
  114. 07/10 commit-graph write: add more descriptive progress outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  115. 08/10 commit-graph write: remove empty line for readabilityÆvar Arnfjörð Bjarmason, Nov 22, 2018
  116. 09/10 commit-graph write: add itermediate progressÆvar Arnfjörð Bjarmason, Nov 22, 2018
  117. 10/10 commit-graph write: emit a percentage for all progressÆvar Arnfjörð Bjarmason, Nov 22, 2018
  118. Eric SunshineNov 22, 2018
  119. 01/10 commit-graph: rename 'num_extra_edges' variable to 'num_large_edges'Ævar Arnfjörð Bjarmason, Nov 22, 2018
  120. 03/10 commit-graph write: rephrase confusing progress outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  121. 02/10 commit-graph: don't call write_graph_chunk_large_edges() unnecessarilyÆvar Arnfjörð Bjarmason, Nov 22, 2018
  122. 04/10 commit-graph write: add "Writing out" progress outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  123. 05/10 commit-graph write: more descriptive "writing out" outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  124. 06/10 commit-graph write: show progress for object searchÆvar Arnfjörð Bjarmason, Nov 22, 2018
  125. 07/10 commit-graph write: add more descriptive progress outputÆvar Arnfjörð Bjarmason, Nov 22, 2018
  126. 08/10 commit-graph write: remove empty line for readabilityÆvar Arnfjörð Bjarmason, Nov 22, 2018
  127. 09/10 commit-graph write: add itermediate progressÆvar Arnfjörð Bjarmason, Nov 22, 2018
  128. 10/10 commit-graph write: emit a percentage for all progressÆvar Arnfjörð Bjarmason, Nov 22, 2018
  129. 3/6 commit-graph write: show progress for object searchÆvar Arnfjörð Bjarmason, Nov 20, 2018
  130. 4/6 commit-graph write: add more describing progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  131. 5/6 commit-graph write: remove empty line for readabilityÆvar Arnfjörð Bjarmason, Nov 20, 2018
  132. 6/6 commit-graph write: add even more progress outputÆvar Arnfjörð Bjarmason, Nov 20, 2018
  133. 2/2 commit-graph verify: add progress outputÆvar Arnfjörð Bjarmason, Sep 17, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.