git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 12/12] receive-pack: send keepalives during quiet periods

From
Jeff King <peff@peff.net>
Date
Jul 20, 2016, 13:28 UTC
Message-ID
<20160720132809.GA18980@sigill.intra.peff.net>
In-Reply-To
<CAGZ79kZ7m94T5+SHjeWcY_cDLYSP9X74eDOOjfmZ1GwAb-C-7g@mail.gmail.com>
On Tue, Jul 19, 2016 at 09:05:14AM -0700, Stefan Beller wrote:
Show 10 quoted lines
> > I know that writing the function myself and then critiquing is the very
> > definition of a strawman. :) But it's the best I could think of.  Maybe
> > you had something more clever in mind?
> 
> Actually no, I had not. I was hoping you could come up with a clever thing.
> My original point was the perceived added complexity to a simple
> seemingly simple function (copy_to_sideband in your original patch),
> so my gut reaction was to shove the complexity away into a helper function.
> But no matter how it is done, there is always this one function that looks
> complex for this problem. So I think your original approach is fine then?

Yeah, I agree the original was nicer, but IMHO none of the "like this" alternatives I showed in this thread are real improvements. So I'm inclined to go with what was originally posted (though I'm open to more input or suggestions).

The only avenue I didn't look at is actually ditching the async muxer completely for the index-pack call, and replacing it with the main thread doing a poll() on stdout/stderr from index-pack. That's probably going to end up as _more_ code, but it potentially makes the "report magic NUL" a little less gross (we could signal with something more sane on stdout). I can explore that if people want me to.

If not, then the only change to the original series is adding "static" to the rev-list progress variables that Junio noticed. A replacement for patch 2 is below in case that makes things easier.

-- >8 --
Subject: [PATCH] rev-list: add optional progress reporting

It's easy to ask rev-list to do a traversal that may takes many seconds (e.g., by calling "--objects --all"). In theory you can monitor its progress by the output you get to stdout, but this isn't always easy.

Some operations, like "--count", don't make any output until the end.

And some callers, like check_everything_connected(), are using it just for the error-checking of the traversal, and throw away stdout entirely.

This patch adds a "--progress" option which can be used to give some eye-candy for a user waiting for a long traversal. This is just a rev-list option and not a regular traversal option, because it needs cooperation from the callbacks in builtin/rev-list.c to do the actual count.

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/rev-list-options.txt |  4 ++++
 builtin/rev-list.c                 | 17 +++++++++++++++++
 2 files changed, 21 insertions(+)
diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt
index c5bd218..f39cb6d 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -274,6 +274,10 @@ ifdef::git-rev-list[]
 	Try to speed up the traversal using the pack bitmap index (if
 	one is available). Note that when traversing with `--objects`,
 	trees and blobs will not have their associated path printed.
+
+--progress=<header>::
+	Show progress reports on stderr as objects are considered. The
+	`<header>` text will be printed with each progress update.
 endif::git-rev-list[]
 
 --
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index b82bcc3..0ba82b1 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -9,6 +9,7 @@
 #include "log-tree.h"
 #include "graph.h"
 #include "bisect.h"
+#include "progress.h"
 
 static const char rev_list_usage[] =
 "git rev-list [OPTION] <commit-id>... [ -- paths... ]\n"
@@ -49,12 +50,17 @@ static const char rev_list_usage[] =
 "    --bisect-all"
 ;
 
+static struct progress *progress;
+static unsigned progress_counter;
+
 static void finish_commit(struct commit *commit, void *data);
 static void show_commit(struct commit *commit, void *data)
 {
 	struct rev_list_info *info = data;
 	struct rev_info *revs = info->revs;
 
+	display_progress(progress, ++progress_counter);
+
 	if (info->flags & REV_LIST_QUIET) {
 		finish_commit(commit, data);
 		return;
@@ -190,6 +196,7 @@ static void show_object(struct object *obj, const char *name, void *cb_data)
 {
 	struct rev_list_info *info = cb_data;
 	finish_object(obj, name, cb_data);
+	display_progress(progress, ++progress_counter);
 	if (info->flags & REV_LIST_QUIET)
 		return;
 	show_object_with_name(stdout, obj, name);
@@ -276,6 +283,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	int bisect_show_vars = 0;
 	int bisect_find_all = 0;
 	int use_bitmap_index = 0;
+	const char *show_progress = NULL;
 
 	git_config(git_default_config, NULL);
 	init_revisions(&revs, prefix);
@@ -325,6 +333,10 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 			test_bitmap_walk(&revs);
 			return 0;
 		}
+		if (skip_prefix(arg, "--progress=", &arg)) {
+			show_progress = arg;
+			continue;
+		}
 		usage(rev_list_usage);
 
 	}
@@ -355,6 +367,9 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	if (bisect_list)
 		revs.limited = 1;
 
+	if (show_progress)
+		progress = start_progress_delay(show_progress, 0, 0, 2);
+
 	if (use_bitmap_index && !revs.prune) {
 		if (revs.count && !revs.left_right && !revs.cherry_mark) {
 			uint32_t commit_count;
@@ -392,6 +407,8 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 
 	traverse_commit_list(&revs, show_commit, show_object, &info);
 
+	stop_progress(&progress);
+
 	if (revs.count) {
 		if (revs.left_right && revs.cherry_mark)
 			printf("%d\t%d\t%d\n", revs.count_left, revs.count_right, revs.count_same);
-- 
2.9.2.506.gb3c8cac
Previous: Stefan BellerNext: Junio C Hamano
Message 25 of 26 in “push progress reporting and keepalives”
  1. 0/12 push progress reporting and keepalivesJeff King, Jul 15, 2016
  2. 01/12 check_everything_connected: always pass --quiet to rev-listJeff King, Jul 15, 2016
  3. 02/12 rev-list: add optional progress reportingJeff King, Jul 15, 2016
  4. Junio C HamanoJul 15, 2016
  5. Jeff KingJul 16, 2016
  6. 03/12 check_everything_connected: convert to argv_arrayJeff King, Jul 15, 2016
  7. 04/12 check_everything_connected: use a struct with named optionsJeff King, Jul 15, 2016
  8. Junio C HamanoJul 15, 2016
  9. Jeff KingJul 16, 2016
  10. 05/12 check_connected: relay errors to alternate descriptorJeff King, Jul 15, 2016
  11. Junio C HamanoJul 15, 2016
  12. Jeff KingJul 16, 2016
  13. 06/12 check_connected: add progress flagJeff King, Jul 15, 2016
  14. 07/12 clone: use a real progress meter for connectivity checkJeff King, Jul 15, 2016
  15. 08/12 index-pack: add flag for showing delta-resolution progressJeff King, Jul 15, 2016
  16. 09/12 receive-pack: turn on index-pack resolving progressJeff King, Jul 15, 2016
  17. 10/12 receive-pack: relay connectivity errors to sidebandJeff King, Jul 15, 2016
  18. 11/12 receive-pack: turn on connectivity progressJeff King, Jul 15, 2016
  19. 12/12 receive-pack: send keepalives during quiet periodsJeff King, Jul 15, 2016
  20. Stefan BellerJul 15, 2016
  21. Jeff KingJul 16, 2016
  22. Stefan BellerJul 19, 2016
  23. Jeff KingJul 19, 2016
  24. Stefan BellerJul 19, 2016
  25. Jeff KingJul 20, 2016
  26. Junio C HamanoJul 15, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.