threads / patch / 46077

patch, 2 partsAdd progress to format-patch and rebase

Subject: [PATCH 0/2] Add progress to format-patch and rebase

## tl;dr

34 messages between May 31, 2017 and Aug 20, 2017. Diffs are folded; open one to read it.

replies: 33people: 7as markdown or json

Kevin Willford· May 31, 2017, 15:04 UTC · lore

This patch series is to add progress for the user when generating patches in format-patch. This is to aid the user when performing large rebases and have some indication to the user that progress is being made.

These patches just add the normal progress indication when generating patches and makes it the default when running rebase. It will also respect the -q[uiet] flag and not show when it is present.

Kevin Willford (2):
  format-patch: have progress option while generating patches
  rebase: turn on progress option by default for format-patch
 Documentation/git-format-patch.txt |  8 ++++++++
 builtin/log.c                      | 10 ++++++++++
 git-rebase--am.sh                  |  5 +++--
 git-rebase.sh                      |  2 ++
 4 files changed, 23 insertions(+), 2 deletions(-)
-- 
2.13.0.92.g73a4ce6a77
Kevin Willford· May 31, 2017, 15:04 UTC · re: Kevin Willford · lore

[PATCH 1/2] format-patch: have progress option while generating patches

When generating patches for the rebase command if the user does not realize the branch they are rebasing onto is thousands of commits different there is no progress indication after initial rewinding message.

This patch allows a progress option to be passed to format-patch so that the user can be informed the progress of generating the patch. This option will then be used by the rebase command when calling format-patch.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 Documentation/git-format-patch.txt |  8 ++++++++
 builtin/log.c                      | 10 ++++++++++
 2 files changed, 18 insertions(+)
Show changes to 2 files +18 −0

Documentation/git-format-patch.txt, builtin/log.c

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index c890328b02..ee5f99f606 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -23,6 +23,7 @@ SYNOPSIS
 		   [(--reroll-count|-v) <n>]
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
+		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
 
@@ -260,6 +261,7 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
 -q::
 --quiet::
 	Do not print the names of the generated files to standard output.
+	Progress is not reported to the standard error stream.
 
 --no-binary::
 	Do not output contents of changes in binary files, instead
@@ -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
 	range are always formatted as creation patches, independently
 	of this flag.
 
+--progress::
+	Progress status is reported on the standard error stream
+	by default when it is attached to a terminal, unless -q
+	is specified. This flag forces progress status even if the
+	standard error stream is not directed to a terminal.
+
 CONFIGURATION
 -------------
 You can specify extra mail header lines to be added to each message,
diff --git a/builtin/log.c b/builtin/log.c
index 631fbc984f..02c50431b6 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -26,6 +26,7 @@
 #include "version.h"
 #include "mailmap.h"
 #include "gpg-interface.h"
+#include "progress.h"
 
 /* Set a default date-time format for git log ("log.date" config variable) */
 static const char *default_date_mode = NULL;
@@ -1409,6 +1410,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	char *branch_name = NULL;
 	char *base_commit = NULL;
 	struct base_tree_info bases;
+	int show_progress = 0;
+	struct progress *progress = NULL;
 
 	const struct option builtin_format_patch_options[] = {
 		{ OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
@@ -1480,6 +1483,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		OPT_FILENAME(0, "signature-file", &signature_file,
 				N_("add a signature from a file")),
 		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
+		OPT_BOOL(0, "progress", &show_progress,
+			 N_("show progress")),
 		OPT_END()
 	};
 
@@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		start_number--;
 	}
 	rev.add_signoff = do_signoff;
+
+	if (show_progress && !quiet)
+		progress = start_progress(_("Generating patch"), total);
 	while (0 <= --nr) {
 		int shown;
+		display_progress(progress, total - nr);
 		commit = list[nr];
 		rev.nr = total - nr + (start_number - 1);
 		/* Make the second and subsequent mails replies to the first */
@@ -1805,6 +1814,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (!use_stdout)
 			fclose(rev.diffopt.file);
 	}
+	stop_progress(&progress);
 	free(list);
 	free(branch_name);
 	string_list_clear(&extra_to, 0);
-- 
2.13.0.92.g73a4ce6a77
Stefan Beller· May 31, 2017, 18:40 UTC · re: Kevin Willford · lore

Re: [PATCH 1/2] format-patch: have progress option while generating patches

On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com> wrote:
Show 9 quoted lines
> When generating patches for the rebase command if the user does
> not realize the branch they are rebasing onto is thousands of
> commits different there is no progress indication after initial
> rewinding message.
>
> This patch allows a progress option to be passed to format-patch
> so that the user can be informed the progress of generating the
> patch.  This option will then be used by the rebase command when
> calling format-patch.

After reading the code, I was looking for some explanation on the underlying assumptions, such as:

  The progress meter as presented in this patch assumes the thousands of
  patches to have a fine granularity as well as assuming to require all the
  same amount of work/time for each, such that a steady progress bar
  is achieved.
  We do not want to estimate the time for each patch based e.g.
  on their size or number of touched files (or parents) as that is too
  expensive for just a progress meter.
Show 94 quoted lines
>
> Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> ---
>  Documentation/git-format-patch.txt |  8 ++++++++
>  builtin/log.c                      | 10 ++++++++++
>  2 files changed, 18 insertions(+)
>
> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
> index c890328b02..ee5f99f606 100644
> --- a/Documentation/git-format-patch.txt
> +++ b/Documentation/git-format-patch.txt
> @@ -23,6 +23,7 @@ SYNOPSIS
>                    [(--reroll-count|-v) <n>]
>                    [--to=<email>] [--cc=<email>]
>                    [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
> +                  [--progress]
>                    [<common diff options>]
>                    [ <since> | <revision range> ]
>
> @@ -260,6 +261,7 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
>  -q::
>  --quiet::
>         Do not print the names of the generated files to standard output.
> +       Progress is not reported to the standard error stream.
>
>  --no-binary::
>         Do not output contents of changes in binary files, instead
> @@ -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
>         range are always formatted as creation patches, independently
>         of this flag.
>
> +--progress::
> +       Progress status is reported on the standard error stream
> +       by default when it is attached to a terminal, unless -q
> +       is specified. This flag forces progress status even if the
> +       standard error stream is not directed to a terminal.
> +
>  CONFIGURATION
>  -------------
>  You can specify extra mail header lines to be added to each message,
> diff --git a/builtin/log.c b/builtin/log.c
> index 631fbc984f..02c50431b6 100644
> --- a/builtin/log.c
> +++ b/builtin/log.c
> @@ -26,6 +26,7 @@
>  #include "version.h"
>  #include "mailmap.h"
>  #include "gpg-interface.h"
> +#include "progress.h"
>
>  /* Set a default date-time format for git log ("log.date" config variable) */
>  static const char *default_date_mode = NULL;
> @@ -1409,6 +1410,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>         char *branch_name = NULL;
>         char *base_commit = NULL;
>         struct base_tree_info bases;
> +       int show_progress = 0;
> +       struct progress *progress = NULL;
>
>         const struct option builtin_format_patch_options[] = {
>                 { OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
> @@ -1480,6 +1483,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>                 OPT_FILENAME(0, "signature-file", &signature_file,
>                                 N_("add a signature from a file")),
>                 OPT__QUIET(&quiet, N_("don't print the patch filenames")),
> +               OPT_BOOL(0, "progress", &show_progress,
> +                        N_("show progress")),
>                 OPT_END()
>         };
>
> @@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>                 start_number--;
>         }
>         rev.add_signoff = do_signoff;
> +
> +       if (show_progress && !quiet)
> +               progress = start_progress(_("Generating patch"), total);
>         while (0 <= --nr) {
>                 int shown;
> +               display_progress(progress, total - nr);
>                 commit = list[nr];
>                 rev.nr = total - nr + (start_number - 1);
>                 /* Make the second and subsequent mails replies to the first */
> @@ -1805,6 +1814,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>                 if (!use_stdout)
>                         fclose(rev.diffopt.file);
>         }
> +       stop_progress(&progress);
>         free(list);
>         free(branch_name);
>         string_list_clear(&extra_to, 0);
> --
> 2.13.0.92.g73a4ce6a77
>
Kevin Willford· May 31, 2017, 19:31 UTC · re: Stefan Beller · lore

RE: [PATCH 1/2] format-patch: have progress option while generating patches

Show 34 quoted lines
> -----Original Message-----
> From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On
> Behalf Of Stefan Beller
> Sent: Wednesday, May 31, 2017 12:40 PM
> To: Kevin Willford <kcwillford@gmail.com>
> Cc: git@vger.kernel.org; Junio C Hamano <gitster@pobox.com>; Kevin
> Willford <kewillf@microsoft.com>
> Subject: Re: [PATCH 1/2] format-patch: have progress option while
> generating patches
> 
> On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com>
> wrote:
> > When generating patches for the rebase command if the user does not
> > realize the branch they are rebasing onto is thousands of commits
> > different there is no progress indication after initial rewinding
> > message.
> >
> > This patch allows a progress option to be passed to format-patch so
> > that the user can be informed the progress of generating the patch.
> > This option will then be used by the rebase command when calling
> > format-patch.
> 
> After reading the code, I was looking for some explanation on the underlying
> assumptions, such as:
> 
>   The progress meter as presented in this patch assumes the thousands of
>   patches to have a fine granularity as well as assuming to require all the
>   same amount of work/time for each, such that a steady progress bar
>   is achieved.
> 
>   We do not want to estimate the time for each patch based e.g.
>   on their size or number of touched files (or parents) as that is too
>   expensive for just a progress meter.
> 
Sounds good.  I will add some explanation to the commit message.
Show 104 quoted lines
> 
> >
> > Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> > ---
> >  Documentation/git-format-patch.txt |  8 ++++++++
> >  builtin/log.c                      | 10 ++++++++++
> >  2 files changed, 18 insertions(+)
> >
> > diff --git a/Documentation/git-format-patch.txt
> > b/Documentation/git-format-patch.txt
> > index c890328b02..ee5f99f606 100644
> > --- a/Documentation/git-format-patch.txt
> > +++ b/Documentation/git-format-patch.txt
> > @@ -23,6 +23,7 @@ SYNOPSIS
> >                    [(--reroll-count|-v) <n>]
> >                    [--to=<email>] [--cc=<email>]
> >                    [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
> > +                  [--progress]
> >                    [<common diff options>]
> >                    [ <since> | <revision range> ]
> >
> > @@ -260,6 +261,7 @@ you can use `--suffix=-patch` to get `0001-
> description-of-my-change-patch`.
> >  -q::
> >  --quiet::
> >         Do not print the names of the generated files to standard output.
> > +       Progress is not reported to the standard error stream.
> >
> >  --no-binary::
> >         Do not output contents of changes in binary files, instead @@
> > -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-
> of-my-change-patch`.
> >         range are always formatted as creation patches, independently
> >         of this flag.
> >
> > +--progress::
> > +       Progress status is reported on the standard error stream
> > +       by default when it is attached to a terminal, unless -q
> > +       is specified. This flag forces progress status even if the
> > +       standard error stream is not directed to a terminal.
> > +
> >  CONFIGURATION
> >  -------------
> >  You can specify extra mail header lines to be added to each message,
> > diff --git a/builtin/log.c b/builtin/log.c index
> > 631fbc984f..02c50431b6 100644
> > --- a/builtin/log.c
> > +++ b/builtin/log.c
> > @@ -26,6 +26,7 @@
> >  #include "version.h"
> >  #include "mailmap.h"
> >  #include "gpg-interface.h"
> > +#include "progress.h"
> >
> >  /* Set a default date-time format for git log ("log.date" config
> > variable) */  static const char *default_date_mode = NULL; @@ -1409,6
> > +1410,8 @@ int cmd_format_patch(int argc, const char **argv, const char
> *prefix)
> >         char *branch_name = NULL;
> >         char *base_commit = NULL;
> >         struct base_tree_info bases;
> > +       int show_progress = 0;
> > +       struct progress *progress = NULL;
> >
> >         const struct option builtin_format_patch_options[] = {
> >                 { OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
> > @@ -1480,6 +1483,8 @@ int cmd_format_patch(int argc, const char **argv,
> const char *prefix)
> >                 OPT_FILENAME(0, "signature-file", &signature_file,
> >                                 N_("add a signature from a file")),
> >                 OPT__QUIET(&quiet, N_("don't print the patch
> > filenames")),
> > +               OPT_BOOL(0, "progress", &show_progress,
> > +                        N_("show progress")),
> >                 OPT_END()
> >         };
> >
> > @@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char
> **argv, const char *prefix)
> >                 start_number--;
> >         }
> >         rev.add_signoff = do_signoff;
> > +
> > +       if (show_progress && !quiet)
> > +               progress = start_progress(_("Generating patch"),
> > + total);
> >         while (0 <= --nr) {
> >                 int shown;
> > +               display_progress(progress, total - nr);
> >                 commit = list[nr];
> >                 rev.nr = total - nr + (start_number - 1);
> >                 /* Make the second and subsequent mails replies to the
> > first */ @@ -1805,6 +1814,7 @@ int cmd_format_patch(int argc, const char
> **argv, const char *prefix)
> >                 if (!use_stdout)
> >                         fclose(rev.diffopt.file);
> >         }
> > +       stop_progress(&progress);
> >         free(list);
> >         free(branch_name);
> >         string_list_clear(&extra_to, 0);
> > --
> > 2.13.0.92.g73a4ce6a77
> >
Jeff King· May 31, 2017, 22:01 UTC · re: Kevin Willford · lore

Re: [PATCH 1/2] format-patch: have progress option while generating patches

On Wed, May 31, 2017 at 08:04:26AM -0700, Kevin Willford wrote:
Show 9 quoted lines
> When generating patches for the rebase command if the user does
> not realize the branch they are rebasing onto is thousands of
> commits different there is no progress indication after initial
> rewinding message.
> 
> This patch allows a progress option to be passed to format-patch
> so that the user can be informed the progress of generating the
> patch.  This option will then be used by the rebase command when
> calling format-patch.

I'm generally in favor of progress meters, though it does seem a little funny to me that we'd need one on format-patch. It's basically the same operation as "git log -p", after all. I guess one difference is that usually the output of "log" would stream to the pager, so the user would see immediate output. That's true of format-patch, too, but in the instance you care about we're probably doing something more like:

  git format-patch "$@" >patches
and the user sees nothing.
That makes me wonder two things:
  1. Should this really be tied to isatty(2), as the documentation
     claims?  It seems like you'd really only want it to kick in for
     certain cases. Arguably whenever stdout is _not_ going to a tty
     you'd want progress, but I think probably the caller should
     probably just decide whether to ask for it or not.
  2. Should this apply to other commands in the log family besides
     format-patch? E.g., should "git log --progress -p >commits" work?
     I added a progress meter to rev-list a while ago (for connectivity
     checks). I don't think we could push this down as far as the
     revision traversal code, because only its callers really understand
     what's the right unit to be counting.
     It may not be worth the trouble, though. Only "format-patch" does a
     pre-pass to find the total number of commits. So "git log
     --progress" would inherently just be counting up, with no clue what
     the final number is.
Show 9 quoted lines
> @@ -283,6 +285,12 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
>  	range are always formatted as creation patches, independently
>  	of this flag.
>  
> +--progress::
> +	Progress status is reported on the standard error stream
> +	by default when it is attached to a terminal, unless -q
> +	is specified. This flag forces progress status even if the
> +	standard error stream is not directed to a terminal.

Checking whether stderr is a tty would match our usual progress meters. But I don't actually see any code in the patch implementing this.

As I said above, I think I'd prefer it to require "--progress", as format-patch is quite often used as plumbing. But we'd need to fix the documentation here to match.

Show 7 quoted lines
> @@ -1739,8 +1744,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  		start_number--;
>  	}
>  	rev.add_signoff = do_signoff;
> +
> +	if (show_progress && !quiet)
> +		progress = start_progress(_("Generating patch"), total);
The meter code here all looks correct, but let me bikeshed for a moment. :)

Should this use start_progress_delay()? In most cases the command will complete very quickly, and the progress report is just noise. For many commands (e.g., checkout) we wait 1-2 seconds before bothering to show progress output.

I would have expected this to say "Generating patches"; most of our other progress messages are pluralized. You could use Q_() to handle the mono/plural case, but I think it's fine to just always say "patches" (that's what other messages do).

One final thought is whether callers would want to customize this message, since it will often be used as plumbing. E.g., would rebase want to say something besides "Generating patches". I'm not sure. Anyway, if you're interested in that direction, there's prior art in the way rev-list handles "--progress" (and its sole caller, check_connected()).

-Peff
Junio C Hamano· Jun 1, 2017, 04:10 UTC · re: Jeff King · lore

Re: [PATCH 1/2] format-patch: have progress option while generating patches

Jeff King <peff@peff.net> writes:
> As I said above, I think I'd prefer it to require "--progress", as
> format-patch is quite often used as plumbing.
Yes, that sounds sensible.

Initially, my reaction was "Why do we even need --progress for format-patch, when it gives one-line per patch output to show the progress anyway?", but if that output is redirected to a file, of course you'd need --progress independently.

> Should this use start_progress_delay()? In most cases the command will
> complete very quickly, and the progress report is just noise. For many
> commands (e.g., checkout) we wait 1-2 seconds before bothering to show
> progress output.

It is better to use the "delay" version for progress meters for commands that may or may not last very long, and this may be a good candidate to heed that principle.

The subcommands that use start_progress() tend to be older and more batch oriented operations, e.g. fsck, pack-objects, etc., that are expected to last longer. It may be a good idea to convert them to the "delay" variant, but obviously that is outside the scope of this patch.

Show 11 quoted lines
> I would have expected this to say "Generating patches"; most of our
> other progress messages are pluralized. You could use Q_() to handle the
> mono/plural case, but I think it's fine to just always say "patches"
> (that's what other messages do).
>
> One final thought is whether callers would want to customize this
> message, since it will often be used as plumbing. E.g., would rebase
> want to say something besides "Generating patches". I'm not sure.
> Anyway, if you're interested in that direction, there's prior art in
> the way rev-list handles "--progress" (and its sole caller,
> check_connected()).
These are all good suggestions.
Johannes Schindelin· Jun 1, 2017, 11:15 UTC · re: Jeff King · lore

Re: [PATCH 1/2] format-patch: have progress option while generating patches

Hi Peff,
On Wed, 31 May 2017, Jeff King wrote:
> I'm generally in favor of progress meters, though it does seem a little
> funny to me that we'd need one on format-patch.

When working with huge repositories with a large number of branches, it is all too easy to pick the wrong branch to rebase to. In that case, it can take a long time for the `git rebase` call to even generate the mbox.

Kevin's patch is a visual indication when this is happening.

Instead of being puzzled by `git rebase` "hanging", the user will now see that (.../21957) patch is generated and will have a chance to say "wait a minute, *that* many patches? I think I meant a different branch".

And since format-patch generates the patches for `git rebase` (actually, it generates an mbox that then has to be split again, but you get the gist), it is format-patch that needs to output the progress.

Ciao, dscho

Jeff King· Jun 1, 2017, 15:54 UTC · re: Johannes Schindelin · lore

Re: [PATCH 1/2] format-patch: have progress option while generating patches

On Thu, Jun 01, 2017 at 01:15:57PM +0200, Johannes Schindelin wrote:
Show 10 quoted lines
> On Wed, 31 May 2017, Jeff King wrote:
> 
> > I'm generally in favor of progress meters, though it does seem a little
> > funny to me that we'd need one on format-patch.
> 
> When working with huge repositories with a large number of branches, it is
> all too easy to pick the wrong branch to rebase to. In that case, it can
> take a long time for the `git rebase` call to even generate the mbox.
> 
> Kevin's patch is a visual indication when this is happening.

Right, sorry if it wasn't clear from the rest of my message. My statement was one of surprise, not objection. I understand his use case.

-Peff
Kevin Willford· May 31, 2017, 15:04 UTC · re: Kevin Willford · lore

[PATCH 2/2] rebase: turn on progress option by default for format-patch

This change passes the progress option of format-patch by default and passes the -q --quiet option through to the format-patch call so that it is respected as well.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 git-rebase--am.sh | 5 +++--
 git-rebase.sh     | 2 ++
 2 files changed, 5 insertions(+), 2 deletions(-)
Show changes to 2 files +5 −2

git-rebase--am.sh, git-rebase.sh

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index 375239341f..ab2be30abf 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -51,8 +51,9 @@ then
 else
 	rm -f "$GIT_DIR/rebased-patches"
 
-	git format-patch -k --stdout --full-index --cherry-pick --right-only \
-		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
+	git format-patch $git_format_patch_opt -k --stdout --full-index \
+		--cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
+		--no-renames --no-cover-letter --progress \
 		"$revisions" ${restrict_revision+^$restrict_revision} \
 		>"$GIT_DIR/rebased-patches"
 	ret=$?
diff --git a/git-rebase.sh b/git-rebase.sh
index db1deed846..b72e319ac9 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -73,6 +73,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
 autostash="$(git config --bool rebase.autostash || echo false)"
 fork_point=auto
 git_am_opt=
+git_format_patch_opt=
 rebase_root=
 force_rebase=
 allow_rerere_autoupdate=
@@ -308,6 +309,7 @@ do
 	--quiet)
 		GIT_QUIET=t
 		git_am_opt="$git_am_opt -q"
+		git_format_patch_opt="$git_format_patch_opt -q"
 		verbose=
 		diffstat=
 		;;
-- 
2.13.0.92.g73a4ce6a77
Stefan Beller· May 31, 2017, 19:08 UTC · re: Kevin Willford · lore

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com> wrote:
> This change passes the progress option of format-patch by
> default and passes the -q --quiet option through to the
> format-patch call so that it is respected as well.

This is not conflicting with Johannes rewrite of rebase in C? (rebase is a huge beast IIUC)

When passing the progress option by default to formatting patches, maybe we should use start_progress_delay in the previous patch instead to omit the progress for short lived patch formatting sessions? (say a delay of one second?)

Thanks, Stefan

Show 46 quoted lines
>
> Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> ---
>  git-rebase--am.sh | 5 +++--
>  git-rebase.sh     | 2 ++
>  2 files changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
> index 375239341f..ab2be30abf 100644
> --- a/git-rebase--am.sh
> +++ b/git-rebase--am.sh
> @@ -51,8 +51,9 @@ then
>  else
>         rm -f "$GIT_DIR/rebased-patches"
>
> -       git format-patch -k --stdout --full-index --cherry-pick --right-only \
> -               --src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> +       git format-patch $git_format_patch_opt -k --stdout --full-index \
> +               --cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
> +               --no-renames --no-cover-letter --progress \
>                 "$revisions" ${restrict_revision+^$restrict_revision} \
>                 >"$GIT_DIR/rebased-patches"
>         ret=$?
> diff --git a/git-rebase.sh b/git-rebase.sh
> index db1deed846..b72e319ac9 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -73,6 +73,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
>  autostash="$(git config --bool rebase.autostash || echo false)"
>  fork_point=auto
>  git_am_opt=
> +git_format_patch_opt=
>  rebase_root=
>  force_rebase=
>  allow_rerere_autoupdate=
> @@ -308,6 +309,7 @@ do
>         --quiet)
>                 GIT_QUIET=t
>                 git_am_opt="$git_am_opt -q"
> +               git_format_patch_opt="$git_format_patch_opt -q"
>                 verbose=
>                 diffstat=
>                 ;;
> --
> 2.13.0.92.g73a4ce6a77
>
Kevin Willford· May 31, 2017, 19:46 UTC · re: Stefan Beller · lore

RE: [PATCH 2/2] rebase: turn on progress option by default for format-patch

Show 17 quoted lines
> -----Original Message-----
> From: Stefan Beller [mailto:sbeller@google.com]
> Sent: Wednesday, May 31, 2017 1:09 PM
> To: Kevin Willford <kcwillford@gmail.com>
> Cc: git@vger.kernel.org; Junio C Hamano <gitster@pobox.com>; Kevin
> Willford <kewillf@microsoft.com>
> Subject: Re: [PATCH 2/2] rebase: turn on progress option by default for
> format-patch
> 
> On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com>
> wrote:
> > This change passes the progress option of format-patch by default and
> > passes the -q --quiet option through to the format-patch call so that
> > it is respected as well.
> 
> This is not conflicting with Johannes rewrite of rebase in C?
> (rebase is a huge beast IIUC)

I will check with Johannes and see what possible conflicts there could be. Since these are flags that get passed to the format-patch code, it shouldn't take much to put it in the C code as well.

Show 6 quoted lines
> 
> When passing the progress option by default to formatting patches, maybe
> we should use start_progress_delay in the previous patch instead to omit
> the progress for short lived patch formatting sessions?
> (say a delay of one second?)
> 
I thought about that and certainly could do it but I have found it nice to have the number of patches that are generated in the output even for a small number or commits.  For example when I run a `git rebase master` and expect there to be only 2 commits, the message "Generating patch: 100% (2/2), done."  Gives me that good feeling that I did it right and didn't mess something up.  I'm good either way though.
Show 49 quoted lines
> Thanks,
> Stefan
> 
> >
> > Signed-off-by: Kevin Willford <kewillf@microsoft.com>
> > ---
> >  git-rebase--am.sh | 5 +++--
> >  git-rebase.sh     | 2 ++
> >  2 files changed, 5 insertions(+), 2 deletions(-)
> >
> > diff --git a/git-rebase--am.sh b/git-rebase--am.sh index
> > 375239341f..ab2be30abf 100644
> > --- a/git-rebase--am.sh
> > +++ b/git-rebase--am.sh
> > @@ -51,8 +51,9 @@ then
> >  else
> >         rm -f "$GIT_DIR/rebased-patches"
> >
> > -       git format-patch -k --stdout --full-index --cherry-pick --right-only \
> > -               --src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> > +       git format-patch $git_format_patch_opt -k --stdout --full-index \
> > +               --cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
> > +               --no-renames --no-cover-letter --progress \
> >                 "$revisions" ${restrict_revision+^$restrict_revision} \
> >                 >"$GIT_DIR/rebased-patches"
> >         ret=$?
> > diff --git a/git-rebase.sh b/git-rebase.sh index
> > db1deed846..b72e319ac9 100755
> > --- a/git-rebase.sh
> > +++ b/git-rebase.sh
> > @@ -73,6 +73,7 @@ test "$(git config --bool rebase.stat)" = true &&
> > diffstat=t  autostash="$(git config --bool rebase.autostash || echo false)"
> >  fork_point=auto
> >  git_am_opt=
> > +git_format_patch_opt=
> >  rebase_root=
> >  force_rebase=
> >  allow_rerere_autoupdate=
> > @@ -308,6 +309,7 @@ do
> >         --quiet)
> >                 GIT_QUIET=t
> >                 git_am_opt="$git_am_opt -q"
> > +               git_format_patch_opt="$git_format_patch_opt -q"
> >                 verbose=
> >                 diffstat=
> >                 ;;
> > --
> > 2.13.0.92.g73a4ce6a77
> >
Stefan Beller· May 31, 2017, 20:27 UTC · re: Kevin Willford · lore

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

On Wed, May 31, 2017 at 12:46 PM, Kevin Willford <kewillf@microsoft.com> wrote:
>
> I thought about that and certainly could do it but I have found it nice to have the number of patches that are generated in the output even for a small number or commits.  For example when I run a `git rebase master` and expect there to be only 2 commits, the message "Generating patch: 100% (2/2), done."  Gives me that good feeling that I did it right and didn't mess something up.  I'm good either way though.
>

Oh I see, that number matching makes sense. Though by reading the code, I have the impression that the final value would not change. So if it would take a really long time for these 2 patches, you'd still get the (2/2), to know it was 2 patches indeed.

When it goes quickly, I'd be more concerned about screen real estate, driving off the recent history from the screen.

Also we maybe want to have
 "Generating patches: 100% (2/2), done."
plural, as (when using the delay)
it is more likely to kick in for more than just one patch?

Sorry, if this comes across as bike shedding. It's meant as food for thought. :)

Thanks, Stefan

Johannes Schindelin· Jun 1, 2017, 11:11 UTC · re: Stefan Beller · lore

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

Hi Stefan,
On Wed, 31 May 2017, Stefan Beller wrote:
Show 7 quoted lines
> On Wed, May 31, 2017 at 8:04 AM, Kevin Willford <kcwillford@gmail.com> wrote:
> > This change passes the progress option of format-patch by
> > default and passes the -q --quiet option through to the
> > format-patch call so that it is respected as well.
> 
> This is not conflicting with Johannes rewrite of rebase in C?
> (rebase is a huge beast IIUC)
No.

Ciao, Dscho

Jeff King· May 31, 2017, 22:11 UTC · re: Kevin Willford · lore

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

On Wed, May 31, 2017 at 08:04:27AM -0700, Kevin Willford wrote:
> This change passes the progress option of format-patch by
> default and passes the -q --quiet option through to the
> format-patch call so that it is respected as well.
That makes sense. Is it a bug that we aren't propagating "-q" already?

I think the answer is "no", because that option only claims to silence the printing of the filenames that format-patch writes. But since we're using --stdout, it wouldn't write those anyway.

Come to think of it, I'm not sure that "-q" silencing "--progress" in your patch 1 actually makes sense. If you do:

  git format-patch -o out/

you don't need progress, because we're already writing the filenames to stdout. So it's only if you did:

  git format-patch -o out/ -q

that you'd actually want progress. But "-q" would override any --progress option! So I actually think we may be better off keeping the two as distinct features (especially if we follow the "no progress unless --progress is given" rule I mentioned in the earlier message).

Show 15 quoted lines
> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
> index 375239341f..ab2be30abf 100644
> --- a/git-rebase--am.sh
> +++ b/git-rebase--am.sh
> @@ -51,8 +51,9 @@ then
>  else
>  	rm -f "$GIT_DIR/rebased-patches"
>  
> -	git format-patch -k --stdout --full-index --cherry-pick --right-only \
> -		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> +	git format-patch $git_format_patch_opt -k --stdout --full-index \
> +		--cherry-pick --right-only --src-prefix=a/ --dst-prefix=b/ \
> +		--no-renames --no-cover-letter --progress \
>  		"$revisions" ${restrict_revision+^$restrict_revision} \
>  		>"$GIT_DIR/rebased-patches"

Here we pass --progress unconditionally, which tells format-patch to output progress information. Shouldn't we be checking whether stderr is a tty before making that decision? And that we're not in --quiet mode?

That explains why you want to pass "-q" in the other hunk; to countermand the explicit --progress here. But if we separate the two as I mentioned above, you'd want logic more like:

  if test -t 2 && test "$GIT_QUIET" != "t"
	git_format_patch_opt="$git_format_patch_opt --progress"
  fi
-Peff
Junio C Hamano· Jun 3, 2017, 23:45 UTC · re: Jeff King · lore

Re: [PATCH 2/2] rebase: turn on progress option by default for format-patch

Jeff King <peff@peff.net> writes:
Show 8 quoted lines
> ...
> That explains why you want to pass "-q" in the other hunk; to
> countermand the explicit --progress here. But if we separate the two as
> I mentioned above, you'd want logic more like:
>
>   if test -t 2 && test "$GIT_QUIET" != "t"
> 	git_format_patch_opt="$git_format_patch_opt --progress"
>   fi

This looks like a sensible line of thought. Kevin, do you want to try this again with the "explicit --progress option" approach? I think the gist of the v1 review is that everybody agrees that the end goal of showing a progress eye-candy from a long-running "format-patch" phase in a "rebase" is a good idea, but it is dubious "format-patch" should show it in cases other than when --progress is explicitly asked for.

Thanks.
Kevin Willford· Aug 10, 2017, 18:32 UTC · re: Kevin Willford · lore

[PATCH v2 0/2] Add progress for format-patch and rebase

Changes since last patch:
1. Use start_progress_delay so progress isn't shown if generating
   the patches is fast enough
2. Updated to have text of "Generating patches"
3. Only show progress when the --progress flag is passed
4. In the rebase script check stderr and the quiet option is not
   set before propagating the progress flag to format-patch
Kevin Willford (2):
  format-patch: have progress option while generating patches
  rebase: turn on progress option by default for format-patch
 Documentation/git-format-patch.txt |  4 ++++
 builtin/log.c                      | 10 ++++++++++
 git-rebase--am.sh                  |  1 +
 git-rebase.sh                      |  6 ++++++
 4 files changed, 21 insertions(+)
-- 
2.14.0.rc0.286.g44127d70e4
Kevin Willford· Aug 10, 2017, 18:32 UTC · re: Kevin Willford · lore

[PATCH v2 2/2] rebase: turn on progress option by default for format-patch

This change passes the progress option of format-patch checking that stderr is attached and rebase is not being run in quiet mode.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 git-rebase--am.sh | 1 +
 git-rebase.sh     | 6 ++++++
 2 files changed, 7 insertions(+)
Show changes to 2 files +7 −0

git-rebase--am.sh, git-rebase.sh

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index 375239341f..ff98fe3a73 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -53,6 +53,7 @@ else
 
 	git format-patch -k --stdout --full-index --cherry-pick --right-only \
 		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
+		$git_format_patch_opt \
 		"$revisions" ${restrict_revision+^$restrict_revision} \
 		>"$GIT_DIR/rebased-patches"
 	ret=$?
diff --git a/git-rebase.sh b/git-rebase.sh
index f8b3d1fd97..ad8415e3cf 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -74,6 +74,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
 autostash="$(git config --bool rebase.autostash || echo false)"
 fork_point=auto
 git_am_opt=
+git_format_patch_opt=
 rebase_root=
 force_rebase=
 allow_rerere_autoupdate=
@@ -445,6 +446,11 @@ else
 	state_dir="$apply_dir"
 fi
 
+if test -t 2 && test -z "$GIT_QUIET"
+then
+	git_format_patch_opt="$git_format_patch_opt --progress"
+fi
+
 if test -z "$rebase_root"
 then
 	case "$#" in
-- 
2.14.0.rc0.286.g44127d70e4
Junio C Hamano· Aug 11, 2017, 22:22 UTC · re: Kevin Willford · lore

Re: [PATCH v2 2/2] rebase: turn on progress option by default for format-patch

Kevin Willford <kcwillford@gmail.com> writes:
> This change passes the progress option of format-patch checking
> that stderr is attached and rebase is not being run in quiet mode.
    Pass the "--progress" option to format-patch when the standard
    error stream goes to the terminal and the command is not run in
    "--quiet" mode.

Thanks for posting a refreshed version. Both patches make sense to me.

>
> Signed-off-by: Kevin Willford <kewillf@microsoft.com>

We want to see our authors and signed-off match. Do you want the employer address appear in our "shortlog -e" output, or your personal gmail address? I'll tenatively "fix" your author address to match the one at @microsoft.com while queuing.

Show 41 quoted lines
> ---
>  git-rebase--am.sh | 1 +
>  git-rebase.sh     | 6 ++++++
>  2 files changed, 7 insertions(+)
>
> diff --git a/git-rebase--am.sh b/git-rebase--am.sh
> index 375239341f..ff98fe3a73 100644
> --- a/git-rebase--am.sh
> +++ b/git-rebase--am.sh
> @@ -53,6 +53,7 @@ else
>  
>  	git format-patch -k --stdout --full-index --cherry-pick --right-only \
>  		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
> +		$git_format_patch_opt \
>  		"$revisions" ${restrict_revision+^$restrict_revision} \
>  		>"$GIT_DIR/rebased-patches"
>  	ret=$?
> diff --git a/git-rebase.sh b/git-rebase.sh
> index f8b3d1fd97..ad8415e3cf 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -74,6 +74,7 @@ test "$(git config --bool rebase.stat)" = true && diffstat=t
>  autostash="$(git config --bool rebase.autostash || echo false)"
>  fork_point=auto
>  git_am_opt=
> +git_format_patch_opt=
>  rebase_root=
>  force_rebase=
>  allow_rerere_autoupdate=
> @@ -445,6 +446,11 @@ else
>  	state_dir="$apply_dir"
>  fi
>  
> +if test -t 2 && test -z "$GIT_QUIET"
> +then
> +	git_format_patch_opt="$git_format_patch_opt --progress"
> +fi
> +
>  if test -z "$rebase_root"
>  then
>  	case "$#" in
Kevin Willford· Aug 10, 2017, 18:32 UTC · re: Kevin Willford · lore

[PATCH v2 1/2] format-patch: have progress option while generating patches

When generating patches for the rebase command if the user does not realize the branch they are rebasing onto is thousands of commits different there is no progress indication after initial rewinding message.

The progress meter as presented in this patch assumes the thousands of patches to have a fine granularity as well as assuming to require all the same amount of work/time for each, such that a steady progress bar is achieved.

We do not want to estimate the time for each patch based e.g. on their size or number of touched files (or parents) as that is too expensive for just a progress meter.

This patch allows a progress option to be passed to format-patch so that the user can be informed the progress of generating the patch. This option is then used by the rebase command when calling format-patch.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
---
 Documentation/git-format-patch.txt |  4 ++++
 builtin/log.c                      | 10 ++++++++++
 2 files changed, 14 insertions(+)
Show changes to 2 files +14 −0

Documentation/git-format-patch.txt, builtin/log.c

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index c890328b02..6cbe462a77 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -23,6 +23,7 @@ SYNOPSIS
 		   [(--reroll-count|-v) <n>]
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
+		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
 
@@ -283,6 +284,9 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
 	range are always formatted as creation patches, independently
 	of this flag.
 
+--progress::
+	Show progress reports on stderr as patches are generated.
+
 CONFIGURATION
 -------------
 You can specify extra mail header lines to be added to each message,
diff --git a/builtin/log.c b/builtin/log.c
index 725c7b8a1a..b07a5529c2 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -27,6 +27,7 @@
 #include "version.h"
 #include "mailmap.h"
 #include "gpg-interface.h"
+#include "progress.h"
 
 /* Set a default date-time format for git log ("log.date" config variable) */
 static const char *default_date_mode = NULL;
@@ -1422,6 +1423,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	char *branch_name = NULL;
 	char *base_commit = NULL;
 	struct base_tree_info bases;
+	int show_progress = 0;
+	struct progress *progress = NULL;
 
 	const struct option builtin_format_patch_options[] = {
 		{ OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
@@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		OPT_FILENAME(0, "signature-file", &signature_file,
 				N_("add a signature from a file")),
 		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
+		OPT_BOOL(0, "progress", &show_progress,
+			 N_("show progress while generating patches")),
 		OPT_END()
 	};
 
@@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		start_number--;
 	}
 	rev.add_signoff = do_signoff;
+
+	if (show_progress)
+		progress = start_progress_delay(_("Generating patches"), total, 0, 1);
 	while (0 <= --nr) {
 		int shown;
+		display_progress(progress, total - nr);
 		commit = list[nr];
 		rev.nr = total - nr + (start_number - 1);
 		/* Make the second and subsequent mails replies to the first */
@@ -1818,6 +1827,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		if (!use_stdout)
 			fclose(rev.diffopt.file);
 	}
+	stop_progress(&progress);
 	free(list);
 	free(branch_name);
 	string_list_clear(&extra_to, 0);
-- 
2.14.0.rc0.286.g44127d70e4
Jeff King· Aug 10, 2017, 23:20 UTC · re: Kevin Willford · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

On Thu, Aug 10, 2017 at 02:32:55PM -0400, Kevin Willford wrote:
Show 6 quoted lines
> @@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  		OPT_FILENAME(0, "signature-file", &signature_file,
>  				N_("add a signature from a file")),
>  		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
> +		OPT_BOOL(0, "progress", &show_progress,
> +			 N_("show progress while generating patches")),

Earlier I suggested allowing --progress="custom text" since this may be driven as plumbing for other commands. But I don't think there's any need to worry about it now. It can be added seamlessly later if we find such a caller.

Show 7 quoted lines
> @@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  		start_number--;
>  	}
>  	rev.add_signoff = do_signoff;
> +
> +	if (show_progress)
> +		progress = start_progress_delay(_("Generating patches"), total, 0, 1);

I don't really have an opinion on a 1 second delay versus 2. I thought we used 2 pretty consistently, though grepping around I do see a couple of 1's. It probably doesn't matter, but just a curiosity.

-Peff
Junio C Hamano· Aug 11, 2017, 22:18 UTC · re: Jeff King · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

Jeff King <peff@peff.net> writes:
Show 25 quoted lines
> On Thu, Aug 10, 2017 at 02:32:55PM -0400, Kevin Willford wrote:
>
>> @@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  		OPT_FILENAME(0, "signature-file", &signature_file,
>>  				N_("add a signature from a file")),
>>  		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
>> +		OPT_BOOL(0, "progress", &show_progress,
>> +			 N_("show progress while generating patches")),
>
> Earlier I suggested allowing --progress="custom text" since this may be
> driven as plumbing for other commands. But I don't think there's any
> need to worry about it now. It can be added seamlessly later if we find
> such a caller.
>
>> @@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  		start_number--;
>>  	}
>>  	rev.add_signoff = do_signoff;
>> +
>> +	if (show_progress)
>> +		progress = start_progress_delay(_("Generating patches"), total, 0, 1);
>
> I don't really have an opinion on a 1 second delay versus 2. I thought
> we used 2 pretty consistently, though grepping around I do see a couple
> of 1's. It probably doesn't matter, but just a curiosity.

Yeah, I also thought 2-second was what we used by default. Perhaps we would want to bring others in line?

Philip Oakley· Aug 12, 2017, 08:06 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

From: "Junio C Hamano" <gitster@pobox.com>
Show 32 quoted lines
> Jeff King <peff@peff.net> writes:
>
>> On Thu, Aug 10, 2017 at 02:32:55PM -0400, Kevin Willford wrote:
>>
>>> @@ -1493,6 +1496,8 @@ int cmd_format_patch(int argc, const char **argv, 
>>> const char *prefix)
>>>  OPT_FILENAME(0, "signature-file", &signature_file,
>>>  N_("add a signature from a file")),
>>>  OPT__QUIET(&quiet, N_("don't print the patch filenames")),
>>> + OPT_BOOL(0, "progress", &show_progress,
>>> + N_("show progress while generating patches")),
>>
>> Earlier I suggested allowing --progress="custom text" since this may be
>> driven as plumbing for other commands. But I don't think there's any
>> need to worry about it now. It can be added seamlessly later if we find
>> such a caller.
>>
>>> @@ -1752,8 +1757,12 @@ int cmd_format_patch(int argc, const char **argv, 
>>> const char *prefix)
>>>  start_number--;
>>>  }
>>>  rev.add_signoff = do_signoff;
>>> +
>>> + if (show_progress)
>>> + progress = start_progress_delay(_("Generating patches"), total, 0, 1);
>>
>> I don't really have an opinion on a 1 second delay versus 2. I thought
>> we used 2 pretty consistently, though grepping around I do see a couple
>> of 1's. It probably doesn't matter, but just a curiosity.
>
> Yeah, I also thought 2-second was what we used by default.  Perhaps
> we would want to bring others in line?

<bikeshed> Surely the choice should depend on the purpose of the delay. IIRC the 1 second between patches on the formal 'sent' time was simply to ensure the patches had the right sequence. Delays for warnings and progress have different purposes, so I think it's more about the purpose than standardising the time (along with expectations [least surprise] if other messages are displayed). -- Philip

Jeff King· Aug 13, 2017, 04:39 UTC · re: Philip Oakley · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

On Sat, Aug 12, 2017 at 09:06:18AM +0100, Philip Oakley wrote:
Show 15 quoted lines
> > > > + progress = start_progress_delay(_("Generating patches"), total, 0, 1);
> > > 
> > > I don't really have an opinion on a 1 second delay versus 2. I thought
> > > we used 2 pretty consistently, though grepping around I do see a couple
> > > of 1's. It probably doesn't matter, but just a curiosity.
> > 
> > Yeah, I also thought 2-second was what we used by default.  Perhaps
> > we would want to bring others in line?
> 
> <bikeshed> Surely the choice should depend on the purpose of the delay. IIRC
> the 1 second between patches on the formal 'sent' time was simply to ensure
> the patches had the right sequence. Delays for warnings and progress have
> different purposes, so I think it's more about the purpose than
> standardising the time (along with expectations [least surprise] if other
> messages are displayed).

I think you're confusing two unrelated things. There's a 1-second fake time-advance on patches, but that's done by send-email, not format-patch.

Here we're just talking about calls to start_progress_delay(), and how long it waits before deciding that the operation is slow enough to show progress. Blame, rename detection, and checkout use 1 second. Prune, prune-packed, and connectivity checks all use 2 seconds. I doubt it matters all that much, but presumably the purpose in all is "how long before a user starts to wonder if things are actually happening", which is probably command-independent.

-Peff
Junio C Hamano· Aug 14, 2017, 16:45 UTC · re: Jeff King · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

Jeff King <peff@peff.net> writes:
Show 16 quoted lines
> On Sat, Aug 12, 2017 at 09:06:18AM +0100, Philip Oakley wrote:
>
>> > > > + progress = start_progress_delay(_("Generating patches"), total, 0, 1);
>> > > 
>> > > I don't really have an opinion on a 1 second delay versus 2. I thought
>> > > we used 2 pretty consistently, though grepping around I do see a couple
>> > > of 1's. It probably doesn't matter, but just a curiosity.
>> > 
> ...
> Here we're just talking about calls to start_progress_delay(), and how
> long it waits before deciding that the operation is slow enough to show
> progress. Blame, rename detection, and checkout use 1 second. Prune,
> prune-packed, and connectivity checks all use 2 seconds. I doubt it
> matters all that much, but presumably the purpose in all is "how long
> before a user starts to wonder if things are actually happening", which
> is probably command-independent.

I feel comfortable moving forward, basing our decisions on the assumption that the "delay before the user wonders is independent of the command" without anybody actually proving it.

Even though I think that it is natural for people to expect longer delay from some operation than others (due to some chicken-and-egg reasons), trying to scientifically measure and to come up with different delay value that suited for each and every operation is waste of our brain cycles.

Perhaps we may want to replace the calls to progress_delay() with a call to a simpler wrapper that does not let the callers give their own delay threashold to simplify the API.

Junio C Hamano· Aug 14, 2017, 18:35 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

Junio C Hamano <gitster@pobox.com> writes:
> Perhaps we may want to replace the calls to progress_delay() with a
> call to a simpler wrapper that does not let the callers give their
> own delay threashold to simplify the API.

... which does not look too bad, but because it makes me wonder if we even need to make the delay-threshold customizable per callers, I'll wait further discussions before committing to the approach.

For example, I do not quite understand why 95 is a good value for prune-packed while 0 is a good value for prune.

The rename detection in diffcore-rename, delay in blame, and checkout (via unpack-trees) all of which use 50-percent threshold with 1 second delay, sort of make sense to me in that if we completed 50 percent within 1 second, it is likely we will finish all in 2 seconds (which is the norm for everybody else), perhaps as a better version of 0-percent 2 seconds rule.

 builtin/blame.c        |  3 +--
 builtin/fsck.c         |  2 +-
 builtin/prune-packed.c |  4 ++--
 builtin/prune.c        |  2 +-
 builtin/rev-list.c     |  2 +-
 diffcore-rename.c      |  4 ++--
 progress.c             | 10 ++++++++--
 progress.h             |  4 ++--
 unpack-trees.c         |  3 +--
 9 files changed, 19 insertions(+), 15 deletions(-)
Show changes to 9 files +19 −15

builtin/blame.c, builtin/fsck.c, builtin/prune-packed.c, builtin/prune.c, builtin/rev-list.c, diffcore-rename.c, progress.c, progress.h, unpack-trees.c

diff --git a/builtin/blame.c b/builtin/blame.c
index bda1a78726..05115900f3 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -925,8 +925,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 	sb.found_guilty_entry = &found_guilty_entry;
 	sb.found_guilty_entry_data = &pi;
 	if (show_progress)
-		pi.progress = start_progress_delay(_("Blaming lines"),
-						   sb.num_lines, 50, 1);
+		pi.progress = start_delayed_progress(_("Blaming lines"), sb.num_lines, 50);
 
 	assign_blame(&sb, opt);
 
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 99dea7adf6..6a26079ebc 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -188,7 +188,7 @@ static int traverse_reachable(void)
 	unsigned int nr = 0;
 	int result = 0;
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0, 0);
 	while (pending.nr) {
 		struct object_array_entry *entry;
 		struct object *obj;
diff --git a/builtin/prune-packed.c b/builtin/prune-packed.c
index ac978ad401..4fc6b175de 100644
--- a/builtin/prune-packed.c
+++ b/builtin/prune-packed.c
@@ -37,8 +37,8 @@ static int prune_object(const struct object_id *oid, const char *path,
 void prune_packed_objects(int opts)
 {
 	if (opts & PRUNE_PACKED_VERBOSE)
-		progress = start_progress_delay(_("Removing duplicate objects"),
-			256, 95, 2);
+		progress = start_delayed_progress(_("Removing duplicate objects"),
+						  256, 95);
 
 	for_each_loose_file_in_objdir(get_object_directory(),
 				      prune_object, NULL, prune_subdir, &opts);
diff --git a/builtin/prune.c b/builtin/prune.c
index c378690545..f32adaa0e9 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -138,7 +138,7 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0, 0);
 
 	mark_reachable_objects(&revs, 1, expire, progress);
 	stop_progress(&progress);
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 95d84d5cda..5cfc4b35f4 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -364,7 +364,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 		revs.limited = 1;
 
 	if (show_progress)
-		progress = start_progress_delay(show_progress, 0, 0, 2);
+		progress = start_delayed_progress(show_progress, 0, 0);
 
 	if (use_bitmap_index && !revs.prune) {
 		if (revs.count && !revs.left_right && !revs.cherry_mark) {
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 786f389498..0eafa43618 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -532,9 +532,9 @@ void diffcore_rename(struct diff_options *options)
 	}
 
 	if (options->show_rename_progress) {
-		progress = start_progress_delay(
+		progress = start_delayed_progress(
 				_("Performing inexact rename detection"),
-				rename_dst_nr * rename_src_nr, 50, 1);
+				rename_dst_nr * rename_src_nr, 50);
 	}
 
 	mx = xcalloc(st_mult(NUM_CANDIDATE_PER_DST, num_create), sizeof(*mx));
diff --git a/progress.c b/progress.c
index 73e36d4a42..eeebe15b6f 100644
--- a/progress.c
+++ b/progress.c
@@ -205,8 +205,8 @@ int display_progress(struct progress *progress, unsigned n)
 	return progress ? display(progress, n, NULL) : 0;
 }
 
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay)
+static struct progress *start_progress_delay(const char *title, unsigned total,
+					     unsigned percent_treshold, unsigned delay)
 {
 	struct progress *progress = malloc(sizeof(*progress));
 	if (!progress) {
@@ -227,6 +227,12 @@ struct progress *start_progress_delay(const char *title, unsigned total,
 	return progress;
 }
 
+struct progress *start_delayed_progress(const char *title,
+					unsigned total,	unsigned percent_treshold)
+{
+	return start_progress_delay(title, total, percent_treshold, 2);
+}
+
 struct progress *start_progress(const char *title, unsigned total)
 {
 	return start_progress_delay(title, total, 0, 0);
diff --git a/progress.h b/progress.h
index 611e4c4d42..c7fc431989 100644
--- a/progress.h
+++ b/progress.h
@@ -6,8 +6,8 @@ struct progress;
 void display_throughput(struct progress *progress, off_t total);
 int display_progress(struct progress *progress, unsigned n);
 struct progress *start_progress(const char *title, unsigned total);
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay);
+struct progress *start_delayed_progress(const char *title, unsigned total,
+					unsigned percent_treshold);
 void stop_progress(struct progress **progress);
 void stop_progress_msg(struct progress **progress, const char *msg);
 
diff --git a/unpack-trees.c b/unpack-trees.c
index dd535bc849..03d1b37578 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -343,8 +343,7 @@ static struct progress *get_progress(struct unpack_trees_options *o)
 			total++;
 	}
 
-	return start_progress_delay(_("Checking out files"),
-				    total, 50, 1);
+	return start_delayed_progress(_("Checking out files"), total, 50);
 }
 
 static int check_updates(struct unpack_trees_options *o)
Jeff King· Aug 14, 2017, 22:29 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

On Mon, Aug 14, 2017 at 11:35:33AM -0700, Junio C Hamano wrote:
Show 9 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Perhaps we may want to replace the calls to progress_delay() with a
> > call to a simpler wrapper that does not let the callers give their
> > own delay threashold to simplify the API.
> 
> ... which does not look too bad, but because it makes me wonder if
> we even need to make the delay-threshold customizable per callers,
> I'll wait further discussions before committing to the approach.

For what it's worth, I had a similar "why is this even part of the API" thought when writing earlier in the thread.

> For example, I do not quite understand why 95 is a good value for
> prune-packed while 0 is a good value for prune.
And I also wondered this same thing. I do not have a good answer.
Show 6 quoted lines
> The rename detection in diffcore-rename, delay in blame, and
> checkout (via unpack-trees) all of which use 50-percent threshold
> with 1 second delay, sort of make sense to me in that if we
> completed 50 percent within 1 second, it is likely we will finish
> all in 2 seconds (which is the norm for everybody else), perhaps as
> a better version of 0-percent 2 seconds rule.
Just thinking what could go wrong for a moment.

If we reach 51% in one second, would we then never show progress? That seems like a misfeature when the counter is spiky. E.g., consider something like object transfer (which doesn't do a delayed progress, but pretend for a moment as it makes a simple example). Imagine we have 100 objects, 99 of which are 10KB and one of which is 100MB. And that the big object comes at slot 75.

No matter how the delay works, the counter is going to jump quickly to 75% as we send the small objects, and then appear to stall on the large one. That's unavoidable without feeding the progress code more data about the items. But what does the user see?

With (0,2), we start the progress meter at 2 seconds, the user sees it stall at 75%, and then eventually it unclogs.

With (50,1), we check the percentage after 1 second and see we are already at 75%. We then disable the meter totally. After 2 seconds, we get the same stall but the user sees nothing.

So in that case we should always base our decision only on time taken.

And that gives me an answer for your question above: the difference is whether we expect the counter to move smoothly, or to be spiky.

If it's smooth, the (50,1) case is slightly nicer in that it puts the progress in front of the user more quickly. I'm not sure if that's actually worth pushing an additional decision onto the person writing the calling code, though (especially when we are just now puzzling out the method for making such a decision from first principles).

So I'd vote to drop that parameter entirely. And if 1 second seems noticeably snappier, then we should probably just move everything to a 1 second delay (I don't have a strong feeling either way).

-Peff
Junio C Hamano· Aug 14, 2017, 22:42 UTC · re: Jeff King · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

Jeff King <peff@peff.net> writes:
Show 9 quoted lines
> If it's smooth, the (50,1) case is slightly nicer in that it puts the
> progress in front of the user more quickly. I'm not sure if that's
> actually worth pushing an additional decision onto the person writing
> the calling code, though (especially when we are just now puzzling out
> the method for making such a decision from first principles).
>
> So I'd vote to drop that parameter entirely. And if 1 second seems
> noticeably snappier, then we should probably just move everything to a 1
> second delay (I don't have a strong feeling either way).
Sounds like a good idea to me.  

I've already locally tweaked Kevin's patch to use (0,2) instead of (0,1) without introducing the simpler wrapper. It should be trivial to do a wrapper to catch and migrate all the (0,2) users to a start_delayed_progress() that takes neither percentage or time with mechanical replacement.

Jeff King· Aug 14, 2017, 23:08 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

On Mon, Aug 14, 2017 at 03:42:14PM -0700, Junio C Hamano wrote:
Show 19 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> > If it's smooth, the (50,1) case is slightly nicer in that it puts the
> > progress in front of the user more quickly. I'm not sure if that's
> > actually worth pushing an additional decision onto the person writing
> > the calling code, though (especially when we are just now puzzling out
> > the method for making such a decision from first principles).
> >
> > So I'd vote to drop that parameter entirely. And if 1 second seems
> > noticeably snappier, then we should probably just move everything to a 1
> > second delay (I don't have a strong feeling either way).
> 
> Sounds like a good idea to me.  
> 
> I've already locally tweaked Kevin's patch to use (0,2) instead of
> (0,1) without introducing the simpler wrapper.  It should be trivial
> to do a wrapper to catch and migrate all the (0,2) users to a
> start_delayed_progress() that takes neither percentage or time with
> mechanical replacement.

I was actually proposing to move (50,1) cases to the simpler wrapper, too. IOW, drop the delayed_percent_treshold code entirely.

-Peff
Junio C Hamano· Aug 14, 2017, 23:23 UTC · re: Jeff King · lore

Re: [PATCH v2 1/2] format-patch: have progress option while generating patches

Jeff King <peff@peff.net> writes:
Show 24 quoted lines
> On Mon, Aug 14, 2017 at 03:42:14PM -0700, Junio C Hamano wrote:
>
>> Jeff King <peff@peff.net> writes:
>> 
>> > If it's smooth, the (50,1) case is slightly nicer in that it puts the
>> > progress in front of the user more quickly. I'm not sure if that's
>> > actually worth pushing an additional decision onto the person writing
>> > the calling code, though (especially when we are just now puzzling out
>> > the method for making such a decision from first principles).
>> >
>> > So I'd vote to drop that parameter entirely. And if 1 second seems
>> > noticeably snappier, then we should probably just move everything to a 1
>> > second delay (I don't have a strong feeling either way).
>> 
>> Sounds like a good idea to me.  
>> 
>> I've already locally tweaked Kevin's patch to use (0,2) instead of
>> (0,1) without introducing the simpler wrapper.  It should be trivial
>> to do a wrapper to catch and migrate all the (0,2) users to a
>> start_delayed_progress() that takes neither percentage or time with
>> mechanical replacement.
>
> I was actually proposing to move (50,1) cases to the simpler wrapper,
> too. IOW, drop the delayed_percent_treshold code entirely.

I should have mentioned that (50,1) should also be treated just like (0,2) case--they should mean the same thing. Other oddness like 95 might merit individual consideration, and I didn't want to add (0,1) as another oddball to the mix.

Junio C Hamano· Aug 19, 2017, 17:39 UTC · re: Jeff King · lore

[PATCH] progress: simplify "delayed" progress API

We used to expose the full power of the delayed progress API to the callers, so that they can specify, not just the message to show and expected total amount of work that is used to compute the percentage of work performed so far, the percent-threshold parameter P and the delay-seconds parameter N. The progress meter starts to show at N seconds into the operation only if the amount of work completed exceeds P.

Most callers used either (0%, 2s) or (50%, 1s) as (P, N), but there are oddballs that chose more random-looking values like 95%.

For a smoother workload, (50%, 1s) would allow us to start showing the progress meter earlier than (0%, 2s), while keeping the chance of not showing progress meter for long running operation the same as the latter. For a task that would take 2s or more to complete, it is likely that less than half of it would complete within the first second, if the workload is smooth. But for a spiky workload whose earlier part is easier, such a setting is likely to fail to show the progress meter entirely and (0%, 2s) is more appropriate.

But that is merely a theory. Realistically, it is of dubious value to ask each codepath to carefully consider smoothness of their workload and specify their own setting by passing two extra parameters. Let's simplify the API by dropping both parameters and have everybody use (0%, 2s).

Oh, by the way, the percent-threshold parameter and the structure member were consistently misspelled, which also is now fixed ;-)

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 * So before I forget all about this topic, here is a patch to
   actually do this.
   > So I'd vote to drop that parameter entirely. And if 1 second seems
   > noticeably snappier, then we should probably just move everything to a 1
   > second delay (I don't have a strong feeling either way).
   I was also tempted to do this, but decided to keep it closer to
   the original for majority of callers by leaving the delay at 2s.
   With this patch, such a change as a follow-up is made a lot
   easier (somebody may want to even make a configuration out of it,
   but that would not be me).
 builtin/blame.c        |  3 +--
 builtin/fsck.c         |  2 +-
 builtin/prune-packed.c |  3 +--
 builtin/prune.c        |  2 +-
 builtin/rev-list.c     |  2 +-
 diffcore-rename.c      |  4 ++--
 progress.c             | 15 ++++++++++-----
 progress.h             |  3 +--
 unpack-trees.c         |  3 +--
 9 files changed, 19 insertions(+), 18 deletions(-)
Show changes to 9 files +19 −18

builtin/blame.c, builtin/fsck.c, builtin/prune-packed.c, builtin/prune.c, builtin/rev-list.c, diffcore-rename.c, progress.c, progress.h, unpack-trees.c

diff --git a/builtin/blame.c b/builtin/blame.c
index bda1a78726..e0daf17548 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -925,8 +925,7 @@ int cmd_blame(int argc, const char **argv, const char *prefix)
 	sb.found_guilty_entry = &found_guilty_entry;
 	sb.found_guilty_entry_data = &pi;
 	if (show_progress)
-		pi.progress = start_progress_delay(_("Blaming lines"),
-						   sb.num_lines, 50, 1);
+		pi.progress = start_delayed_progress(_("Blaming lines"), sb.num_lines);
 
 	assign_blame(&sb, opt);
 
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 99dea7adf6..0031439fc4 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -188,7 +188,7 @@ static int traverse_reachable(void)
 	unsigned int nr = 0;
 	int result = 0;
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0);
 	while (pending.nr) {
 		struct object_array_entry *entry;
 		struct object *obj;
diff --git a/builtin/prune-packed.c b/builtin/prune-packed.c
index ac978ad401..8f41f7c20e 100644
--- a/builtin/prune-packed.c
+++ b/builtin/prune-packed.c
@@ -37,8 +37,7 @@ static int prune_object(const struct object_id *oid, const char *path,
 void prune_packed_objects(int opts)
 {
 	if (opts & PRUNE_PACKED_VERBOSE)
-		progress = start_progress_delay(_("Removing duplicate objects"),
-			256, 95, 2);
+		progress = start_delayed_progress(_("Removing duplicate objects"), 256);
 
 	for_each_loose_file_in_objdir(get_object_directory(),
 				      prune_object, NULL, prune_subdir, &opts);
diff --git a/builtin/prune.c b/builtin/prune.c
index c378690545..cddabf26a9 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -138,7 +138,7 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (show_progress)
-		progress = start_progress_delay(_("Checking connectivity"), 0, 0, 2);
+		progress = start_delayed_progress(_("Checking connectivity"), 0);
 
 	mark_reachable_objects(&revs, 1, expire, progress);
 	stop_progress(&progress);
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 95d84d5cda..dfad8e847a 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -364,7 +364,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 		revs.limited = 1;
 
 	if (show_progress)
-		progress = start_progress_delay(show_progress, 0, 0, 2);
+		progress = start_delayed_progress(show_progress, 0);
 
 	if (use_bitmap_index && !revs.prune) {
 		if (revs.count && !revs.left_right && !revs.cherry_mark) {
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 786f389498..0d8c3d2ee4 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -532,9 +532,9 @@ void diffcore_rename(struct diff_options *options)
 	}
 
 	if (options->show_rename_progress) {
-		progress = start_progress_delay(
+		progress = start_delayed_progress(
 				_("Performing inexact rename detection"),
-				rename_dst_nr * rename_src_nr, 50, 1);
+				rename_dst_nr * rename_src_nr);
 	}
 
 	mx = xcalloc(st_mult(NUM_CANDIDATE_PER_DST, num_create), sizeof(*mx));
diff --git a/progress.c b/progress.c
index 73e36d4a42..289678d43d 100644
--- a/progress.c
+++ b/progress.c
@@ -34,7 +34,7 @@ struct progress {
 	unsigned total;
 	unsigned last_percent;
 	unsigned delay;
-	unsigned delayed_percent_treshold;
+	unsigned delayed_percent_threshold;
 	struct throughput *throughput;
 	uint64_t start_ns;
 };
@@ -88,7 +88,7 @@ static int display(struct progress *progress, unsigned n, const char *done)
 			return 0;
 		if (progress->total) {
 			unsigned percent = n * 100 / progress->total;
-			if (percent > progress->delayed_percent_treshold) {
+			if (percent > progress->delayed_percent_threshold) {
 				/* inhibit this progress report entirely */
 				clear_progress_signal();
 				progress->delay = -1;
@@ -205,8 +205,8 @@ int display_progress(struct progress *progress, unsigned n)
 	return progress ? display(progress, n, NULL) : 0;
 }
 
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay)
+static struct progress *start_progress_delay(const char *title, unsigned total,
+					     unsigned percent_threshold, unsigned delay)
 {
 	struct progress *progress = malloc(sizeof(*progress));
 	if (!progress) {
@@ -219,7 +219,7 @@ struct progress *start_progress_delay(const char *title, unsigned total,
 	progress->total = total;
 	progress->last_value = -1;
 	progress->last_percent = -1;
-	progress->delayed_percent_treshold = percent_treshold;
+	progress->delayed_percent_threshold = percent_threshold;
 	progress->delay = delay;
 	progress->throughput = NULL;
 	progress->start_ns = getnanotime();
@@ -227,6 +227,11 @@ struct progress *start_progress_delay(const char *title, unsigned total,
 	return progress;
 }
 
+struct progress *start_delayed_progress(const char *title, unsigned total)
+{
+	return start_progress_delay(title, total, 0, 2);
+}
+
 struct progress *start_progress(const char *title, unsigned total)
 {
 	return start_progress_delay(title, total, 0, 0);
diff --git a/progress.h b/progress.h
index 611e4c4d42..6392b63371 100644
--- a/progress.h
+++ b/progress.h
@@ -6,8 +6,7 @@ struct progress;
 void display_throughput(struct progress *progress, off_t total);
 int display_progress(struct progress *progress, unsigned n);
 struct progress *start_progress(const char *title, unsigned total);
-struct progress *start_progress_delay(const char *title, unsigned total,
-				       unsigned percent_treshold, unsigned delay);
+struct progress *start_delayed_progress(const char *title, unsigned total);
 void stop_progress(struct progress **progress);
 void stop_progress_msg(struct progress **progress, const char *msg);
 
diff --git a/unpack-trees.c b/unpack-trees.c
index dd535bc849..e5ae7fe183 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -343,8 +343,7 @@ static struct progress *get_progress(struct unpack_trees_options *o)
 			total++;
 	}
 
-	return start_progress_delay(_("Checking out files"),
-				    total, 50, 1);
+	return start_delayed_progress(_("Checking out files"), total);
 }
 
 static int check_updates(struct unpack_trees_options *o)
-- 
2.14.1-396-gef255185d2
Junio C Hamano· Aug 19, 2017, 20:58 UTC · re: Junio C Hamano · lore

Re: [PATCH] progress: simplify "delayed" progress API

Junio C Hamano <gitster@pobox.com> writes:
Show 7 quoted lines
> We used to expose the full power of the delayed progress API to the
> callers, so that they can specify, not just the message to show and
> expected total amount of work that is used to compute the percentage
> of work performed so far, the percent-threshold parameter P and the
> delay-seconds parameter N.  The progress meter starts to show at N
> seconds into the operation only if the amount of work completed
> exceeds P.

Oops, we inspect the doneness at N seconds and show only if we have not yet completed P percent of the total work.

Perhaps
	... at N seconds into the operation only if we have not
	completed P per-cent of the total work.
Jeff King· Aug 20, 2017, 07:43 UTC · re: Junio C Hamano · lore

Re: [PATCH] progress: simplify "delayed" progress API

On Sat, Aug 19, 2017 at 10:39:41AM -0700, Junio C Hamano wrote:
Show 25 quoted lines
> We used to expose the full power of the delayed progress API to the
> callers, so that they can specify, not just the message to show and
> expected total amount of work that is used to compute the percentage
> of work performed so far, the percent-threshold parameter P and the
> delay-seconds parameter N.  The progress meter starts to show at N
> seconds into the operation only if the amount of work completed
> exceeds P.
> 
> Most callers used either (0%, 2s) or (50%, 1s) as (P, N), but there
> are oddballs that chose more random-looking values like 95%.
> 
> For a smoother workload, (50%, 1s) would allow us to start showing
> the progress meter earlier than (0%, 2s), while keeping the chance
> of not showing progress meter for long running operation the same as
> the latter.  For a task that would take 2s or more to complete, it
> is likely that less than half of it would complete within the first
> second, if the workload is smooth.  But for a spiky workload whose
> earlier part is easier, such a setting is likely to fail to show the
> progress meter entirely and (0%, 2s) is more appropriate.
> 
> But that is merely a theory.  Realistically, it is of dubious value
> to ask each codepath to carefully consider smoothness of their
> workload and specify their own setting by passing two extra
> parameters.  Let's simplify the API by dropping both parameters and
> have everybody use (0%, 2s).

Nicely explained (modulo the reversal you noticed in the first paragraph). The patch looks good to me.

> Oh, by the way, the percent-threshold parameter and the structure
> member were consistently misspelled, which also is now fixed ;-)
Heh, that was bugging me, too. :)
Show 12 quoted lines
>  * So before I forget all about this topic, here is a patch to
>    actually do this.
> 
>    > So I'd vote to drop that parameter entirely. And if 1 second seems
>    > noticeably snappier, then we should probably just move everything to a 1
>    > second delay (I don't have a strong feeling either way).
> 
>    I was also tempted to do this, but decided to keep it closer to
>    the original for majority of callers by leaving the delay at 2s.
>    With this patch, such a change as a follow-up is made a lot
>    easier (somebody may want to even make a configuration out of it,
>    but that would not be me).
Yeah, I agree that it can come later on top (if at all).
Thanks for finishing this off.
-Peff
Junio C Hamano· Aug 10, 2017, 22:48 UTC · re: Kevin Willford · lore

Re: [PATCH v2 0/2] Add progress for format-patch and rebase

Kevin Willford <kcwillford@gmail.com> writes:
Show 11 quoted lines
> Changes since last patch:
> 1. Use start_progress_delay so progress isn't shown if generating
>    the patches is fast enough
> 2. Updated to have text of "Generating patches"
> 3. Only show progress when the --progress flag is passed
> 4. In the rebase script check stderr and the quiet option is not
>    set before propagating the progress flag to format-patch
>
> Kevin Willford (2):
>   format-patch: have progress option while generating patches
>   rebase: turn on progress option by default for format-patch

Do you have a pointer to the previous discussion handy (if you do not, that is OK---I think I can dig the list archive myself)?

Also, your patch messages appear from you at gmail yet signed off by you at microsoft. Please make them consistent (I can fix them up while queuing _this_ time, but I do not want forever doing that for future patches from you or other people). One easy workaround to do so without convincing your mailer to put your microsoft address on the sender's From: line in the e-mail is to put an extra

    From: Kevin Without <kewillf@microsoft.com>

line, followed by a blank line, at the very beginning of the body of the e-mail message.

Thanks.
Jeff King· Aug 10, 2017, 23:17 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 0/2] Add progress for format-patch and rebase

On Thu, Aug 10, 2017 at 03:48:31PM -0700, Junio C Hamano wrote:
Show 16 quoted lines
> Kevin Willford <kcwillford@gmail.com> writes:
> 
> > Changes since last patch:
> > 1. Use start_progress_delay so progress isn't shown if generating
> >    the patches is fast enough
> > 2. Updated to have text of "Generating patches"
> > 3. Only show progress when the --progress flag is passed
> > 4. In the rebase script check stderr and the quiet option is not
> >    set before propagating the progress flag to format-patch
> >
> > Kevin Willford (2):
> >   format-patch: have progress option while generating patches
> >   rebase: turn on progress option by default for format-patch
> 
> Do you have a pointer to the previous discussion handy (if you do
> not, that is OK---I think I can dig the list archive myself)?
https://public-inbox.org/git/20170531150427.7820-1-kewillf@microsoft.com/
is what I turned up.

Overall this version looks good to me, and addresses all of the previous points. I have two minor points which I'll make inline.

-Peff

← back to recent threads