patch, 2 partsworktree: fix incorrectly-ordered messages on Windows

This patch series fixes a problem in which a chatty "Preparing worktree" message and a subsequent fatal error message appear in the wrong order on Microsoft Windows, which may confuse readers into thinking that the operation somehow succeeded despite the error.

Unlike the original RFC attempt[*] to fix this problem at a low level in a generalized fashion, patch [1/2] localizes the fix to git-worktree itself by making it conform to common Git practice of issuing chatty messages to stderr rather than to stdout as is currently the case.

Patch [2/2] is just a drive-by fix for a minor documentation problem I noticed along the way.

[*]: https://lore.kernel.org/git/20211130043946.19987-1-sunshine@sunshineco.com/

Eric Sunshine (2):
  worktree: send "chatty" messages to stderr
  git-worktree.txt: add missing `-v` to synopsis for `worktree list`

 Documentation/git-worktree.txt |  2 +-
 builtin/worktree.c             | 14 +++++++-------
 t/t2401-worktree-prune.sh      | 14 +++++++-------
 t/t2402-worktree-list.sh       |  2 +-
 t/t2406-worktree-repair.sh     | 30 ++++++++++++------------------
 5 files changed, 28 insertions(+), 34 deletions(-)

The order in which the stdout and stderr streams are flushed is not guaranteed to be the same across platforms or `libc` implementations. This lack of determinism can lead to anomalous and potentially confusing output if normal (stdout) output is flushed after error (stderr) output. For instance, the following output which clearly indicates a failure due to a fatal error:

    % git worktree add ../foo bar
    Preparing worktree (checking out 'bar')
    fatal: 'bar' is already checked out at '.../wherever'

has been reported[1] on Microsoft Windows to appear as:

    % git worktree add ../foo bar
    fatal: 'bar' is already checked out at '.../wherever'
    Preparing worktree (checking out 'bar')

which may confuse the reader into thinking that the command somehow recovered and ran to completion despite the error.

This problem crops up because the "chatty" status message "Preparing worktree" is sent to stdout, whereas the "fatal" error message is sent to stderr. One way to fix this would be to flush stdout manually before git-worktree reports any errors to stderr.

However, common practice in Git is for "chatty" messages to be sent to stderr. Therefore, a more appropriate fix is to adjust git-worktree to conform to that practice by sending its "chatty" messages to stderr rather than stdout as is currently the case.

There may be concern that relocating messages from stdout to stderr could break existing tooling, however, these messages are already internationalized, thus are unstable. And, indeed, the "Preparing worktree" message has already been the subject of somewhat significant changes in 2c27002a0a (worktree: improve message when creating a new worktree, 2018-04-24). Moreover, there is existing precedent, such as 68b939b2f0 (clone: send diagnostic messages to stderr, 2013-09-18) which likewise relocated "chatty" messages from stdout to stderr for git-clone.

[1]: https://lore.kernel.org/git/CA+34VNLj6VB1kCkA=MfM7TZR+6HgqNi5-UaziAoCXacSVkch4A@mail.gmail.com/T/

Reported-by: Baruch Burstein <bmburstein@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 builtin/worktree.c         | 14 +++++++-------
 t/t2401-worktree-prune.sh  | 14 +++++++-------
 t/t2402-worktree-list.sh   |  2 +-
 t/t2406-worktree-repair.sh | 30 ++++++++++++------------------
 4 files changed, 27 insertions(+), 33 deletions(-)

On Thu, Dec 02 2021, Eric Sunshine wrote:

Makes sense.

This is just a "for bonus points", but maybe we could/should while we're at it harden and make the tests more exhaustive by checking the full output of both, e.g.

	cat >actual.out <<-\EOF &&
	Preparing worktree (checking out 'bar')
	EOF
	cat >actual.err <<-\EOF &&
	fatal: 'bar' is already checked out at '.../wherever'
	EOF
        <cmd> [...]
	test_cmp expect.out actual.out &&
	test_cmp expect.err actual.err

Doesn't need a re-roll etc., just if you're interested... :)

On Fri, Dec 3, 2021 at 4:19 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

To be clear, with the application of the current patch, both of those messages would need to be in the `actual.err` file, and `actual.out` would be empty; not one message in each file as in your snippet.

That aside, there's still potentially output which is outside the control of git-worktree. In the case of this particular negative test, your suggestion should work, but for a positive test, it would be harder and uglier (though, of course, not impossible). For instance, for a successful `git worktree add`, the output is:

    Preparing worktree (new branch 'foobar')
    Updating files: 100% (3993/3993), done.
    HEAD is now at abe6bb3905 Gobbledygook

The subsequent lines come from git-reset (which, by the way, is sending "HEAD is now at" to stdout, though they probably should be on stderr, but that's a separate issue).

Anyhow, such a change to the tests should be a separate topic. The user-facing problem addressed by the current patch series need not be held up by a behind-the-scenes change to testing.

When verbose mode was added to `git worktree list` by 076b444a62 (worktree: teach `list` verbose mode, 2021-01-27), although the documentation was updated to reflect the new functionality, the synopsis was overlooked. Correct this minor oversight.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 Documentation/git-worktree.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

On Thu, Dec 02 2021, Eric Sunshine wrote:

Looks good!

Aside: I've been thinking of hacking something up to just change all
these "[verse]" bits in the *.txt source to:

    [verse]
    $(git worktree -h)

And then have the doc build process pick that up, run 'git $name -h', do some light search/replacement (e.g. "$cmd" to "'$cmd'") and build the docs like that.

Seems far preferrable to dual-maintaining all of these forever.

But in the meantime this small fix is obviously correct.

On Fri, Dec 3, 2021 at 4:15 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

One caution that springs to mind is that there may be external tooling which processes these documentation files directly, and such a change might break them. (The one which popped to mind immediately was the git-scm.{org,com} website, though I don't know what their tooling looks like.)

On Fri, Dec 03 2021, Eric Sunshine wrote:

True, I hadn't looked into how that worked before, but behold! https://github.com/git/git-scm.com/blob/main/lib/tasks/index.rake

It seems to be a re-implementation of a non-trivial part of the doc building process.

In any case, if we do end up generating more of the documentation ourselves presumably any such concerns will be brought up then.

On Fri, Dec 03, 2021 at 03:57:41PM +0100, Ævar Arnfjörð Bjarmason wrote:

Yeah. It's kind of ugly, but the complication there is that the docs are updated on a running heroku dyno which does not actually have a clone of the new version of Git, let alone an actual build.

It's also hard for it to just use the output of our "make" anyway, since there's some munging that happens to fit the page content inside the rest of the site, changing links, etc. We could in theory operate on the result of "make html" more directly, but it would definitely require some changes.

IMHO the way the site operates now (with "live" updates by extracting content from git.git and shoving it into a database) is not ideal. It's not like we're importing new Git versions once per minute. It would be easier to reason about as a "static site" which is built by a process which actually has a clone of git.git and invokes "make html" there, post-processes the pages, and saves the whole thing as a Git tree. And then run that build occasionally (at new releases, or changes to the source, but also periodically via GitHub Actions or similar to pick up changed book content). It's just a big enough change (and there are some gotcha around things like site search) that I've never gotten around to it.

-Peff

Eric Sunshine <sunshine@sunshineco.com> writes:

Also it would slow us down by making the .txt variant we see in the source tree harder to read (or in this case, impossible to see without building the documentation).

On Sun, Dec 5, 2021 at 4:12 AM Junio C Hamano <gitster@pobox.com> wrote:

Taking this point into consideration, a middle-ground alternative to Ævar's idea would be to add tooling which only compares (by some definition of "compare") the output of `git blah -h` with the synopsis in `git-blah.txt` and complains if there are significant differences (by some definition "significant" and "difference"). It doesn't automate-away the work of keeping the synopsis up-to-date, but at least would flag inconsistencies.

On Mon, Dec 06 2021, Eric Sunshine wrote:

Or we could do the reverse and move the source code version of it to be generated from the [verse] sections in the documentation.

Anyway, it's not something I was planning to work on any time soon, just something I'd thought was a good idea for a while, especially given the differences and divergenge. That can be viewed with:

    parallel -k '
        git {} -h 2>&1 | grep -e "^usage" -e "^   or";
        git help {} 2>&1 | grep -A20 SYNOPSIS | grep -B20 DESCRIPTION
    ' ::: $(git --list-cmds=builtins) | less

It's a long-standing UX issue, and we keep re-introducing divergence between the two.

If we're going to insist that the version in the *.txt file isn't generated that categorically closes the door to some logical follow-ups.

E.g. having parse-options automatically generate alternates in cases where options are mutually exclusive, or adding color output to this where we color short/long options differently than arguments etc.

That and e.g. translators needing to do less work for the translated manpages (we already have the translated output in the C code).

Well, I suppose we could have a generating step and then commit the equivalent of the compiled file (or section) into git.git every time we add/change an option.

In general I don't think it's a worthwhile goal to keep the .txt versions of the docs as some human-readable 1=1 mapping to what you'd get if you generated them. That's already not the case due to includes, and e.g. in this case accepting some reasonable amount of auto-generation would make them easier to maintain.

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

Either would be far more preferrable than committing generated files, or committing us to work on unreadable sources.

Thanks.

Eric Sunshine <sunshine@sunshineco.com> writes:

Oops! Thanks for cleaning up after me.