Re: [PATCH 0/7] [RFC] advice: refuse to output if stderr not TTY

From: Patrick Steinhardt <ps@pks.im>
Date: Aug 22, 2024, 06:19 UTC
Message-ID: <ZsbYYo3pLUAmBU0e@tanuki>
In-Reply-To: <xmqqbk1l25p3.fsf@gitster.g>

On Wed, Aug 21, 2024 at 09:36:56AM -0700, Junio C Hamano wrote:

Show 43 quoted lines

> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
> > Advice is supposed to be for humans, not machines. Why do we output it when
> > stderr is not a terminal? Let's stop doing that.
> 
> Last night while skimming the series on my phone (read: not a real
> review at all), I found it very annoying that GIT_ADVICE=1 had to be
> sprinkled all over the place.  I wonder if we want to instead set
> and export it in t/test-lib.sh and turn it off as needed?
> 
> The end-to-end tests we have are primarily to guarantee the
> continuity of the end-user experience by humans, and ensuring that
> an advice message is given when appropriate and it does not get
> shown otherwise is very much inherent part of them.  An alternative
> workaround to counteract the breakage this series causes of course
> is to run everything under test_terminal and it probably is much
> more kosher philosophically ;-), but compared to that, globally
> disabling the "if (!isatty(2))" while running the tests, and
> temporarily lifting that disabling during tests of the new feature
> added by this series would be easier to reason about, I would
> suspect.
> 
> > This series is motivated by an internal tool breaking due to the advice
> > message added to Git 2.46.0 by 9479a31d603 (advice: warn when sparse index
> > expands, 2024-07-08). This tool is assuming that any output to stderr is an
> > error, and in this case is attempting to parse it to determine what kind of
> > error (warning, error, or failure).
> 
> The "anything on stderr is an error" attitude needs to be fixed
> regardless of where it comes from (tcl/tk scripts have, or at least
> used to have, the tendency, which I found annoying), but regardless,
> I thought we added a mechanism to squelch all advice messages for
> this exact purpose at f0e21837 (Merge branch 'jl/git-no-advice',
> 2024-05-16).  Why isn't the tool using the mechanism that already
> exists?
> 
> I would have supported the behaviour proposed by this series 100% if
> it were on the table when we were introducing the advise mechanism,
> but unfortunately nobody seemed have suggested it back then.  I am
> willing to go with an "experiment" to change the behaviour,
> deliberately breaking "backward compatibility", if we have a wide
> support here during the review period.  FWIW, I think any scripts
> that scrape the advice messages are already broken.

I continue to believe that the biggest issue in this context is that there is no proper interface between Git and its caller that would allow the caller to learn about errors in a machine-parseable way. Matching error messages against regular expressions is bad, and can easily be broken by the output changing in whatever way. This may be because the error message itself was changed, or it may be because we have started to show advice messages. It's extremely fragile, and from my point of view there is no good way to classify errors right now.

I won't argue that checking whether stderr is empty or not is good -- it almost certainly feels wrong to me. But that's only one small part of a more widespread issue. Having structured error handling in Git, e.g. via a new structure that represents errors as discussed a couple of months ago [1] would go a long way. I didn't quite like the approach chosen by that patch series, but think that the idea certainly has merit.

The other question is why advice is being shown in the first place. In theory, all one should ever use in scripted usecases are plumbing tools. And as plumbing tools are explicitly not designed for users, they should never show advice in the first place. I guess chances are high though that the scripts in question used porcelain. That is also understandable though: our plumbing tools are often not as powerful as the porcelain ones, which has been lamented on the mailing list several times.

So I certainly get the sentiment of this patch series, but feel like we continue to work around the underlying problems. Those are rooted rather deep though, so fixing them is nothing we can do in a release or two, but rather on the order of years. Meanwhile I guess we have to find short-term solutions.

Patrick

[1]: https://lore.kernel.org/git/pull.1666.git.git.1708241612.gitgitgadget@gmail.com/