Re: [PATCH] cmd-list.perl: fix identifying man sections

From: Martin Ågren <martin.agren@gmail.com>
Date: Sep 26, 2022, 07:16 UTC
Message-ID: <CAN0heSpPGuSYb1nEyA3RuV+ACrXjHu+YqxAsfZN62EHVqcQ9Mg@mail.gmail.com>
In-Reply-To: <Yy4uDW9YjpleFUHW@coredump.intra.peff.net>

On Sat, 24 Sept 2022 at 00:07, Jeff King <peff@peff.net> wrote:

Show 18 quoted lines

>
> I wondered if we might have caught this in a more automatic way. The
> output of:
>
>   cd Documentation
>   ./doc-diff cc75e556a9^ cc75e556a9
>
> makes the problem apparent, but I don't fault reviewers for not running
> it. I rarely remember to do so. And in general you need a human looking
> at doc-diff output to know if it was the intended change or not.
>
> I wondered if it might be worth running
>
>   ./doc-diff v2.37.0 v2.38.0-rc1
>
> near a release to scan over all of the changes. But the diff is over
> 8000 lines, and I admit my eyes glazed over before I got to the
> problematic hunks (even though I knew I was looking for them!).

I know how you felt when you looked at that doc-diff... I was lucky enough to first look at

  ./doc-diff v2.37.0 v2.38.0-rc0

so that when I then looked at

  ./doc-diff v2.38.0-rc0 v2.38.0-rc1

this stood out quite well.

> So I dunno. I think doc-diff is a potentially useful tool, but I'm not
> sure how to point the human attention at the right spot to find a bug.

I try to do some such doc-diffing every now and then, and in particular around rc time. It has caught a few buglets, usually nothing major.

I've also done

  ./doc-diff origin/next origin/seen

at times to catch such things a lot earlier, but it's not often that I find the time to do so / think about doing it.

> Maybe "given enough eyeballs, all bugs are shallow" is our best bet
> here. After all, it did find this bug before the release. :)

Yeah, and luck never hurts -- I think it's fair to assume that I would have missed this bug if it hadn't come in after rc0.

Martin