Re: [PATCH v2 1/3] test-lib: allow selecting tests by substring/regex with --run

From: Jeff King <peff@peff.net>
Date: Oct 16, 2020, 20:06 UTC
Message-ID: <20201016200642.GB3355643@coredump.intra.peff.net>
In-Reply-To: <xmqq7drqdu2u.fsf@gitster.c.googlers.com>

On Fri, Oct 16, 2020 at 09:16:57AM -0700, Junio C Hamano wrote:

Show 11 quoted lines

> > I'm not sure where he suggests worse names. I'd think if anything we'd
> > have better names, because they'd be even more meaningful (if people
> > start using them for test selectors). FWIW, I also grep like that when
> > looking for scripts.
> 
> I didn't mean Jonathan suggested worse names.  Unlike "I don't tend
> to discover test scripts based on their filename", which was what
> Jonathan said, I do look for tests based on their filename, so
> having a good name matters (on the other hand, if you are the kind
> of person who does not look for them by name, the naming may not
> matter to you).

Ah, I misunderstood. Thanks for clarifying (and I am very much in your camp that the names are useful).

Show 10 quoted lines

> As long as it is known that "filesystem" and "update-ref" can serve
> as tokens to uniquely identify these two tests, it would be fine for
> my purpose.  But 0050 (under the rule that numbering must be unique)
> would give me such an assurance much better without having to look
> at any other test file.
> 
> The word "filesystem"?  Unless we have a rule that we can use each
> unique word in test names only once (which of course is impractical)
> I am not sure I can use it in place of 0050 without checking names
> of other tests first.

With your follow-up response:

> So if the rule is to always spell out the full name if I wanted
> uniqueness, then it would work.

I think we are on the same page, and my intent was to match full names.

So now you get "t0050" from some failed-test output (prove, or just the output from make failing), and you copy it into the command-line to use with "--test". And instead you'd just copy the full text name. It's a little less convenient because t1234 is short enough that I'd type it, and I'd probably cut-and-paste the text name. But other than that, I'd expect the procedure to be the same.

The substring matches added by Elijah's series make sense for individual test snippets within a script, I think. And I think we could even add script-name matching now[1], without getting rid of the numbers. But if we do so, we should be careful to introduce it as an anchored match and not a substring match, to avoid having to switch it later.

-Peff

[1] And by "now" I don't mean we should hold up Elijah's patches for this
    feature, but that anybody who wishes to build it on top is free to
    do so without us having to make a decision on ditching the numbers
    entirely.