threads / patch / 61191

patcht/README: mention test files are make targets

Subject: [PATCH] t/README: mention test files are make targets

## tl;dr

6 messages between Mar 24, 2024 and Mar 25, 2024. Diffs are folded; open one to read it.

replies: 5people: 3as markdown or json

Philippe Blain via GitGitGadget· Mar 24, 2024, 15:14 UTC · lore

From: Philippe Blain <levraiphilippeblain@gmail.com>

Since 23fc63bf8f (make tests ignorable with "make -i", 2005-11-08), each test file defines a target in the test Makefile, such that one can invoke:

	make *checkout*

to run all tests with 'checkout' in their filename. This is useful to run a subset of tests when you have a good idea of what part of the code is touched by the changes your are testing.

Document that in t/README to help new (or more seasoned) contributors that might not be aware.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
---
    t/README: mention test files are make targets

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1701%2Fphil-blain%2Ftests-makefile-targets-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1701/phil-blain/tests-makefile-targets-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1701

 t/README | 7 +++++++
 1 file changed, 7 insertions(+)

Show changes to t/README +7 −0

diff --git a/t/README b/t/README
index 621d3b8c095..71211109338 100644
--- a/t/README
+++ b/t/README
@@ -32,6 +32,13 @@ the tests.
     ok 2 - plain with GIT_WORK_TREE
     ok 3 - plain bare
 
+t/Makefile defines a target for each test file, such that you can also use
+shell pattern matching to run a subset of the tests:
+
+    make *checkout*
+
+will run all tests with 'checkout' in their filename.
+
 Since the tests all output TAP (see https://testanything.org) they can
 be run with any TAP harness. Here's an example of parallel testing
 powered by a recent version of prove(1):

base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b

-- 
gitgitgadget

Junio C Hamano· Mar 24, 2024, 16:10 UTC · re: Philippe Blain via GitGitGadget · lore

Re: [PATCH] t/README: mention test files are make targets

"Philippe Blain via GitGitGadget" <gitgitgadget@gmail.com> writes:

Show 11 quoted lines

> From: Philippe Blain <levraiphilippeblain@gmail.com>
>
> Since 23fc63bf8f (make tests ignorable with "make -i", 2005-11-08), each
> test file defines a target in the test Makefile, such that one can
> invoke:
>
> 	make *checkout*
>
> to run all tests with 'checkout' in their filename. This is useful to
> run a subset of tests when you have a good idea of what part of the code
> is touched by the changes your are testing.

While I agree with the patch that this is a useful "feature" of t/Makefile, I've always felt it was ugly to use a file itself that we do not consider a build product, rather a source, as the target to trigger some action. Are we comfortable casting this behaviour in stone by documenting it here? Just checking by asking others, as you are obviously comfortable enough to write this patch ;-)

Thanks.

Show 34 quoted lines

> Document that in t/README to help new (or more seasoned) contributors
> that might not be aware.
>
> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
> ---
>     t/README: mention test files are make targets
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1701%2Fphil-blain%2Ftests-makefile-targets-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1701/phil-blain/tests-makefile-targets-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1701
>
>  t/README | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/t/README b/t/README
> index 621d3b8c095..71211109338 100644
> --- a/t/README
> +++ b/t/README
> @@ -32,6 +32,13 @@ the tests.
>      ok 2 - plain with GIT_WORK_TREE
>      ok 3 - plain bare
>  
> +t/Makefile defines a target for each test file, such that you can also use
> +shell pattern matching to run a subset of the tests:
> +
> +    make *checkout*
> +
> +will run all tests with 'checkout' in their filename.
> +
>  Since the tests all output TAP (see https://testanything.org) they can
>  be run with any TAP harness. Here's an example of parallel testing
>  powered by a recent version of prove(1):
>
> base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b

Philippe Blain· Mar 24, 2024, 17:04 UTC · re: Junio C Hamano · lore

Re: [PATCH] t/README: mention test files are make targets

Hi Junio,

Le 2024-03-24 à 12:10, Junio C Hamano a écrit :

Show 19 quoted lines

> "Philippe Blain via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> From: Philippe Blain <levraiphilippeblain@gmail.com>
>>
>> Since 23fc63bf8f (make tests ignorable with "make -i", 2005-11-08), each
>> test file defines a target in the test Makefile, such that one can
>> invoke:
>>
>> 	make *checkout*
>>
>> to run all tests with 'checkout' in their filename. This is useful to
>> run a subset of tests when you have a good idea of what part of the code
>> is touched by the changes your are testing.
> 
> While I agree with the patch that this is a useful "feature" of
> t/Makefile, I've always felt it was ugly to use a file itself that
> we do not consider a build product, rather a source, as the target
> to trigger some action.  Are we comfortable casting this behaviour
> in stone by documenting it here?

Since '$(T)' is listed at the bottom of the Makefile as .PHONY, I think it is OK and not that ugly since this uses a documented feature of make.

Cheers, Philippe.

Junio C Hamano· Mar 25, 2024, 01:24 UTC · re: Philippe Blain · lore

Re: [PATCH] t/README: mention test files are make targets

Philippe Blain <levraiphilippeblain@gmail.com> writes:

> Since '$(T)' is listed at the bottom of the Makefile as .PHONY,
> I think it is OK and not that ugly since this uses a documented feature
> of make.

You're prehaps right.

I've always felt that the documented .PHONY feature was to mark targets that do not correspond to any filename on the filesystem, e.g., "all", "clean", "install". Of course these can exist as filenames as well, and .PHONY works as an instruction that says "the existence or freshness of these targets do not matter at all". For our use, it is OK.

I however wonder if marking $(T) as .PHONY is the right thing to begin with. Declaring that the existence or freshness of t0000-basic.sh does not matter means we will not be able to later write rules other than "just run it!" that does depend on the freshness of t0000-basic.sh file, no? IOW, if we wanted to add a target like this at the end of the t/Makefile:

        t-combined.sh: t0000-basic.sh t0001-init.sh
                cat t0000-basic.sh t0001-init.sh >"$@"

then "make -C t t-combined.sh" would end up running these two test scripts (because they are .PHONY) and then leave the concatenation in t-combined.sh file. Without changing anything, doing the same "make -C t t-combined.sh" again will again run these two tests and recreate the same t-combined.sh file, even though there is no need to.

So I think that is what I felt ugly.

As long as we do not use these $(T) files as an input to some other thing and list them as the dependencies, we are OK, though.

Chris Torek· Mar 25, 2024, 09:59 UTC · re: Junio C Hamano · lore

Re: [PATCH] t/README: mention test files are make targets

On Mon, Mar 25, 2024 at 2:49 AM Junio C Hamano <gitster@pobox.com> wrote:

> As long as we do not use these $(T) files as an input to some other
> thing and list them as the dependencies, we are OK, though.

You could (maybe later / at need) stop listing them as `.PHONY` and instead use:

    $(T)::
        sh -c ./$@

or similar, so that some $(T) *can* be an input. Note that this requires using double-colon rules earlier to build the test.

I wouldn't do this without a pretty strong reason though.

Chris

Junio C Hamano· Mar 25, 2024, 19:01 UTC · re: Chris Torek · lore

Re: [PATCH] t/README: mention test files are make targets

Chris Torek <chris.torek@gmail.com> writes:

Show 14 quoted lines

> On Mon, Mar 25, 2024 at 2:49 AM Junio C Hamano <gitster@pobox.com> wrote:
>> As long as we do not use these $(T) files as an input to some other
>> thing and list them as the dependencies, we are OK, though.
>
> You could (maybe later / at need) stop listing them as `.PHONY` and
> instead use:
>
>     $(T)::
>         sh -c ./$@
>
> or similar, so that some $(T) *can* be an input. Note that this requires
> using double-colon rules earlier to build the test.
>
> I wouldn't do this without a pretty strong reason though.

Me neither.

I personally think a target that is marked as .PHONY and does not use double-colon rule is a bug by itself but that is a separate story.

In any case, just to avoid leaving the thread hanging, I'll take the patch as is, as it documents a useful trick in the status quo.

Thanks, all.