threads / patch / 62374

patcht/meson.build: fix up tests for 'seen' branch

Subject: [PATCH] t/meson.build: fix up tests for 'seen' branch

## tl;dr

4 messages between Oct 20, 2024 and Oct 22, 2024. Diffs are folded; open one to read it.

replies: 3people: 4as markdown or json

Ramsay Jones· Oct 20, 2024, 16:00 UTC · lore

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
---

Hi Patrick,

So, having got the build to work, I noticed that 'ninja test' was running less tests than 'make test': 1028 v. 1035. Having a fixed list of tests to run has 'issues'. :) This patch fixes up the list for the 'seen' branch.

The 'new' tests in 'seen' are:

  t1017-cat-file-remote-object-info.sh
    branch: 'ej/cat-file-remote-object-info'
    commit: ef30c4557c ("cat-file: add remote-object-info to batch-command",
    2024-09-25)

  t5710-promisor-remote-capability.sh
    branch: 'cc/promisor-remote-capability'
    commit: 3cb155a011 ("Add 'promisor-remote' capability to protocol v2",
    2024-09-10)

  t6601-path-walk.sh
    branch: 'ds/path-walk'
    commit: d7e16a4ca9 ("t6601: add helper for testing path-walk API",
    2024-10-08)

  t8015-blame-default-ignore-revs.sh
    branch: 'am/git-blame-ignore-revs-by-default'
    commit: eca5438740 ("blame: respect .git-blame-ignore-revs automatically",
    2024-10-12)

  t8016-blame-override-ignore-revs.sh
    branch: 'am/git-blame-ignore-revs-by-default'
    commit: 0386bb36aa ("blame: introduce --override-ignore-revs to bypass
    ignore revisions list", 2024-10-12)

A 'new' test in 'next':

  t2408-worktree-relative.sh
    branch: 'cw/worktree-relative'
    commit: 717af916cd ("worktree: link worktrees with relative paths",
    2024-10-07)

A missing test which (as you know ;) ) is in maint:

  unit-tests/t-reftable-reader.c
    commit: 0a148a8eda ("reftable/reader: make table iterator reseekable",
    2024-09-16)

Hmm, this is going to be a PITA as far as maintenance is concerned! :( If I am reading it correctly, the cmake solution uses file globbing to get the list of test files to run - could meson do the same?

ATB, Ramsay Jones

 t/meson.build | 7 +++++++
 1 file changed, 7 insertions(+)

Show changes to t/meson.build +7 −0

diff --git a/t/meson.build b/t/meson.build
index 152fd7dc13..a956e6db75 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -43,6 +43,7 @@ unit_test_programs = [
   'unit-tests/t-reftable-block.c',
   'unit-tests/t-reftable-merged.c',
   'unit-tests/t-reftable-pq.c',
+  'unit-tests/t-reftable-reader.c',
   'unit-tests/t-reftable-readwrite.c',
   'unit-tests/t-reftable-record.c',
   'unit-tests/t-reftable-stack.c',
@@ -166,6 +167,7 @@ integration_tests = [
   't1014-read-tree-confusing.sh',
   't1015-read-index-unmerged.sh',
   't1016-compatObjectFormat.sh',
+  't1017-cat-file-remote-object-info.sh',
   't1020-subdirectory.sh',
   't1021-rerere-in-workdir.sh',
   't1022-read-tree-partial-clone.sh',
@@ -295,6 +297,7 @@ integration_tests = [
   't2405-worktree-submodule.sh',
   't2406-worktree-repair.sh',
   't2407-worktree-heads.sh',
+  't2408-worktree-relative.sh',
   't2500-untracked-overwriting.sh',
   't2501-cwd-empty.sh',
   't3000-ls-files-others.sh',
@@ -716,6 +719,7 @@ integration_tests = [
   't5703-upload-pack-ref-in-want.sh',
   't5704-protocol-violations.sh',
   't5705-session-id-in-capabilities.sh',
+  't5710-promisor-remote-capability.sh',
   't5730-protocol-v2-bundle-uri-file.sh',
   't5731-protocol-v2-bundle-uri-git.sh',
   't5732-protocol-v2-bundle-uri-http.sh',
@@ -818,6 +822,7 @@ integration_tests = [
   't6500-gc.sh',
   't6501-freshen-objects.sh',
   't6600-test-reach.sh',
+  't6601-path-walk.sh',
   't6700-tree-depth.sh',
   't7001-mv.sh',
   't7002-mv-sparse-checkout.sh',
@@ -944,6 +949,8 @@ integration_tests = [
   't8012-blame-colors.sh',
   't8013-blame-ignore-revs.sh',
   't8014-blame-ignore-fuzzy.sh',
+  't8015-blame-default-ignore-revs.sh',
+  't8016-blame-override-ignore-revs.sh',
   't9001-send-email.sh',
   't9002-column.sh',
   't9003-help-autocorrect.sh',

-- 
2.47.0

Patrick Steinhardt· Oct 21, 2024, 06:04 UTC · re: Ramsay Jones · lore

Re: [PATCH] t/meson.build: fix up tests for 'seen' branch

On Sun, Oct 20, 2024 at 05:00:44PM +0100, Ramsay Jones wrote:

Thanks, I'll pick these up.

> Hmm, this is going to be a PITA as far as maintenance is concerned! :(
> If I am reading it correctly, the cmake solution uses file globbing
> to get the list of test files to run - could meson do the same?

In theory we can, yes. But there's a big problem with it, both in Meson and in CMake: the instructions to deduce source files only get executed at configure time. Consequently, when new files get added, the build instructions do not get updated accordingly and are thus broken. So CMake does get around this, but not in a way that is feasible for use as our main build system, and the same would be true for Meson.

For our integration-style tests I'd be okay with not listing the files individually, such that we instead use e.g. prove(1) to run all tests via a single test target. It would be a regression in functionality as we now cannot easily run e.g. "meson test t0000*", but at least we would not have to maintain the list of test scripts anymore.

But for our source files I don't really see an alternative to listing them out explicitly. You don't want things like git-rebase(1) or git-bisect(1) to be broken just because one of the commits happens to add or remove a file. And this is true for whichever build system we want to adopt as additional official buildsystem, whether that is CMake or Meson.

I guess we have less churn here anyway, and we would notice breakage more readily because things would stop compiling. So this is likely less of a problem compared to our tests.

Patrick

Taylor Blau· Oct 21, 2024, 21:14 UTC · re: Patrick Steinhardt · lore

Re: [PATCH] t/meson.build: fix up tests for 'seen' branch

On Mon, Oct 21, 2024 at 08:04:16AM +0200, Patrick Steinhardt wrote:

> On Sun, Oct 20, 2024 at 05:00:44PM +0100, Ramsay Jones wrote:
>
> Thanks, I'll pick these up.

Thanks. Please make sure to let me know which topic(s) that the new version of ps/build depends on as a result so I can queue it properly.

Show 16 quoted lines

> > Hmm, this is going to be a PITA as far as maintenance is concerned! :(
> > If I am reading it correctly, the cmake solution uses file globbing
> > to get the list of test files to run - could meson do the same?
>
> In theory we can, yes. But there's a big problem with it, both in Meson
> and in CMake: the instructions to deduce source files only get executed
> at configure time. Consequently, when new files get added, the build
> instructions do not get updated accordingly and are thus broken. So
> CMake does get around this, but not in a way that is feasible for use as
> our main build system, and the same would be true for Meson.
>
> For our integration-style tests I'd be okay with not listing the files
> individually, such that we instead use e.g. prove(1) to run all tests
> via a single test target. It would be a regression in functionality as
> we now cannot easily run e.g. "meson test t0000*", but at least we would
> not have to maintain the list of test scripts anymore.

I quite like the ability to run tNNNN-*.sh individually when spelled like that (i.e. including the test number, but not the full script name).

I find that for the areas that I have worked on most over the years that I have the relevant test numbers committed by heart, but often don't know the full script name. So it is nice to be able to refer to them by number only.

Am I reading it write that this suggests that Meson would lose that ability?

Thanks, Taylor

Eli Schwartz· Oct 22, 2024, 01:38 UTC · re: Taylor Blau · lore

Re: [PATCH] t/meson.build: fix up tests for 'seen' branch

On 10/21/24 5:14 PM, Taylor Blau wrote:

Show 18 quoted lines

> On Mon, Oct 21, 2024 at 08:04:16AM +0200, Patrick Steinhardt wrote:
>> For our integration-style tests I'd be okay with not listing the files
>> individually, such that we instead use e.g. prove(1) to run all tests
>> via a single test target. It would be a regression in functionality as
>> we now cannot easily run e.g. "meson test t0000*", but at least we would
>> not have to maintain the list of test scripts anymore.
> 
> I quite like the ability to run tNNNN-*.sh individually when spelled
> like that (i.e. including the test number, but not the full script
> name).
> 
> I find that for the areas that I have worked on most over the years that
> I have the relevant test numbers committed by heart, but often don't
> know the full script name. So it is nice to be able to refer to them by
> number only.
> 
> Am I reading it write that this suggests that Meson would lose that
> ability?

If "meson test" simply ran the prove(1) utility then it would still be possible to do:

``` meson test --test-args "--args --to --prove" ```

This would not actually do what you want, since prove(1) doesn't implement "t0000*" style globbing, does it?

So your reading is correct -- meson's own globbing support relies on meson knowing the list of all test names, which means you do have to maintain that list somehow (and have that list update when you do "git pull" and new tests materialize).

-- 
Eli Schwartz