{"thread":{"id":"63397","subject":"[PATCH] ci: fix aggregation of test results with Meson","startedAt":"2025-05-05T10:55:50Z","lastAt":"2025-05-05T21:19:02Z","messageCount":3,"participants":["Patrick Steinhardt","Johannes Schindelin","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"517243","messageId":"20250505-b4-pks-meson-aggregate-results-v1-1-f38899a0a2cc@pks.im","threadId":"63397","inReplyTo":null,"subject":"[PATCH] ci: fix aggregation of test results with Meson","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-05-05T10:55:45Z","receivedAt":"2025-05-05T10:55:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Our CI needs to be aware of the location of the test output directory so\nthat it knows where to find test results. Some of our CI jobs achieve\nthis by setting the `TEST_OUTPUT_DIRECTORY` environment variable, which\nensures that the output will be written to that directory. Other jobs,\nespecially on GitHub Workflows, don't set that environment variable and\ninstead expect test results to be located in the source directory in\n\"t/\".\n\nThe latter logic does not work with Meson though, as the test results\nare not written into the source directory by default, but instead into\nthe build directory. As such, any job that uses Meson without setting\nthe environment variable will be unable to locate and aggregate results.\n\nFix this by explicitly setting the test output directory when we set up\nthe Meson build directory. Like this, we can easily default to \"t/\" in\nthe source directory when the value hasn't been set explicitly.\n\nReported-by: Johannes Schindelin \nSigned-off-by: Patrick Steinhardt \n---\nHi,\n\nJohannes reported to me off-list that aggregation of test results\ndoesn't work on GitHub Workflow when using Meson, as can be seen e.g. in\n[1].\n\nAs it turns out, the issue is that we don't set `TEST_OUTPUT_DIRECTORY`\nfor many of the GitHub Workflows jobs. And because Meson by default puts\ntest results into the build instead of into the source directory our\nassumption that the results can be found in \"t/\" is broken. This isn't \n\nI never noticed this failure myself because in GitLab we always set the\nabove environment variable there. In any case, this patch fixes it as\ncan be seen at [2].\n\nThanks!\n\nPatrick\n\n[1]: https://github.com/git-for-windows/git/actions/runs/14806194960/job/41574766327#step:9:2125\n[2]: https://github.com/git/git/actions/runs/14833082023/job/41638408968\n---\n ci/run-build-and-tests.sh | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\nindex f99b7db2ee8..bd300a05db8 100755\n--- a/ci/run-build-and-tests.sh\n+++ b/ci/run-build-and-tests.sh\n@@ -56,6 +56,7 @@ case \"$jobname\" in\n \t\t--warnlevel 2 --werror \\\n \t\t--wrap-mode nofallback \\\n \t\t-Dfuzzers=true \\\n+\t\t-Dtest_output_directory=\"${TEST_OUTPUT_DIRECTORY:-$(pwd)/t}\" \\\n \t\t$MESONFLAGS\n \tgroup \"Build\" meson compile -C build --\n \tif test -n \"$run_tests\"\n\n---\nbase-commit: 6c0bd1fc70efaf053abe4e57c976afdc72d15377\nchange-id: 20250505-b4-pks-meson-aggregate-results-60ef175dd424\n\n"},{"id":"517249","messageId":"b98d128e-0c16-aa50-e5d5-1717a611b7f6@gmx.de","threadId":"63397","inReplyTo":"20250505-b4-pks-meson-aggregate-results-v1-1-f38899a0a2cc@pks.im","subject":"Re: [PATCH] ci: fix aggregation of test results with Meson","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-05-05T14:00:07Z","receivedAt":"2025-05-05T14:00:12Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Patrick,\n\nOn Mon, 5 May 2025, Patrick Steinhardt wrote:\n\n> Our CI needs to be aware of the location of the test output directory so\n> that it knows where to find test results. Some of our CI jobs achieve\n> this by setting the `TEST_OUTPUT_DIRECTORY` environment variable, which\n> ensures that the output will be written to that directory. Other jobs,\n> especially on GitHub Workflows, don't set that environment variable and\n> instead expect test results to be located in the source directory in\n> \"t/\".\n> \n> The latter logic does not work with Meson though, as the test results\n> are not written into the source directory by default, but instead into\n> the build directory. As such, any job that uses Meson without setting\n> the environment variable will be unable to locate and aggregate results.\n> \n> Fix this by explicitly setting the test output directory when we set up\n> the Meson build directory. Like this, we can easily default to \"t/\" in\n> the source directory when the value hasn't been set explicitly.\n> \n> Reported-by: Johannes Schindelin \n> Signed-off-by: Patrick Steinhardt \n> ---\n> Hi,\n> \n> Johannes reported to me off-list that aggregation of test results\n> doesn't work on GitHub Workflow when using Meson, as can be seen e.g. in\n> [1].\n> \n> As it turns out, the issue is that we don't set `TEST_OUTPUT_DIRECTORY`\n> for many of the GitHub Workflows jobs. And because Meson by default puts\n> test results into the build instead of into the source directory our\n> assumption that the results can be found in \"t/\" is broken. This isn't \n> \n> I never noticed this failure myself because in GitLab we always set the\n> above environment variable there. In any case, this patch fixes it as\n> can be seen at [2].\n> \n> Thanks!\n\nThank _you_!\nJohannes\n\n> \n> Patrick\n> \n> [1]: https://github.com/git-for-windows/git/actions/runs/14806194960/job/41574766327#step:9:2125\n> [2]: https://github.com/git/git/actions/runs/14833082023/job/41638408968\n> ---\n> ci/run-build-and-tests.sh | 1 +\n> 1 file changed, 1 insertion(+)\n> \n> diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\n> index f99b7db2ee8..bd300a05db8 100755\n> --- a/ci/run-build-and-tests.sh\n> +++ b/ci/run-build-and-tests.sh\n> @@ -56,6 +56,7 @@ case \"$jobname\" in\n> \t\t--warnlevel 2 --werror \\\n> \t\t--wrap-mode nofallback \\\n> \t\t-Dfuzzers=true \\\n> +\t\t-Dtest_output_directory=\"${TEST_OUTPUT_DIRECTORY:-$(pwd)/t}\" \\\n> \t\t$MESONFLAGS\n> \tgroup \"Build\" meson compile -C build --\n> \tif test -n \"$run_tests\"\n> \n> ---\n> base-commit: 6c0bd1fc70efaf053abe4e57c976afdc72d15377\n> change-id: 20250505-b4-pks-meson-aggregate-results-60ef175dd424\n> \n> \n"},{"id":"517280","messageId":"xmqq34dioiwr.fsf@gitster.g","threadId":"63397","inReplyTo":"b98d128e-0c16-aa50-e5d5-1717a611b7f6@gmx.de","subject":"Re: [PATCH] ci: fix aggregation of test results with Meson","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-05-05T21:19:00Z","receivedAt":"2025-05-05T21:19:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> Johannes reported to me off-list that aggregation of test results\n>> doesn't work on GitHub Workflow when using Meson, as can be seen e.g. in\n>> [1].\n>> ...\n>> \n>> Thanks!\n>\n> Thank _you_!\n> Johannes\n\nThank both of you ;-)\n"}]}