# [PATCH] gitlab-ci: add smoke test for fuzzers 6 messages from 2024-04-29 to 2024-05-06. Participants: Patrick Steinhardt, Junio C Hamano. Thread: https://gitlist.dev/t/61377 ## Patrick Steinhardt, 2024-04-29 06:13 Subject: [PATCH] gitlab-ci: add smoke test for fuzzers Message-ID: <01fb94999f8e2014ba4d09ce7451a4f5d315ee72.1714371146.git.ps@pks.im> URL: https://gitlist.dev/e/01fb94999f8e2014ba4d09ce7451a4f5d315ee72.1714371146.git.ps%40pks.im ``` Our GitLab CI setup has a test gap where the fuzzers aren't exercised at all. Add a smoke test, similar to the one we have in GitHub Workflows. Signed-off-by: Patrick Steinhardt --- As identified by Junio in . Patrick .gitlab-ci.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c0fa2fe90b..33d53b3ec7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -93,6 +93,15 @@ test:osx: - t/failed-test-artifacts when: on_failure +test:fuzz-smoke-tests: + image: ubuntu:latest + variables: + CC: clang + before_script: + - ./ci/install-docker-dependencies.sh + script: + - ./ci/run-build-and-minimal-fuzzers.sh + static-analysis: image: ubuntu:22.04 variables: -- 2.45.0-rc1 ``` ## Patrick Steinhardt, 2024-04-29 06:14 Subject: Re: [PATCH] gitlab-ci: add smoke test for fuzzers Message-ID: URL: https://gitlist.dev/e/Zi86uFE5RlDG2RFN%40tanuki In-Reply-To: <01fb94999f8e2014ba4d09ce7451a4f5d315ee72.1714371146.git.ps@pks.im> ``` On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote: > Our GitLab CI setup has a test gap where the fuzzers aren't exercised at > all. Add a smoke test, similar to the one we have in GitHub Workflows. > > Signed-off-by: Patrick Steinhardt > --- > > As identified by Junio in . > > Patrick I forgot to add the link to a successful run of this job: https://gitlab.com/gitlab-org/git/-/jobs/6735705569 Patrick ``` ## Junio C Hamano, 2024-04-29 15:37 Subject: Re: [PATCH] gitlab-ci: add smoke test for fuzzers Message-ID: URL: https://gitlist.dev/e/xmqqfrv4yyb6.fsf%40gitster.g In-Reply-To: ``` Patrick Steinhardt writes: > On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote: >> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at >> all. Add a smoke test, similar to the one we have in GitHub Workflows. >> >> Signed-off-by: Patrick Steinhardt >> --- >> >> As identified by Junio in . >> >> Patrick > > I forgot to add the link to a successful run of this job: > https://gitlab.com/gitlab-org/git/-/jobs/6735705569 Thanks. I wonder if we can somehow automate a change like this. Seeing how simple this fix has become thanks to the use of before_script/script pair that merely point at ci/*.sh scripts, perhaps we have already extracted enough commonalities as a set of shell scripts in ci/ hierarchy. I wonder if we can have a common "source" that is "compiled" into .gitlab-ci.yml and its counterpart for GitHub Actions? Or perhaps a linter that can say things like "ah, you are adding this new test to one, but not touching the other, shouldn't you?", and "you are tweaking this existing test in one, but shouldn't you be doing the same to the other?" ``` ## Patrick Steinhardt, 2024-04-30 04:30 Subject: Re: [PATCH] gitlab-ci: add smoke test for fuzzers Message-ID: URL: https://gitlist.dev/e/ZjBz_V31u_9CfXDn%40tanuki In-Reply-To: ``` On Mon, Apr 29, 2024 at 08:37:49AM -0700, Junio C Hamano wrote: > Patrick Steinhardt writes: > > > On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote: > >> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at > >> all. Add a smoke test, similar to the one we have in GitHub Workflows. > >> > >> Signed-off-by: Patrick Steinhardt > >> --- > >> > >> As identified by Junio in . > >> > >> Patrick > > > > I forgot to add the link to a successful run of this job: > > https://gitlab.com/gitlab-org/git/-/jobs/6735705569 > > Thanks. I wonder if we can somehow automate a change like this. > > Seeing how simple this fix has become thanks to the use of > before_script/script pair that merely point at ci/*.sh scripts, > perhaps we have already extracted enough commonalities as a set of > shell scripts in ci/ hierarchy. I wonder if we can have a common > "source" that is "compiled" into .gitlab-ci.yml and its counterpart > for GitHub Actions? > > Or perhaps a linter that can say things like "ah, you are adding > this new test to one, but not touching the other, shouldn't you?", > and "you are tweaking this existing test in one, but shouldn't you > be doing the same to the other?" We probably could, yeah. The question is whether it would really be worth it in the end. GitLab CI is still a relatively new addition, and thus it needs to catch up with what GitHub Workflows has. But once that is done I don't expect there to be a ton of changes to the CI setup, and the few new additions that we gain once in a while should be relatively easy to spot during review. So if anybody is up for it then I'm happy to review that. But I don't think there would be enough value to do it myself. Patrick ``` ## Patrick Steinhardt, 2024-05-06 10:08 Subject: Re: [PATCH] gitlab-ci: add smoke test for fuzzers Message-ID: URL: https://gitlist.dev/e/ZjisGhU66kDKkZsZ%40tanuki In-Reply-To: <01fb94999f8e2014ba4d09ce7451a4f5d315ee72.1714371146.git.ps@pks.im> ``` On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote: > Our GitLab CI setup has a test gap where the fuzzers aren't exercised at > all. Add a smoke test, similar to the one we have in GitHub Workflows. > > Signed-off-by: Patrick Steinhardt Junio, I haven't seen this topic getting picked up in your "What's cooking" report yet. Is this intentional or did it simply fall through the cracks? Thanks! Patrick ``` ## Junio C Hamano, 2024-05-06 18:52 Subject: Re: [PATCH] gitlab-ci: add smoke test for fuzzers Message-ID: URL: https://gitlist.dev/e/xmqqy18mydrh.fsf%40gitster.g In-Reply-To: ``` Patrick Steinhardt writes: > On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote: >> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at >> all. Add a smoke test, similar to the one we have in GitHub Workflows. >> >> Signed-off-by: Patrick Steinhardt > > Junio, I haven't seen this topic getting picked up in your "What's > cooking" report yet. Is this intentional or did it simply fall through > the cracks? The latter. Thanks for a ping that is designed to make it easy to find the patch (by going to the message it is a response to). ```