[PATCH 2/2] ci: build and run minimal fuzzers in GitHub CI

From: Josh Steadmon <steadmon@google.com>
Date: Jan 19, 2024, 21:38 UTC
Message-ID: <eb38274459cde83a22745172a1ed1d20aebc58a7.1705700054.git.steadmon@google.com>
In-Reply-To: <cover.1705700054.git.steadmon@google.com>

To prevent bitrot, we would like to regularly exercise the fuzz tests in order to make sure they still link & run properly. We already compile the fuzz test objects as part of the default `make` target, but we do not link the executables due to the fuzz tests needing specific compilers and compiler features. This has lead to frequent build breakages for the fuzz tests.

To remedy this, we can add a CI step to actually link the fuzz executables, and run them (with finite input rather than the default infinite random input mode) to verify that they execute properly.

Since the main use of the fuzz tests is via OSS-Fuzz [1], and OSS-Fuzz only runs tests on Linux [2], we only set up a CI test for the fuzzers on Linux.

[1] https://github.com/google/oss-fuzz [2] https://google.github.io/oss-fuzz/further-reading/fuzzer-environment/

Signed-off-by: Josh Steadmon <steadmon@google.com>
---
 .github/workflows/main.yml          | 11 +++++++++++
 Makefile                            |  3 +++
 ci/run-build-and-minimal-fuzzers.sh | 19 +++++++++++++++++++
 3 files changed, 33 insertions(+)
 create mode 100755 ci/run-build-and-minimal-fuzzers.sh

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 9fdbd54028..4d97da57ec 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -309,6 +309,17 @@ jobs:
       with:
         name: failed-tests-${{matrix.vector.jobname}}
         path: ${{env.FAILED_TEST_ARTIFACTS}}
+  fuzz-smoke-test:
+    name: fuzz smoke test
+    needs: ci-config
+    if: needs.ci-config.outputs.enabled == 'yes'
+    env:
+      CC: clang
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - run: ci/install-dependencies.sh
+    - run: ci/run-build-and-minimal-fuzzers.sh
   dockerized:
     name: ${{matrix.vector.jobname}} (${{matrix.vector.image}})
     needs: ci-config
diff --git a/Makefile b/Makefile
index 1e9bd6430f..2e94c566e0 100644
--- a/Makefile
+++ b/Makefile
@@ -752,6 +752,9 @@ SCRIPTS = $(SCRIPT_SH_GEN) \
 
 ETAGS_TARGET = TAGS
 
+# If you add a new fuzzer, please also make sure to run it in
+# ci/run-build-and-minimal-fuzzers.sh so that we make sure it still links and
+# runs in the future.
 FUZZ_OBJS += oss-fuzz/dummy-cmd-main.o
 FUZZ_OBJS += oss-fuzz/fuzz-commit-graph.o
 FUZZ_OBJS += oss-fuzz/fuzz-date.o
diff --git a/ci/run-build-and-minimal-fuzzers.sh b/ci/run-build-and-minimal-fuzzers.sh
new file mode 100755
index 0000000000..8ba486f659
--- /dev/null
+++ b/ci/run-build-and-minimal-fuzzers.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+#
+# Build and test Git's fuzzers
+#
+
+. ${0%/*}/lib.sh
+
+group "Build fuzzers" make \
+	CC=clang \
+	CXX=clang++ \
+	CFLAGS="-fsanitize=fuzzer-no-link,address" \
+	LIB_FUZZING_ENGINE="-fsanitize=fuzzer,address" \
+	fuzz-all
+
+for fuzzer in commit-graph date pack-headers pack-idx ; do
+	begin_group "fuzz-$fuzzer"
+	./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
+	end_group "fuzz-$fuzzer"
+done

-- 
2.43.0.429.g432eaa2c6b-goog