[PATCH v4 0/5] meson: wire up support for benchmarks

Hi,

this small patch series implements support for running our benchmarks in "t/perf" via Meson. The series does not aim to replace "t/perf/run", which is more fully-featured and allows running benchmarks against multiple different trees. Instead, this series only allows running the benchmarks against the current tree. Users are thus expected to continue using "t/perf/run" for more advanced usecases.

Changes in v2:
  - Adapt "aggregate.perl" to use a "/usr/bin/env perl" shebang.
  - Link to v1: https://lore.kernel.org/r/20250331-pks-meson-benchmarks-v1-0-b2ace85616a3@pks.im

Changes in v3:
  - Document how to run benchmarks in "meson.build".
  - Expand the message for the commit that enables out-of-tree
    benchmarking.
  - Link to v2: https://lore.kernel.org/r/20250414-pks-meson-benchmarks-v2-0-04377080a167@pks.im

Changes in v4:
  - The patch series was rebased on top of f65182a99e5 (The ninth batch,
    2025-04-24). This is due to a conflict with ps/test-wo-perl-prereq.
  - Introduce a 'benchmarks' option. This allows developers to require
    benchmarks as desired. By default, we auto-detect whether the host
    system has all dependencies available and enable or disable them
    accordingly.
  - Report whether or not benchmarks are enabled via `summary()`.
  - Our benchmarks depend on Perl, so add this dependency accordingly.
  - Link to v3: https://lore.kernel.org/r/20250422-pks-meson-benchmarks-v3-0-7aad68bac6fd@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (5):
      t/perf: fix benchmarks with alternate repo formats
      t/perf: use configured PERL_PATH
      t/perf: fix benchmarks with out-of-tree builds
      meson: wire up benchmarks
      meson: wire up benchmarking options

 meson.build           | 16 ++++++++---
 meson_options.txt     |  8 ++++++
 t/meson.build         | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++
 t/perf/aggregate.perl |  2 +-
 t/perf/perf-lib.sh    | 32 ++++++++++++++++++---
 t/perf/run            |  4 +--
 6 files changed, 128 insertions(+), 11 deletions(-)

Range-diff versus v3:

1:  2375a16c2c0 = 1:  174804805c2 t/perf: fix benchmarks with alternate repo formats
2:  69782035246 = 2:  41faa3d9b3b t/perf: use configured PERL_PATH
3:  cca40e9dd0e = 3:  87ea3ab1a45 t/perf: fix benchmarks with out-of-tree builds
4:  50e9b507db4 ! 4:  723a0a2e449 meson: wire up benchmarks
    @@ meson.build
      #   Test execution is parallelized by default and scales with the number of
      #   processor cores available. You can change the number of processes by passing
      #   the `-jN` flag to `meson test`.
    -@@ meson.build: git = find_program('git', dirs: program_path, required: false)
    - sed = find_program('sed', dirs: program_path)
    - shell = find_program('sh', dirs: program_path)
    - tar = find_program('tar', dirs: program_path)
    -+time = find_program('time', dirs: program_path, required: false)
    +@@ meson.build: git = find_program('git', dirs: program_path, native: true, required: false)
    + sed = find_program('sed', dirs: program_path, native: true)
    + shell = find_program('sh', dirs: program_path, native: true)
    + tar = find_program('tar', dirs: program_path, native: true)
    ++time = find_program('time', dirs: program_path, required: get_option('benchmarks'))
      
    - # Sanity-check that programs required for the build exist.
    - foreach tool : ['cat', 'cut', 'grep', 'sort', 'tr', 'uname']
    + target_shell = find_program('sh', dirs: program_path, native: false)
    + 
    +@@ meson.build: endif
    + # features. It is optional if you want to neither execute tests nor use any of
    + # these optional features.
    + perl_required = get_option('perl')
    +-if get_option('gitweb').enabled() or 'netrc' in get_option('credential_helpers') or get_option('docs') != []
    ++if get_option('benchmarks').enabled() or get_option('gitweb').enabled() or 'netrc' in get_option('credential_helpers') or get_option('docs') != []
    +   perl_required = true
    + endif
    + 
    +@@ meson.build: meson.add_dist_script(
    + )
    + 
    + summary({
    ++  'benchmarks': get_option('tests') and perl.found() and time.found(),
    +   'curl': curl.found(),
    +   'expat': expat.found(),
    +   'gettext': intl.found(),
    +
    + ## meson_options.txt ##
    +@@ meson_options.txt: option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto
    +   description: 'Which backend to use to generate documentation.')
    + 
    + # Testing.
    ++option('benchmarks', type: 'feature', value: 'auto',
    ++  description: 'Enable benchmarks. This requires Perl and GNU time.')
    + option('coccinelle', type: 'feature', value: 'auto',
    +   description: 'Provide a coccicheck target that generates a Coccinelle patch.')
    + option('tests', type: 'boolean', value: true,
     
      ## t/meson.build ##
     @@ t/meson.build: integration_tests = [
    @@ t/meson.build: foreach integration_test : integration_tests
        )
      endforeach
     +
    -+if time.found()
    ++if perl.found() and time.found()
     +  benchmark_environment = test_environment
     +  benchmark_environment.set('GTIME', time.full_path())
     +
5:  52cca4b17ae ! 5:  16c0230656f meson: wire up benchmarking options
    @@ meson.build: builtin_sources += custom_target(
     
      ## meson_options.txt ##
     @@ meson_options.txt: option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto
    -   description: 'Which backend to use to generate documentation.')
    - 
      # Testing.
    + option('benchmarks', type: 'feature', value: 'auto',
    +   description: 'Enable benchmarks. This requires Perl and GNU time.')
     +option('benchmark_repo', type: 'string', value: '',
     +  description: 'Repository to copy for the performance tests. Should be at least the size of the Git repository.')
     +option('benchmark_large_repo', type: 'string', value: '',

--- base-commit: f65182a99e545d2f2bc22e6c1c2da192133b16a3 change-id: 20250328-pks-meson-benchmarks-a8fac5f69467