threads / patch / 62866

patch, 11 partsmeson: cleanups, improvements, smallish fixes

Subject: [PATCH 00/11] meson: cleanups, improvements, smallish fixes

## tl;dr

35 messages between Jan 29, 2025 and Feb 7, 2025. Diffs are folded; open one to read it.

replies: 34people: 3as markdown or json

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · lore

Hi,

this patch series contains a mostly-random set of smallish improvements, simplifications and fixes to the Meson build instructions.

Thanks!

Patrick

---
Patrick Steinhardt (11):
      meson: fix exec path with enabled runtime prefix
      meson: inline the static 'git' library
      meson: simplify use of the common-main library
      meson: stop linking libcurl into all executables
      meson: introduce `libgit_curl` dependency
      meson: drop separate version library
      meson: deduplicate the list of required programs
      meson: simplify setup of PATH environment variable
      meson: prevent finding sed(1) in a loop
      meson: fix overwritten `git` variable
      meson: consistently use custom program paths to resolve programs

 Documentation/howto/meson.build |   2 +-
 Documentation/meson.build       |  13 ++---
 meson.build                     | 124 +++++++++++++++++-----------------------
 t/helper/meson.build            |   4 +-
 t/meson.build                   |   4 +-
 5 files changed, 63 insertions(+), 84 deletions(-)

--- base-commit: da898a5c645ce9b6d72c2d39abe1bc3d48cb0fdb change-id: 20250121-b4-pks-meson-improvements-3e575363e91c

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · re: Patrick Steinhardt · lore

[PATCH 01/11] meson: fix exec path with enabled runtime prefix

When the runtime prefix option is enabled, Git is built such that it knows to locate its binaries relative to the directory a binary is being executed from. This requires us to figure out relative paths, which is handled in `system_prefix()` by trying to strip a couple of well-known paths.

One of these paths, GIT_EXEC_PATH, is expected to be absolute when runtime prefixes are enabled, but relative otherwise. And while our Makefile gets this correcty, in Meson we always wire up the absolute path, which may result in us not being able to find binaries.

Fix this by conditionally injecting the paths depending on whether or not the `runtime_prefix` option is enabled.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Show changes to meson.build +3 −1

diff --git a/meson.build b/meson.build
index c54ccd2162..fd83df8c42 100644
--- a/meson.build
+++ b/meson.build
@@ -675,7 +675,6 @@ libgit_c_args = [
   '-DETC_GITATTRIBUTES="' + get_option('gitattributes') + '"',
   '-DETC_GITCONFIG="' + get_option('gitconfig') + '"',
   '-DFALLBACK_RUNTIME_PREFIX="' + get_option('prefix') + '"',
-  '-DGIT_EXEC_PATH="' + get_option('prefix') / get_option('libexecdir') / 'git-core"',
   '-DGIT_HOST_CPU="' + host_machine.cpu_family() + '"',
   '-DGIT_HTML_PATH="' + get_option('datadir') / 'doc/git-doc"',
   '-DGIT_INFO_PATH="' + get_option('infodir') + '"',
@@ -1437,6 +1436,7 @@ endif
 if get_option('runtime_prefix')
   libgit_c_args += '-DRUNTIME_PREFIX'
   build_options_config.set('RUNTIME_PREFIX', 'true')
+  git_exec_path = get_option('libexecdir') / 'git-core'
 
   if compiler.has_header('mach-o/dyld.h')
     libgit_c_args += '-DHAVE_NS_GET_EXECUTABLE_PATH'
@@ -1473,7 +1473,9 @@ if get_option('runtime_prefix')
   endif
 else
   build_options_config.set('RUNTIME_PREFIX', 'false')
+  git_exec_path = get_option('prefix') / get_option('libexecdir') / 'git-core'
 endif
+libgit_c_args += '-DGIT_EXEC_PATH="' + git_exec_path + '"'
 
 git_version_file = custom_target(
   command: [

-- 
2.48.1.362.g079036d154.dirty

Justin Tobler· Jan 29, 2025, 20:12 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 01/11] meson: fix exec path with enabled runtime prefix

On 25/01/29 08:11AM, Patrick Steinhardt wrote:

Show 5 quoted lines

> When the runtime prefix option is enabled, Git is built such that it
> knows to locate its binaries relative to the directory a binary is being
> executed from. This requires us to figure out relative paths, which is
> handled in `system_prefix()` by trying to strip a couple of well-known
> paths.

Ok if I understand this correctly, when the runtime prefix option is enabled, the prefix that gets setup by `system_prefix()` is expected to be relative from the directory the binary is being executed at.

> One of these paths, GIT_EXEC_PATH, is expected to be absolute when
> runtime prefixes are enabled, but relative otherwise. And while our
> Makefile gets this correcty, in Meson we always wire up the absolute

s/correcty/correctly/

> path, which may result in us not being able to find binaries.

So the problem is that since GIT_EXEC_PATH is always defined as absolute, when the runtime prefix option is enabled, the relative prefix is not able to be correctly set and thus always uses the `FALLBACK_RUNTIME_PREFIX`.

Show 35 quoted lines

> Fix this by conditionally injecting the paths depending on whether or
> not the `runtime_prefix` option is enabled.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  meson.build | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/meson.build b/meson.build
> index c54ccd2162..fd83df8c42 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -675,7 +675,6 @@ libgit_c_args = [
>    '-DETC_GITATTRIBUTES="' + get_option('gitattributes') + '"',
>    '-DETC_GITCONFIG="' + get_option('gitconfig') + '"',
>    '-DFALLBACK_RUNTIME_PREFIX="' + get_option('prefix') + '"',
> -  '-DGIT_EXEC_PATH="' + get_option('prefix') / get_option('libexecdir') / 'git-core"',
>    '-DGIT_HOST_CPU="' + host_machine.cpu_family() + '"',
>    '-DGIT_HTML_PATH="' + get_option('datadir') / 'doc/git-doc"',
>    '-DGIT_INFO_PATH="' + get_option('infodir') + '"',
> @@ -1437,6 +1436,7 @@ endif
>  if get_option('runtime_prefix')
>    libgit_c_args += '-DRUNTIME_PREFIX'
>    build_options_config.set('RUNTIME_PREFIX', 'true')
> +  git_exec_path = get_option('libexecdir') / 'git-core'
>  
>    if compiler.has_header('mach-o/dyld.h')
>      libgit_c_args += '-DHAVE_NS_GET_EXECUTABLE_PATH'
> @@ -1473,7 +1473,9 @@ if get_option('runtime_prefix')
>    endif
>  else
>    build_options_config.set('RUNTIME_PREFIX', 'false')
> +  git_exec_path = get_option('prefix') / get_option('libexecdir') / 'git-core'
>  endif
> +libgit_c_args += '-DGIT_EXEC_PATH="' + git_exec_path + '"'

Ok, so now we conditionally set `GIT_EXEC_PATH` depending on whether the runtime prefix option is enabled. Makes sense

Show 8 quoted lines

>  
>  git_version_file = custom_target(
>    command: [
> 
> -- 
> 2.48.1.362.g079036d154.dirty
> 
>

Patrick Steinhardt· Jan 30, 2025, 07:06 UTC · re: Justin Tobler · lore

Re: [PATCH 01/11] meson: fix exec path with enabled runtime prefix

On Wed, Jan 29, 2025 at 02:12:35PM -0600, Justin Tobler wrote:

Show 23 quoted lines

> On 25/01/29 08:11AM, Patrick Steinhardt wrote:
> > When the runtime prefix option is enabled, Git is built such that it
> > knows to locate its binaries relative to the directory a binary is being
> > executed from. This requires us to figure out relative paths, which is
> > handled in `system_prefix()` by trying to strip a couple of well-known
> > paths.
> 
> Ok if I understand this correctly, when the runtime prefix option is
> enabled, the prefix that gets setup by `system_prefix()` is expected to
> be relative from the directory the binary is being executed at.
> 
> > One of these paths, GIT_EXEC_PATH, is expected to be absolute when
> > runtime prefixes are enabled, but relative otherwise. And while our
> > Makefile gets this correcty, in Meson we always wire up the absolute
> 
> s/correcty/correctly/
> 
> > path, which may result in us not being able to find binaries.
> 
> So the problem is that since GIT_EXEC_PATH is always defined as
> absolute, when the runtime prefix option is enabled, the relative prefix
> is not able to be correctly set and thus always uses the
> `FALLBACK_RUNTIME_PREFIX`.

Yup, excatly.

Patrick

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · re: Patrick Steinhardt · lore

[PATCH 02/11] meson: inline the static 'git' library

When setting up `libgit.a` we first create the static library itself, and then declare it as part of a dependency such that compile arguments, include directories and transitive dependencies get propagated to the users of that library. As such, the static library isn't expected to be used by anything but the declared dependency.

Inline the static library so that we don't even use a separate variable for it. This avoids any kind of confusion that may arise and clarifies how the library is supposed to be used.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

Show changes to meson.build +7 −9

diff --git a/meson.build b/meson.build
index fd83df8c42..84d100fd25 100644
--- a/meson.build
+++ b/meson.build
@@ -1521,17 +1521,15 @@ libgit_version_library = static_library('git-version',
   include_directories: libgit_include_directories,
 )
 
-libgit_library = static_library('git',
-  sources: libgit_sources,
-  c_args: libgit_c_args,
-  link_with: libgit_version_library,
-  dependencies: libgit_dependencies,
-  include_directories: libgit_include_directories,
-)
-
 libgit = declare_dependency(
+  link_with: static_library('git',
+    sources: libgit_sources,
+    c_args: libgit_c_args,
+    link_with: libgit_version_library,
+    dependencies: libgit_dependencies,
+    include_directories: libgit_include_directories,
+  ),
   compile_args: libgit_c_args,
-  link_with: libgit_library,
   dependencies: libgit_dependencies,
   include_directories: libgit_include_directories,
 )

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · re: Patrick Steinhardt · lore

[PATCH 04/11] meson: stop linking libcurl into all executables

We set up libcurl via the `libgit_dependencies` variable, which gets propagated into every user of the `libgit` dependency. This is not necessary though, as most of our executables aren't even supposed to link against libcurl.

Fix this by only linking against libcurl as required.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

Show changes to meson.build +7 −6

diff --git a/meson.build b/meson.build
index 82b6e62029..6f62728117 100644
--- a/meson.build
+++ b/meson.build
@@ -911,7 +911,6 @@ if curl.found()
     use_curl_for_imap_send = true
   endif
 
-  libgit_dependencies += curl
   libgit_c_args += '-DCURL_DISABLE_TYPECHECK'
   build_options_config.set('NO_CURL', '')
 else
@@ -1631,7 +1630,7 @@ if get_option('curl').enabled()
 
   git_remote_http = executable('git-remote-http',
     sources: curl_sources + 'remote-curl.c',
-    dependencies: [libgit_commonmain],
+    dependencies: [libgit_commonmain, curl],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
@@ -1639,7 +1638,7 @@ if get_option('curl').enabled()
 
   test_dependencies += executable('git-http-fetch',
     sources: curl_sources + 'http-fetch.c',
-    dependencies: [libgit_commonmain],
+    dependencies: [libgit_commonmain, curl],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
@@ -1647,7 +1646,7 @@ if get_option('curl').enabled()
   if expat.found()
     test_dependencies += executable('git-http-push',
       sources: curl_sources + 'http-push.c',
-      dependencies: [libgit_commonmain],
+      dependencies: [libgit_commonmain, curl],
       install: true,
       install_dir: get_option('libexecdir') / 'git-core',
     )
@@ -1656,7 +1655,7 @@ if get_option('curl').enabled()
   foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ]
     test_dependencies += executable(alias,
       objects: git_remote_http.extract_all_objects(recursive: false),
-      dependencies: [libgit],
+      dependencies: [libgit, curl],
     )
 
     install_symlink(alias + executable_suffix,
@@ -1667,13 +1666,15 @@ if get_option('curl').enabled()
 endif
 
 imap_send_sources = ['imap-send.c']
+imap_send_dependencies = [libgit_commonmain]
 if use_curl_for_imap_send
   imap_send_sources += curl_sources
+  imap_send_dependencies += curl
 endif
 
 test_dependencies += executable('git-imap-send',
   sources: imap_send_sources,
-  dependencies: [libgit_commonmain],
+  dependencies: imap_send_dependencies,
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · re: Patrick Steinhardt · lore

[PATCH 03/11] meson: simplify use of the common-main library

The "common-main.c" file is used by multiple executables. In order to make it easy to set it up we have created a separate library that these executables can link against. All of these executables also want to link against `libgit.a` though, which makes it necessary to specify both of these as dependencies for every executable.

Simplify this a bit by declaring the library as a source dependency: instead of creating a static library, we now instead compile the common set of files into each executable separately.

This change surfaces an issue when linking aliases for git-remote-http: we extract all objects from `git-remote-http` et al and then link them into the new executable. As such, these objects would already contain a `main()` function. But now that we also compile "common-main.c" into these aliased executables we see a linker error due to `main()` being defined twice. We fix this by only linking against `libgit.a`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build          | 34 +++++++++++++++-------------------
 t/helper/meson.build |  4 ++--
 t/meson.build        |  4 ++--
 3 files changed, 19 insertions(+), 23 deletions(-)

Show changes to 3 files +19 −23

meson.build, t/helper/meson.build, t/meson.build

diff --git a/meson.build b/meson.build
index 84d100fd25..82b6e62029 100644
--- a/meson.build
+++ b/meson.build
@@ -1570,15 +1570,11 @@ if host_machine.system() == 'windows'
     error('Unsupported compiler ' + compiler.get_id())
   endif
 endif
-common_main_library = static_library('common-main',
+
+libgit_commonmain = declare_dependency(
   sources: common_main_sources,
-  c_args: libgit_c_args,
-  dependencies: libgit_dependencies,
-  include_directories: libgit_include_directories,
-)
-common_main = declare_dependency(
-  link_with: common_main_library,
   link_args: common_main_link_args,
+  dependencies: [ libgit ],
 )
 
 bin_wrappers = [ ]
@@ -1586,7 +1582,7 @@ test_dependencies = [ ]
 
 git = executable('git',
   sources: builtin_sources + 'git.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
@@ -1594,35 +1590,35 @@ bin_wrappers += git
 
 test_dependencies += executable('git-daemon',
   sources: 'daemon.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 test_dependencies += executable('git-sh-i18n--envsubst',
   sources: 'sh-i18n--envsubst.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 bin_wrappers += executable('git-shell',
   sources: 'shell.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 test_dependencies += executable('git-http-backend',
   sources: 'http-backend.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 bin_wrappers += executable('scalar',
   sources: 'scalar.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
@@ -1635,7 +1631,7 @@ if get_option('curl').enabled()
 
   git_remote_http = executable('git-remote-http',
     sources: curl_sources + 'remote-curl.c',
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
@@ -1643,7 +1639,7 @@ if get_option('curl').enabled()
 
   test_dependencies += executable('git-http-fetch',
     sources: curl_sources + 'http-fetch.c',
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
@@ -1651,7 +1647,7 @@ if get_option('curl').enabled()
   if expat.found()
     test_dependencies += executable('git-http-push',
       sources: curl_sources + 'http-push.c',
-      dependencies: [libgit, common_main],
+      dependencies: [libgit_commonmain],
       install: true,
       install_dir: get_option('libexecdir') / 'git-core',
     )
@@ -1660,7 +1656,7 @@ if get_option('curl').enabled()
   foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ]
     test_dependencies += executable(alias,
       objects: git_remote_http.extract_all_objects(recursive: false),
-      dependencies: [libgit, common_main],
+      dependencies: [libgit],
     )
 
     install_symlink(alias + executable_suffix,
@@ -1677,7 +1673,7 @@ endif
 
 test_dependencies += executable('git-imap-send',
   sources: imap_send_sources,
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
@@ -1685,7 +1681,7 @@ test_dependencies += executable('git-imap-send',
 foreach alias : [ 'git-receive-pack', 'git-upload-archive', 'git-upload-pack' ]
   bin_wrappers += executable(alias,
     objects: git.extract_all_objects(recursive: false),
-    dependencies: [libgit, common_main],
+    dependencies: [libgit],
   )
 
   install_symlink(alias + executable_suffix,
diff --git a/t/helper/meson.build b/t/helper/meson.build
index 5e83884246..05bf35bd26 100644
--- a/t/helper/meson.build
+++ b/t/helper/meson.build
@@ -78,14 +78,14 @@ test_tool_sources = [
 
 test_tool = executable('test-tool',
   sources: test_tool_sources,
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
 )
 bin_wrappers += test_tool
 test_dependencies += test_tool
 
 test_fake_ssh = executable('test-fake-ssh',
   sources: 'test-fake-ssh.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
 )
 bin_wrappers += test_fake_ssh
 test_dependencies += test_fake_ssh
diff --git a/t/meson.build b/t/meson.build
index 14fea8dddf..8896314b88 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -39,7 +39,7 @@ clar_sources += custom_target(
 
 clar_unit_tests = executable('unit-tests',
   sources: clar_sources + clar_test_suites,
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
 )
 test('unit-tests', clar_unit_tests)
 
@@ -72,7 +72,7 @@ foreach unit_test_program : unit_test_programs
       'unit-tests/lib-reftable.c',
       unit_test_program,
     ],
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
   )
   test(unit_test_name, unit_test,
     workdir: meson.current_source_dir(),

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · re: Patrick Steinhardt · lore

[PATCH 05/11] meson: introduce `libgit_curl` dependency

We've got a set of common source files that we use for those executables that link against libcurl. The setup is somewhat repetitive though. Simplify it by declaring a `libgit_curl` dependency that bundles all of it together.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 41 ++++++++++++++++++-----------------------
 1 file changed, 18 insertions(+), 23 deletions(-)

Show changes to meson.build +18 −23

diff --git a/meson.build b/meson.build
index 6f62728117..e022235815 100644
--- a/meson.build
+++ b/meson.build
@@ -1623,30 +1623,32 @@ bin_wrappers += executable('scalar',
 )
 
 if get_option('curl').enabled()
-  curl_sources = [
-    'http.c',
-    'http-walker.c',
-  ]
-
-  git_remote_http = executable('git-remote-http',
-    sources: curl_sources + 'remote-curl.c',
+  libgit_curl = declare_dependency(
+    sources: [
+      'http.c',
+      'http-walker.c',
+    ],
     dependencies: [libgit_commonmain, curl],
+  )
+
+  test_dependencies += executable('git-remote-http',
+    sources: 'remote-curl.c',
+    dependencies: libgit_curl,
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
-  test_dependencies += git_remote_http
 
   test_dependencies += executable('git-http-fetch',
-    sources: curl_sources + 'http-fetch.c',
-    dependencies: [libgit_commonmain, curl],
+    sources: 'http-fetch.c',
+    dependencies: libgit_curl,
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
 
   if expat.found()
     test_dependencies += executable('git-http-push',
-      sources: curl_sources + 'http-push.c',
-      dependencies: [libgit_commonmain, curl],
+      sources: 'http-push.c',
+      dependencies: libgit_curl,
       install: true,
       install_dir: get_option('libexecdir') / 'git-core',
     )
@@ -1654,8 +1656,8 @@ if get_option('curl').enabled()
 
   foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ]
     test_dependencies += executable(alias,
-      objects: git_remote_http.extract_all_objects(recursive: false),
-      dependencies: [libgit, curl],
+      sources: 'remote-curl.c',
+      dependencies: libgit_curl,
     )
 
     install_symlink(alias + executable_suffix,
@@ -1665,16 +1667,9 @@ if get_option('curl').enabled()
   endforeach
 endif
 
-imap_send_sources = ['imap-send.c']
-imap_send_dependencies = [libgit_commonmain]
-if use_curl_for_imap_send
-  imap_send_sources += curl_sources
-  imap_send_dependencies += curl
-endif
-
 test_dependencies += executable('git-imap-send',
-  sources: imap_send_sources,
-  dependencies: imap_send_dependencies,
+  sources: 'imap-send.c',
+  dependencies: use_curl_for_imap_send ? [libgit_curl] : [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:12 UTC · re: Patrick Steinhardt · lore

[PATCH 07/11] meson: deduplicate the list of required programs

When setting up Meson we detect a couple of executables that we rely on in order to build Git and prepend their respective directories to PATH. This is done so that Windows can locate these tools at build time in case they aren't included in the default PATH, as we know to pick up those required build tools from the Git for Windows path explicitly.

The list of executables we check against is somewhat excessive though, as many of the tools are all part of coreutils. Let's deduplicate them so that we only check for one binary that is part of it, namely cat(1).

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Show changes to meson.build +1 −1

diff --git a/meson.build b/meson.build
index 1e1e478d17..48eb068fd8 100644
--- a/meson.build
+++ b/meson.build
@@ -187,7 +187,7 @@ shell = find_program('sh', dirs: program_path)
 tar = find_program('tar', dirs: program_path)
 
 script_environment = environment()
-foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname']
+foreach tool : ['cat', 'grep', 'sed']
   program = find_program(tool, dirs: program_path)
   script_environment.prepend('PATH', fs.parent(program.full_path()))
 endforeach

-- 
2.48.1.362.g079036d154.dirty

Justin Tobler· Jan 29, 2025, 20:28 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 07/11] meson: deduplicate the list of required programs

On 25/01/29 08:12AM, Patrick Steinhardt wrote:

Show 25 quoted lines

> When setting up Meson we detect a couple of executables that we rely on
> in order to build Git and prepend their respective directories to PATH.
> This is done so that Windows can locate these tools at build time in
> case they aren't included in the default PATH, as we know to pick up
> those required build tools from the Git for Windows path explicitly.
> 
> The list of executables we check against is somewhat excessive though,
> as many of the tools are all part of coreutils. Let's deduplicate them
> so that we only check for one binary that is part of it, namely cat(1).
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  meson.build | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meson.build b/meson.build
> index 1e1e478d17..48eb068fd8 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -187,7 +187,7 @@ shell = find_program('sh', dirs: program_path)
>  tar = find_program('tar', dirs: program_path)
>  
>  script_environment = environment()
> -foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname']
> +foreach tool : ['cat', 'grep', 'sed']

It might be nice to leave a comment here noting that checking for cat(1) alone is sufficient to account for the other tools part of coreutils.

Show 8 quoted lines

>    program = find_program(tool, dirs: program_path)
>    script_environment.prepend('PATH', fs.parent(program.full_path()))
>  endforeach
> 
> -- 
> 2.48.1.362.g079036d154.dirty
> 
>

Patrick Steinhardt· Jan 29, 2025, 07:11 UTC · re: Patrick Steinhardt · lore

[PATCH 06/11] meson: drop separate version library

When building `libgit.a` we link it against a `libgit_version.a` library that contains the version information that we inject at build time. The intent of this is to avoid rebuilding all of `libgit.a` whenever the version changes. But that wouldn't happen in the first place, as we know to just rebuild the files that depend on the generated "version-def.h" file.

This is an artifact of an earlier version of the Meson build infra that didn't ultimately land. We didn't yet have "version-def.h", and instead injected the version via preprocessor directives. And here we would have rebuilt all of `libgit.a` indeed in case the version changes, because the preprocessor directive applied to all files.

Stop building the separate library and instead add "version-def.h" to the list of source files directly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

Show changes to meson.build +5 −16

diff --git a/meson.build b/meson.build
index e022235815..1e1e478d17 100644
--- a/meson.build
+++ b/meson.build
@@ -462,6 +462,7 @@ libgit_sources = [
   'userdiff.c',
   'utf8.c',
   'varint.c',
+  'version.c',
   'versioncmp.c',
   'walker.c',
   'wildmatch.c',
@@ -1505,26 +1506,14 @@ version_def_h = custom_target(
   depends: [git_version_file],
   env: version_gen_environment,
 )
-
-# Build a separate library for "version.c" so that we do not have to rebuild
-# everything when the current Git commit changes.
-libgit_version_library = static_library('git-version',
-  sources: [
-    'version.c',
-    version_def_h,
-  ],
-  c_args: libgit_c_args + [
-    '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
-  ],
-  dependencies: libgit_dependencies,
-  include_directories: libgit_include_directories,
-)
+libgit_sources += version_def_h
 
 libgit = declare_dependency(
   link_with: static_library('git',
     sources: libgit_sources,
-    c_args: libgit_c_args,
-    link_with: libgit_version_library,
+    c_args: libgit_c_args + [
+      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
+    ],
     dependencies: libgit_dependencies,
     include_directories: libgit_include_directories,
   ),

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:12 UTC · re: Patrick Steinhardt · lore

[PATCH 08/11] meson: simplify setup of PATH environment variable

We're setting up the PATH environment variable such that a set of necessary build tools can be found at build time. Make this step a bit less repetitive by only looping through the set of found programs once.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

Show changes to meson.build +8 −9

diff --git a/meson.build b/meson.build
index 48eb068fd8..e3829f2365 100644
--- a/meson.build
+++ b/meson.build
@@ -181,22 +181,21 @@ if host_machine.system() == 'windows'
   program_path += [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ]
 endif
 
+cat = find_program('cat', dirs: program_path)
 cygpath = find_program('cygpath', dirs: program_path, required: false)
 diff = find_program('diff', dirs: program_path)
+git = find_program('git', dirs: program_path, required: false)
+grep = find_program('grep', dirs: program_path)
+sed = find_program('sed', dirs: program_path)
 shell = find_program('sh', dirs: program_path)
 tar = find_program('tar', dirs: program_path)
 
 script_environment = environment()
-foreach tool : ['cat', 'grep', 'sed']
-  program = find_program(tool, dirs: program_path)
-  script_environment.prepend('PATH', fs.parent(program.full_path()))
+foreach program : [cat, cygpath, diff, git, grep, sed, shell, tar]
+  if program.found()
+    script_environment.prepend('PATH', fs.parent(program.full_path()))
+  endif
 endforeach
-
-git = find_program('git', dirs: program_path, required: false)
-if git.found()
-  script_environment.prepend('PATH', fs.parent(git.full_path()))
-endif
-
 if get_option('sane_tool_path') != ''
   script_environment.prepend('PATH', get_option('sane_tool_path'))
 endif

-- 
2.48.1.362.g079036d154.dirty

Justin Tobler· Jan 29, 2025, 20:42 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 08/11] meson: simplify setup of PATH environment variable

On 25/01/29 08:12AM, Patrick Steinhardt wrote:

Show 25 quoted lines

> We're setting up the PATH environment variable such that a set of
> necessary build tools can be found at build time. Make this step a bit
> less repetitive by only looping through the set of found programs once.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  meson.build | 17 ++++++++---------
>  1 file changed, 8 insertions(+), 9 deletions(-)
> 
> diff --git a/meson.build b/meson.build
> index 48eb068fd8..e3829f2365 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -181,22 +181,21 @@ if host_machine.system() == 'windows'
>    program_path += [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ]
>  endif
>  
> +cat = find_program('cat', dirs: program_path)
>  cygpath = find_program('cygpath', dirs: program_path, required: false)
>  diff = find_program('diff', dirs: program_path)
> +git = find_program('git', dirs: program_path, required: false)
> +grep = find_program('grep', dirs: program_path)
> +sed = find_program('sed', dirs: program_path)
>  shell = find_program('sh', dirs: program_path)
>  tar = find_program('tar', dirs: program_path)

At first I was curious to why we wouldn't just invoke `find_program()` during the loop as well, but some of these programs are not required and marked as such here.

Show 8 quoted lines

>  script_environment = environment()
> -foreach tool : ['cat', 'grep', 'sed']
> -  program = find_program(tool, dirs: program_path)
> -  script_environment.prepend('PATH', fs.parent(program.full_path()))
> +foreach program : [cat, cygpath, diff, git, grep, sed, shell, tar]
> +  if program.found()
> +    script_environment.prepend('PATH', fs.parent(program.full_path()))
> +  endif

It looks like cygpath, diff, shell, and tar were previously not being appended to the path environment. With this change now they are.

Show 15 quoted lines

>  endforeach
> -
> -git = find_program('git', dirs: program_path, required: false)
> -if git.found()
> -  script_environment.prepend('PATH', fs.parent(git.full_path()))
> -endif
> -
>  if get_option('sane_tool_path') != ''
>    script_environment.prepend('PATH', get_option('sane_tool_path'))
>  endif
> 
> -- 
> 2.48.1.362.g079036d154.dirty
> 
>

Patrick Steinhardt· Jan 30, 2025, 07:06 UTC · re: Justin Tobler · lore

Re: [PATCH 08/11] meson: simplify setup of PATH environment variable

On Wed, Jan 29, 2025 at 02:42:07PM -0600, Justin Tobler wrote:

Show 12 quoted lines

> On 25/01/29 08:12AM, Patrick Steinhardt wrote:
> >  script_environment = environment()
> > -foreach tool : ['cat', 'grep', 'sed']
> > -  program = find_program(tool, dirs: program_path)
> > -  script_environment.prepend('PATH', fs.parent(program.full_path()))
> > +foreach program : [cat, cygpath, diff, git, grep, sed, shell, tar]
> > +  if program.found()
> > +    script_environment.prepend('PATH', fs.parent(program.full_path()))
> > +  endif
> 
> It looks like cygpath, diff, shell, and tar were previously not being
> appended to the path environment. With this change now they are.

You know, I think I've been approaching this from the wrong angle. It's not like we need to add these tools to PATH in case they have been found via the usual PATH lookup: Meson knows to remember PATH just fine, so the scripts would be able to find them anyway.

The actual issue is that we sometimes end up looking up programs via something else but PATH, namely on Windows, where we may instead look up programs via the Git for Windows installation. So the proper way to handle this is to add these system-specific paths to PATH, not every single binary's parent directory.

Will adapt, thanks for making me rethink.

Patrick

Patrick Steinhardt· Jan 29, 2025, 07:12 UTC · re: Patrick Steinhardt · lore

[PATCH 09/11] meson: prevent finding sed(1) in a loop

We're searching for the sed(1) executable in a loop, which will make us try to find it multiple times. Starting with the preceding commit we already declare a variable for that program in the top-level build file. Use it so that we only need to search for the program once.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/howto/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Show changes to Documentation/howto/meson.build +1 −1

diff --git a/Documentation/howto/meson.build b/Documentation/howto/meson.build
index c023c10416..92a08b13ee 100644
--- a/Documentation/howto/meson.build
+++ b/Documentation/howto/meson.build
@@ -41,7 +41,7 @@ custom_target(
 foreach howto : howto_sources
   howto_stripped = custom_target(
     command: [
-      find_program('sed'),
+      sed,
       '-e',
       '1,/^$/d',
       '@INPUT@',

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:12 UTC · re: Patrick Steinhardt · lore

[PATCH 10/11] meson: fix overwritten `git` variable

We're assigning the `git` variable in three places:

  - In "meson.build" to store the external Git executable.

  - In "meson.build" to store the compiled Git executable.

  - In "Documentation/meson.build" to store the external Git executable,
    a second time.

The last case is only needed because we overwrite the original variable with the built version. Rename the variable used for the built Git executable so that we don't have to resolve the external Git executable multiple times.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 1 -
 meson.build               | 6 +++---
 2 files changed, 3 insertions(+), 4 deletions(-)

Show changes to 2 files +3 −4

Documentation/meson.build, meson.build

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 2a26fa8a5f..6438fa6792 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -283,7 +283,6 @@ elif docs_backend == 'asciidoctor'
   ]
 endif
 
-git = find_program('git', required: false)
 xmlto = find_program('xmlto')
 
 cmd_lists = [
diff --git a/meson.build b/meson.build
index e3829f2365..828fbae8b0 100644
--- a/meson.build
+++ b/meson.build
@@ -1567,13 +1567,13 @@ libgit_commonmain = declare_dependency(
 bin_wrappers = [ ]
 test_dependencies = [ ]
 
-git = executable('git',
+git_builtin = executable('git',
   sources: builtin_sources + 'git.c',
   dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
-bin_wrappers += git
+bin_wrappers += git_builtin
 
 test_dependencies += executable('git-daemon',
   sources: 'daemon.c',
@@ -1664,7 +1664,7 @@ test_dependencies += executable('git-imap-send',
 
 foreach alias : [ 'git-receive-pack', 'git-upload-archive', 'git-upload-pack' ]
   bin_wrappers += executable(alias,
-    objects: git.extract_all_objects(recursive: false),
+    objects: git_builtin.extract_all_objects(recursive: false),
     dependencies: [libgit],
   )

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 29, 2025, 07:12 UTC · re: Patrick Steinhardt · lore

[PATCH 11/11] meson: consistently use custom program paths to resolve programs

The calls to `find_program()` in our documentation don't use our custom program path. This variable gets populated on Windows with the location of Git for Windows so that we can use it to provide our build tools. Consequently, we may not be able to find all necessary binaries on Windows.

Adapt the calls to use the program path to fix this. While at it, drop `required: true` arguments, which are the default anyway.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

Show changes to Documentation/meson.build +6 −6

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 6438fa6792..c6117366ff 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -206,9 +206,9 @@ manpages = {
 
 docs_backend = get_option('docs_backend')
 if docs_backend == 'auto'
-  if find_program('asciidoc', required: false).found()
+  if find_program('asciidoc', dirs: program_path, required: false).found()
     docs_backend = 'asciidoc'
-  elif find_program('asciidoctor', required: false).found()
+  elif find_program('asciidoctor', dirs: program_path, required: false).found()
     docs_backend = 'asciidoctor'
   else
     error('Neither asciidoc nor asciidoctor were found.')
@@ -216,7 +216,7 @@ if docs_backend == 'auto'
 endif
 
 if docs_backend == 'asciidoc'
-  asciidoc = find_program('asciidoc', required: true)
+  asciidoc = find_program('asciidoc', dirs: program_path)
   asciidoc_html = 'xhtml11'
   asciidoc_docbook = 'docbook'
   xmlto_extra = [ ]
@@ -245,7 +245,7 @@ if docs_backend == 'asciidoc'
     asciidoc_conf,
   ]
 elif docs_backend == 'asciidoctor'
-  asciidoctor = find_program('asciidoctor', required: true)
+  asciidoctor = find_program('asciidoctor', dirs: program_path)
   asciidoc_html = 'xhtml5'
   asciidoc_docbook = 'docbook5'
   xmlto_extra = [
@@ -283,7 +283,7 @@ elif docs_backend == 'asciidoctor'
   ]
 endif
 
-xmlto = find_program('xmlto')
+xmlto = find_program('xmlto', dirs: program_path)
 
 cmd_lists = [
   'cmds-ancillaryinterrogators.txt',
@@ -404,7 +404,7 @@ if get_option('docs').contains('html')
     pointing_to: 'git.html',
   )
 
-  xsltproc = find_program('xsltproc')
+  xsltproc = find_program('xsltproc', dirs: program_path)
 
   user_manual_xml = custom_target(
     command: asciidoc_common_options + [

-- 
2.48.1.362.g079036d154.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 00/13] meson: cleanups, improvements, smallish fixes

Hi,

this patch series contains a mostly-random set of smallish improvements, simplifications and fixes to the Meson build instructions.

Changes in v2:
  - Redo the two commits regarding PATH handling. We now only prepend
    system-specific paths to PATH and handle the `-Dsane_tool_path`
    option the same.
  - Continue to propagate libcurl include directories into libgit, as we
    need it for "help.c" to resolve LIBCURL_VERSION. Reorder the patch
    to minimize the required changes.
  - Fix a commit message typo.
  - Include another commit that fixes the OpenSSL fallback on Windows.
  - Include another commit to fix linker errors due to out-of-memory
    situations on Windows with GitLab CI.
  - Link to v1: https://lore.kernel.org/r/20250129-b4-pks-meson-improvements-v1-0-ab709f0be12c@pks.im

I have furthermore pulled in ps/build-meson-fixes at 7304bd2bc3 (ci: wire up Visual Studio build with Meson, 2025-01-22) as a dependency of this series. This is due to a semantic conflict with 28911f7dca (meson: wire up fuzzers, 2025-01-22).

Thanks!

Patrick

---
Patrick Steinhardt (13):
      meson: fix exec path with enabled runtime prefix
      meson: fix OpenSSL fallback when not explicitly required
      meson: inline the static 'git' library
      meson: simplify use of the common-main library
      meson: introduce `libgit_curl` dependency
      meson: stop linking libcurl into all executables
      meson: drop separate version library
      meson: improve PATH handling
      meson: improve handling of `sane_tool_path` option
      meson: prevent finding sed(1) in a loop
      meson: fix overwritten `git` variable
      meson: consistently use custom program paths to resolve programs
      gitlab-ci: restrict maximum number of link jobs on Windows

 .gitlab-ci.yml                  |   2 +-
 Documentation/howto/meson.build |   2 +-
 Documentation/meson.build       |  13 ++--
 meson.build                     | 151 +++++++++++++++++++---------------------
 meson_options.txt               |   4 +-
 oss-fuzz/meson.build            |   2 +-
 t/helper/meson.build            |   4 +-
 t/meson.build                   |   4 +-
 8 files changed, 85 insertions(+), 97 deletions(-)

Range-diff versus v1:

 1:  a4f1eb695a !  1:  932d79a814 meson: fix exec path with enabled runtime prefix
    @@ Commit message
     
         One of these paths, GIT_EXEC_PATH, is expected to be absolute when
         runtime prefixes are enabled, but relative otherwise. And while our
    -    Makefile gets this correcty, in Meson we always wire up the absolute
    +    Makefile gets this correctly, in Meson we always wire up the absolute
         path, which may result in us not being able to find binaries.
     
         Fix this by conditionally injecting the paths depending on whether or
 -:  ---------- >  2:  6bd07f35a6 meson: fix OpenSSL fallback when not explicitly required
 2:  da61c2a679 =  3:  94f9e45c56 meson: inline the static 'git' library
 3:  744754959e !  4:  669c811eb6 meson: simplify use of the common-main library
    @@ meson.build: test_dependencies += executable('git-imap-send',
      
        install_symlink(alias + executable_suffix,
     
    + ## oss-fuzz/meson.build ##
    +@@ oss-fuzz/meson.build: foreach fuzz_program : fuzz_programs
    +       'dummy-cmd-main.c',
    +       fuzz_program,
    +     ],
    +-    dependencies: [libgit, common_main],
    ++    dependencies: [libgit_commonmain],
    +   )
    + endforeach
    +
      ## t/helper/meson.build ##
     @@ t/helper/meson.build: test_tool_sources = [
      
 4:  172d3e07a5 <  -:  ---------- meson: stop linking libcurl into all executables
 5:  a0a1e548cc !  5:  edb9fba4b3 meson: introduce `libgit_curl` dependency
    @@ Commit message
         Simplify it by declaring a `libgit_curl` dependency that bundles all of
         it together.
     
    +    Note that we don't include curl itself as a dependency. This is because
    +    we already pull it in transitively via the libgit dependency, which is
    +    unfortunate because libgit itself shouldn't actually link against curl
    +    in the first place. This will get fixed in the next commit.
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## meson.build ##
    @@ meson.build: bin_wrappers += executable('scalar',
     +      'http.c',
     +      'http-walker.c',
     +    ],
    -     dependencies: [libgit_commonmain, curl],
    +     dependencies: [libgit_commonmain],
     +  )
     +
     +  test_dependencies += executable('git-remote-http',
     +    sources: 'remote-curl.c',
    -+    dependencies: libgit_curl,
    ++    dependencies: [libgit_curl],
          install: true,
          install_dir: get_option('libexecdir') / 'git-core',
        )
    @@ meson.build: bin_wrappers += executable('scalar',
      
        test_dependencies += executable('git-http-fetch',
     -    sources: curl_sources + 'http-fetch.c',
    --    dependencies: [libgit_commonmain, curl],
    +-    dependencies: [libgit_commonmain],
     +    sources: 'http-fetch.c',
    -+    dependencies: libgit_curl,
    ++    dependencies: [libgit_curl],
          install: true,
          install_dir: get_option('libexecdir') / 'git-core',
        )
    @@ meson.build: bin_wrappers += executable('scalar',
        if expat.found()
          test_dependencies += executable('git-http-push',
     -      sources: curl_sources + 'http-push.c',
    --      dependencies: [libgit_commonmain, curl],
    +-      dependencies: [libgit_commonmain],
     +      sources: 'http-push.c',
    -+      dependencies: libgit_curl,
    ++      dependencies: [libgit_curl],
            install: true,
            install_dir: get_option('libexecdir') / 'git-core',
          )
    @@ meson.build: if get_option('curl').enabled()
        foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ]
          test_dependencies += executable(alias,
     -      objects: git_remote_http.extract_all_objects(recursive: false),
    --      dependencies: [libgit, curl],
    +-      dependencies: [libgit],
     +      sources: 'remote-curl.c',
    -+      dependencies: libgit_curl,
    ++      dependencies: [libgit_curl],
          )
      
          install_symlink(alias + executable_suffix,
    @@ meson.build: if get_option('curl').enabled()
      endif
      
     -imap_send_sources = ['imap-send.c']
    --imap_send_dependencies = [libgit_commonmain]
     -if use_curl_for_imap_send
     -  imap_send_sources += curl_sources
    --  imap_send_dependencies += curl
     -endif
     -
      test_dependencies += executable('git-imap-send',
     -  sources: imap_send_sources,
    --  dependencies: imap_send_dependencies,
    +-  dependencies: [libgit_commonmain],
     +  sources: 'imap-send.c',
    -+  dependencies: use_curl_for_imap_send ? [libgit_curl] : [libgit_commonmain],
    ++  dependencies: [ use_curl_for_imap_send ? libgit_curl : libgit_commonmain ],
        install: true,
        install_dir: get_option('libexecdir') / 'git-core',
      )
 -:  ---------- >  6:  95ceafc2f4 meson: stop linking libcurl into all executables
 6:  ab55efa559 =  7:  65d00e36b8 meson: drop separate version library
 7:  7226cd6388 <  -:  ---------- meson: deduplicate the list of required programs
 8:  55a372a559 <  -:  ---------- meson: simplify setup of PATH environment variable
 -:  ---------- >  8:  3dc35872a3 meson: improve PATH handling
 -:  ---------- >  9:  e996dc571c meson: improve handling of `sane_tool_path` option
 9:  2abab952c0 ! 10:  2730e0166b meson: prevent finding sed(1) in a loop
    @@ Documentation/howto/meson.build: custom_target(
            '-e',
            '1,/^$/d',
            '@INPUT@',
    +
    + ## meson.build ##
    +@@ meson.build: endif
    + cygpath = find_program('cygpath', dirs: program_path, required: false)
    + diff = find_program('diff', dirs: program_path)
    + git = find_program('git', dirs: program_path, required: false)
    ++sed = find_program('sed', dirs: program_path)
    + shell = find_program('sh', dirs: program_path)
    + tar = find_program('tar', dirs: program_path)
    + 
    + # Sanity-check that programs required for the build exist.
    +-foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname']
    ++foreach tool : ['cat', 'cut', 'grep', 'sort', 'tr', 'uname']
    +   find_program(tool, dirs: program_path)
    + endforeach
    + 
10:  a36529b4dd = 11:  9a54f2c2b7 meson: fix overwritten `git` variable
11:  2c60ae4349 = 12:  4604460ba2 meson: consistently use custom program paths to resolve programs
 -:  ---------- > 13:  9a6fd5435d gitlab-ci: restrict maximum number of link jobs on Windows

--- base-commit: 6dd4e543f2219290db9faa61b30f7f30a34e4b50 change-id: 20250121-b4-pks-meson-improvements-3e575363e91c

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 02/13] meson: fix OpenSSL fallback when not explicitly required

When OpenSSL isn't provided by the system we know to fall back to the subproject wrapper. This is especially helpful on Windows systems, where you typically don't have OpenSSL available, in order to reduce the number of required dependencies.

The fallback is broken though when the OpenSSL backend is set to 'auto' as we end up calling `dependency('openssl', required: false)` in that case, which implicitly disables falling back to the wrapper.

Fix the issue by re-allowing the fallback in case either OpenSSL is required or in case the backend is set to 'auto'. While at it, fix reporting of the backend in case the user asked us to pick no HTTPS backend at all.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

Show changes to meson.build +6 −1

diff --git a/meson.build b/meson.build
index 1f83dc58c2..83ed55c75c 100644
--- a/meson.build
+++ b/meson.build
@@ -1352,7 +1352,11 @@ if https_backend == 'auto' and security_framework.found()
 endif
 
 openssl_required = 'openssl' in [csprng_backend, https_backend, sha1_backend, sha1_unsafe_backend, sha256_backend]
-openssl = dependency('openssl', required: openssl_required, default_options: ['default_library=static'])
+openssl = dependency('openssl',
+  required: openssl_required,
+  allow_fallback: openssl_required or https_backend == 'auto',
+  default_options: ['default_library=static'],
+)
 if https_backend == 'auto' and openssl.found()
   https_backend = 'openssl'
 endif
@@ -1366,6 +1370,7 @@ elif https_backend == 'openssl'
 else
   # We either couldn't find any dependencies with 'auto' or the user requested
   # 'none'. Both cases are benign.
+  https_backend = 'none'
 endif
 
 if https_backend != 'openssl'

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 01/13] meson: fix exec path with enabled runtime prefix

One of these paths, GIT_EXEC_PATH, is expected to be absolute when runtime prefixes are enabled, but relative otherwise. And while our Makefile gets this correctly, in Meson we always wire up the absolute path, which may result in us not being able to find binaries.

Fix this by conditionally injecting the paths depending on whether or not the `runtime_prefix` option is enabled.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Show changes to meson.build +3 −1

diff --git a/meson.build b/meson.build
index dbc1dab605..1f83dc58c2 100644
--- a/meson.build
+++ b/meson.build
@@ -691,7 +691,6 @@ libgit_c_args = [
   '-DETC_GITATTRIBUTES="' + get_option('gitattributes') + '"',
   '-DETC_GITCONFIG="' + get_option('gitconfig') + '"',
   '-DFALLBACK_RUNTIME_PREFIX="' + get_option('prefix') + '"',
-  '-DGIT_EXEC_PATH="' + get_option('prefix') / get_option('libexecdir') / 'git-core"',
   '-DGIT_HOST_CPU="' + host_machine.cpu_family() + '"',
   '-DGIT_HTML_PATH="' + get_option('datadir') / 'doc/git-doc"',
   '-DGIT_INFO_PATH="' + get_option('infodir') + '"',
@@ -1466,6 +1465,7 @@ endif
 if get_option('runtime_prefix')
   libgit_c_args += '-DRUNTIME_PREFIX'
   build_options_config.set('RUNTIME_PREFIX', 'true')
+  git_exec_path = get_option('libexecdir') / 'git-core'
 
   if compiler.has_header('mach-o/dyld.h')
     libgit_c_args += '-DHAVE_NS_GET_EXECUTABLE_PATH'
@@ -1502,7 +1502,9 @@ if get_option('runtime_prefix')
   endif
 else
   build_options_config.set('RUNTIME_PREFIX', 'false')
+  git_exec_path = get_option('prefix') / get_option('libexecdir') / 'git-core'
 endif
+libgit_c_args += '-DGIT_EXEC_PATH="' + git_exec_path + '"'
 
 git_version_file = custom_target(
   command: [

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 03/13] meson: inline the static 'git' library

Inline the static library so that we don't even use a separate variable for it. This avoids any kind of confusion that may arise and clarifies how the library is supposed to be used.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

Show changes to meson.build +7 −9

diff --git a/meson.build b/meson.build
index 83ed55c75c..a124101a73 100644
--- a/meson.build
+++ b/meson.build
@@ -1555,17 +1555,15 @@ libgit_version_library = static_library('git-version',
   include_directories: libgit_include_directories,
 )
 
-libgit_library = static_library('git',
-  sources: libgit_sources,
-  c_args: libgit_c_args,
-  link_with: libgit_version_library,
-  dependencies: libgit_dependencies,
-  include_directories: libgit_include_directories,
-)
-
 libgit = declare_dependency(
+  link_with: static_library('git',
+    sources: libgit_sources,
+    c_args: libgit_c_args,
+    link_with: libgit_version_library,
+    dependencies: libgit_dependencies,
+    include_directories: libgit_include_directories,
+  ),
   compile_args: libgit_c_args,
-  link_with: libgit_library,
   dependencies: libgit_dependencies,
   include_directories: libgit_include_directories,
 )

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 04/13] meson: simplify use of the common-main library

Simplify this a bit by declaring the library as a source dependency: instead of creating a static library, we now instead compile the common set of files into each executable separately.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build          | 34 +++++++++++++++-------------------
 oss-fuzz/meson.build |  2 +-
 t/helper/meson.build |  4 ++--
 t/meson.build        |  4 ++--
 4 files changed, 20 insertions(+), 24 deletions(-)

Show changes to 4 files +20 −24

meson.build, oss-fuzz/meson.build, t/helper/meson.build, t/meson.build

diff --git a/meson.build b/meson.build
index a124101a73..c7d597eda8 100644
--- a/meson.build
+++ b/meson.build
@@ -1604,15 +1604,11 @@ if host_machine.system() == 'windows'
     error('Unsupported compiler ' + compiler.get_id())
   endif
 endif
-common_main_library = static_library('common-main',
+
+libgit_commonmain = declare_dependency(
   sources: common_main_sources,
-  c_args: libgit_c_args,
-  dependencies: libgit_dependencies,
-  include_directories: libgit_include_directories,
-)
-common_main = declare_dependency(
-  link_with: common_main_library,
   link_args: common_main_link_args,
+  dependencies: [ libgit ],
 )
 
 bin_wrappers = [ ]
@@ -1620,7 +1616,7 @@ test_dependencies = [ ]
 
 git = executable('git',
   sources: builtin_sources + 'git.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
@@ -1628,35 +1624,35 @@ bin_wrappers += git
 
 test_dependencies += executable('git-daemon',
   sources: 'daemon.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 test_dependencies += executable('git-sh-i18n--envsubst',
   sources: 'sh-i18n--envsubst.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 bin_wrappers += executable('git-shell',
   sources: 'shell.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 test_dependencies += executable('git-http-backend',
   sources: 'http-backend.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
 
 bin_wrappers += executable('scalar',
   sources: 'scalar.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
@@ -1669,7 +1665,7 @@ if get_option('curl').enabled()
 
   git_remote_http = executable('git-remote-http',
     sources: curl_sources + 'remote-curl.c',
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
@@ -1677,7 +1673,7 @@ if get_option('curl').enabled()
 
   test_dependencies += executable('git-http-fetch',
     sources: curl_sources + 'http-fetch.c',
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
@@ -1685,7 +1681,7 @@ if get_option('curl').enabled()
   if expat.found()
     test_dependencies += executable('git-http-push',
       sources: curl_sources + 'http-push.c',
-      dependencies: [libgit, common_main],
+      dependencies: [libgit_commonmain],
       install: true,
       install_dir: get_option('libexecdir') / 'git-core',
     )
@@ -1694,7 +1690,7 @@ if get_option('curl').enabled()
   foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ]
     test_dependencies += executable(alias,
       objects: git_remote_http.extract_all_objects(recursive: false),
-      dependencies: [libgit, common_main],
+      dependencies: [libgit],
     )
 
     install_symlink(alias + executable_suffix,
@@ -1711,7 +1707,7 @@ endif
 
 test_dependencies += executable('git-imap-send',
   sources: imap_send_sources,
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
@@ -1719,7 +1715,7 @@ test_dependencies += executable('git-imap-send',
 foreach alias : [ 'git-receive-pack', 'git-upload-archive', 'git-upload-pack' ]
   bin_wrappers += executable(alias,
     objects: git.extract_all_objects(recursive: false),
-    dependencies: [libgit, common_main],
+    dependencies: [libgit],
   )
 
   install_symlink(alias + executable_suffix,
diff --git a/oss-fuzz/meson.build b/oss-fuzz/meson.build
index ed79665501..878afd8426 100644
--- a/oss-fuzz/meson.build
+++ b/oss-fuzz/meson.build
@@ -15,6 +15,6 @@ foreach fuzz_program : fuzz_programs
       'dummy-cmd-main.c',
       fuzz_program,
     ],
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
   )
 endforeach
diff --git a/t/helper/meson.build b/t/helper/meson.build
index f502d1aaa3..ae01b3fc45 100644
--- a/t/helper/meson.build
+++ b/t/helper/meson.build
@@ -79,14 +79,14 @@ test_tool_sources = [
 
 test_tool = executable('test-tool',
   sources: test_tool_sources,
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
 )
 bin_wrappers += test_tool
 test_dependencies += test_tool
 
 test_fake_ssh = executable('test-fake-ssh',
   sources: 'test-fake-ssh.c',
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
 )
 bin_wrappers += test_fake_ssh
 test_dependencies += test_fake_ssh
diff --git a/t/meson.build b/t/meson.build
index 35f25ca4a1..dae50601fe 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -39,7 +39,7 @@ clar_sources += custom_target(
 
 clar_unit_tests = executable('unit-tests',
   sources: clar_sources + clar_test_suites,
-  dependencies: [libgit, common_main],
+  dependencies: [libgit_commonmain],
 )
 test('unit-tests', clar_unit_tests)
 
@@ -72,7 +72,7 @@ foreach unit_test_program : unit_test_programs
       'unit-tests/lib-reftable.c',
       unit_test_program,
     ],
-    dependencies: [libgit, common_main],
+    dependencies: [libgit_commonmain],
   )
   test(unit_test_name, unit_test,
     workdir: meson.current_source_dir(),

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 06/13] meson: stop linking libcurl into all executables

Fix this by only propagating include directories as a libgit dependency and propagating the full curl dependency via `libgit_curl`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Show changes to meson.build +4 −2

diff --git a/meson.build b/meson.build
index 820008f6de..4b641095a1 100644
--- a/meson.build
+++ b/meson.build
@@ -927,7 +927,9 @@ if curl.found()
     use_curl_for_imap_send = true
   endif
 
-  libgit_dependencies += curl
+  # Most executables don't have to link against libcurl, but we still need its
+  # include directories so that we can resolve LIBCURL_VERSION in "help.c".
+  libgit_dependencies += curl.partial_dependency(includes: true)
   libgit_c_args += '-DCURL_DISABLE_TYPECHECK'
   build_options_config.set('NO_CURL', '')
 else
@@ -1663,7 +1665,7 @@ if get_option('curl').enabled()
       'http.c',
       'http-walker.c',
     ],
-    dependencies: [libgit_commonmain],
+    dependencies: [libgit_commonmain, curl],
   )
 
   test_dependencies += executable('git-remote-http',

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 05/13] meson: introduce `libgit_curl` dependency

Note that we don't include curl itself as a dependency. This is because we already pull it in transitively via the libgit dependency, which is unfortunate because libgit itself shouldn't actually link against curl in the first place. This will get fixed in the next commit.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 39 ++++++++++++++++++---------------------
 1 file changed, 18 insertions(+), 21 deletions(-)

Show changes to meson.build +18 −21

diff --git a/meson.build b/meson.build
index c7d597eda8..820008f6de 100644
--- a/meson.build
+++ b/meson.build
@@ -1658,30 +1658,32 @@ bin_wrappers += executable('scalar',
 )
 
 if get_option('curl').enabled()
-  curl_sources = [
-    'http.c',
-    'http-walker.c',
-  ]
-
-  git_remote_http = executable('git-remote-http',
-    sources: curl_sources + 'remote-curl.c',
+  libgit_curl = declare_dependency(
+    sources: [
+      'http.c',
+      'http-walker.c',
+    ],
     dependencies: [libgit_commonmain],
+  )
+
+  test_dependencies += executable('git-remote-http',
+    sources: 'remote-curl.c',
+    dependencies: [libgit_curl],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
-  test_dependencies += git_remote_http
 
   test_dependencies += executable('git-http-fetch',
-    sources: curl_sources + 'http-fetch.c',
-    dependencies: [libgit_commonmain],
+    sources: 'http-fetch.c',
+    dependencies: [libgit_curl],
     install: true,
     install_dir: get_option('libexecdir') / 'git-core',
   )
 
   if expat.found()
     test_dependencies += executable('git-http-push',
-      sources: curl_sources + 'http-push.c',
-      dependencies: [libgit_commonmain],
+      sources: 'http-push.c',
+      dependencies: [libgit_curl],
       install: true,
       install_dir: get_option('libexecdir') / 'git-core',
     )
@@ -1689,8 +1691,8 @@ if get_option('curl').enabled()
 
   foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ]
     test_dependencies += executable(alias,
-      objects: git_remote_http.extract_all_objects(recursive: false),
-      dependencies: [libgit],
+      sources: 'remote-curl.c',
+      dependencies: [libgit_curl],
     )
 
     install_symlink(alias + executable_suffix,
@@ -1700,14 +1702,9 @@ if get_option('curl').enabled()
   endforeach
 endif
 
-imap_send_sources = ['imap-send.c']
-if use_curl_for_imap_send
-  imap_send_sources += curl_sources
-endif
-
 test_dependencies += executable('git-imap-send',
-  sources: imap_send_sources,
-  dependencies: [libgit_commonmain],
+  sources: 'imap-send.c',
+  dependencies: [ use_curl_for_imap_send ? libgit_curl : libgit_commonmain ],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:43 UTC · re: Patrick Steinhardt · lore

[PATCH v2 07/13] meson: drop separate version library

Stop building the separate library and instead add "version-def.h" to the list of source files directly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 21 +++++----------------
 1 file changed, 5 insertions(+), 16 deletions(-)

Show changes to meson.build +5 −16

diff --git a/meson.build b/meson.build
index 4b641095a1..e9af093024 100644
--- a/meson.build
+++ b/meson.build
@@ -478,6 +478,7 @@ libgit_sources = [
   'userdiff.c',
   'utf8.c',
   'varint.c',
+  'version.c',
   'versioncmp.c',
   'walker.c',
   'wildmatch.c',
@@ -1542,26 +1543,14 @@ version_def_h = custom_target(
   depends: [git_version_file],
   env: version_gen_environment,
 )
-
-# Build a separate library for "version.c" so that we do not have to rebuild
-# everything when the current Git commit changes.
-libgit_version_library = static_library('git-version',
-  sources: [
-    'version.c',
-    version_def_h,
-  ],
-  c_args: libgit_c_args + [
-    '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
-  ],
-  dependencies: libgit_dependencies,
-  include_directories: libgit_include_directories,
-)
+libgit_sources += version_def_h
 
 libgit = declare_dependency(
   link_with: static_library('git',
     sources: libgit_sources,
-    c_args: libgit_c_args,
-    link_with: libgit_version_library,
+    c_args: libgit_c_args + [
+      '-DGIT_VERSION_H="' + version_def_h.full_path() + '"',
+    ],
     dependencies: libgit_dependencies,
     include_directories: libgit_include_directories,
   ),

-- 
2.48.1.468.gbf5f394be8.dirty

Toon Claes· Feb 7, 2025, 13:24 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 07/13] meson: drop separate version library

Patrick Steinhardt <ps@pks.im> writes:

Show 15 quoted lines

> When building `libgit.a` we link it against a `libgit_version.a` library
> that contains the version information that we inject at build time. The
> intent of this is to avoid rebuilding all of `libgit.a` whenever the
> version changes. But that wouldn't happen in the first place, as we know
> to just rebuild the files that depend on the generated "version-def.h"
> file.
>
> This is an artifact of an earlier version of the Meson build infra that
> didn't ultimately land. We didn't yet have "version-def.h", and instead
> injected the version via preprocessor directives. And here we would have
> rebuilt all of `libgit.a` indeed in case the version changes, because
> the preprocessor directive applied to all files.
>
> Stop building the separate library and instead add "version-def.h" to
> the list of source files directly.

I'm happy to see this "optimazation" go. The solution using "version-def.h" seems simpler and more straightforward. Thanks!

-- Toon

Patrick Steinhardt· Jan 30, 2025, 14:44 UTC · re: Patrick Steinhardt · lore

[PATCH v2 08/13] meson: improve PATH handling

When locating programs required for the build we give some special treatment to Windows systems so that we know to also look up tools provided by a Git for Windows installation. This ensures that the build doesn't have any prerequisites other than Microsoft Visual Studio, Meson and Git for Windows.

Consequently, some of the programs returned by `find_program()` may not be found via PATH, but via these extra directories. But while Meson can use these tools directly without any special treatment, any scripts that we execute may not be able to find those programs. To help them we thus prepend the directories of a subset of the found programs to PATH.

This doesn't make much sense though: we don't need to prepend PATH for any program that was found via PATH, but we really only need to do so for programs located via the extraneous Windows-specific paths. So instead of prepending all programs paths, we really only need to prepend the Windows-specific paths.

Adapt the code accordingly by only prepeding Windows-specific paths to PATH, which both simplifies the code and clarifies intent.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

Show changes to meson.build +7 −7

diff --git a/meson.build b/meson.build
index e9af093024..bc42cd994c 100644
--- a/meson.build
+++ b/meson.build
@@ -198,19 +198,19 @@ endif
 
 cygpath = find_program('cygpath', dirs: program_path, required: false)
 diff = find_program('diff', dirs: program_path)
+git = find_program('git', dirs: program_path, required: false)
 shell = find_program('sh', dirs: program_path)
 tar = find_program('tar', dirs: program_path)
 
-script_environment = environment()
+# Sanity-check that programs required for the build exist.
 foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname']
-  program = find_program(tool, dirs: program_path)
-  script_environment.prepend('PATH', fs.parent(program.full_path()))
+  find_program(tool, dirs: program_path)
 endforeach
 
-git = find_program('git', dirs: program_path, required: false)
-if git.found()
-  script_environment.prepend('PATH', fs.parent(git.full_path()))
-endif
+script_environment = environment()
+foreach path : program_path
+  script_environment.prepend('PATH', path)
+endforeach
 
 if get_option('sane_tool_path') != ''
   script_environment.prepend('PATH', get_option('sane_tool_path'))

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:44 UTC · re: Patrick Steinhardt · lore

[PATCH v2 09/13] meson: improve handling of `sane_tool_path` option

The `sane_tool_path` option can be used to override the PATH variable from which the build process, tests and ultimately Git will end up picking programs from. It is currently lacking though because we only use it to populate the PATH environment variable for executed scripts and for the `BROKEN_PATH_FIX` mechanism, but we don't use it to find programs used in the build process itself.

Fix this issue by treating it similar to the Windows-specific paths, which will make us use it both to find programs and to populate the PATH environment variable.

To help with this fix, change the type of the option to be an array of paths, which makes the handling a bit easier for us. It's also the correct thing to do as the input indeed is a list of paths.

Furthermore, the option now overrides the default behaviour on Windows, which si to pick up tools from Git for Windows. This is done so that it becomes easier to override that default behaviour in case it's not desired.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 meson.build       | 17 ++++++++---------
 meson_options.txt |  4 ++--
 2 files changed, 10 insertions(+), 11 deletions(-)

Show changes to 2 files +10 −11

meson.build, meson_options.txt

diff --git a/meson.build b/meson.build
index bc42cd994c..7180fd58fd 100644
--- a/meson.build
+++ b/meson.build
@@ -191,9 +191,11 @@ project('git', 'c',
 fs = import('fs')
 
 program_path = []
-# Git for Windows provides all the tools we need to build Git.
-if host_machine.system() == 'windows'
-  program_path += [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ]
+if get_option('sane_tool_path').length() != 0
+  program_path = get_option('sane_tool_path')
+elif host_machine.system() == 'windows'
+  # Git for Windows provides all the tools we need to build Git.
+  program_path = [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ]
 endif
 
 cygpath = find_program('cygpath', dirs: program_path, required: false)
@@ -212,10 +214,6 @@ foreach path : program_path
   script_environment.prepend('PATH', path)
 endforeach
 
-if get_option('sane_tool_path') != ''
-  script_environment.prepend('PATH', get_option('sane_tool_path'))
-endif
-
 # The environment used by GIT-VERSION-GEN. Note that we explicitly override
 # environment variables that might be set by the user. This is by design so
 # that we always use whatever Meson has configured instead of what is present
@@ -671,8 +669,9 @@ build_options_config.set('GIT_TEST_UTF8_LOCALE', '')
 build_options_config.set_quoted('LOCALEDIR', fs.as_posix(get_option('prefix') / get_option('localedir')))
 build_options_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb'))
 
-if get_option('sane_tool_path') != ''
-  build_options_config.set_quoted('BROKEN_PATH_FIX', 's|^\# @BROKEN_PATH_FIX@$|git_broken_path_fix "' + get_option('sane_tool_path') + '"|')
+if get_option('sane_tool_path').length() != 0
+  sane_tool_path = (host_machine.system() == 'windows' ? ';' : ':').join(get_option('sane_tool_path'))
+  build_options_config.set_quoted('BROKEN_PATH_FIX', 's|^\# @BROKEN_PATH_FIX@$|git_broken_path_fix "' + sane_tool_path + '"|')
 else
   build_options_config.set_quoted('BROKEN_PATH_FIX', '/^\# @BROKEN_PATH_FIX@$/d')
 endif
diff --git a/meson_options.txt b/meson_options.txt
index c102185ed5..e0e8089891 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -13,8 +13,8 @@ option('perl_cpan_fallback', type: 'boolean', value: true,
   description: 'Install bundled copies of CPAN modules that serve as a fallback in case the modules are not available on the system.')
 option('runtime_prefix', type: 'boolean', value: false,
   description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.')
-option('sane_tool_path', type: 'string', value: '',
-  description: 'A colon-separated list of paths to prepend to PATH if your tools in /usr/bin are broken.')
+option('sane_tool_path', type: 'array', value: [],
+  description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.')
 
 # Build information compiled into Git and other parts like documentation.
 option('build_date', type: 'string', value: '',

-- 
2.48.1.468.gbf5f394be8.dirty

Toon Claes· Feb 7, 2025, 13:49 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 09/13] meson: improve handling of `sane_tool_path` option

Patrick Steinhardt <ps@pks.im> writes:

Show 14 quoted lines

> The `sane_tool_path` option can be used to override the PATH variable
> from which the build process, tests and ultimately Git will end up
> picking programs from. It is currently lacking though because we only
> use it to populate the PATH environment variable for executed scripts
> and for the `BROKEN_PATH_FIX` mechanism, but we don't use it to find
> programs used in the build process itself.
>
> Fix this issue by treating it similar to the Windows-specific paths,
> which will make us use it both to find programs and to populate the PATH
> environment variable.
>
> To help with this fix, change the type of the option to be an array of
> paths, which makes the handling a bit easier for us. It's also the
> correct thing to do as the input indeed is a list of paths.

Should we consider this a breaking change, or aren't we because using Meson is still not the "official" installation method? I've noticed it breaks my "build" directory which was set up before this change and now am trying to use these changes on:

   $ cd build
   $ meson configure

   ERROR: The value of option "sane_tool_path" is "[]", which is not a string.

   $ meson compile
   INFO: autodetecting backend as ninja
   INFO: calculating backend command to run: /home/toon/.local/bin/ninja
   [0/1] Regenerating build files.
   The Meson build system
   Version: 1.6.1
   Source dir: /home/toon/devel/git
   Build dir: /home/toon/devel/git/build
   Build type: native build
   Program sh found: YES (/usr/bin/sh)

   ../meson.build:171:0: ERROR: The value of option "sane_tool_path" is "[]", which is not a string.

   A full log can be found at /home/toon/devel/git/build/meson-logs/meson-log.txt
   FAILED: build.ninja
   /home/toon/.local/bin/meson --internal regenerate /home/toon/devel/git .
   ninja: error: rebuilding 'build.ninja': subcommand failed

-- 
Toon

Patrick Steinhardt· Feb 7, 2025, 14:29 UTC · re: Toon Claes · lore

Re: [PATCH v2 09/13] meson: improve handling of `sane_tool_path` option

On Fri, Feb 07, 2025 at 02:49:12PM +0100, Toon Claes wrote:

Show 21 quoted lines

> Patrick Steinhardt <ps@pks.im> writes:
> 
> > The `sane_tool_path` option can be used to override the PATH variable
> > from which the build process, tests and ultimately Git will end up
> > picking programs from. It is currently lacking though because we only
> > use it to populate the PATH environment variable for executed scripts
> > and for the `BROKEN_PATH_FIX` mechanism, but we don't use it to find
> > programs used in the build process itself.
> >
> > Fix this issue by treating it similar to the Windows-specific paths,
> > which will make us use it both to find programs and to populate the PATH
> > environment variable.
> >
> > To help with this fix, change the type of the option to be an array of
> > paths, which makes the handling a bit easier for us. It's also the
> > correct thing to do as the input indeed is a list of paths.
> 
> Should we consider this a breaking change, or aren't we because using
> Meson is still not the "official" installation method? I've noticed it
> breaks my "build" directory which was set up before this change and now
> am trying to use these changes on:

Meson is still marked as experimental right now, so I don't think this needs to be explicitly called out. We should start to be more mindful of any backwards-incompatible changes once Meson support has matured a bit, say for example starting with Git v2.49 or v2.50. But until then it's kind of expected that we'll still have to iterate a bit.

Patrick

Patrick Steinhardt· Jan 30, 2025, 14:44 UTC · re: Patrick Steinhardt · lore

[PATCH v2 10/13] meson: prevent finding sed(1) in a loop

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/howto/meson.build | 2 +-
 meson.build                     | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

Show changes to 2 files +3 −2

Documentation/howto/meson.build, meson.build

diff --git a/Documentation/howto/meson.build b/Documentation/howto/meson.build
index c023c10416..92a08b13ee 100644
--- a/Documentation/howto/meson.build
+++ b/Documentation/howto/meson.build
@@ -41,7 +41,7 @@ custom_target(
 foreach howto : howto_sources
   howto_stripped = custom_target(
     command: [
-      find_program('sed'),
+      sed,
       '-e',
       '1,/^$/d',
       '@INPUT@',
diff --git a/meson.build b/meson.build
index 7180fd58fd..b5e8aca288 100644
--- a/meson.build
+++ b/meson.build
@@ -201,11 +201,12 @@ endif
 cygpath = find_program('cygpath', dirs: program_path, required: false)
 diff = find_program('diff', dirs: program_path)
 git = find_program('git', dirs: program_path, required: false)
+sed = find_program('sed', dirs: program_path)
 shell = find_program('sh', dirs: program_path)
 tar = find_program('tar', dirs: program_path)
 
 # Sanity-check that programs required for the build exist.
-foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname']
+foreach tool : ['cat', 'cut', 'grep', 'sort', 'tr', 'uname']
   find_program(tool, dirs: program_path)
 endforeach

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:44 UTC · re: Patrick Steinhardt · lore

[PATCH v2 11/13] meson: fix overwritten `git` variable

We're assigning the `git` variable in three places:

  - In "meson.build" to store the external Git executable.

  - In "meson.build" to store the compiled Git executable.

  - In "Documentation/meson.build" to store the external Git executable,
    a second time.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 1 -
 meson.build               | 6 +++---
 2 files changed, 3 insertions(+), 4 deletions(-)

Show changes to 2 files +3 −4

Documentation/meson.build, meson.build

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 2a26fa8a5f..6438fa6792 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -283,7 +283,6 @@ elif docs_backend == 'asciidoctor'
   ]
 endif
 
-git = find_program('git', required: false)
 xmlto = find_program('xmlto')
 
 cmd_lists = [
diff --git a/meson.build b/meson.build
index b5e8aca288..cf3f0d59d5 100644
--- a/meson.build
+++ b/meson.build
@@ -1605,13 +1605,13 @@ libgit_commonmain = declare_dependency(
 bin_wrappers = [ ]
 test_dependencies = [ ]
 
-git = executable('git',
+git_builtin = executable('git',
   sources: builtin_sources + 'git.c',
   dependencies: [libgit_commonmain],
   install: true,
   install_dir: get_option('libexecdir') / 'git-core',
 )
-bin_wrappers += git
+bin_wrappers += git_builtin
 
 test_dependencies += executable('git-daemon',
   sources: 'daemon.c',
@@ -1702,7 +1702,7 @@ test_dependencies += executable('git-imap-send',
 
 foreach alias : [ 'git-receive-pack', 'git-upload-archive', 'git-upload-pack' ]
   bin_wrappers += executable(alias,
-    objects: git.extract_all_objects(recursive: false),
+    objects: git_builtin.extract_all_objects(recursive: false),
     dependencies: [libgit],
   )

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:44 UTC · re: Patrick Steinhardt · lore

[PATCH v2 12/13] meson: consistently use custom program paths to resolve programs

Adapt the calls to use the program path to fix this. While at it, drop `required: true` arguments, which are the default anyway.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/meson.build | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

Show changes to Documentation/meson.build +6 −6

diff --git a/Documentation/meson.build b/Documentation/meson.build
index 6438fa6792..c6117366ff 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -206,9 +206,9 @@ manpages = {
 
 docs_backend = get_option('docs_backend')
 if docs_backend == 'auto'
-  if find_program('asciidoc', required: false).found()
+  if find_program('asciidoc', dirs: program_path, required: false).found()
     docs_backend = 'asciidoc'
-  elif find_program('asciidoctor', required: false).found()
+  elif find_program('asciidoctor', dirs: program_path, required: false).found()
     docs_backend = 'asciidoctor'
   else
     error('Neither asciidoc nor asciidoctor were found.')
@@ -216,7 +216,7 @@ if docs_backend == 'auto'
 endif
 
 if docs_backend == 'asciidoc'
-  asciidoc = find_program('asciidoc', required: true)
+  asciidoc = find_program('asciidoc', dirs: program_path)
   asciidoc_html = 'xhtml11'
   asciidoc_docbook = 'docbook'
   xmlto_extra = [ ]
@@ -245,7 +245,7 @@ if docs_backend == 'asciidoc'
     asciidoc_conf,
   ]
 elif docs_backend == 'asciidoctor'
-  asciidoctor = find_program('asciidoctor', required: true)
+  asciidoctor = find_program('asciidoctor', dirs: program_path)
   asciidoc_html = 'xhtml5'
   asciidoc_docbook = 'docbook5'
   xmlto_extra = [
@@ -283,7 +283,7 @@ elif docs_backend == 'asciidoctor'
   ]
 endif
 
-xmlto = find_program('xmlto')
+xmlto = find_program('xmlto', dirs: program_path)
 
 cmd_lists = [
   'cmds-ancillaryinterrogators.txt',
@@ -404,7 +404,7 @@ if get_option('docs').contains('html')
     pointing_to: 'git.html',
   )
 
-  xsltproc = find_program('xsltproc')
+  xsltproc = find_program('xsltproc', dirs: program_path)
 
   user_manual_xml = custom_target(
     command: asciidoc_common_options + [

-- 
2.48.1.468.gbf5f394be8.dirty

Patrick Steinhardt· Jan 30, 2025, 14:44 UTC · re: Patrick Steinhardt · lore

[PATCH v2 13/13] gitlab-ci: restrict maximum number of link jobs on Windows

The hosted Windows runners on GitLab.com only have 7.5GB of RAM. Given that "link.exe" provided by Microsoft Visual Studio is multi-threaded by itself already and thus quite memory hungry this can quickly lead to memory starvation, out-of-memory situations and thus failed CI jobs.

Fix the issue by limiting the number of concurrent linker jobs. The same issue hasn't been observed on GitHub Actions yet, probably because it got more than twice the amount of RAM with 16GB.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .gitlab-ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Show changes to .gitlab-ci.yml +1 −1

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 4976e18a05..7e1cecc6a7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -169,7 +169,7 @@ build:msvc-meson:
   extends: .msvc-meson
   stage: build
   script:
-    - meson setup build -Dperl=disabled
+    - meson setup build -Dperl=disabled -Dbackend_max_links=1
     - meson compile -C build
   artifacts:
     paths:

-- 
2.48.1.468.gbf5f394be8.dirty

Justin Tobler· Feb 7, 2025, 15:22 UTC · re: Patrick Steinhardt · lore

Re: [PATCH v2 00/13] meson: cleanups, improvements, smallish fixes

On 25/01/30 03:43PM, Patrick Steinhardt wrote:

Show 22 quoted lines

> Hi,
> 
> this patch series contains a mostly-random set of smallish improvements,
> simplifications and fixes to the Meson build instructions.
> 
> Changes in v2:
>   - Redo the two commits regarding PATH handling. We now only prepend
>     system-specific paths to PATH and handle the `-Dsane_tool_path`
>     option the same.
>   - Continue to propagate libcurl include directories into libgit, as we
>     need it for "help.c" to resolve LIBCURL_VERSION. Reorder the patch
>     to minimize the required changes.
>   - Fix a commit message typo.
>   - Include another commit that fixes the OpenSSL fallback on Windows.
>   - Include another commit to fix linker errors due to out-of-memory
>     situations on Windows with GitLab CI.
>   - Link to v1: https://lore.kernel.org/r/20250129-b4-pks-meson-improvements-v1-0-ab709f0be12c@pks.im
> 
> I have furthermore pulled in ps/build-meson-fixes at 7304bd2bc3 (ci:
> wire up Visual Studio build with Meson, 2025-01-22) as a dependency of
> this series. This is due to a semantic conflict with 28911f7dca (meson:
> wire up fuzzers, 2025-01-22).

Thanks Patrick! I've reviewed the changes in this version and tested building locally. This version looks good to me.

-Justin