# [PATCH v3 01/13] meson: fix exec path with enabled runtime prefix 14 messages from 2025-02-26 to 2025-02-26. Participants: Patrick Steinhardt. Thread: https://gitlist.dev/t/63006 ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 01/13] meson: fix exec path with enabled runtime prefix Message-ID: <20250226-b4-pks-meson-improvements-v3-1-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-1-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` When the runtime prefix option is enabled, Git is built such that it knows to locate its binaries relative to the directory a binary is being executed from. This requires us to figure out relative paths, which is handled in `system_prefix()` by trying to strip a couple of well-known paths. One of these paths, GIT_EXEC_PATH, is expected to be absolute when runtime prefixes are enabled, but relative otherwise. And while our Makefile gets this correctly, in Meson we always wire up the absolute path, which may result in us not being able to find binaries. Fix this by conditionally injecting the paths depending on whether or not the `runtime_prefix` option is enabled. Signed-off-by: Patrick Steinhardt --- meson.build | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/meson.build b/meson.build index dbc1dab605e..1f83dc58c24 100644 --- a/meson.build +++ b/meson.build @@ -691,7 +691,6 @@ libgit_c_args = [ '-DETC_GITATTRIBUTES="' + get_option('gitattributes') + '"', '-DETC_GITCONFIG="' + get_option('gitconfig') + '"', '-DFALLBACK_RUNTIME_PREFIX="' + get_option('prefix') + '"', - '-DGIT_EXEC_PATH="' + get_option('prefix') / get_option('libexecdir') / 'git-core"', '-DGIT_HOST_CPU="' + host_machine.cpu_family() + '"', '-DGIT_HTML_PATH="' + get_option('datadir') / 'doc/git-doc"', '-DGIT_INFO_PATH="' + get_option('infodir') + '"', @@ -1466,6 +1465,7 @@ endif if get_option('runtime_prefix') libgit_c_args += '-DRUNTIME_PREFIX' build_options_config.set('RUNTIME_PREFIX', 'true') + git_exec_path = get_option('libexecdir') / 'git-core' if compiler.has_header('mach-o/dyld.h') libgit_c_args += '-DHAVE_NS_GET_EXECUTABLE_PATH' @@ -1502,7 +1502,9 @@ if get_option('runtime_prefix') endif else build_options_config.set('RUNTIME_PREFIX', 'false') + git_exec_path = get_option('prefix') / get_option('libexecdir') / 'git-core' endif +libgit_c_args += '-DGIT_EXEC_PATH="' + git_exec_path + '"' git_version_file = custom_target( command: [ -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 00/13] meson: cleanups, improvements, smallish fixes Message-ID: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae%40pks.im In-Reply-To: <20250129-b4-pks-meson-improvements-v1-0-ab709f0be12c@pks.im> ``` Hi, this patch series contains a mostly-random set of smallish improvements, simplifications and fixes to the Meson build instructions. Changes in v2: - Redo the two commits regarding PATH handling. We now only prepend system-specific paths to PATH and handle the `-Dsane_tool_path` option the same. - Continue to propagate libcurl include directories into libgit, as we need it for "help.c" to resolve LIBCURL_VERSION. Reorder the patch to minimize the required changes. - Fix a commit message typo. - Include another commit that fixes the OpenSSL fallback on Windows. - Include another commit to fix linker errors due to out-of-memory situations on Windows with GitLab CI. - Link to v1: https://lore.kernel.org/r/20250129-b4-pks-meson-improvements-v1-0-ab709f0be12c@pks.im Changes in v3: - Fix missing linker arguments for a couple of aliased executables on Windows with MSVC. - Link to v2: https://lore.kernel.org/r/20250130-b4-pks-meson-improvements-v2-0-2f05581ffb44@pks.im I've kicked off an extra pipeline at [1] to verify that this version works on next and doesn't break anything. Thanks! Patrick [1]: https://github.com/git/git/pull/1898 --- Patrick Steinhardt (13): meson: fix exec path with enabled runtime prefix meson: fix OpenSSL fallback when not explicitly required meson: inline the static 'git' library meson: simplify use of the common-main library meson: introduce `libgit_curl` dependency meson: stop linking libcurl into all executables meson: drop separate version library meson: improve PATH handling meson: improve handling of `sane_tool_path` option meson: prevent finding sed(1) in a loop meson: fix overwritten `git` variable meson: consistently use custom program paths to resolve programs gitlab-ci: restrict maximum number of link jobs on Windows .gitlab-ci.yml | 2 +- Documentation/howto/meson.build | 2 +- Documentation/meson.build | 13 ++-- meson.build | 156 +++++++++++++++++++--------------------- meson_options.txt | 4 +- oss-fuzz/meson.build | 2 +- t/helper/meson.build | 4 +- t/meson.build | 4 +- 8 files changed, 89 insertions(+), 98 deletions(-) Range-diff versus v2: 1: 94730a8bdb4 = 1: fd7097ab8bd meson: fix exec path with enabled runtime prefix 2: 97710a79d0f = 2: ff8d4cd70c6 meson: fix OpenSSL fallback when not explicitly required 3: 66a8c6f24d0 = 3: 238a45c8bb0 meson: inline the static 'git' library 4: f717119b1a0 ! 4: 1bd9f892310 meson: simplify use of the common-main library @@ Commit message instead of creating a static library, we now instead compile the common set of files into each executable separately. - This change surfaces an issue when linking aliases for git-remote-http: - we extract all objects from `git-remote-http` et al and then link them - into the new executable. As such, these objects would already contain - a `main()` function. But now that we also compile "common-main.c" into - these aliased executables we see a linker error due to `main()` being - defined twice. We fix this by only linking against `libgit.a`. - Signed-off-by: Patrick Steinhardt ## meson.build ## @@ meson.build: if host_machine.system() == 'windows' endif endif -common_main_library = static_library('common-main', -+ -+libgit_commonmain = declare_dependency( - sources: common_main_sources, +- sources: common_main_sources, - c_args: libgit_c_args, - dependencies: libgit_dependencies, - include_directories: libgit_include_directories, -) -common_main = declare_dependency( - link_with: common_main_library, ++ ++libgit_commonmain = declare_dependency( ++ link_with: static_library('common-main', ++ sources: common_main_sources, ++ dependencies: [ libgit ], ++ ), link_args: common_main_link_args, + dependencies: [ libgit ], ) @@ meson.build: if get_option('curl').enabled() test_dependencies += executable(alias, objects: git_remote_http.extract_all_objects(recursive: false), - dependencies: [libgit, common_main], -+ dependencies: [libgit], ++ dependencies: [libgit_commonmain], ) install_symlink(alias + executable_suffix, @@ meson.build: test_dependencies += executable('git-imap-send', bin_wrappers += executable(alias, objects: git.extract_all_objects(recursive: false), - dependencies: [libgit, common_main], -+ dependencies: [libgit], ++ dependencies: [libgit_commonmain], ) install_symlink(alias + executable_suffix, 5: 1b61303b130 ! 5: 024f4297f5e meson: introduce `libgit_curl` dependency @@ meson.build: if get_option('curl').enabled() foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ] test_dependencies += executable(alias, - objects: git_remote_http.extract_all_objects(recursive: false), -- dependencies: [libgit], +- dependencies: [libgit_commonmain], + sources: 'remote-curl.c', + dependencies: [libgit_curl], ) 6: 70dc5edc05c = 6: 7fa4957e256 meson: stop linking libcurl into all executables 7: 7fe6d1d4e74 = 7: b24c4e06deb meson: drop separate version library 8: 1d0597eb7cf = 8: 8ca171f9694 meson: improve PATH handling 9: 9219ab697a7 = 9: beedc776a7b meson: improve handling of `sane_tool_path` option 10: 7e3e0bb409f = 10: 018f549375f meson: prevent finding sed(1) in a loop 11: c1243c17dbe ! 11: 3da8f4f105d meson: fix overwritten `git` variable @@ meson.build: test_dependencies += executable('git-imap-send', bin_wrappers += executable(alias, - objects: git.extract_all_objects(recursive: false), + objects: git_builtin.extract_all_objects(recursive: false), - dependencies: [libgit], + dependencies: [libgit_commonmain], ) 12: f06585cd99e = 12: b2febf1acac meson: consistently use custom program paths to resolve programs 13: 02f0ffdd362 = 13: f5bb92726dc gitlab-ci: restrict maximum number of link jobs on Windows --- base-commit: 6dd4e543f2219290db9faa61b30f7f30a34e4b50 change-id: 20250121-b4-pks-meson-improvements-3e575363e91c ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 03/13] meson: inline the static 'git' library Message-ID: <20250226-b4-pks-meson-improvements-v3-3-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-3-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` When setting up `libgit.a` we first create the static library itself, and then declare it as part of a dependency such that compile arguments, include directories and transitive dependencies get propagated to the users of that library. As such, the static library isn't expected to be used by anything but the declared dependency. Inline the static library so that we don't even use a separate variable for it. This avoids any kind of confusion that may arise and clarifies how the library is supposed to be used. Signed-off-by: Patrick Steinhardt --- meson.build | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/meson.build b/meson.build index 83ed55c75c6..a124101a73a 100644 --- a/meson.build +++ b/meson.build @@ -1555,17 +1555,15 @@ libgit_version_library = static_library('git-version', include_directories: libgit_include_directories, ) -libgit_library = static_library('git', - sources: libgit_sources, - c_args: libgit_c_args, - link_with: libgit_version_library, - dependencies: libgit_dependencies, - include_directories: libgit_include_directories, -) - libgit = declare_dependency( + link_with: static_library('git', + sources: libgit_sources, + c_args: libgit_c_args, + link_with: libgit_version_library, + dependencies: libgit_dependencies, + include_directories: libgit_include_directories, + ), compile_args: libgit_c_args, - link_with: libgit_library, dependencies: libgit_dependencies, include_directories: libgit_include_directories, ) -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 02/13] meson: fix OpenSSL fallback when not explicitly required Message-ID: <20250226-b4-pks-meson-improvements-v3-2-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-2-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` When OpenSSL isn't provided by the system we know to fall back to the subproject wrapper. This is especially helpful on Windows systems, where you typically don't have OpenSSL available, in order to reduce the number of required dependencies. The fallback is broken though when the OpenSSL backend is set to 'auto' as we end up calling `dependency('openssl', required: false)` in that case, which implicitly disables falling back to the wrapper. Fix the issue by re-allowing the fallback in case either OpenSSL is required or in case the backend is set to 'auto'. While at it, fix reporting of the backend in case the user asked us to pick no HTTPS backend at all. Signed-off-by: Patrick Steinhardt --- meson.build | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/meson.build b/meson.build index 1f83dc58c24..83ed55c75c6 100644 --- a/meson.build +++ b/meson.build @@ -1352,7 +1352,11 @@ if https_backend == 'auto' and security_framework.found() endif openssl_required = 'openssl' in [csprng_backend, https_backend, sha1_backend, sha1_unsafe_backend, sha256_backend] -openssl = dependency('openssl', required: openssl_required, default_options: ['default_library=static']) +openssl = dependency('openssl', + required: openssl_required, + allow_fallback: openssl_required or https_backend == 'auto', + default_options: ['default_library=static'], +) if https_backend == 'auto' and openssl.found() https_backend = 'openssl' endif @@ -1366,6 +1370,7 @@ elif https_backend == 'openssl' else # We either couldn't find any dependencies with 'auto' or the user requested # 'none'. Both cases are benign. + https_backend = 'none' endif if https_backend != 'openssl' -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 04/13] meson: simplify use of the common-main library Message-ID: <20250226-b4-pks-meson-improvements-v3-4-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-4-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` The "common-main.c" file is used by multiple executables. In order to make it easy to set it up we have created a separate library that these executables can link against. All of these executables also want to link against `libgit.a` though, which makes it necessary to specify both of these as dependencies for every executable. Simplify this a bit by declaring the library as a source dependency: instead of creating a static library, we now instead compile the common set of files into each executable separately. Signed-off-by: Patrick Steinhardt --- meson.build | 39 +++++++++++++++++++-------------------- oss-fuzz/meson.build | 2 +- t/helper/meson.build | 4 ++-- t/meson.build | 4 ++-- 4 files changed, 24 insertions(+), 25 deletions(-) diff --git a/meson.build b/meson.build index a124101a73a..7236c163374 100644 --- a/meson.build +++ b/meson.build @@ -1604,15 +1604,14 @@ if host_machine.system() == 'windows' error('Unsupported compiler ' + compiler.get_id()) endif endif -common_main_library = static_library('common-main', - sources: common_main_sources, - c_args: libgit_c_args, - dependencies: libgit_dependencies, - include_directories: libgit_include_directories, -) -common_main = declare_dependency( - link_with: common_main_library, + +libgit_commonmain = declare_dependency( + link_with: static_library('common-main', + sources: common_main_sources, + dependencies: [ libgit ], + ), link_args: common_main_link_args, + dependencies: [ libgit ], ) bin_wrappers = [ ] @@ -1620,7 +1619,7 @@ test_dependencies = [ ] git = executable('git', sources: builtin_sources + 'git.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1628,35 +1627,35 @@ bin_wrappers += git test_dependencies += executable('git-daemon', sources: 'daemon.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) test_dependencies += executable('git-sh-i18n--envsubst', sources: 'sh-i18n--envsubst.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) bin_wrappers += executable('git-shell', sources: 'shell.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) test_dependencies += executable('git-http-backend', sources: 'http-backend.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) bin_wrappers += executable('scalar', sources: 'scalar.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1669,7 +1668,7 @@ if get_option('curl').enabled() git_remote_http = executable('git-remote-http', sources: curl_sources + 'remote-curl.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1677,7 +1676,7 @@ if get_option('curl').enabled() test_dependencies += executable('git-http-fetch', sources: curl_sources + 'http-fetch.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1685,7 +1684,7 @@ if get_option('curl').enabled() if expat.found() test_dependencies += executable('git-http-push', sources: curl_sources + 'http-push.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1694,7 +1693,7 @@ if get_option('curl').enabled() foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ] test_dependencies += executable(alias, objects: git_remote_http.extract_all_objects(recursive: false), - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) install_symlink(alias + executable_suffix, @@ -1711,7 +1710,7 @@ endif test_dependencies += executable('git-imap-send', sources: imap_send_sources, - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1719,7 +1718,7 @@ test_dependencies += executable('git-imap-send', foreach alias : [ 'git-receive-pack', 'git-upload-archive', 'git-upload-pack' ] bin_wrappers += executable(alias, objects: git.extract_all_objects(recursive: false), - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) install_symlink(alias + executable_suffix, diff --git a/oss-fuzz/meson.build b/oss-fuzz/meson.build index ed796655016..878afd8426f 100644 --- a/oss-fuzz/meson.build +++ b/oss-fuzz/meson.build @@ -15,6 +15,6 @@ foreach fuzz_program : fuzz_programs 'dummy-cmd-main.c', fuzz_program, ], - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) endforeach diff --git a/t/helper/meson.build b/t/helper/meson.build index f502d1aaa36..ae01b3fc45d 100644 --- a/t/helper/meson.build +++ b/t/helper/meson.build @@ -79,14 +79,14 @@ test_tool_sources = [ test_tool = executable('test-tool', sources: test_tool_sources, - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) bin_wrappers += test_tool test_dependencies += test_tool test_fake_ssh = executable('test-fake-ssh', sources: 'test-fake-ssh.c', - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) bin_wrappers += test_fake_ssh test_dependencies += test_fake_ssh diff --git a/t/meson.build b/t/meson.build index 35f25ca4a1d..dae50601fec 100644 --- a/t/meson.build +++ b/t/meson.build @@ -39,7 +39,7 @@ clar_sources += custom_target( clar_unit_tests = executable('unit-tests', sources: clar_sources + clar_test_suites, - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) test('unit-tests', clar_unit_tests) @@ -72,7 +72,7 @@ foreach unit_test_program : unit_test_programs 'unit-tests/lib-reftable.c', unit_test_program, ], - dependencies: [libgit, common_main], + dependencies: [libgit_commonmain], ) test(unit_test_name, unit_test, workdir: meson.current_source_dir(), -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 05/13] meson: introduce `libgit_curl` dependency Message-ID: <20250226-b4-pks-meson-improvements-v3-5-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-5-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` We've got a set of common source files that we use for those executables that link against libcurl. The setup is somewhat repetitive though. Simplify it by declaring a `libgit_curl` dependency that bundles all of it together. Note that we don't include curl itself as a dependency. This is because we already pull it in transitively via the libgit dependency, which is unfortunate because libgit itself shouldn't actually link against curl in the first place. This will get fixed in the next commit. Signed-off-by: Patrick Steinhardt --- meson.build | 39 ++++++++++++++++++--------------------- 1 file changed, 18 insertions(+), 21 deletions(-) diff --git a/meson.build b/meson.build index 7236c163374..e58462ac4fe 100644 --- a/meson.build +++ b/meson.build @@ -1661,30 +1661,32 @@ bin_wrappers += executable('scalar', ) if get_option('curl').enabled() - curl_sources = [ - 'http.c', - 'http-walker.c', - ] - - git_remote_http = executable('git-remote-http', - sources: curl_sources + 'remote-curl.c', + libgit_curl = declare_dependency( + sources: [ + 'http.c', + 'http-walker.c', + ], dependencies: [libgit_commonmain], + ) + + test_dependencies += executable('git-remote-http', + sources: 'remote-curl.c', + dependencies: [libgit_curl], install: true, install_dir: get_option('libexecdir') / 'git-core', ) - test_dependencies += git_remote_http test_dependencies += executable('git-http-fetch', - sources: curl_sources + 'http-fetch.c', - dependencies: [libgit_commonmain], + sources: 'http-fetch.c', + dependencies: [libgit_curl], install: true, install_dir: get_option('libexecdir') / 'git-core', ) if expat.found() test_dependencies += executable('git-http-push', - sources: curl_sources + 'http-push.c', - dependencies: [libgit_commonmain], + sources: 'http-push.c', + dependencies: [libgit_curl], install: true, install_dir: get_option('libexecdir') / 'git-core', ) @@ -1692,8 +1694,8 @@ if get_option('curl').enabled() foreach alias : [ 'git-remote-https', 'git-remote-ftp', 'git-remote-ftps' ] test_dependencies += executable(alias, - objects: git_remote_http.extract_all_objects(recursive: false), - dependencies: [libgit_commonmain], + sources: 'remote-curl.c', + dependencies: [libgit_curl], ) install_symlink(alias + executable_suffix, @@ -1703,14 +1705,9 @@ if get_option('curl').enabled() endforeach endif -imap_send_sources = ['imap-send.c'] -if use_curl_for_imap_send - imap_send_sources += curl_sources -endif - test_dependencies += executable('git-imap-send', - sources: imap_send_sources, - dependencies: [libgit_commonmain], + sources: 'imap-send.c', + dependencies: [ use_curl_for_imap_send ? libgit_curl : libgit_commonmain ], install: true, install_dir: get_option('libexecdir') / 'git-core', ) -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 06/13] meson: stop linking libcurl into all executables Message-ID: <20250226-b4-pks-meson-improvements-v3-6-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-6-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` We set up libcurl via the `libgit_dependencies` variable, which gets propagated into every user of the `libgit` dependency. This is not necessary though, as most of our executables aren't even supposed to link against libcurl. Fix this by only propagating include directories as a libgit dependency and propagating the full curl dependency via `libgit_curl`. Signed-off-by: Patrick Steinhardt --- meson.build | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meson.build b/meson.build index e58462ac4fe..39d15ee6cb8 100644 --- a/meson.build +++ b/meson.build @@ -927,7 +927,9 @@ if curl.found() use_curl_for_imap_send = true endif - libgit_dependencies += curl + # Most executables don't have to link against libcurl, but we still need its + # include directories so that we can resolve LIBCURL_VERSION in "help.c". + libgit_dependencies += curl.partial_dependency(includes: true) libgit_c_args += '-DCURL_DISABLE_TYPECHECK' build_options_config.set('NO_CURL', '') else @@ -1666,7 +1668,7 @@ if get_option('curl').enabled() 'http.c', 'http-walker.c', ], - dependencies: [libgit_commonmain], + dependencies: [libgit_commonmain, curl], ) test_dependencies += executable('git-remote-http', -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 07/13] meson: drop separate version library Message-ID: <20250226-b4-pks-meson-improvements-v3-7-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-7-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` When building `libgit.a` we link it against a `libgit_version.a` library that contains the version information that we inject at build time. The intent of this is to avoid rebuilding all of `libgit.a` whenever the version changes. But that wouldn't happen in the first place, as we know to just rebuild the files that depend on the generated "version-def.h" file. This is an artifact of an earlier version of the Meson build infra that didn't ultimately land. We didn't yet have "version-def.h", and instead injected the version via preprocessor directives. And here we would have rebuilt all of `libgit.a` indeed in case the version changes, because the preprocessor directive applied to all files. Stop building the separate library and instead add "version-def.h" to the list of source files directly. Signed-off-by: Patrick Steinhardt --- meson.build | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-) diff --git a/meson.build b/meson.build index 39d15ee6cb8..c8df19804ae 100644 --- a/meson.build +++ b/meson.build @@ -478,6 +478,7 @@ libgit_sources = [ 'userdiff.c', 'utf8.c', 'varint.c', + 'version.c', 'versioncmp.c', 'walker.c', 'wildmatch.c', @@ -1542,26 +1543,14 @@ version_def_h = custom_target( depends: [git_version_file], env: version_gen_environment, ) - -# Build a separate library for "version.c" so that we do not have to rebuild -# everything when the current Git commit changes. -libgit_version_library = static_library('git-version', - sources: [ - 'version.c', - version_def_h, - ], - c_args: libgit_c_args + [ - '-DGIT_VERSION_H="' + version_def_h.full_path() + '"', - ], - dependencies: libgit_dependencies, - include_directories: libgit_include_directories, -) +libgit_sources += version_def_h libgit = declare_dependency( link_with: static_library('git', sources: libgit_sources, - c_args: libgit_c_args, - link_with: libgit_version_library, + c_args: libgit_c_args + [ + '-DGIT_VERSION_H="' + version_def_h.full_path() + '"', + ], dependencies: libgit_dependencies, include_directories: libgit_include_directories, ), -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 08/13] meson: improve PATH handling Message-ID: <20250226-b4-pks-meson-improvements-v3-8-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-8-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` When locating programs required for the build we give some special treatment to Windows systems so that we know to also look up tools provided by a Git for Windows installation. This ensures that the build doesn't have any prerequisites other than Microsoft Visual Studio, Meson and Git for Windows. Consequently, some of the programs returned by `find_program()` may not be found via PATH, but via these extra directories. But while Meson can use these tools directly without any special treatment, any scripts that we execute may not be able to find those programs. To help them we thus prepend the directories of a subset of the found programs to PATH. This doesn't make much sense though: we don't need to prepend PATH for any program that was found via PATH, but we really only need to do so for programs located via the extraneous Windows-specific paths. So instead of prepending all programs paths, we really only need to prepend the Windows-specific paths. Adapt the code accordingly by only prepeding Windows-specific paths to PATH, which both simplifies the code and clarifies intent. Signed-off-by: Patrick Steinhardt --- meson.build | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/meson.build b/meson.build index c8df19804ae..acd6074b32d 100644 --- a/meson.build +++ b/meson.build @@ -198,19 +198,19 @@ endif cygpath = find_program('cygpath', dirs: program_path, required: false) diff = find_program('diff', dirs: program_path) +git = find_program('git', dirs: program_path, required: false) shell = find_program('sh', dirs: program_path) tar = find_program('tar', dirs: program_path) -script_environment = environment() +# Sanity-check that programs required for the build exist. foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname'] - program = find_program(tool, dirs: program_path) - script_environment.prepend('PATH', fs.parent(program.full_path())) + find_program(tool, dirs: program_path) endforeach -git = find_program('git', dirs: program_path, required: false) -if git.found() - script_environment.prepend('PATH', fs.parent(git.full_path())) -endif +script_environment = environment() +foreach path : program_path + script_environment.prepend('PATH', path) +endforeach if get_option('sane_tool_path') != '' script_environment.prepend('PATH', get_option('sane_tool_path')) -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 09/13] meson: improve handling of `sane_tool_path` option Message-ID: <20250226-b4-pks-meson-improvements-v3-9-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-9-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` The `sane_tool_path` option can be used to override the PATH variable from which the build process, tests and ultimately Git will end up picking programs from. It is currently lacking though because we only use it to populate the PATH environment variable for executed scripts and for the `BROKEN_PATH_FIX` mechanism, but we don't use it to find programs used in the build process itself. Fix this issue by treating it similar to the Windows-specific paths, which will make us use it both to find programs and to populate the PATH environment variable. To help with this fix, change the type of the option to be an array of paths, which makes the handling a bit easier for us. It's also the correct thing to do as the input indeed is a list of paths. Furthermore, the option now overrides the default behaviour on Windows, which si to pick up tools from Git for Windows. This is done so that it becomes easier to override that default behaviour in case it's not desired. Signed-off-by: Patrick Steinhardt --- meson.build | 17 ++++++++--------- meson_options.txt | 4 ++-- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/meson.build b/meson.build index acd6074b32d..ed857aaa4ab 100644 --- a/meson.build +++ b/meson.build @@ -191,9 +191,11 @@ project('git', 'c', fs = import('fs') program_path = [] -# Git for Windows provides all the tools we need to build Git. -if host_machine.system() == 'windows' - program_path += [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ] +if get_option('sane_tool_path').length() != 0 + program_path = get_option('sane_tool_path') +elif host_machine.system() == 'windows' + # Git for Windows provides all the tools we need to build Git. + program_path = [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ] endif cygpath = find_program('cygpath', dirs: program_path, required: false) @@ -212,10 +214,6 @@ foreach path : program_path script_environment.prepend('PATH', path) endforeach -if get_option('sane_tool_path') != '' - script_environment.prepend('PATH', get_option('sane_tool_path')) -endif - # The environment used by GIT-VERSION-GEN. Note that we explicitly override # environment variables that might be set by the user. This is by design so # that we always use whatever Meson has configured instead of what is present @@ -671,8 +669,9 @@ build_options_config.set('GIT_TEST_UTF8_LOCALE', '') build_options_config.set_quoted('LOCALEDIR', fs.as_posix(get_option('prefix') / get_option('localedir'))) build_options_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb')) -if get_option('sane_tool_path') != '' - build_options_config.set_quoted('BROKEN_PATH_FIX', 's|^\# @BROKEN_PATH_FIX@$|git_broken_path_fix "' + get_option('sane_tool_path') + '"|') +if get_option('sane_tool_path').length() != 0 + sane_tool_path = (host_machine.system() == 'windows' ? ';' : ':').join(get_option('sane_tool_path')) + build_options_config.set_quoted('BROKEN_PATH_FIX', 's|^\# @BROKEN_PATH_FIX@$|git_broken_path_fix "' + sane_tool_path + '"|') else build_options_config.set_quoted('BROKEN_PATH_FIX', '/^\# @BROKEN_PATH_FIX@$/d') endif diff --git a/meson_options.txt b/meson_options.txt index c102185ed5e..e0e8089891b 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -13,8 +13,8 @@ option('perl_cpan_fallback', type: 'boolean', value: true, description: 'Install bundled copies of CPAN modules that serve as a fallback in case the modules are not available on the system.') option('runtime_prefix', type: 'boolean', value: false, description: 'Resolve ancillary tooling and support files relative to the location of the runtime binary instead of hard-coding them into the binary.') -option('sane_tool_path', type: 'string', value: '', - description: 'A colon-separated list of paths to prepend to PATH if your tools in /usr/bin are broken.') +option('sane_tool_path', type: 'array', value: [], + description: 'An array of paths to pick up tools from in case the normal tools are broken or lacking.') # Build information compiled into Git and other parts like documentation. option('build_date', type: 'string', value: '', -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 10/13] meson: prevent finding sed(1) in a loop Message-ID: <20250226-b4-pks-meson-improvements-v3-10-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-10-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` We're searching for the sed(1) executable in a loop, which will make us try to find it multiple times. Starting with the preceding commit we already declare a variable for that program in the top-level build file. Use it so that we only need to search for the program once. Signed-off-by: Patrick Steinhardt --- Documentation/howto/meson.build | 2 +- meson.build | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Documentation/howto/meson.build b/Documentation/howto/meson.build index c023c104161..92a08b13eed 100644 --- a/Documentation/howto/meson.build +++ b/Documentation/howto/meson.build @@ -41,7 +41,7 @@ custom_target( foreach howto : howto_sources howto_stripped = custom_target( command: [ - find_program('sed'), + sed, '-e', '1,/^$/d', '@INPUT@', diff --git a/meson.build b/meson.build index ed857aaa4ab..5bf6a914ead 100644 --- a/meson.build +++ b/meson.build @@ -201,11 +201,12 @@ endif cygpath = find_program('cygpath', dirs: program_path, required: false) diff = find_program('diff', dirs: program_path) git = find_program('git', dirs: program_path, required: false) +sed = find_program('sed', dirs: program_path) shell = find_program('sh', dirs: program_path) tar = find_program('tar', dirs: program_path) # Sanity-check that programs required for the build exist. -foreach tool : ['cat', 'cut', 'grep', 'sed', 'sort', 'tr', 'uname'] +foreach tool : ['cat', 'cut', 'grep', 'sort', 'tr', 'uname'] find_program(tool, dirs: program_path) endforeach -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 12/13] meson: consistently use custom program paths to resolve programs Message-ID: <20250226-b4-pks-meson-improvements-v3-12-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-12-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` The calls to `find_program()` in our documentation don't use our custom program path. This variable gets populated on Windows with the location of Git for Windows so that we can use it to provide our build tools. Consequently, we may not be able to find all necessary binaries on Windows. Adapt the calls to use the program path to fix this. While at it, drop `required: true` arguments, which are the default anyway. Signed-off-by: Patrick Steinhardt --- Documentation/meson.build | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Documentation/meson.build b/Documentation/meson.build index 6438fa67920..c6117366ff9 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -206,9 +206,9 @@ manpages = { docs_backend = get_option('docs_backend') if docs_backend == 'auto' - if find_program('asciidoc', required: false).found() + if find_program('asciidoc', dirs: program_path, required: false).found() docs_backend = 'asciidoc' - elif find_program('asciidoctor', required: false).found() + elif find_program('asciidoctor', dirs: program_path, required: false).found() docs_backend = 'asciidoctor' else error('Neither asciidoc nor asciidoctor were found.') @@ -216,7 +216,7 @@ if docs_backend == 'auto' endif if docs_backend == 'asciidoc' - asciidoc = find_program('asciidoc', required: true) + asciidoc = find_program('asciidoc', dirs: program_path) asciidoc_html = 'xhtml11' asciidoc_docbook = 'docbook' xmlto_extra = [ ] @@ -245,7 +245,7 @@ if docs_backend == 'asciidoc' asciidoc_conf, ] elif docs_backend == 'asciidoctor' - asciidoctor = find_program('asciidoctor', required: true) + asciidoctor = find_program('asciidoctor', dirs: program_path) asciidoc_html = 'xhtml5' asciidoc_docbook = 'docbook5' xmlto_extra = [ @@ -283,7 +283,7 @@ elif docs_backend == 'asciidoctor' ] endif -xmlto = find_program('xmlto') +xmlto = find_program('xmlto', dirs: program_path) cmd_lists = [ 'cmds-ancillaryinterrogators.txt', @@ -404,7 +404,7 @@ if get_option('docs').contains('html') pointing_to: 'git.html', ) - xsltproc = find_program('xsltproc') + xsltproc = find_program('xsltproc', dirs: program_path) user_manual_xml = custom_target( command: asciidoc_common_options + [ -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 11/13] meson: fix overwritten `git` variable Message-ID: <20250226-b4-pks-meson-improvements-v3-11-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-11-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` We're assigning the `git` variable in three places: - In "meson.build" to store the external Git executable. - In "meson.build" to store the compiled Git executable. - In "Documentation/meson.build" to store the external Git executable, a second time. The last case is only needed because we overwrite the original variable with the built version. Rename the variable used for the built Git executable so that we don't have to resolve the external Git executable multiple times. Signed-off-by: Patrick Steinhardt --- Documentation/meson.build | 1 - meson.build | 6 +++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Documentation/meson.build b/Documentation/meson.build index 2a26fa8a5fe..6438fa67920 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -283,7 +283,6 @@ elif docs_backend == 'asciidoctor' ] endif -git = find_program('git', required: false) xmlto = find_program('xmlto') cmd_lists = [ diff --git a/meson.build b/meson.build index 5bf6a914ead..930b85a6dfe 100644 --- a/meson.build +++ b/meson.build @@ -1608,13 +1608,13 @@ libgit_commonmain = declare_dependency( bin_wrappers = [ ] test_dependencies = [ ] -git = executable('git', +git_builtin = executable('git', sources: builtin_sources + 'git.c', dependencies: [libgit_commonmain], install: true, install_dir: get_option('libexecdir') / 'git-core', ) -bin_wrappers += git +bin_wrappers += git_builtin test_dependencies += executable('git-daemon', sources: 'daemon.c', @@ -1705,7 +1705,7 @@ test_dependencies += executable('git-imap-send', foreach alias : [ 'git-receive-pack', 'git-upload-archive', 'git-upload-pack' ] bin_wrappers += executable(alias, - objects: git.extract_all_objects(recursive: false), + objects: git_builtin.extract_all_objects(recursive: false), dependencies: [libgit_commonmain], ) -- 2.48.1.741.g8a9f3a5cdc.dirty ``` ## Patrick Steinhardt, 2025-02-26 08:22 Subject: [PATCH v3 13/13] gitlab-ci: restrict maximum number of link jobs on Windows Message-ID: <20250226-b4-pks-meson-improvements-v3-13-60c77cf673ae@pks.im> URL: https://gitlist.dev/e/20250226-b4-pks-meson-improvements-v3-13-60c77cf673ae%40pks.im In-Reply-To: <20250226-b4-pks-meson-improvements-v3-0-60c77cf673ae@pks.im> ``` The hosted Windows runners on GitLab.com only have 7.5GB of RAM. Given that "link.exe" provided by Microsoft Visual Studio is multi-threaded by itself already and thus quite memory hungry this can quickly lead to memory starvation, out-of-memory situations and thus failed CI jobs. Fix the issue by limiting the number of concurrent linker jobs. The same issue hasn't been observed on GitHub Actions yet, probably because it got more than twice the amount of RAM with 16GB. Signed-off-by: Patrick Steinhardt --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 4976e18a050..7e1cecc6a70 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -169,7 +169,7 @@ build:msvc-meson: extends: .msvc-meson stage: build script: - - meson setup build -Dperl=disabled + - meson setup build -Dperl=disabled -Dbackend_max_links=1 - meson compile -C build artifacts: paths: -- 2.48.1.741.g8a9f3a5cdc.dirty ```