{"thread":{"id":"62927","subject":"Meson build leaks host 'sh' path to target build when cross-compiled","startedAt":"2025-02-09T12:30:29Z","lastAt":"2025-02-18T14:42:15Z","messageCount":9,"participants":["Peter Seiderer","Patrick Steinhardt","Junio C Hamano","Eli Schwartz"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"512161","messageId":"20250209133027.64a865aa@gmx.net","threadId":"62927","inReplyTo":null,"subject":"Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Peter Seiderer","fromEmail":"ps.report@gmx.net","sentAt":"2025-02-09T12:30:27Z","receivedAt":"2025-02-09T12:30:29Z","isPatch":false,"sender":{"key":"ps.report@gmx.net","avatar":null},"body":"What did you do before the bug happened? (Steps to reproduce your issue)\n\n  Cross compile git from source (git-2.48.1) using the meson build system\n  (Buildroot package experimental converted from autoconf to meson).\n\nWhat did you expect to happen? (Expected behavior)\n\n  Execute command 'git fetch' on the target without failure.\n\nWhat happened instead? (Actual behavior)\n\n  Command 'git fetch' on the target failed.\n\nWhat's different between what you expected and what actually happened?\n\n  Debug with strace showed the following difference:\n\n  - o.k (autoconf):\n\n    679   execve(\"/bin/sh\", [\"/bin/sh\", \"-c\", \"git-upload-pack '/home/git-repo\"..., \"git-upload-pack '/home/git-repo\"...], 0x55c342ce4420 /* 17 vars */ <unfinished ...>\n\n  - failure (meson build):\n\n    6861  execve(\"/usr/bin/sh\", [\"/usr/bin/sh\", \"-c\", \"git-upload-pack '/home/git-repo\"..., \"git-upload-pack '/home/git-repo\"...], 0x5639ab382210 /* 17 vars */) = -1 ENOENT (No such file or directory)\n\n  The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n  '/bin/sh' as the autoconf build), 'which sh' on the host returns\n  '/usr/bin/sh'...\n\n  From meson.build\n\n   [...]\n   186 shell = find_program('sh', dirs: program_path)\n   [...]\n   685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n\n  Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n  (at least not for cross-compile), use fix '/bin/sh' instead or make it\n  configurable via a meson option?\n\nRegards,\nPeter\n\n\n[System Info]\ngit version:\ngit version 2.48.1\ncpu: x86_64\nno commit associated with this build\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nlibcurl: 8.11.1\nOpenSSL: OpenSSL 3.2.3 3 Sep 2024\nzlib: 1.3.1\nuname: Linux 6.13.1-1-default #1 SMP PREEMPT_DYNAMIC Mon Feb  3 05:33:25 UTC 2025 (1918d13) x86_64\ncompiler info: gnuc: 14.2\nlibc info: glibc: 2.40\n$SHELL (typically, interactive shell): /bin/bash\n\n\n[Enabled Hooks]\nnot run from a git repository - no hooks to show\n"},{"id":"512168","messageId":"Z6mtnmvKMsIOEVz5@pks.im","threadId":"62927","inReplyTo":"20250209133027.64a865aa@gmx.net","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-10T07:41:18Z","receivedAt":"2025-02-10T07:41:26Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Feb 09, 2025 at 01:30:27PM +0100, Peter Seiderer wrote:\n[snip]\n>   The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n>   '/bin/sh' as the autoconf build), 'which sh' on the host returns\n>   '/usr/bin/sh'...\n> \n>   From meson.build\n> \n>    [...]\n>    186 shell = find_program('sh', dirs: program_path)\n>    [...]\n>    685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n> \n>   Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n>   (at least not for cross-compile), use fix '/bin/sh' instead or make it\n>   configurable via a meson option?\n\nHm, very true. We're mixing up concerns here by treating the build\nenvironment and the target environment the same.\n\nI guess the proper fix is to wire up the \"native:\" parameter when we\ncall `find_program()`, which allows us to tell Meson whether it should\nfind an executable for the build or the target host. And then, for those\nbinaries where we actually need to know about both the build and target\nhost's locations, we'd end up calling `find_program()` twice.\n\nFor executables that are supposed to be used on the target host Meson\nwould then know to first consult the cross file, which could look like\nthis:\n\n    [binaries]\n    sh = '/target/path/to/sh'\n    perl = '/target/path/to/perl'\n\nMeson would then pick up that file via `meson setup --cross-file\n<CROSSFILE_PATH> <BUILDDIR>`.\n\nThe patch should look somewhat like the attached patch, but it conflicts\nwith my in-flight patch series at [1]. I'll wait for that series to be\nmerged to `next` before sending out the fix.\n\nThanks for your report!\n\nPatrick\n\n[1]: <20250129-b4-pks-meson-improvements-v1-0-ab709f0be12c@pks.im>\n\n-- >8 --\n\ndiff --git a/Documentation/meson.build b/Documentation/meson.build\nindex c6117366ff..b033f4a93a 100644\n--- a/Documentation/meson.build\n+++ b/Documentation/meson.build\n@@ -206,9 +206,9 @@ manpages = {\n \n docs_backend = get_option('docs_backend')\n if docs_backend == 'auto'\n-  if find_program('asciidoc', dirs: program_path, required: false).found()\n+  if find_program('asciidoc', dirs: program_path, native: true, required: false).found()\n     docs_backend = 'asciidoc'\n-  elif find_program('asciidoctor', dirs: program_path, required: false).found()\n+  elif find_program('asciidoctor', dirs: program_path, native: true, required: false).found()\n     docs_backend = 'asciidoctor'\n   else\n     error('Neither asciidoc nor asciidoctor were found.')\n@@ -216,7 +216,7 @@ if docs_backend == 'auto'\n endif\n \n if docs_backend == 'asciidoc'\n-  asciidoc = find_program('asciidoc', dirs: program_path)\n+  asciidoc = find_program('asciidoc', native: true, dirs: program_path)\n   asciidoc_html = 'xhtml11'\n   asciidoc_docbook = 'docbook'\n   xmlto_extra = [ ]\n@@ -245,7 +245,7 @@ if docs_backend == 'asciidoc'\n     asciidoc_conf,\n   ]\n elif docs_backend == 'asciidoctor'\n-  asciidoctor = find_program('asciidoctor', dirs: program_path)\n+  asciidoctor = find_program('asciidoctor', native: true, dirs: program_path)\n   asciidoc_html = 'xhtml5'\n   asciidoc_docbook = 'docbook5'\n   xmlto_extra = [\n@@ -283,7 +283,7 @@ elif docs_backend == 'asciidoctor'\n   ]\n endif\n \n-xmlto = find_program('xmlto', dirs: program_path)\n+xmlto = find_program('xmlto', dirs: program_path, native: true)\n \n cmd_lists = [\n   'cmds-ancillaryinterrogators.txt',\n@@ -404,7 +404,7 @@ if get_option('docs').contains('html')\n     pointing_to: 'git.html',\n   )\n \n-  xsltproc = find_program('xsltproc', dirs: program_path)\n+  xsltproc = find_program('xsltproc', dirs: program_path, native: true)\n \n   user_manual_xml = custom_target(\n     command: asciidoc_common_options + [\ndiff --git a/gitweb/meson.build b/gitweb/meson.build\nindex 89b403dc9d..88a54b4dc9 100644\n--- a/gitweb/meson.build\n+++ b/gitweb/meson.build\n@@ -1,5 +1,5 @@\n gitweb_config = configuration_data()\n-gitweb_config.set_quoted('PERL_PATH', perl.full_path())\n+gitweb_config.set_quoted('PERL_PATH', target_perl.full_path())\n gitweb_config.set_quoted('CSSMIN', '')\n gitweb_config.set_quoted('JSMIN', '')\n gitweb_config.set_quoted('GIT_BINDIR', get_option('prefix') / get_option('bindir'))\ndiff --git a/meson.build b/meson.build\nindex e153a43918..5a5662bc02 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -173,7 +173,7 @@ project('git', 'c',\n   # The version is only of cosmetic nature, so if we cannot find a shell yet we\n   # simply don't set up a version at all. This may be the case for example on\n   # Windows systems, where we first have to bootstrap the host environment.\n-  version: find_program('sh', required: false).found() ? run_command(\n+  version: find_program('sh', native: true, required: false).found() ? run_command(\n     'GIT-VERSION-GEN', meson.current_source_dir(), '--format=@GIT_VERSION@',\n     capture: true,\n     check: true,\n@@ -198,16 +198,18 @@ elif host_machine.system() == 'windows'\n   program_path = [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ]\n endif\n \n-cygpath = find_program('cygpath', dirs: program_path, required: false)\n-diff = find_program('diff', dirs: program_path)\n-git = find_program('git', dirs: program_path, required: false)\n-sed = find_program('sed', dirs: program_path)\n-shell = find_program('sh', dirs: program_path)\n-tar = find_program('tar', dirs: program_path)\n+cygpath = find_program('cygpath', dirs: program_path, native: true, required: false)\n+diff = find_program('diff', dirs: program_path, native: true)\n+git = find_program('git', dirs: program_path, native: true, required: false)\n+sed = find_program('sed', dirs: program_path, native: true)\n+shell = find_program('sh', dirs: program_path, native: true)\n+tar = find_program('tar', dirs: program_path, native: true)\n+\n+target_shell = find_program('sh', dirs: program_path, native: false)\n \n # Sanity-check that programs required for the build exist.\n foreach tool : ['cat', 'cut', 'grep', 'sort', 'tr', 'uname']\n-  find_program(tool, dirs: program_path)\n+  find_program(tool, dirs: program_path, native: true)\n endforeach\n \n script_environment = environment()\n@@ -758,6 +760,7 @@ endif\n build_options_config.set_quoted('X', executable_suffix)\n \n python = import('python').find_installation('python3', required: get_option('python'))\n+target_python = find_program('python3', native: false, required: python.found())\n if python.found()\n   build_options_config.set('NO_PYTHON', '')\n else\n@@ -775,7 +778,8 @@ endif\n \n # Note that we only set NO_PERL if the Perl features were disabled by the user.\n # It may not be set when we have found Perl, but only use it to run tests.\n-perl = find_program('perl', version: '>=5.8.1', dirs: program_path, required: perl_required)\n+perl = find_program('perl', version: '>=5.8.1', dirs: program_path, native: true, required: perl_required)\n+target_perl = find_program('perl', version: '>=5.8.1', native: false, required: perl.found())\n perl_features_enabled = perl.found() and get_option('perl').allowed()\n if perl_features_enabled\n   build_options_config.set('NO_PERL', '')\n@@ -825,7 +829,7 @@ else\n   build_options_config.set('NO_PTHREADS', '1')\n endif\n \n-msgfmt = find_program('msgfmt', dirs: program_path, required: false)\n+msgfmt = find_program('msgfmt', dirs: program_path, native: true, required: false)\n gettext_option = get_option('gettext').disable_auto_if(not msgfmt.found())\n if not msgfmt.found() and gettext_option.enabled()\n   error('Internationalization via libintl requires msgfmt')\n@@ -1954,9 +1958,9 @@ foreach key, value : {\n   'GIT_TEST_TEMPLATE_DIR': meson.project_build_root() / 'templates',\n   'GIT_TEST_TEXTDOMAINDIR': meson.project_build_root() / 'po',\n   'PAGER_ENV': get_option('pager_environment'),\n-  'PERL_PATH': perl.found() ? perl.full_path() : '',\n-  'PYTHON_PATH': python.found () ? python.full_path() : '',\n-  'SHELL_PATH': shell.full_path(),\n+  'PERL_PATH': target_perl.found() ? target_perl.full_path() : '',\n+  'PYTHON_PATH': target_python.found () ? target_python.full_path() : '',\n+  'SHELL_PATH': target_shell.full_path(),\n   'TAR': tar.full_path(),\n   'TEST_OUTPUT_DIRECTORY': test_output_directory,\n   'TEST_SHELL_PATH': shell.full_path(),\ndiff --git a/templates/meson.build b/templates/meson.build\nindex 1faf9a44ce..986c2e03be 100644\n--- a/templates/meson.build\n+++ b/templates/meson.build\n@@ -1,6 +1,6 @@\n template_config = configuration_data()\n-template_config.set('PERL_PATH', perl.found() ? fs.as_posix(perl.full_path()) : '')\n-template_config.set('SHELL_PATH', fs.as_posix(shell.full_path()))\n+template_config.set('PERL_PATH', perl.found() ? fs.as_posix(target_perl.full_path()) : '')\n+template_config.set('SHELL_PATH', fs.as_posix(target_shell.full_path()))\n template_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb'))\n \n configure_file(\n"},{"id":"512173","messageId":"20250210122603.5130e309@gmx.net","threadId":"62927","inReplyTo":"Z6mtnmvKMsIOEVz5@pks.im","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Peter Seiderer","fromEmail":"ps.report@gmx.net","sentAt":"2025-02-10T11:26:03Z","receivedAt":"2025-02-10T11:26:06Z","isPatch":false,"sender":{"key":"ps.report@gmx.net","avatar":null},"body":"Hello Patrick,\n\nOn Mon, 10 Feb 2025 08:41:18 +0100, Patrick Steinhardt <ps@pks.im> wrote:\n\n> On Sun, Feb 09, 2025 at 01:30:27PM +0100, Peter Seiderer wrote:\n> [snip]\n> >   The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n> >   '/bin/sh' as the autoconf build), 'which sh' on the host returns\n> >   '/usr/bin/sh'...\n> >\n> >   From meson.build\n> >\n> >    [...]\n> >    186 shell = find_program('sh', dirs: program_path)\n> >    [...]\n> >    685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n> >\n> >   Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n> >   (at least not for cross-compile), use fix '/bin/sh' instead or make it\n> >   configurable via a meson option?\n>\n> Hm, very true. We're mixing up concerns here by treating the build\n> environment and the target environment the same.\n>\n> I guess the proper fix is to wire up the \"native:\" parameter when we\n> call `find_program()`, which allows us to tell Meson whether it should\n> find an executable for the build or the target host. And then, for those\n> binaries where we actually need to know about both the build and target\n> host's locations, we'd end up calling `find_program()` twice.\n>\n> For executables that are supposed to be used on the target host Meson\n> would then know to first consult the cross file, which could look like\n> this:\n>\n>     [binaries]\n>     sh = '/target/path/to/sh'\n>     perl = '/target/path/to/perl'\n>\n> Meson would then pick up that file via `meson setup --cross-file\n> <CROSSFILE_PATH> <BUILDDIR>`.\n\nSorry, I believe this will not work..., the description of the native\nparameter in find_program ([2]) on the first sight sounds like doing the\nright thing, but as far as I read the 'Cross compilation' page ([3], [4]) the\ntools under the '[binaries]' section are the tools used while cross-compiling\n(running on the build machine) and not the paths/tools on the target\n(or as meson nomenclature host/target)...\n\nOne tiny finding below...\n\n>\n> The patch should look somewhat like the attached patch, but it conflicts\n> with my in-flight patch series at [1]. I'll wait for that series to be\n> merged to `next` before sending out the fix.\n>\n> Thanks for your report!\n>\n> Patrick\n>\n> [1]: <20250129-b4-pks-meson-improvements-v1-0-ab709f0be12c@pks.im>\n>\n> -- >8 --\n>\n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index c6117366ff..b033f4a93a 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -206,9 +206,9 @@ manpages = {\n>\n>  docs_backend = get_option('docs_backend')\n>  if docs_backend == 'auto'\n> -  if find_program('asciidoc', dirs: program_path, required: false).found()\n> +  if find_program('asciidoc', dirs: program_path, native: true, required: false).found()\n>      docs_backend = 'asciidoc'\n> -  elif find_program('asciidoctor', dirs: program_path, required: false).found()\n> +  elif find_program('asciidoctor', dirs: program_path, native: true, required: false).found()\n>      docs_backend = 'asciidoctor'\n>    else\n>      error('Neither asciidoc nor asciidoctor were found.')\n> @@ -216,7 +216,7 @@ if docs_backend == 'auto'\n>  endif\n>\n>  if docs_backend == 'asciidoc'\n> -  asciidoc = find_program('asciidoc', dirs: program_path)\n> +  asciidoc = find_program('asciidoc', native: true, dirs: program_path)\n>    asciidoc_html = 'xhtml11'\n>    asciidoc_docbook = 'docbook'\n>    xmlto_extra = [ ]\n> @@ -245,7 +245,7 @@ if docs_backend == 'asciidoc'\n>      asciidoc_conf,\n>    ]\n>  elif docs_backend == 'asciidoctor'\n> -  asciidoctor = find_program('asciidoctor', dirs: program_path)\n> +  asciidoctor = find_program('asciidoctor', native: true, dirs: program_path)\n>    asciidoc_html = 'xhtml5'\n>    asciidoc_docbook = 'docbook5'\n>    xmlto_extra = [\n> @@ -283,7 +283,7 @@ elif docs_backend == 'asciidoctor'\n>    ]\n>  endif\n>\n> -xmlto = find_program('xmlto', dirs: program_path)\n> +xmlto = find_program('xmlto', dirs: program_path, native: true)\n>\n>  cmd_lists = [\n>    'cmds-ancillaryinterrogators.txt',\n> @@ -404,7 +404,7 @@ if get_option('docs').contains('html')\n>      pointing_to: 'git.html',\n>    )\n>\n> -  xsltproc = find_program('xsltproc', dirs: program_path)\n> +  xsltproc = find_program('xsltproc', dirs: program_path, native: true)\n>\n>    user_manual_xml = custom_target(\n>      command: asciidoc_common_options + [\n> diff --git a/gitweb/meson.build b/gitweb/meson.build\n> index 89b403dc9d..88a54b4dc9 100644\n> --- a/gitweb/meson.build\n> +++ b/gitweb/meson.build\n> @@ -1,5 +1,5 @@\n>  gitweb_config = configuration_data()\n> -gitweb_config.set_quoted('PERL_PATH', perl.full_path())\n> +gitweb_config.set_quoted('PERL_PATH', target_perl.full_path())\n>  gitweb_config.set_quoted('CSSMIN', '')\n>  gitweb_config.set_quoted('JSMIN', '')\n>  gitweb_config.set_quoted('GIT_BINDIR', get_option('prefix') / get_option('bindir'))\n> diff --git a/meson.build b/meson.build\n> index e153a43918..5a5662bc02 100644\n> --- a/meson.build\n> +++ b/meson.build\n> @@ -173,7 +173,7 @@ project('git', 'c',\n>    # The version is only of cosmetic nature, so if we cannot find a shell yet we\n>    # simply don't set up a version at all. This may be the case for example on\n>    # Windows systems, where we first have to bootstrap the host environment.\n> -  version: find_program('sh', required: false).found() ? run_command(\n> +  version: find_program('sh', native: true, required: false).found() ? run_command(\n>      'GIT-VERSION-GEN', meson.current_source_dir(), '--format=@GIT_VERSION@',\n>      capture: true,\n>      check: true,\n> @@ -198,16 +198,18 @@ elif host_machine.system() == 'windows'\n>    program_path = [ 'C:/Program Files/Git/bin', 'C:/Program Files/Git/usr/bin' ]\n>  endif\n>\n> -cygpath = find_program('cygpath', dirs: program_path, required: false)\n> -diff = find_program('diff', dirs: program_path)\n> -git = find_program('git', dirs: program_path, required: false)\n> -sed = find_program('sed', dirs: program_path)\n> -shell = find_program('sh', dirs: program_path)\n> -tar = find_program('tar', dirs: program_path)\n> +cygpath = find_program('cygpath', dirs: program_path, native: true, required: false)\n> +diff = find_program('diff', dirs: program_path, native: true)\n> +git = find_program('git', dirs: program_path, native: true, required: false)\n> +sed = find_program('sed', dirs: program_path, native: true)\n> +shell = find_program('sh', dirs: program_path, native: true)\n> +tar = find_program('tar', dirs: program_path, native: true)\n> +\n> +target_shell = find_program('sh', dirs: program_path, native: false)\n>\n>  # Sanity-check that programs required for the build exist.\n>  foreach tool : ['cat', 'cut', 'grep', 'sort', 'tr', 'uname']\n> -  find_program(tool, dirs: program_path)\n> +  find_program(tool, dirs: program_path, native: true)\n>  endforeach\n>\n>  script_environment = environment()\n> @@ -758,6 +760,7 @@ endif\n>  build_options_config.set_quoted('X', executable_suffix)\n>\n>  python = import('python').find_installation('python3', required: get_option('python'))\n> +target_python = find_program('python3', native: false, required: python.found())\n>  if python.found()\n>    build_options_config.set('NO_PYTHON', '')\n>  else\n> @@ -775,7 +778,8 @@ endif\n>\n>  # Note that we only set NO_PERL if the Perl features were disabled by the user.\n>  # It may not be set when we have found Perl, but only use it to run tests.\n> -perl = find_program('perl', version: '>=5.8.1', dirs: program_path, required: perl_required)\n> +perl = find_program('perl', version: '>=5.8.1', dirs: program_path, native: true, required: perl_required)\n> +target_perl = find_program('perl', version: '>=5.8.1', native: false, required: perl.found())\n>  perl_features_enabled = perl.found() and get_option('perl').allowed()\n>  if perl_features_enabled\n>    build_options_config.set('NO_PERL', '')\n> @@ -825,7 +829,7 @@ else\n>    build_options_config.set('NO_PTHREADS', '1')\n>  endif\n>\n> -msgfmt = find_program('msgfmt', dirs: program_path, required: false)\n> +msgfmt = find_program('msgfmt', dirs: program_path, native: true, required: false)\n>  gettext_option = get_option('gettext').disable_auto_if(not msgfmt.found())\n>  if not msgfmt.found() and gettext_option.enabled()\n>    error('Internationalization via libintl requires msgfmt')\n> @@ -1954,9 +1958,9 @@ foreach key, value : {\n>    'GIT_TEST_TEMPLATE_DIR': meson.project_build_root() / 'templates',\n>    'GIT_TEST_TEXTDOMAINDIR': meson.project_build_root() / 'po',\n>    'PAGER_ENV': get_option('pager_environment'),\n> -  'PERL_PATH': perl.found() ? perl.full_path() : '',\n> -  'PYTHON_PATH': python.found () ? python.full_path() : '',\n> -  'SHELL_PATH': shell.full_path(),\n> +  'PERL_PATH': target_perl.found() ? target_perl.full_path() : '',\n> +  'PYTHON_PATH': target_python.found () ? target_python.full_path() : '',\n> +  'SHELL_PATH': target_shell.full_path(),\n>    'TAR': tar.full_path(),\n>    'TEST_OUTPUT_DIRECTORY': test_output_directory,\n>    'TEST_SHELL_PATH': shell.full_path(),\n> diff --git a/templates/meson.build b/templates/meson.build\n> index 1faf9a44ce..986c2e03be 100644\n> --- a/templates/meson.build\n> +++ b/templates/meson.build\n> @@ -1,6 +1,6 @@\n>  template_config = configuration_data()\n> -template_config.set('PERL_PATH', perl.found() ? fs.as_posix(perl.full_path()) : '')\n> -template_config.set('SHELL_PATH', fs.as_posix(shell.full_path()))\n> +template_config.set('PERL_PATH', perl.found() ? fs.as_posix(target_perl.full_path()) : '')\n\nAbove should read (perl.found() vs. target_perl.found()):\n\n   +template_config.set('PERL_PATH', target_perl.found() ? fs.as_posix(target_perl.full_path()) : '')\n\n> +template_config.set('SHELL_PATH', fs.as_posix(target_shell.full_path()))\n>  template_config.set('GITWEBDIR', fs.as_posix(get_option('prefix') / get_option('datadir') / 'gitweb'))\n>\n>  configure_file(\n\nRegards,\nPeter\n\n[2] https://mesonbuild.com/Reference-manual_functions.html#find_program\n[3] https://mesonbuild.com/Cross-compilation.html\n[4] https://mesonbuild.com/Machine-files.html\n\n"},{"id":"512180","messageId":"xmqqlduddb8b.fsf@gitster.g","threadId":"62927","inReplyTo":"Z6mtnmvKMsIOEVz5@pks.im","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-10T16:18:44Z","receivedAt":"2025-02-10T16:18:46Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> On Sun, Feb 09, 2025 at 01:30:27PM +0100, Peter Seiderer wrote:\n> [snip]\n>>   The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n>>   '/bin/sh' as the autoconf build), 'which sh' on the host returns\n>>   '/usr/bin/sh'...\n>> \n>>   From meson.build\n>> \n>>    [...]\n>>    186 shell = find_program('sh', dirs: program_path)\n>>    [...]\n>>    685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n>> \n>>   Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n>>   (at least not for cross-compile), use fix '/bin/sh' instead or make it\n>>   configurable via a meson option?\n>\n> Hm, very true. We're mixing up concerns here by treating the build\n> environment and the target environment the same.\n> ...\n> The patch should look somewhat like the attached patch, but it conflicts\n> with my in-flight patch series at [1]. I'll wait for that series to be\n> merged to `next` before sending out the fix.\n\nInteresting.  When we did our make-based build, we never seriously\nconsidered cross building into a platform where the path to the\nbasic tools differed between the host and target hosts.  At least in\nour build procedure in olden times, I think we used to assume that\nwhat we just built can be run inside the build procedure on the host\nplatform even outside the tests, which would make cross building\nimpossible.\n\nNow, since we are \"fixing\" this aspect of the build for meson-based\nbuild, should we also make the same fix for make-based build as well?\n\nI'd have to say that I prefer to see it done out of pure principle\n(i.e. we earlier declared that meson is not yet replacing make, so\nadding new shinies only to meson-world is like making the make-world\nbitrot as if we do not care).\n\nBut on the other hand, nobody complained that they cannot cross\nbuild with make-based build seriously enough to cause us consider\ndoing something about it for the past 20 years, so the pragmast in\nme tells me that it is not worth it doing it in make-based build.\n\n> Thanks for your report!\n\nYup, thanks.\n"},{"id":"512182","messageId":"20250210174335.6d6d2af2@gmx.net","threadId":"62927","inReplyTo":"xmqqlduddb8b.fsf@gitster.g","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Peter Seiderer","fromEmail":"ps.report@gmx.net","sentAt":"2025-02-10T16:43:35Z","receivedAt":"2025-02-10T16:43:43Z","isPatch":false,"sender":{"key":"ps.report@gmx.net","avatar":null},"body":"Hello Junio,\n\nOn Mon, 10 Feb 2025 08:18:44 -0800, Junio C Hamano <gitster@pobox.com> wrote:\n\n> Patrick Steinhardt <ps@pks.im> writes:\n>\n> > On Sun, Feb 09, 2025 at 01:30:27PM +0100, Peter Seiderer wrote:\n> > [snip]\n> >>   The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n> >>   '/bin/sh' as the autoconf build), 'which sh' on the host returns\n> >>   '/usr/bin/sh'...\n> >>\n> >>   From meson.build\n> >>\n> >>    [...]\n> >>    186 shell = find_program('sh', dirs: program_path)\n> >>    [...]\n> >>    685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n> >>\n> >>   Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n> >>   (at least not for cross-compile), use fix '/bin/sh' instead or make it\n> >>   configurable via a meson option?\n> >\n> > Hm, very true. We're mixing up concerns here by treating the build\n> > environment and the target environment the same.\n> > ...\n> > The patch should look somewhat like the attached patch, but it conflicts\n> > with my in-flight patch series at [1]. I'll wait for that series to be\n> > merged to `next` before sending out the fix.\n>\n> Interesting.  When we did our make-based build, we never seriously\n> considered cross building into a platform where the path to the\n> basic tools differed between the host and target hosts.  At least in\n> our build procedure in olden times, I think we used to assume that\n> what we just built can be run inside the build procedure on the host\n> platform even outside the tests, which would make cross building\n> impossible.\n>\n> Now, since we are \"fixing\" this aspect of the build for meson-based\n> build, should we also make the same fix for make-based build as well?\n>\n> I'd have to say that I prefer to see it done out of pure principle\n> (i.e. we earlier declared that meson is not yet replacing make, so\n> adding new shinies only to meson-world is like making the make-world\n> bitrot as if we do not care).\n>\n> But on the other hand, nobody complained that they cannot cross\n> build with make-based build seriously enough to cause us consider\n> doing something about it for the past 20 years, so the pragmast in\n> me tells me that it is not worth it doing it in make-based build.\n\nMaybe all doing (autoconf) cross builds where happy with the defaults\nfrom the Makefile (SHELL_PATH = /bin/sh, PERL_PATH = /usr/bin/perl)\non host and target (as the buildroot autoconf package since 2013) and\nonly users doing native builds fiddled around with non-default values?\n\nRegards,\nPeter\n\n>\n> > Thanks for your report!\n>\n> Yup, thanks.\n\n"},{"id":"512196","messageId":"xmqqpljpa83y.fsf@gitster.g","threadId":"62927","inReplyTo":"20250210174335.6d6d2af2@gmx.net","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-02-10T19:54:25Z","receivedAt":"2025-02-10T19:54:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Peter Seiderer <ps.report@gmx.net> writes:\n\n> Maybe all doing (autoconf) cross builds where happy with the defaults\n> from the Makefile (SHELL_PATH = /bin/sh, PERL_PATH = /usr/bin/perl)\n> on host and target (as the buildroot autoconf package since 2013) and\n> only users doing native builds fiddled around with non-default values?\n\nI somehow doubt it.\n\nThe problem I see is that there is no distinction between \"this is\nthe path for the shell on the target system\" vs \"this is the shell\nwe run on the host while building the package\" in the Makefile.  Use\nof autoconf would not magically change it; the Makefile that\nincludes the config.mak.autogen needs to be aware of the\ndistinction.\n"},{"id":"512225","messageId":"20250211095544.66226abc@gmx.net","threadId":"62927","inReplyTo":"xmqqpljpa83y.fsf@gitster.g","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Peter Seiderer","fromEmail":"ps.report@gmx.net","sentAt":"2025-02-11T08:55:44Z","receivedAt":"2025-02-11T08:55:49Z","isPatch":false,"sender":{"key":"ps.report@gmx.net","avatar":null},"body":"Hello Junio,\n\nOn Mon, 10 Feb 2025 11:54:25 -0800, Junio C Hamano <gitster@pobox.com> wrote:\n\n> Peter Seiderer <ps.report@gmx.net> writes:\n>\n> > Maybe all doing (autoconf) cross builds where happy with the defaults\n> > from the Makefile (SHELL_PATH = /bin/sh, PERL_PATH = /usr/bin/perl)\n> > on host and target (as the buildroot autoconf package since 2013) and\n> > only users doing native builds fiddled around with non-default values?\n>\n> I somehow doubt it.\n>\n> The problem I see is that there is no distinction between \"this is\n> the path for the shell on the target system\" vs \"this is the shell\n> we run on the host while building the package\" in the Makefile.  Use\n> of autoconf would not magically change it; the Makefile that\n> includes the config.mak.autogen needs to be aware of the\n> distinction.\n\nFound no problem/special-handling in buildroot ([1]) or yocto ([2]) for\nSHELL_PATH, but you are right regarding the PERL_PATH in yocto using\n\n  38 EXTRA_OECONF = \"--with-perl=${STAGING_BINDIR_NATIVE}/perl-native/perl \\\n\nand\n\n  49 EXTRA_OEMAKE += \"'PERL_PATH=/usr/bin/env perl'\"\n\nand a fixup for the resulting files\n\n  76 perl_native_fixup () {\n  77 \tsed -i -e 's#${STAGING_BINDIR_NATIVE}/perl-native/#${bindir}/#' \\\n  78 \t       -e 's#${libdir}/perl-native/#${libdir}/#' \\\n  79 \t    ${@d.getVar(\"PERLTOOLS\").replace(' /',d.getVar('D') + '/')}\n\nYes, doing a distinct handling for host/target path for sh/perl for\nmeson/autoconf is for sure the right thing ;-)\n\nRegards,\nPeter\n\n[1] https://gitlab.com/buildroot.org/buildroot/-/blob/master/package/git/git.mk\n[2] https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/git/git_2.48.1.bb\n"},{"id":"512600","messageId":"Z7RzPkiBfbYPhKp_@pks.im","threadId":"62927","inReplyTo":"20250210122603.5130e309@gmx.net","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-02-18T11:47:10Z","receivedAt":"2025-02-18T11:47:19Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Feb 10, 2025 at 12:26:03PM +0100, Peter Seiderer wrote:\n> On Mon, 10 Feb 2025 08:41:18 +0100, Patrick Steinhardt <ps@pks.im> wrote:\n> > On Sun, Feb 09, 2025 at 01:30:27PM +0100, Peter Seiderer wrote:\n> > [snip]\n> > >   The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n> > >   '/bin/sh' as the autoconf build), 'which sh' on the host returns\n> > >   '/usr/bin/sh'...\n> > >\n> > >   From meson.build\n> > >\n> > >    [...]\n> > >    186 shell = find_program('sh', dirs: program_path)\n> > >    [...]\n> > >    685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n> > >\n> > >   Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n> > >   (at least not for cross-compile), use fix '/bin/sh' instead or make it\n> > >   configurable via a meson option?\n> >\n> > Hm, very true. We're mixing up concerns here by treating the build\n> > environment and the target environment the same.\n> >\n> > I guess the proper fix is to wire up the \"native:\" parameter when we\n> > call `find_program()`, which allows us to tell Meson whether it should\n> > find an executable for the build or the target host. And then, for those\n> > binaries where we actually need to know about both the build and target\n> > host's locations, we'd end up calling `find_program()` twice.\n> >\n> > For executables that are supposed to be used on the target host Meson\n> > would then know to first consult the cross file, which could look like\n> > this:\n> >\n> >     [binaries]\n> >     sh = '/target/path/to/sh'\n> >     perl = '/target/path/to/perl'\n> >\n> > Meson would then pick up that file via `meson setup --cross-file\n> > <CROSSFILE_PATH> <BUILDDIR>`.\n> \n> Sorry, I believe this will not work..., the description of the native\n> parameter in find_program ([2]) on the first sight sounds like doing the\n> right thing, but as far as I read the 'Cross compilation' page ([3], [4]) the\n> tools under the '[binaries]' section are the tools used while cross-compiling\n> (running on the build machine) and not the paths/tools on the target\n> (or as meson nomenclature host/target)...\n\nQuoting the documentation of `find_program()`'s `native` parameter [1]:\n\n    Defines how this executable should be searched. By default it is set to\n    false, which causes Meson to first look for the executable in the cross\n    file (when cross building) and if it is not defined there, then from the\n    system. If set to true, the cross file is ignored and the program is\n    only searched from the system.\n\nSo I think this should work as expected when passing the file via\n`--cross-file`, shouldn't it? If we are cross-compiling we'd find the\ntarget binaries via the cross file when `native: false`, which is\nexactly what we want.\n\nBut I see what you're saying. The _intent_ is to specify the environment\nof the cross-compiling environment, and not to describe the target\nenvironment.\n\nI can think of two alternatives:\n\n  - We can introduce build options for this. If unset, we continue to\n    use the result of `find_program()`. Otherwise, we use the value\n    specified by the user.\n\n  - We can introduce properties into the cross file that allow the user\n    to specify those parameters. We can then retrieve them by calling\n    `meson.get_external_property()`, but only when cross-compiling.\n\nLet me also Cc Eli, he might have an opinion on how to do this.\n\n> > diff --git a/templates/meson.build b/templates/meson.build\n> > index 1faf9a44ce..986c2e03be 100644\n> > --- a/templates/meson.build\n> > +++ b/templates/meson.build\n> > @@ -1,6 +1,6 @@\n> >  template_config = configuration_data()\n> > -template_config.set('PERL_PATH', perl.found() ? fs.as_posix(perl.full_path()) : '')\n> > -template_config.set('SHELL_PATH', fs.as_posix(shell.full_path()))\n> > +template_config.set('PERL_PATH', perl.found() ? fs.as_posix(target_perl.full_path()) : '')\n> \n> Above should read (perl.found() vs. target_perl.found()):\n> \n>    +template_config.set('PERL_PATH', target_perl.found() ? fs.as_posix(target_perl.full_path()) : '')\n\nAh, yes, indeed.\n\nPatrick\n\n[1]: https://mesonbuild.com/Reference-manual_functions.html#find_program\n"},{"id":"512601","messageId":"24df8aa2-760f-4da3-88b0-ab97796373fd@gentoo.org","threadId":"62927","inReplyTo":"Z7RzPkiBfbYPhKp_@pks.im","subject":"Re: Meson build leaks host 'sh' path to target build when cross-compiled","fromName":"Eli Schwartz","fromEmail":"eschwartz@gentoo.org","sentAt":"2025-02-18T14:41:23Z","receivedAt":"2025-02-18T14:42:15Z","isPatch":false,"sender":{"key":"eschwartz@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/6551424?v=4"},"body":"On 2/18/25 6:47 AM, Patrick Steinhardt wrote:\n> On Mon, Feb 10, 2025 at 12:26:03PM +0100, Peter Seiderer wrote:\n>> On Mon, 10 Feb 2025 08:41:18 +0100, Patrick Steinhardt <ps@pks.im> wrote:\n>>> On Sun, Feb 09, 2025 at 01:30:27PM +0100, Peter Seiderer wrote:\n>>> [snip]\n>>>>   The meson build tries to execute the non-existent '/usr/bin/sh' (instead of\n>>>>   '/bin/sh' as the autoconf build), 'which sh' on the host returns\n>>>>   '/usr/bin/sh'...\n>>>>\n>>>>   From meson.build\n>>>>\n>>>>    [...]\n>>>>    186 shell = find_program('sh', dirs: program_path)\n>>>>    [...]\n>>>>    685   '-DSHELL_PATH=\"' + fs.as_posix(shell.full_path()) + '\"',\n>>>>\n>>>>   Do not use the result of 'find_program('sh',...)' for '-DSHELL_PATH='\n>>>>   (at least not for cross-compile), use fix '/bin/sh' instead or make it\n>>>>   configurable via a meson option?\n>>>\n>>> Hm, very true. We're mixing up concerns here by treating the build\n>>> environment and the target environment the same.\n>>>\n>>> I guess the proper fix is to wire up the \"native:\" parameter when we\n>>> call `find_program()`, which allows us to tell Meson whether it should\n>>> find an executable for the build or the target host. And then, for those\n>>> binaries where we actually need to know about both the build and target\n>>> host's locations, we'd end up calling `find_program()` twice.\n>>>\n>>> For executables that are supposed to be used on the target host Meson\n>>> would then know to first consult the cross file, which could look like\n>>> this:\n>>>\n>>>     [binaries]\n>>>     sh = '/target/path/to/sh'\n>>>     perl = '/target/path/to/perl'\n>>>\n>>> Meson would then pick up that file via `meson setup --cross-file\n>>> <CROSSFILE_PATH> <BUILDDIR>`.\n>>\n>> Sorry, I believe this will not work..., the description of the native\n>> parameter in find_program ([2]) on the first sight sounds like doing the\n>> right thing, but as far as I read the 'Cross compilation' page ([3], [4]) the\n>> tools under the '[binaries]' section are the tools used while cross-compiling\n>> (running on the build machine) and not the paths/tools on the target\n>> (or as meson nomenclature host/target)...\n> \n> Quoting the documentation of `find_program()`'s `native` parameter [1]:\n> \n>     Defines how this executable should be searched. By default it is set to\n>     false, which causes Meson to first look for the executable in the cross\n>     file (when cross building) and if it is not defined there, then from the\n>     system. If set to true, the cross file is ignored and the program is\n>     only searched from the system.\n> \n> So I think this should work as expected when passing the file via\n> `--cross-file`, shouldn't it? If we are cross-compiling we'd find the\n> target binaries via the cross file when `native: false`, which is\n> exactly what we want.\n> \n> But I see what you're saying. The _intent_ is to specify the environment\n> of the cross-compiling environment, and not to describe the target\n> environment.\n> \n> I can think of two alternatives:\n> \n>   - We can introduce build options for this. If unset, we continue to\n>     use the result of `find_program()`. Otherwise, we use the value\n>     specified by the user.\n> \n>   - We can introduce properties into the cross file that allow the user\n>     to specify those parameters. We can then retrieve them by calling\n>     `meson.get_external_property()`, but only when cross-compiling.\n> \n> Let me also Cc Eli, he might have an opinion on how to do this.\n\n\nFor the specific case of detecting sh, the portable \"API filename\" is\nexactly /bin/sh and nothing else. It should always exist on pretty much\nany unix system ever... except for the ones where it exists but is a\npre-POSIX shell and the actual POSIX one isn't always on PATH at all. Hi\nthere, Solaris /usr/xpg4/bin/sh ! :)\n\nOverriding it via the cross file would be fine -- if your goal is to\nonly ever find_program(..., native: false) in order to detect a path and\nembed it, then it doesn't matter whether cross files are for running\ncross tools on the build machine or for looking up cross tools to detect\na path and embed it, since the two goals would never *come into\nconflict*. And that's what actually matters -- if you are concerned that\ncross files will be wrong as they specify the cross-compile environment\nnot the install environment, then you shouldn't be using find_program()\neither, you should be exclusively using build options.\n\n\n\nBut again -- that's the general case, and for the specific case you\nshould be defaulting to /bin/sh\n\nThis does have a highly practical application to it. Gentoo supports\nsplit-usr systems, and binary package support needs to consistently use\npaths that are present on both merged- and split-usr systems. There is\nno /usr/bin/sh on a split-usr system, but on a merged-usr system they\nare the same directory, so it will exist \"anyway\".\n\nNote that this is NOT a cross compile environment. I'm compiling for the\ncurrent x86-64 environment, but also tarring it up for installation on\nother x86-64 environments that happen to use a different filesystem\npartition scheme.\n\n\nCc Sam as we will need to fix this in our Gentoo packaging one way or\nanother:\n\n$ ebuild git-2.48.1.ebuild clean install\n[...]\n$ ag usr/bin /var/tmp/portage/dev-vcs/git-2.48.1/image\n\nIt is finding and embedding various hits for /usr/bin/sh because I\nhappened to build on a merged-usr profile.\n\n\n\n-- \nEli Schwartz\n"}]}