[PATCH v2 00/13] miscellaneous build mods (part 1)

changes in v2: --------------

- patch #3 has been split into two (new patches #3,4). The motivation
  for the original patch seems not to have been too obvious. The new
  patch #3 combines the suppressing the -Defines for standard default
  values for editor and pager, with the setting of the missing docs
  attributes. The new patch #4 combines the suppressing of the -Define
  for the default-help-format, with the overriding of the default value
  on the windows platform.

- patch #6 (new #7) updates the commit message along with skipping the
  'failing' test #12 in t7815-grep-binary.sh on cygwin. (With the cygwin
  platform regex library the '.' pattern matches a NUL character).

- patch #11 (new #12) fixes a simple typo in the patch subject.

- patch #12 (new #13) updates the commit message and makes a choice for
  the default CSPRNG_METHOD on Linux; 'getrandom' as advised by Brian
  here [0].

[0] https://lore.kernel.org/git/Z9YbJFJjtXNYnTzk@tapette.crustytoothpaste.net/

As before, this series is based on commit 683c54c999 ("Git 2.49", 2025-03-14).

Also, I did a test merge to 'master'@5b97a56fa0, 'next'@ff1f9ca942 and 'seen'@1b09671a54 and they all 'auto-merged' without issue.

As an experiment, I am using 'git sendemail' to send this series for the first time (I have sent single patches before, but not a series), so if anything goes awry I apologize in advance! :)

A range-diff is given below.

The original v1 cover-letter follows: -------------------------------------

This series contains some miscellaneous changes to the make, meson and autoconf build systems. These are only the patches I have managed to write a commit message for; I have some others with no commit message, some need more testing and others are yet to be written. (So, there will be a 'part #2' series later... :) ).

(I have notes from a couple of months ago - as an example, I have a note that the meson build incorrectly sets ETC_GITATTRIBUTES and ETC_GITCONFIG to e.g. '/etc/gitattributes' rather than 'etc/gitattributes').

This series is based on the master branch at commit 683c54c999 ("Git 2.49", 2025-03-14).

As a simple means for comparing the build products, we can compare the symbols exported by git. So, starting at current master:

  $ git checkout master
  Switched to branch 'master'
  Your branch is up-to-date with 'origin/master'.
  $ git describe
  v2.49.0
  $ 

  $ make >out 2>&1
  $ 

  $ meson setup build/
  The Meson build system
  ...
  Found ninja-1.11.1 at /usr/bin/ninja
  $ 

  $ cd build
  $ meson compile
  INFO: autodetecting backend as ninja
  INFO: calculating backend command to run: /usr/bin/ninja
  [661/661] Copying file contrib/completion/git-completion.zsh
  $ cd ..
  $ 

  $ rm *.syms
  $ ./syms.pl git >git.syms 2>/dev/null
  $ ./syms.pl build/git >mgit.syms 2>/dev/null
  $ 

  $ diff git.syms mgit.syms
  88a89
  > U -- arc4random_buf@GLIBC_2.36
  98a100,101
  > U -- atoi@GLIBC_2.2.5
  > U -- atol@GLIBC_2.2.5
  195a199
  > U -- bsearch@GLIBC_2.2.5
  586,587d589
  < U -- dcgettext@GLIBC_2.2.5
  < U -- dcngettext@GLIBC_2.2.5
  746,747c748,749
  < B -- __environ@GLIBC_2.2.5
  < V -- environ@GLIBC_2.2.5
  ---
  > U -- __environ@GLIBC_2.2.5
  > U -- environ@GLIBC_2.2.5
  909c911
  < U -- __fprintf_chk@GLIBC_2.3.4
  ---
  > U -- fprintf@GLIBC_2.2.5
  914d915
  < U -- __fread_chk@GLIBC_2.7
  1007c1008
  < U -- getc@GLIBC_2.2.5
  ---
  > U -- getchar@GLIBC_2.2.5
  1018a1020
  > U -- getc_unlocked@GLIBC_2.2.5
  1107a1110
  > U -- gettext@GLIBC_2.2.5
  1253d1255
  < T -- gitstrlcpy
  1355a1358
  > U -- htons@GLIBC_2.2.5
  1666d1668
  < U -- __memcpy_chk@GLIBC_2.3.4
  1673d1674
  < U -- __memmove_chk@GLIBC_2.3.4
  1684d1684
  < U -- __memset_chk@GLIBC_2.3.4
  1737a1738
  > U -- ngettext@GLIBC_2.2.5
  1759a1761
  > U -- ntohs@GLIBC_2.2.5
  1819d1820
  < U -- __open64_2@GLIBC_2.7
  1988a1990,2007
  > U -- pcre2_code_free_8
  > U -- pcre2_compile_8
  > U -- pcre2_compile_context_create_8
  > U -- pcre2_compile_context_free_8
  > U -- pcre2_config_8
  > U -- pcre2_general_context_create_8
  > U -- pcre2_general_context_free_8
  > U -- pcre2_get_error_message_8
  > U -- pcre2_get_ovector_pointer_8
  > U -- pcre2_jit_compile_8
  > U -- pcre2_jit_match_8
  > U -- pcre2_maketables_8
  > U -- pcre2_maketables_free_8
  > U -- pcre2_match_8
  > U -- pcre2_match_data_create_from_pattern_8
  > U -- pcre2_match_data_free_8
  > U -- pcre2_pattern_info_8
  > U -- pcre2_set_character_tables_8
  2045c2064
  < U -- __printf_chk@GLIBC_2.3.4
  ---
  > U -- printf@GLIBC_2.2.5
  2080a2100
  > U -- pthread_equal@GLIBC_2.2.5
  2767c2787
  < U -- __snprintf_chk@GLIBC_2.3.4
  ---
  > U -- snprintf@GLIBC_2.2.5
  2925a2946
  > U -- strlcpy@GLIBC_2.38
  3185d3205
  < U -- __uflow@GLIBC_2.2.5
  3289,3290c3309,3311
  < U -- __vfprintf_chk@GLIBC_2.3.4
  < U -- __vsnprintf_chk@GLIBC_2.3.4
  ---
  > U -- vfprintf@GLIBC_2.2.5
  > U -- vprintf@GLIBC_2.2.5
  > U -- vsnprintf@GLIBC_2.2.5
  $ 

Looking at the above, you can see that quite a few differences relate to symbols which differ due to the level of optimization during compilation. This reflects, among other things, the different 'default' settings between the Makefile and the meson build; e.g. prefix=$HOME vs. /usr/local, initial CFLAGS='-g -O2 -Wall' vs. '-g -O0 -Wall'. Also, note that the make build does not include the (optional) pcre2 library, even if it is installed on the system.

[On cygwin the diff is slightly different, but also contains more symbols, such as all symbols relating to the compat regex routines, clock_gettime, getdelim, sysinfo and so on ... ;) ]

If we change the meson setup, thus:

  $ meson setup --optimization=2 -Dprefix=$HOME -Dpcre2=disabled build/

After we re-build and repeat the above symbol generation, we find:

  $ diff git.syms mgit.syms
  88a89
  > U -- arc4random_buf@GLIBC_2.36
  746,747c747,748
  < B -- __environ@GLIBC_2.2.5
  < V -- environ@GLIBC_2.2.5
  ---
  > U -- __environ@GLIBC_2.2.5
  > U -- environ@GLIBC_2.2.5
  1253d1253
  < T -- gitstrlcpy
  2925a2926
  > U -- strlcpy@GLIBC_2.38
  $ 

After applying these patches and adding settings to the config.mak file: 

  $ vim config.mak # add NO_STRLCPY and CSPRNG_METHOD
  $ 
  $ cat config.mak
  DEFAULT_TEST_TARGET=prove
  GIT_PROVE_OPTS='--timer'
  PYTHON_PATH=/usr/bin/python3
  GIT_TEST_HTTPD=false
  export GIT_TEST_HTTPD
  #DEVELOPER=1
  SPARSE_FLAGS += -D__STDC_NO_VLA__
  NO_STRLCPY=
  CSPRNG_METHOD=arc4random
  $ 

[Note that I commented out DEVELOPER=1. I'm not entirely sure why I did that. :) Maybe it was one less variable? ]

  $ make >out 2>&1
  $ 

  $ meson setup --optimization=2 -Dprefix=$HOME -Dpcre2=disabled build/
  The Meson build system
  ...
    User defined options
      optimization: 2
      prefix      : /home/ramsay
      pcre2       : disabled
  
  Found ninja-1.11.1 at /usr/bin/ninja
  $ 

  $ cd build
  $ meson compile
  INFO: autodetecting backend as ninja
  INFO: calculating backend command to run: /usr/bin/ninja
  [661/661] Copying file contrib/completion/git-prompt.sh
  $ cd ..
  $ 

  $ rm *.syms
  $ ./syms.pl git >git.syms 2>/dev/null
  $ ./syms.pl build/git >mgit.syms 2>/dev/null
  $ 

  $ diff git.syms mgit.syms
  747,748c747,748
  < B -- __environ@GLIBC_2.2.5
  < V -- environ@GLIBC_2.2.5
  ---
  > U -- __environ@GLIBC_2.2.5
  > U -- environ@GLIBC_2.2.5
  $ 
 
On cygwin, this diff is now empty. ;)

The above diff seems to imply a difference in the linking of main() and the C library startup routines (or something similar). However, I happened to notice that the meson build included '-fPIC' in the build flags, so I thought it was worth a punt:

  $ make CFLAGS='-g -O2 -Wall -fPIC' >out 2>&1

[with the advent of ASLR, I thought -fPIC/-fPIE was the default anyway!]

  $ diff git.syms mgit.syms
  2809,2811c2809,2811
  < U -- stderr@GLIBC_2.2.5
  < U -- stdin@GLIBC_2.2.5
  < U -- stdout@GLIBC_2.2.5
  ---
  > B -- stderr@GLIBC_2.2.5
  > B -- stdin@GLIBC_2.2.5
  > B -- stdout@GLIBC_2.2.5
  $ 

Hmm, so one step forward, one step back! (or is it 2 steps forward, 3 steps back? ;) ). So, something for another day.

Note that both test suite runs ('make test' and 'meson test') pass all tests (excluding the http, cvs, svn, and p4 tests) on Linux. (I am still waiting for the v2.49.0 test run to finish on cygwin! :( ).

Also, patches #5 and #12 only add/modify a comment, so they could be dropped, if people feel that they are not appropriate.

[Just in case you want to reproduce the above, I have appended the syms.pl script below.]

ATB, Ramsay Jones

Ramsay Jones (13):
  meson.build: remove -DCURL_DISABLE_TYPECHECK
  Makefile: only set some BASIC_CFLAGS when RUNTIME_PREFIX is set
  meson.build: only set build variables for non-default values
  meson.build: set default help format to html on windows
  Makefile: remove NEEDS_LIBRT build variable
  config.mak.uname: add a note about NO_STRLCPY for Linux
  config.mak.uname: only set NO_REGEX on cygwin for v1.7
  config.mak.uname: add HAVE_GETDELIM to the cygwin section
  config.mak.uname: add clock_gettime() to the cygwin build
  builtin/gc.c: correct RAM calculation when using sysinfo
  config.mak.uname: add sysinfo() configuration for cygwin
  config.mak.uname: add arc4random to the cygwin build
  config.mak.uname: set CSPRNG_METHOD to getrandom on Linux

 Documentation/meson.build | 20 +++++++++++++++++
 Makefile                  | 45 ++++++++++++++++++++-------------------
 builtin/gc.c              |  9 ++++++--
 config.mak.uname          | 15 +++++++++----
 configure.ac              |  8 +++++++
 meson.build               | 27 +++++++++++++++++++----
 meson_options.txt         |  2 +-
 t/t7815-grep-binary.sh    |  2 +-
 8 files changed, 94 insertions(+), 34 deletions(-)

range-diff against v1:

 1:  50ce21363e =  1:  fe969deec0 meson.build: remove -DCURL_DISABLE_TYPECHECK
 2:  b7097a71b4 =  2:  8df541634a Makefile: only set some BASIC_CFLAGS when RUNTIME_PREFIX is set
 3:  85363624b2 <  -:  ---------- meson.build: only set build variables for non-default values
 -:  ---------- >  3:  280363cd56 meson.build: only set build variables for non-default values
 -:  ---------- >  4:  34349bf98c meson.build: set default help format to html on windows
 4:  4bda84bbf2 =  5:  e73c2c3219 Makefile: remove NEEDS_LIBRT build variable
 5:  45d570f6d4 =  6:  8c76671f0e config.mak.uname: add a note about NO_STRLCPY for Linux
 6:  45ef2d4268 !  7:  324bb21342 config.mak.uname: only set NO_REGEX on cygwin for v1.7
    @@ Commit message
         some of the tests (t4018 and t4034), which passed just fine with the
         compat library.
     
    -    After some time (may a year or two), the platform library had been
    +    After some time (maybe a year or two), the platform library had been
         updated (with an import from FreeBSD, I believe) and now passed the full
         test-suite. This would be about the time of the v1.7 -> v2.0 transition
         in 2015. I had a patch ready to send, but just didn't get around to
    @@ Commit message
         NO_REGEX variable to use the platform regex library functions. The new
         meson build system does likewise.
     
    +    The cygwin platform regex library, in addition to now passing the tests
    +    which formerly failed, now passes an 'test_expect_failure' test in the
    +    t7815-grep-binary test file. In particular, test #12 'git grep .fi a'
    +    which determines that the regex pattern '.' matches a NUL character.
    +    The commit f96e56733a ("grep: use REG_STARTEND for all matching if
    +    available", 2010-05-22) added the test in question, but it does not
    +    give any indication as to why the test was framed as an expected fail,
    +    rather than a 'positive' test that the 'git grep' command fails to
    +    match a NUL. Note that the previous test #11 was also originally
    +    marked in that commit as a 'test_expect_failure', but was flipped to
    +    an 'success' test in commit 7e36de5859 ("t/t7008-grep-binary.sh: un-TODO
    +    a test that needs REG_STARTEND", 2010-08-17).
    +
         In order to produce the same NO_REGEX configuration from autoconf, meson
         and make, modify config.mak.uname to only set NO_REGEX for cygwin v1.7.
    +    In addition, skip test t7815.12 on cygwin, by adding the !CYGWIN pre-
    +    requisite to the test header, which (among other things) removes an
    +    '...; please update test(s)' comment.
     
      ## config.mak.uname ##
     @@ config.mak.uname: ifeq ($(uname_O),Cygwin)
    @@ config.mak.uname: ifeq ($(uname_O),Cygwin)
              endif
      	HAVE_DEV_TTY = YesPlease
      	HAVE_ALLOCA_H = YesPlease
    +
    + ## t/t7815-grep-binary.sh ##
    +@@ t/t7815-grep-binary.sh: test_expect_success 'git grep ile a' '
    + 	git grep ile a
    + '
    + 
    +-test_expect_failure 'git grep .fi a' '
    ++test_expect_failure !CYGWIN 'git grep .fi a' '
    + 	git grep .fi a
    + '
    + 
 7:  aa3e298fde =  8:  0a3ce7e80a config.mak.uname: add HAVE_GETDELIM to the cygwin section
 8:  dc30aa5653 =  9:  3bd6e40cd2 config.mak.uname: add clock_gettime() to the cygwin build
 9:  1e92c641a6 = 10:  a2eb9ab117 builtin/gc.c: correct RAM calculation when using sysinfo
10:  6e208c53dd = 11:  e45b9df3a4 config.mak.uname: add sysinfo() configuration for cygwin
11:  990fde71bb ! 12:  422d4b2571 config.mak.uname; add arc4random to the cygwin build
    @@ Metadata
     Author: Ramsay Jones <ramsay@ramsayjones.plus.com>
     
      ## Commit message ##
    -    config.mak.uname; add arc4random to the cygwin build
    +    config.mak.uname: add arc4random to the cygwin build
     
         The arc4random_buf() function has been available in cygwin since
         about 2016 (somewhere in the v2.x branch). Set the CSPRNG_METHOD
12:  847ca6b16c ! 13:  7b133af24b config.mak.uname: add a note about CSPRNG_METHOD for Linux
    @@ Metadata
     Author: Ramsay Jones <ramsay@ramsayjones.plus.com>
     
      ## Commit message ##
    -    config.mak.uname: add a note about CSPRNG_METHOD for Linux
    +    config.mak.uname: set CSPRNG_METHOD to getrandom on Linux
     
         Commit 05cd988dce ("wrapper: add a helper to generate numbers from a
         CSPRNG", 2022-01-17) added a csprng_bytes() function which used one
    @@ Commit message
         Commit 05cd988dce did not set CSPRNG_METHOD in the Linux section of
         the config.mak.uname file, so it defaults to using '/dev/urandom' as
         the source of random bytes. The 'backend' values which could be used
    -    on Linux, in order of preference, are 'arc4random', 'getrandom' or
    -    'getentropy' ('openssl' is an option, but seems to be discouraged).
    +    on Linux are 'arc4random', 'getrandom' or 'getentropy' ('openssl' is
    +    an option, but seems to be discouraged).
     
         The arc4random routines (ar4random_buf() is the one actually used) were
         added to glibc in version 2.36, while both getrandom() and getentropy()
         were included in 2.25. So, some of the more up-to-date distributions of
    -    Linux (eg Debian 12, Ubuntu 24.04) would be able to use the preferred
    -    'arc4random' setting.
    +    Linux (eg Debian 12, Ubuntu 24.04) would be able to use the 'arc4random'
    +    setting. All currently supported distributions have glibc 2.25 or later
    +    (RHEL 8 has v2.28) and, therefore, have support for the 'getrandom' and
    +    'getentropy' settings.
    +
    +    The arc4random routines on the *BSDs (along with cygwin) implement the
    +    ChaCha20 stream cipher algorithm (see RFC8439) in userspace, rather than
    +    as a system call, and are thus somewhat faster (having avoided a context
    +    switch to the kernel). In contrast, on Linux all three functions are
    +    simple wrappers around the same kernel CSPRNG syscall.
     
         If the meson build system is used on a newer platform, then they will be
         configured to use 'arc4random', whereas the make build will currently
    -    default to using '/dev/urandom'. Add a note to the config.mak.uname file,
    -    in the Linux section, to prompt make users to override CSPRNG_METHOD in
    -    the config.mak file, if appropriate.
    +    default to using '/dev/urandom' on Linux. Since there is no advantage,
    +    in terms of performance, to the 'arc4random' setting, the 'getrandom'
    +    setting should be preferred from an availability perspective. (Also, the
    +    current uses of csprng_bytes() are not in any hot path).
    +
    +    In order to set an appropriate default, set the CSPRNG_METHOD build
    +    variable to 'getrandom' in the Linux section of the 'config.mak.uname'
    +    file.
     
      ## config.mak.uname ##
     @@ config.mak.uname: ifeq ($(uname_S),Linux)
      	HAVE_ALLOCA_H = YesPlease
      	# override in config.mak if you have glibc >= 2.38
      	NO_STRLCPY = YesPlease
    -+	# set to arc4random (in config.mak) if you have glibc >= 2.36
    -+	CSPRNG_METHOD =
    ++	CSPRNG_METHOD = getrandom
      	HAVE_PATHS_H = YesPlease
      	LIBC_CONTAINS_LIBINTL = YesPlease
      	HAVE_DEV_TTY = YesPlease