git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/9] cmake: use GIT_TEST_BUILD_DIR instead of editing hack

From
PWPhillip Wood <phillip.wood123@gmail.com>
Date
Oct 21, 2022, 14:18 UTC
Message-ID
<d6e2baab-be26-66d1-f07c-134eb6fe06fd@dunelm.org.uk>
In-Reply-To
<patch-6.9-45f1a4e6f93-20221021T091013Z-avarab@gmail.com>
Hi Ævar
On 21/10/2022 10:44, Ævar Arnfjörð Bjarmason wrote:
Show 15 quoted lines
> When cmake builds git in "contrib/buildsystems/out" it will create a
> "t/" directory there containing only the "t/helper/test-tool", but for
> running the tests with "cmake" it cd's to the "real" t/ directory, and
> runs the tests from there.
> 
> To get the test-lib.sh to locate "git" and other binaries in
> "../contrib/buildsystems/out/" rather than "../" we have since [1]
> been editing the "GIT_BUILD_DIR" assignment in test-lib.sh in-place.
> 
> This has meant that when testing cmake we've had to "git reset --hard"
> before running "make" again.
> 
> What this build infrastructure really wanted was some feature like the
> "GIT_TEST_BUILD_DIR" variable added in the preceding commit, so let's
> make use of it.

Lets squash that commit into this one, so we can see how it is used when it is added.

Show 14 quoted lines
> Even though "ctest" will work with this approach, one advantage of the
> previous arrangement was that we could:
> 
> 	A. Build with cmake
> 	B. cd t
> 	C. Run a test
> 
> And have the test itself know to locate and use the cmake binaries,
> this workflow was documented in [2]. The "t/test-lib".sh modification
> here is so that we can support this use-case.
> 
> As [3] notes "contrib/buildsystems/out" isn't just the directory that
> happens to be documented in "contrib/buildsystems/CMakeLists.txt", but
> the one that VS will use when building git.

That may be the directory that VS uses when building git, but it is possible to specify a different build directory when running cmake.

Show 53 quoted lines
> To make it clear what's happening we emit a "setup: " from the
> test-lib.sh to note that we used this fallback method:
> 
> 	$ ./t0001-init.sh
> 	setup: had no ../git, but found & used cmake built git in ../contrib/buildsystems/out/git
> 	ok 1 - plain
> 	[...]
> 
> Note: the "On Windows[...]" part of this is lifted from [4].
> 
> 1. 7f5397a07c6 (cmake: support for testing git when building out of
>     the source tree, 2020-06-26)
> 2. f2f1250c47f (cmake (Windows): recommend using Visual Studio's
>     built-in CMake support, 2020-09-30)
> 3. 3eccc7b99d4 (cmake: ignore files generated by CMake as run in
>     Visual Studio, 2020-09-25)
> 4. https://lore.kernel.org/git/5b0c2a150e9fce1ca0284d65628b42ed5a7aad9a.1666090745.git.gitgitgadget@gmail.com/
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>   contrib/buildsystems/CMakeLists.txt | 15 +--------------
>   t/test-lib.sh                       | 19 +++++++++++++++++++
>   2 files changed, 20 insertions(+), 14 deletions(-)
> 
> diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
> index 725b3f2ac82..91b7009f4fd 100644
> --- a/contrib/buildsystems/CMakeLists.txt
> +++ b/contrib/buildsystems/CMakeLists.txt
> @@ -1080,25 +1080,12 @@ if(USE_VCPKG)
>   	file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS "PATH=\"$PATH:$TEST_DIRECTORY/../compat/vcbuild/vcpkg/installed/x64-windows/bin\"\n")
>   endif()
>   
> -#Make the tests work when building out of the source tree
> -get_filename_component(CACHE_PATH ${CMAKE_CURRENT_LIST_DIR}/../../CMakeCache.txt ABSOLUTE)
> -if(NOT ${CMAKE_BINARY_DIR}/CMakeCache.txt STREQUAL ${CACHE_PATH})
> -	file(RELATIVE_PATH BUILD_DIR_RELATIVE ${CMAKE_SOURCE_DIR} ${CMAKE_BINARY_DIR}/CMakeCache.txt)
> -	string(REPLACE "/CMakeCache.txt" "" BUILD_DIR_RELATIVE ${BUILD_DIR_RELATIVE})
> -	#Setting the build directory in test-lib.sh before running tests
> -	file(WRITE ${CMAKE_BINARY_DIR}/CTestCustom.cmake
> -		"file(STRINGS ${CMAKE_SOURCE_DIR}/t/test-lib.sh GIT_BUILD_DIR_REPL REGEX \"GIT_BUILD_DIR=(.*)\")\n"
> -		"file(STRINGS ${CMAKE_SOURCE_DIR}/t/test-lib.sh content NEWLINE_CONSUME)\n"
> -		"string(REPLACE \"\${GIT_BUILD_DIR_REPL}\" \"GIT_BUILD_DIR=\\\"$TEST_DIRECTORY/../${BUILD_DIR_RELATIVE}\\\"\" content \"\${content}\")\n"
> -		"file(WRITE ${CMAKE_SOURCE_DIR}/t/test-lib.sh \${content})")
> -endif()
> -
>   file(GLOB test_scipts "${CMAKE_SOURCE_DIR}/t/t[0-9]*.sh")
>   
>   #test
>   foreach(tsh ${test_scipts})
>   	add_test(NAME ${tsh}
> -		COMMAND ${SH_EXE} ${tsh}
> +		COMMAND env GIT_TEST_BUILD_DIR=${CMAKE_BINARY_DIR} ${SH_EXE} ${tsh}

I'm not sure about using env on windows, can we use ${SH_EXE} -c instead to avoid creating an extra process?

	COMMAND ${SH_EXE} -c [[GIT_TEST_BUILD_DIR="$1"; . "$2"]] "${tsh}" 
"${CMAKE_BINARY_DIR}" "${tsh}"
Show 17 quoted lines
>   		WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/t)
>   endforeach()
>   
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 41b1ddc96ff..284b619708a 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -47,9 +47,21 @@ fi
>   # its build directory.
>   GIT_SOURCE_DIR="${TEST_DIRECTORY%/t}"
>   GIT_BUILD_DIR="$GIT_SOURCE_DIR"
> +GIT_AUTO_CONTRIB_BUILDSYSTEMS_OUT=
>   if test -n "$GIT_TEST_BUILD_DIR"
>   then
>   	GIT_BUILD_DIR="$GIT_TEST_BUILD_DIR"
> +elif ! test -x "$GIT_BUILD_DIR/git" &&
> +     test -x "$GIT_BUILD_DIR/contrib/buildsystems/out/git"

I'm really not keen on hard coding the CMAKE_BINARY_DIR. One of the positive things about dscho's approach is that it does not hard code the build directory. I'm not convinced this approach is an improvement.

Best Wishes
Phillip
Show 26 quoted lines
> +then
> +	GIT_BUILD_DIR="$GIT_SOURCE_DIR/contrib/buildsystems/out"
> +	GIT_AUTO_CONTRIB_BUILDSYSTEMS_OUT=t
> +	# On Windows, we must convert Windows paths lest they contain a colon
> +	case "$(uname -s)" in
> +	*MINGW*)
> +		GIT_BUILD_DIR="$(cygpath -au "$GIT_BUILD_DIR")"
> +		;;
> +	esac
>   fi
>   
>   if test "$TEST_DIRECTORY" = "$GIT_BUILD_DIR"
> @@ -1630,6 +1642,13 @@ remove_trash_directory "$TRASH_DIRECTORY" || {
>   	BAIL_OUT 'cannot prepare test area'
>   }
>   
> +# Emitting this now because earlier we didn't have "say", but not in
> +# anything using lib-subtest.sh
> +if test -n "$GIT_AUTO_CONTRIB_BUILDSYSTEMS_OUT" && test -t 1
> +then
> +	say "setup: had no ../git, but found & used cmake built git in ../contrib/buildsystems/out/git"
> +fi
> +
>   remove_trash=t
>   if test -z "$TEST_NO_CREATE_REPO"
>   then
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 9 of 113 in “cmake: fix *nix & general issues, no test-lib.sh editing, ctest in CI”
  1. 0/9 cmake: fix *nix & general issues, no test-lib.sh editing, ctest in CIÆvar Arnfjörð Bjarmason, Oct 21, 2022
  2. 1/9 cmake: don't copy chainlint.pl to build directoryÆvar Arnfjörð Bjarmason, Oct 21, 2022
  3. 2/9 cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4Ævar Arnfjörð Bjarmason, Oct 21, 2022
  4. Phillip WoodOct 21, 2022
  5. Ævar Arnfjörð BjarmasonOct 21, 2022
  6. 4/9 cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.shÆvar Arnfjörð Bjarmason, Oct 21, 2022
  7. 5/9 test-lib.sh: support a "GIT_TEST_BUILD_DIR"Ævar Arnfjörð Bjarmason, Oct 21, 2022
  8. 6/9 cmake: use GIT_TEST_BUILD_DIR instead of editing hackÆvar Arnfjörð Bjarmason, Oct 21, 2022
  9. Phillip WoodOct 21, 2022
  10. Ævar Arnfjörð BjarmasonOct 21, 2022
  11. Phillip WoodOct 25, 2022
  12. 3/9 cmake & test-lib.sh: add a $GIT_SOURCE_DIR variableÆvar Arnfjörð Bjarmason, Oct 21, 2022
  13. 7/9 cmake: support using GIT_TEST_OPTS from the environmentÆvar Arnfjörð Bjarmason, Oct 21, 2022
  14. Phillip WoodOct 21, 2022
  15. Ævar Arnfjörð BjarmasonOct 21, 2022
  16. Phillip WoodOct 25, 2022
  17. Ævar Arnfjörð BjarmasonOct 25, 2022
  18. 8/9 cmake: copy over git-p4.py for t983[56] perforce testÆvar Arnfjörð Bjarmason, Oct 21, 2022
  19. 9/9 CI: add a "linux-cmake-test" to run cmake & ctest on linuxÆvar Arnfjörð Bjarmason, Oct 21, 2022
  20. Johannes SchindelinOct 21, 2022
  21. Phillip WoodOct 21, 2022
  22. Ævar Arnfjörð BjarmasonOct 21, 2022
  23. Phillip WoodOct 25, 2022
  24. 00/11 cmake: document, fix on *nix, add CIÆvar Arnfjörð Bjarmason, Oct 27, 2022
  25. 01/11 cmake: don't "mkdir -p" and "cd" in build instructionsÆvar Arnfjörð Bjarmason, Oct 27, 2022
  26. 02/11 cmake: update instructions for portable CMakeLists.txtÆvar Arnfjörð Bjarmason, Oct 27, 2022
  27. Eric SunshineOct 27, 2022
  28. 03/11 cmake: don't copy chainlint.pl to build directoryÆvar Arnfjörð Bjarmason, Oct 27, 2022
  29. 05/11 cmake & test-lib.sh: add a $GIT_SOURCE_DIR variableÆvar Arnfjörð Bjarmason, Oct 27, 2022
  30. 04/11 cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4Ævar Arnfjörð Bjarmason, Oct 27, 2022
  31. 06/11 cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.shÆvar Arnfjörð Bjarmason, Oct 27, 2022
  32. 08/11 Makefile + cmake: use environment, not GIT-BUILD-DIRÆvar Arnfjörð Bjarmason, Oct 27, 2022
  33. 07/11 test-lib.sh: support a "GIT_TEST_BUILD_DIR"Ævar Arnfjörð Bjarmason, Oct 27, 2022
  34. 09/11 cmake: support GIT_TEST_OPTS, abstract away WIN32 defaultsÆvar Arnfjörð Bjarmason, Oct 27, 2022
  35. 10/11 cmake: copy over git-p4.py for t983[56] perforce testÆvar Arnfjörð Bjarmason, Oct 27, 2022
  36. 11/11 CI: add a "linux-cmake-test" to run cmake & ctest on linuxÆvar Arnfjörð Bjarmason, Oct 27, 2022
  37. 00/12 cmake: document, fix on *nix, add CIÆvar Arnfjörð Bjarmason, Nov 1, 2022
  38. 01/12 cmake: don't "mkdir -p" and "cd" in build instructionsÆvar Arnfjörð Bjarmason, Nov 1, 2022
  39. Phillip WoodNov 3, 2022
  40. 02/12 cmake: update instructions for portable CMakeLists.txtÆvar Arnfjörð Bjarmason, Nov 1, 2022
  41. Eric SunshineNov 1, 2022
  42. Phillip WoodNov 3, 2022
  43. Ævar Arnfjörð BjarmasonNov 3, 2022
  44. 03/12 cmake: don't copy chainlint.pl to build directoryÆvar Arnfjörð Bjarmason, Nov 1, 2022
  45. 04/12 cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4Ævar Arnfjörð Bjarmason, Nov 1, 2022
  46. 05/12 cmake & test-lib.sh: add a $GIT_SOURCE_DIR variableÆvar Arnfjörð Bjarmason, Nov 1, 2022
  47. 06/12 cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.shÆvar Arnfjörð Bjarmason, Nov 1, 2022
  48. 07/12 test-lib.sh: support a "GIT_TEST_BUILD_DIR"Ævar Arnfjörð Bjarmason, Nov 1, 2022
  49. 08/12 Makefile + cmake: use environment, not GIT-BUILD-DIRÆvar Arnfjörð Bjarmason, Nov 1, 2022
  50. 09/12 cmake: support GIT_TEST_OPTS, abstract away WIN32 defaultsÆvar Arnfjörð Bjarmason, Nov 1, 2022
  51. Phillip WoodNov 3, 2022
  52. Ævar Arnfjörð BjarmasonNov 3, 2022
  53. 11/12 cmake: copy over git-p4.py for t983[56] perforce testÆvar Arnfjörð Bjarmason, Nov 1, 2022
  54. 10/12 cmake: increase test timeout on Windows onlyÆvar Arnfjörð Bjarmason, Nov 1, 2022
  55. 12/12 CI: add a "linux-cmake-test" to run cmake & ctest on linuxÆvar Arnfjörð Bjarmason, Nov 1, 2022
  56. 00/14 cmake: document, fix on *nix, add CIÆvar Arnfjörð Bjarmason, Nov 3, 2022
  57. 02/14 cmake: use "-S" and "-B" to specify source and build directoriesÆvar Arnfjörð Bjarmason, Nov 3, 2022
  58. 05/14 cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4Ævar Arnfjörð Bjarmason, Nov 3, 2022
  59. 04/14 cmake: don't copy chainlint.pl to build directoryÆvar Arnfjörð Bjarmason, Nov 3, 2022
  60. 01/14 cmake: don't invoke msgfmt with --statisticsÆvar Arnfjörð Bjarmason, Nov 3, 2022
  61. 03/14 cmake: update instructions for portable CMakeLists.txtÆvar Arnfjörð Bjarmason, Nov 3, 2022
  62. 06/14 cmake & test-lib.sh: add a $GIT_SOURCE_DIR variableÆvar Arnfjörð Bjarmason, Nov 3, 2022
  63. 07/14 cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.shÆvar Arnfjörð Bjarmason, Nov 3, 2022
  64. 13/14 cmake: copy over git-p4.py for t983[56] perforce testÆvar Arnfjörð Bjarmason, Nov 3, 2022
  65. 09/14 Makefile + cmake: use environment, not GIT-BUILD-DIRÆvar Arnfjörð Bjarmason, Nov 3, 2022
  66. 08/14 test-lib.sh: support a "GIT_TEST_BUILD_DIR"Ævar Arnfjörð Bjarmason, Nov 3, 2022
  67. 11/14 cmake: increase test timeout on Windows onlyÆvar Arnfjörð Bjarmason, Nov 3, 2022
  68. 12/14 cmake: only look for "sh" in "C:/Program Files" on WindowsÆvar Arnfjörð Bjarmason, Nov 3, 2022
  69. 14/14 CI: add a "linux-cmake-test" to run cmake & ctest on linuxÆvar Arnfjörð Bjarmason, Nov 3, 2022
  70. 10/14 cmake: support GIT_TEST_OPTS, abstract away WIN32 defaultsÆvar Arnfjörð Bjarmason, Nov 3, 2022
  71. Taylor BlauNov 5, 2022
  72. Phillip WoodNov 8, 2022
  73. 00/15 cmake: document, fix on *nix, add CIÆvar Arnfjörð Bjarmason, Dec 2, 2022
  74. 01/15 cmake: don't invoke msgfmt with --statisticsÆvar Arnfjörð Bjarmason, Dec 2, 2022
  75. 03/15 cmake: update instructions for portable CMakeLists.txtÆvar Arnfjörð Bjarmason, Dec 2, 2022
  76. 02/15 cmake: use "-S" and "-B" to specify source and build directoriesÆvar Arnfjörð Bjarmason, Dec 2, 2022
  77. Eric SunshineDec 3, 2022
  78. 04/15 cmake: don't copy chainlint.pl to build directoryÆvar Arnfjörð Bjarmason, Dec 2, 2022
  79. 05/15 cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4Ævar Arnfjörð Bjarmason, Dec 2, 2022
  80. 07/15 cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.shÆvar Arnfjörð Bjarmason, Dec 2, 2022
  81. 06/15 cmake & test-lib.sh: add a $GIT_SOURCE_DIR variableÆvar Arnfjörð Bjarmason, Dec 2, 2022
  82. 08/15 Makefile + test-lib.sh: don't prefer cmake-built to make-built gitÆvar Arnfjörð Bjarmason, Dec 2, 2022
  83. 09/15 test-lib.sh: support a "GIT_TEST_BUILD_DIR"Ævar Arnfjörð Bjarmason, Dec 2, 2022
  84. Eric SunshineDec 3, 2022
  85. 10/15 cmake: optionally be able to run tests before "ctest"Ævar Arnfjörð Bjarmason, Dec 2, 2022
  86. 11/15 cmake: support GIT_TEST_OPTS, abstract away WIN32 defaultsÆvar Arnfjörð Bjarmason, Dec 2, 2022
  87. Eric SunshineDec 3, 2022
  88. Ævar Arnfjörð BjarmasonDec 3, 2022
  89. Eric SunshineDec 3, 2022
  90. 12/15 cmake: increase test timeout on Windows onlyÆvar Arnfjörð Bjarmason, Dec 2, 2022
  91. 13/15 cmake: only look for "sh" in "C:/Program Files" on WindowsÆvar Arnfjörð Bjarmason, Dec 2, 2022
  92. 14/15 cmake: copy over git-p4.py for t983[56] perforce testÆvar Arnfjörð Bjarmason, Dec 2, 2022
  93. 15/15 CI: add a "linux-cmake-test" to run cmake & ctest on linuxÆvar Arnfjörð Bjarmason, Dec 2, 2022
  94. Eric SunshineDec 3, 2022
  95. 00/15 cmake: document, fix on *nix, add CIÆvar Arnfjörð Bjarmason, Dec 6, 2022
  96. 01/15 cmake: don't invoke msgfmt with --statisticsÆvar Arnfjörð Bjarmason, Dec 6, 2022
  97. 02/15 cmake: use "-S" and "-B" to specify source and build directoriesÆvar Arnfjörð Bjarmason, Dec 6, 2022
  98. 04/15 cmake: don't copy chainlint.pl to build directoryÆvar Arnfjörð Bjarmason, Dec 6, 2022
  99. 03/15 cmake: update instructions for portable CMakeLists.txtÆvar Arnfjörð Bjarmason, Dec 6, 2022
  100. 05/15 cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4Ævar Arnfjörð Bjarmason, Dec 6, 2022
  101. 06/15 cmake & test-lib.sh: add a $GIT_SOURCE_DIR variableÆvar Arnfjörð Bjarmason, Dec 6, 2022
  102. 07/15 cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.shÆvar Arnfjörð Bjarmason, Dec 6, 2022
  103. 08/15 Makefile + test-lib.sh: don't prefer cmake-built to make-built gitÆvar Arnfjörð Bjarmason, Dec 6, 2022
  104. 09/15 test-lib.sh: support a "GIT_TEST_BUILD_DIR"Ævar Arnfjörð Bjarmason, Dec 6, 2022
  105. 10/15 cmake: optionally be able to run tests before "ctest"Ævar Arnfjörð Bjarmason, Dec 6, 2022
  106. 11/15 cmake: support GIT_TEST_OPTS, abstract away WIN32 defaultsÆvar Arnfjörð Bjarmason, Dec 6, 2022
  107. 12/15 cmake: increase test timeout on Windows onlyÆvar Arnfjörð Bjarmason, Dec 6, 2022
  108. 14/15 cmake: copy over git-p4.py for t983[56] perforce testÆvar Arnfjörð Bjarmason, Dec 6, 2022
  109. 13/15 cmake: only look for "sh" in "C:/Program Files" on WindowsÆvar Arnfjörð Bjarmason, Dec 6, 2022
  110. 15/15 CI: add a "linux-cmake-test" to run cmake & ctest on linuxÆvar Arnfjörð Bjarmason, Dec 6, 2022
  111. Phillip WoodDec 7, 2022
  112. Ævar Arnfjörð BjarmasonDec 8, 2022
  113. Phillip WoodNov 3, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.