git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 3/5] Makefile: simplify $(test_bindir_programs) rule by splitting it up

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Sep 1, 2022, 13:17 UTC
Message-ID
<patch-3.5-0f66992d5f7-20220901T130817Z-avarab@gmail.com>
In-Reply-To
<cover-0.5-00000000000-20220901T130817Z-avarab@gmail.com>

When the @@PROG@@ substitution was added in [1] it was a simple matter of doing a:

	's|@@PROG@@|$(@F)|'

Then when t/helpers were added in [2] followed by the ".exe" needing to be appended in [3] this previously simple rule ended up as a dense one-liner.

It has been pointed out that this is hard to read[4], but the problem isn't the "design of the Makefile syntax". It's we now have to after-the-fact determine if we were dealing with a bin-wrapper/ that needed to have $(X) appended, a t/helper/, or a non-binary (currently only git-cvsserver).

That would be a problem in any language. We're starting out with three lists, and then end up having to heuristically determine given a member of any of those lists which list that member came from. Let's just stop doing that and keep track of what member belongs to which list.

We can do this by splitting up the single "bin-wrappers/%" rule into a rule for each of the three lists. With the "cmd_munge_script_sed_shell_path_arg" define added in a preceding commit this is easy, we just need to add a sister template to the existing "cmd_munge_script" added in [5].

There is then an an in-flight change[6] that generalizes this to support "scalar". That change needed to further special-case the pre-image here, as we couldn't rely on the "git%" heuristic to mean "not from the $(TEST_PROGRAMS_NEED_X) variable".

With this conflicting change that special-casing becomes unnecessary. The conflict between the two can be resolved entirely in favor of this change, as the other change adds "scalar" to "$(BINDIR_PROGRAMS_NEED_X)", and we are no longer losing track of which members belong to that list.

This change can be tested with e.g.:
	git checkout master &&
	make SHELL_PATH=/bin/bash X=.exe &&
	mv bin-wrappers bin-wrappers.master &&
	<apply this change> &&
	make SHELL_PATH=/bin/bash X=.exe &&
	diff -ru bin-wrappers{.master,}

Which will show an empty diff, i.e. we've correctly dealt with the combination of $(SHELL_PATH), $(X) and these three variables here.

1. ea925196f1b (build dashless "bin-wrappers" directory similar to
   installed bindir, 2009-12-02)
2. e6e7530d10b (test helpers: move test-* to t/helper/ subdirectory,
   2016-04-13)
3. 3a94cb31d52 (bin-wrappers: append `.exe` to target paths if
   necessary, 2019-07-29)
4. https://lore.kernel.org/git/sso99so6-n28s-rq86-8q20-4456r3pn869r@tzk.qr/
5. 46bac904581 (Do not install shell libraries executable, 2010-01-31)
6. https://lore.kernel.org/git/4d69e5eaccb8873eece666a3d2bb2b22abdde7ea.1661961746.git.gitgitgadget@gmail.com/
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)
diff --git a/Makefile b/Makefile
index 0d6b2caa7d9..96de9e55864 100644
--- a/Makefile
+++ b/Makefile
@@ -597,6 +597,7 @@ export TCL_PATH TCLTK_PATH
 PTHREAD_LIBS = -lpthread
 
 # Guard against environment variables
+BIN_WRAPPERS =
 BUILTIN_OBJS =
 BUILT_INS =
 COMPAT_CFLAGS =
@@ -3061,16 +3062,34 @@ GIT-PYTHON-VARS: FORCE
             fi
 endif
 
-test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))
+define cmd_munge_bin_wrappers_script
+sed \
+	-e $(call cmd_munge_script_sed_shell_path_arg) \
+	-e 's|@@BUILD_DIR@@|$(shell pwd)|' \
+	-e 's|@@PROG@@|$(2)$(1)$(3)|' \
+	<$< >$@ && \
+	chmod +x $@
+endef
 
-all:: $(test_bindir_programs)
+BW_BINDIR_PROGRAMS_NEED_X = $(BINDIR_PROGRAMS_NEED_X:%=bin-wrappers/%)
+BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NEED_X)
+$(BW_BINDIR_PROGRAMS_NEED_X): wrap-for-bin.sh
+	$(call mkdir_p_parent_template)
+	$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),,$X)
 
-bin-wrappers/%: wrap-for-bin.sh
+BW_BINDIR_PROGRAMS_NO_X = $(BINDIR_PROGRAMS_NO_X:%=bin-wrappers/%)
+BIN_WRAPPERS += $(BW_BINDIR_PROGRAMS_NO_X)
+$(BW_BINDIR_PROGRAMS_NO_X): wrap-for-bin.sh
 	$(call mkdir_p_parent_template)
-	$(QUIET_GEN)sed -e $(call cmd_munge_script_sed_shell_path_arg) \
-	     -e 's|@@BUILD_DIR@@|$(shell pwd)|' \
-	     -e 's|@@PROG@@|$(patsubst test-%,t/helper/test-%$(X),$(@F))$(patsubst git%,$(X),$(filter $(@F),$(BINDIR_PROGRAMS_NEED_X)))|' < $< > $@ && \
-	chmod +x $@
+	$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F))
+
+BW_TEST_PROGRAMS_NEED_X = $(TEST_PROGRAMS_NEED_X:%=bin-wrappers/%)
+BIN_WRAPPERS += $(BW_TEST_PROGRAMS_NEED_X)
+$(BW_TEST_PROGRAMS_NEED_X): wrap-for-bin.sh
+	$(call mkdir_p_parent_template)
+	$(QUIET_GEN)$(call cmd_munge_bin_wrappers_script,$(@F),t/helper/,$X)
+
+all:: $(BIN_WRAPPERS)
 
 # GNU make supports exporting all variables by "export" without parameters.
 # However, the environment gets quite big, and some programs have problems
@@ -3403,7 +3422,7 @@ OTHER_PROGRAMS += $(shell echo *.dll t/helper/*.dll)
 endif
 
 artifacts-tar:: $(ALL_COMMANDS_TO_INSTALL) $(SCRIPT_LIB) $(OTHER_PROGRAMS) \
-		GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \
+		GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(BIN_WRAPPERS) \
 		$(MOFILES)
 	$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \
 		SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)'
-- 
2.37.3.1426.g360dd7cf8ca
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 9 of 70 in “scalar: integrate into core Git”
  1. 0/8 scalar: integrate into core GitVictoria Dye via GitGitGadget, Aug 31, 2022
  2. 1/8 scalar: fix command documentation section headerVictoria Dye via GitGitGadget, Aug 31, 2022
  3. 3/8 git help: special-case `scalar`Johannes Schindelin via GitGitGadget, Aug 31, 2022
  4. 2/8 scalar: include in standard Git build & installationVictoria Dye via GitGitGadget, Aug 31, 2022
  5. Johannes SchindelinSep 1, 2022
  6. 0/5 Makefile: split up $(test_bindir_programs)Ævar Arnfjörð Bjarmason, Sep 1, 2022
  7. 1/5 Makefile: factor sed-powered '#!/bin/sh' munging into a variableÆvar Arnfjörð Bjarmason, Sep 1, 2022
  8. 2/5 Makefile: define "TEST_{PROGRAM,OBJS}" variables earlierÆvar Arnfjörð Bjarmason, Sep 1, 2022
  9. 3/5 Makefile: simplify $(test_bindir_programs) rule by splitting it upÆvar Arnfjörð Bjarmason, Sep 1, 2022
  10. 5/5 Makefile: fix "make clean && make bin-wrappers/$NAME" dependenciesÆvar Arnfjörð Bjarmason, Sep 1, 2022
  11. 4/5 Makefile: define bin-wrappers/% rules with a templateÆvar Arnfjörð Bjarmason, Sep 1, 2022
  12. Derrick StoleeSep 1, 2022
  13. Johannes SchindelinSep 2, 2022
  14. 0/3 Makefile: fix issues with bin-wrappers/% ruleÆvar Arnfjörð Bjarmason, Oct 26, 2022
  15. 1/3 Makefile: factor sed-powered '#!/bin/sh' munging into a variableÆvar Arnfjörð Bjarmason, Oct 26, 2022
  16. Junio C HamanoOct 26, 2022
  17. 3/3 Makefile: simplify $(test_bindir_programs) rule by splitting it upÆvar Arnfjörð Bjarmason, Oct 26, 2022
  18. Junio C HamanoOct 26, 2022
  19. Ævar Arnfjörð BjarmasonOct 26, 2022
  20. Junio C HamanoOct 26, 2022
  21. Ævar Arnfjörð BjarmasonOct 26, 2022
  22. 2/3 Makefile: define "TEST_{PROGRAM,OBJS}" variables earlierÆvar Arnfjörð Bjarmason, Oct 26, 2022
  23. Junio C HamanoOct 26, 2022
  24. Ævar Arnfjörð BjarmasonOct 26, 2022
  25. Junio C HamanoOct 26, 2022
  26. Jeff KingOct 28, 2022
  27. Ævar Arnfjörð BjarmasonOct 28, 2022
  28. 0/4 Makefile: untangle bin-wrappers/% rule complexityÆvar Arnfjörð Bjarmason, Oct 31, 2022
  29. 1/4 Makefile: factor sed-powered '#!/bin/sh' munging into a variableÆvar Arnfjörð Bjarmason, Oct 31, 2022
  30. 3/4 Makefile: rename "test_bindir_programs" variable, pre-declareÆvar Arnfjörð Bjarmason, Oct 31, 2022
  31. 2/4 Makefile: define "TEST_{PROGRAM,OBJS}" variables earlierÆvar Arnfjörð Bjarmason, Oct 31, 2022
  32. 4/4 Makefile: simplify $(test_bindir_programs) rule by splitting it upÆvar Arnfjörð Bjarmason, Oct 31, 2022
  33. Taylor BlauOct 31, 2022
  34. Ævar Arnfjörð BjarmasonNov 1, 2022
  35. 4/8 scalar: implement the `help` subcommandJohannes Schindelin via GitGitGadget, Aug 31, 2022
  36. Ævar Arnfjörð BjarmasonAug 31, 2022
  37. Victoria DyeSep 1, 2022
  38. Johannes SchindelinSep 1, 2022
  39. Johannes SchindelinSep 1, 2022
  40. 5/8 scalar-clone: add test coverageVictoria Dye via GitGitGadget, Aug 31, 2022
  41. Johannes SchindelinSep 1, 2022
  42. Victoria DyeSep 1, 2022
  43. Johannes SchindelinSep 2, 2022
  44. Junio C HamanoSep 2, 2022
  45. 7/8 t/perf: add 'GIT_PERF_USE_SCALAR' run optionVictoria Dye via GitGitGadget, Aug 31, 2022
  46. Johannes SchindelinSep 1, 2022
  47. Victoria DyeSep 2, 2022
  48. Johannes SchindelinSep 2, 2022
  49. 6/8 t/perf: add Scalar performance testsVictoria Dye via GitGitGadget, Aug 31, 2022
  50. Johannes SchindelinSep 1, 2022
  51. Victoria DyeSep 1, 2022
  52. Victoria DyeSep 1, 2022
  53. Johannes SchindelinSep 2, 2022
  54. Junio C HamanoSep 1, 2022
  55. 8/8 Documentation/technical: include Scalar technical docVictoria Dye via GitGitGadget, Aug 31, 2022
  56. Ævar Arnfjörð BjarmasonAug 31, 2022
  57. Victoria DyeAug 31, 2022
  58. Johannes SchindelinSep 1, 2022
  59. 0/9 scalar: integrate into core GitVictoria Dye via GitGitGadget, Sep 2, 2022
  60. 4/9 scalar: implement the `help` subcommandJohannes Schindelin via GitGitGadget, Sep 2, 2022
  61. 3/9 git help: special-case `scalar`Johannes Schindelin via GitGitGadget, Sep 2, 2022
  62. 1/9 scalar: fix command documentation section headerVictoria Dye via GitGitGadget, Sep 2, 2022
  63. 7/9 t/perf: add Scalar performance testsVictoria Dye via GitGitGadget, Sep 2, 2022
  64. 6/9 scalar-clone: add test coverageVictoria Dye via GitGitGadget, Sep 2, 2022
  65. 2/9 scalar: include in standard Git build & installationVictoria Dye via GitGitGadget, Sep 2, 2022
  66. 8/9 t/perf: add 'GIT_PERF_USE_SCALAR' run optionVictoria Dye via GitGitGadget, Sep 2, 2022
  67. 9/9 Documentation/technical: include Scalar technical docVictoria Dye via GitGitGadget, Sep 2, 2022
  68. 5/9 scalar: add to 'git help -a' command listVictoria Dye via GitGitGadget, Sep 2, 2022
  69. Johannes SchindelinSep 5, 2022
  70. Derrick StoleeSep 8, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.