threads / patch / 57100

patch, 11 partsMakefile: fix "Perl" and "Python" rules, speed up re-builds

Subject: [PATCH 00/11] Makefile: fix "Perl" and "Python" rules, speed up re-builds

## tl;dr

13 messages between Dec 17, 2021 and Feb 21, 2022. Diffs are folded; open one to read it.

replies: 12people: 1as markdown or json

Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · lore

This topic is split-up from v4 of ab/make-dependency[1]. Range-diff to it below. As noted in [2] I opted to split up that series rather than having it grow too large.

This is miscellanious cleanups related to the "Perl" rules, and some fixes for bad dependency rules in 08/11. Then in 10/11 we make the "Python" rules use the new template we created for "Perl",

We are then able to speed up "make" re-builds a bit in 11/11, as we don't need to inject @@GIT_VERSION@@ into the Perl scripts anymore.

The 09/11 is new. It's a trivial sorting of the PERL_DEFINES list, which'll help to maintain it in the future.

1. https://lore.kernel.org/git/cover-v4-00.23-00000000000-20211117T101807Z-avarab@gmail.com/
2. https://lore.kernel.org/git/211217.86a6h09km9.gmgdl@evledraar.gmail.com/
Ævar Arnfjörð Bjarmason (11):
  Makefile: clean perl/build/ even with NO_PERL=Y
  Makefile: don't set up "perl/build" rules under NO_PERL=Y
  Makefile: use "=" not ":=" for po/* and perl/*
  Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR
  Makefile: guard Perl-only variable assignments
  Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"
  Makefile: adjust Perl-related comments & whitespace
  Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph
  Makefile: sort PERL_DEFINES list
  Makefile: create a GIT-PYTHON-DEFINES, like "PERL"
  Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts
 .gitignore          |   2 +-
 Makefile            | 160 +++++++++++++++++++++++---------------------
 git-cvsserver.perl  |   6 +-
 git-send-email.perl |   7 +-
 git-svn.perl        |   2 +-
 5 files changed, 90 insertions(+), 87 deletions(-)
Range-diff:
 1:  1621ca72c1d <  -:  ----------- Makefile: don't invoke msgfmt with --statistics
 4:  daead5ec293 =  1:  185f5a9a9cf Makefile: clean perl/build/ even with NO_PERL=Y
 2:  b7c36c9fea0 =  2:  37eeccc5646 Makefile: don't set up "perl/build" rules under NO_PERL=Y
 3:  29b000eb0f1 =  3:  572a82c71a1 Makefile: use "=" not ":=" for po/* and perl/*
 5:  3c987590740 =  4:  5baff22f5af Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR
 6:  b57f582ccd3 =  5:  992e325491d Makefile: guard Perl-only variable assignments
 7:  fcdee92f64c =  6:  5e94ef729fe Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"
 8:  1e25b532ca2 =  7:  4eeef8571aa Makefile: adjust Perl-related comments & whitespace
 9:  77d9855bfcf =  8:  8aa35537b08 Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph
 -:  ----------- >  9:  5ff768a5cec Makefile: sort PERL_DEFINES list
10:  6004cdcd8d9 = 10:  fe79797f45e Makefile: create a GIT-PYTHON-DEFINES, like "PERL"
11:  17b30e96057 = 11:  defbd9b6db2 Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts
12:  30ddf7da2c8 <  -:  ----------- Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
13:  f378a7dc35e <  -:  ----------- Makefile: move $(comma), $(empty) and $(space) to shared.mak
14:  13cbb851d32 <  -:  ----------- Makefile: re-add and use the "shellquote" macros
15:  337953e4994 <  -:  ----------- Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...}
16:  5bb597c1993 <  -:  ----------- Makefile: add "$(QUIET)" boilerplate to shared.mak
17:  3c4d0589667 <  -:  ----------- Makefile: use $(wspfx) for $(QUIET...) in shared.mak
18:  be5882b2c99 <  -:  ----------- Makefiles: add and use wildcard "mkdir -p" template
19:  2710f8af6cd <  -:  ----------- Makefile: correct the dependency graph of hook-list.h
20:  59f22a0269a <  -:  ----------- Makefile: use $(file) I/O instead of "FORCE" when possible
21:  dd569a59c74 <  -:  ----------- Makefile: disable GNU make built-in wildcard rules
22:  4168a7e3b30 <  -:  ----------- Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
23:  48a3927d972 <  -:  ----------- Makefile: move ".SUFFIXES" rule to shared.mak
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 01/11] Makefile: clean perl/build/ even with NO_PERL=Y

Fix a regression in 499c29394ce (Makefile: allow building without perl, 2009-04-03) where we'd stop cleaning the perl/* directory because NO_PERL was defined, thus leaving behind litter if the flag at "clean" time didn't match that of build time.

In 499c29394ce this was done to avoid relying on the perl/Makefile.PL, but since my 20d2a30f8ff (Makefile: replace perl/Makefile.PL with simple make rules, 2017-12-10) we can clean things in that directory unconditionally.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to Makefile +1 −1
diff --git a/Makefile b/Makefile
index 9c00a793e47..b21ffe25c2a 100644
--- a/Makefile
+++ b/Makefile
@@ -3280,6 +3280,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json
 	$(RM) -r po/build/
+	$(RM) -r perl/build/
 	$(RM) *.pyc *.pyo */*.pyc */*.pyo $(GENERATED_H) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r .dist-tmp-dir .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz
@@ -3288,7 +3289,6 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) Documentation/GIT-EXCLUDED-PROGRAMS
 ifndef NO_PERL
 	$(MAKE) -C gitweb clean
-	$(RM) -r perl/build/
 endif
 	$(MAKE) -C templates/ clean
 	$(MAKE) -C t/ clean
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 02/11] Makefile: don't set up "perl/build" rules under NO_PERL=Y

Do not define LIB_{PERL,CPAN}{,_GEN} if NO_PERL is defined. This changes no functionality, but makes it clear which of these rules are needed under NO_PERL=Y. See 20d2a30f8ff (Makefile: replace perl/Makefile.PL with simple make rules, 2017-12-10) for the initial implementation.

We do for better or worse rely on "install-doc" calling "install-man-perl" regardless of whether NO_PERL=Y is defined or not, i.e. we'll always end up with that manual page, even if we don't have any of the Perl code installed. Let's add a comment about that adjacent to the rules that build perl/build.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
Show changes to Makefile +9 −8
diff --git a/Makefile b/Makefile
index b21ffe25c2a..d4c31a2718f 100644
--- a/Makefile
+++ b/Makefile
@@ -2731,19 +2731,12 @@ endif
 po/build/locale/%/LC_MESSAGES/git.mo: po/%.po
 	$(QUIET_MSGFMT)mkdir -p $(dir $@) && $(MSGFMT) -o $@ $<
 
+ifndef NO_PERL
 LIB_PERL := $(wildcard perl/Git.pm perl/Git/*.pm perl/Git/*/*.pm perl/Git/*/*/*.pm)
 LIB_PERL_GEN := $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_PERL))
 LIB_CPAN := $(wildcard perl/FromCPAN/*.pm perl/FromCPAN/*/*.pm)
 LIB_CPAN_GEN := $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_CPAN))
 
-ifndef NO_PERL
-all:: $(LIB_PERL_GEN)
-ifndef NO_PERL_CPAN_FALLBACKS
-all:: $(LIB_CPAN_GEN)
-endif
-NO_PERL_CPAN_FALLBACKS_SQ = $(subst ','\'',$(NO_PERL_CPAN_FALLBACKS))
-endif
-
 perl/build/lib/%.pm: perl/%.pm GIT-PERL-DEFINES
 	$(QUIET_GEN)mkdir -p $(dir $@) && \
 	sed -e 's|@@LOCALEDIR@@|$(perl_localedir_SQ)|g' \
@@ -2751,6 +2744,14 @@ perl/build/lib/%.pm: perl/%.pm GIT-PERL-DEFINES
 	    -e 's|@@NO_PERL_CPAN_FALLBACKS@@|$(NO_PERL_CPAN_FALLBACKS_SQ)|g' \
 	< $< > $@
 
+all:: $(LIB_PERL_GEN)
+ifndef NO_PERL_CPAN_FALLBACKS
+all:: $(LIB_CPAN_GEN)
+endif
+NO_PERL_CPAN_FALLBACKS_SQ = $(subst ','\'',$(NO_PERL_CPAN_FALLBACKS))
+endif
+
+# install-man depends on Git.3pm even with NO_PERL=Y
 perl/build/man/man3/Git.3pm: perl/Git.pm
 	$(QUIET_GEN)mkdir -p $(dir $@) && \
 	pod2man $< $@
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 03/11] Makefile: use "=" not ":=" for po/* and perl/*

Change these variable definitions from being simply-expanded to be recursively expanded instead. I.e. they'll be lazily expanded when used.

I added these in 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) and 20d2a30f8ff (Makefile: replace perl/Makefile.PL with simple make rules, 2017-12-10), the reason for using ":=" over "=" was that I didn't know the difference in 2011 (I think I copied some POC code), and in 2017 I used the 2011 commit for reference.

This doesn't make much if any of a practical difference, doing this is cheap either way, but as simply-expanded variables in our Makefile generally indicate special behavior (e.g. making a copy now, and modifying the RHS later) let's change these to show that nothing odd is going on here).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
Show changes to Makefile +8 −8
diff --git a/Makefile b/Makefile
index d4c31a2718f..cad9795ec0a 100644
--- a/Makefile
+++ b/Makefile
@@ -2719,11 +2719,11 @@ po/git.pot: $(GENERATED_H) FORCE
 pot: po/git.pot
 
 ifdef NO_GETTEXT
-POFILES :=
-MOFILES :=
+POFILES =
+MOFILES =
 else
-POFILES := $(wildcard po/*.po)
-MOFILES := $(patsubst po/%.po,po/build/locale/%/LC_MESSAGES/git.mo,$(POFILES))
+POFILES = $(wildcard po/*.po)
+MOFILES = $(patsubst po/%.po,po/build/locale/%/LC_MESSAGES/git.mo,$(POFILES))
 
 all:: $(MOFILES)
 endif
@@ -2732,10 +2732,10 @@ po/build/locale/%/LC_MESSAGES/git.mo: po/%.po
 	$(QUIET_MSGFMT)mkdir -p $(dir $@) && $(MSGFMT) -o $@ $<
 
 ifndef NO_PERL
-LIB_PERL := $(wildcard perl/Git.pm perl/Git/*.pm perl/Git/*/*.pm perl/Git/*/*/*.pm)
-LIB_PERL_GEN := $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_PERL))
-LIB_CPAN := $(wildcard perl/FromCPAN/*.pm perl/FromCPAN/*/*.pm)
-LIB_CPAN_GEN := $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_CPAN))
+LIB_PERL = $(wildcard perl/Git.pm perl/Git/*.pm perl/Git/*/*.pm perl/Git/*/*/*.pm)
+LIB_PERL_GEN = $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_PERL))
+LIB_CPAN = $(wildcard perl/FromCPAN/*.pm perl/FromCPAN/*/*.pm)
+LIB_CPAN_GEN = $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_CPAN))
 
 perl/build/lib/%.pm: perl/%.pm GIT-PERL-DEFINES
 	$(QUIET_GEN)mkdir -p $(dir $@) && \
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 04/11] Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR

Since 7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29) we don't need to guard the clobbering of $@ with this sort of "mv $@+ $@" pattern in these cases where we're merely generating a file that'll be used as a dependency for other files, as in this case for GIT-PERL-HEADER.

We have had a hard dependency on .DELETE_ON_ERROR since 7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29), so this is a pure cleanup as a follow-up to that commit. Support for the ".DELETE_ON_ERROR" target itself is much older than any GNU make version we support, it was added to GNU make in 1994.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Show changes to Makefile +1 −2
diff --git a/Makefile b/Makefile
index cad9795ec0a..bd8cb7c463d 100644
--- a/Makefile
+++ b/Makefile
@@ -2359,8 +2359,7 @@ GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES Makefile
 	    -e 's=@@PERLLIBDIR_REL@@=$(perllibdir_relative_SQ)=g' \
 	    -e 's=@@GITEXECDIR_REL@@=$(gitexecdir_relative_SQ)=g' \
 	    -e 's=@@LOCALEDIR_REL@@=$(localedir_relative_SQ)=g' \
-	    $< >$@+ && \
-	mv $@+ $@
+	    $< >$@
 
 .PHONY: perllibdir
 perllibdir:
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 05/11] Makefile: guard Perl-only variable assignments

Move the "ifndef NO_PERL" a few lines earlier to encompass the "perl_localedir_SQ" variable. We'll only use it under !NO_PERL.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to Makefile +1 −1
diff --git a/Makefile b/Makefile
index bd8cb7c463d..780f222face 100644
--- a/Makefile
+++ b/Makefile
@@ -2301,11 +2301,11 @@ git.res: git.rc GIT-VERSION-FILE GIT-PREFIX
 # This makes sure we depend on the NO_PERL setting itself.
 $(SCRIPT_PERL_GEN): GIT-BUILD-OPTIONS
 
+ifndef NO_PERL
 # Used for substitution in Perl modules. Disabled when using RUNTIME_PREFIX
 # since the locale directory is injected.
 perl_localedir_SQ = $(localedir_SQ)
 
-ifndef NO_PERL
 PERL_HEADER_TEMPLATE = perl/header_templates/fixed_prefix.template.pl
 PERL_DEFINES =
 PERL_DEFINES += $(PERL_PATH_SQ)
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 06/11] Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"

Change the NO_PERL variable assignments so that they declare the much smaller fallback condition first.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
Show changes to Makefile +9 −9
diff --git a/Makefile b/Makefile
index 780f222face..4bb2da36e4e 100644
--- a/Makefile
+++ b/Makefile
@@ -2301,7 +2301,15 @@ git.res: git.rc GIT-VERSION-FILE GIT-PREFIX
 # This makes sure we depend on the NO_PERL setting itself.
 $(SCRIPT_PERL_GEN): GIT-BUILD-OPTIONS
 
-ifndef NO_PERL
+ifdef NO_PERL
+$(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh
+	$(QUIET_GEN) \
+	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
+	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
+	    unimplemented.sh >$@+ && \
+	chmod +x $@+ && \
+	mv $@+ $@
+else # NO_PERL
 # Used for substitution in Perl modules. Disabled when using RUNTIME_PREFIX
 # since the locale directory is injected.
 perl_localedir_SQ = $(localedir_SQ)
@@ -2373,14 +2381,6 @@ git-instaweb: git-instaweb.sh GIT-SCRIPT-DEFINES
 	$(QUIET_GEN)$(cmd_munge_script) && \
 	chmod +x $@+ && \
 	mv $@+ $@
-else # NO_PERL
-$(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh
-	$(QUIET_GEN) \
-	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
-	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
-	    unimplemented.sh >$@+ && \
-	chmod +x $@+ && \
-	mv $@+ $@
 endif # NO_PERL
 
 # This makes sure we depend on the NO_PYTHON setting itself.
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 07/11] Makefile: adjust Perl-related comments & whitespace

Folllow-up my 4070c9e09fc (Makefile: don't re-define PERL_DEFINES, 2021-05-05) and move the rest of the assignments to PERL_DEFINES to one place.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
Show changes to Makefile +6 −9
diff --git a/Makefile b/Makefile
index 4bb2da36e4e..d488a723b79 100644
--- a/Makefile
+++ b/Makefile
@@ -2322,21 +2322,19 @@ PERL_DEFINES += $(perllibdir_SQ)
 PERL_DEFINES += $(RUNTIME_PREFIX)
 PERL_DEFINES += $(NO_PERL_CPAN_FALLBACKS)
 PERL_DEFINES += $(NO_GETTEXT)
+PERL_DEFINES += $(gitexecdir)
+PERL_DEFINES += $(perllibdir)
+PERL_DEFINES += $(localedir)
+PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES))
 
-# Support Perl runtime prefix. In this mode, a different header is installed
-# into Perl scripts.
 ifdef RUNTIME_PREFIX
-
 PERL_HEADER_TEMPLATE = perl/header_templates/runtime_prefix.template.pl
 
-# Don't export a fixed $(localedir) path; it will be resolved by the Perl header
-# at runtime.
+# The RUNTIME_PREFIX header defines $Git::I18N::TEXTDOMAINDIR, so
+# $(perl_localedir_SQ) won't be needed
 perl_localedir_SQ =
-
 endif
 
-PERL_DEFINES += $(gitexecdir) $(perllibdir) $(localedir)
-
 $(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
 	$(QUIET_GEN) \
 	sed -e '1{' \
@@ -2349,7 +2347,6 @@ $(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
 	chmod +x $@+ && \
 	mv $@+ $@
 
-PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES))
 GIT-PERL-DEFINES: FORCE
 	@FLAGS='$(PERL_DEFINES)'; \
 	    if test x"$$FLAGS" != x"`cat $@ 2>/dev/null`" ; then \
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 08/11] Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph

Fix several small issues with the dependency graph of the generated "GIT-PERL-DEFINES" and "GIT-PERL-HEADER" files:

1. Don't have "GIT-PERL-HEADER" depend on the "Makefile". That was a
   lazy way to over-declare the dependencies added in
   f6a0ad4be71 (Makefile: generate Perl header from template file,
   2018-04-10). Let's correct our dependency graph instead.
2. Don't have $(SCRIPT_PERL_GEN) (such as "git-send-email") depend on
   GIT-BUILD-OPTIONS. Let's instead use GIT-PERL-DEFINES.
   The reason for depending on "GIT-BUILD-OPTIONS" was to trigger a
   re-build if NO_PERL=Y was defined. We can instead add that variable
   to "PERL_DEFINES", and have "GIT-PERL-DEFINES" created and updated
   if "NO_PERL=Y" is defined.
3. Due to #2 we'll need to have GIT-PERL-DEFINES be generated even
   under NO_PERL, since that variable will be used by the
   "unimplemented.sh" script.
4. Don't depend on $(gitexecdir), $(perllibdir) and $(localedir),
   instead depend on the $(*_relative_SQ) versions of those.
   The latter is what we'll actually use, while it's unlikely to
   matter in practice (we'd just skip re-building these under
   RUNTIME_PREFIX if the relative path was the same) it makes the code
   easier to read.
   That's because this brings us to a 1=1 mapping of these variables
   and what's subsequently used in the "GIT-PERL-DEFINES",
   "GIT-PERL-HEADER" and "perl/build/lib/%.pm" rules below.
5. We don't need the substitution of " " for ":" added in
   07d90eadb50 (Makefile: add Perl runtime prefix support,
   2018-04-10), let's drop it. This doesn't matter for the correctness
   of these files, because unlike GIT-BUILD-OPTIONS nothing is
   consuming them except the Makefile itself.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 42 +++++++++++++++++++++++++-----------------
 1 file changed, 25 insertions(+), 17 deletions(-)
Show changes to Makefile +25 −17
diff --git a/Makefile b/Makefile
index d488a723b79..356c98dff45 100644
--- a/Makefile
+++ b/Makefile
@@ -2298,10 +2298,14 @@ git.res: git.rc GIT-VERSION-FILE GIT-PREFIX
 	    $(shell echo $(GIT_VERSION) 0 0 0 0 | tr '.a-zA-Z-' ' '))) \
 	  -DGIT_VERSION="\\\"$(GIT_VERSION)\\\"" -i $< -o $@
 
-# This makes sure we depend on the NO_PERL setting itself.
-$(SCRIPT_PERL_GEN): GIT-BUILD-OPTIONS
+# Under NO_PERL=Y we'll still make GIT-PERL-DEFINES. We need to depend
+# on NO_PERL=Y itself for creating "unimplemented.sh" scripts.
+PERL_DEFINES =
+$(SCRIPT_PERL_GEN): GIT-PERL-DEFINES
 
 ifdef NO_PERL
+PERL_DEFINES += $(NO_PERL)
+
 $(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh
 	$(QUIET_GEN) \
 	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
@@ -2310,22 +2314,26 @@ $(SCRIPT_PERL_GEN) git-instaweb: % : unimplemented.sh
 	chmod +x $@+ && \
 	mv $@+ $@
 else # NO_PERL
-# Used for substitution in Perl modules. Disabled when using RUNTIME_PREFIX
-# since the locale directory is injected.
+# The localedir is only used in Perl modules if !NO_GETTEXT
+ifndef NO_GETTEXT
 perl_localedir_SQ = $(localedir_SQ)
+endif
 
 PERL_HEADER_TEMPLATE = perl/header_templates/fixed_prefix.template.pl
-PERL_DEFINES =
+
 PERL_DEFINES += $(PERL_PATH_SQ)
 PERL_DEFINES += $(PERLLIB_EXTRA_SQ)
 PERL_DEFINES += $(perllibdir_SQ)
 PERL_DEFINES += $(RUNTIME_PREFIX)
 PERL_DEFINES += $(NO_PERL_CPAN_FALLBACKS)
 PERL_DEFINES += $(NO_GETTEXT)
-PERL_DEFINES += $(gitexecdir)
-PERL_DEFINES += $(perllibdir)
-PERL_DEFINES += $(localedir)
-PERL_DEFINES := $(subst $(space),:,$(PERL_DEFINES))
+ifdef RUNTIME_PREFIX
+PERL_DEFINES += $(gitexecdir_relative_SQ)
+PERL_DEFINES += $(perllibdir_relative_SQ)
+PERL_DEFINES += $(localedir_relative_SQ)
+else
+PERL_DEFINES += $(perllocaledir_SQ)
+endif
 
 ifdef RUNTIME_PREFIX
 PERL_HEADER_TEMPLATE = perl/header_templates/runtime_prefix.template.pl
@@ -2347,14 +2355,7 @@ $(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
 	chmod +x $@+ && \
 	mv $@+ $@
 
-GIT-PERL-DEFINES: FORCE
-	@FLAGS='$(PERL_DEFINES)'; \
-	    if test x"$$FLAGS" != x"`cat $@ 2>/dev/null`" ; then \
-		echo >&2 "    * new perl-specific parameters"; \
-		echo "$$FLAGS" >$@; \
-	    fi
-
-GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES Makefile
+GIT-PERL-HEADER: $(PERL_HEADER_TEMPLATE) GIT-PERL-DEFINES
 	$(QUIET_GEN) \
 	INSTLIBDIR='$(perllibdir_SQ)' && \
 	INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \
@@ -2380,6 +2381,13 @@ git-instaweb: git-instaweb.sh GIT-SCRIPT-DEFINES
 	mv $@+ $@
 endif # NO_PERL
 
+GIT-PERL-DEFINES: FORCE
+	@FLAGS='$(PERL_DEFINES)'; \
+	    if test x"$$FLAGS" != x"`cat $@ 2>/dev/null`" ; then \
+		echo >&2 "    * new perl-specific parameters"; \
+		echo "$$FLAGS" >$@; \
+	    fi
+
 # This makes sure we depend on the NO_PYTHON setting itself.
 $(SCRIPT_PYTHON_GEN): GIT-BUILD-OPTIONS
 
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 09/11] Makefile: sort PERL_DEFINES list

Sort the PERL_DEFINES list, this will make it easier to maintain it in the future.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
Show changes to Makefile +5 −5
diff --git a/Makefile b/Makefile
index 356c98dff45..3be3dc9621d 100644
--- a/Makefile
+++ b/Makefile
@@ -2321,16 +2321,16 @@ endif
 
 PERL_HEADER_TEMPLATE = perl/header_templates/fixed_prefix.template.pl
 
-PERL_DEFINES += $(PERL_PATH_SQ)
+PERL_DEFINES += $(NO_GETTEXT)
+PERL_DEFINES += $(NO_PERL_CPAN_FALLBACKS)
 PERL_DEFINES += $(PERLLIB_EXTRA_SQ)
-PERL_DEFINES += $(perllibdir_SQ)
+PERL_DEFINES += $(PERL_PATH_SQ)
 PERL_DEFINES += $(RUNTIME_PREFIX)
-PERL_DEFINES += $(NO_PERL_CPAN_FALLBACKS)
-PERL_DEFINES += $(NO_GETTEXT)
+PERL_DEFINES += $(perllibdir_SQ)
 ifdef RUNTIME_PREFIX
 PERL_DEFINES += $(gitexecdir_relative_SQ)
-PERL_DEFINES += $(perllibdir_relative_SQ)
 PERL_DEFINES += $(localedir_relative_SQ)
+PERL_DEFINES += $(perllibdir_relative_SQ)
 else
 PERL_DEFINES += $(perllocaledir_SQ)
 endif
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 10/11] Makefile: create a GIT-PYTHON-DEFINES, like "PERL"

Create a new "GIT-PYTHON-DEFINES" file, and untangle the dependency issues of the Python by copying over the patterns established for building the adjacent Perl code in preceding commits.

As with Perl, there's no real reason to depend on GIT-BUILD-OPTIONS or GIT-CFLAGS when building the Python code, nor did we need GIT-PREFIX. Let's instead add those variables we care about to a "GIT-PYTHON-DEFINES" and depend on that.

This changes code originally added in ca3bcabf118 (auto-detect changed prefix and/or changed build flags, 2006-06-15), and adjusted in 96a4647fca5 (Makefile: detect when PYTHON_PATH changes, 2012-12-18).

The relevant code for the "Perl" targets was then added in 07981dce81e (Makefile: rebuild perl scripts when perl paths change, 2013-11-18), and has been adjusted in preceding commits.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore |  2 +-
 Makefile   | 49 +++++++++++++++++++++++++------------------------
 2 files changed, 26 insertions(+), 25 deletions(-)
Show changes to 2 files +26 −25

.gitignore, Makefile

diff --git a/.gitignore b/.gitignore
index 054249b20a8..845e5d0c355 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,7 +8,7 @@
 /GIT-PREFIX
 /GIT-PERL-DEFINES
 /GIT-PERL-HEADER
-/GIT-PYTHON-VARS
+/GIT-PYTHON-DEFINES
 /GIT-SCRIPT-DEFINES
 /GIT-USER-AGENT
 /GIT-VERSION-FILE
diff --git a/Makefile b/Makefile
index 3be3dc9621d..f2cee13ea5d 100644
--- a/Makefile
+++ b/Makefile
@@ -2388,18 +2388,15 @@ GIT-PERL-DEFINES: FORCE
 		echo "$$FLAGS" >$@; \
 	    fi
 
-# This makes sure we depend on the NO_PYTHON setting itself.
-$(SCRIPT_PYTHON_GEN): GIT-BUILD-OPTIONS
+# As with NO_PERL=Y we'll still make GIT-PYTHON-DEFINES if "NO_PYTHON"
+# is defined, for creating the "unimplemented.sh" scripts.
+PYTHON_DEFINES =
+$(SCRIPT_PYTHON_GEN): GIT-PYTHON-DEFINES
+
+ifdef NO_PYTHON
+PYTHON_DEFINES += $(SHELL_PATH_SQ)
+PYTHON_DEFINES += $(NO_PYTHON)
 
-ifndef NO_PYTHON
-$(SCRIPT_PYTHON_GEN): GIT-CFLAGS GIT-PREFIX GIT-PYTHON-VARS
-$(SCRIPT_PYTHON_GEN): % : %.py
-	$(QUIET_GEN) \
-	sed -e '1s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
-	    $< >$@+ && \
-	chmod +x $@+ && \
-	mv $@+ $@
-else # NO_PYTHON
 $(SCRIPT_PYTHON_GEN): % : unimplemented.sh
 	$(QUIET_GEN) \
 	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
@@ -2407,8 +2404,24 @@ $(SCRIPT_PYTHON_GEN): % : unimplemented.sh
 	    unimplemented.sh >$@+ && \
 	chmod +x $@+ && \
 	mv $@+ $@
+else # NO_PYTHON
+PYTHON_DEFINES += $(PYTHON_PATH_SQ)
+
+$(SCRIPT_PYTHON_GEN): % : %.py GIT-PYTHON-DEFINES
+	$(QUIET_GEN) \
+	sed -e '1s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
+	    $< >$@+ && \
+	chmod +x $@+ && \
+	mv $@+ $@
 endif # NO_PYTHON
 
+GIT-PYTHON-DEFINES: FORCE
+	@FLAGS='$(PYTHON_DEFINES)'; \
+	    if test x"$$FLAGS" != x"`cat $@ 2>/dev/null`" ; then \
+		echo >&2 "    * new python-specific parameters"; \
+		echo "$$FLAGS" >$@; \
+	    fi
+
 CONFIGURE_RECIPE = sed -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
 			configure.ac >configure.ac+ && \
 		   autoconf -o configure configure.ac+ && \
@@ -2895,18 +2908,6 @@ else
 endif
 	@if cmp $@+ $@ >/dev/null 2>&1; then $(RM) $@+; else mv $@+ $@; fi
 
-### Detect Python interpreter path changes
-ifndef NO_PYTHON
-TRACK_PYTHON = $(subst ','\'',-DPYTHON_PATH='$(PYTHON_PATH_SQ)')
-
-GIT-PYTHON-VARS: FORCE
-	@VARS='$(TRACK_PYTHON)'; \
-	    if test x"$$VARS" != x"`cat $@ 2>/dev/null`" ; then \
-		echo >&2 "    * new Python interpreter location"; \
-		echo "$$VARS" >$@; \
-            fi
-endif
-
 test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X))
 
 all:: $(TEST_PROGRAMS) $(test_bindir_programs)
@@ -3303,7 +3304,7 @@ ifndef NO_TCLTK
 endif
 	$(RM) GIT-VERSION-FILE GIT-CFLAGS GIT-LDFLAGS GIT-BUILD-OPTIONS
 	$(RM) GIT-USER-AGENT GIT-PREFIX
-	$(RM) GIT-SCRIPT-DEFINES GIT-PERL-DEFINES GIT-PERL-HEADER GIT-PYTHON-VARS
+	$(RM) GIT-SCRIPT-DEFINES GIT-PERL-DEFINES GIT-PERL-HEADER GIT-PYTHON-DEFINES
 ifdef MSVC
 	$(RM) $(patsubst %.o,%.o.pdb,$(OBJECTS))
 	$(RM) $(patsubst %.exe,%.pdb,$(OTHER_PROGRAMS))
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Dec 17, 2021, 00:26 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH 11/11] Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts

Change the hardcoding of @@GIT_VERSION@@ in generated *.perl scripts to instead shell out to "git version". This means that we can stop re-building during development every time the HEAD changes.

These codepaths are not "hot", so shelling out to get the version shouldn't matter to users, in the one case where it potentially would in send-email (the loop for each E-Mail we send) we now cache the value, so we'll only retrieve it once.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile            | 2 +-
 git-cvsserver.perl  | 6 +++---
 git-send-email.perl | 7 ++-----
 git-svn.perl        | 2 +-
 4 files changed, 7 insertions(+), 10 deletions(-)
Show changes to 4 files +7 −10

Makefile, git-cvsserver.perl, git-send-email.perl, git-svn.perl

diff --git a/Makefile b/Makefile
index f2cee13ea5d..9ccfb1109f5 100644
--- a/Makefile
+++ b/Makefile
@@ -2343,7 +2343,7 @@ PERL_HEADER_TEMPLATE = perl/header_templates/runtime_prefix.template.pl
 perl_localedir_SQ =
 endif
 
-$(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
+$(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER
 	$(QUIET_GEN) \
 	sed -e '1{' \
 	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 4c8118010a8..4ec0cc1cc3c 100755
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -26,8 +26,6 @@
 use File::Basename;
 use Getopt::Long qw(:config require_order no_ignore_case);
 
-my $VERSION = '@@GIT_VERSION@@';
-
 my $log = GITCVS::log->new();
 my $cfg;
 
@@ -126,7 +124,9 @@
     or die $usage;
 
 if ($state->{version}) {
-    print "git-cvsserver version $VERSION\n";
+    my $version = qx[git version];
+    $version =~ s/^(git)\b/$1-cvsserver/;
+    print $version;
     exit;
 }
 if ($state->{help}) {
diff --git a/git-send-email.perl b/git-send-email.perl
index 04087221aa7..390e8512627 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1504,6 +1504,7 @@ sub file_name_is_absolute {
 #
 # If an error occurs sending the email, this just dies.
 
+my $gitversion;
 sub send_message {
 	my @recipients = unique_email_list(@to);
 	@cc = (grep { my $cc = extract_valid_address_or_die($_);
@@ -1514,11 +1515,6 @@ sub send_message {
 	@recipients = unique_email_list(@recipients,@cc,@initial_bcc);
 	@recipients = (map { extract_valid_address_or_die($_) } @recipients);
 	my $date = format_2822_time($time++);
-	my $gitversion = '@@GIT_VERSION@@';
-	if ($gitversion =~ m/..GIT_VERSION../) {
-	    $gitversion = Git::version();
-	}
-
 	my $cc = join(",\n\t", unique_email_list(@cc));
 	my $ccline = "";
 	if ($cc ne '') {
@@ -1533,6 +1529,7 @@ sub send_message {
 Message-Id: $message_id
 ";
 	if ($use_xmailer) {
+		$gitversion ||= Git::version();
 		$header .= "X-Mailer: git-send-email $gitversion\n";
 	}
 	if ($in_reply_to) {
diff --git a/git-svn.perl b/git-svn.perl
index be987e316f9..727431412be 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -9,7 +9,6 @@
 		$_revision $_repository
 		$_q $_authors $_authors_prog %users/;
 $AUTHOR = 'Eric Wong <normalperson@yhbt.net>';
-$VERSION = '@@GIT_VERSION@@';
 
 use Carp qw/croak/;
 use File::Basename qw/dirname basename/;
@@ -47,6 +46,7 @@
 	command_close_bidi_pipe
 	get_record
 );
+$VERSION = Git::version();
 
 BEGIN {
 	Memoize::memoize 'Git::config';
-- 
2.34.1.1119.g7a3fc8778ee
Ævar Arnfjörð Bjarmason· Feb 21, 2022, 20:14 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH 00/11] Makefile: fix "Perl" and "Python" rules, speed up re-builds

On Fri, Dec 17 2021, Ævar Arnfjörð Bjarmason wrote:

A gentle reminder about considering picking up this topic that got lost around the holidays.

This thread doesn't have any reviews, but as the linked-to range-diff shows it's all had reviews already as part of the larger topic I split it up from (except for the trivial addition noted below).

Show 63 quoted lines
> This topic is split-up from v4 of ab/make-dependency[1]. Range-diff to
> it below. As noted in [2] I opted to split up that series rather than
> having it grow too large.
>
> This is miscellanious cleanups related to the "Perl" rules, and some
> fixes for bad dependency rules in 08/11. Then in 10/11 we make the
> "Python" rules use the new template we created for "Perl",
>
> We are then able to speed up "make" re-builds a bit in 11/11, as we
> don't need to inject @@GIT_VERSION@@ into the Perl scripts anymore.
>
> The 09/11 is new. It's a trivial sorting of the PERL_DEFINES list,
> which'll help to maintain it in the future.
>
> 1. https://lore.kernel.org/git/cover-v4-00.23-00000000000-20211117T101807Z-avarab@gmail.com/
> 2. https://lore.kernel.org/git/211217.86a6h09km9.gmgdl@evledraar.gmail.com/
>
>
> Ævar Arnfjörð Bjarmason (11):
>   Makefile: clean perl/build/ even with NO_PERL=Y
>   Makefile: don't set up "perl/build" rules under NO_PERL=Y
>   Makefile: use "=" not ":=" for po/* and perl/*
>   Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR
>   Makefile: guard Perl-only variable assignments
>   Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"
>   Makefile: adjust Perl-related comments & whitespace
>   Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph
>   Makefile: sort PERL_DEFINES list
>   Makefile: create a GIT-PYTHON-DEFINES, like "PERL"
>   Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts
>
>  .gitignore          |   2 +-
>  Makefile            | 160 +++++++++++++++++++++++---------------------
>  git-cvsserver.perl  |   6 +-
>  git-send-email.perl |   7 +-
>  git-svn.perl        |   2 +-
>  5 files changed, 90 insertions(+), 87 deletions(-)
>
> Range-diff:
>  1:  1621ca72c1d <  -:  ----------- Makefile: don't invoke msgfmt with --statistics
>  4:  daead5ec293 =  1:  185f5a9a9cf Makefile: clean perl/build/ even with NO_PERL=Y
>  2:  b7c36c9fea0 =  2:  37eeccc5646 Makefile: don't set up "perl/build" rules under NO_PERL=Y
>  3:  29b000eb0f1 =  3:  572a82c71a1 Makefile: use "=" not ":=" for po/* and perl/*
>  5:  3c987590740 =  4:  5baff22f5af Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR
>  6:  b57f582ccd3 =  5:  992e325491d Makefile: guard Perl-only variable assignments
>  7:  fcdee92f64c =  6:  5e94ef729fe Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL"
>  8:  1e25b532ca2 =  7:  4eeef8571aa Makefile: adjust Perl-related comments & whitespace
>  9:  77d9855bfcf =  8:  8aa35537b08 Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph
>  -:  ----------- >  9:  5ff768a5cec Makefile: sort PERL_DEFINES list
> 10:  6004cdcd8d9 = 10:  fe79797f45e Makefile: create a GIT-PYTHON-DEFINES, like "PERL"
> 11:  17b30e96057 = 11:  defbd9b6db2 Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts
> 12:  30ddf7da2c8 <  -:  ----------- Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
> 13:  f378a7dc35e <  -:  ----------- Makefile: move $(comma), $(empty) and $(space) to shared.mak
> 14:  13cbb851d32 <  -:  ----------- Makefile: re-add and use the "shellquote" macros
> 15:  337953e4994 <  -:  ----------- Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...}
> 16:  5bb597c1993 <  -:  ----------- Makefile: add "$(QUIET)" boilerplate to shared.mak
> 17:  3c4d0589667 <  -:  ----------- Makefile: use $(wspfx) for $(QUIET...) in shared.mak
> 18:  be5882b2c99 <  -:  ----------- Makefiles: add and use wildcard "mkdir -p" template
> 19:  2710f8af6cd <  -:  ----------- Makefile: correct the dependency graph of hook-list.h
> 20:  59f22a0269a <  -:  ----------- Makefile: use $(file) I/O instead of "FORCE" when possible
> 21:  dd569a59c74 <  -:  ----------- Makefile: disable GNU make built-in wildcard rules
> 22:  4168a7e3b30 <  -:  ----------- Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
> 23:  48a3927d972 <  -:  ----------- Makefile: move ".SUFFIXES" rule to shared.mak

← back to recent threads