git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2] Makefile: make the "sparse" target non-.PHONY

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Sep 23, 2021, 00:07 UTC
Message-ID
<patch-v2-1.1-059829f2195-20210923T000654Z-avarab@gmail.com>
In-Reply-To
<cover-0.3-00000000000-20210921T224944Z-avarab@gmail.com>

Change the "sparse" target and its *.sp dependencies to be non-.PHONY. Before this change "make sparse" would take ~5s to re-run all the *.c files through "cgcc", after it it'll create an empty *.sp file sitting alongside the *.c file, only if the *.c file or its dependencies are newer than the *.sp is the *.sp re-made.

We ensure that the recursive dependencies are correct by depending on the *.o file, which in turn will have correct dependencies by either depending on all header files, or under "COMPUTE_HEADER_DEPENDENCIES=yes" the headers it needs.

This means that a plain "make sparse" is much slower, as we'll now need to make the *.o files just to create the *.sp files, but incrementally creating the *.sp files is *much* faster and less verbose, it thus becomes viable to run "sparse" along with "all" as e.g. "git rebase --exec 'make all sparse'".

On my box with -j8 "make sparse" was fast before, or around 5 seconds, now it only takes that long the first time, and the common case is <100ms, or however long it takes GNU make to stat the *.sp file and see that all the corresponding *.c file and its dependencies are older.

See 0bcd9ae85d7 (sparse: Fix errors due to missing target-specific variables, 2011-04-21) for the modern implementation of the sparse target being changed here.

It is critical that we use -Wsparse-error here, otherwise the error would only show up once, but we'd successfully create the empty *.sp file, and running a second time wouldn't show the error. I'm therefore not putting it into SPARSE_FLAGS or SP_EXTRA_FLAGS, it's not optional, the Makefile logic won't behave properly without it.

Appending to $@ without a move is OK here because we're using the .DELETE_ON_ERROR Makefile feature. See 7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29). GNU make ensures that on error this file will be removed.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
Range-diff against v1:
1:  e03fde1b642 ! 1:  059829f2195 Makefile: make the "sparse" target non-.PHONY
    @@ Commit message
         Makefile: make the "sparse" target non-.PHONY
     
         Change the "sparse" target and its *.sp dependencies to be
    -    non-.PHONY. It's now viable to run it as part of a normal compilation
    -    target, as we'll only re-generate these checks if the source *.c file
    -    has changed.
    +    non-.PHONY. Before this change "make sparse" would take ~5s to re-run
    +    all the *.c files through "cgcc", after it it'll create an empty *.sp
    +    file sitting alongside the *.c file, only if the *.c file or its
    +    dependencies are newer than the *.sp is the *.sp re-made.
     
    -    On my box with -j8 it was fast before, or around 5 seconds, now it
    -    only takes that long the first time, and the common case is <100ms, or
    -    however long it takes GNU make to stat the *.sp file and see that all
    -    the corresponding *.c files are older.
    +    We ensure that the recursive dependencies are correct by depending on
    +    the *.o file, which in turn will have correct dependencies by either
    +    depending on all header files, or under
    +    "COMPUTE_HEADER_DEPENDENCIES=yes" the headers it needs.
    +
    +    This means that a plain "make sparse" is much slower, as we'll now
    +    need to make the *.o files just to create the *.sp files, but
    +    incrementally creating the *.sp files is *much* faster and less
    +    verbose, it thus becomes viable to run "sparse" along with "all" as
    +    e.g. "git rebase --exec 'make all sparse'".
    +
    +    On my box with -j8 "make sparse" was fast before, or around 5 seconds,
    +    now it only takes that long the first time, and the common case is
    +    <100ms, or however long it takes GNU make to stat the *.sp file and
    +    see that all the corresponding *.c file and its dependencies are
    +    older.
     
         See 0bcd9ae85d7 (sparse: Fix errors due to missing target-specific
         variables, 2011-04-21) for the modern implementation of the sparse
    @@ Makefile: check-sha1:: t/helper/test-tool$X
      SP_OBJ = $(patsubst %.o,%.sp,$(C_OBJ))
      
     -$(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
    -+$(SP_OBJ): %.sp: %.c GIT-CFLAGS
    ++$(SP_OBJ): %.sp: %.c %.o GIT-CFLAGS
      	$(QUIET_SP)cgcc -no-compile $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) \
     -		$(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $<
     +		-Wsparse-error \
    @@ Makefile: check-sha1:: t/helper/test-tool$X
      sparse: $(SP_OBJ)
      
      EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
    +@@ Makefile: clean: profile-clean coverage-clean cocciclean
    + 	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
    + 	$(RM) $(TEST_PROGRAMS)
    + 	$(RM) $(FUZZ_PROGRAMS)
    ++	$(RM) $(SP_OBJ)
    + 	$(RM) $(HCC)
    + 	$(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json
    + 	$(RM) -r po/build/
2:  9c4cedacdce < -:  ----------- Makefile: do one append in %.hcc rule
3:  e2ad1700f9e < -:  ----------- Makefile: make the "hdr-check" target non-.PHONY
 .gitignore | 1 +
 Makefile   | 9 ++++++---
 2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/.gitignore b/.gitignore
index 311841f9bed..b02250a50c4 100644
--- a/.gitignore
+++ b/.gitignore
@@ -224,6 +224,7 @@
 *.lib
 *.res
 *.sln
+*.sp
 *.suo
 *.ncb
 *.vcproj
diff --git a/Makefile b/Makefile
index a9f9b689f0c..0bae65889fe 100644
--- a/Makefile
+++ b/Makefile
@@ -2896,11 +2896,13 @@ check-sha1:: t/helper/test-tool$X
 
 SP_OBJ = $(patsubst %.o,%.sp,$(C_OBJ))
 
-$(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
+$(SP_OBJ): %.sp: %.c %.o GIT-CFLAGS
 	$(QUIET_SP)cgcc -no-compile $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) \
-		$(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $<
+		-Wsparse-error \
+		$(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $< && \
+	>$@
 
-.PHONY: sparse $(SP_OBJ)
+.PHONY: sparse
 sparse: $(SP_OBJ)
 
 EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
@@ -3227,6 +3229,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
 	$(RM) $(TEST_PROGRAMS)
 	$(RM) $(FUZZ_PROGRAMS)
+	$(RM) $(SP_OBJ)
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json
 	$(RM) -r po/build/
-- 
2.33.0.1225.g9f062250122
Previous: Junio C HamanoNext: Jeff King
Message 15 of 28 in “Makefile: make "sparse" and "hdr-check" non-.PHONY”
  1. 0/3 Makefile: make "sparse" and "hdr-check" non-.PHONYÆvar Arnfjörð Bjarmason, Sep 21, 2021
  2. 1/3 Makefile: make the "sparse" target non-.PHONYÆvar Arnfjörð Bjarmason, Sep 21, 2021
  3. Jeff KingSep 22, 2021
  4. 2/3 Makefile: do one append in %.hcc ruleÆvar Arnfjörð Bjarmason, Sep 21, 2021
  5. 3/3 Makefile: make the "hdr-check" target non-.PHONYÆvar Arnfjörð Bjarmason, Sep 21, 2021
  6. Jeff KingSep 22, 2021
  7. Ramsay JonesSep 22, 2021
  8. Jeff KingSep 22, 2021
  9. Ramsay JonesSep 22, 2021
  10. Junio C HamanoSep 22, 2021
  11. Ævar Arnfjörð BjarmasonSep 23, 2021
  12. Junio C HamanoSep 23, 2021
  13. Ævar Arnfjörð BjarmasonSep 23, 2021
  14. Junio C HamanoSep 22, 2021
  15. Makefile: make the "sparse" target non-.PHONYÆvar Arnfjörð Bjarmason, Sep 23, 2021
  16. Jeff KingSep 23, 2021
  17. Ævar Arnfjörð BjarmasonSep 23, 2021
  18. Jeff KingSep 23, 2021
  19. Junio C HamanoSep 23, 2021
  20. Ramsay JonesSep 23, 2021
  21. Ævar Arnfjörð BjarmasonSep 24, 2021
  22. Ramsay JonesSep 24, 2021
  23. Ævar Arnfjörð BjarmasonSep 24, 2021
  24. Junio C HamanoSep 24, 2021
  25. Makefile: add a non-.PHONY "sparse-incr" targetÆvar Arnfjörð Bjarmason, Sep 28, 2021
  26. Makefile: add a non-.PHONY "sparse-incr" targetÆvar Arnfjörð Bjarmason, Sep 28, 2021
  27. Junio C HamanoSep 28, 2021
  28. Ævar Arnfjörð BjarmasonSep 28, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.