git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Makefile: add support for generating JSON compilation database

From
Philippe Blain via GitGitGadget <gitgitgadget@gmail.com>
Date
Aug 30, 2020, 19:28 UTC
Message-ID
<pull.714.git.1598815707540.gitgitgadget@gmail.com>
From: Philippe Blain <levraiphilippeblain@gmail.com>

Tools based on LibClang [1] can make use of a 'JSON Compilation Database' [2] that keeps track of the exact options used to compile a set of source files.

The Clang compiler can generate JSON fragments when compiling [3], using the `-MJ` flag. These JSON fragments (one per compiled source file) can then be concatenated to create the compilation database, commonly called 'compile_commands.json'.

Add support to the Makefile for generating these JSON fragments as well as the compilation database itself, if the environment variable 'GENERATE_COMPILATION_DATABASE' is set.

If this variable is set, check that $(CC) indeed supports the `-MJ` flag, following what is done for automatic dependencies.

All JSON fragments are placed in the 'compile_commands/' directory, and the compilation database 'compile_commands.json' is generated as a dependency of the 'all' target using a `sed` invocation.

[1] https://clang.llvm.org/docs/Tooling.html [2] https://clang.llvm.org/docs/JSONCompilationDatabase.html [3] https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-mj-arg

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
---
    Add support for generating JSON compilation database
    
    I don't have a lot of knowledge of Make double-colon rules, or insight
    into why they are used for the 'all' target, but I think the approach I
    chose makes sense. In particular, I do not list any prerequisite for the
    'compile_commands.json' file, but from what I tested it is still rebuilt
    anytime the 'all' target is rebuilt, which is what we want.
    
    Note: CMakeLists.txt in contrib/buildsystems does not need to be updated
    to also support this feature because CMake supports it out-of-the-box
    [1].
    
    [1] 
    https://cmake.org/cmake/help/latest/variable/CMAKE_EXPORT_COMPILE_COMMANDS.html
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-714%2Fphil-blain%2Fcompiledb-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-714/phil-blain/compiledb-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/714
 .gitignore |  2 ++
 Makefile   | 52 +++++++++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 49 insertions(+), 5 deletions(-)
diff --git a/.gitignore b/.gitignore
index ee509a2ad2..f4c51300e0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -197,6 +197,7 @@
 /git.spec
 *.exe
 *.[aos]
+*.o.json
 *.py[co]
 .depend/
 *.gcda
@@ -218,6 +219,7 @@
 /tags
 /TAGS
 /cscope*
+/compile_commands.json
 *.hcc
 *.obj
 *.lib
diff --git a/Makefile b/Makefile
index 65f8cfb236..954bd2aa47 100644
--- a/Makefile
+++ b/Makefile
@@ -462,6 +462,12 @@ all::
 # the global variable _wpgmptr containing the absolute path of the current
 # executable (this is the case on Windows).
 #
+# Define GENERATE_COMPILATION_DATABASE to generate JSON compilation database
+# entries during compilation if your compiler supports it, using the `-MJ` flag.
+# The JSON entries will be placed in the `compile_commands/` directory,
+# and the JSON compilation database can be created afterwards with
+# `make compile_commands.json`.
+#
 # Define DEVELOPER to enable more compiler warnings. Compiler version
 # and family are auto detected, but could be overridden by defining
 # COMPILER_FEATURES (see config.mak.dev). You can still set
@@ -1258,6 +1264,20 @@ $(error please set COMPUTE_HEADER_DEPENDENCIES to yes, no, or auto \
 endif
 endif
 
+ifdef GENERATE_COMPILATION_DATABASE
+compdb_check = $(shell $(CC) $(ALL_CFLAGS) \
+	-c -MJ /dev/null \
+	-x c /dev/null -o /dev/null 2>&1; \
+	echo $$?)
+ifeq ($(compdb_check),0)
+override GENERATE_COMPILATION_DATABASE = yes
+else
+override GENERATE_COMPILATION_DATABASE = no
+$(warning GENERATE_COMPILATION_DATABASE is set, but your compiler does not \
+support generating compilation database entries)
+endif
+endif
+
 ifdef SANE_TOOL_PATH
 SANE_TOOL_PATH_SQ = $(subst ','\'',$(SANE_TOOL_PATH))
 BROKEN_PATH_FIX = 's|^\# @@BROKEN_PATH_FIX@@$$|git_broken_path_fix "$(SANE_TOOL_PATH_SQ)"|'
@@ -2381,16 +2401,30 @@ missing_dep_dirs =
 dep_args =
 endif
 
+compdb_dir = compile_commands/
+
+ifeq ($(GENERATE_COMPILATION_DATABASE),yes)
+missing_compdb_dir = $(compdb_dir)
+$(missing_compdb_dir):
+	@mkdir -p $@
+
+compdb_file = $(compdb_dir)$(subst .-,,$(subst /,-,$(dir $@)))$(notdir $@).json
+compdb_args = -MJ $(compdb_file)
+else
+missing_compdb_dir =
+compdb_args =
+endif
+
 ASM_SRC := $(wildcard $(OBJECTS:o=S))
 ASM_OBJ := $(ASM_SRC:S=o)
 C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))
 
 .SUFFIXES:
 
-$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs)
-	$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
-$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs)
-	$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
+$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
+	$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
+$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
+	$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
 
 %.s: %.c GIT-CFLAGS FORCE
 	$(QUIET_CC)$(CC) -o $@ -S $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
@@ -2413,6 +2447,14 @@ else
 $(OBJECTS): $(LIB_H) $(GENERATED_H)
 endif
 
+ifeq ($(GENERATE_COMPILATION_DATABASE),yes)
+all:: compile_commands.json
+compile_commands.json:
+	@$(RM) $@
+	$(QUIET_GEN)sed -e '1s/^/[/' -e '$$s/,$$/]/' $(compdb_dir)*.o.json > $@+
+	@if test -s $@+; then mv $@+ $@; else $(RM) $@+; fi
+endif
+
 exec-cmd.sp exec-cmd.s exec-cmd.o: GIT-PREFIX
 exec-cmd.sp exec-cmd.s exec-cmd.o: EXTRA_CPPFLAGS = \
 	'-DGIT_EXEC_PATH="$(gitexecdir_SQ)"' \
@@ -3117,7 +3159,7 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(TEST_PROGRAMS)
 	$(RM) $(FUZZ_PROGRAMS)
 	$(RM) $(HCC)
-	$(RM) -r bin-wrappers $(dep_dirs)
+	$(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json
 	$(RM) -r po/build/
 	$(RM) *.pyc *.pyo */*.pyc */*.pyo $(GENERATED_H) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir

base-commit: d9cd4331470f4d9d78677f12dc79063dab832f53

-- 
gitgitgadget
Next: brian m. carlson
Message 1 of 15 in “Makefile: add support for generating JSON compilation database”
  1. Makefile: add support for generating JSON compilation databasePhilippe Blain via GitGitGadget, Aug 30, 2020
  2. brian m. carlsonAug 30, 2020
  3. Philippe BlainAug 31, 2020
  4. Junio C HamanoAug 31, 2020
  5. Jeff KingSep 1, 2020
  6. Philippe BlainSep 1, 2020
  7. brian m. carlsonSep 2, 2020
  8. Jeff KingSep 2, 2020
  9. Philippe BlainAug 30, 2020
  10. Makefile: add support for generating JSON compilation databasePhilippe Blain via GitGitGadget, Sep 1, 2020
  11. Junio C HamanoSep 2, 2020
  12. Philippe BlainSep 3, 2020
  13. Junio C HamanoSep 3, 2020
  14. Philippe BlainSep 3, 2020
  15. Makefile: add support for generating JSON compilation databasePhilippe Blain via GitGitGadget, Sep 3, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.