using iwyu (include-what-you-use) to analyze includes (was: [PATCH] bisect.c: remove unused includes)

On Thu, Mar 31 2022, Junio C Hamano wrote:

[Changed $subject to make this easier to find]

Also: Some built-ins don't include builtin.h as they should, a fix (or
even basic CI check) for that would be most welcome.

	git grep -C2 -n -F -e builtin.h -e cache.h -e git-compat-util.h -- builtin

I.e. we have this saying a lot of those are redundant:
	
	Documentation/CodingGuidelines- - The first #include in C files, except in platform specific compat/
	Documentation/CodingGuidelines-   implementations, must be either "git-compat-util.h", "cache.h" or
	Documentation/CodingGuidelines:   "builtin.h".  You do not have to include more than one of these.

But maybe it's not worth it, anyway...

... For anyone interested in pursuing this, I think using the excellent include-what-you-use tool would be a nice start.

We could even eventually add it to our CI if the false positive rate isn't bad (I haven't checked much): https://github.com/include-what-you-use/include-what-you-use

E.g. in this case (I manually omitted the rest of the output, there's probably a iwyu option to omit it, but I didn't see how do that from skimming the docs):

	$ sudo apt install iwyu # YMMV
	$ make bisect.o CC=include-what-you-use CFLAGS="-Xiwyu --verbose=1" 2>&1 | grep -v -E -e '^#include <' -e '^#include "(cache|git-compat-util|gettext)\.h"'
	    CC bisect.o
	
	(bisect.h has correct #includes/fwd-decls)
	
	bisect.c should add these lines:
	#include "hash.h"              // for oideq, object_id, oidcmp, oidcpy, GIT_...
	#include "object.h"            // for object, repo_clear_commit_marks
	#include "path.h"              // for GIT_PATH_FUNC, git_pathdup
	#include "pretty.h"            // for CMIT_FMT_UNSPECIFIED, format_commit_me...
	#include "repository.h"        // for repository (ptr only), the_repository
	#include "strbuf.h"            // for strbuf_release, strbuf, strbuf_getline_lf
	#include "string-list.h"       // for string_list_append, string_list_clear
	
	bisect.c should remove these lines:
	- #include "hash-lookup.h"  // lines 9-9
	- struct commit_weight;  // lines 76-76

Then if I patch it as:
	
	diff --git a/bisect.c b/bisect.c
	index 9e6a2b7f201..512430e3cc8 100644
	--- a/bisect.c
	+++ b/bisect.c
	@@ -6,7 +6,6 @@
	 #include "refs.h"
	 #include "list-objects.h"
	 #include "quote.h"
	-#include "hash-lookup.h"
	 #include "run-command.h"
	 #include "log-tree.h"
	 #include "bisect.h"
	@@ -16,6 +15,13 @@
	 #include "commit-reach.h"
	 #include "object-store.h"
	 #include "dir.h"
	+#include "hash.h"
	+#include "object.h"
	+#include "path.h"
	+#include "pretty.h"
	+#include "repository.h"
	+#include "strbuf.h"
	+#include "string-list.h"
	 
	 static struct oid_array good_revs;
	 static struct oid_array skipped_revs;

It's happier, but probably needs to be told to ignore define_commit_slab() somehow:

	$ make bisect.o CC=include-what-you-use CFLAGS="-Xiwyu --verbose=1" 2>&1 | grep -v -E -e '^#include <' -e '^#include "(cache|git-compat-util|gettext)\.h"'
	    CC bisect.o
	
	(bisect.h has correct #includes/fwd-decls)
	
	bisect.c should add these lines:
	
	bisect.c should remove these lines:
	- struct commit_weight;  // lines 82-82

That still needs to be massaged a bit, e.g. we should probably omit hash.h and anything else in cache.h and git-compat-util.h.

Or maybe not & we should make those headers even lighter. It is rather annoying that changing some of those things leads to a complete re-build, but there's a trade-off there where we probably want things like gettext.h and other used-almost-everywhere headers in included by those.

So take all the above with a huge grain of salt. I haven't used iwyu much, but it seems to be something that'll help us go in the direction Junio noted above.

I think starting with:

	make -k git-objs <the CC etc. params above>

And tackling the "should remove these lines" issues first would be a
good start, e.g. for serve.c it says:
	
	serve.c should remove these lines:
	- #include "cache.h"  // lines 1-1
	- #include "strvec.h"  // lines 6-6

We don't want that first one, but it's right about the second one. It's been orphaned since f0a35c9ce52 (serve: drop "keys" strvec, 2021-09-15), I skimmed some of the rist and they all seem like good suggestions. E.g. lockfile.h for builtin/apply.c, which isn't needed since 6d058c88264 (apply: move lockfile into `apply_state`, 2017-10-05).