threads / rfc / 29526

RFC patchFix build problems related to profile-directed optimization

Subject: [PATCH, RFC] Fix build problems related to profile-directed optimization

## tl;dr

18 messages between Feb 2, 2012 and Feb 9, 2012. Diffs are folded; open one to read it.

replies: 17people: 5as markdown or json

Theodore Ts'o· Feb 2, 2012, 19:03 UTC · lore

There was a number of problems I ran into when trying the profile-directed optimizations added by Andi Kleen in git commit 7ddc2710b9. (This was using gcc 4.4 found on many enterprise distros.)

1) The -fprofile-generate and -fprofile-use commands are incompatible
with ccache; the code ends up looking in the wrong place for the gcda
files based on the ccache object names.

2) If the makefile notices that CFLAGS are different, it will rebuild
all of the binaries.  Hence the recipe originally specified by the
INSTALL file ("make profile-all" followed by "make install") doesn't
work.  It will appear to work, but the binaries will end up getting
built with no optimization.

This patch fixes this by using an explicit set of options passed via PROFILE_GEN and PROFILE_USE and then using these to directly manipulate CFLAGS and EXTLIBS.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andi Kleen <ak@linux.intel.com>
---
 INSTALL  |    4 ++--
 Makefile |   31 +++++++++++++++++++++++--------
 2 files changed, 25 insertions(+), 10 deletions(-)

Show changes to 2 files +25 −10

INSTALL, Makefile

diff --git a/INSTALL b/INSTALL
index 6fa83fe..978ed09 100644
--- a/INSTALL
+++ b/INSTALL
@@ -28,8 +28,8 @@ set up install paths (via config.mak.autogen), so you can write instead
 If you're willing to trade off (much) longer build time for a later
 faster git you can also do a profile feedback build with
 
-	$ make profile-all
-	# make prefix=... install
+	$ make prefix=... profile-all
+	# make prefix=... PROFILE_USE=t install
 
 This will run the complete test suite as training workload and then
 rebuild git with the generated profile feedback. This results in a git
diff --git a/Makefile b/Makefile
index c457c34..15d1df4 100644
--- a/Makefile
+++ b/Makefile
@@ -1772,6 +1772,24 @@ ifdef ASCIIDOC7
 	export ASCIIDOC7
 endif
 
+### profile feedback build
+#
+
+# Can adjust this to be a global directory if you want to do extended
+# data gathering
+PROFILE_DIR := $(CURDIR)
+
+ifdef PROFILE_GEN
+	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
+	EXTLIBS += -lgcov
+	export CCACHE_DISABLE=t
+endif
+
+ifdef PROFILE_USE
+	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
+	export CCACHE_DISABLE=t
+endif
+
 # Shell quote (do not use $(call) to accommodate ancient setups);
 
 SHA1_HEADER_SQ = $(subst ','\'',$(SHA1_HEADER))
@@ -2701,14 +2719,11 @@ cover_db_html: cover_db
 #
 .PHONY: profile-all profile-clean
 
-PROFILE_GEN_CFLAGS := $(CFLAGS) -fprofile-generate -DNO_NORETURN=1
-PROFILE_USE_CFLAGS := $(CFLAGS) -fprofile-use -fprofile-correction -DNO_NORETURN=1
-
 profile-clean:
-	$(RM) $(addsuffix *.gcda,$(object_dirs))
-	$(RM) $(addsuffix *.gcno,$(object_dirs))
+	$(RM) $(addsuffix *.gcda,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
+	$(RM) $(addsuffix *.gcno,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
 
 profile-all: profile-clean
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" all
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" -j1 test
-	$(MAKE) CFLAGS="$(PROFILE_USE_CFLAGS)" all
+	$(MAKE) PROFILE_GEN=t all
+	$(MAKE) PROFILE_GEN=t -j1 test
+	$(MAKE) PROFILE_USE=t all

-- 
1.7.8.11.gefc1f.dirty

Junio C Hamano· Feb 2, 2012, 20:02 UTC · re: Theodore Ts'o · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

Theodore Ts'o <tytso@mit.edu> writes:

Show 12 quoted lines

> diff --git a/INSTALL b/INSTALL
> index 6fa83fe..978ed09 100644
> --- a/INSTALL
> +++ b/INSTALL
> @@ -28,8 +28,8 @@ set up install paths (via config.mak.autogen), so you can write instead
>  If you're willing to trade off (much) longer build time for a later
>  faster git you can also do a profile feedback build with
>  
> -	$ make profile-all
> -	# make prefix=... install
> +	$ make prefix=... profile-all
> +	# make prefix=... PROFILE_USE=t install

Thanks for a patch. How does this compare with what was discussed in the other thread?

  http://thread.gmane.org/gmane.comp.version-control.git/188992/focus=189172

I would wish a solution ideally would support

	make PROFILE_BUILD=YesPlease
        make PROFILE_BUILD=YesPlease install

or even

	echo >>config.mak PROFILE_BUILD
        make
        su make install

and I think your patch takes us in the right direction.

Ted Ts'o· Feb 2, 2012, 20:12 UTC · re: Junio C Hamano · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

On Thu, Feb 02, 2012 at 12:02:27PM -0800, Junio C Hamano wrote:

Show 5 quoted lines

> 
> Thanks for a patch.  How does this compare with what was discussed in the
> other thread?
> 
>   http://thread.gmane.org/gmane.comp.version-control.git/188992/focus=189172

I wasn't aware of this other approach when I created this patch (I must have missed the e-mail thread, sorry).

One of the reasons why I did it this way was for more flexibility. I wanted to be able to do:

$ make PROFILE_GEN=YesPlease PROFILE_DIR=/var/cache/FDO all # make PROFILE_GEN=YesPlease PROFILE_DIR=/var/cache/FDO install

run a bunch of git commands on various git repositories to get real-life usage...

Then do...

$ make PROFILE_USE=YesPlease PROFILE_DIR=/var/cache/FDO all # make PROFILE_GEN=YesPlease PROFILE_DIR=/var/cache/FDO install

But for many people they would probably be satisfied with something that builds git using a single magic recipe, even if they give up some fractional performance improvement (keep in mind that the feedback directed optimization seems to buy you only a single digit percentage improvement according according to Andi's original experiment; I just got interested in this more for amusement value than any thought that it would save me serious amounts of time).

> I would wish a solution ideally would support
> 
> 	make PROFILE_BUILD=YesPlease
>         make PROFILE_BUILD=YesPlease install

At least in theory, it should be possible to have something which supports both PROFILE_GEN/PROFILE_USE as well as a combined PROFILE_BUILD.

The hard part is that PROFILE_BUILD requires a multi-pass process; you need to build with one set of CFLAGS, then run the sample workload to get the data for your feedback directed optimizations, and then re-run the build with another set of CFLAGS. I think what we could to check for PROFILE_BUILD, and if it is set, do the first PROFILE_GEN / make test commands as part of the top-level Makefile's all: rule, and then do the normal build after that.

It's a little kludgy, but does that sound acceptable to you?

       	      	      	       	    	  - Ted

Ted Ts'o· Feb 2, 2012, 20:14 UTC · re: Ted Ts'o · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

On Thu, Feb 02, 2012 at 03:12:26PM -0500, Ted Ts'o wrote:

> Then do...
> 
> $ make PROFILE_USE=YesPlease PROFILE_DIR=/var/cache/FDO all
> # make PROFILE_GEN=YesPlease PROFILE_DIR=/var/cache/FDO install

Err, that last line should have been:

# make PROFILE_USE=YesPlease PROFILE_DIR=/var/cache/FDO install

of course...

					- Ted

Junio C Hamano· Feb 3, 2012, 00:58 UTC · re: Ted Ts'o · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

Ted Ts'o <tytso@mit.edu> writes:

Show 9 quoted lines

> ...
> At least in theory, it should be possible to have something which
> supports both PROFILE_GEN/PROFILE_USE as well as a combined
> PROFILE_BUILD.
>
> The hard part is that PROFILE_BUILD requires a multi-pass process; you
> need to build with one set of CFLAGS, then run the sample workload to
> get the data for your feedback directed optimizations, and then re-run
> the build with another set of CFLAGS.

Yeah, I can see how that forces us to some kludgy solution, but I tend to agree that the separation between GEN/USE is a good thing.

> I think what we could to check
> for PROFILE_BUILD, and if it is set, do the first PROFILE_GEN / make
> test commands as part of the top-level Makefile's all: rule, and then
> do the normal build after that.

Yeah, something like that would emulate the "make profile-all" well enough for people not to notice the change while giving us the flexibility of GEN/USE separation. I kinda like it.

Thanks.

Ted Ts'o· Feb 3, 2012, 02:07 UTC · re: Junio C Hamano · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

What do you think of this? I'm still running a test build --- "make PROFILE=BUILD all" takes quite a long time, so this is still an RFC; I figure there will still be some places where people will point out more nits to be polished. :-)

(In particular, I just noticed I left the V=1 for debugging purposes in this version....)

	      	   	       	      - Ted

>From 4bf14e732216fd1327da2e3c8c6dfc0a3f689e1b Mon Sep 17 00:00:00 2001

From: Theodore Ts'o <tytso@mit.edu>
Date: Thu, 2 Feb 2012 13:56:22 -0500
Subject: [PATCH] Fix build problems related to profile-directed optimization

There was a number of problems I ran into when trying the profile-directed optimizations added by Andi Kleen in git commit 7ddc2710b9. (This was using gcc 4.4 found on many enterprise distros.)

1) The -fprofile-generate and -fprofile-use commands are incompatible
with ccache; the code ends up looking in the wrong place for the gcda
files based on the ccache object names.

2) If the makefile notices that CFLAGS are different, it will rebuild
all of the binaries.  Hence the recipe originally specified by the
INSTALL file ("make profile-all" followed by "make install") doesn't
work.  It will appear to work, but the binaries will end up getting
built with no optimization.

This patch fixes this by using an explicit set of options passed via the PROFILE variable then using this to directly manipulate CFLAGS and EXTLIBS.

The developer can run "make PROFILE=BUILD all ; make PROFILE=BUILD install" to do an automatic two-pass build using the test suite as the sample workload for the purpose of profiling.

Alternatively, the profiling version of binaries can be built using:

	make PROFILE=GEN PROFILE_DIR=/var/cache/profile all
	make PROFILE=GEN install

and then after git has been used a number of times, the optimized version of the binary can be built as follows:

	make PROFILE=USE PROFILE_DIR=/var/cache/profile all
	make PROFILE=USE install

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andi Kleen <ak@linux.intel.com>
---
 INSTALL  |    4 ++--
 Makefile |   41 ++++++++++++++++++++++++++++++-----------
 2 files changed, 32 insertions(+), 13 deletions(-)

Show changes to 2 files +32 −13

INSTALL, Makefile

diff --git a/INSTALL b/INSTALL
index 6fa83fe..73b654b 100644
--- a/INSTALL
+++ b/INSTALL
@@ -28,8 +28,8 @@ set up install paths (via config.mak.autogen), so you can write instead
 If you're willing to trade off (much) longer build time for a later
 faster git you can also do a profile feedback build with
 
-	$ make profile-all
-	# make prefix=... install
+	$ make --prefix=/usr PROFILE=BUILD all
+	# make --prefix=/usr PROFILE=BUILD install
 
 This will run the complete test suite as training workload and then
 rebuild git with the generated profile feedback. This results in a git
diff --git a/Makefile b/Makefile
index c457c34..7d66d5c 100644
--- a/Makefile
+++ b/Makefile
@@ -1772,6 +1772,24 @@ ifdef ASCIIDOC7
 	export ASCIIDOC7
 endif
 
+### profile feedback build
+#
+
+# Can adjust this to be a global directory if you want to do extended
+# data gathering
+PROFILE_DIR := $(CURDIR)
+
+ifeq "$(PROFILE)" "GEN"
+	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
+	EXTLIBS += -lgcov
+	export CCACHE_DISABLE=t
+	V=1
+else ifneq "$PROFILE" ""
+	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
+	export CCACHE_DISABLE=t
+	V=1
+endif
+
 # Shell quote (do not use $(call) to accommodate ancient setups);
 
 SHA1_HEADER_SQ = $(subst ','\'',$(SHA1_HEADER))
@@ -1828,7 +1846,15 @@ export DIFF TAR INSTALL DESTDIR SHELL_PATH
 
 SHELL = $(SHELL_PATH)
 
-all:: shell_compatibility_test $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
+all:: shell_compatibility_test
+
+ifeq "$(PROFILE)" "BUILD"
+all:: profile-clean
+	$(MAKE) PROFILE=GEN all
+	$(MAKE) PROFILE=GEN -j1 test
+endif
+
+all:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
 ifneq (,$X)
 	$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_PROGRAMS) $(BUILT_INS) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)
 endif
@@ -2699,16 +2725,9 @@ cover_db_html: cover_db
 
 ### profile feedback build
 #
-.PHONY: profile-all profile-clean
-
-PROFILE_GEN_CFLAGS := $(CFLAGS) -fprofile-generate -DNO_NORETURN=1
-PROFILE_USE_CFLAGS := $(CFLAGS) -fprofile-use -fprofile-correction -DNO_NORETURN=1
+.PHONY: profile-clean
 
 profile-clean:
-	$(RM) $(addsuffix *.gcda,$(object_dirs))
-	$(RM) $(addsuffix *.gcno,$(object_dirs))
+	$(RM) $(addsuffix *.gcda,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
+	$(RM) $(addsuffix *.gcno,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
 
-profile-all: profile-clean
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" all
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" -j1 test
-	$(MAKE) CFLAGS="$(PROFILE_USE_CFLAGS)" all

-- 
1.7.8.11.gefc1f.dirty

Junio C Hamano· Feb 3, 2012, 06:00 UTC · re: Ted Ts'o · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

Ted Ts'o <tytso@mit.edu> writes:

Show 7 quoted lines

> What do you think of this?  I'm still running a test build --- "make
> PROFILE=BUILD all" takes quite a long time, so this is still an RFC; I
> figure there will still be some places where people will point out
> more nits to be polished.  :-)
>
> (In particular, I just noticed I left the V=1 for debugging purposes
> in this version....)

Thanks.

Three comments:

 * I am happy that this version handles this well:

   $ make PROFILE=BUILD install

   even though you did not advertise as such in INSTALL ;-).

 * However, I think "clean" target should remove *.gcda unconditionally.

   $ make PROFILE=BUILD install ; make clean ; git clean -n -x | grep gcda

 * Running "make PROFILE=BUILD install" immediately after another one,
   without "make clean" in between, resulted in full rebuild and test
   before the second "install", which somewhat surprised me.  I however do
   not think this is a big show-stopper problem.

Theodore Tso· Feb 3, 2012, 18:19 UTC · re: Junio C Hamano · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

On Feb 3, 2012, at 1:00 AM, Junio C Hamano wrote:

Show 6 quoted lines

> 
> * I am happy that this version handles this well:
> 
>   $ make PROFILE=BUILD install
> 
>   even though you did not advertise as such in INSTALL ;-).

I can mention it, although it will mean adding more verbiage about profile-directed optimization into the INSTALL.

My assumption was that people who did this would usually be installing into --prefix=/usr as root, but there certainly will be anal people like myself who want to install profile-optimized binaries into ~/bin.  :-)

> * However, I think "clean" target should remove *.gcda unconditionally.
> 
>   $ make PROFILE=BUILD install ; make clean ; git clean -n -x | grep gcda

Will fix.

> * Running "make PROFILE=BUILD install" immediately after another one,
>   without "make clean" in between, resulted in full rebuild and test
>   before the second "install", which somewhat surprised me.  I however do
>   not think this is a big show-stopper problem.

Hmm… that surprises me too.  If

	make PROFILE=BUILD all
	make PROFILE=BUILD install

works correctly, I don't understand why a second "make PROFILE=BUILD install" issued after the above sequence would result in complete rebuild and test pass, unless something in the "make install" rules is modifying the build tree as a side-effect of the install pass, which I'd argue is a bug.  I'll take a look at it.

-- Ted

Junio C Hamano· Feb 3, 2012, 19:58 UTC · re: Theodore Tso · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

Theodore Tso <tytso@MIT.EDU> writes:

Show 11 quoted lines

> On Feb 3, 2012, at 1:00 AM, Junio C Hamano wrote:
>
>> 
>> * I am happy that this version handles this well:
>> 
>>   $ make PROFILE=BUILD install
>> 
>>   even though you did not advertise as such in INSTALL ;-).
>
> I can mention it, although it will mean adding more verbiage about
> profile-directed optimization into the INSTALL.

Oh, sorry, I didn't mean it that way. Please read it as: "Something that is a natural thing for people to expect after reading what is in INSTALL works correctly. Yay! Thanks."

Theodore Ts'o· Feb 6, 2012, 00:44 UTC · re: Junio C Hamano · lore

[PATCH] Fix build problems related to profile-directed optimization

There was a number of problems I ran into when trying the profile-directed optimizations added by Andi Kleen in git commit 7ddc2710b9. (This was using gcc 4.4 found on many enterprise distros.)

1) The -fprofile-generate and -fprofile-use commands are incompatible
with ccache; the code ends up looking in the wrong place for the gcda
files based on the ccache object names.

2) If the makefile notices that CFLAGS are different, it will rebuild
all of the binaries.  Hence the recipe originally specified by the
INSTALL file ("make profile-all" followed by "make install") doesn't
work.  It will appear to work, but the binaries will end up getting
built with no optimization.

This patch fixes this by using an explicit set of options passed via the PROFILE variable then using this to directly manipulate CFLAGS and EXTLIBS.

The developer can run "make PROFILE=BUILD all ; sudo make PROFILE=BUILD install" automatically run a two-pass build with the test suite run in between as the sample workload for the purpose of recording profiling information to do the profile-directed optimization.

Alternatively, the profiling version of binaries can be built using:

	make PROFILE=GEN PROFILE_DIR=/var/cache/profile all
	make PROFILE=GEN install

and then after git has been used for a while, the optimized version of the binary can be built as follows:

	make PROFILE=USE PROFILE_DIR=/var/cache/profile all
	make PROFILE=USE install

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andi Kleen <ak@linux.intel.com>
---
 INSTALL  |   17 +++++++++++++----
 Makefile |   53 +++++++++++++++++++++++++++++++++++------------------
 2 files changed, 48 insertions(+), 22 deletions(-)

Show changes to 2 files +48 −22

INSTALL, Makefile

diff --git a/INSTALL b/INSTALL
index 6fa83fe..5b7eec1 100644
--- a/INSTALL
+++ b/INSTALL
@@ -28,16 +28,25 @@ set up install paths (via config.mak.autogen), so you can write instead
 If you're willing to trade off (much) longer build time for a later
 faster git you can also do a profile feedback build with
 
-	$ make profile-all
-	# make prefix=... install
+	$ make --prefix=/usr PROFILE=BUILD all
+	# make --prefix=/usr PROFILE=BUILD install
 
 This will run the complete test suite as training workload and then
 rebuild git with the generated profile feedback. This results in a git
 which is a few percent faster on CPU intensive workloads.  This
 may be a good tradeoff for distribution packagers.
 
-Note that the profile feedback build stage currently generates
-a lot of additional compiler warnings.
+Or if you just want to install a profile-optimized version of git into
+your home directory, you could run:
+
+	$ make PROFILE=BUILD install
+
+As a caveat: a profile-optimized build takes a *lot* longer since it
+is the sources have to be built twice, and in order for the profiling
+measurements to work properly, ccache must be disabled and the test
+suite has to be run using only a single CPU.  In addition, the profile
+feedback build stage currently generates a lot of additional compiler
+warnings.
 
 Issues of note:
 
diff --git a/Makefile b/Makefile
index c457c34..8cea247 100644
--- a/Makefile
+++ b/Makefile
@@ -1772,6 +1772,24 @@ ifdef ASCIIDOC7
 	export ASCIIDOC7
 endif
 
+### profile feedback build
+#
+
+# Can adjust this to be a global directory if you want to do extended
+# data gathering
+PROFILE_DIR := $(CURDIR)
+
+ifeq "$(PROFILE)" "GEN"
+	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
+	EXTLIBS += -lgcov
+	export CCACHE_DISABLE=t
+	V=1
+else ifneq "$PROFILE" ""
+	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
+	export CCACHE_DISABLE=t
+	V=1
+endif
+
 # Shell quote (do not use $(call) to accommodate ancient setups);
 
 SHA1_HEADER_SQ = $(subst ','\'',$(SHA1_HEADER))
@@ -1828,7 +1846,17 @@ export DIFF TAR INSTALL DESTDIR SHELL_PATH
 
 SHELL = $(SHELL_PATH)
 
-all:: shell_compatibility_test $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
+all:: shell_compatibility_test
+
+ifeq "$(PROFILE)" "BUILD"
+ifeq ($(filter all,$(MAKECMDGOALS)),all)
+all:: profile-clean
+	$(MAKE) PROFILE=GEN all
+	$(MAKE) PROFILE=GEN -j1 test
+endif
+endif
+
+all:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
 ifneq (,$X)
 	$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_PROGRAMS) $(BUILT_INS) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)
 endif
@@ -2557,7 +2585,11 @@ distclean: clean
 	$(RM) configure
 	$(RM) po/git.pot
 
-clean:
+profile-clean:
+	$(RM) $(addsuffix *.gcda,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
+	$(RM) $(addsuffix *.gcno,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
+
+clean: profile-clean
 	$(RM) *.o block-sha1/*.o ppc/*.o compat/*.o compat/*/*.o xdiff/*.o vcs-svn/*.o \
 		builtin/*.o $(LIB_FILE) $(XDIFF_LIB) $(VCSSVN_LIB)
 	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
@@ -2587,7 +2619,7 @@ ifndef NO_TCLTK
 endif
 	$(RM) GIT-VERSION-FILE GIT-CFLAGS GIT-LDFLAGS GIT-GUI-VARS GIT-BUILD-OPTIONS
 
-.PHONY: all install clean strip
+.PHONY: all install profile-clean clean strip
 .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell
 .PHONY: FORCE cscope
 
@@ -2697,18 +2729,3 @@ cover_db: coverage-report
 cover_db_html: cover_db
 	cover -report html -outputdir cover_db_html cover_db
 
-### profile feedback build
-#
-.PHONY: profile-all profile-clean
-
-PROFILE_GEN_CFLAGS := $(CFLAGS) -fprofile-generate -DNO_NORETURN=1
-PROFILE_USE_CFLAGS := $(CFLAGS) -fprofile-use -fprofile-correction -DNO_NORETURN=1
-
-profile-clean:
-	$(RM) $(addsuffix *.gcda,$(object_dirs))
-	$(RM) $(addsuffix *.gcno,$(object_dirs))
-
-profile-all: profile-clean
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" all
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" -j1 test
-	$(MAKE) CFLAGS="$(PROFILE_USE_CFLAGS)" all

-- 
1.7.9.107.g8e04a

Jeff King· Feb 6, 2012, 04:18 UTC · re: Theodore Ts'o · lore

Re: [PATCH] Fix build problems related to profile-directed optimization

On Sun, Feb 05, 2012 at 07:44:50PM -0500, Theodore Ts'o wrote:

Show 12 quoted lines

> diff --git a/INSTALL b/INSTALL
> index 6fa83fe..5b7eec1 100644
> --- a/INSTALL
> +++ b/INSTALL
> @@ -28,16 +28,25 @@ set up install paths (via config.mak.autogen), so you can write instead
>  If you're willing to trade off (much) longer build time for a later
>  faster git you can also do a profile feedback build with
>  
> -	$ make profile-all
> -	# make prefix=... install
> +	$ make --prefix=/usr PROFILE=BUILD all
> +	# make --prefix=/usr PROFILE=BUILD install

Eh? --prefix?

> +As a caveat: a profile-optimized build takes a *lot* longer since it
> +is the sources have to be built twice, and in order for the profiling

s/it is//

Show 16 quoted lines

> diff --git a/Makefile b/Makefile
> index c457c34..8cea247 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1772,6 +1772,24 @@ ifdef ASCIIDOC7
> [...]
> +ifeq "$(PROFILE)" "GEN"
> +	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
> +	EXTLIBS += -lgcov
> +	export CCACHE_DISABLE=t
> +	V=1
> +else ifneq "$PROFILE" ""
> +	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
> +	export CCACHE_DISABLE=t
> +	V=1
> +endif

Did you mean "$(PROFILE)" in the second conditional?

-Peff

Ted Ts'o· Feb 6, 2012, 05:57 UTC · re: Jeff King · lore

Re: [PATCH] Fix build problems related to profile-directed optimization

On Sun, Feb 05, 2012 at 11:18:39PM -0500, Jeff King wrote:

Show 6 quoted lines

> > -	$ make profile-all
> > -	# make prefix=... install
> > +	$ make --prefix=/usr PROFILE=BUILD all
> > +	# make --prefix=/usr PROFILE=BUILD install
> 
> Eh? --prefix?

Oops, configure meme strikes; will fix.

> > +As a caveat: a profile-optimized build takes a *lot* longer since it
> > +is the sources have to be built twice, and in order for the profiling
> 
> s/it is//

Thanks, will fix.

Show 8 quoted lines

> > +ifeq "$(PROFILE)" "GEN"
> > +	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
> > +	EXTLIBS += -lgcov
> > +	export CCACHE_DISABLE=t
> > +	V=1
> > +else ifneq "$PROFILE" ""
> 
> Did you mean "$(PROFILE)" in the second conditional?

Yes, thanks.   Will fix.

						- Ted

Theodore Ts'o· Feb 6, 2012, 06:00 UTC · re: Ted Ts'o · lore

[PATCH] Fix build problems related to profile-directed optimization

There was a number of problems I ran into when trying the profile-directed optimizations added by Andi Kleen in git commit 7ddc2710b9. (This was using gcc 4.4 found on many enterprise distros.)

1) The -fprofile-generate and -fprofile-use commands are incompatible
with ccache; the code ends up looking in the wrong place for the gcda
files based on the ccache object names.

2) If the makefile notices that CFLAGS are different, it will rebuild
all of the binaries.  Hence the recipe originally specified by the
INSTALL file ("make profile-all" followed by "make install") doesn't
work.  It will appear to work, but the binaries will end up getting
built with no optimization.

This patch fixes this by using an explicit set of options passed via the PROFILE variable then using this to directly manipulate CFLAGS and EXTLIBS.

Alternatively, the profiling version of binaries can be built using:

	make PROFILE=GEN PROFILE_DIR=/var/cache/profile all
	make PROFILE=GEN install

and then after git has been used for a while, the optimized version of the binary can be built as follows:

	make PROFILE=USE PROFILE_DIR=/var/cache/profile all
	make PROFILE=USE install

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andi Kleen <ak@linux.intel.com>
---
 INSTALL  |   17 +++++++++++++----
 Makefile |   53 +++++++++++++++++++++++++++++++++++------------------
 2 files changed, 48 insertions(+), 22 deletions(-)

Show changes to 2 files +48 −22

INSTALL, Makefile

diff --git a/INSTALL b/INSTALL
index 6fa83fe..58b2b86 100644
--- a/INSTALL
+++ b/INSTALL
@@ -28,16 +28,25 @@ set up install paths (via config.mak.autogen), so you can write instead
 If you're willing to trade off (much) longer build time for a later
 faster git you can also do a profile feedback build with
 
-	$ make profile-all
-	# make prefix=... install
+	$ make prefix=/usr PROFILE=BUILD all
+	# make prefix=/usr PROFILE=BUILD install
 
 This will run the complete test suite as training workload and then
 rebuild git with the generated profile feedback. This results in a git
 which is a few percent faster on CPU intensive workloads.  This
 may be a good tradeoff for distribution packagers.
 
-Note that the profile feedback build stage currently generates
-a lot of additional compiler warnings.
+Or if you just want to install a profile-optimized version of git into
+your home directory, you could run:
+
+	$ make PROFILE=BUILD install
+
+As a caveat: a profile-optimized build takes a *lot* longer since the
+git tree must be built twice, and in order for the profiling
+measurements to work properly, ccache must be disabled and the test
+suite has to be run using only a single CPU.  In addition, the profile
+feedback build stage currently generates a lot of additional compiler
+warnings.
 
 Issues of note:
 
diff --git a/Makefile b/Makefile
index c457c34..719ffca 100644
--- a/Makefile
+++ b/Makefile
@@ -1772,6 +1772,24 @@ ifdef ASCIIDOC7
 	export ASCIIDOC7
 endif
 
+### profile feedback build
+#
+
+# Can adjust this to be a global directory if you want to do extended
+# data gathering
+PROFILE_DIR := $(CURDIR)
+
+ifeq "$(PROFILE)" "GEN"
+	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
+	EXTLIBS += -lgcov
+	export CCACHE_DISABLE=t
+	V=1
+else ifneq "$(PROFILE)" ""
+	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
+	export CCACHE_DISABLE=t
+	V=1
+endif
+
 # Shell quote (do not use $(call) to accommodate ancient setups);
 
 SHA1_HEADER_SQ = $(subst ','\'',$(SHA1_HEADER))
@@ -1828,7 +1846,17 @@ export DIFF TAR INSTALL DESTDIR SHELL_PATH
 
 SHELL = $(SHELL_PATH)
 
-all:: shell_compatibility_test $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
+all:: shell_compatibility_test
+
+ifeq "$(PROFILE)" "BUILD"
+ifeq ($(filter all,$(MAKECMDGOALS)),all)
+all:: profile-clean
+	$(MAKE) PROFILE=GEN all
+	$(MAKE) PROFILE=GEN -j1 test
+endif
+endif
+
+all:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) GIT-BUILD-OPTIONS
 ifneq (,$X)
 	$(QUIET_BUILT_IN)$(foreach p,$(patsubst %$X,%,$(filter %$X,$(ALL_PROGRAMS) $(BUILT_INS) git$X)), test -d '$p' -o '$p' -ef '$p$X' || $(RM) '$p';)
 endif
@@ -2557,7 +2585,11 @@ distclean: clean
 	$(RM) configure
 	$(RM) po/git.pot
 
-clean:
+profile-clean:
+	$(RM) $(addsuffix *.gcda,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
+	$(RM) $(addsuffix *.gcno,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))
+
+clean: profile-clean
 	$(RM) *.o block-sha1/*.o ppc/*.o compat/*.o compat/*/*.o xdiff/*.o vcs-svn/*.o \
 		builtin/*.o $(LIB_FILE) $(XDIFF_LIB) $(VCSSVN_LIB)
 	$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
@@ -2587,7 +2619,7 @@ ifndef NO_TCLTK
 endif
 	$(RM) GIT-VERSION-FILE GIT-CFLAGS GIT-LDFLAGS GIT-GUI-VARS GIT-BUILD-OPTIONS
 
-.PHONY: all install clean strip
+.PHONY: all install profile-clean clean strip
 .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell
 .PHONY: FORCE cscope
 
@@ -2697,18 +2729,3 @@ cover_db: coverage-report
 cover_db_html: cover_db
 	cover -report html -outputdir cover_db_html cover_db
 
-### profile feedback build
-#
-.PHONY: profile-all profile-clean
-
-PROFILE_GEN_CFLAGS := $(CFLAGS) -fprofile-generate -DNO_NORETURN=1
-PROFILE_USE_CFLAGS := $(CFLAGS) -fprofile-use -fprofile-correction -DNO_NORETURN=1
-
-profile-clean:
-	$(RM) $(addsuffix *.gcda,$(object_dirs))
-	$(RM) $(addsuffix *.gcno,$(object_dirs))
-
-profile-all: profile-clean
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" all
-	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" -j1 test
-	$(MAKE) CFLAGS="$(PROFILE_USE_CFLAGS)" all

-- 
1.7.9.107.g8e04a

Ted Ts'o· Feb 8, 2012, 18:53 UTC · re: Theodore Ts'o · lore

Re: [PATCH] Fix build problems related to profile-directed optimization

Junio, any comments on my most recent spin of this patch? Any changes you'd like to see?

Thanks,

					- Ted

Junio C Hamano· Feb 9, 2012, 04:03 UTC · re: Ted Ts'o · lore

Re: [PATCH] Fix build problems related to profile-directed optimization

Ted Ts'o <tytso@mit.edu> writes:

> Junio, any comments on my most recent spin of this patch?  Any changes
> you'd like to see?

Nothing from me; all looked good.

Let's cook it in 'next' for a few days to give developers a chance to play with it and merge down to 'master'.

Thanks.

Johannes Sixt· Feb 9, 2012, 08:22 UTC · re: Ted Ts'o · lore

Re: [PATCH] Fix build problems related to profile-directed optimization

Am 2/8/2012 19:53, schrieb Ted Ts'o:

> Junio, any comments on my most recent spin of this patch?  Any changes
> you'd like to see?

I need the following to unbreak my build on Windows.

--- >8 ---
From: Johannes Sixt <j6t@kdbg.org>
Subject: [PATCH] Makefile: fix syntax for older make

It is necessary to write the else branch as a nested conditional. Also, write the conditions with parentheses because we use them throughout the Makefile.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 Makefile |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

Show changes to Makefile +4 −2

diff --git a/Makefile b/Makefile
index bfc5daa..01a3c77 100644
--- a/Makefile
+++ b/Makefile
@@ -1784,16 +1784,18 @@ endif
 # data gathering
 PROFILE_DIR := $(CURDIR)
 
-ifeq "$(PROFILE)" "GEN"
+ifeq ("$(PROFILE)","GEN")
 	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
 	EXTLIBS += -lgcov
 	export CCACHE_DISABLE=t
 	V=1
-else ifneq "$(PROFILE)" ""
+else
+ifneq ("$(PROFILE)","")
 	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
 	export CCACHE_DISABLE=t
 	V=1
 endif
+endif
 
 # Shell quote (do not use $(call) to accommodate ancient setups);

-- 
1.7.9.1420.gae2d6

Junio C Hamano· Feb 9, 2012, 18:22 UTC · re: Johannes Sixt · lore

Re: [PATCH] Fix build problems related to profile-directed optimization

Johannes Sixt <j.sixt@viscovery.net> writes:

Show 5 quoted lines

> Am 2/8/2012 19:53, schrieb Ted Ts'o:
>> Junio, any comments on my most recent spin of this patch?  Any changes
>> you'd like to see?
>
> I need the following to unbreak my build on Windows.

Thanks; will apply.

Show 37 quoted lines

> --- >8 ---
> From: Johannes Sixt <j6t@kdbg.org>
> Subject: [PATCH] Makefile: fix syntax for older make
>
> It is necessary to write the else branch as a nested conditional. Also,
> write the conditions with parentheses because we use them throughout the
> Makefile.
>
> Signed-off-by: Johannes Sixt <j6t@kdbg.org>
> ---
>  Makefile |    6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index bfc5daa..01a3c77 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1784,16 +1784,18 @@ endif
>  # data gathering
>  PROFILE_DIR := $(CURDIR)
>  
> -ifeq "$(PROFILE)" "GEN"
> +ifeq ("$(PROFILE)","GEN")
>  	CFLAGS += -fprofile-generate=$(PROFILE_DIR) -DNO_NORETURN=1
>  	EXTLIBS += -lgcov
>  	export CCACHE_DISABLE=t
>  	V=1
> -else ifneq "$(PROFILE)" ""
> +else
> +ifneq ("$(PROFILE)","")
>  	CFLAGS += -fprofile-use=$(PROFILE_DIR) -fprofile-correction -DNO_NORETURN=1
>  	export CCACHE_DISABLE=t
>  	V=1
>  endif
> +endif
>  
>  # Shell quote (do not use $(call) to accommodate ancient setups);

Andi Kleen· Feb 3, 2012, 18:39 UTC · re: Junio C Hamano · lore

Re: [PATCH, RFC] Fix build problems related to profile-directed optimization

Thanks everyone for improving this.

I should add that any improvements will depend on your compiler version. I would also expect better numbers when combined with LTO in gcc 4.7, but haven't tried so far.

BTW it would be really nice to figure out a subset of the test suite that runs faster and gives similar speedup like the full one.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only