# [PATCH 2/2] Add profile feedback build to git 7 messages from 2011-06-08 to 2011-06-09. Participants: Andi Kleen, Junio C Hamano, Jeff King, Erik Faye-Lund. Thread: https://gitlist.dev/t/27586 ## Andi Kleen, 2011-06-08 21:43 Subject: [PATCH 1/2] Remove noreturn function pointers in usage.c Message-ID: <1307569417-8924-1-git-send-email-andi@firstfloor.org> URL: https://gitlist.dev/e/1307569417-8924-1-git-send-email-andi%40firstfloor.org ``` From: Andi Kleen Due to a bug in gcc 4.6+ it can crash when doing profile feedback with a noreturn function pointer (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) Remove the NORETURNs from the die functions for now to work around this. Doesn't seem to make any difference. Signed-off-by: Andi Kleen --- usage.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/usage.c b/usage.c index b5e67e3..4045574 100644 --- a/usage.c +++ b/usage.c @@ -12,13 +12,13 @@ void vreportf(const char *prefix, const char *err, va_list params) fprintf(stderr, "%s%s\n", prefix, msg); } -static NORETURN void usage_builtin(const char *err, va_list params) +static void usage_builtin(const char *err, va_list params) { vreportf("usage: ", err, params); exit(129); } -static NORETURN void die_builtin(const char *err, va_list params) +static void die_builtin(const char *err, va_list params) { vreportf("fatal: ", err, params); exit(128); @@ -36,8 +36,8 @@ static void warn_builtin(const char *warn, va_list params) /* If we are in a dlopen()ed .so write to a global variable would segfault * (ugh), so keep things static. */ -static NORETURN_PTR void (*usage_routine)(const char *err, va_list params) = usage_builtin; -static NORETURN_PTR void (*die_routine)(const char *err, va_list params) = die_builtin; +static void (*usage_routine)(const char *err, va_list params) = usage_builtin; +static void (*die_routine)(const char *err, va_list params) = die_builtin; static void (*error_routine)(const char *err, va_list params) = error_builtin; static void (*warn_routine)(const char *err, va_list params) = warn_builtin; -- 1.7.4.4 ``` ## Andi Kleen, 2011-06-08 21:43 Subject: [PATCH 2/2] Add profile feedback build to git Message-ID: <1307569417-8924-2-git-send-email-andi@firstfloor.org> URL: https://gitlist.dev/e/1307569417-8924-2-git-send-email-andi%40firstfloor.org In-Reply-To: <1307569417-8924-1-git-send-email-andi@firstfloor.org> ``` From: Andi Kleen Add a gcc profile feedback build option "profile-all" to the main Makefile. It simply runs the test suite to generate feedback data and the recompiles the main executables with that. The basic structure is similar to the existing gcov code. gcc is often able to generate better code with profile feedback data. The training load also doesn't need to be too similar to the actual load, it still gives benefits. The test suite run is unfortunately quite long. It would be good to find a suitable subset that runs faster and still gives reasonable feedback. For now the test suite runs single threaded (I had some trouble running the test suite with -jX) I tested it with git gc and git blame kernel/sched.c on a Linux kernel tree. For gc I get about 2.7% improvement in wall clock time by using the feedback build, for blame about 2.4%. That's not gigantic, but not shabby either for a very small patch. If anyone has any favourite CPU intensive git benchmarks feel free to try them too. I hope distributors will switch to use a feedback build in their packages. Signed-off-by: Andi Kleen --- Makefile | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/Makefile b/Makefile index e40ac0c..85e2679 100644 --- a/Makefile +++ b/Makefile @@ -2486,3 +2486,20 @@ cover_db: coverage-report cover_db_html: cover_db cover -report html -outputdir cover_db_html cover_db + +### profile feedback build +# +.PHONY: profile-all profile-clean + +PROFILE_GEN_CFLAGS := $(CFLAGS) -fprofile-generate +PROFILE_USE_CFLAGS := $(CFLAGS) -fprofile-use -fprofile-correction + +profile-clean: + $(RM) $(addsuffix *.gcda,$(object_dirs)) + $(RM) $(addsuffix *.gcno,$(object_dirs)) + +profile-all: profile-clean + $(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" all + $(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" -j1 test + $(MAKE) CFLAGS="$(PROFILE_USE_CFLAGS)" all + -- 1.7.4.4 ``` ## Junio C Hamano, 2011-06-09 00:36 Subject: Re: [PATCH 1/2] Remove noreturn function pointers in usage.c Message-ID: <7v4o3z264s.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7v4o3z264s.fsf%40alter.siamese.dyndns.org In-Reply-To: <1307569417-8924-1-git-send-email-andi@firstfloor.org> ``` Andi Kleen writes: > From: Andi Kleen > > Due to a bug in gcc 4.6+ it can crash when doing profile feedback > with a noreturn function pointer > > (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) > > Remove the NORETURNs from the die functions for now to work > around this. Doesn't seem to make any difference. > > Signed-off-by: Andi Kleen I would expect a better patch from a well respected kernel person, though. - There are many more NORETURN and NORETURN_PTR in the code, and the proposed commit log message does not explain why these two are the only ones that are problematic and needs to be worked around. It does not guide other people who might want to add NORETURN/NORETURN_PTR when deciding if their change would break the "fix" this change brought in. - Potential impact to people who do not use Gcc 4.6 with profile feedback is not explained away well, except for "Doesn't seem to make any difference." - If other NORETURN/NORETURN_PTR could/should also go (I don't know due to the first bullet point above) when using the problematic compiler with the profile feedback feature, wouldn't it be a better workaround would be to introduce a Makefile variable to ask git-compat-util.h to make these two a no-op, perhaps? A patch to do so may look like this. I did not like the triple negation "make NO_NORETURN=NoThanks" and wanted to name this AVOID_NORETURN instead, but decided to go with other existing Makefile variables. Makefile | 6 ++++++ git-compat-util.h | 2 +- 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/Makefile b/Makefile index d2e2ea1..70c814c 100644 --- a/Makefile +++ b/Makefile @@ -144,6 +144,9 @@ all:: # that tells runtime paths to dynamic libraries; # "-Wl,-rpath=/path/lib" is used instead. # +# Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, +# as the compiler can crash (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) +# # Define USE_NSEC below if you want git to care about sub-second file mtimes # and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and # it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely @@ -1349,6 +1352,9 @@ endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif +ifdef NO_NORETURN + BASIC_CFLAGS += -DNO_NORETURN +endif ifdef NO_NSEC BASIC_CFLAGS += -DNO_NSEC endif diff --git a/git-compat-util.h b/git-compat-util.h index 40498b3..13bc26f 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -218,7 +218,7 @@ extern char *gitbasename(char *); #if __HP_cc >= 61000 #define NORETURN __attribute__((noreturn)) #define NORETURN_PTR -#elif defined(__GNUC__) +#elif defined(__GNUC__) && !defined(NO_NORETURN) #define NORETURN __attribute__((__noreturn__)) #define NORETURN_PTR __attribute__((__noreturn__)) #elif defined(_MSC_VER) ``` ## Andi Kleen, 2011-06-09 04:59 Subject: Re: [PATCH 1/2] Remove noreturn function pointers in usage.c Message-ID: <20110609045915.GA15448@one.firstfloor.org> URL: https://gitlist.dev/e/20110609045915.GA15448%40one.firstfloor.org In-Reply-To: <7v4o3z264s.fsf@alter.siamese.dyndns.org> ``` > - There are many more NORETURN and NORETURN_PTR in the code, and the > proposed commit log message does not explain why these two are the only > ones that are problematic and needs to be worked around. It does not > guide other people who might want to add NORETURN/NORETURN_PTR when > deciding if their change would break the "fix" this change brought in. This was the only place where it crashed the compiler. I don't have a good criterium to decide which cases do crash the compiler or not except for trying it. But I believe the crash is relatively unlikely (needs quite a lot of conditions to line up), so it doesn't deserve extensive changes all over. > > - Potential impact to people who do not use Gcc 4.6 with profile feedback > is not explained away well, except for "Doesn't seem to make any > difference." I merely went by "there are no new warnings" (I assume that's the main motivation) > > - If other NORETURN/NORETURN_PTR could/should also go (I don't know due > to the first bullet point above) when using the problematic compiler > with the profile feedback feature, wouldn't it be a better workaround > would be to introduce a Makefile variable to ask git-compat-util.h to > make these two a no-op, perhaps? I don't think we need to remove the others for now. > > A patch to do so may look like this. > > I did not like the triple negation "make NO_NORETURN=NoThanks" and wanted > to name this AVOID_NORETURN instead, but decided to go with other existing > Makefile variables. Given the explanation above (I can update the description with that) do you still want the complete disabling? -Andi ``` ## Jeff King, 2011-06-09 05:52 Subject: Re: [PATCH 1/2] Remove noreturn function pointers in usage.c Message-ID: <20110609055224.GA27780@sigill.intra.peff.net> URL: https://gitlist.dev/e/20110609055224.GA27780%40sigill.intra.peff.net In-Reply-To: <20110609045915.GA15448@one.firstfloor.org> ``` On Thu, Jun 09, 2011 at 06:59:15AM +0200, Andi Kleen wrote: > > - Potential impact to people who do not use Gcc 4.6 with profile feedback > > is not explained away well, except for "Doesn't seem to make any > > difference." > > I merely went by "there are no new warnings" (I assume that's the main > motivation) On your compiler and settings, perhaps. With your patch I get: usage.c: In function ‘die’: usage.c:70:1: error: ‘noreturn’ function does return [-Werror] And rightfully so: void NORETURN die(const char *err, ...) { va_list params; va_start(params, err); die_routine(err, params); va_end(params); } You've stripped the NORETURN from die_routine, so of course it looks like we end up returning. This is with: $ gcc --version | head -n 1 gcc (Debian 4.6.0-11) 4.6.1 20110604 (prerelease) -Peff ``` ## Andi Kleen, 2011-06-09 06:31 Subject: Re: [PATCH 1/2] Remove noreturn function pointers in usage.c Message-ID: <20110609063145.GC15448@one.firstfloor.org> URL: https://gitlist.dev/e/20110609063145.GC15448%40one.firstfloor.org In-Reply-To: <20110609055224.GA27780@sigill.intra.peff.net> ``` On Thu, Jun 09, 2011 at 01:52:24AM -0400, Jeff King wrote: > On Thu, Jun 09, 2011 at 06:59:15AM +0200, Andi Kleen wrote: > > > > - Potential impact to people who do not use Gcc 4.6 with profile feedback > > > is not explained away well, except for "Doesn't seem to make any > > > difference." > > > > I merely went by "there are no new warnings" (I assume that's the main > > motivation) > > On your compiler and settings, perhaps. With your patch I get: > > usage.c: In function ‘die’: > usage.c:70:1: error: ‘noreturn’ function does return [-Werror] Ok. Hmm, all I can say it compiled here. Ok then we have to remove it. I didn't really like Junio's approach to only do it for a single file because that would break with LTO / link time optimization which requires declarations to match between translation units. Maybe it's better to stick an extra exit() at the end to shut up the extra warning. I'll do that I guess. BTW 4.6.2 or so will have the problem fixed. -Andi ``` ## Erik Faye-Lund, 2011-06-09 21:13 Subject: Re: [PATCH 1/2] Remove noreturn function pointers in usage.c Message-ID: URL: https://gitlist.dev/e/BANLkTimxA9yRHFt8jX2kEM8JvNoO5AOcYA%40mail.gmail.com In-Reply-To: <20110609063145.GC15448@one.firstfloor.org> ``` On Thu, Jun 9, 2011 at 8:31 AM, Andi Kleen wrote: > On Thu, Jun 09, 2011 at 01:52:24AM -0400, Jeff King wrote: >> On Thu, Jun 09, 2011 at 06:59:15AM +0200, Andi Kleen wrote: >> >> > >  - Potential impact to people who do not use Gcc 4.6 with profile feedback >> > >    is not explained away well, except for "Doesn't seem to make any >> > >    difference." >> > >> > I merely went by "there are no new warnings" (I assume that's the main >> > motivation) >> >> On your compiler and settings, perhaps. With your patch I get: >> >>   usage.c: In function ‘die’: >>   usage.c:70:1: error: ‘noreturn’ function does return [-Werror] > > Ok.  Hmm, all I can say it compiled here. > > Ok then we have to remove it. I didn't really like Junio's approach > to only do it for a single file because that would break with LTO / link > time optimization which requires declarations to match between > translation units. Junio's approach didn't do it for a single file, it disabled the NORETURN mechanism all together, by having a Makefile-switch. > BTW 4.6.2 or so will have the problem fixed. If it's only in 4.6 through 4.6.2, then we probably don't even need a Makefile-switch for junio's approach at all; just checking the GCC version should be reliable enough, no? ```