threads / patch / 56225

patchmakefile: update detect-compiler for newer Xcode version

Subject: [PATCH] makefile: update detect-compiler for newer Xcode version

## tl;dr

22 messages between Aug 6, 2021 and Aug 9, 2021. Diffs are folded; open one to read it.

replies: 21people: 7as markdown or json

Carlo Marcelo Arenas Belón· Aug 6, 2021, 08:06 UTC · lore

1da1580e4c (Makefile: detect compiler and enable more warnings in DEVELOPER=1, 2018-04-14) uses the output of the compiler banner to detect the compiler family.

Apple had since changed the wording used to refer to its compiler as clang instead of LLVM as shown by:

  $ cc --version
  Apple clang version 12.0.5 (clang-1205.0.22.9)
  Target: x86_64-apple-darwin20.6.0
  Thread model: posix
  InstalledDir: /Library/Developer/CommandLineTools/usr/bin

so update the script to match, and allow DEVELOPER=1 to work as expected again in macOS.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
---
 detect-compiler | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to detect-compiler +1 −1
diff --git a/detect-compiler b/detect-compiler
index 70b754481c..c85be83c64 100755
--- a/detect-compiler
+++ b/detect-compiler
@@ -44,7 +44,7 @@ clang)
 "FreeBSD clang")
 	print_flags clang
 	;;
-"Apple LLVM")
+"Apple LLVM"|"Apple clang")
 	print_flags clang
 	;;
 *)

-- 
2.33.0.rc0.443.g98cc19b6c0
Bagas Sanjaya· Aug 6, 2021, 12:00 UTC · re: Carlo Marcelo Arenas Belón · lore

Re: [PATCH] makefile: update detect-compiler for newer Xcode version

On 06/08/21 15.06, Carlo Marcelo Arenas Belón wrote:
> -"Apple LLVM")
> +"Apple LLVM"|"Apple clang")
>   	print_flags clang
Why not just s/Apple LLVM/Apple clang/?
-- 
An old man doll... just what I always wanted! - Clara
Carlo Arenas· Aug 6, 2021, 13:32 UTC · re: Bagas Sanjaya · lore

Re: [PATCH] makefile: update detect-compiler for newer Xcode version

On Fri, Aug 6, 2021 at 5:00 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>
> Why not just s/Apple LLVM/Apple clang/?

because that would break it for older versions of Xcode that reported themselves as "Apple LLVM" and I am not even sure when the change was made to even consider those versions as obsolete.

Either way, I am sure once we figure out which versions are affected and which versions macOS developers care enough for, that would be the obvious next step.

note that DEVELOPER=1 was broken for a while in macOS and that also affects CI, and it is still known to be suboptimal, since version numbers for Xcode compilers are not always representative of what features can be found in the opensource version with similar numbers.

Carlo
Carlo
Eric Sunshine· Aug 6, 2021, 16:37 UTC · re: Carlo Arenas · lore

Re: [PATCH] makefile: update detect-compiler for newer Xcode version

On Fri, Aug 6, 2021 at 9:33 AM Carlo Arenas <carenas@gmail.com> wrote:
Show 10 quoted lines
> On Fri, Aug 6, 2021 at 5:00 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> > Why not just s/Apple LLVM/Apple clang/?
>
> because that would break it for older versions of Xcode that reported
> themselves as "Apple LLVM" and I am not even sure when the change was
> made to even consider those versions as obsolete.
>
> Either way, I am sure once we figure out which versions are affected
> and which versions macOS developers care enough for, that would be the
> obvious next step.

My daily development machine still reports "Apple LLVM", so it would indeed be premature to `s/Apple LLVM/Apple clang/`. Moreover, there's simply no good reason to `s/Apple LLVM/Apple clang/` since it's not a maintenance burden to leave "Apple LLVM" in the match pattern.

Atharva Raykar· Aug 6, 2021, 13:42 UTC · re: Carlo Marcelo Arenas Belón · lore

Re: [PATCH] makefile: update detect-compiler for newer Xcode version

Carlo Marcelo Arenas Belón <carenas@gmail.com> writes:
Show 15 quoted lines
> 1da1580e4c (Makefile: detect compiler and enable more warnings in
> DEVELOPER=1, 2018-04-14) uses the output of the compiler banner to
> detect the compiler family.
>
> Apple had since changed the wording used to refer to its compiler
> as clang instead of LLVM as shown by:
>
>   $ cc --version
>   Apple clang version 12.0.5 (clang-1205.0.22.9)
>   Target: x86_64-apple-darwin20.6.0
>   Thread model: posix
>   InstalledDir: /Library/Developer/CommandLineTools/usr/bin
>
> so update the script to match, and allow DEVELOPER=1 to work as
> expected again in macOS.
Thanks for submitting this enhancement!

For those of us using Homebrew and using the LLVM installation from there, we get:

$ cc --version
Homebrew clang version 12.0.1
Target: arm64-apple-darwin20.5.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
Show 18 quoted lines
> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
> ---
>  detect-compiler | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/detect-compiler b/detect-compiler
> index 70b754481c..c85be83c64 100755
> --- a/detect-compiler
> +++ b/detect-compiler
> @@ -44,7 +44,7 @@ clang)
>  "FreeBSD clang")
>  	print_flags clang
>  	;;
> -"Apple LLVM")
> +"Apple LLVM"|"Apple clang")
>  	print_flags clang
>  	;;
>  *)
So maybe we could add another case for "Homebrew clang"?

--- Atharva Raykar ಅಥರ್ವ ರಾಯ್ಕರ್ अथर्व रायकर

Junio C Hamano· Aug 6, 2021, 18:11 UTC · re: Atharva Raykar · lore

Re: [PATCH] makefile: update detect-compiler for newer Xcode version

Atharva Raykar <raykar.ath@gmail.com> writes:
Show 23 quoted lines
> Carlo Marcelo Arenas Belón <carenas@gmail.com> writes:
>
> For those of us using Homebrew and using the LLVM installation from
> there, we get:
>
> $ cc --version
> Homebrew clang version 12.0.1
> ...
>> diff --git a/detect-compiler b/detect-compiler
>> index 70b754481c..c85be83c64 100755
>> --- a/detect-compiler
>> +++ b/detect-compiler
>> @@ -44,7 +44,7 @@ clang)
>>  "FreeBSD clang")
>>  	print_flags clang
>>  	;;
>> -"Apple LLVM")
>> +"Apple LLVM"|"Apple clang")
>>  	print_flags clang
>>  	;;
>>  *)
>
> So maybe we could add another case for "Homebrew clang"?

$ clang --version 2>&1 | sed -ne 's/ version .*//p' Debian clang

It might be necessary to cope with this "$VENDOR clang version" convention better with something like the following.

I am afraid that this patch is being a bit too aggressive about LLVM, as I do not know if "$VENDOR LLVM version" is also a thing, or it is just oddity only at Apple, though.

Show changes to diff +1 −7
diff --git c/detect-compiler w/detect-compiler
index 70b754481c..a80442a327 100755
--- c/detect-compiler
+++ w/detect-compiler
@@ -38,13 +38,7 @@ case "$(get_family)" in
 gcc)
 	print_flags gcc
 	;;
-clang)
-	print_flags clang
-	;;
-"FreeBSD clang")
-	print_flags clang
-	;;
-"Apple LLVM")
+clang | *" clang" | *" LLVM")
 	print_flags clang
 	;;
 *)
Jeff King· Aug 6, 2021, 19:20 UTC · re: Junio C Hamano · lore

Re: [PATCH] makefile: update detect-compiler for newer Xcode version

On Fri, Aug 06, 2021 at 11:11:31AM -0700, Junio C Hamano wrote:
Show 7 quoted lines
> > So maybe we could add another case for "Homebrew clang"?
> 
> $ clang --version 2>&1 | sed -ne 's/ version .*//p'
> Debian clang
> 
> It might be necessary to cope with this "$VENDOR clang version"
> convention better with something like the following.

Good catch. Unfortunately your patch below isn't sufficient because get_family() is broken, too. :(

It insists on there being an extra word after the actual version. There is for gcc, but not for clang on my system:

  $ CC=gcc get_version_line
  gcc version 10.2.1 20210110 (Debian 10.2.1-6)
  $ CC=clang get_version_line
  Debian clang version 11.0.1-2

Doing this on top of your patch makes "./detect-compiler clang" behave as expected:

Show changes to detect-compiler +2 −3
diff --git a/detect-compiler b/detect-compiler
index a80442a327..fd388ae783 100755
--- a/detect-compiler
+++ b/detect-compiler
@@ -13,11 +13,11 @@ get_version_line() {
 }
 
 get_family() {
-	get_version_line | sed 's/^\(.*\) version [0-9][^ ]* .*/\1/'
+	get_version_line | sed 's/^\(.*\) version [0-9].*/\1/'
 }
 
 get_version() {
-	get_version_line | sed 's/^.* version \([0-9][^ ]*\) .*/\1/'
+	get_version_line | sed 's/^.* version \([0-9][^ ]*\).*/\1/'
 }
 
 print_flags() {

> I am afraid that this patch is being a bit too aggressive about
> LLVM, as I do not know if "$VENDOR LLVM version" is also a thing, or
> it is just oddity only at Apple, though.

I think even before this issue, all of the versioning of Apple's clang
is suspect. The "Apple LLVM" bit comes from Eric in:

  https://lore.kernel.org/git/20180318090607.GA26226@flurp.local/

But downthread we realized that the version numbers there don't match
the clang ones:

  https://lore.kernel.org/git/CAPig+cRQXQ_DowS2Dsc1x3TAGJjnWig7P4eYS4kQ+C2piAdSWA@mail.gmail.com/

Duy indicated in the cover letter:

  https://lore.kernel.org/git/20180324125348.6614-1-pclouds@gmail.com/

that the apple support was probably wrong, but it looks like nobody
stepped up in the meantime to fix it. In practice I think it mostly
works anyway because "clang4" is the only version check we have for
clang. So if we err "ahead" a few versions (which is what Apple's
scheme does), you only get bit if you have a pretty old version of
Xcode.

I think if we wanted to get it right, we'd need to encode into the
script some form of the version table here:

  https://en.wikipedia.org/wiki/Xcode#Xcode_11.x_-_13.x_%28since_SwiftUI_framework%29

-Peff
Junio C Hamano· Aug 6, 2021, 20:52 UTC · re: Jeff King · lore

[PATCH v2 0/3] detect-compiler: clang updates

So here is a mini-series that summarizes what has been suggested so far on the topic.

Carlo Marcelo Arenas Belón (1):
  build: update detect-compiler for newer Xcode version
Jeff King (1):
  build: clang version may not be followed by extra words
Junio C Hamano (1):
  build: catch clang that identifies itself as "$VENDOR clang"
 detect-compiler | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
-- 
2.33.0-rc1-159-gdd7297d7fa
Junio C Hamano· Aug 6, 2021, 20:52 UTC · re: Junio C Hamano · lore

[PATCH v2 1/3] build: update detect-compiler for newer Xcode version

From: Carlo Marcelo Arenas Belón <carenas@gmail.com>

1da1580e4c (Makefile: detect compiler and enable more warnings in DEVELOPER=1, 2018-04-14) uses the output of the compiler banner to detect the compiler family.

Apple had since changed the wording used to refer to its compiler as clang instead of LLVM as shown by:

  $ cc --version
  Apple clang version 12.0.5 (clang-1205.0.22.9)
  Target: x86_64-apple-darwin20.6.0
  Thread model: posix
  InstalledDir: /Library/Developer/CommandLineTools/usr/bin

so update the script to match, and allow DEVELOPER=1 to work as expected again in macOS.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 detect-compiler | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to detect-compiler +1 −1
diff --git a/detect-compiler b/detect-compiler
index 70b754481c..c85be83c64 100755
--- a/detect-compiler
+++ b/detect-compiler
@@ -44,7 +44,7 @@ clang)
 "FreeBSD clang")
 	print_flags clang
 	;;
-"Apple LLVM")
+"Apple LLVM"|"Apple clang")
 	print_flags clang
 	;;
 *)

-- 
2.33.0-rc1-159-gdd7297d7fa
Junio C Hamano· Aug 6, 2021, 20:52 UTC · re: Junio C Hamano · lore

[PATCH v2 2/3] build: clang version may not be followed by extra words

From: Jeff King <peff@peff.net>

The get_family and get_version helpers of detect-compiler assume that the line to identify the version from the compilers have a token "version", followed by the version number, followed by some other string, e.g.

  $ CC=gcc get_version_line
  gcc version 10.2.1 20210110 (Debian 10.2.1-6)
But that is not necessarily true, e.g.
  $ CC=clang get_version_line
  Debian clang version 11.0.1-2
Tweak the script not to require extra string after the version.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 detect-compiler | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to detect-compiler +2 −2
diff --git a/detect-compiler b/detect-compiler
index c85be83c64..955be1c906 100755
--- a/detect-compiler
+++ b/detect-compiler
@@ -13,11 +13,11 @@ get_version_line() {
 }
 
 get_family() {
-	get_version_line | sed 's/^\(.*\) version [0-9][^ ]* .*/\1/'
+	get_version_line | sed 's/^\(.*\) version [0-9].*/\1/'
 }
 
 get_version() {
-	get_version_line | sed 's/^.* version \([0-9][^ ]*\) .*/\1/'
+	get_version_line | sed 's/^.* version \([0-9][^ ]*\).*/\1/'
 }
 
 print_flags() {

-- 
2.33.0-rc1-159-gdd7297d7fa
Junio C Hamano· Aug 6, 2021, 20:52 UTC · re: Junio C Hamano · lore

[PATCH v2 3/3] build: catch clang that identifies itself as "$VENDOR clang"

The case statement in detect-compiler notices 'clang', 'FreeBSD clang' and 'Apple clang', but there are other platforms that follow the '$VENDOR clang' pattern (e.g. Debian).

Generalize the pattern to catch them.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 detect-compiler | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
Show changes to detect-compiler +2 −5
diff --git a/detect-compiler b/detect-compiler
index 955be1c906..11d60da5b7 100755
--- a/detect-compiler
+++ b/detect-compiler
@@ -38,13 +38,10 @@ case "$(get_family)" in
 gcc)
 	print_flags gcc
 	;;
-clang)
+clang | *" clang")
 	print_flags clang
 	;;
-"FreeBSD clang")
-	print_flags clang
-	;;
-"Apple LLVM"|"Apple clang")
+"Apple LLVM")
 	print_flags clang
 	;;
 *)

-- 
2.33.0-rc1-159-gdd7297d7fa
Jeff King· Aug 7, 2021, 02:09 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 3/3] build: catch clang that identifies itself as "$VENDOR clang"

On Fri, Aug 06, 2021 at 01:52:35PM -0700, Junio C Hamano wrote:
Show 19 quoted lines
> diff --git a/detect-compiler b/detect-compiler
> index 955be1c906..11d60da5b7 100755
> --- a/detect-compiler
> +++ b/detect-compiler
> @@ -38,13 +38,10 @@ case "$(get_family)" in
>  gcc)
>  	print_flags gcc
>  	;;
> -clang)
> +clang | *" clang")
>  	print_flags clang
>  	;;
> -"FreeBSD clang")
> -	print_flags clang
> -	;;
> -"Apple LLVM"|"Apple clang")
> +"Apple LLVM")
>  	print_flags clang
>  	;;

All three patches look fine to me, and functionality-wise are a strict improvement over the status quo. But I suspect in the long run we'd need to keep all of the Apple bits in their own case-arm, like:

  # this must come first, so we prefer it over "* clang".
  "Apple LLVM" | "Apple clang")
	print_apple_magic
        ;;
  clang | *" clang")
        print_flags clang
        ;;

and then apple_magic does the version conversion from Wikipedia I linked to earlier.

I don't think your patch is really making it significantly harder to get there, though splitting up "Apple LLVM" and "Apple clang" feels a bit like it's the wrong direction.

I wasn't personally planning to take that next step, as I lack the platform to test it on. And as noted, unless you have a pretty old version of Xcode, it doesn't matter either way (so I'm content to leave it until dev with a mac is bitten by it and cares enough to make it more accurate).

-Peff
Ævar Arnfjörð Bjarmason· Aug 7, 2021, 02:02 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Fri, Aug 06 2021, Junio C Hamano wrote:
Show 14 quoted lines
> So here is a mini-series that summarizes what has been suggested so
> far on the topic.
>
> Carlo Marcelo Arenas Belón (1):
>   build: update detect-compiler for newer Xcode version
>
> Jeff King (1):
>   build: clang version may not be followed by extra words
>
> Junio C Hamano (1):
>   build: catch clang that identifies itself as "$VENDOR clang"
>
>  detect-compiler | 9 +++------
>  1 file changed, 3 insertions(+), 6 deletions(-)

Perhaps I've missed some obvious reason not to do this, but why are we parsing the --version output of two modern compilers, as opposed to just asking them what type/version they are via their usual macro facilities? I.e. something like the below:

Show changes to detect-compiler +35 −13
diff --git a/detect-compiler b/detect-compiler
index 70b754481c8..37908ac9ea8 100755
--- a/detect-compiler
+++ b/detect-compiler
@@ -5,19 +5,47 @@
 
 CC="$*"
 
-# we get something like (this is at least true for gcc and clang)
+#!/bin/sh
 #
-# FreeBSD clang version 3.4.1 (tags/RELEASE...)
-get_version_line() {
-	$CC -v 2>&1 | grep ' version '
-}
+# Probe the compiler for vintage, version, etc. This is used for setting
+# optional make knobs under the DEVELOPER knob.
+
+CC="$*"
+
+v=$($CC -E - <<-EOF 2>&1 | grep -e '=')
+GNUC=__GNUC__
+GNUC_MINOR=__GNUC_MINOR__
+GNUC_PATCHLEVEL=__GNUC_PATCHLEVEL__
+clang=__clang__
+clang_major=__clang_major__
+clang_minor=__clang_minor__
+clang_patchlevel=__clang_patchlevel__
+EOF
+eval "$v"
 
 get_family() {
-	get_version_line | sed 's/^\(.*\) version [0-9][^ ]* .*/\1/'
+	# Clang also sets the GNUC macros, but GCC does not set
+	# clang's.
+	if test "$clang" != "__clang__"
+	then
+		echo clang
+	elif test "$GNUC" != "__GNUC__"
+	then
+		echo gcc
+	else
+		echo unknown
+	fi
 }
 
 get_version() {
-	get_version_line | sed 's/^.* version \([0-9][^ ]*\) .*/\1/'
+	case "$(get_family)" in
+	clang)
+		echo "$clang_major.$clang_minor.$clang_patchlevel"
+		;;
+	gcc)
+		echo "$GNUC.$GNUC_MINOR.$GNUC_PATCHLEVEL"
+		;;
+	esac
 }
 
 print_flags() {
@@ -41,12 +69,6 @@ gcc)
 clang)
 	print_flags clang
 	;;
-"FreeBSD clang")
-	print_flags clang
-	;;
-"Apple LLVM")
-	print_flags clang
-	;;
 *)
 	: unknown compiler family
 	;;
Jeff King· Aug 7, 2021, 02:15 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07, 2021 at 04:02:45AM +0200, Ævar Arnfjörð Bjarmason wrote:
> Perhaps I've missed some obvious reason not to do this, but why are we
> parsing the --version output of two modern compilers, as opposed to just
> asking them what type/version they are via their usual macro facilities?
> I.e. something like the below:

That would probably work OK in practice, but it actually seems more complex to me (how do other random compilers react to "-E -"? Is it possible for us to get other output from the preprocessor that would confuse an eval?).

It could perhaps make the Apple version-string confusion go away, though, which might make it worth doing.

-Peff
Ævar Arnfjörð Bjarmason· Aug 7, 2021, 02:56 UTC · re: Jeff King · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Fri, Aug 06 2021, Jeff King wrote:
Show 9 quoted lines
> On Sat, Aug 07, 2021 at 04:02:45AM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> Perhaps I've missed some obvious reason not to do this, but why are we
>> parsing the --version output of two modern compilers, as opposed to just
>> asking them what type/version they are via their usual macro facilities?
>> I.e. something like the below:
>
> That would probably work OK in practice, but it actually seems more
> complex to me (how do other random compilers react to "-E -"?

We only care about gcc and clang in that script, which I think have supported that form of "-E" on stdin input for any version we're likely to care about for the purposes of config.mak.dev. It seems unlikely that we'll care about non-modern compilers in config.mak.dev, so using more modern features there seems fine (it's all for opting us into even more modern warning flags and the like...).

> Is it possible for us to get other output from the preprocessor that
> would confuse an eval?).

Probably, I just meant that as a POC. We could pipe it into some awk/grep/cut/perl or whatever that would be more strict.

Jeff King· Aug 7, 2021, 14:13 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07, 2021 at 04:56:04AM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 16 quoted lines
> > On Sat, Aug 07, 2021 at 04:02:45AM +0200, Ævar Arnfjörð Bjarmason wrote:
> >
> >> Perhaps I've missed some obvious reason not to do this, but why are we
> >> parsing the --version output of two modern compilers, as opposed to just
> >> asking them what type/version they are via their usual macro facilities?
> >> I.e. something like the below:
> >
> > That would probably work OK in practice, but it actually seems more
> > complex to me (how do other random compilers react to "-E -"?
> 
> We only care about gcc and clang in that script, which I think have
> supported that form of "-E" on stdin input for any version we're likely
> to care about for the purposes of config.mak.dev. It seems unlikely that
> we'll care about non-modern compilers in config.mak.dev, so using more
> modern features there seems fine (it's all for opting us into even more
> modern warning flags and the like...).

Yeah, but we don't find out what we have until we run the script in question. I guess it is OK as long as we redirect stderr, ignore the exit code, and only look for a positive outcome in the output (your patch does the latter two already).

I also wondered how this might interact with CC="ccache gcc" (where caching might fail to notice version changes). But from some quick testing, it looks like it doesn't cache in this case (neither stdin, nor with -E).

Show 5 quoted lines
> > Is it possible for us to get other output from the preprocessor that
> > would confuse an eval?).
> 
> Probably, I just meant that as a POC. We could pipe it into some
> awk/grep/cut/perl or whatever that would be more strict.

That would probably be better. I would be curious to hear from somebody with a mac if this technique gives more sensible version numbers for the Apple-clang compiler.

-Peff
Ævar Arnfjörð Bjarmason· Aug 7, 2021, 14:26 UTC · re: Jeff King · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07 2021, Jeff King wrote:
Show 38 quoted lines
> On Sat, Aug 07, 2021 at 04:56:04AM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> > On Sat, Aug 07, 2021 at 04:02:45AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> >
>> >> Perhaps I've missed some obvious reason not to do this, but why are we
>> >> parsing the --version output of two modern compilers, as opposed to just
>> >> asking them what type/version they are via their usual macro facilities?
>> >> I.e. something like the below:
>> >
>> > That would probably work OK in practice, but it actually seems more
>> > complex to me (how do other random compilers react to "-E -"?
>> 
>> We only care about gcc and clang in that script, which I think have
>> supported that form of "-E" on stdin input for any version we're likely
>> to care about for the purposes of config.mak.dev. It seems unlikely that
>> we'll care about non-modern compilers in config.mak.dev, so using more
>> modern features there seems fine (it's all for opting us into even more
>> modern warning flags and the like...).
>
> Yeah, but we don't find out what we have until we run the script in
> question. I guess it is OK as long as we redirect stderr, ignore the
> exit code, and only look for a positive outcome in the output (your
> patch does the latter two already).
>
> I also wondered how this might interact with CC="ccache gcc" (where
> caching might fail to notice version changes). But from some quick
> testing, it looks like it doesn't cache in this case (neither stdin, nor
> with -E).
>
>> > Is it possible for us to get other output from the preprocessor that
>> > would confuse an eval?).
>> 
>> Probably, I just meant that as a POC. We could pipe it into some
>> awk/grep/cut/perl or whatever that would be more strict.
>
> That would probably be better. I would be curious to hear from somebody
> with a mac if this technique gives more sensible version numbers for the
> Apple-clang compiler.
It does, on the gcc304 box on the gccfarm (recent apple M1 Mac Mini):
    avar@minimac ~ % uname -a
    Darwin minimac.moose.housegordon.com 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021; root:xnu-7195.101.2~1/RELEASE_ARM64_T8101 arm64    
    avar@minimac ~ % clang --version
    Apple clang version 12.0.5 (clang-1205.0.22.9)
    Target: arm64-apple-darwin20.4.0
    Thread model: posix
    InstalledDir: /Library/Developer/CommandLineTools/usr/bin
    avar@minimac ~ % cat >f  
    GNUC=__GNUC__
    GNUC_MINOR=__GNUC_MINOR__
    GNUC_PATCHLEVEL=__GNUC_PATCHLEVEL__
    clang=__clang__
    clang_major=__clang_major__
    clang_minor=__clang_minor__
    clang_patchlevel=__clang_patchlevel__
    
    ^C
    avar@minimac ~ % clang -E - <f
    # 1 "<stdin>"
    # 1 "<built-in>" 1
    # 1 "<built-in>" 3
    # 384 "<built-in>" 3
    # 1 "<command line>" 1
    # 1 "<built-in>" 2
    # 1 "<stdin>" 2
    GNUC=4
    GNUC_MINOR=2
    GNUC_PATCHLEVEL=1
    clang=1
    clang_major=12
    clang_minor=0
    clang_patchlevel=5

I think nobody who's using clang derivatives is screwing with these macro variables, they're just changing whatever the "product name" or whatever is in the --version output.

Jeff King· Aug 7, 2021, 14:40 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07, 2021 at 04:26:33PM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 40 quoted lines
> > That would probably be better. I would be curious to hear from somebody
> > with a mac if this technique gives more sensible version numbers for the
> > Apple-clang compiler.
> 
> It does, on the gcc304 box on the gccfarm (recent apple M1 Mac Mini):
> 
>     avar@minimac ~ % uname -a
>     Darwin minimac.moose.housegordon.com 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021; root:xnu-7195.101.2~1/RELEASE_ARM64_T8101 arm64    
>     avar@minimac ~ % clang --version
>     Apple clang version 12.0.5 (clang-1205.0.22.9)
>     Target: arm64-apple-darwin20.4.0
>     Thread model: posix
>     InstalledDir: /Library/Developer/CommandLineTools/usr/bin
> 
>     avar@minimac ~ % cat >f  
>     GNUC=__GNUC__
>     GNUC_MINOR=__GNUC_MINOR__
>     GNUC_PATCHLEVEL=__GNUC_PATCHLEVEL__
>     clang=__clang__
>     clang_major=__clang_major__
>     clang_minor=__clang_minor__
>     clang_patchlevel=__clang_patchlevel__
>     
>     ^C
> 
>     avar@minimac ~ % clang -E - <f
>     # 1 "<stdin>"
>     # 1 "<built-in>" 1
>     # 1 "<built-in>" 3
>     # 384 "<built-in>" 3
>     # 1 "<command line>" 1
>     # 1 "<built-in>" 2
>     # 1 "<stdin>" 2
>     GNUC=4
>     GNUC_MINOR=2
>     GNUC_PATCHLEVEL=1
>     clang=1
>     clang_major=12
>     clang_minor=0
>     clang_patchlevel=5

Hmm, now I'm really confused, though. Is that really clang 12 (for which there is no 12.0.5; 12.0.1 is the latest version, shipped in July)? Or is it XCode 12, shipping with LLVM 11, according to the table in:

  https://en.wikipedia.org/wiki/Xcode#Xcode_11.x_-_13.x_(since_SwiftUI_framework)

(sorry, there are actually _two_ tables with that same anchor on the page; the one you want is the second one, under "Toolchain versions").

The distinction does not matter for our script (where we only care about "clang4" and up). I guess the most relevant test would be to get XCode 8.x and see what it says. I expect it to claim "clang 8.1.0" or similar, but actually be clang-3. And therefore not support -Wtautological-constant-out-of-range-compare.

If we can't get easily get hold of such a platform, then maybe that is a good indication that this conversation is too academic for now, and we should wait until somebody wants to add a more recent version-specifier to config.mak.dev. ;)

-Peff
Ævar Arnfjörð Bjarmason· Aug 7, 2021, 15:36 UTC · re: Jeff King · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07 2021, Jeff King wrote:
Show 64 quoted lines
> On Sat, Aug 07, 2021 at 04:26:33PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> > That would probably be better. I would be curious to hear from somebody
>> > with a mac if this technique gives more sensible version numbers for the
>> > Apple-clang compiler.
>> 
>> It does, on the gcc304 box on the gccfarm (recent apple M1 Mac Mini):
>> 
>>     avar@minimac ~ % uname -a
>>     Darwin minimac.moose.housegordon.com 20.4.0 Darwin Kernel
>> Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021;
>> root:xnu-7195.101.2~1/RELEASE_ARM64_T8101 arm64
>>     avar@minimac ~ % clang --version
>>     Apple clang version 12.0.5 (clang-1205.0.22.9)
>>     Target: arm64-apple-darwin20.4.0
>>     Thread model: posix
>>     InstalledDir: /Library/Developer/CommandLineTools/usr/bin
>> 
>>     avar@minimac ~ % cat >f  
>>     GNUC=__GNUC__
>>     GNUC_MINOR=__GNUC_MINOR__
>>     GNUC_PATCHLEVEL=__GNUC_PATCHLEVEL__
>>     clang=__clang__
>>     clang_major=__clang_major__
>>     clang_minor=__clang_minor__
>>     clang_patchlevel=__clang_patchlevel__
>>     
>>     ^C
>> 
>>     avar@minimac ~ % clang -E - <f
>>     # 1 "<stdin>"
>>     # 1 "<built-in>" 1
>>     # 1 "<built-in>" 3
>>     # 384 "<built-in>" 3
>>     # 1 "<command line>" 1
>>     # 1 "<built-in>" 2
>>     # 1 "<stdin>" 2
>>     GNUC=4
>>     GNUC_MINOR=2
>>     GNUC_PATCHLEVEL=1
>>     clang=1
>>     clang_major=12
>>     clang_minor=0
>>     clang_patchlevel=5
>
> Hmm, now I'm really confused, though. Is that really clang 12 (for which
> there is no 12.0.5; 12.0.1 is the latest version, shipped in July)? Or
> is it XCode 12, shipping with LLVM 11, according to the table in:
>
>   https://en.wikipedia.org/wiki/Xcode#Xcode_11.x_-_13.x_(since_SwiftUI_framework)
>
> (sorry, there are actually _two_ tables with that same anchor on the
> page; the one you want is the second one, under "Toolchain versions").
>
> The distinction does not matter for our script (where we only care about
> "clang4" and up). I guess the most relevant test would be to get XCode
> 8.x and see what it says. I expect it to claim "clang 8.1.0" or similar,
> but actually be clang-3. And therefore not support
> -Wtautological-constant-out-of-range-compare.
>
> If we can't get easily get hold of such a platform, then maybe that is a
> good indication that this conversation is too academic for now, and we
> should wait until somebody wants to add a more recent version-specifier
> to config.mak.dev. ;)

I think it's clang 12.0.5, and Apple just takes upstream versions and increments them, e.g. I found this: https://gist.github.com/yamaya/2924292

So you can presumably rely on it for having clang 12 features, and we'd only ever care about the clang_major...

Jeff King· Aug 9, 2021, 18:10 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07, 2021 at 05:36:57PM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 26 quoted lines
> > Hmm, now I'm really confused, though. Is that really clang 12 (for which
> > there is no 12.0.5; 12.0.1 is the latest version, shipped in July)? Or
> > is it XCode 12, shipping with LLVM 11, according to the table in:
> >
> >   https://en.wikipedia.org/wiki/Xcode#Xcode_11.x_-_13.x_(since_SwiftUI_framework)
> >
> > (sorry, there are actually _two_ tables with that same anchor on the
> > page; the one you want is the second one, under "Toolchain versions").
> >
> > The distinction does not matter for our script (where we only care about
> > "clang4" and up). I guess the most relevant test would be to get XCode
> > 8.x and see what it says. I expect it to claim "clang 8.1.0" or similar,
> > but actually be clang-3. And therefore not support
> > -Wtautological-constant-out-of-range-compare.
> >
> > If we can't get easily get hold of such a platform, then maybe that is a
> > good indication that this conversation is too academic for now, and we
> > should wait until somebody wants to add a more recent version-specifier
> > to config.mak.dev. ;)
> 
> I think it's clang 12.0.5, and Apple just takes upstream versions and
> increments them, e.g. I found this:
> https://gist.github.com/yamaya/2924292
> 
> So you can presumably rely on it for having clang 12 features, and we'd
> only ever care about the clang_major...

From the reading I've done, I'm unconvinced that this is _actually_ clang 12, and they aren't simply doing funky things with the version numbers. But since we lack an easy test of how the different versions behave (even my suggested clang4 test turned out not to be robust!), it probably doesn't matter much either way.

-Peff
Carlo Arenas· Aug 8, 2021, 00:30 UTC · re: Jeff King · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 7, 2021 at 7:40 AM Jeff King <peff@peff.net> wrote:
Show 5 quoted lines
> The distinction does not matter for our script (where we only care about
> "clang4" and up). I guess the most relevant test would be to get XCode
> 8.x and see what it says. I expect it to claim "clang 8.1.0" or similar,
> but actually be clang-3. And therefore not support
> -Wtautological-constant-out-of-range-compare.

uses Xcode 7.3 (based on clang 3.8) and either does support that flag or ignores it silently

  https://www.travis-ci.com/github/carenas/git/builds/234772346
the same was observed with Xcode 8

both error later and fail to build because of a valid (but harmless) -Wformat-extra-args warning that doesn't trigger in later versions

Carlo
Jeff King· Aug 9, 2021, 18:08 UTC · re: Carlo Arenas · lore

Re: [PATCH v2 0/3] detect-compiler: clang updates

On Sat, Aug 07, 2021 at 05:30:39PM -0700, Carlo Arenas wrote:
Show 16 quoted lines
> On Sat, Aug 7, 2021 at 7:40 AM Jeff King <peff@peff.net> wrote:
> > The distinction does not matter for our script (where we only care about
> > "clang4" and up). I guess the most relevant test would be to get XCode
> > 8.x and see what it says. I expect it to claim "clang 8.1.0" or similar,
> > but actually be clang-3. And therefore not support
> > -Wtautological-constant-out-of-range-compare.
> 
> uses Xcode 7.3 (based on clang 3.8) and either does support that flag
> or ignores it silently
> 
>   https://www.travis-ci.com/github/carenas/git/builds/234772346
> 
> the same was observed with Xcode 8
> 
> both error later and fail to build because of a valid (but harmless)
> -Wformat-extra-args warning that doesn't trigger in later versions

Thanks for testing. I think I was wrong that clang4 is the limit for that option, though. It comes originally from:

  https://lore.kernel.org/git/20180317160832.GB15772@sigill.intra.peff.net/

where clang4 just happened to be the oldest thing I had access to at the time, so we used that as a minimum. So probably all of our "clang4" could really be "any clang" (but it is probably OK to leave it as-is).

-Peff

← back to recent threads