threads / patch / 58265

patchtests: cache glibc version check

Subject: [PATCH] tests: cache glibc version check

## tl;dr

6 messages between Aug 4, 2022 and Aug 4, 2022. Diffs are folded; open one to read it.

replies: 5people: 2as markdown or json

Phillip Wood via GitGitGadget· Aug 4, 2022, 13:38 UTC · lore
From: Phillip Wood <phillip.wood@dunelm.org.uk>

131b94a10a ("test-lib.sh: Use GLIBC_TUNABLES instead of MALLOC_CHECK_ on glibc >= 2.34", 2022-03-04) introduced a check for the version of glibc that is in use. This check is performed as part of setup_malloc_check() which is called at least once for each test. As the test involves forking `getconf` and `expr` cache the result and use that within setup_malloc_check() to avoid forking these extra processes for each test.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---
    tests: cache glibc version check
    
    A recent discussion on the list[1] reminded me that this patch was
    waiting to be sent.
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1311%2Fphillipwood%2Fwip%2Ftest-cache-glibc-tunables-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1311/phillipwood/wip/test-cache-glibc-tunables-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1311
 t/test-lib.sh | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
Show changes to t/test-lib.sh +8 −3
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 7726d1da88a..ad81c78fce7 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -557,14 +557,19 @@ then
 		: nothing
 	}
 else
+	_USE_GLIBC_TUNABLES=
+	if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
+	   _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
+	   expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
+	then
+		_USE_GLIBC_TUNABLES=YesPlease
+	fi
 	setup_malloc_check () {
 		local g
 		local t
 		MALLOC_CHECK_=3	MALLOC_PERTURB_=165
 		export MALLOC_CHECK_ MALLOC_PERTURB_
-		if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
-		   _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
-		   expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
+		if test -n "$_USE_GLIBC_TUNABLES"
 		then
 			g=
 			LD_PRELOAD="libc_malloc_debug.so.0"

base-commit: 23b219f8e3f2adfb0441e135f0a880e6124f766c
-- 
gitgitgadget
Phillip Wood· Aug 4, 2022, 14:26 UTC · re: Phillip Wood via GitGitGadget · lore

Re: [PATCH] tests: cache glibc version check

On 04/08/2022 14:38, Phillip Wood via GitGitGadget wrote:
Show 16 quoted lines
> From: Phillip Wood <phillip.wood@dunelm.org.uk>
> 
> 131b94a10a ("test-lib.sh: Use GLIBC_TUNABLES instead of MALLOC_CHECK_
> on glibc >= 2.34", 2022-03-04) introduced a check for the version of
> glibc that is in use. This check is performed as part of
> setup_malloc_check() which is called at least once for each test. As
> the test involves forking `getconf` and `expr` cache the result and
> use that within setup_malloc_check() to avoid forking these extra
> processes for each test.
> 
> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> ---
>      tests: cache glibc version check
>      
>      A recent discussion on the list[1] reminded me that this patch was
>      waiting to be sent
[1] https://lore.kernel.org/git/YuL7EotrIpnOn5BT@coredump.intra.peff.net/
Show 36 quoted lines
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1311%2Fphillipwood%2Fwip%2Ftest-cache-glibc-tunables-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1311/phillipwood/wip/test-cache-glibc-tunables-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1311
> 
>   t/test-lib.sh | 11 ++++++++---
>   1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 7726d1da88a..ad81c78fce7 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -557,14 +557,19 @@ then
>   		: nothing
>   	}
>   else
> +	_USE_GLIBC_TUNABLES=
> +	if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
> +	   _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
> +	   expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
> +	then
> +		_USE_GLIBC_TUNABLES=YesPlease
> +	fi
>   	setup_malloc_check () {
>   		local g
>   		local t
>   		MALLOC_CHECK_=3	MALLOC_PERTURB_=165
>   		export MALLOC_CHECK_ MALLOC_PERTURB_
> -		if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
> -		   _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
> -		   expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
> +		if test -n "$_USE_GLIBC_TUNABLES"
>   		then
>   			g=
>   			LD_PRELOAD="libc_malloc_debug.so.0"
> 
> base-commit: 23b219f8e3f2adfb0441e135f0a880e6124f766c
Junio C Hamano· Aug 4, 2022, 18:08 UTC · re: Phillip Wood via GitGitGadget · lore

Re: [PATCH] tests: cache glibc version check

"Phillip Wood via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 51 quoted lines
> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>
> 131b94a10a ("test-lib.sh: Use GLIBC_TUNABLES instead of MALLOC_CHECK_
> on glibc >= 2.34", 2022-03-04) introduced a check for the version of
> glibc that is in use. This check is performed as part of
> setup_malloc_check() which is called at least once for each test. As
> the test involves forking `getconf` and `expr` cache the result and
> use that within setup_malloc_check() to avoid forking these extra
> processes for each test.
>
> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> ---
>     tests: cache glibc version check
>     
>     A recent discussion on the list[1] reminded me that this patch was
>     waiting to be sent.
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1311%2Fphillipwood%2Fwip%2Ftest-cache-glibc-tunables-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1311/phillipwood/wip/test-cache-glibc-tunables-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1311
>
>  t/test-lib.sh | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 7726d1da88a..ad81c78fce7 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -557,14 +557,19 @@ then
>  		: nothing
>  	}
>  else
> +	_USE_GLIBC_TUNABLES=
> +	if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
> +	   _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
> +	   expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
> +	then
> +		_USE_GLIBC_TUNABLES=YesPlease
> +	fi
>  	setup_malloc_check () {
>  		local g
>  		local t
>  		MALLOC_CHECK_=3	MALLOC_PERTURB_=165
>  		export MALLOC_CHECK_ MALLOC_PERTURB_
> -		if _GLIBC_VERSION=$(getconf GNU_LIBC_VERSION 2>/dev/null) &&
> -		   _GLIBC_VERSION=${_GLIBC_VERSION#"glibc "} &&
> -		   expr 2.34 \<= "$_GLIBC_VERSION" >/dev/null
> +		if test -n "$_USE_GLIBC_TUNABLES"
>  		then
>  			g=
>  			LD_PRELOAD="libc_malloc_debug.so.0"

Between USE_LIBC_MALLOC_DEBUG, which is the name Peff originally gave this intermediate variable, and the one you use here, I am undecided. If the only thing the GLIBC_TUNABLES mechanism can do were to tweak the malloc checking, then both names are good, but that is not the case. We are only seeing if we are going to use the malloc check feature given by glibc here, so the original name feels more to the point, and use of GLIBC_TUNABLE mechanism to trigger that malloc check feature is a mere implementation detail.

But that is minor. Let's queue the patch to help me not to forget about it, and we'll amend it if necessary, as we'd probably need a helped-by or signed-off-by from Peff anyway before this hits 'next'.

Thanks.
Phillip Wood· Aug 4, 2022, 19:16 UTC · re: Junio C Hamano · lore

Re: [PATCH] tests: cache glibc version check

Hi Junio
On 04/08/2022 19:08, Junio C Hamano wrote:
> "Phillip Wood via GitGitGadget" <gitgitgadget@gmail.com> writes:
 >
Show 12 quoted lines
> Between USE_LIBC_MALLOC_DEBUG, which is the name Peff originally
> gave this intermediate variable, and the one you use here, I am
> undecided.  If the only thing the GLIBC_TUNABLES mechanism can do
> were to tweak the malloc checking, then both names are good, but
> that is not the case.  We are only seeing if we are going to use the
> malloc check feature given by glibc here, so the original name feels
> more to the point, and use of GLIBC_TUNABLE mechanism to trigger
> that malloc check feature is a mere implementation detail.
> 
> But that is minor.  Let's queue the patch to help me not to forget
> about it, and we'll amend it if necessary, as we'd probably need a
> helped-by or signed-off-by from Peff anyway before this hits 'next'.

Oh, sorry I'd missed that message where Peff posted essentially the same patch. I wrote this at the same time as 067109a5e7 ("tests: make SANITIZE=address imply TEST_NO_MALLOC_CHECK", 2022-04-09) but did not post in then as we were in a rc period and then forgot about it. Having just read Peff's message this does not make much difference to the test timings and if I'd seen that before I wouldn't have posted this.

As for the variable name I don't mind particularly either way, I chose this name as the variable is checking whether we should use the glibc tunables mechanism or not.

Best Wishes
Phillip
> Thanks.
Junio C Hamano· Aug 4, 2022, 20:20 UTC · re: Phillip Wood · lore

Re: [PATCH] tests: cache glibc version check

Phillip Wood <phillip.wood123@gmail.com> writes:
> it. Having just read Peff's message this does not make much difference
> to the test timings and if I'd seen that before I wouldn't have posted
> this.

Ah, I forgot about that conclusion, i.e. while this does look the right thing to do, it didn't make much difference in practice.

It still is tempting to take it, though :-), if only to stop people sending in the same patch in the future without benchmarking, as long as it does not make it worse.

Thanks.
Junio C Hamano· Aug 4, 2022, 20:41 UTC · re: Phillip Wood · lore

Re: [PATCH] tests: cache glibc version check

Phillip Wood <phillip.wood123@gmail.com> writes:
> As for the variable name I don't mind particularly either way, I chose
> this name as the variable is checking whether we should use the glibc
> tunables mechanism or not.

Yup, but that becomes awkward when we decide to use the tunabules mechanism for something other than malloc debugging, and that is where my "is this the right name?" comes from.

Thanks.

← back to recent threads