threads / patch / 24698

patchgit-notes: Run partial expensive test everywhere

Subject: [PATCH] git-notes: Run partial expensive test everywhere

## tl;dr

9 messages between Aug 10, 2010 and Aug 29, 2010. Diffs are folded; open one to read it.

replies: 8people: 4as markdown or json

Ævar Arnfjörð Bjarmason· Aug 10, 2010, 19:56 UTC · lore

The git-notes expensive timing test is only expensive because it either did 1k iterations or nothing. Change it to do 10 by default, with an option to run the expensive version with the old GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Since nobody was ostensibly running this test under TAP the code had bitrotted so that it emitted invalid TAP. This change fixes that.

The old version would also mysteriously fail on systems without /usr/bin/time, there's now a check for that using the test prerequisite facility.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3302-notes-index-expensive.sh |   39 ++++++++++++++++++++++++++-----------
 1 files changed, 27 insertions(+), 12 deletions(-)
Show changes to t/t3302-notes-index-expensive.sh +27 −12
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 361a10a..3b8313c 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -7,11 +7,16 @@ test_description='Test commit notes index (expensive!)'
 
 . ./test-lib.sh
 
-test -z "$GIT_NOTES_TIMING_TESTS" && {
-	skip_all="Skipping timing tests"
-	test_done
-	exit
-}
+test_set_prereq NOT_EXPENSIVE
+test -n "$GIT_NOTES_TIMING_TESTS" && test_set_prereq EXPENSIVE
+
+if test -x /usr/bin/time
+then
+	# Hack around multiple test prerequisites not supporting AND-ing
+	# of terms
+	test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
+	test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
+fi
 
 create_repo () {
 	number_of_commits=$1
@@ -102,17 +107,27 @@ time_notes () {
 	done
 }
 
-for count in 10 100 1000 10000; do
+do_tests () {
+	pr=$1
+	count=$2
+
+	test_expect_success $pr 'setup / mkdir' '
+		mkdir $count &&
+		cd $count
+	'
 
-	mkdir $count
-	(cd $count;
+	test_expect_success $pr "setup $count" "create_repo $count"
 
-	test_expect_success "setup $count" "create_repo $count"
+	test_expect_success $pr 'notes work' "test_notes $count"
 
-	test_expect_success 'notes work' "test_notes $count"
+	test_expect_success USR_BIN_TIME+$pr 'notes timing with /usr/bin/time' "time_notes 100"
+
+	test_expect_success $pr 'teardown / cd ..' 'cd ..'
+}
 
-	test_expect_success 'notes timing' "time_notes 100"
-	)
+do_tests NOT_EXPENSIVE 10
+for count in 100 1000 10000; do
+	do_tests EXPENSIVE $count
 done
 
 test_done
-- 
1.7.2.1.295.gd03d
Sverre Rabbelier· Aug 10, 2010, 20:29 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH] git-notes: Run partial expensive test everywhere

Heya,
On Tue, Aug 10, 2010 at 14:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> The git-notes expensive timing test is only expensive because it
> either did 1k iterations or nothing. Change it to do 10 by default,
> with an option to run the expensive version with the old
> GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.
Nice, why 10 though? Any motivation for that particular value?
> Since nobody was ostensibly running this test under TAP the code had
> bitrotted so that it emitted invalid TAP. This change fixes that.
Nice catch.
> The old version would also mysteriously fail on systems without
> /usr/bin/time, there's now a check for that using the test
> prerequisite facility.
Should this patch be split up?
-- 
Cheers,

Sverre Rabbelier
Ævar Arnfjörð Bjarmason· Aug 10, 2010, 21:00 UTC · re: Sverre Rabbelier · lore

Re: [PATCH] git-notes: Run partial expensive test everywhere

On Tue, Aug 10, 2010 at 20:29, Sverre Rabbelier <srabbelier@gmail.com> wrote:
Show 9 quoted lines
> Heya,
>
> On Tue, Aug 10, 2010 at 14:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> The git-notes expensive timing test is only expensive because it
>> either did 1k iterations or nothing. Change it to do 10 by default,
>> with an option to run the expensive version with the old
>> GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.
>
> Nice, why 10 though? Any motivation for that particular value?

The old version had "for count in 10 100 1000 10000; do". Mine has 10 as non-expensive, and "for count in 100 1000 10000; do" as expensive.

I.e. I'm running the first test batch from the old tests.

I have no idea whether it actually needs to run 10..10k times, I didn't try to grok the actual test code.

Show 5 quoted lines
>> The old version would also mysteriously fail on systems without
>> /usr/bin/time, there's now a check for that using the test
>> prerequisite facility.
>
> Should this patch be split up?
It all touched the same bits, it'd be nastier to split it up IMO.
Ævar Arnfjörð Bjarmason· Aug 10, 2010, 21:56 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH] git-notes: Run partial expensive test everywhere

On Tue, Aug 10, 2010 at 19:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
Show 5 quoted lines
> +       # Hack around multiple test prerequisites not supporting AND-ing
> +       # of terms
> +       test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
> +       test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
> +fi

In retrospect this may have been some brainfried code, I'll check it out tomorrow.

Ævar Arnfjörð Bjarmason· Aug 10, 2010, 23:37 UTC · re: Ævar Arnfjörð Bjarmason · lore

[PATCH v2] git-notes: Run partial expensive test everywhere

The git-notes expensive timing test is only expensive because it either did 10,100,1k and 10k iterations or nothing.

Change it to do 10 by default, with an option to run the expensive version with the old GIT_NOTES_TIMING_TESTS=ZomgYesPlease variable.

Since nobody was ostensibly running this test under TAP the code had bitrotted so that it emitted invalid TAP. This change fixes that.

The old version would also mysteriously fail on systems without /usr/bin/time, there's now a check for that using the multiple test prerequisite facility.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
On Tue, Aug 10, 2010 at 21:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
Show 9 quoted lines
> On Tue, Aug 10, 2010 at 19:56, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> +       # Hack around multiple test prerequisites not supporting AND-ing
>> +       # of terms
>> +       test_set_prereq USR_BIN_TIME+NOT_EXPENSIVE
>> +       test_have_prereq EXPENSIVE && test_set_prereq USR_BIN_TIME+EXPENSIVE
>> +fi
>
> In retrospect this may have been some brainfried code, I'll check it
> out tomorrow.

Here's a patch that's not crazy. In v1 I was hacking around not having a facility I already added to the test-lib (tired).

This patch goes on top of my "test-lib: Multi-prereq support only checked the last prereq" patch, which fixes up the test prereq facility so that it actually works.

 t/t3302-notes-index-expensive.sh |   32 ++++++++++++++++++++------------
 1 files changed, 20 insertions(+), 12 deletions(-)
Show changes to t/t3302-notes-index-expensive.sh +20 −12
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 361a10a..7c08e99 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -7,11 +7,9 @@ test_description='Test commit notes index (expensive!)'
 
 . ./test-lib.sh
 
-test -z "$GIT_NOTES_TIMING_TESTS" && {
-	skip_all="Skipping timing tests"
-	test_done
-	exit
-}
+test_set_prereq NOT_EXPENSIVE
+test -n "$GIT_NOTES_TIMING_TESTS" && test_set_prereq EXPENSIVE
+test -x /usr/bin/time && test_set_prereq USR_BIN_TIME
 
 create_repo () {
 	number_of_commits=$1
@@ -102,17 +100,27 @@ time_notes () {
 	done
 }
 
-for count in 10 100 1000 10000; do
+do_tests () {
+	pr=$1
+	count=$2
+
+	test_expect_success $pr 'setup / mkdir' '
+		mkdir $count &&
+		cd $count
+	'
 
-	mkdir $count
-	(cd $count;
+	test_expect_success $pr "setup $count" "create_repo $count"
 
-	test_expect_success "setup $count" "create_repo $count"
+	test_expect_success $pr 'notes work' "test_notes $count"
 
-	test_expect_success 'notes work' "test_notes $count"
+	test_expect_success USR_BIN_TIME,$pr 'notes timing with /usr/bin/time' "time_notes 100"
+
+	test_expect_success $pr 'teardown / cd ..' 'cd ..'
+}
 
-	test_expect_success 'notes timing' "time_notes 100"
-	)
+do_tests NOT_EXPENSIVE 10
+for count in 100 1000 10000; do
+	do_tests EXPENSIVE $count
 done
 
 test_done
-- 
1.7.2.1.295.gd03d
Ævar Arnfjörð Bjarmason· Aug 19, 2010, 15:50 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

On Tue, Aug 10, 2010 at 23:37, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> +test -x /usr/bin/time && test_set_prereq USR_BIN_TIME
It turns out that this fails on Solaris because its /usr/bin/time is different.
Maybe this should just check if /usr/bin/time --version is ^GNU ?
Jonathan Nieder· Aug 24, 2010, 07:14 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

Ævar Arnfjörð Bjarmason wrote:
> It turns out that this fails on Solaris because its /usr/bin/time is different.

Odd. Different how? As far as I can tell, all that test asks of time is to execv() its arguments and pass on a 0 exit status.

Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh. Does the following help?

Patch is against next.  Untested except on Linux where it wouldn't
make a difference.
-- 8< --
Subject: t3302 (notes): Port to Solaris

The time_notes script, which uses POSIX shell features, is currently sometimes run with a non-POSIX /bin/sh.

Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Show changes to t/t3302-notes-index-expensive.sh +1 −1
diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
index 7c08e99..e35d781 100755
--- a/t/t3302-notes-index-expensive.sh
+++ b/t/t3302-notes-index-expensive.sh
@@ -96,7 +96,7 @@ time_notes () {
 	for mode in no-notes notes
 	do
 		echo $mode
-		/usr/bin/time sh ../time_notes $mode $1
+		/usr/bin/time "$SHELL_PATH" ../time_notes $mode $1
 	done
 }
 
Junio C Hamano· Aug 24, 2010, 16:46 UTC · re: Jonathan Nieder · lore

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

Jonathan Nieder <jrnieder@gmail.com> writes:
Show 8 quoted lines
> Odd.  Different how?  As far as I can tell, all that test asks
> of time is to execv() its arguments and pass on a 0 exit status.
>
> Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh.  Does the
> following help?
>
> Patch is against next.  Untested except on Linux where it wouldn't
> make a difference.
Makes sense, although I don't have access to Solaris boxes right now...
Show 23 quoted lines
> -- 8< --
> Subject: t3302 (notes): Port to Solaris
>
> The time_notes script, which uses POSIX shell features, is
> currently sometimes run with a non-POSIX /bin/sh.
>
> Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
> ---
> diff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh
> index 7c08e99..e35d781 100755
> --- a/t/t3302-notes-index-expensive.sh
> +++ b/t/t3302-notes-index-expensive.sh
> @@ -96,7 +96,7 @@ time_notes () {
>  	for mode in no-notes notes
>  	do
>  		echo $mode
> -		/usr/bin/time sh ../time_notes $mode $1
> +		/usr/bin/time "$SHELL_PATH" ../time_notes $mode $1
>  	done
>  }
>  
> -- 
Ævar Arnfjörð Bjarmason· Aug 29, 2010, 19:20 UTC · re: Junio C Hamano · lore

Re: [PATCH v2] git-notes: Run partial expensive test everywhere

On Tue, Aug 24, 2010 at 16:46, Junio C Hamano <gitster@pobox.com> wrote:
Show 12 quoted lines
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> Odd.  Different how?  As far as I can tell, all that test asks
>> of time is to execv() its arguments and pass on a 0 exit status.
>>
>> Ah, maybe this is it: perhaps /usr/bin/time sh runs /bin/sh.  Does the
>> following help?
>>
>> Patch is against next.  Untested except on Linux where it wouldn't
>> make a difference.
>
> Makes sense, although I don't have access to Solaris boxes right now...
I tested it FWIW. It works, thanks both.

← back to recent threads