threads / patch / 20494

patch, 3 partsmoving test trash directories

Subject: [PATCH 0/3] moving test trash directories

## tl;dr

7 messages between Aug 9, 2009 and Aug 9, 2009. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Jeff King· Aug 9, 2009, 08:35 UTC · lore

This series adds support for making trash directories for the test scripts outside of the usual hierarchy. The main benefit is improved performance. By putting the trash directories onto a tmpfs (rather than SATA disks), my "make -j32 test" time went from around 1m51s to around 1m01s. I can only imagine what improvement it would have for people on NFS. :)

It also has the nice side effect that running the test suite doesn't thrash your disk as much, in case you wanted to do something else while it was running.

  [1/3] tests: use "$TEST_DIRECTORY" instead of ".."
  [2/3] tests: provide $TRASH_DIRECTORY variable
  [3/3] tests: allow user to specify trash directory location
-Peff
Jeff King· Aug 9, 2009, 08:37 UTC · re: Jeff King · lore

[PATCH 1/3] tests: use "$TEST_DIRECTORY" instead of ".."

The $TEST_DIRECTORY variable allows tests to find the top-level test directory regardless of the current working directory.

In the past, this has been used to accomodate tests which change directories, but it is also the first step to being able to move trash directories outside of the $TEST_DIRECTORY hierarchy.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t3404-rebase-interactive.sh            |    2 +-
 t/t3411-rebase-preserve-around-merges.sh |    2 +-
 t/t3414-rebase-preserve-onto.sh          |    2 +-
 t/t4020-diff-external.sh                 |    2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
Show changes to 4 files +4 −4

t/t3404-rebase-interactive.sh, t/t3411-rebase-preserve-around-merges.sh, t/t3414-rebase-preserve-onto.sh, t/t4020-diff-external.sh

diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index a973628..4cae019 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -10,7 +10,7 @@ that the result still makes sense.
 '
 . ./test-lib.sh
 
-. ../lib-rebase.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
 
 set_fake_editor
 
diff --git a/t/t3411-rebase-preserve-around-merges.sh b/t/t3411-rebase-preserve-around-merges.sh
index 6533505..14a23cd 100755
--- a/t/t3411-rebase-preserve-around-merges.sh
+++ b/t/t3411-rebase-preserve-around-merges.sh
@@ -10,7 +10,7 @@ a merge to before the merge.
 '
 . ./test-lib.sh
 
-. ../lib-rebase.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
 
 set_fake_editor
 
diff --git a/t/t3414-rebase-preserve-onto.sh b/t/t3414-rebase-preserve-onto.sh
index 80019ee..ee0a6cc 100755
--- a/t/t3414-rebase-preserve-onto.sh
+++ b/t/t3414-rebase-preserve-onto.sh
@@ -10,7 +10,7 @@ aren'"'"'t on top of $ONTO, even if they are on top of $UPSTREAM.
 '
 . ./test-lib.sh
 
-. ../lib-rebase.sh
+. "$TEST_DIRECTORY"/lib-rebase.sh
 
 # Set up branches like this:
 # A1---B1---E1---F1---G1
diff --git a/t/t4020-diff-external.sh b/t/t4020-diff-external.sh
index 4ea42e0..a7602cf 100755
--- a/t/t4020-diff-external.sh
+++ b/t/t4020-diff-external.sh
@@ -166,7 +166,7 @@ test_expect_success 'diff --cached' '
 	git update-index --assume-unchanged file &&
 	echo second >file &&
 	git diff --cached >actual &&
-	test_cmp ../t4020/diff.NUL actual
+	test_cmp "$TEST_DIRECTORY"/t4020/diff.NUL actual
 '
 
 test_done
-- 
1.6.4.178.g7a987
Jeff King· Aug 9, 2009, 08:38 UTC · re: Jeff King · lore

[PATCH 2/3] tests: provide $TRASH_DIRECTORY variable

Most scripts don't care about the absolute path to the trash directory. The one exception was t4014 script, which pieced together $TEST_DIRECTORY and $test itself to get an absolute directory.

Instead, let's provide a $TRASH_DIRECTORY which specifies the same thing. This keeps the $test variable internal to test-lib.sh and paves the way for trash directories in other locations.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t4014-format-patch.sh |    3 +--
 t/test-lib.sh           |    3 ++-
 2 files changed, 3 insertions(+), 3 deletions(-)
Show changes to 2 files +3 −3

t/t4014-format-patch.sh, t/test-lib.sh

diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 922a894..531f5b7 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -493,13 +493,12 @@ test_expect_success 'format-patch from a subdirectory (2)' '
 '
 
 test_expect_success 'format-patch from a subdirectory (3)' '
-	here="$TEST_DIRECTORY/$test" &&
 	rm -f 0* &&
 	filename=$(
 		rm -rf sub &&
 		mkdir -p sub/dir &&
 		cd sub/dir &&
-		git format-patch -1 -o "$here"
+		git format-patch -1 -o "$TRASH_DIRECTORY"
 	) &&
 	basename=$(expr "$filename" : ".*/\(.*\)") &&
 	test -f "$basename"
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 5fdc5d9..e6a7bb1 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -645,7 +645,8 @@ fi
 
 # Test repository
 test="trash directory.$(basename "$0" .sh)"
-test ! -z "$debug" || remove_trash="$TEST_DIRECTORY/$test"
+TRASH_DIRECTORY="$TEST_DIRECTORY/$test"
+test ! -z "$debug" || remove_trash=$TRASH_DIRECTORY
 rm -fr "$test" || {
 	GIT_EXIT_OK=t
 	echo >&5 "FATAL: Cannot prepare test area"
-- 
1.6.4.178.g7a987
Jeff King· Aug 9, 2009, 08:39 UTC · re: Jeff King · lore

[PATCH 3/3] tests: allow user to specify trash directory location

The tests generate a large amount of I/O activity creating and destroying repositories and files. We can improve the time it takes to run the test suite by creating trash directories on filesystems with better performance characteristic, even though we may not want the rest of the git repository on those filesystems (e.g., because they are not network connected, or because they are temporary ramdisks).

For example, on a dual processor system:
  $ cd t && time make -j32
  real    1m51.562s
  user    0m59.260s
  sys     1m20.933s
  # /dev/shm is tmpfs
  $ cd t && time make -j32 GIT_TEST_OPTS="--root=/dev/shm"
  real    1m1.484s
  user    0m53.555s
  sys     1m5.264s

We almost halve the wall clock time, and we utilize the dual processors much better.

Signed-off-by: Jeff King <peff@peff.net>
---
There is a test below for absolute versus relative path in the root
provided.  Do we need some extra magic to make it work on non-Unix
platforms?
 t/test-lib.sh |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)
Show changes to t/test-lib.sh +8 −1
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e6a7bb1..a5b8d03 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -114,6 +114,9 @@ do
 		valgrind=t; verbose=t; shift ;;
 	--tee)
 		shift ;; # was handled already
+	--root=*)
+		root=$(expr "z$1" : 'z[^=]*=\(.*\)')
+		shift ;;
 	*)
 		echo "error: unknown test option '$1'" >&2; exit 1 ;;
 	esac
@@ -645,7 +648,11 @@ fi
 
 # Test repository
 test="trash directory.$(basename "$0" .sh)"
-TRASH_DIRECTORY="$TEST_DIRECTORY/$test"
+test -n "$root" && test="$root/$test"
+case "$test" in
+/*) TRASH_DIRECTORY="$test" ;;
+ *) TRASH_DIRECTORY="$TEST_DIRECTORY/$test" ;;
+esac
 test ! -z "$debug" || remove_trash=$TRASH_DIRECTORY
 rm -fr "$test" || {
 	GIT_EXIT_OK=t
-- 
1.6.4.178.g7a987
Johannes Sixt· Aug 9, 2009, 13:33 UTC · re: Jeff King · lore

Re: [PATCH 3/3] tests: allow user to specify trash directory location

On Sonntag, 9. August 2009, Jeff King wrote:
> There is a test below for absolute versus relative path in the root
> provided.  Do we need some extra magic to make it work on non-Unix
> platforms?

Not for Windows. Those devolpers who want to use an absolute path can use the POSIX path notation:

   GIT_TEST_OPTS=--root=/c/temp/gittests
This works in t4014.
-- Hannes
Johannes Schindelin· Aug 9, 2009, 17:42 UTC · re: Johannes Sixt · lore

Re: [PATCH 3/3] tests: allow user to specify trash directory location

Hi,
On Sun, 9 Aug 2009, Johannes Sixt wrote:
Show 11 quoted lines
> On Sonntag, 9. August 2009, Jeff King wrote:
> > There is a test below for absolute versus relative path in the root
> > provided.  Do we need some extra magic to make it work on non-Unix
> > platforms?
> 
> Not for Windows. Those devolpers who want to use an absolute path can use the 
> POSIX path notation:
> 
>    GIT_TEST_OPTS=--root=/c/temp/gittests
> 
> This works in t4014.

But I am not aware of any reasonable tmpfs equivalent on Windows. Are you? I would be _very_ interested.

Ciao, Dscho

Jeff King· Aug 9, 2009, 08:50 UTC · re: Jeff King · lore

[PATCH 4/3] t/Makefile: include config.mak

This is useful if you want to specify GIT_TEST_OPTS that you always use.

Signed-off-by: Jeff King <peff@peff.net>
---
On Sun, Aug 09, 2009 at 04:35:19AM -0400, Jeff King wrote:
>   [1/3] tests: use "$TEST_DIRECTORY" instead of ".."
>   [2/3] tests: provide $TRASH_DIRECTORY variable
>   [3/3] tests: allow user to specify trash directory location

I think a 4/3 like this is nice to let you use --root without extra typing each time. And when you do "cd t && make t$WHATEVER.sh GIT_TEST_OPTS='-v -i'", it will automatically be overridden, which is probably what you want, so you can inspect the failed tests in t/trash*.

 t/Makefile |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
Show changes to t/Makefile +2 −0
diff --git a/t/Makefile b/t/Makefile
index bf816fc..bd09390 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -3,6 +3,8 @@
 # Copyright (c) 2005 Junio C Hamano
 #
 
+-include ../config.mak
+
 #GIT_TEST_OPTS=--verbose --debug
 SHELL_PATH ?= $(SHELL)
 TAR ?= $(TAR)
-- 
1.6.4.178.g7a987

← back to recent threads