Re: [PATCH 2/2] Support Out-Of-Tree Valgrind Tests

From: Junio C Hamano <gitster@pobox.com>
Date: Mar 6, 2012, 22:37 UTC
Message-ID: <7v7gyxqrty.fsf@alter.siamese.dyndns.org>
In-Reply-To: <7vboo9qskb.fsf@alter.siamese.dyndns.org>

Junio C Hamano <gitster@pobox.com> writes:

Show 17 quoted lines

> "David A. Greene" <dag@cray.com> writes:
>
>> Junio C Hamano <gitster@pobox.com> writes:
>> ...
>>> Could it be that the reason for the breakage is because you are
>>> setting TEST_DIRECTORY to the directory that contains out-of-tree
>>> tests, instead of $GIT_BUILD_DIR/t/ directory?
>> ...
>>> Shouldn't TEST_DIRECTORY merely a short-hand for GIT_BUILD_DIR/t?
>>> What do you find relative to $TEST_DIRECTORY that cannot be found
>>> relative to GIT_BUILD_DIR/t?
>> ...
>>
>> Thanks for clarifying!
>
> Not so fast. The questions in the message you are responding to were
> not rhetorical.

So I ended up looking at t/test-lib.sh, sigh...

We have this bit from 62f5390 (test-lib: Allow overriding of TEST_DIRECTORY, 2010-08-19):

   # Test the binaries we have just built.  The tests are kept in
   # t/ subdirectory and are run in 'trash directory' subdirectory.
   if test -z "$TEST_DIRECTORY"
   then
           # We allow tests to override this, in case they want to run tests
           # outside of t/, e.g. for running tests on the test library
           # itself.
           TEST_DIRECTORY=$(pwd)
   fi
   GIT_BUILD_DIR="$TEST_DIRECTORY"/..

that says "As a side benefit this change also makes it easy for us to move the t/*.sh tests into subdirectories if we ever want to do that."

This expects that an out-of-tree test script is expected to set TEST_DIRECTORY before dot-sourcing test-lib.sh, e.g.

	#!/bin/sh
	TEST_DIRECTORY=/srv/project/git/git.git/t
        test_description='an out-of-tree test'
        . "$TEST_DIRECTORY/test-lib.sh"

which in turn lets the test framework to learn GIT_BUILD_DIR. From there, 'git' will be found in GIT_BUILD_DIR/bin-wrappers and the valgrind variants are found in a similar way.

One thing that is potentially missing is a way for such an out-of-tree test scripts to ship with supporting material in a separate file, relative to the test script. The in-tree t/t4013-diff-various.sh has its test vectors kept in t/t4013/ directory and finds them by doing

	expect="$TEST_DIRECTORY/t4013/diff.$test"

This is because the working directory after test-lib comes back to us may not be "trash" directory under TEST_DIRECTORY, and ../t4013/ is not the right way to find it. If an out-of-tree test t9999 wants to do something similar, it needs to do something like:

	#!/bin/sh
        HERE=$(PWD)
	TEST_DIRECTORY=/srv/project/git/git.git/t
        test_description='an out-of-tree test'
        . "$TEST_DIRECTORY/test-lib.sh"

and find it relative to $HERE, e.g. "$HERE/../t9999/diff.$test"

Of course, it would be nice to use a name better than $HERE for such a purpose ;-)