Re: [PATCH] test-lib: write test results to test-results/<basename>-<pid>

From: SZEDER Gábor <szeder@ira.uka.de>
Date: Mar 14, 2009, 13:59 UTC
Message-ID: <20090314135904.GL6808@neumann>
In-Reply-To: <fabb9a1e0903140616q3770f89axff84755abb1f47c7@mail.gmail.com>

Hi,

On Sat, Mar 14, 2009 at 02:16:57PM +0100, Sverre Rabbelier wrote:

Show 10 quoted lines

> On Sat, Mar 14, 2009 at 13:28, SZEDER Gábor <szeder@ira.uka.de> wrote:
> > With my proposed change there would be no need to clean 'test-results'
> > before running the tests, because test-lib.sh would take care of that
> > (not by removing and recreating 'test-results/', but by overwriting
> > (IOW: removing and recreating, but in one step) individual test result
> > files).
> 
> Wouldn't that result in possible stale files being counted in the
> result (e.g., if those tests were not run this time, but they were run
> previously)?

It depends.

If you run only a few tests, then you do it with a command like 'make t1234-foo.sh t5678-bar.sh'.

Currently this doesn't run the 'pre-clean' target, therefore if you run different tests (e.g. 'make t1234-foo.sh ; make t5678-bar.sh'), then a 'make aggregate-results' will include the result of both of those tests. The same happens with my proposal, too, because the test of the last run will not overwrite the results of the test in the first run.

Now suppose that you want to run the same set of tests twice (or more; e.g. 'make t1234-foo.sh ; make t1234-foo.sh ; make aggregate-results'). Since currently the pid of the test is included in the test result file name, there will be two files (t1234-<pid1> and t1234-<pid2>) created under 'test-results/', and _both_ will be counted by 'aggregate-results'. In this case my proposal is better, because the last round will overwrite the result of the previous runs, therefore no stale files will be counted.

Best, Gábor