From: Jon Loeliger <jdl@freescale.com>
Date: Mon, 11 Jun 2007 18:51:32 GMT
Subject: Re: [RFC] git integrated bugtracking
Message-ID: <1181587892.3380.37.camel@ld0161-tx32>
In-Reply-To: <20070610085044.GD4084@efreet.light.src>

On Sun, 2007-06-10 at 03:50, Jan Hudec wrote:

> 
> It would be really useful to have a tool, that could link a bug report to
> a test case demonstrating it and reporting whenever output of that test case
> changes. This would make it much easier for developer to see which bugs he
> might have fixed when doing a refactoring.

I think the inverse is interesting too.  That is, applying
keywords to tests and then making the test database
keywords searchable.

I used to work for a company that had a huge collection
of test cases for its product.  It would take days to run
a validation run for some of the simplest of changes that
were introduced during development.

Often what I wanted, and what was needed, was a simple
sanity check that verified that the proposed change was
not brain-dead from the start.

Over time, I found that certain tests proved to be good
at detecting failures in particular portions of the code.
Often, that test was not necessarily directly related to
the concept it was _supposed_ to be designed to test, but
incidentally was good at some _other_ concept too.

The developer who made the change, and ran the tests,
then was able to state "Test t42 from suite Frotz is good
at detecting changes to the <SomeModule>."

The trick is to now associate with that test the keyword
"SomeModule" so that in the future, another developer
could ask:  "Say, I just modified <SomeModule>, are there
any tests that are good at proving my changes sound?"
A test driver could then focus some test cycles quickly.

Now, for git, it's not likely a problem to just run
through all of its tests.  At ${PriorCompany} it was
essential to pare down the test suites to a manageable
size, yet still have assurance that your coverage was
reasonably good.

jdl