Re: [RFC] git integrated bugtracking

From: Jon Loeliger <jdl@freescale.com>
Date: Jun 11, 2007, 18:51 UTC
Message-ID: <1181587892.3380.37.camel@ld0161-tx32>
In-Reply-To: <20070610085044.GD4084@efreet.light.src>

On Sun, 2007-06-10 at 03:50, Jan Hudec wrote:

Show 5 quoted lines

> 
> It would be really useful to have a tool, that could link a bug report to
> a test case demonstrating it and reporting whenever output of that test case
> changes. This would make it much easier for developer to see which bugs he
> might have fixed when doing a refactoring.

I think the inverse is interesting too. That is, applying keywords to tests and then making the test database keywords searchable.

I used to work for a company that had a huge collection of test cases for its product. It would take days to run a validation run for some of the simplest of changes that were introduced during development.

Often what I wanted, and what was needed, was a simple sanity check that verified that the proposed change was not brain-dead from the start.

Over time, I found that certain tests proved to be good at detecting failures in particular portions of the code. Often, that test was not necessarily directly related to the concept it was _supposed_ to be designed to test, but incidentally was good at some _other_ concept too.

The developer who made the change, and ran the tests, then was able to state "Test t42 from suite Frotz is good at detecting changes to the <SomeModule>."

The trick is to now associate with that test the keyword "SomeModule" so that in the future, another developer could ask: "Say, I just modified <SomeModule>, are there any tests that are good at proving my changes sound?" A test driver could then focus some test cycles quickly.

Now, for git, it's not likely a problem to just run through all of its tests. At ${PriorCompany} it was essential to pare down the test suites to a manageable size, yet still have assurance that your coverage was reasonably good.

jdl