Re: [PATCH 3/6] t5516 (fetch-push): introduce mk_test_with_name()

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Mar 20, 2013, 20:00 UTC
Message-ID: <20130320200010.GR3655@google.com>
In-Reply-To: <20130320185844.GB30165@sigill.intra.peff.net>

Jeff King wrote:

Show 7 quoted lines

>                                                                    I
> tend to read the tests in a top-down manner: a test is interesting
> (usually because it fails), and then I want to see what it is doing, so
> I look at any functions it calls, and so forth.
>
> What I usually find _much_ harder to debug is when there is hidden state
> leftover from other tests.

Thanks for articulating this. I agree that keeping track of state is the hardest part of working with git's tests.

To clarify my earlier comment, I was reading the test script from the point of view of someone who wants to add an additional test, rather than someone debugging an existing one. That person has a difficult task: she needs to understand

 * What do the existing tests in the script do?  This information
   is important in deciding whether the new test will be redundant.

 * How do I work with the particular dialect used in the script,
   including helpers?  A new test should fit in with the style of its
   surroundings to avoid contributing to an unmaintainable mess.

 * What is the intended scope of the test script?  Does my new test
   belong in this script?

 * What is the logical progression of the script?  What story does this
   script tell?  Where should I insert my test to maintain a logical
   ordering?

 * What state that later tests rely on do I need to avoid clobbering?

Generally the best way to help such a person is to make the script very simple. In particular, using standard helpers instead of custom functions when appropriate and documenting helpers used to give new readers a quick introduction to the dialect can help a lot.