debugging git tests, was: Re: [PATCH v4 2/8] t5520: test no merge candidates cases

From: Jeff King <peff@peff.net>
Date: May 18, 2015, 18:55 UTC
Message-ID: <20150518185554.GB11463@peff.net>
In-Reply-To: <xmqqoalhdbhx.fsf@gitster.dls.corp.google.com>

On Mon, May 18, 2015 at 10:46:50AM -0700, Junio C Hamano wrote:

Show 26 quoted lines

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> >> +test_expect_success 'fail if wildcard spec does not match any refs' '
> >> +	git checkout -b test copy^ &&
> >> +	test_when_finished "git checkout -f copy && git branch -D test" &&
> >
> > When I read this line, I immediately asked myself whether the
> > branch would be deleted even if the test case failed. I then
> > tested this theory by editing the first test case ("setup") like
> > this:
> > ...
> > and indeed, the file "file" was gone, even if the test case
> > failed. I therefore believe that this "test_when_finished" cleanup
> > might make debugging substantially harder. Maybe we can drop these
> > lines from this patch?
> 
> The test framework is aware of the fact that it needs to help the
> people who are debugging the scripts.  The support is limited to the
> case in which you run it under the -i option, i.e.
> 
> 	$ cd t
>         $ sh ./t5520-pull.sh -i -v
> 
> will refrain from running test_when_finished scripts when the test
> piece fails.  Even though this is only limited to -i, I found it
> often sufficient for debugging.

If you don't use "-i", you are pretty much screwed anyway, because the subsequent tests will stomp all over the state of the test directory. Many a head-scratching session has been caused by looking at the wrong state, and these days my go-to options for debugging a test are "-v -i". But since we are talking about it in a related thread, I will advertise the new "-x" here, too. :)

As a side note, I've also considered better support for running the debugger on git commands inside a test (right now, I usually stick a "gdb --args" in the pipeline, but you have to remember to run with "-v", and to redirect stdin appropriately). Do other people have this annoyance, too?

I'm vaguely thinking of something like putting debug support into bin-wrappers/git, but activating it only for certain tests (so you could say "t5520-pull.sh --gdb=10", and git would start under the debugger only for test 10). I think we'd also have to use gdbserver for I/O sanity, and maybe provide short script to do:

   gdb -ex "target remote localhost:$some_port" "$TEST_DIRECTORY"/../git

That still doesn't cover all cases (when git spawns an external command, you probably want to run the debugger on that; likewise, I have a git-remote-debug hack for debugging remote-curl). I suspect with clever use of gdb options that you could convince the original gdb invocation to end up tracing the process you care about, though.

-Peff