Re: [PATCH 3/6] Facilitate debugging Git executables in tests with gdb

From: Jeff King <peff@peff.net>
Date: Oct 27, 2015, 23:58 UTC
Message-ID: <20151027235823.GD4172@sigill.intra.peff.net>
In-Reply-To: <CAGZ79karRbOTSEfFHRU6MG21T1L5GyuZW2ATqfdP4NE7wHMmHQ@mail.gmail.com>

On Tue, Oct 27, 2015 at 04:39:37PM -0700, Stefan Beller wrote:

Show 9 quoted lines

> On Tue, Oct 27, 2015 at 4:28 PM, Jeff King <peff@peff.net> wrote:
> > I agree doing so would be crazy. But would:
> >
> >   ./t1234-frotz.sh --gdb=17
> >
> > be sane to run gdb only inside test 17?
> 
> OT:
> We have two ways of addressing tests, by number and by name.

Yeah. The numbers are not stable if the script gets new test, but they are usually fine for within a debugging session. Names are annoying to type (and also not guaranteed unique).

Show 6 quoted lines

> Usually when a test fails ("Foo gobbles the bar correctly" failed),
> I want to run tests 1,17 (1 is the correct setup and 17 is the failing test)
> But coming up with that tuple is hard.
>   * How do I know we need to run 1 as the setup ? (usually we do,
>     sometimes we don't and other times we also need 2,3 to completely
> setup the tests)

I think trying to deduce that tuple is a fool's errand. It takes a lot of manual work, and even if you _think_ you have it, sometimes state left from earlier tests is accidentally important. But it's usually not that expensive to run earlier tests at all; it's just expensive to run them with extra debugging. That's why we have options like "--valgrind-only=17". We still _run_ tests 1..16, but we do it quickly, and then execute the expensive and slow valgrind git only on the suspicious one.

And I'd propose --gdb to work the same way (run all the other tests, but only kick in gdb for the suspicious one).

If you had multiple "git" invocations inside test 17, you could even do something like "--gdb=17:4" to kick in only for the 4th git invocation or something. But counting up git invocations is probably too irritating to be worth doing manually.

Show 5 quoted lines

>   * How do I know it's test 17 which is failing? My workflow up to now
>     I just searched the test title in the file, such that I'd be there anyway
>     to inspect it further. But still I found it inconvenient to
> mentally map between
>     17 and the test title.

I usually just run the test script and look at the output. Here's a failure (which I obviously induced with an extra line):

  $ ./t4103-apply-binary.sh -v -i
  [...]
  ok 5 - check binary diff -- should fail.
  
  expecting success: 
          git checkout master &&
          echo whoops, we fail here && false &&
          test_must_fail git apply --check C.diff
  
  Already on 'master'
  whoops, we fail here
  not ok 6 - check binary diff (copy) -- should fail.
  #
  #               git checkout master &&
  #               echo whoops, we fail here && false &&
  #               test_must_fail git apply --check C.diff
  #

I'd pull the test number from the "not ok" above (it's actually even easier to see if you drop the "-v", but I usually start my debugging with "-v" anyway, since error messages often make the problem obvious).

-Peff