Re: Drastic jump in the time required for the test suite

From: Jeff King <peff@peff.net>
Date: Oct 20, 2016, 23:20 UTC
Message-ID: <20161020232052.uggm2w7qitrj5f7g@sigill.intra.peff.net>
In-Reply-To: <20161020210013.57wkyhmf2ism6vt2@sigill.intra.peff.net>

On Thu, Oct 20, 2016 at 05:00:13PM -0400, Jeff King wrote:

Show 10 quoted lines

> > I am not an expert on perl nor tracing, but is it feasible to find out
> > how many internal calls there are? i.e. either some shell script (rebase,
> > submodule) calling git itself a couple of times or even from compile/git/git
> > itself, e.g. some submodule operations use forking in there.
> 
> The script below is my attempt, though I think it is not quite right, as
> "make" should be the single apex of the graph. You can run it like:
> 
>   strace -f -o /tmp/foo.out -e clone,execve make test
>   perl graph.pl /tmp/foo.out | less -S

Ah, I see why it is sometimes wrong. We may see parent/child interactions out of order. E.g., we may see:

  1. Process B execs "git".

  2. Process B exits.

  3. Process A forks pid B.

because strace does not manage to ptrace "A" before "B" finishes running. It's hard to tell in step 3 if this is the same "A" we saw earlier. You cannot just go by PID, because the tests run enough processes that we end up reusing PIDs. My script has a hack which increments a counter when processes go away, but that happens in step 2 above (and steps 1 and 3 think they are seeing different processes, even though they are the same).

I'm sure it could be cleverly hacked around, but the easiest thing is probably to just make sure the tests are not run in parallel. High load makes that kind of out-of-order sequence much more likely.

Making sure I run it with "make -j1", the script I posted earlier gives pretty reasonable output. I won't post mine here, as it's double-digit megabytes, but you should be able to recreate it yourself.

Of course, the call graph by itself isn't that interesting. But you might be able to do some interesting analysis, or just find spots of interest to you (like git-submodule).

-Peff