Re: [PATCH] t/Makefile: add a rule to re-run previously-failed tests

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Jul 1, 2016, 14:00 UTC
Message-ID: <alpine.DEB.2.20.1607011558010.12947@virtualbox>
In-Reply-To: <20160630063725.GC15380@sigill.intra.peff.net>

Hi Peff,

On Thu, 30 Jun 2016, Jeff King wrote:

Show 11 quoted lines

> On Wed, Jun 29, 2016 at 09:02:37AM +0200, Johannes Schindelin wrote:
> 
> > It is the most convenient way to determine which tests failed after
> > running the entire test suite, in parallel, to look for left-over "trash
> > directory.t*" subdirectories in the t/ subdirectory.
> 
> As Junio noted, this doesn't work with --root. I have sometimes used:
> 
>   grep 'failed [^0]' test-results/*
> 
> for this purpose.

True, I could also do that. Looking for directories rather than spawning a full-fledged grep is more light-weight, though.

Show 8 quoted lines

> > This patch automates the process of determinig which tests failed
> > previously and re-running them; It turned out to be quite convenient
> > when trying to squash bugs that crept in during rebases.
> 
> I suspect your response will be "perl tools on Windows are too painful
> to use", but the "prove" tool which comes with perl can do this and more
> (e.g., running the failed tests first, and then following up with the
> others to double-check), and our test suite supports it quite well.

It will surprise you to learn that I did use `prove` extensively. There have been enough problems with it, though, that I stopped it.

Modern Windows does not have too many problems with it, but it appears as if Windows Server 2008 R2 (which I used for quite some time for my principal development) requires too many work-arounds for Perl to work reliably so that every once in a while, `prove` hangs without any real reason.

That is when I stopped using it.

Ciao, Dscho