Re: [PATCH v3] doc-diff: don't `cd_to_toplevel`

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Feb 7, 2019, 15:41 UTC
Message-ID: <nycvar.QRO.7.76.6.1902071632280.41@tvgsbejvaqbjf.bet>
In-Reply-To: <20190206185520.GD10231@sigill.intra.peff.net>

Hi Peff,

On Wed, 6 Feb 2019, Jeff King wrote:

Show 13 quoted lines

> On Tue, Feb 05, 2019 at 10:45:35AM -0800, Junio C Hamano wrote:
> 
> >  - Perhaps find the fork point, run tests to find known breakages
> >    and exclude them?  This would simply be not practical, as it
> >    doubles the number of tests run, for individual topic branches
> >    because there are an order of magnitude more of them than the
> >    primary integration branches.
> 
> I think this can be limited to the tests that failed, which makes things
> much faster. I.e., we run the tests at the tip of topic X and see that
> t1234 fails. We then go back to the fork point and we just need to run
> t1234 again. If it succeeds, then we blame X for the failure. If it
> fails, then we consider it a false positive.

If you mean merge bases by fork points, I wrote an Azure Pipeline to do that (so that I could use the cloud as kind of a fast computer), but that was still too slow.

Even when there are even only as much as 12 merge bases to test (which is the current number of merge bases between `next` and `pu`), a build takes roughly 6 minutes on Windows, and many tests take 1 minute or more to run (offenders like t7003 and t7610 take over 400 seconds, i.e. roughly 6 minutes), we are talking about roughly 1.5h *just* to test the merge bases.

And I sadly have to report that that's not the end of it. Back when I implemented the automatic bisect after failed builds (for details, see https://github.com/git-for-windows/build-extra/commit/c7e01e82c), I had to turn it off real quickly because the dumb bisect between `next` and `pu` regularly ran into the 4h timeout.

Ciao, Dscho

Show 5 quoted lines

> You do pay the price to do a full build at the fork point, but in my
> experience that is much quicker than the tests.
> 
> -Peff
>