Re: 'make test' fails in pu

> On Tue, Feb 17, 2015 at 09:39:17AM +0100, Dennis Kaarsemaker wrote:
>
>> Make test has been failing for 'pu' yesterday for and today at
>> t4016-diff-quote.sh. Full log:
>> http://ci.kaarsemaker.net/git/refs/heads/pu/1df29c71a731c679de9055ae5e407f3a4e18740a/artefact/test/log
>>
>> I noticed this a few times before and it tends to get fixed again
>> relatively quickly. So I'm wondering:
>>
>> - Should I even mention that it's failing, or is that just useless
>>   noise?
>> - If I should report this, I could also make my testing thing send
>>   mails. Would that be useful?
>
> If you bisect this, it turns up commit 30cd8f94f, which says:
>
>     WIP: diff-b-m
>
>     [...]
>
>     This update is still broken and breaks a handful of tests:
>
>          4016 4023 4047 4130 6022 6031 6032 9300 9200 9300 9350
>
> Sometimes a breakage in pu is surprising (e.g., it breaks only on a
> platform that the maintainer does not run "make test" on) and we would
> want to know about it. But sometimes it is merely that there is a
> work-in-progress. And it probably requires a human to tell the
> difference.
>
> So no, I do not think automatically mailing on test failures in pu is a
> good idea. Manually peeking at them and sending fixes before the series
> is merged to next _is_ very much encouraged, though. :)
>
> Unlike "pu", "next" and "master" should never fail tests (I think that
> Junio will not push them out if the tests have failed on his system). So
> failures there are much more likely to be interesting platform bugs (but
> of course, testing "pu" is still encouraged, as we may catch problems).
>
> But even for "next", I would say blind automated emails are not nearly
> as useful as a human who has looked at the problem (and especially
> bisected).
>
> -Peff
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html