threads / discuss / 46954

v2.15.0-rc1 test failure

Subject: v2.15.0-rc1 test failure

## tl;dr

6 messages between Oct 11, 2017 and Oct 12, 2017.

replies: 5people: 3as markdown or json

Ramsay Jones· Oct 11, 2017, 22:15 UTC · lore
Hi Adam,

I had a test failure on the v2.15.0-rc1 build tonight. The test in question being t0021-conversion.sh #15 ('required process filter should filter data'). I didn't have any test failures on v2.15.0-rc0, and I don't see any change that would have affected this test.

Also, I ran this test by hand (well, in a shell loop) at least 70 times tonight (after the test-suite run), without any failures, so ... (unfortunately, I don't have a trash directory to look at. :( )

I have just kicked off another full test-suite run.
Just a heads up! ;-)

ATB, Ramsay Jones

Ramsay Jones· Oct 11, 2017, 22:25 UTC · re: Ramsay Jones · lore

Re: v2.15.0-rc1 test failure

On 11/10/17 23:15, Ramsay Jones wrote:
Show 16 quoted lines
> Hi Adam,
> 
> I had a test failure on the v2.15.0-rc1 build tonight.
> The test in question being t0021-conversion.sh #15
> ('required process filter should filter data'). I didn't
> have any test failures on v2.15.0-rc0, and I don't see
> any change that would have affected this test.
> 
> Also, I ran this test by hand (well, in a shell loop) at
> least 70 times tonight (after the test-suite run), without
> any failures, so ... (unfortunately, I don't have a trash
> directory to look at. :( )
> 
> I have just kicked off another full test-suite run.
> 
> Just a heads up! ;-)

Oops, for mailing list reader, I should have made clear that this failure is only on cygwin. :-D

ATB, Ramsay Jones

Adam Dinwoodie· Oct 11, 2017, 22:34 UTC · re: Ramsay Jones · lore

Re: v2.15.0-rc1 test failure

On Wed, Oct 11, 2017 at 11:15:57PM +0100, Ramsay Jones wrote:
Show 16 quoted lines
> Hi Adam,
> 
> I had a test failure on the v2.15.0-rc1 build tonight.
> The test in question being t0021-conversion.sh #15
> ('required process filter should filter data'). I didn't
> have any test failures on v2.15.0-rc0, and I don't see
> any change that would have affected this test.
> 
> Also, I ran this test by hand (well, in a shell loop) at
> least 70 times tonight (after the test-suite run), without
> any failures, so ... (unfortunately, I don't have a trash
> directory to look at. :( )
> 
> I have just kicked off another full test-suite run.
> 
> Just a heads up! ;-)
Hi Ramsay,
I assume, given you're emailing me, that this is a Cygwin failure?

t0021.15 has PERL as a requirement, and I see semi-regular failures from Git tests that are Perl-based in one way or another (git-svn tests are the most common problems). I've not spotted t0021 failing in that way, but it sounds like the same class of problem.

I dig into these failures when I see them, mostly by running the script a few hundred times until I get the failure again, and they've always been Perl itself segfaulting. That points to the problem being in Cygwin's Perl package rather than Git, and it's very unlikely to be anything that's got worse in v2.15.0.

I've never managed to get further than pinning the blame on Perl, though. If you manage to reproduce the failure and it turns out to be something different, or you manage to dig into the failure in Perl itself, that'd be some very interesting news.

Adam
Jonathan Nieder· Oct 11, 2017, 23:08 UTC · re: Adam Dinwoodie · lore

Re: v2.15.0-rc1 test failure

Hi,
Adam Dinwoodie wrote:
Show 10 quoted lines
> t0021.15 has PERL as a requirement, and I see semi-regular failures from
> Git tests that are Perl-based in one way or another (git-svn tests are
> the most common problems).  I've not spotted t0021 failing in that way,
> but it sounds like the same class of problem.
>
> I dig into these failures when I see them, mostly by running the script
> a few hundred times until I get the failure again, and they've always
> been Perl itself segfaulting.  That points to the problem being in
> Cygwin's Perl package rather than Git, and it's very unlikely to be
> anything that's got worse in v2.15.0.

That reminds me of https://bugs.debian.org/868738, which I tracked down to perl's "die" helper using errno to determine the exit status instead of deterministically using 128. I wasn't able to track it down further than that.

t/t0021/rot13-filter.pl doesn't have any similar suspect constructs, but thought I should mention it anyway.

Thanks, Jonathan

Ramsay Jones· Oct 12, 2017, 00:27 UTC · re: Adam Dinwoodie · lore

Re: v2.15.0-rc1 test failure

On 11/10/17 23:34, Adam Dinwoodie wrote: [snip]

> Hi Ramsay,
> 
> I assume, given you're emailing me, that this is a Cygwin failure?
Yes, sorry, I should have made that clear.
> t0021.15 has PERL as a requirement, and I see semi-regular failures from
> Git tests that are Perl-based in one way or another (git-svn tests are
> the most common problems).  I've not spotted t0021 failing in that way,
> but it sounds like the same class of problem.

Yep, many moons ago, I used to run the svn tests (on Linux and cygwin) which would fail intermittently on cygwin. I didn't notice any problem with perl though.

Show 5 quoted lines
> I dig into these failures when I see them, mostly by running the script
> a few hundred times until I get the failure again, and they've always
> been Perl itself segfaulting.  That points to the problem being in
> Cygwin's Perl package rather than Git, and it's very unlikely to be
> anything that's got worse in v2.15.0.

Since I stopped running the svn tests, the number of intermittent test failures on cygwin have dropped significantly, but haven't gone away completely.

I just finished the second test-suite run and, of course, t0021 ran without problem this time. Hmm, I don't think I have time to chase this down at the moment. I will keep your 'perl hypothesis' in mind for next time, however.

Thanks.

ATB, Ramsay Jones

Adam Dinwoodie· Oct 12, 2017, 09:06 UTC · re: Ramsay Jones · lore

Re: v2.15.0-rc1 test failure

On Thu, Oct 12, 2017 at 01:27:57AM +0100, Ramsay Jones wrote:
Show 30 quoted lines
> On 11/10/17 23:34, Adam Dinwoodie wrote:
> [snip]
> > Hi Ramsay,
> > 
> > I assume, given you're emailing me, that this is a Cygwin failure?
> 
> Yes, sorry, I should have made that clear.
> 
> > t0021.15 has PERL as a requirement, and I see semi-regular failures from
> > Git tests that are Perl-based in one way or another (git-svn tests are
> > the most common problems).  I've not spotted t0021 failing in that way,
> > but it sounds like the same class of problem.
> 
> Yep, many moons ago, I used to run the svn tests (on Linux and cygwin)
> which would fail intermittently on cygwin. I didn't notice any problem
> with perl though.
> 
> > I dig into these failures when I see them, mostly by running the script
> > a few hundred times until I get the failure again, and they've always
> > been Perl itself segfaulting.  That points to the problem being in
> > Cygwin's Perl package rather than Git, and it's very unlikely to be
> > anything that's got worse in v2.15.0.
> 
> Since I stopped running the svn tests, the number of intermittent test failures on cygwin have dropped significantly, but haven't gone away
> completely.
> 
> I just finished the second test-suite run and, of course, t0021 ran
> without problem this time. Hmm, I don't think I have time to chase
> this down at the moment. I will keep your 'perl hypothesis' in mind
> for next time, however.

Evidence for my Perl hypothesis, which I offer at least as much so other people can check my logic as anything else:

Here's a fairly typical set of verbose output from a test failing in this way [0]. The critical bit is the line "error: git-svn died of signal 11". Since git-svn is a Perl script, and Perl is the sort of interpreted language that would throw its own errors if it encountered a script bug, the fact that it's hitting a segfault means there's a problem of some ilk with the Perl interpreter itself.

[0]: https://github.com/me-and/Cygwin-Git/issues/13#issuecomment-211372448
Adam

← back to recent threads