Re: [RFC][PATCH] index-pack: add testcases found using AFL

From: Jeff King <peff@peff.net>
Date: Mar 10, 2017, 19:42 UTC
Message-ID: <20170310194245.p37w6mew4que6oya@sigill.intra.peff.net>
In-Reply-To: <eec5ab2a-7fe7-b47f-8073-a8212a9634f1@oracle.com>

On Fri, Mar 10, 2017 at 08:34:45PM +0100, Vegard Nossum wrote:

Show 11 quoted lines

> > That's something I guess, but I'm not enthused by the idea of just
> > dumping a bunch of binary test cases that nobody, not even the author,
> > understands.
> 
> I understand your concern. This is how I see it:
> 
> Negatives:
> 
>  - 'make test' takes 1 second longer to run
> 
>  - 548K data added to git.git

My real concern is that this is the tip of the ice berg. So we increased coverage in one program by a few percent. But wouldn't this procedure be applicable to lots of _other_ parts of Git, too?

IOW, I'm worried about a day when we've added dozens or hundreds of seconds to the test suite. Sure, we can quit adding at any time, but I feel like it's easier to make a decision from the outset.

I'm tempted to say these should go into a different test-suite, or be marked with a special flag or something. But then I guess nobody runs them.

-Peff