Re: [PATCH] t1050-large: replace dd by test-genrandom

From: Jeff King <peff@peff.net>
Date: Jan 13, 2015, 21:47 UTC
Message-ID: <20150113214733.GA16582@peff.net>
In-Reply-To: <54B5579B.4080607@kdbg.org>

On Tue, Jan 13, 2015 at 06:36:27PM +0100, Johannes Sixt wrote:

Show 5 quoted lines

> For some unknown reason, the dd on my Windows box segfaults every now
> and than, but since recently, it does so much more often than it used
> to, which makes running the test suite burdensome.
> 
> Get rid of four invocations of dd and use test-genrandom instead.

There are a dozen other uses of dd in the test suite. Do they all need to go?

Show 5 quoted lines

> The new code does change some properties of the generated files:
> 
>  - They are a bit smaller.
>  - They are not sparse anymore.
>  - They do not compress well anymore.

This is unfortunate, as it means other platforms will be slower. I measured a best-of-five on running t1050 going from 0.780s to 1.750s. That's on an SSD. Doing it on a RAM disk the numbers are 0.600s and 1.394s. Better, but not great.

One second on the test suite probably isn't breaking the bank, but these sorts of things do add up. I wonder if we can shrink the test size. We use 2000k files with a 200k core.bigfilethreshold, and a 1500k GIT_ALLOC_LIMIT. Skimming through the history, the sizes seem fairly arbitrary. We can't go _too_ low, or GIT_ALLOC_LIMIT will prevent us from even allocating heap memory for non-objects.

I tried dropping it by a factor of 10, but sadly that hits several cases. The commit-slab code wants 512k chunks (which seems like rather a lot to me), and pack-objects starts at just over 150k for the set of objects. It would be nice to have a finer-grained tool than GIT_ALLOC_LIMIT that applied only to objects, but I guess then we would not be as sure of catching stray code paths (each caller would have to annotate "this is for an object").

-Peff