Re: [PATCH] Make some commit hashes in tests reproducible

From: Jeff King <peff@peff.net>
Date: Jul 7, 2020, 21:52 UTC
Message-ID: <20200707215206.GA2300296@coredump.intra.peff.net>
In-Reply-To: <xmqq1rln3t4y.fsf@gitster.c.googlers.com>

On Tue, Jul 07, 2020 at 02:35:57PM -0700, Junio C Hamano wrote:

Show 9 quoted lines

> Jeff King <peff@peff.net> writes:
> 
> > That's using the same start point as test_tick, though really it could
> > be anything. I've intentionally _not_ called test_tick at the beginning
> > of each script, because that would throw off all of the scripts that do
> > use it by one tick (whereas the first test_tick will overwrite these
> > values).
> 
> Yup, I think that is a sensible approach.

OK, I'll see if I can polish it into a series.

Show 8 quoted lines

> Another thing we could do is under DEVELOPER=YesPlease we can set
> timestamps you just added here to some random time.
> 
> The ones that do care about reproducibility guarantee by using
> test_tick would not be affected, and those that were happy with the
> current time should be happy with such a random timestamp.  Or we
> could just drop what this patch does only under DEVELOPER=YesPlease
> which amounts to be the same as setting random time.

That should be easy to do. I wonder if it's really worth the trouble to maintain two parallel worlds. I.e., one of the goals would be to stop worrying about this non-determinism. If we keep it as a requirement, then we might as well do so all the time (and making it random under DEVELOPER is effectively keeping it as a requirement, since we expect all tests to pass under that flag).

I'm also skeptical how often we use system times anyway, because _any_ use of test_commit or test_tick in a script is enough to make all of the subsequent commands deterministic. I'd be more inclined to let a particular script say "I'm interested in random times". But then, I'd think such a script would be better written to trigger its interesting cases with a well-crafted set of deterministic times.

-Peff