Re: [PATCH 3/9] test-mergesort: add test subcommand
René Scharfe <l.s.r@web.de> writes:
Show 8 quoted lines
> Repeatable tests are not essential (the original paper didn't mention
> seeding), but shouldn't be much trouble to implement and would simplify
> comparisons across versions, systems and among testers.
>
> The only downside I can think of is that it may perhaps also simplify
> over-fitting, i.e. I might find micro-tweaks that only work for our
> specific rand() sequence and then misinterpret them as general
> improvements..
Yup, I think you summarized the pros-and-cons nicely.
Junio C Hamano <gitster@pobox.com>