Re: [PATCH 3/9] test-mergesort: add test subcommand
- From
René Scharfe <l.s.r@web.de>
- Date
- Oct 7, 2021, 20:00 UTC
- Message-ID
- <a3eeb255-56e3-eb22-ebf7-e9f04c95fa44@web.de>
- In-Reply-To
- <xmqqee9210a1.fsf@gitster.g>
Am 03.10.21 um 19:33 schrieb Junio C Hamano:
Show 12 quoted lines
> René Scharfe <l.s.r@web.de> writes: > >> Repeatable tests are not essential (the original paper didn't mention >> seeding), but shouldn't be much trouble to implement and would simplify >> comparisons across versions, systems and among testers. >> >> The only downside I can think of is that it may perhaps also simplify >> over-fitting, i.e. I might find micro-tweaks that only work for our >> specific rand() sequence and then misinterpret them as general >> improvements.. > > Yup, I think you summarized the pros-and-cons nicely.
Seeing that the series is in next already, here's a bonus patch for that.
--- >8 --- Subject: [PATCH 10/9] test-mergesort: use repeatable random numbers
Use MINSTD to generate pseudo-random numbers consistently instead of using rand(3), whose output can vary from system to system, and reset its seed before filling in the test values. This gives repeatable results across versions and systems, which simplifies sharing and comparing of results between developers.
Signed-off-by: René Scharfe <l.s.r@web.de> --- t/helper/test-mergesort.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/t/helper/test-mergesort.c b/t/helper/test-mergesort.c index 43ec74e2d3..8d128ae437 100644 --- a/t/helper/test-mergesort.c +++ b/t/helper/test-mergesort.c @@ -2,6 +2,12 @@ #include "cache.h" #include "mergesort.h" +static unsigned int minstd_rand(unsigned int *state) +{ + *state = (uint64_t)*state * 48271 % 2147483647; + return *state; +} + struct line { char *text; struct line *next; @@ -60,8 +66,9 @@ static void dist_sawtooth(int *arr, int n, int m) static void dist_rand(int *arr, int n, int m) { int i; + unsigned int seed = 1; for (i = 0; i < n; i++) - arr[i] = rand() % m; + arr[i] = minstd_rand(&seed) % m; } static void dist_stagger(int *arr, int n, int m) @@ -81,8 +88,9 @@ static void dist_plateau(int *arr, int n, int m) static void dist_shuffle(int *arr, int n, int m) { int i, j, k; + unsigned int seed = 1; for (i = j = 0, k = 1; i < n; i++) - arr[i] = (rand() % m) ? (j += 2) : (k += 2); + arr[i] = minstd_rand(&seed) % m ? (j += 2) : (k += 2); } #define DIST(name) { #name, dist_##name } -- 2.33.0