Re: script to pre-build all commits in a git-bisect, and use them during testing

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 21, 2024, 23:20 UTC
Message-ID: <xmqqo7c9l809.fsf@gitster.g>
In-Reply-To: <CAC4O8c-tgDQ42upWorMi3Cw+KdPoHT2YxvZEirt_dtjQXreXAw@mail.gmail.com>

Britton Kerin <britton.kerin@gmail.com> writes:

Show 5 quoted lines

> Problem: it's annoying to have to build each commit when
> git-bisect'ing for something that isn't being tested automatically (so
> no git bisect run).
>
> Solution: https://github.com/bkerin/git-batch-bisect

So, because "git bisect run" would not be usable for whatever reason to run a "build and then test" from the command line, you would build all revisions using

    git batch-bisect runinall 'autoreconf --install && ./configure && make'

before starting to test any revision, and then after building everything, you would do:

    git batch-bisect runincurrent ./test_program
    git batch-bisect good
    git batch-bisect runincurrent ./test_program
    git batch-bisect bad
    ...

where ./test_program cannot signal if it is good or bad mechanically (e.g., by exiting with non-zero status for failure) but the testers need to judge the good/bad manually before they can say "git batch-bisect good/bad", I presume.

It's an interesting workaround for a test program that cannot be automated. You are willing to waste CPU cycles and diskspace to build upfront and hold the build products for all N revisions in the range, where with bisection you would have to build far fewer revisions (which is the point of bisection), to optimize for developer/tester latency, which tends to be more expensive. The trade-off may make good sense.