Re: A potential approach to making tests faster on Windows

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Apr 3, 2018, 11:43 UTC
Message-ID: <nycvar.QRO.7.76.6.1804031150030.5026@qfpub.tvgsbejvaqbjf.bet>
In-Reply-To: <87fu4hwgfa.fsf@evledraar.gmail.com>

Hi Ævar,

On Fri, 30 Mar 2018, Ævar Arnfjörð Bjarmason wrote:

> On Fri, Mar 30 2018, Johannes Schindelin wrote [expressing frustrations
> about Windows test suite slowness]:

To be precise (and I think it is important to be precise here): it is not the Windows test suite about which I talked, it is Git's test suite, as run on Windows. It might sound like a small difference, but it is not: the fault really lies with Git because it wants to be a portable software.

> I've wondered for a while whether it wouldn't be a viable approach to
> make something like an interpreter for our test suite to get around this
> problem, i.e. much of it's very repetitive and just using a few shell
> functions we've defined, what if we had C equivalents of those?

There has even been an attempt to do this by Linus Torvalds himself:

https://public-inbox.org/git/Pine.LNX.4.64.0602232229340.3771@g5.osdl.org/

It has not really gone anywhere...

To be honest, I had a different idea (because I do not really want to maintain yet another piece of software): BusyBox. The source code is clean enough, and it should, in theory, allow us to go really fast.

> Duy had a WIP patch set a while ago to add C test suite support, but I
> thought what if we turn that inside-out, and instead have a shell
> interpreter that knows about the likes of test_cmp, and executes them
> directly?

The problem, of course, is: if you add Git-test-suite-specific stuff to any Unix shell, you are going to have to maintain this fork, and all of a sudden it has become a lot harder to develop Git, and to port it.

Quite frankly, I would rather go with Duy's original approach, or a variation thereof, as snuck into the wildmatch discussion here:

	https://public-inbox.org/git/20180110090724.GA2893@ash/

Show 45 quoted lines

> Here's proof of concept as a patch to the dash shell:
> 
>     u dash (debian/master=) $ git diff
>     diff --git a/src/builtins.def.in b/src/builtins.def.in
>     index 4441fe4..b214a17 100644
>     --- a/src/builtins.def.in
>     +++ b/src/builtins.def.in
>     @@ -92,3 +92,4 @@ ulimitcmd     ulimit
>      #endif
>      testcmd                test [
>      killcmd                -u kill
>     +testcmpcmd     test_cmp
>     diff --git a/src/jobs.c b/src/jobs.c
>     index c2c2332..905563f 100644
>     --- a/src/jobs.c
>     +++ b/src/jobs.c
>     @@ -1502,3 +1502,12 @@ getstatus(struct job *job) {
>                     jobno(job), job->nprocs, status, retval));
>             return retval;
>      }
>     +
>     +#include <stdio.h>
>     +int
>     +testcmpcmd(argc, argv)
>     +       int argc;
>     +       char **argv;
>     +{
>     +       fprintf(stderr, "Got %d arguments\n", argc);
>     +}
> 
> I just added that to jobs.c because it was easiest, then test_cmp
> becomes a builtin:
> 
>     u dash (debian/master=) $ src/dash -c 'type test_cmp'
>     test_cmp is a shell builtin
>     u dash (debian/master=) $ src/dash -c 'echo foo && test_cmp 1 2 3'
>     foo
>     Got 4 arguments
> 
> I.e. it's really easy to add new built in commands to the dash shell
> (and probably other shells, but dash is really small & fast).
> 
> We could carry some patch like that to dash, and also patch it so
> test-lib.sh could know that that was our own custom shell, and we'd then
> skip defining functions like test_cmp, and instead use that new builtin.

Or even use the output of `type test_cmp` as a tell-tale.

Show 11 quoted lines

> Similarly, it could then be linked to our own binaries, and the
> test-tool would be a builtin that would appropriately dispatch, and we
> could even eventually make "git" a shell builtin.
> 
> I don't have time or interest to work on this now, but thought it was
> interesting to share. This assumes that something in shellscript like:
> 
>     while echo foo; do echo bar; done
> 
> Is no slower on Windows than *nix, since it's purely using built-ins, as
> opposed to something that would shell out.

It is still interpreting stuff. And it still goes through the POSIX emulation layer.

I did see reports on the Git for Windows bug tracker that gave me the impression that such loops in Unix shell scripts may not, in fact, be as performant in MSYS2's Bash as you would like to believe:

https://github.com/git-for-windows/git/issues/1533#issuecomment-372025449

Ciao, Dscho