Re: scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Nov 2, 2021, 13:52 UTC
Message-ID: <nycvar.QRO.7.76.6.2111021446140.56@tvgsbejvaqbjf.bet>
In-Reply-To: <20211026201448.GA29480@dcvr>

Hi Eric,

On Tue, 26 Oct 2021, Eric Wong wrote:

Show 14 quoted lines

> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > * Test suite is slow. Shell scripts and process forking.
> >
> >    * What if we had a special shell that interpreted the commands in a
> >      single process?
> >
> >    * Even Git commands like rev-parse and hash-object, as long as that’s
> >      not the command you’re trying to test
>
> This is something I've wanted in a very long time as a scripter.
> fast-import has been great over the years, as is
> "cat-file --batch(-check)", but there's gaps should be filled
> (preferably without fragile linkage of shared libraries into a
> script process)

The conclusion reached at the Summit seemed to be that we don't want to get into that rabbit hole. We might very well end up maintaining a POSIX-compatible shell inside Git. Definitely out of scope.

Show 11 quoted lines

> >    * Dscho wants to slip in a C-based solution
> >
> >    * Jonathan tan commented: going back to your custom shell for tests
> >      idea, one thing we could do is have a custom command that generates
> >      the repo commits that we want (and that saves process spawns and
> >      might make the tests simpler too)
>
> Perhaps a not-seriously-proposed patch from 2006 could be
> modernized for our now-libified internals:
>
> https://yhbt.net/lore/git/Pine.LNX.4.64.0602232229340.3771@g5.osdl.org/

Thanks for digging that out. I had looked for it multiple times over the years, but searched using the wrong search terms.

However, as you can see, it went nowhere. Probably the (implicit) conclusion was the same as above.

Show 7 quoted lines

> >       * We could replace several “setup repo” steps with “git fast-import”
> >         instead.
> >
> >    * Dscho measured: 0.5 sec - 30 sec in setup steps. Can use fast-import,
> >      or can make a new format that helps us set up the test scenario
>
> 0.5s - 30s across the whole suite or individual tests?

That was just vague recollection, but it was for setup steps, i.e. the initial test cases that do not even test Git's functionality but merely want to set up a repository/worktree for the subsequent test cases to play with.

> Having a way to disable fsync globally should further improve
> things, especially for people on slower storage.  libeatmydata
> is available, but perhaps not widely available/known.

What was missing from the notes was the crucial fact that I did this on Windows, i.e. a platform that is pretty darned good at multi-tasking (something with which Linux has historically struggled a bit), but not so good at spawning wholesale processes.

So the problem really is that calling, say, `git commit` in a `for $(test_seq 100)` loop is ridiculously expensive.

Even rewriting those setup test cases to something as verbose as a `fast-import` stream accelerates them like you wouldn't believe.

I even thought I threw out the idea of implementing a test helper that could turn the output of `git log --graph --oneline` into a branch replicating that structure, but it might have gotten lost in the noise.

I doubt that my test suite-centered commentary is very helpful for your use cases, though.

Ciao, Dscho