Re: [PATCH v4 00/10] The final building block for a faster rebase -i

Hi Ævar,

On Mon, 29 May 2017, Ævar Arnfjörð Bjarmason wrote:

Okay, here you go, my weekly (or so) clarification about the POSIX emulation layer called MSYS2 (which itself kind of a portable Cygwin).

Whenever Git for Windows has to execute Unix shell scripts (which are not native to Windows, as the "Unix" in "Unix shell scripts" so clearly suggests), we resort to calling the Bash from the MSYS2 project, which spins up a POSIX emulation layer. Git for Windows' own .exe files (and in particular, git.exe) is *not* affected by the POSIX emulation layer, as they are real Win32 programs.

Whenever execution has to bridge into, or out of, the POSIX emulation layer, a few things need to be done. To emulate signal handling, for example, a completely new process has to be spun up that itself has the non-MSYS2 process as a child. The environment has to be converted, to reflect the fact that some things are Unix-y paths (or path lists) inside the POSIX emulation layer and Windows paths outside.

Even when staying within the POSIX emulation layer, some operations are not as cheap as "Linux folks" are used to. For example, to spawn a subshell, due to the absence of a spawn syscall fork() is called, followed by exec(). However, fork() itself is not native to Windows and has to be emulated. The POSIX emulation layer spins up a new process, meticulously copies the entire memory, tries to reopen the file descriptors, network connections, etc (to emulate the fork() semantics).

Obviously, this is anything but cheap.

And this is only a glimpse into the entire problem, as I am not aware of any thorough analysis what is going on in msys-2.0.dll when shell scripts run. All I know is that it slows things down dramatically.

As a consequence, even the simple act of creating a repository, or spawning Win32 processes from within a shell, become quite the contributing factors to the noise of the measurements.

In my hands, a repeat count of 3 always resulted in quite a bit of noise previously.

Mind you, I am working my machine. It has to run two VMs in the background, has multiple browsers and dozens of tabs open, checks for mails and Tweets and RSS feeds and a couple of Skypes are open, too.

So yeah, obviously there is a bit of noise involved.

Test ------------------------------------------------------ 3400.2: rebase on top of a lot of unrelated changes

v2.10.0 v2.12.0 v2.13.0 ------------------------------------------------------------------ 60.65(0.01+0.03) 55.75(0.01+0.07) -8.1% 55.97(0.04+0.09) -7.7%

(wrapped myself, as the ./run output is a lot wider than the 80 columns allowed in plain text email format)

And what does it tell you?

Not much, right? You have no idea about the trend line of the three tests, not even of the standard deviation (not that it would be meaningful for N=3). It is not immediately obvious whether the first run is always a tad slower (or faster), or whether there is no noticable difference between the first and any subsequent runs.

In other words, there is no measure of confidence in those results. We can't say how reliable those numbers are.

And we certainly can't know how much the shell scripting hurts.

Although... let's try something. Let's run the same command in a *Linux VM* on the same machine! Yes, that should give us an idea. So here goes:

Test ------------------------------------------------------ 3400.2: rebase on top of a lot of unrelated changes

v2.10.0 v2.12.0 v2.13.0 --------------------------------------------------------------- 2.08(1.76+0.15) 2.10(1.76+0.15) +1.0% 2.00(1.65+0.15) -3.8%

A ha! Not only does this show a curious *increase* in v2.12.0 (but I'd not put much stock into that, again N=3 is way too low a repetition number), it also shows that the Linux VM runs the same thing roughly 30x faster.

I did see a few speed differences between native git.exe on Windows and the git executable on Linux, but it was barely in the two-digit *percentage* region. Nowhere near the four-digit percentage region.

So now you know how much shell scripting hurts performance testing.

A lot.

It pretty much renders the entire endeavor of testing performance completely and utterly useless.

pu does not build for me:

2017-05-30T11:38:50.0089681Z libgit.a(grep.o): In function `pcre1match': 2017-05-30T11:38:50.0289250Z .../grep.c:411: undefined reference to `__imp_pcre_jit_exec' 2017-05-30T11:38:50.0329160Z collect2.exe: error: ld returned 1 exit status

You mean a test helper designed to do the timing and the setting up so as to time *just* the operations that should be timed?

If so: I am all for it.

If you already go HTML, it would make much more sense to go d3.js. I would even prefer to go c3.js (which uses d3.js) right away. Would make everything so much easier.

Not to mention more portable.

Sure.

It does not need to be static. It can use, say, c3.js, for the added benefit of being able to toggle multiple graphs in the same diagram.

Ciao, Dscho