Re: [PATCH 6/8] t/perf: add Scalar performance tests

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Sep 1, 2022, 09:39 UTC
Message-ID: <59s7o8n8-5797-q2qo-4q0o-93qppnr92q52@tzk.qr>
In-Reply-To: <42ab39f21212d3da1af3546d3425aa790637056f.1661961746.git.gitgitgadget@gmail.com>

Hi Victoria,

On Wed, 31 Aug 2022, Victoria Dye via GitGitGadget wrote:

Show 14 quoted lines

> From: Victoria Dye <vdye@github.com>
>
> Create 'p9210-scalar.sh' for testing Scalar performance and comparing
> performance of Git operations in Scalar registrations and standard
> repositories. Example results:
>
> Test                                                   this tree
> ------------------------------------------------------------------------
> 9210.2: scalar clone                                   14.82(18.00+3.63)
> 9210.3: git clone                                      26.15(36.67+6.90)
> 9210.4: git status (scalar)                            0.04(0.01+0.01)
> 9210.5: git status (non-scalar)                        0.10(0.02+0.11)
> 9210.6: test_commit --append --no-tag A (scalar)       0.08(0.02+0.03)
> 9210.7: test_commit --append --no-tag A (non-scalar)   0.13(0.03+0.11)

Excellent!

Show 21 quoted lines

> [...]
> +
> +test_compare_perf () {
> +	command="$@"
> +	test_perf "$command (scalar)" "
> +		(
> +			cd scalar-clone/src &&
> +			$command
> +		)
> +	"
> +
> +	test_perf "$command (non-scalar)" "
> +		(
> +			cd git-clone &&
> +			$command
> +		)
> +	"
> +}
> +
> +test_compare_perf git status
> +test_compare_perf test_commit --append --no-tag A

Given the small numbers presented in the commit message, I suspect that even so much as running the command in a subshell might skew the timings at least on Windows, where subshells are very, very expensive.

Given that both `git` and `test_commit` understand the `-C <directory>` syntax, this variant would resolve my concern:

	test_compare_perf () {
	     command=$1
             shift
	     args="$*"

	     test_perf "$command (scalar)" "
	             $command -C scalar-clone/src $args
	     "

	     test_perf "$command (non-scalar)" "
	             $command -C git-clone $args
	     "
	}

What do you think?

Ciao, Dscho