Re: [PATCH v1 4/4] rm: integrate with sparse-index

From: Derrick Stolee <derrickstolee@github.com>
Date: Aug 4, 2022, 14:48 UTC
Message-ID: <999169c6-a727-af2a-3361-51ac7b1f1d80@github.com>
In-Reply-To: <20220803045118.1243087-5-shaoxuan.yuan02@gmail.com>

On 8/3/2022 12:51 AM, Shaoxuan Yuan wrote:

> Enable the sparse index within the `git-rm` command.
> 
> The `p2000` tests demonstrate a ~96% execution time reduction for
> 'git rm' using a sparse index.

Sorry that I got sidetracked yesterday when I was reviewing this
series, but I noticed something looking at these results:

> Test                                     before  after
> -------------------------------------------------------------
> 2000.74: git rm -f f2/f4/a (full-v3)     0.66    0.88 +33.0%
> 2000.75: git rm -f f2/f4/a (full-v4)     0.67    0.75 +12.0%

The range of _growth_ here seemed odd, so I wanted to check if this was due to a small sample size or not.

> 2000.76: git rm -f f2/f4/a (sparse-v3)   1.99    0.08 -96.0%
> 2000.77: git rm -f f2/f4/a (sparse-v4)   2.06    0.07 -96.6%

These numbers are as expected.

>  test_perf_on_all git read-tree -mu HEAD
>  test_perf_on_all git checkout-index -f --all
>  test_perf_on_all git update-index --add --remove $SPARSE_CONE/a
> +test_perf_on_all git rm -f $SPARSE_CONE/a

At first, I was confused why we needed '-f' and thought that maybe this was turning into a no-op after the first deletion. However, the test_perf_on_all helper does an "echo >>$SPARSE_CONE/a" before hand, so the file exists _in the worktree_ every time. That requires '-f' since otherwise Git complains that we have modifications.

However, after the first instance the file no longer exists in the index, so we are losing some testing of the index modification.

We can fix this by resetting the index in each test loop:

  test_perf_on_all "git rm -f $SPARSE_CONE/a && git checkout HEAD -- $SPARSE_CONE/a"

Running this version of the test with GIT_PERF_REPEAT_COUNT=10 and using the Git repository itself, I get these numbers:

Test HEAD~1 HEAD -------------------------------------------------------------------------- 2000.74: git rm ... (full-v3) 0.41(0.37+0.05) 0.43(0.36+0.07) +4.9% 2000.75: git rm ... (full-v4) 0.38(0.34+0.05) 0.39(0.35+0.05) +2.6% 2000.76: git rm ... (sparse-v3) 0.57(0.56+0.01) 0.05(0.05+0.00) -91.2% 2000.77: git rm ... (sparse-v4) 0.57(0.55+0.02) 0.03(0.03+0.00) -94.7%

Yes, the 'git checkout' command is contributing to the overall numbers, but it also already has the performance improvements of the sparse-index, so it contributes only a little to the performance on the left.

(Also note that the full index cases change only by amounts within reasonable noise. The repeat count helps there.)

Thanks, -Stolee