Re: Merge ORT performance in the wild

From: Elijah Newren <newren@gmail.com>
Date: Oct 6, 2021, 14:58 UTC
Message-ID: <CABPp-BF8R-i0CnYE9mQjvwTWuZCgW0VsTTajWTpi15fk9F48Lg@mail.gmail.com>
In-Reply-To: <9e4411aa-e757-0653-0c2a-24894b13158f@gmail.com>

On Tue, Oct 5, 2021 at 12:42 PM Derrick Stolee <stolee@gmail.com> wrote:

Show 30 quoted lines

>
> Hello all, especially Elijah.
>
> We finally shipped Git 2.33.0 (with VFS for Git) to the Windows OS
> engineers, and the microsoft/git fork enables the ORT merge strategy
> by default in this version.
>
> I know that we are queued to include the ORT merge strategy as on
> by default in Git 2.34.0, and to further support that change (and
> thank Elijah for working hard on the feature), I wanted to share
> some early data on our users interacting with it.
>
> We have about 250 users who have upgraded, relative to ~2,300 users
> who were active on the previous version. However, we saw sufficiently
> high use of 'git cherry-pick' and 'git merge' in these early users to
> share these results for how they are experiencing the new world:
>
> | Builtin     | Percentile | Recursive | ORT   |
> |-------------|------------|-----------|-------|
> | cherry-pick | 50         |  18.4s    | 14.1s |
> | cherry-pick | 80         |  34.9s    | 15.4s |
> | cherry-pick | 95         | 117.9s    | 17.7s |
> | merge       | 50         |   7.7s    |  1.2s |
> | merge       | 80         |  17.9s    | 12.7s |
> | merge       | 95         |  58.9s    | 22.3s |
>
> This matches the results from a synthetic performance test I ran
> in our monorepos: ORT is always faster, but its outlier performance
> is far faster than the outlier performance of the 'recursive'
> strategy.

Great news, thanks for sharing.

This is basically just testing one of the optimizations, too. I'm still curious what the performance difference would be on your large repositories, comparing git <= v2.30's recursive (i.e. before my optimizations, some of which benefitted recursive) with rename detection _not_ turned off, and current ort. I suspect you'd have a hard time convincing users to use the former, though. IIRC, Ben reported one example merge took on the order of an hour, but I don't know which repository that was with or if it even had many renames. Anyway, it's nice to know that even without the consideration of rename detection in the old comparison numbers, you're still seeing nice speedups.