Re: [GSoC] Git Blog 13

From: Christian Couder <christian.couder@gmail.com>
Date: Aug 15, 2021, 10:28 UTC
Message-ID: <CAP8UFD3FpnmOi50iUuFa9qMzi=PVP_Kcod6T+8BzJNq5X-cmgw@mail.gmail.com>
In-Reply-To: <CAOLTT8Q6f-B05bQ+XCDuGGk5KTS1o3QatGbeJ-FQiqFCYGW7Og@mail.gmail.com>

On Sun, Aug 15, 2021 at 9:08 AM ZheNing Hu <adlternative@gmail.com> wrote:

>
> My thirteenth week blog finished:
> The web version is here:
> https://adlternative.github.io/GSOC-Git-Blog-13/

Thanks!

Show 6 quoted lines

> ### Project Progress
>
> This week I continue to try to optimize the performance of `git
> cat-file --batch`.
>
> You can see them here:

Show 10 quoted lines

> Test                                        upstream/master   this tree
> ------------------------------------------------------------------------------------
> 1006.2: cat-file --batch-check              0.08(0.06+0.01)
> 0.08(0.07+0.01) +0.0%
> 1006.3: cat-file --batch-check with atoms   0.06(0.05+0.00)
> 0.07(0.06+0.01) +16.7%
> 1006.4: cat-file --batch                    0.48(0.45+0.03)
> 0.50(0.46+0.03) +4.2%
> 1006.5: cat-file --batch with atoms         0.47(0.43+0.03)
> 0.49(0.46+0.02) +4.3%

Show 11 quoted lines

> The performance of `git cat-file --batch-check` is very close
> to `upstream/master`!
>
> The performance difference of `git cat-file --batch` has also
> changed from 25.0% to 4.2%!
>
> This result is far better than my expectations, I have reason to
> believe that the performance of `git cat-file --batch` can be
> improved again!
>
> Good job!

Yeah, great!

> I think the key to continuing to optimize is still to reduce unnecessary copies.

Do you have data or hints about this?

Have you looked at why there is still +16.7% in the cat-file --batch-check with atoms case? Maybe solving this would improve things in the other cases too.

Show 7 quoted lines

> ### Additional advice
>
> During the optimization process this week, I found that I might want
> to use a `strbuf_move()` function, although I did not adopt it in my work
> (because it did not allow my work to be greatly optimized), but I think it might
> be useful in some situations: We don’t want to copy the data of strbuf,
> but just want to move its buf pointer:

Yeah, if we are sure that it improves performance, that might be worth it.

About your patch series (https://lore.kernel.org/git/pull.1016.git.1628842990.gitgitgadget@gmail.com/), I wonder if it might be possible and better to have some performance improvements before the commit that uses the ref-filter code ("cat-file: reuse ref-filter logic"). If you are going this way, it might make sense to add a few performance tests, if some don't already exist, to show the effect of these performance improvements on the ref-filter code when it's used by other commands like for-each-ref.

This could show that these performance improvements are worth doing even if we didn't want to reuse the ref-filter code in cat-file. And then perhaps these performance improvements could be merged as part of one or more small patch series, before the patch series that reuses the ref-filter code in cat-file.

Best, Christian.