Re: [GSOC] How to improve the performance of git cat-file --batch

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jul 24, 2021, 21:20 UTC
Message-ID: <87im0zs8wn.fsf@evledraar.gmail.com>
In-Reply-To: <CAOLTT8RR3nvtXotqhSO8xPCzGQpGUA8dnNgraAjREZ6uLf4n4w@mail.gmail.com>

On Sat, Jul 24 2021, ZheNing Hu wrote:

Show 11 quoted lines

> After reusing ref-filter logic for cat-file --batch, the function of
> cat-file --batch has been enhanced,
> but the performance of cat-file is severely degraded. So we need a
> better solution to solve it.
> its last version is here:
> https://lore.kernel.org/git/pull.993.v2.git.1626363626.gitgitgadget@gmail.com/
>
> Use google doc to show some of my recent ideas:
> https://docs.google.com/document/d/1hyM-ecMd8C_TJ3OsOn46nMr8ZxCUxeK7HKj-bRkq3sk/edit?usp=sharing
>
> Anyone is welcome to comment and suggest better solutions!

Having looked at that Google Doc it seems to just discuss "Plan A", or there some multi-doc magic and there's a "Plan B", or...? I'm not very familiar with Google Docs.

Anyway, I'd encourage you to just send this as an E-Mail to the mailing list.

Having skimmed it I'm a bit confused about this in reference to performance generally. I haven't looked into the case you're discussing, but as I noted in https://lore.kernel.org/git/87im1p6x34.fsf@evledraar.gmail.com/ the profiling clearly shows that the main problem is that you've added object lookups we skipped before.

I think any plan to improve performance should start with a perf test, see which of your patches have performance regressions, if some don't and are otherwise OK perhaps those can go in earlier.

Then we'll have the smallest possible set of changes that's correct, but introduces performance regressions, and can look at what those are doing to slow things down...