Re: [GSOC] How to improve the performance of git cat-file --batch

From: Christian Couder <christian.couder@gmail.com>
Date: Jul 26, 2021, 09:38 UTC
Message-ID: <CAP8UFD1szPO-qmusSqiLRWxynV0gcy8UsnZORNn0USYAmRUGVw@mail.gmail.com>
In-Reply-To: <CAOLTT8Sbusr8=iJbG1qXcSerivZqP5xm-GS8R7TqDMh7QXSZaA@mail.gmail.com>

On Sun, Jul 25, 2021 at 2:04 PM ZheNing Hu <adlternative@gmail.com> wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> 于2021年7月25日周日 上午5:23写道：

Show 8 quoted lines

> > Having skimmed it I'm a bit confused about this in reference to
> > performance generally. I haven't looked into the case you're discussing,
> > but as I noted in
> > https://lore.kernel.org/git/87im1p6x34.fsf@evledraar.gmail.com/ the
> > profiling clearly shows that the main problem is that you've added
> > object lookups we skipped before.
>
> Yeah, you showed me last time that lookup_object() took up a lot of time.

Could the document explain with some details why there are more calls to lookup_object()? For example it could take an example `git cat-file --batch ...` command (if possible a simple one), and say which functions like lookup_object() it was using (and how many times) to get the data it needs before using the ref-filter logic, and then the same information after using the ref-filter logic.

It could be nice if there were also some data about how much time used to be spent in lookup_object() and how much time is now spent there, and how this compares with the whole slowdown we are seeing. If Ævar already showed that, you can of course reuse what he already did.

The GIT_TRACE_PERFORMANCE, GIT_TRACE2_PERF env variables and the associated trace_*() and trace2_*() functions might help you with measuring how much time is spent in different functions.