Re: [PATCH 2/2] for-each-ref: add --count-matches option

From: Junio C Hamano <gitster@pobox.com>
Date: Jun 28, 2023, 17:08 UTC
Message-ID: <xmqqzg4jcy62.fsf@gitster.g>
In-Reply-To: <776c3682-d2eb-d2d7-3ea8-4a7db8cd7842@gmail.com>

Phillip Wood <phillip.wood123@gmail.com> writes:

> So it seems most of the slowdown I was seeing yesterday was due it
> looking up a loose object. I'm surprised repacking makes such a
> difference in a repository that only contains two objects.

If we compare what is done in packfile.c:packed_object_info() and object-file.c:loose_object_info() when we are only interested in finding out the object type, there aren't that many differences in the set of system calls each codepath needs to make.

 * The packfile codepath needs to open and mmap *.pack and *.idx,
   binary search in the .idx for the object location, then read a
   few bytes from .pack, before being able to decode the header to
   find out the type.

 * The loose object codepath needs to open and mmap the loose object
   file, read a few bytes from there, before being abole to decode
   the header to find out the type. After that, it needs to munmap.

The cost of open/mmap for packfile codepath amortises over number of objects (hence number of refs) very well. If there are many refs that point at the same object, cache object layer will kick in to avoid disk access for second and subsequent accesses to the same object, but it helps both codepaths equally, so there should not be much difference either way.

Thanks for a interesting piece of food for thought.