Re: [PATCH 00/30] [RFC] extensions.refFormat and packed-refs v2 file format

From: Taylor Blau <me@ttaylorr.com>
Date: Nov 30, 2022, 16:37 UTC
Message-ID: <Y4eG1SDxBqjwUhwM@nand.local>
In-Reply-To: <f1c45bd5-692e-85db-90c3-c516003f47e5@github.com>

On Wed, Nov 30, 2022 at 10:16:52AM -0500, Derrick Stolee wrote:

Show 9 quoted lines

> > Do you mean 'enumerate refs' ? Why would you want to count refs by prefix?
>
> * There was a GitHub feature that counted refs and tags, but wanted to
>   ignore internal ref prefixes (outside of refs/heads/* or refs/tags/*).
>   It turns out that we didn't actually need the full count but an
>   existence indicator, but it would be helpful to quickly identify how
>   many branches or tags are in a repository at a glance. Packed-refs v1
>   requires scanning the whole file while packed-refs v2 does a fixed
>   number of binary searches followed by a subtraction of row indexes.

True. On the surface, it seemed odd to use a function which returns something like:

    { "refs/heads/*" => NNNN, "refs/tags/*" => MMMM }

only to check whether or not NNNN and MMMM are zero or non-zero.

But there's a little more to the story. That emptiness check does occur at the beginning of many page loads. But when it responds "non-empty", we then care about how many branches and tags there actually are.

So calling count_refs() (the name of the internal RPC that powers all of this) was an optimization written under the assumption that we actually are going to ask about the exact number of branches/tags very shortly after querying for emptiness.

It turns out that empirically it's faster to do something like:

    $ git show-ref --[heads|tags] | head -n 1

to check if there are any branches and tags at all[^1], and then a follow up 'git show-ref --heads | wc -l' to check how many there are.

But it would be nice to do both operations quickly without having actually scan all of the entries in each prefix.

Thanks, Taylor

[^1]: Some may remember my series in
  https://lore.kernel.org/git/cover.1654552560.git.me@ttaylorr.com/
  which replaced '| head -n 1' with a '--count=1' option. This matches
  what GitHub runs in production where piping one command to another
  from Ruby is unfortunately quite complicated.