threads / patch / 65767

patchdescribe: limit default ref iteration to tags

Subject: [PATCH] describe: limit default ref iteration to tags

## tl;dr

5 messages between Jun 7, 2026 and Jun 8, 2026. Diffs are folded; open one to read it.

replies: 4people: 3as markdown or json

Tamir Duberstein· Jun 7, 2026, 20:51 UTC · lore

Unless --all is given, get_name() rejects every ref outside refs/tags/. The rejection happens only after the ref backend has enumerated the ref, so repositories with many other refs spend most of a simple describe invocation visiting refs which cannot affect its result.

Commit 8a5a1884e9 (Avoid accessing non-tag refs in git-describe unless --all is requested, 2008-02-24) moved this rejection before object lookup, but left iteration unscoped. Pass the existing refs/tags/ restriction to the iterator unless --all is given so the backend can avoid unrelated refs.

On a checkout with 124,357 refs, of which 330 were tags, I ran the following command with the parent and patched binaries:

    hyperfine --warmup 3 --runs 15 \
        'git describe --always --long --abbrev=40 HEAD'
The results were:
             parent       this commit
  elapsed    196.2 ms      63.3 ms
  user        69.5 ms      48.0 ms
  system     123.0 ms      12.0 ms

The wall-time standard deviations were 13.2 ms and 2.6 ms, respectively, for a 3.10x speedup.

Both revisions were built with -O3, -mcpu=native, and ThinLTO using Apple clang 21.0.0 on macOS 26.5. The machine was a MacBook Pro (Mac16,6) with a 16-core Apple M4 Max (12 performance and four efficiency cores) and 128 GB RAM.

Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
 builtin/describe.c       |  3 +++
 t/perf/p6100-describe.sh | 20 ++++++++++++++++++++
 2 files changed, 23 insertions(+)
Show changes to 2 files +23 −1

builtin/describe.c, t/perf/p6100-describe.sh

diff --git a/builtin/describe.c b/builtin/describe.c
index 1c47d7c0b7..3532c8ff22 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -740,6 +740,9 @@ int cmd_describe(int argc,
 		return ret;
 	}
 
+	if (!all)
+		for_each_ref_opts.prefix = "refs/tags/";
+
 	hashmap_init(&names, commit_name_neq, NULL, 0);
 	refs_for_each_ref_ext(get_main_ref_store(the_repository),
 			      get_name, NULL, &for_each_ref_opts);
diff --git a/t/perf/p6100-describe.sh b/t/perf/p6100-describe.sh
index 069f91ce49..dfcaf59e90 100755
--- a/t/perf/p6100-describe.sh
+++ b/t/perf/p6100-describe.sh
@@ -5,6 +5,12 @@ test_description='performance of git-describe'
 
 test_perf_default_repo
 
+test_lazy_prereq PERF_REFFILES '
+	test "$(git rev-parse --show-ref-format)" = files
+'
+
+ref_count=10000
+
 # clear out old tags and give us a known state
 test_expect_success 'set up tags' '
 	git for-each-ref --format="delete %(refname)" refs/tags >to-delete &&
@@ -27,4 +33,18 @@ test_perf 'describe HEAD with one tag' '
 	git describe --match=new HEAD
 '
 
+test_expect_success PERF_REFFILES 'set up many unrelated refs' '
+	git tag -m tip tip HEAD &&
+	for i in $(test_seq $ref_count)
+	do
+		printf "create refs/heads/describe-perf/%05d HEAD\n" $i ||
+		return 1
+	done >instructions &&
+	git update-ref --stdin <instructions
+'
+
+test_perf 'describe exact tag with many loose refs' --prereq PERF_REFFILES '
+	git describe --exact-match HEAD
+'
+
 test_done

---
base-commit: 9ac3f193c05c2237e2b14ebaa1149e9fc8a1abe0
change-id: 20260607-describe-tag-ref-scope-7d00ae140a58

Best regards,
--  
Tamir Duberstein <tamird@gmail.com>
Patrick Steinhardt· Jun 8, 2026, 06:59 UTC · re: Tamir Duberstein · lore

Re: [PATCH] describe: limit default ref iteration to tags

On Sun, Jun 07, 2026 at 04:51:53PM -0400, Tamir Duberstein wrote:
> Unless --all is given, get_name() rejects every ref outside refs/tags/.
> The rejection happens only after the ref backend has enumerated the ref,
> so repositories with many other refs spend most of a simple describe
> invocation visiting refs which cannot affect its result.
Right. The relevant block is this one:
	if (skip_prefix(ref->name, "refs/tags/", &path_to_match)) {
		is_tag = 1;
	} else if (all) {
		if ((exclude_patterns.nr || patterns.nr) &&
		    !skip_prefix(ref->name, "refs/heads/", &path_to_match) &&
		    !skip_prefix(ref->name, "refs/remotes/", &path_to_match)) {
			/* Only accept reference of known type if there are match/exclude patterns */
			return 0;
		}
	} else {
		/* Reject anything outside refs/tags/ unless --all */
		return 0;
	}
So we really only use tags unless "--all" is given.
Show 18 quoted lines
> Commit 8a5a1884e9 (Avoid accessing non-tag refs in git-describe unless
> --all is requested, 2008-02-24) moved this rejection before object
> lookup, but left iteration unscoped. Pass the existing refs/tags/
> restriction to the iterator unless --all is given so the backend can
> avoid unrelated refs.
> 
> On a checkout with 124,357 refs, of which 330 were tags, I ran the
> following command with the parent and patched binaries:
> 
>     hyperfine --warmup 3 --runs 15 \
>         'git describe --always --long --abbrev=40 HEAD'
> 
> The results were:
> 
>              parent       this commit
>   elapsed    196.2 ms      63.3 ms
>   user        69.5 ms      48.0 ms
>   system     123.0 ms      12.0 ms

It's a bit curious that you don't post the hyperfine(1) results as-is here.

> The wall-time standard deviations were 13.2 ms and 2.6 ms, respectively,
> for a 3.10x speedup.

Makes sense that this would result in a sizeable speedup, depending of course on the shape of the existing refs in the repository.

Show 14 quoted lines
> diff --git a/builtin/describe.c b/builtin/describe.c
> index 1c47d7c0b7..3532c8ff22 100644
> --- a/builtin/describe.c
> +++ b/builtin/describe.c
> @@ -740,6 +740,9 @@ int cmd_describe(int argc,
>  		return ret;
>  	}
>  
> +	if (!all)
> +		for_each_ref_opts.prefix = "refs/tags/";
> +
>  	hashmap_init(&names, commit_name_neq, NULL, 0);
>  	refs_for_each_ref_ext(get_main_ref_store(the_repository),
>  			      get_name, NULL, &for_each_ref_opts);

Another performance optimization that we could do here is to wire up the exclude patterns via `for_each_ref_opts.exclude_patterns`. But that's outside the scope of this patch series, and also much less likely to help many use cases out there.

Show 13 quoted lines
> diff --git a/t/perf/p6100-describe.sh b/t/perf/p6100-describe.sh
> index 069f91ce49..dfcaf59e90 100755
> --- a/t/perf/p6100-describe.sh
> +++ b/t/perf/p6100-describe.sh
> @@ -5,6 +5,12 @@ test_description='performance of git-describe'
>  
>  test_perf_default_repo
>  
> +test_lazy_prereq PERF_REFFILES '
> +	test "$(git rev-parse --show-ref-format)" = files
> +'
> +
> +ref_count=10000
Let's not declare this variable outside of tests.
Show 13 quoted lines
> @@ -27,4 +33,18 @@ test_perf 'describe HEAD with one tag' '
>  	git describe --match=new HEAD
>  '
>  
> +test_expect_success PERF_REFFILES 'set up many unrelated refs' '
> +	git tag -m tip tip HEAD &&
> +	for i in $(test_seq $ref_count)
> +	do
> +		printf "create refs/heads/describe-perf/%05d HEAD\n" $i ||
> +		return 1
> +	done >instructions &&
> +	git update-ref --stdin <instructions
> +'

Why is this limited to the "files" backend, only? The logic should work for both backends as-is.

Thanks!
Patrick
Tamir Duberstein· Jun 8, 2026, 15:46 UTC · re: Patrick Steinhardt · lore

Re: [PATCH] describe: limit default ref iteration to tags

On Sun, Jun 7, 2026 at 11:59 PM Patrick Steinhardt <ps@pks.im> wrote:
Show 46 quoted lines
>
> On Sun, Jun 07, 2026 at 04:51:53PM -0400, Tamir Duberstein wrote:
> > Unless --all is given, get_name() rejects every ref outside refs/tags/.
> > The rejection happens only after the ref backend has enumerated the ref,
> > so repositories with many other refs spend most of a simple describe
> > invocation visiting refs which cannot affect its result.
>
> Right. The relevant block is this one:
>
>         if (skip_prefix(ref->name, "refs/tags/", &path_to_match)) {
>                 is_tag = 1;
>         } else if (all) {
>                 if ((exclude_patterns.nr || patterns.nr) &&
>                     !skip_prefix(ref->name, "refs/heads/", &path_to_match) &&
>                     !skip_prefix(ref->name, "refs/remotes/", &path_to_match)) {
>                         /* Only accept reference of known type if there are match/exclude patterns */
>                         return 0;
>                 }
>         } else {
>                 /* Reject anything outside refs/tags/ unless --all */
>                 return 0;
>         }
>
> So we really only use tags unless "--all" is given.
>
> > Commit 8a5a1884e9 (Avoid accessing non-tag refs in git-describe unless
> > --all is requested, 2008-02-24) moved this rejection before object
> > lookup, but left iteration unscoped. Pass the existing refs/tags/
> > restriction to the iterator unless --all is given so the backend can
> > avoid unrelated refs.
> >
> > On a checkout with 124,357 refs, of which 330 were tags, I ran the
> > following command with the parent and patched binaries:
> >
> >     hyperfine --warmup 3 --runs 15 \
> >         'git describe --always --long --abbrev=40 HEAD'
> >
> > The results were:
> >
> >              parent       this commit
> >   elapsed    196.2 ms      63.3 ms
> >   user        69.5 ms      48.0 ms
> >   system     123.0 ms      12.0 ms
>
> It's a bit curious that you don't post the hyperfine(1) results as-is
> here.
Agreed, will include that in v2. For reference:
        Benchmark 1: parent
          Time (mean ± σ):     171.7 ms ±  18.5 ms    [User: 23.9 ms,
System: 133.6 ms]
          Range (min … max):   142.3 ms … 198.3 ms    15 runs
        Benchmark 2: this commit
          Time (mean ± σ):       9.9 ms ±   1.1 ms    [User: 3.3 ms,
System: 4.7 ms]
          Range (min … max):     8.8 ms …  13.1 ms    15 runs
Show 26 quoted lines
>
> > The wall-time standard deviations were 13.2 ms and 2.6 ms, respectively,
> > for a 3.10x speedup.
>
> Makes sense that this would result in a sizeable speedup, depending of
> course on the shape of the existing refs in the repository.
>
> > diff --git a/builtin/describe.c b/builtin/describe.c
> > index 1c47d7c0b7..3532c8ff22 100644
> > --- a/builtin/describe.c
> > +++ b/builtin/describe.c
> > @@ -740,6 +740,9 @@ int cmd_describe(int argc,
> >               return ret;
> >       }
> >
> > +     if (!all)
> > +             for_each_ref_opts.prefix = "refs/tags/";
> > +
> >       hashmap_init(&names, commit_name_neq, NULL, 0);
> >       refs_for_each_ref_ext(get_main_ref_store(the_repository),
> >                             get_name, NULL, &for_each_ref_opts);
>
> Another performance optimization that we could do here is to wire up the
> exclude patterns via `for_each_ref_opts.exclude_patterns`. But that's
> outside the scope of this patch series, and also much less likely to
> help many use cases out there.
I tried this and have a separate patch prepared.

The patterns cannot be passed through verbatim: `git describe --exclude=foo` excludes the exact name `foo`, while the refs API would treat `foo` as a directory prefix and also skip `foo/*`. The patch therefore passes only patterns consisting of a literal prefix followed by trailing asterisks, adds back the applicable ref namespace, and retains the existing callback filtering.

With 30,000 packed remote-tracking refs under an excluded prefix, the perf test invokes `git describe` ten times per run:

```
                                  master           patched
describe excluding many refs   0.16(0.07+0.05)  0.12(0.04+0.05)
```

That is a 25% wall-time reduction, with user CPU falling from 0.07 to 0.04 seconds.

I also tested a larger checkout with 62,170 refs under `refs/remotes/origin/`:

``` git describe --all --exact-match --exclude='origin/*' HEAD ```

This improved from 176.7 ms to 161.3 ms, or about 10%. Startup work unrelated to ref iteration dominates more of that repository's runtime.

Show 16 quoted lines
>
> > diff --git a/t/perf/p6100-describe.sh b/t/perf/p6100-describe.sh
> > index 069f91ce49..dfcaf59e90 100755
> > --- a/t/perf/p6100-describe.sh
> > +++ b/t/perf/p6100-describe.sh
> > @@ -5,6 +5,12 @@ test_description='performance of git-describe'
> >
> >  test_perf_default_repo
> >
> > +test_lazy_prereq PERF_REFFILES '
> > +     test "$(git rev-parse --show-ref-format)" = files
> > +'
> > +
> > +ref_count=10000
>
> Let's not declare this variable outside of tests.
Done in v2.
Show 17 quoted lines
>
> > @@ -27,4 +33,18 @@ test_perf 'describe HEAD with one tag' '
> >       git describe --match=new HEAD
> >  '
> >
> > +test_expect_success PERF_REFFILES 'set up many unrelated refs' '
> > +     git tag -m tip tip HEAD &&
> > +     for i in $(test_seq $ref_count)
> > +     do
> > +             printf "create refs/heads/describe-perf/%05d HEAD\n" $i ||
> > +             return 1
> > +     done >instructions &&
> > +     git update-ref --stdin <instructions
> > +'
>
> Why is this limited to the "files" backend, only? The logic should work
> for both backends as-is.
You're right, fixed in v2.
>
> Thanks!
Thanks for the quick review!
Junio C Hamano· Jun 8, 2026, 12:36 UTC · re: Tamir Duberstein · lore

Re: [PATCH] describe: limit default ref iteration to tags

Tamir Duberstein <tamird@gmail.com> writes:
[jc: Removing Shawn from CC who passed away quite a while ago, RIP].
Show 15 quoted lines
> Unless --all is given, get_name() rejects every ref outside refs/tags/.
> The rejection happens only after the ref backend has enumerated the ref,
> so repositories with many other refs spend most of a simple describe
> invocation visiting refs which cannot affect its result.
> ...
> Both revisions were built with -O3, -mcpu=native, and ThinLTO using
> Apple clang 21.0.0 on macOS 26.5. The machine was a MacBook Pro
> (Mac16,6) with a 16-core Apple M4 Max (12 performance and four
> efficiency cores) and 128 GB RAM.
>
> Signed-off-by: Tamir Duberstein <tamird@gmail.com>
> ---
>  builtin/describe.c       |  3 +++
>  t/perf/p6100-describe.sh | 20 ++++++++++++++++++++
>  2 files changed, 23 insertions(+)

Interesting. How would this relate to and work well with <20260601233727.43558-1-jacob.e.keller@intel.com>?

Show 27 quoted lines
> +test_lazy_prereq PERF_REFFILES '
> +	test "$(git rev-parse --show-ref-format)" = files
> +'
> +
> +ref_count=10000
> +
>  # clear out old tags and give us a known state
>  test_expect_success 'set up tags' '
>  	git for-each-ref --format="delete %(refname)" refs/tags >to-delete &&
> @@ -27,4 +33,18 @@ test_perf 'describe HEAD with one tag' '
>  	git describe --match=new HEAD
>  '
>  
> +test_expect_success PERF_REFFILES 'set up many unrelated refs' '
> +	git tag -m tip tip HEAD &&
> +	for i in $(test_seq $ref_count)
> +	do
> +		printf "create refs/heads/describe-perf/%05d HEAD\n" $i ||
> +		return 1
> +	done >instructions &&
> +	git update-ref --stdin <instructions
> +'
> +
> +test_perf 'describe exact tag with many loose refs' --prereq PERF_REFFILES '
> +	git describe --exact-match HEAD
> +'
> +

Is there a strong reason to guard this new test behind `PERF_REFFILES`?

Even though the penalty of enumerating 10,000 unrelated loose references may be most pronounced in the `files` backend, skipping unnecessary reference enumeration is an architectural win for other backends (like `reftable` or a fully packed repository) as well.

If we drop `PERF_REFFILES` and retitle the test to "describe exact tag with many unrelated refs", we could run it unconditionally to benchmark the improvement across all storage formats.

Tamir Duberstein· Jun 8, 2026, 15:53 UTC · re: Junio C Hamano · lore

Re: [PATCH] describe: limit default ref iteration to tags

On Mon, Jun 8, 2026 at 5:36 AM Junio C Hamano <gitster@pobox.com> wrote:
Show 23 quoted lines
>
> Tamir Duberstein <tamird@gmail.com> writes:
>
> [jc: Removing Shawn from CC who passed away quite a while ago, RIP].
>
> > Unless --all is given, get_name() rejects every ref outside refs/tags/.
> > The rejection happens only after the ref backend has enumerated the ref,
> > so repositories with many other refs spend most of a simple describe
> > invocation visiting refs which cannot affect its result.
> > ...
> > Both revisions were built with -O3, -mcpu=native, and ThinLTO using
> > Apple clang 21.0.0 on macOS 26.5. The machine was a MacBook Pro
> > (Mac16,6) with a 16-core Apple M4 Max (12 performance and four
> > efficiency cores) and 128 GB RAM.
> >
> > Signed-off-by: Tamir Duberstein <tamird@gmail.com>
> > ---
> >  builtin/describe.c       |  3 +++
> >  t/perf/p6100-describe.sh | 20 ++++++++++++++++++++
> >  2 files changed, 23 insertions(+)
>
> Interesting.  How would this relate to and work well with
> <20260601233727.43558-1-jacob.e.keller@intel.com>?

They are orthogonal. That patch changes the argument construction inside the `contains` block, which invokes `cmd_name_rev()` and returns. This patch changes the ref iterator used after that block, so it only affects the ordinary, non-`--contains` path.

Show 40 quoted lines
>
> > +test_lazy_prereq PERF_REFFILES '
> > +     test "$(git rev-parse --show-ref-format)" = files
> > +'
> > +
> > +ref_count=10000
> > +
> >  # clear out old tags and give us a known state
> >  test_expect_success 'set up tags' '
> >       git for-each-ref --format="delete %(refname)" refs/tags >to-delete &&
> > @@ -27,4 +33,18 @@ test_perf 'describe HEAD with one tag' '
> >       git describe --match=new HEAD
> >  '
> >
> > +test_expect_success PERF_REFFILES 'set up many unrelated refs' '
> > +     git tag -m tip tip HEAD &&
> > +     for i in $(test_seq $ref_count)
> > +     do
> > +             printf "create refs/heads/describe-perf/%05d HEAD\n" $i ||
> > +             return 1
> > +     done >instructions &&
> > +     git update-ref --stdin <instructions
> > +'
> > +
> > +test_perf 'describe exact tag with many loose refs' --prereq PERF_REFFILES '
> > +     git describe --exact-match HEAD
> > +'
> > +
>
> Is there a strong reason to guard this new test behind
> `PERF_REFFILES`?
>
> Even though the penalty of enumerating 10,000 unrelated loose
> references may be most pronounced in the `files` backend, skipping
> unnecessary reference enumeration is an architectural win for other
> backends (like `reftable` or a fully packed repository) as well.
>
> If we drop `PERF_REFFILES` and retitle the test to "describe exact
> tag with many unrelated refs", we could run it unconditionally to
> benchmark the improvement across all storage formats.

Yeah, there's no good reason - and Patrick made the same observation. In v2 I will remove the prerequisite and rename the case to refer to unrelated rather than loose refs.

← back to recent threads