threads / patch / 66544

patchrefs: fix loose-ref seeking across directory boundaries

Subject: [PATCH] refs: fix loose-ref seeking across directory boundaries

## tl;dr

A patch fixes cache_ref_iterator_seek() returning an already-seen ref when a similarly named ref sorts before a directory. Read the story.

replies: 10people: 6as markdown or json

Venkat Ragavan via GitGitGadget· Oct 15, 2026, 12:30 UTC · lore
From: Venkat Ragavan S <mail2venkat21@gmail.com>

When seeking through loose refs, cache_ref_iterator_seek() compares each directory component without its trailing slash. This can confuse a directory with a similarly named ref that sorts before it.

For example, with these refs:
    refs/tags/releases-z
    refs/tags/releases/v1
    refs/tags/releases/v2

Starting after refs/tags/releases/v1 can position the iterator at releases-z instead of descending into releases/. Iteration then returns v1 again, even though it should return only v2.

Include the trailing slash when comparing directory components so that the seek follows the correct directory.

Also handle two related positioning errors. If every entry in a directory precedes the cursor, stop searching that level rather than descending into its last entry. If a leaf sorts after the cursor, leave the iterator positioned to return it, even when the cursor has remaining path components.

Add regression tests for shared-prefix names, missing nested markers, and markers beyond the end of the ref listing, using loose, packed, and mixed refs.

Signed-off-by: Venkat Ragavan S <mail2venkat21@gmail.com>
---
    refs: fix loose-ref seeking across directory boundaries
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2258%2Fvenkatr21%2Fvenkatr21%2Ffix-loose-ref-seek-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2258/venkatr21/venkatr21/fix-loose-ref-seek-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2258
 refs/ref-cache.c               |  6 ++--
 t/t6302-for-each-ref-filter.sh | 55 ++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 2 deletions(-)
Show changes to 2 files +59 −2

refs/ref-cache.c, t/t6302-for-each-ref-filter.sh

diff --git a/refs/ref-cache.c b/refs/ref-cache.c
index ffef01a597..a3bc2dc11b 100644
--- a/refs/ref-cache.c
+++ b/refs/ref-cache.c
@@ -506,7 +506,7 @@ static int cache_ref_iterator_seek(struct ref_iterator *ref_iterator,
 			sort_ref_dir(dir);
 
 			slash = strchr(slash, '/');
-			len = slash ? (size_t)(slash - refname) : strlen(refname);
+			len = slash ? (size_t)(slash - refname + 1) : strlen(refname);
 
 			for (idx = 0; idx < dir->nr; idx++) {
 				cmp = strncmp(refname, dir->entries[idx]->name, len);
@@ -520,6 +520,8 @@ static int cache_ref_iterator_seek(struct ref_iterator *ref_iterator,
 				slash = slash + 1;
 
 			level->index = idx;
+			if (cmp > 0)
+				break;
 			if (dir->entries[idx]->flag & REF_DIR) {
 				/* push down a level */
 				dir = get_ref_dir(dir->entries[idx]);
@@ -532,7 +534,7 @@ static int cache_ref_iterator_seek(struct ref_iterator *ref_iterator,
 				level->prefix_state = PREFIX_CONTAINS_DIR;
 			} else {
 				/* reduce the index so the leaf node is iterated over */
-				if (cmp <= 0 && !slash)
+				if (cmp < 0 || (cmp == 0 && !slash))
 					level->index = idx - 1;
 				/*
 				 * while the seek path may not be exhausted, our
diff --git a/t/t6302-for-each-ref-filter.sh b/t/t6302-for-each-ref-filter.sh
index 7f060d97bf..826fc1b9f2 100755
--- a/t/t6302-for-each-ref-filter.sh
+++ b/t/t6302-for-each-ref-filter.sh
@@ -746,6 +746,61 @@ test_expect_success 'start after used with a pattern' '
 	test_cmp expect actual
 '
 
+for storage in loose packed mixed
+do
+	test_expect_success "start after respects directory boundaries ($storage)" '
+		test_when_finished "rm -rf repo" &&
+		git init repo &&
+		(
+			cd repo &&
+			test_commit --no-tag base &&
+			git update-ref refs/tags/packed HEAD &&
+			if test "$storage" = mixed
+			then
+				git pack-refs --all || return 1
+			fi &&
+			git update-ref --stdin <<-\EOF &&
+			create refs/tags/a-leaf HEAD
+			create refs/tags/a/nested HEAD
+			create refs/tags/a0 HEAD
+			create refs/tags/b HEAD
+			create refs/tags/releases-z HEAD
+			create refs/tags/releases/v1 HEAD
+			create refs/tags/releases/v2 HEAD
+			EOF
+			if test "$storage" = packed
+			then
+				git pack-refs --all || return 1
+			fi &&
+			echo refs/tags/releases/v2 >expect &&
+			git for-each-ref --no-sort --format="%(refname)" \
+				--start-after=refs/tags/releases/v1 >actual &&
+			test_cmp expect actual &&
+			cat >expect <<-\EOF &&
+			refs/tags/a0
+			refs/tags/b
+			refs/tags/packed
+			refs/tags/releases-z
+			refs/tags/releases/v1
+			refs/tags/releases/v2
+			EOF
+			git for-each-ref --no-sort --format="%(refname)" \
+				--start-after=refs/tags/a/next >actual &&
+			test_cmp expect actual &&
+			tail -n 5 expect >expect-later &&
+			git for-each-ref --no-sort --format="%(refname)" \
+				--start-after=refs/tags/aa/missing >actual &&
+			test_cmp expect-later actual &&
+			for marker in refs/tags/zzzz refs/zzzz zzzz
+			do
+				git for-each-ref --no-sort --format="%(refname)" \
+					--start-after="$marker" >actual &&
+				test_must_be_empty actual || return 1
+			done
+		)
+	'
+done
+
 test_expect_success 'start after used with custom sort order' '
 	cat >expect <<-\EOF &&
 	fatal: cannot use --start-after with custom sort options

base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
-- 
gitgitgadget
Harald Nordgren· Oct 15, 2026, 13:36 UTC · re: Venkat Ragavan via GitGitGadget · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

The general idea looks sound.
> +			if test "$storage" = mixed
> +			then
> +				git pack-refs --all || return 1
> +			fi &&

I don't see any reason why a test would need an if block, we know what data we expect, no?

Harald
D. Ben Knoble· Oct 15, 2026, 17:08 UTC · re: Harald Nordgren · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

On Thu, Oct 15, 2026 at 10:20 AM Harald Nordgren <haraldnordgren@gmail.com> wrote: [snip]

Show 7 quoted lines
> > +                     if test "$storage" = mixed
> > +                     then
> > +                             git pack-refs --all || return 1
> > +                     fi &&
>
> I don't see any reason why a test would need an if block, we know what data we
> expect, no?

I didn't examine too closely, but this test is wrapped in a loop over the storage variable that appears to determine whether and when we pack-refs, so I think the conditional makes sense for that?

-- 
D. Ben Knoble
Junio C Hamano· Oct 15, 2026, 19:13 UTC · re: Harald Nordgren · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

Harald Nordgren <haraldnordgren@gmail.com> writes:
Show 9 quoted lines
> The general idea looks sound.
>
>> +			if test "$storage" = mixed
>> +			then
>> +				git pack-refs --all || return 1
>> +			fi &&
>
> I don't see any reason why a test would need an if block, we know what data we
> expect, no?

This is three tests implemented as a single test with a for-loop around it. The variable instructs the single test_expect_success to behave slightly differently depending on its setting.

+for storage in loose packed mixed +do + test_expect_success "start after respects directory boundaries ($storage)" ' + test_when_finished "rm -rf repo" && + git init repo && + ( + cd repo && + test_commit --no-tag base && + git update-ref refs/tags/packed HEAD && + if test "$storage" = mixed + then + git pack-refs --all || return 1 + fi && + git update-ref --stdin <<-\EOF && + create refs/tags/a-leaf HEAD + ... + EOF + if test "$storage" = packed + then + git pack-refs --all || return 1 + fi && + ... real test in the repository begins ... + ) + ' +done

It creates a tag "packed", and packs when storage is "mixed". Then we create many other refs, and packs the whole refs when storage is "packed". So during these three runs of the for-loop, the preparation part quoted above behaves this way:

    storage=loose	no 'git pack-refs' run and every ref is loose
    storage=packed	the later 'git pack-refs' packs everything
    storage=mixed	'packed' tag is packed, everything else is loose

and the test checks what happens in this repository with refs packed in three different ways.

Harald Nordgren· Oct 15, 2026, 21:17 UTC · re: Junio C Hamano · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

Show 13 quoted lines
> > The general idea looks sound.
> >
> >> +                    if test "$storage" = mixed
> >> +                    then
> >> +                            git pack-refs --all || return 1
> >> +                    fi &&
> >
> > I don't see any reason why a test would need an if block, we know what data we
> > expect, no?
>
> This is three tests implemented as a single test with a for-loop
> around it.  The variable instructs the single test_expect_success
> to behave slightly differently depending on its setting.

I realize now that I didn't read the code properly. I didn't expect to see a loop encapsulating a call to test_expect_success. That explains the if case as well.

Harald
Junio C Hamano· Oct 27, 2026, 22:22 UTC · re: Venkat Ragavan via GitGitGadget · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

"Venkat Ragavan via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 13 quoted lines
> From: Venkat Ragavan S <mail2venkat21@gmail.com>
>
> When seeking through loose refs, cache_ref_iterator_seek() compares
> each directory component without its trailing slash. This can confuse
> a directory with a similarly named ref that sorts before it.
> ...
> Add regression tests for shared-prefix names, missing nested markers,
> and markers beyond the end of the ref listing, using loose, packed,
> and mixed refs.
>
> Signed-off-by: Venkat Ragavan S <mail2venkat21@gmail.com>
> ---
>     refs: fix loose-ref seeking across directory boundaries

I overlooked this topic, but re-reading the discussion, aside from a test that confused a reader, it seems the topic was already ready to go?

Any objections to the idea of making the topic ready for 'next'?
Thanks.
Patrick Steinhardt· Oct 28, 2026, 08:58 UTC · re: Junio C Hamano · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

On Tue, Oct 27, 2026 at 03:22:48PM -0700, Junio C Hamano wrote:
Show 21 quoted lines
> "Venkat Ragavan via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
> > From: Venkat Ragavan S <mail2venkat21@gmail.com>
> >
> > When seeking through loose refs, cache_ref_iterator_seek() compares
> > each directory component without its trailing slash. This can confuse
> > a directory with a similarly named ref that sorts before it.
> > ...
> > Add regression tests for shared-prefix names, missing nested markers,
> > and markers beyond the end of the ref listing, using loose, packed,
> > and mixed refs.
> >
> > Signed-off-by: Venkat Ragavan S <mail2venkat21@gmail.com>
> > ---
> >     refs: fix loose-ref seeking across directory boundaries
> 
> I overlooked this topic, but re-reading the discussion, aside from a
> test that confused a reader, it seems the topic was already ready to
> go?
> 
> Any objections to the idea of making the topic ready for 'next'?

Sorry, I lost track of this patch series, but I've reviewed it now. Overall this looks good, but I wonder whether this really should be split up into three patches because as far as I understand, this single patch solves three separate bugs at once.

Patrick
Junio C Hamano· Oct 28, 2026, 16:26 UTC · re: Patrick Steinhardt · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

Patrick Steinhardt <ps@pks.im> writes:
Show 29 quoted lines
> On Tue, Oct 27, 2026 at 03:22:48PM -0700, Junio C Hamano wrote:
>> "Venkat Ragavan via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> 
>> > From: Venkat Ragavan S <mail2venkat21@gmail.com>
>> >
>> > When seeking through loose refs, cache_ref_iterator_seek() compares
>> > each directory component without its trailing slash. This can confuse
>> > a directory with a similarly named ref that sorts before it.
>> > ...
>> > Add regression tests for shared-prefix names, missing nested markers,
>> > and markers beyond the end of the ref listing, using loose, packed,
>> > and mixed refs.
>> >
>> > Signed-off-by: Venkat Ragavan S <mail2venkat21@gmail.com>
>> > ---
>> >     refs: fix loose-ref seeking across directory boundaries
>> 
>> I overlooked this topic, but re-reading the discussion, aside from a
>> test that confused a reader, it seems the topic was already ready to
>> go?
>> 
>> Any objections to the idea of making the topic ready for 'next'?
>
> Sorry, I lost track of this patch series, but I've reviewed it now.
> Overall this looks good, but I wonder whether this really should be
> split up into three patches because as far as I understand, this single
> patch solves three separate bugs at once.
>
> Patrick
Thanks.  Let's hear from the author, then.
Patrick Steinhardt· Oct 28, 2026, 08:57 UTC · re: Venkat Ragavan via GitGitGadget · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

On Thu, Oct 15, 2026 at 12:30:55PM +0000, Venkat Ragavan via GitGitGadget wrote:
Show 5 quoted lines
> From: Venkat Ragavan S <mail2venkat21@gmail.com>
> 
> When seeking through loose refs, cache_ref_iterator_seek() compares
> each directory component without its trailing slash. This can confuse
> a directory with a similarly named ref that sorts before it.

This observes the status quo, but it doesn't give us any context _why_ we even did this. I'm sure there must've been a reason for this, but the reader will have to figure it out by themselves.

Show 9 quoted lines
> For example, with these refs:
> 
>     refs/tags/releases-z
>     refs/tags/releases/v1
>     refs/tags/releases/v2
> 
> Starting after refs/tags/releases/v1 can position the iterator at
> releases-z instead of descending into releases/. Iteration then
> returns v1 again, even though it should return only v2.

It would have been great to demonstrate how this is visible to the user. E.g., can this be shown with a simple call to git-for-each-ref(1)?

Show 8 quoted lines
> Include the trailing slash when comparing directory components so
> that the seek follows the correct directory.
> 
> Also handle two related positioning errors. If every entry in a
> directory precedes the cursor, stop searching that level rather than
> descending into its last entry. If a leaf sorts after the cursor,
> leave the iterator positioned to return it, even when the cursor has
> remaining path components.

Are thse separate bugs? If so, it would've made sense to split this series into three separate commit so that we can fix the individual bugs one by one, right?

Show 13 quoted lines
> diff --git a/refs/ref-cache.c b/refs/ref-cache.c
> index ffef01a597..a3bc2dc11b 100644
> --- a/refs/ref-cache.c
> +++ b/refs/ref-cache.c
> @@ -532,7 +534,7 @@ static int cache_ref_iterator_seek(struct ref_iterator *ref_iterator,
>  				level->prefix_state = PREFIX_CONTAINS_DIR;
>  			} else {
>  				/* reduce the index so the leaf node is iterated over */
> -				if (cmp <= 0 && !slash)
> +				if (cmp < 0 || (cmp == 0 && !slash))
>  					level->index = idx - 1;
>  				/*
>  				 * while the seek path may not be exhausted, our

Tiny nit: we usually don't explicitly compare for equality (or inequality) with 0, but rather write `cmp` or `!cmp`.

Show 21 quoted lines
> diff --git a/t/t6302-for-each-ref-filter.sh b/t/t6302-for-each-ref-filter.sh
> index 7f060d97bf..826fc1b9f2 100755
> --- a/t/t6302-for-each-ref-filter.sh
> +++ b/t/t6302-for-each-ref-filter.sh
> @@ -746,6 +746,61 @@ test_expect_success 'start after used with a pattern' '
>  	test_cmp expect actual
>  '
>  
> +for storage in loose packed mixed
> +do
> +	test_expect_success "start after respects directory boundaries ($storage)" '
> +		test_when_finished "rm -rf repo" &&
> +		git init repo &&
> +		(
> +			cd repo &&
> +			test_commit --no-tag base &&
> +			git update-ref refs/tags/packed HEAD &&
> +			if test "$storage" = mixed
> +			then
> +				git pack-refs --all || return 1
> +			fi &&

Okay. These are a bit funny in the context of the reftable backend, but I agree it makes sense to have this anyway. Doesn't hurt much to have extra coverage.

Thanks!
Patrick
D. Ben Knoble· Oct 15, 2026, 19:04 UTC · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

[resending with corrected plain-text ':D]
Show 16 quoted lines
> On Thu, 15 Oct 2026 at 10:38 PM, D. Ben Knoble <ben.knoble@gmail.com> wrote:
>>
>> On Thu, Oct 15, 2026 at 10:20 AM Harald Nordgren
>> <haraldnordgren@gmail.com> wrote:
>> [snip]
>> > > +                     if test "$storage" = mixed
>> > > +                     then
>> > > +                             git pack-refs --all || return 1
>> > > +                     fi &&
>> >
>> > I don't see any reason why a test would need an if block, we know what data we
>> > expect, no?
>>
>> I didn't examine too closely, but this test is wrapped in a loop over
>> the storage variable that appears to determine whether and when we
>> pack-refs, so I think the conditional makes sense for that?
On Thu, Oct 15, 2026 at 1:13 PM Venkat Ragavan <mail2venkat21@gmail.com> wrote:
>
> Yes it does, I interpreted Harald’s suggestion was to write separate tests instead of looping over a variable outside the test block, either ways the logic of the test is going to remain the same.
[we bottom-post here]

I don’t mind the loop, personally; it might be worth looking around for other tests and see how they approach it.

You can also extract shared sections into functions if that makes writing separate tests easier.

-- 
D. Ben Knoble
Venkat Ragavan· Oct 16, 2026, 03:19 UTC · lore

Re: [PATCH] refs: fix loose-ref seeking across directory boundaries

[Resending as plain text after the mailing list rejected the HTML version.]
> I realize now that I didn't read the code properly. I didn't expect to
> see a loop encapsulating a call to test_expect_success. That explains
> the if case as well.

Thanks for following up, and thanks Junio for explaining the setup. Since the loop intentionally exercises loose, packed, and mixed ref layouts, I'll leave the test unchanged.

On Fri, Oct 16, 2026 at 8:47 AM Venkat Ragavan <mail2venkat21@gmail.com> wrote:
Show 31 quoted lines
>
> > I realize now that I didn't read the code properly. I didn't expect to
> > see a loop encapsulating a call to test_expect_success. That explains
> > the if case as well.
>
> Thanks for following up, and thanks Junio for explaining the setup. Since
> the loop intentionally exercises loose, packed, and mixed ref layouts, I’ll
> leave the test unchanged.
>
> On Fri, Oct 16, 2026 at 2:47 AM Harald Nordgren <haraldnordgren@gmail.com> wrote:
>>
>> > > The general idea looks sound.
>> > >
>> > >> +                    if test "$storage" = mixed
>> > >> +                    then
>> > >> +                            git pack-refs --all || return 1
>> > >> +                    fi &&
>> > >
>> > > I don't see any reason why a test would need an if block, we know what data we
>> > > expect, no?
>> >
>> > This is three tests implemented as a single test with a for-loop
>> > around it.  The variable instructs the single test_expect_success
>> > to behave slightly differently depending on its setting.
>>
>> I realize now that I didn't read the code properly. I didn't expect to
>> see a loop encapsulating a call to test_expect_success. That explains
>> the if case as well.
>>
>>
>> Harald

← back to recent threads