threads / patch / 61379

patch, 8 partstightening ref handling outside of refs/

Subject: [PATCH 0/8] tightening ref handling outside of refs/

## tl;dr

7 messages between Apr 29, 2024 and Apr 30, 2024. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Jeff King· Apr 29, 2024, 08:15 UTC · lore
This is picking up the discussion from:
  https://lore.kernel.org/git/20240426211529.GD13703@coredump.intra.peff.net/

The basic issue is that we don't really enforce pseudoref syntax for names outside of "refs/", so you update random files in .git like:

  git update-ref objects/info/commit-graphs/commit-graph-chain HEAD
This is mitigated a bit by:
  1. You can't usually _overwrite_ files unless they look vaguely
     sha1-ish (in this case the chain file contains hashes of graph
     files, which is enough). So high-ticket items like "config" should
     be immune.
  2. Receive-pack is a bit more careful here, and refuses anything
     outside of "refs/". So you can't get up to any mischief via "git
     push".

But I still find it a bit scary/weird in general. And as noted in that thread, there's some attempt to enforce this that is done inconsistently. So you can update and read such refs, but are forbidden to delete them.

Of course all of this becomes a non-issue with reftables, where those names are not used in the filesystem at all. But even there I think we'd probably want to consistently enforce the syntax rules (both between delete/update/read, but also consistency with the files backend).

This series teaches check_refname_format() to enforce these rules (when instructed; see patch 6 for a discussion of all sorts of complications).

These changes are not backwards-compatible! But that is kind of the point. This is stuff that was never supposed to work. My concern would just be that somehow somebody is relying on it. Pay attention specifically to patches 4, 7, and 8, which are where the behavior changes are.

  [1/8]: t0600: don't create ref outside of refs/
  [2/8]: t5619: use fully qualified refname for branch
  [3/8]: refs: move is_pseudoref_syntax() definition earlier
  [4/8]: refs: disallow dash in pseudoref syntax
  [5/8]: refs: use is_pseudoref_syntax() in refname_is_safe()
  [6/8]: check_refname_format(): add FULLY_QUALIFIED flag
  [7/8]: refs: check refnames as fully qualified when writing
  [8/8]: refs: check refnames as fully qualified when resolving
 refs.c                                     | 59 ++++++++++++----------
 refs.h                                     |  1 +
 t/t0600-reffiles-backend.sh                |  2 +-
 t/t1430-bad-ref-name.sh                    | 20 ++++++++
 t/t5619-clone-local-ambiguous-transport.sh |  4 +-
 5 files changed, 57 insertions(+), 29 deletions(-)
-Peff
Jeff King· Apr 29, 2024, 08:16 UTC · re: Jeff King · lore

[PATCH 1/8] t0600: don't create ref outside of refs/

We have a test that tries to manipulate the branch refs/heads/referrent, mostly using either the fully qualified refname or git-branch (which implies refs/heads/). However, the call to update-ref uses the unqualified name, meaning we were quietly creating ".git/referrent", which was otherwise unused by the test.

Fix this to specify refs/heads/referrent.

I _think_ it actually doesn't affect the test outcome either way. The point of the test is that expiring reflogs for "the_symref" should not fail when we cannot lock "refs/heads/referrent" that it points to (because we have created a fake .lock file for it). And that is true even if the "referrent" file does not even exist. After all, the process holding the lock could be in the act of creating it.

So I suspect this "update-ref" line could just be dropped entirely. Which you can verify by going back to its origin in 41d796ed5c (refs: on symref reflog expire, lock symref not referrent, 2016-04-07) and removing it. The test fails without the matching code change and passes with it.

But I think it's worth keeping the update-ref call, as it creates a situation which is more likely to match what we'd see in the real world. Even if it does not matter now, it's possible it could in the future.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t0600-reffiles-backend.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to t/t0600-reffiles-backend.sh +1 −1
diff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh
index 64214340e7..41db9e30d4 100755
--- a/t/t0600-reffiles-backend.sh
+++ b/t/t0600-reffiles-backend.sh
@@ -343,7 +343,7 @@ test_expect_success SHA1 'parsing reverse reflogs at BUFSIZ boundaries' '
 test_expect_success 'reflog expire operates on symref not referrent' '
 	git branch --create-reflog the_symref &&
 	git branch --create-reflog referrent &&
-	git update-ref referrent HEAD &&
+	git update-ref refs/heads/referrent HEAD &&
 	git symbolic-ref refs/heads/the_symref refs/heads/referrent &&
 	test_when_finished "rm -f .git/refs/heads/referrent.lock" &&
 	touch .git/refs/heads/referrent.lock &&
-- 
2.45.0.rc1.416.gbe2a76c799
Jeff King· Apr 29, 2024, 08:36 UTC · re: Jeff King · lore

Re: [PATCH 0/8] tightening ref handling outside of refs/

On Mon, Apr 29, 2024 at 04:15:13AM -0400, Jeff King wrote:
Show 8 quoted lines
>   [1/8]: t0600: don't create ref outside of refs/
>   [2/8]: t5619: use fully qualified refname for branch
>   [3/8]: refs: move is_pseudoref_syntax() definition earlier
>   [4/8]: refs: disallow dash in pseudoref syntax
>   [5/8]: refs: use is_pseudoref_syntax() in refname_is_safe()
>   [6/8]: check_refname_format(): add FULLY_QUALIFIED flag
>   [7/8]: refs: check refnames as fully qualified when writing
>   [8/8]: refs: check refnames as fully qualified when resolving

Ugh, sorry, I managed to break the threading due to some too-clever use of mutt. ;)

The other messages are on the list, and hopefully shouldn't be too hard to find by date.

-Peff
Jeff King· Apr 29, 2024, 08:42 UTC · re: Jeff King · lore

Re: [PATCH 0/8] tightening ref handling outside of refs/

On Mon, Apr 29, 2024 at 04:15:13AM -0400, Jeff King wrote:
>   [1/8]: t0600: don't create ref outside of refs/
>   [2/8]: t5619: use fully qualified refname for branch

You can probably guess that I found these test cleanups only after writing the rest of the series and seeing them fail. :)

It turns out there is one more spot that is run only with reftables (so CI caught it, but my local testing did not):

Show changes to t/t0610-reftable-basics.sh +2 −3
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index 178791e086..c6dbd2b5c4 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -343,11 +343,11 @@ test_expect_success 'ref transaction: env var disables compaction' '
 	for i in $(test_seq $iterations)
 	do
 		GIT_TEST_REFTABLE_AUTOCOMPACTION=false \
-		git -C repo update-ref branch-$i HEAD || return 1
+		git -C repo update-ref refs/heads/branch-$i HEAD || return 1
 	done &&
 	test_line_count = $expected repo/.git/reftable/tables.list &&
 
-	git -C repo update-ref foo HEAD &&
+	git -C repo update-ref refs/heads/foo HEAD &&
 	test_line_count -lt $expected repo/.git/reftable/tables.list
 '
 

I'll wait for comments before re-rolling, but I'll make sure that gets
added in.

-Peff
Patrick Steinhardt· Apr 29, 2024, 09:28 UTC · re: Jeff King · lore

Re: [PATCH 0/8] tightening ref handling outside of refs/

On Mon, Apr 29, 2024 at 04:42:38AM -0400, Jeff King wrote:
Show 10 quoted lines
> On Mon, Apr 29, 2024 at 04:15:13AM -0400, Jeff King wrote:
> 
> >   [1/8]: t0600: don't create ref outside of refs/
> >   [2/8]: t5619: use fully qualified refname for branch
> 
> You can probably guess that I found these test cleanups only after
> writing the rest of the series and seeing them fail. :)
> 
> It turns out there is one more spot that is run only with reftables (so
> CI caught it, but my local testing did not):

Yeah, that's an issue by itself in my opinion. It's ultimately the reason why I change this to always run the backend-specific tests in [1].

Show 20 quoted lines
> diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
> index 178791e086..c6dbd2b5c4 100755
> --- a/t/t0610-reftable-basics.sh
> +++ b/t/t0610-reftable-basics.sh
> @@ -343,11 +343,11 @@ test_expect_success 'ref transaction: env var disables compaction' '
>  	for i in $(test_seq $iterations)
>  	do
>  		GIT_TEST_REFTABLE_AUTOCOMPACTION=false \
> -		git -C repo update-ref branch-$i HEAD || return 1
> +		git -C repo update-ref refs/heads/branch-$i HEAD || return 1
>  	done &&
>  	test_line_count = $expected repo/.git/reftable/tables.list &&
>  
> -	git -C repo update-ref foo HEAD &&
> +	git -C repo update-ref refs/heads/foo HEAD &&
>  	test_line_count -lt $expected repo/.git/reftable/tables.list
>  '
>  
> I'll wait for comments before re-rolling, but I'll make sure that gets
> added in.
The fix looks reasonable.
Patrick
[1]: https://lore.kernel.org/git/acf0c285506fe7ba275b08cdaf6b2245ec66b565.1712896869.git.ps@pks.im/
Jeff King· Apr 30, 2024, 10:45 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 0/8] tightening ref handling outside of refs/

On Mon, Apr 29, 2024 at 11:28:38AM +0200, Patrick Steinhardt wrote:
Show 15 quoted lines
> On Mon, Apr 29, 2024 at 04:42:38AM -0400, Jeff King wrote:
> > On Mon, Apr 29, 2024 at 04:15:13AM -0400, Jeff King wrote:
> > 
> > >   [1/8]: t0600: don't create ref outside of refs/
> > >   [2/8]: t5619: use fully qualified refname for branch
> > 
> > You can probably guess that I found these test cleanups only after
> > writing the rest of the series and seeing them fail. :)
> > 
> > It turns out there is one more spot that is run only with reftables (so
> > CI caught it, but my local testing did not):
> 
> Yeah, that's an issue by itself in my opinion. It's ultimately the
> reason why I change this to always run the backend-specific tests in
> [1].

Ah, I hadn't seen that series. Yes, I'd be very much in favor of that change.

-Peff
Junio C Hamano· Apr 29, 2024, 15:01 UTC · re: Jeff King · lore

Re: [PATCH 0/8] tightening ref handling outside of refs/

Jeff King <peff@peff.net> writes:
Show 6 quoted lines
> This series teaches check_refname_format() to enforce these rules (when
> instructed; see patch 6 for a discussion of all sorts of complications).
>
> These changes are not backwards-compatible! But that is kind of the
> point. This is stuff that was never supposed to work. My concern would
> just be that somehow somebody is relying on it.

I would of course be worried about the same, but these all look like reasonable "I wish we did them in this way from the beginning" changes.

← back to recent threads