git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v5 3/4] refs: allow reference location in refstorage config

From
Patrick Steinhardt <ps@pks.im>
Date
Feb 9, 2026, 16:34 UTC
Message-ID
<aYoMk5HmpSyN1azt@pks.im>
In-Reply-To
<20260209-kn-alternate-ref-dir-v5-3-740899834ceb@gmail.com>
On Mon, Feb 09, 2026 at 04:58:20PM +0100, Karthik Nayak wrote:
Show 12 quoted lines
> The 'extensions.refStorage' config is used to specify the reference
> backend for a given repository. Both the 'files' and 'reftable' backends
> utilize the $GIT_DIR as the reference folder by default in
> `get_main_ref_store()`.
> 
> Since the reference backends are pluggable, this means that they could
> work with out-of-tree reference directories too. Extend the 'refStorage'
> config to also support taking an URI input, where users can specify the
> reference backend and the location.
> 
> Add the required changes to obtain and propagate this value to the
> individual backends also add the necessary documentation and tests.
This reads as if this should have been two sentences.
Show 10 quoted lines
> diff --git a/builtin/worktree.c b/builtin/worktree.c
> index fbdaf2eb2e..94480be5c4 100644
> --- a/builtin/worktree.c
> +++ b/builtin/worktree.c
> @@ -425,6 +425,39 @@ static int make_worktree_orphan(const char * ref, const struct add_opts *opts,
>  	return run_command(&cp);
>  }
>  
> +/*
> + * References for worktress are generally stored in '$GIT_DIR/worktrees/<wt_id>'.
s/worktress/worktrees/
Show 92 quoted lines
> diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
> new file mode 100755
> index 0000000000..9c777b79f3
> --- /dev/null
> +++ b/t/t1423-ref-backend.sh
> @@ -0,0 +1,159 @@
> +#!/bin/sh
> +
> +test_description='Test reference backend URIs'
> +
> +. ./test-lib.sh
> +
> +# Run a git command with the provided reference storage. Reset the backend
> +# post running the command.
> +# Usage: run_with_uri <repo> <backend> <uri> <cmd>
> +#   <repo> is the relative path to the repo to run the command in.
> +#   <backend> is the original ref storage of the repo.
> +#   <uri> is the new URI to be set for the ref storage.
> +#   <cmd> is the git subcommand to be run in the repository.
> +run_with_uri() {
> +	repo=$1 &&
> +	backend=$2 &&
> +	uri=$3 &&
> +	cmd=$4 &&
> +
> +	git -C "$repo" config set core.repositoryformatversion 1
> +	git -C "$repo" config set extensions.refStorage "$uri" &&
> +	git -C "$repo" $cmd &&
> +	git -C "$repo" config set extensions.refStorage "$backend"
> +}
> +
> +# Test a repository with a given reference storage by running and comparing
> +# 'git refs list' before and after setting the new reference backend. If
> +# err_msg is set, expect the command to fail and grep for the provided err_msg.
> +# Usage: run_with_uri <repo> <backend> <uri> <cmd>
> +#   <repo> is the relative path to the repo to run the command in.
> +#   <backend> is the original ref storage of the repo.
> +#   <uri> is the new URI to be set for the ref storage.
> +#   <err_msg> (optional) if set, check if 'git-refs(1)' failed with the provided msg.
> +test_refs_backend() {
> +	repo=$1 &&
> +	backend=$2 &&
> +	uri=$3 &&
> +	err_msg=$4 &&
> +
> +	git -C "$repo" config set core.repositoryformatversion 1 &&
> +	if test -n "$err_msg";
> +	then
> +		git -C "$repo" config set extensions.refStorage "$uri" &&
> +		test_must_fail git -C "$repo" refs list 2>err &&
> +		test_grep "$err_msg" err
> +	else
> +		git -C "$repo" refs list >expect &&
> +		run_with_uri "$repo" "$backend" "$uri" "refs list" >actual &&
> +		test_cmp expect actual
> +	fi
> +}
> +
> +test_expect_success 'URI is invalid' '
> +	test_when_finished "rm -rf repo" &&
> +	git init repo &&
> +	test_refs_backend repo files "reftable@/home/reftable" \
> +		"invalid value for ${SQ}extensions.refstorage${SQ}"
> +'
> +
> +test_expect_success 'URI ends with colon' '
> +	test_when_finished "rm -rf repo" &&
> +	git init repo &&
> +	test_refs_backend repo files "reftable:" \
> +		"invalid value for ${SQ}extensions.refstorage${SQ}"
> +'
> +
> +test_expect_success 'unknown reference backend' '
> +	test_when_finished "rm -rf repo" &&
> +	git init repo &&
> +	test_refs_backend repo files "db://.git" \
> +		"invalid value for ${SQ}extensions.refstorage${SQ}"
> +'
> +
> +ref_formats="files reftable"
> +for from_format in $ref_formats
> +do
> +
> +for to_format in $ref_formats
> +do
> +	if test "$from_format" = "$to_format"
> +	then
> +		continue
> +	fi
> +
> +
> +	for dir in "$(pwd)/repo/.git" "./"

As "./" is a relative directory I expect it to be resolved relative to "$GIT_DIR", right? Also, I don't see any tests that create the ref directory outside of the repository. Should we maybe add one?

Show 12 quoted lines
> +	do
> +
> +		test_expect_success "$read from $to_format backend, $dir dir" '
> +			test_when_finished "rm -rf repo" &&
> +			git init --ref-format=$from_format repo &&
> +			(
> +				cd repo &&
> +				test_commit 1 &&
> +				test_commit 2 &&
> +				test_commit 3 &&
> +
> +				git refs migrate --dry-run --ref-format=$to_format >out &&

Okay, we do the migration, but with "--dry-run". This should result in a temporary staging directory, which is in fact somewhat interesting given that "to_format" can now contain a payload. I assume it wouldn't have an impact if such a payload was set here?

> +				BACKEND_PATH="$dir/$(sed "s/.* ${SQ}.git\/\(.*\)${SQ}/\1/" out)" &&
Hm. I have no idea what this is doing :)
> +				test_refs_backend . $from_format "$to_format://$BACKEND_PATH" "$method"
> +			)
> +		'

In general I think it would be sensible to also have a couple tests here that exercise specific formats directly. Like:

  - Are the files created in the right spots for the files and reftable
    backend with a specific backend?
  - Does `git refs migrate` know to write the files into the correct
    location in case "--dry-run" wasn't passed?
  - Does git-init(1) and git-clone(1) initialize the refstore in a
    different location as expected?
  - Does creating a worktree work?
Patrick
Previous: Karthik NayakNext: Karthik Nayak
Message 48 of 101 in “refs: allow setting the reference directory”
  1. 0/2 refs: allow setting the reference directoryKarthik Nayak, Nov 19, 2025
  2. 1/2 refs: support obtaining ref_store for given dirKarthik Nayak, Nov 19, 2025
  3. Justin ToblerNov 20, 2025
  4. Karthik NayakNov 21, 2025
  5. 2/2 refs: add GIT_REF_URI to specify reference backend and directoryKarthik Nayak, Nov 19, 2025
  6. Eric SunshineNov 19, 2025
  7. Karthik NayakNov 19, 2025
  8. Jean-Noël AvilaNov 20, 2025
  9. Karthik NayakNov 21, 2025
  10. Justin ToblerNov 20, 2025
  11. Karthik NayakNov 24, 2025
  12. Toon ClaesNov 21, 2025
  13. Junio C HamanoNov 21, 2025
  14. Karthik NayakNov 24, 2025
  15. Toon ClaesNov 26, 2025
  16. Karthik NayakNov 24, 2025
  17. Patrick SteinhardtDec 1, 2025
  18. Karthik NayakDec 2, 2025
  19. Junio C HamanoNov 23, 2025
  20. Patrick SteinhardtDec 1, 2025
  21. Junio C HamanoDec 2, 2025
  22. Karthik NayakDec 2, 2025
  23. 0/4 refs: allow setting the reference directoryKarthik Nayak, Feb 2, 2026
  24. 1/4 refs: allow reference location in refstorage configKarthik Nayak, Feb 2, 2026
  25. Patrick SteinhardtFeb 6, 2026
  26. Karthik NayakFeb 9, 2026
  27. 2/4 refs: extract out `refs_create_refdir_stubs()`Karthik Nayak, Feb 2, 2026
  28. Patrick SteinhardtFeb 6, 2026
  29. Karthik NayakFeb 9, 2026
  30. 3/4 refs: parse and use the reference storage payloadKarthik Nayak, Feb 2, 2026
  31. Patrick SteinhardtFeb 6, 2026
  32. Karthik NayakFeb 9, 2026
  33. 4/4 refs: add GIT_REFERENCE_BACKEND to specify reference backendKarthik Nayak, Feb 2, 2026
  34. Patrick SteinhardtFeb 6, 2026
  35. Karthik NayakFeb 9, 2026
  36. Patrick SteinhardtFeb 6, 2026
  37. Junio C HamanoFeb 6, 2026
  38. Karthik NayakFeb 9, 2026
  39. 0/4 refs: allow setting the reference directoryKarthik Nayak, Feb 9, 2026
  40. 1/4 refs: extract out `refs_create_refdir_stubs()`Karthik Nayak, Feb 9, 2026
  41. 2/4 refs: forward and use the reference storage payloadKarthik Nayak, Feb 9, 2026
  42. Patrick SteinhardtFeb 9, 2026
  43. Karthik NayakFeb 10, 2026
  44. Jeff KingFeb 10, 2026
  45. Karthik NayakFeb 13, 2026
  46. Jeff KingFeb 15, 2026
  47. 3/4 refs: allow reference location in refstorage configKarthik Nayak, Feb 9, 2026
  48. Patrick SteinhardtFeb 9, 2026
  49. Karthik NayakFeb 10, 2026
  50. Jeff KingFeb 10, 2026
  51. Karthik NayakFeb 11, 2026
  52. 4/4 refs: add GIT_REFERENCE_BACKEND to specify reference backendKarthik Nayak, Feb 9, 2026
  53. Patrick SteinhardtFeb 9, 2026
  54. Junio C HamanoFeb 9, 2026
  55. Karthik NayakFeb 10, 2026
  56. Junio C HamanoFeb 10, 2026
  57. 0/6 refs: allow setting the reference directoryKarthik Nayak, Feb 14, 2026
  58. 1/6 setup: don't modify repo in `create_reference_database()`Karthik Nayak, Feb 14, 2026
  59. Patrick SteinhardtFeb 17, 2026
  60. Karthik NayakFeb 17, 2026
  61. 2/6 refs: extract out `refs_create_refdir_stubs()`Karthik Nayak, Feb 14, 2026
  62. 3/6 refs: receive and use the reference storage payloadKarthik Nayak, Feb 14, 2026
  63. Patrick SteinhardtFeb 17, 2026
  64. Karthik NayakFeb 17, 2026
  65. 4/6 refs: move out stub modification to generic layerKarthik Nayak, Feb 14, 2026
  66. Patrick SteinhardtFeb 17, 2026
  67. Karthik NayakFeb 17, 2026
  68. Toon ClaesFeb 18, 2026
  69. Karthik NayakFeb 19, 2026
  70. 5/6 refs: allow reference location in refstorage configKarthik Nayak, Feb 14, 2026
  71. 6/6 refs: add GIT_REFERENCE_BACKEND to specify reference backendKarthik Nayak, Feb 14, 2026
  72. Patrick SteinhardtFeb 17, 2026
  73. Karthik NayakFeb 17, 2026
  74. Patrick SteinhardtFeb 17, 2026
  75. Toon ClaesFeb 18, 2026
  76. Karthik NayakFeb 19, 2026
  77. 0/6 refs: allow setting the reference directoryKarthik Nayak, Feb 23, 2026
  78. 1/6 setup: don't modify repo in `create_reference_database()`Karthik Nayak, Feb 23, 2026
  79. 2/6 refs: extract out `refs_create_refdir_stubs()`Karthik Nayak, Feb 23, 2026
  80. 3/6 refs: move out stub modification to generic layerKarthik Nayak, Feb 23, 2026
  81. 4/6 refs: receive and use the reference storage payloadKarthik Nayak, Feb 23, 2026
  82. 5/6 refs: allow reference location in refstorage configKarthik Nayak, Feb 23, 2026
  83. Kristoffer HaugsbakkFeb 23, 2026
  84. Karthik NayakFeb 24, 2026
  85. Kristoffer HaugsbakkFeb 24, 2026
  86. Karthik NayakFeb 24, 2026
  87. 6/6 refs: add GIT_REFERENCE_BACKEND to specify reference backendKarthik Nayak, Feb 23, 2026
  88. Toon ClaesFeb 25, 2026
  89. Karthik NayakFeb 25, 2026
  90. Patrick SteinhardtFeb 23, 2026
  91. Karthik NayakFeb 23, 2026
  92. Junio C HamanoFeb 23, 2026
  93. Karthik NayakFeb 25, 2026
  94. 0/6 refs: allow setting the reference directoryKarthik Nayak, Feb 25, 2026
  95. 1/6 setup: don't modify repo in `create_reference_database()`Karthik Nayak, Feb 25, 2026
  96. 2/6 refs: extract out `refs_create_refdir_stubs()`Karthik Nayak, Feb 25, 2026
  97. 3/6 refs: move out stub modification to generic layerKarthik Nayak, Feb 25, 2026
  98. 4/6 refs: receive and use the reference storage payloadKarthik Nayak, Feb 25, 2026
  99. 5/6 refs: allow reference location in refstorage configKarthik Nayak, Feb 25, 2026
  100. Junio C HamanoFeb 25, 2026
  101. 6/6 refs: add GIT_REFERENCE_BACKEND to specify reference backendKarthik Nayak, Feb 25, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.