git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 06/12] setup: introduce "extensions.refStorage" extension

From
Patrick Steinhardt <ps@pks.im>
Date
Dec 20, 2023, 10:55 UTC
Message-ID
<beeb182f2844bd8a9e1f2a2767838282f705e0a2.1703067989.git.ps@pks.im>
In-Reply-To
<cover.1703067989.git.ps@pks.im>

Introduce a new "extensions.refStorage" extension that allows us to specify the ref storage format used by a repository. For now, the only supported format is the "files" format, but this list will likely soon be extended to also support the upcoming "reftable" format.

There have been discussions on the Git mailing list in the past around how exactly this extension should look like. One alternative [1] that was discussed was whether it would make sense to model the extension in such a way that backends are arbitrarily stackable. This would allow for a combined value of e.g. "loose,packed-refs" or "loose,reftable", which indicates that new refs would be written via "loose" files backend and compressed into "packed-refs" or "reftable" backends, respectively.

It is arguable though whether this flexibility and the complexity that it brings with it is really required for now. It is not foreseeable that there will be a proliferation of backends in the near-term future, and the current set of existing formats and formats which are on the horizon can easily be configured with the much simpler proposal where we have a single value, only.

Furthermore, if we ever see that we indeed want to gain the ability to arbitrarily stack the ref formats, then we can adapt the current extension rather easily. Given that Git clients will refuse any unknown value for the "extensions.refStorage" extension they would also know to ignore a stacked "loose,packed-refs" in the future.

So let's stick with the easy proposal for the time being and wire up the extension.

[1]: <pull.1408.git.1667846164.gitgitgadget@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/config/extensions.txt           | 11 ++++++++
 Documentation/ref-storage-format.txt          |  1 +
 .../technical/repository-version.txt          |  5 ++++
 builtin/clone.c                               |  2 +-
 setup.c                                       | 23 +++++++++++++---
 setup.h                                       |  3 ++-
 t/t0001-init.sh                               | 26 +++++++++++++++++++
 t/test-lib.sh                                 |  2 +-
 8 files changed, 67 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/ref-storage-format.txt
diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt
index bccaec7a96..66db0e15da 100644
--- a/Documentation/config/extensions.txt
+++ b/Documentation/config/extensions.txt
@@ -7,6 +7,17 @@ Note that this setting should only be set by linkgit:git-init[1] or
 linkgit:git-clone[1].  Trying to change it after initialization will not
 work and will produce hard-to-diagnose issues.
 
+extensions.refStorage::
+	Specify the ref storage format to use. The acceptable values are:
++
+include::../ref-storage-format.txt[]
++
+It is an error to specify this key unless `core.repositoryFormatVersion` is 1.
++
+Note that this setting should only be set by linkgit:git-init[1] or
+linkgit:git-clone[1]. Trying to change it after initialization will not
+work and will produce hard-to-diagnose issues.
+
 extensions.worktreeConfig::
 	If enabled, then worktrees will load config settings from the
 	`$GIT_DIR/config.worktree` file in addition to the
diff --git a/Documentation/ref-storage-format.txt b/Documentation/ref-storage-format.txt
new file mode 100644
index 0000000000..1a65cac468
--- /dev/null
+++ b/Documentation/ref-storage-format.txt
@@ -0,0 +1 @@
+* `files` for loose files with packed-refs. This is the default.
diff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt
index 045a76756f..27be3741e6 100644
--- a/Documentation/technical/repository-version.txt
+++ b/Documentation/technical/repository-version.txt
@@ -100,3 +100,8 @@ If set, by default "git config" reads from both "config" and
 multiple working directory mode, "config" file is shared while
 "config.worktree" is per-working directory (i.e., it's in
 GIT_COMMON_DIR/worktrees/<id>/config.worktree)
+
+==== `refStorage`
+
+Specifies the file format for the ref database. The only valid value
+is `files` (loose references with a packed-refs file).
diff --git a/builtin/clone.c b/builtin/clone.c
index 6840064b59..20c161e94a 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -1289,7 +1289,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	 * ours to the same thing.
 	 */
 	hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport));
-	initialize_repository_version(hash_algo, 1);
+	initialize_repository_version(hash_algo, the_repository->ref_storage_format, 1);
 	repo_set_hash_algo(the_repository, hash_algo);
 	create_reference_database(the_repository->ref_storage_format, NULL, 1);
 
diff --git a/setup.c b/setup.c
index 5dffea6e2f..7eb4ae8ea0 100644
--- a/setup.c
+++ b/setup.c
@@ -590,6 +590,17 @@ static enum extension_result handle_extension(const char *var,
 				     "extensions.objectformat", value);
 		data->hash_algo = format;
 		return EXTENSION_OK;
+	} else if (!strcmp(ext, "refstorage")) {
+		int format;
+
+		if (!value)
+			return config_error_nonbool(var);
+		format = ref_storage_format_by_name(value);
+		if (format == REF_STORAGE_FORMAT_UNKNOWN)
+			return error(_("invalid value for '%s': '%s'"),
+				     "extensions.refstorage", value);
+		data->ref_storage_format = format;
+		return EXTENSION_OK;
 	}
 	return EXTENSION_UNKNOWN;
 }
@@ -1869,12 +1880,14 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree)
 	return 1;
 }
 
-void initialize_repository_version(int hash_algo, int reinit)
+void initialize_repository_version(int hash_algo, int ref_storage_format,
+				   int reinit)
 {
 	char repo_version_string[10];
 	int repo_version = GIT_REPO_VERSION;
 
-	if (hash_algo != GIT_HASH_SHA1)
+	if (hash_algo != GIT_HASH_SHA1 ||
+	    ref_storage_format != REF_STORAGE_FORMAT_FILES)
 		repo_version = GIT_REPO_VERSION_READ;
 
 	/* This forces creation of new config file */
@@ -1887,6 +1900,10 @@ void initialize_repository_version(int hash_algo, int reinit)
 			       hash_algos[hash_algo].name);
 	else if (reinit)
 		git_config_set_gently("extensions.objectformat", NULL);
+
+	if (ref_storage_format != REF_STORAGE_FORMAT_FILES)
+		git_config_set("extensions.refstorage",
+			       ref_storage_format_to_name(ref_storage_format));
 }
 
 static int is_reinit(void)
@@ -2028,7 +2045,7 @@ static int create_default_files(const char *template_path,
 		adjust_shared_perm(get_git_dir());
 	}
 
-	initialize_repository_version(fmt->hash_algo, 0);
+	initialize_repository_version(fmt->hash_algo, fmt->ref_storage_format, 0);
 
 	/* Check filemode trustability */
 	path = git_path_buf(&buf, "config");
diff --git a/setup.h b/setup.h
index 4927abf2cf..89033ae1d9 100644
--- a/setup.h
+++ b/setup.h
@@ -180,7 +180,8 @@ int init_db(const char *git_dir, const char *real_git_dir,
 	    int ref_storage_format,
 	    const char *initial_branch, int init_shared_repository,
 	    unsigned int flags);
-void initialize_repository_version(int hash_algo, int reinit);
+void initialize_repository_version(int hash_algo, int ref_storage_format,
+				   int reinit);
 void create_reference_database(int ref_storage_format,
 			       const char *initial_branch, int quiet);
 
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 2b78e3be47..38b3e4c39e 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -532,6 +532,32 @@ test_expect_success 'init rejects attempts to initialize with different hash' '
 	test_must_fail git -C sha256 init --object-format=sha1
 '
 
+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage is not allowed with repo version 0' '
+	test_when_finished "rm -rf refstorage" &&
+	git init refstorage &&
+	git -C refstorage config extensions.refStorage files &&
+	test_must_fail git -C refstorage rev-parse 2>err &&
+	grep "repo version is 0, but v1-only extension found" err
+'
+
+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with files backend' '
+	test_when_finished "rm -rf refstorage" &&
+	git init refstorage &&
+	git -C refstorage config core.repositoryformatversion 1 &&
+	git -C refstorage config extensions.refStorage files &&
+	test_commit -C refstorage A &&
+	git -C refstorage rev-parse --verify HEAD
+'
+
+test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown backend' '
+	test_when_finished "rm -rf refstorage" &&
+	git init refstorage &&
+	git -C refstorage config core.repositoryformatversion 1 &&
+	git -C refstorage config extensions.refStorage garbage &&
+	test_must_fail git -C refstorage rev-parse 2>err &&
+	grep "invalid value for ${SQ}extensions.refstorage${SQ}: ${SQ}garbage${SQ}" err
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index dc03f06b8e..4685cc3d48 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1937,7 +1937,7 @@ test_lazy_prereq SHA1 '
 '
 
 test_lazy_prereq DEFAULT_REPO_FORMAT '
-	test_have_prereq SHA1
+	test_have_prereq SHA1,REFFILES
 '
 
 # Ensure that no test accidentally triggers a Git command

-- 
2.43.GIT
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 18 of 60 in “Introduce `refStorage` extension”
  1. 00/12 Introduce `refStorage` extensionPatrick Steinhardt, Dec 20, 2023
  2. 01/12 t: introduce DEFAULT_REPO_FORMAT prereqPatrick Steinhardt, Dec 20, 2023
  3. Karthik NayakDec 22, 2023
  4. Patrick SteinhardtDec 28, 2023
  5. 02/12 worktree: skip reading HEAD when repairing worktreesPatrick Steinhardt, Dec 20, 2023
  6. Karthik NayakDec 22, 2023
  7. 03/12 refs: refactor logic to look up storage backendsPatrick Steinhardt, Dec 20, 2023
  8. Karthik NayakDec 22, 2023
  9. Patrick SteinhardtDec 28, 2023
  10. 04/12 setup: start tracking ref storage format whenPatrick Steinhardt, Dec 20, 2023
  11. Junio C HamanoDec 20, 2023
  12. Patrick SteinhardtDec 28, 2023
  13. Junio C HamanoDec 28, 2023
  14. Patrick SteinhardtDec 28, 2023
  15. Karthik NayakDec 22, 2023
  16. Patrick SteinhardtDec 28, 2023
  17. 05/12 setup: set repository's formats on initPatrick Steinhardt, Dec 20, 2023
  18. 06/12 setup: introduce "extensions.refStorage" extensionPatrick Steinhardt, Dec 20, 2023
  19. 07/12 setup: introduce GIT_DEFAULT_REF_FORMAT envvarPatrick Steinhardt, Dec 20, 2023
  20. 08/12 t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvarPatrick Steinhardt, Dec 20, 2023
  21. 09/12 builtin/rev-parse: introduce `--show-ref-format` flagPatrick Steinhardt, Dec 20, 2023
  22. 10/12 builtin/init: introduce `--ref-format=` value flagPatrick Steinhardt, Dec 20, 2023
  23. 11/12 builtin/clone: introduce `--ref-format=` value flagPatrick Steinhardt, Dec 20, 2023
  24. 12/12 t9500: write "extensions.refstorage" into configPatrick Steinhardt, Dec 20, 2023
  25. Karthik NayakDec 22, 2023
  26. Karthik NayakDec 22, 2023
  27. Junio C HamanoDec 26, 2023
  28. 00/12 Introduce `refStorage` extensionPatrick Steinhardt, Dec 28, 2023
  29. 01/12 t: introduce DEFAULT_REPO_FORMAT prereqPatrick Steinhardt, Dec 28, 2023
  30. 02/12 worktree: skip reading HEAD when repairing worktreesPatrick Steinhardt, Dec 28, 2023
  31. Eric SunshineDec 28, 2023
  32. Eric SunshineDec 28, 2023
  33. Patrick SteinhardtDec 28, 2023
  34. 03/12 refs: refactor logic to look up storage backendsPatrick Steinhardt, Dec 28, 2023
  35. Junio C HamanoDec 28, 2023
  36. Patrick SteinhardtDec 28, 2023
  37. Junio C HamanoDec 28, 2023
  38. 04/12 setup: start tracking ref storage formatPatrick Steinhardt, Dec 28, 2023
  39. 05/12 setup: set repository's formats on initPatrick Steinhardt, Dec 28, 2023
  40. 06/12 setup: introduce "extensions.refStorage" extensionPatrick Steinhardt, Dec 28, 2023
  41. 07/12 setup: introduce GIT_DEFAULT_REF_FORMAT envvarPatrick Steinhardt, Dec 28, 2023
  42. 08/12 t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvarPatrick Steinhardt, Dec 28, 2023
  43. 09/12 builtin/rev-parse: introduce `--show-ref-format` flagPatrick Steinhardt, Dec 28, 2023
  44. 10/12 builtin/init: introduce `--ref-format=` value flagPatrick Steinhardt, Dec 28, 2023
  45. 11/12 builtin/clone: introduce `--ref-format=` value flagPatrick Steinhardt, Dec 28, 2023
  46. 12/12 t9500: write "extensions.refstorage" into configPatrick Steinhardt, Dec 28, 2023
  47. 00/12 Introduce `refStorage` extensionPatrick Steinhardt, Dec 29, 2023
  48. 01/12 t: introduce DEFAULT_REPO_FORMAT prereqPatrick Steinhardt, Dec 29, 2023
  49. 02/12 worktree: skip reading HEAD when repairing worktreesPatrick Steinhardt, Dec 29, 2023
  50. 03/12 refs: refactor logic to look up storage backendsPatrick Steinhardt, Dec 29, 2023
  51. 04/12 setup: start tracking ref storage formatPatrick Steinhardt, Dec 29, 2023
  52. 05/12 setup: set repository's formats on initPatrick Steinhardt, Dec 29, 2023
  53. 06/12 setup: introduce "extensions.refStorage" extensionPatrick Steinhardt, Dec 29, 2023
  54. 07/12 setup: introduce GIT_DEFAULT_REF_FORMAT envvarPatrick Steinhardt, Dec 29, 2023
  55. 08/12 t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvarPatrick Steinhardt, Dec 29, 2023
  56. 09/12 builtin/rev-parse: introduce `--show-ref-format` flagPatrick Steinhardt, Dec 29, 2023
  57. 10/12 builtin/init: introduce `--ref-format=` value flagPatrick Steinhardt, Dec 29, 2023
  58. 11/12 builtin/clone: introduce `--ref-format=` value flagPatrick Steinhardt, Dec 29, 2023
  59. 12/12 t9500: write "extensions.refstorage" into configPatrick Steinhardt, Dec 29, 2023
  60. Eric SunshineDec 29, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.