git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 09/13] setup: refactor how we configure the ref storage format

From
Patrick Steinhardt <ps@pks.im>
Date
Sep 9, 2026, 11:12 UTC
Message-ID
<20260909-b4-pks-unify-ref-storage-format-v3-9-ca041fb40ad8@pks.im>
In-Reply-To
<20260909-b4-pks-unify-ref-storage-format-v3-0-ca041fb40ad8@pks.im>

When (re)initializing a repository we need to figure out the ref storage format that the repository ought to use. This logic is surprisingly complex, as we have grown a lot of different mechanisms over time to configure the format. Unfortunately, as a result of this organic growth, the logic that configures the storage format has grown very complex.

The biggest culprit here is that we're mixing the logic that determines the desired storage format with the logic that validates whether the end result is sane. This leads to some repetitive code, and makes it very easy to forget validation for some of the branches.

In fact, the way we handle GIT_REFERENCE_BACKEND shows exactly one such edge case where we don't properly validate. When initializing a repository with one storage format and then reinitializing it with the environment variable set to a different format then we'd corrupt the repository because we silently change the format:

    $ git init repo
    $ git -C repo commit --allow-empty -m message
    $ GIT_REFERENCE_BACKEND=reftable git -C repo init
    fatal: could not open '.../refs/heads' for writing: Is a directory
    $ git -C repo log
    fatal: your current branch appears to be broken

Refactor the code so that we clearly distinguish between these two different concerns. This lets us clearly spell out the precedence order and makes the whole logic significantly easier to extend going forward.

Note that the new logic intentionally changes the precedence order so that "GIT_REFERENCE_BACKEND" is now overridden by the "--ref-storage-format=" command line option. This matches our usual precedence order, where explicit command line arguments override environment variables.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 setup.c         | 103 +++++++++++++++++++++++++++++++++++---------------------
 t/t0001-init.sh |  12 ++++++-
 2 files changed, 75 insertions(+), 40 deletions(-)
diff --git a/setup.c b/setup.c
index 3be7dac452..38fa5e854c 100644
--- a/setup.c
+++ b/setup.c
@@ -2674,7 +2674,7 @@ static void separate_git_dir(struct repository *repo,
 
 struct default_format_config {
 	int hash;
-	enum ref_storage_format ref_format;
+	enum ref_storage_format ref_storage_format;
 };
 
 static int read_default_format_config(const char *key, const char *value,
@@ -2699,8 +2699,8 @@ static int read_default_format_config(const char *key, const char *value,
 		ret = git_config_string(&str, key, value);
 		if (ret)
 			goto out;
-		cfg->ref_format = ref_storage_format_by_name(str);
-		if (cfg->ref_format == REF_STORAGE_FORMAT_UNKNOWN)
+		cfg->ref_storage_format = ref_storage_format_by_name(str);
+		if (cfg->ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			warning(_("unknown ref storage format '%s'"), str);
 		goto out;
 	}
@@ -2710,9 +2710,9 @@ static int read_default_format_config(const char *key, const char *value,
 	 * "init.defaultRefFormat" takes precedence over this setting.
 	 */
 	if (!strcmp(key, "feature.experimental") &&
-	    cfg->ref_format == REF_STORAGE_FORMAT_UNKNOWN &&
+	    cfg->ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN &&
 	    git_config_bool(key, value)) {
-		cfg->ref_format = REF_STORAGE_FORMAT_REFTABLE;
+		cfg->ref_storage_format = REF_STORAGE_FORMAT_REFTABLE;
 		ret = 0;
 		goto out;
 	}
@@ -2724,18 +2724,18 @@ static int read_default_format_config(const char *key, const char *value,
 }
 
 static void repository_format_configure(struct repository_format *repo_fmt,
-					int hash, enum ref_storage_format ref_format)
+					int hash, enum ref_storage_format ref_storage_format)
 {
 	struct default_format_config cfg = {
 		.hash = GIT_HASH_UNKNOWN,
-		.ref_format = REF_STORAGE_FORMAT_UNKNOWN,
+		.ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN,
 	};
 	struct config_options opts = {
 		.respect_includes = 1,
 		.ignore_repo = 1,
 		.ignore_worktree = 1,
 	};
-	const char *ref_backend_uri;
+	char *ref_storage_payload = NULL;
 	const char *env;
 
 	config_with_options(read_default_format_config, &cfg, NULL, NULL, &opts);
@@ -2761,40 +2761,65 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 		repo_fmt->hash_algo = cfg.hash;
 	}
 
-	env = getenv("GIT_DEFAULT_REF_FORMAT");
-	if (repo_fmt->version >= 0 &&
-	    ref_format != REF_STORAGE_FORMAT_UNKNOWN &&
-	    ref_format != repo_fmt->ref_storage_format) {
-		die(_("attempt to reinitialize repository with different reference storage format"));
-	} else if (ref_format != REF_STORAGE_FORMAT_UNKNOWN) {
-		repo_fmt->ref_storage_format = ref_format;
-	} else if (env) {
-		ref_format = ref_storage_format_by_name(env);
-		if (ref_format == REF_STORAGE_FORMAT_UNKNOWN)
-			die(_("unknown ref storage format '%s'"), env);
-		if (repo_fmt->version < 0 ||
-		    repo_fmt->ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
-			repo_fmt->ref_storage_format = ref_format;
-	} else if (cfg.ref_format != REF_STORAGE_FORMAT_UNKNOWN) {
-		repo_fmt->ref_storage_format = cfg.ref_format;
+	/*
+	 * We have the following order of preference when configuring the ref
+	 * storage format:
+	 *
+	 *   1. Explicit override via the command line, like in `git init
+	 *      --ref-storage-format=`.
+	 *
+	 *   2. Explicit override via the environment with
+	 *      GIT_REFERENCE_BACKEND.
+	 *
+	 *   3. Existing repository format. All the subsequent sources only
+	 *      kick in when there is no repository yet.
+	 *
+	 *   4. The default ref storage format for new repositories as
+	 *      configured via "GIT_DEFAULT_REF_FORMAT".
+	 *
+	 *   5. The default ref storage format for new repositories as
+	 *      configured via "init.defaultRefFormat"
+	 *
+	 *   6. Otherwise, we fall back to the default ref storage format
+	 *      compiled into Git.
+	 */
+	if (ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) {
+		/* nothing to do */
+	} else if ((env = getenv(GIT_REFERENCE_BACKEND_ENVIRONMENT))) {
+		ref_storage_format = ref_storage_format_by_uri(env, &ref_storage_payload);
+		if (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
+			die(_("unknown ref storage format specified via %s: '%s'"),
+			    GIT_REFERENCE_BACKEND_ENVIRONMENT, env);
+	} else if (repo_fmt->version >= 0) {
+		ref_storage_format = repo_fmt->ref_storage_format;
+		ref_storage_payload = xstrdup_or_null(repo_fmt->ref_storage_payload);
+	} else if ((env = getenv("GIT_DEFAULT_REF_FORMAT"))) {
+		ref_storage_format = ref_storage_format_by_name(env);
+		if (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
+			die(_("unknown ref storage format specified via %s: '%s'"),
+			    "GIT_DEFAULT_REF_FORMAT", env);
+	} else if (cfg.ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) {
+		ref_storage_format = cfg.ref_storage_format;
 	} else {
-		repo_fmt->ref_storage_format = REF_STORAGE_FORMAT_DEFAULT;
+		ref_storage_format = REF_STORAGE_FORMAT_DEFAULT;
 	}
 
-
-	ref_backend_uri = getenv(GIT_REFERENCE_BACKEND_ENVIRONMENT);
-	if (ref_backend_uri) {
-		enum ref_storage_format format;
-		char *payload;
-
-		format = ref_storage_format_by_uri(ref_backend_uri, &payload);
-		if (format == REF_STORAGE_FORMAT_UNKNOWN)
-			die(_("unknown ref storage format: '%s'"), ref_backend_uri);
-
-		repo_fmt->ref_storage_format = format;
-		free(repo_fmt->ref_storage_payload);
-		repo_fmt->ref_storage_payload = payload;
-	}
+	/*
+	 * If we have a preexisting repository we need to verify that its
+	 * current ref storage format does not change.
+	 */
+	if (repo_fmt->version >= 0) {
+		if (ref_storage_format != repo_fmt->ref_storage_format)
+			die(_("attempt to reinitialize repository with different reference storage format"));
+		if ((ref_storage_payload || repo_fmt->ref_storage_payload) &&
+		    strcmp(ref_storage_payload ? ref_storage_payload : "",
+			   repo_fmt->ref_storage_payload ? repo_fmt->ref_storage_payload : ""))
+			die(_("attempt to reinitialize repository with different reference storage payload"));
+	}
+
+	free(repo_fmt->ref_storage_payload);
+	repo_fmt->ref_storage_format = ref_storage_format;
+	repo_fmt->ref_storage_payload = ref_storage_payload;
 }
 
 int init_db(struct repository *repo,
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 6f4431bed7..ca44dfc1ce 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -643,12 +643,22 @@ test_expect_success DEFAULT_REPO_FORMAT 'extensions.refStorage with unknown back
 test_expect_success 'init with GIT_DEFAULT_REF_FORMAT=garbage' '
 	test_when_finished "rm -rf refformat" &&
 	cat >expect <<-EOF &&
-	fatal: unknown ref storage format ${SQ}garbage${SQ}
+	fatal: unknown ref storage format specified via GIT_DEFAULT_REF_FORMAT: ${SQ}garbage${SQ}
 	EOF
 	test_must_fail env GIT_DEFAULT_REF_FORMAT=garbage git init refformat 2>err &&
 	test_cmp expect err
 '
 
+test_expect_success 'GIT_REFERENCE_BACKEND refuses to reinitialize with different storage format' '
+	test_when_finished "rm -rf refbackend" &&
+	git init --ref-storage-format=files refbackend &&
+	cat >expect <<-EOF &&
+	fatal: attempt to reinitialize repository with different reference storage format
+	EOF
+	test_must_fail env GIT_REFERENCE_BACKEND=reftable git init refbackend 2>err &&
+	test_cmp expect err
+'
+
 test_expect_success 'init warns about invalid init.defaultRefFormat' '
 	test_when_finished "rm -rf repo" &&
 	test_config_global init.defaultRefFormat garbage &&
-- 
2.55.0.1074.ge7621b4bad.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 13 of 20 in “Fix inconsistent ref storage format terminology”
  1. 00/13 Fix inconsistent ref storage format terminologyPatrick Steinhardt, Sep 9, 2026
  2. 01/13 parse-options: allow for hidden aliasesPatrick Steinhardt, Sep 9, 2026
  3. Karthik NayakSep 14, 2026
  4. Kaartic SivaraamSep 15, 2026
  5. Junio C HamanoSep 15, 2026
  6. 02/13 builtin/init: rename "--ref-format=" to "--ref-storage-format="Patrick Steinhardt, Sep 9, 2026
  7. 03/13 builtin/clone: rename "--ref-format=" to "--ref-storage-format="Patrick Steinhardt, Sep 9, 2026
  8. 04/13 builtin/refs: rename "--ref-format=" to "--ref-storage-format="Patrick Steinhardt, Sep 9, 2026
  9. 05/13 builtin/submodule: rename "--ref-format=" to "--ref-storage-format="Patrick Steinhardt, Sep 9, 2026
  10. 06/13 builtin/rev-parse: rename "--show-ref-format" to "--show-ref-storage-format"Patrick Steinhardt, Sep 9, 2026
  11. 07/13 help: rename "default-ref-format" to "default-ref-storage-format"Patrick Steinhardt, Sep 9, 2026
  12. 08/13 refs: expose function to parse reference URIsPatrick Steinhardt, Sep 9, 2026
  13. 09/13 setup: refactor how we configure the ref storage formatPatrick Steinhardt, Sep 9, 2026
  14. 10/13 setup: rename ref storage format environment variablesPatrick Steinhardt, Sep 9, 2026
  15. 11/13 t: rename GIT_TEST_DEFAULT_REF_FORMATPatrick Steinhardt, Sep 9, 2026
  16. 12/13 setup: rename "init.defaultRefFormat" to "init.defaultRefStorageFormat"Patrick Steinhardt, Sep 9, 2026
  17. 13/13 setup: allow "--ref-storage-format=" to specify a payloadPatrick Steinhardt, Sep 9, 2026
  18. Karthik NayakSep 14, 2026
  19. Junio C HamanoSep 14, 2026
  20. Kaartic SivaraamSep 14, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.