threads / patch / 66784

patch, 5 partsImprovements for storage format handling

Subject: [PATCH 0/5] Improvements for storage format handling

## tl;dr

The patches add GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT and storage-format payloads. In a first trial, 1105 of about 35,000 tests fail with a non-default object path. Read the story.

replies: 12people: 3as markdown or json

Patrick Steinhardt· Nov 5, 2026, 15:02 UTC · lore
Hi,

this patch series includes a couple of improvements for how we handle ref and object storage formats.

Most importantly, this series allows us to run tests with a different object storage backend via `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT=`. As we don't have an alternative backend, I also introduce the ability to specify a default object (and ref) storage format that carries a payload so that you can run all tests with "files://objdir", for example. That surfaces a bunch of tests that assume objects exist in ".git/objects".

Overall it's a surprisingly small number of test failures, only 1105 out of 35000 tests fail. Fixing those is not part of this patch series, but we'll start upstreaming fixes once this lands. Such tests will either have to be adapted to use plumbing commands, or they'll need to be tagged with a prerequisite. It's going to be a bunch of work, but it's basically preparing for the introduction of alternative backends already.

The series is built on top of 0ffa3e35c7c (The 15th batch, 2026-11-04) with ps/odb-object-storage-extension at 87a55d790ca (builtin/submodule: propagate "--object-storage-format=", 2026-10-27) merged into it.

Thanks!
Patrick
---
Patrick Steinhardt (5):
      refs: create leading directories
      odb/files: create leading directories
      setup: allow speciyfing payloads with default storage formats
      builtin/rev-parse: print payload for the ref storage format
      t: allow testing with a non-default object storage foramt
 builtin/rev-parse.c     |  6 +++++-
 odb/source-files.c      |  1 +
 refs/files-backend.c    |  1 +
 refs/reftable-backend.c |  1 +
 setup.c                 | 24 ++++++++++++++++-----
 t/t0001-init.sh         | 57 ++++++++++++++++++++++++++++++++++++++++++++++---
 t/t1423-ref-backend.sh  | 24 +++++++++++++++++++++
 t/t1500-rev-parse.sh    |  9 ++++++++
 t/test-lib.sh           |  4 ++++
 9 files changed, 118 insertions(+), 9 deletions(-)

--- base-commit: df904e7114cda6ca4ebf514115835a2dcd2f2b64 change-id: 20261105-pks-storage-format-improvements-7b716599bada

Patrick Steinhardt· Nov 5, 2026, 15:02 UTC · re: Patrick Steinhardt · lore

[PATCH 1/5] refs: create leading directories

Both the "files" and "reftable" backends accept a payload that allows the user to use a non-default location for their files. So e.g. if the user specifies "files://refdir" as the ref storage format, then we would put files into ".git/refdir/refs" instead of ".git/refs".

Using this is somewhat unwieldy though, as the directory must exist before we create the ref storage as we'd otherwise die. This isn't that bad in the case where the user specifies an absolute path that is outside of the repository. But it becomes quite weird when asked to create the directory relative to commondir, as the user would first have to "mkdir -p repo/.git/refdir".

Adapt the reference backends to create leading directories. Note that we use the `noshare()` variant for this because it's very much possible that it will exist outside of the repository, and if so we shouldn't adapt permissions on such leading directories.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 refs/files-backend.c    |  1 +
 refs/reftable-backend.c |  1 +
 t/t1423-ref-backend.sh  | 24 ++++++++++++++++++++++++
 3 files changed, 26 insertions(+)
Show changes to 3 files +26 −0

refs/files-backend.c, refs/reftable-backend.c, t/t1423-ref-backend.sh

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 87d04557a53..1bdb16c0e7b 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3699,6 +3699,7 @@ static int files_ref_store_create_on_disk(struct ref_store *ref_store,
 	 *   they do not understand the reference format extension.
 	 */
 	strbuf_addf(&sb, "%s/refs", ref_store->gitdir);
+	safe_create_leading_directories_no_share(sb.buf);
 	safe_create_dir(refs->base.repo, sb.buf, 1);
 	adjust_shared_perm(refs->base.repo, sb.buf);
 
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index d0de0663552..3ba6a9ac0db 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -504,6 +504,7 @@ static int reftable_be_create_on_disk(struct ref_store *ref_store,
 	struct strbuf sb = STRBUF_INIT;
 
 	strbuf_addf(&sb, "%s/reftable", refs->base.gitdir);
+	safe_create_leading_directories_no_share(sb.buf);
 	safe_create_dir(ref_store->repo, sb.buf, 1);
 	strbuf_reset(&sb);
 
diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
index ee2bb66b99c..85d1e876b82 100755
--- a/t/t1423-ref-backend.sh
+++ b/t/t1423-ref-backend.sh
@@ -284,6 +284,30 @@ test_expect_success 'initializing repository with --ref-storage-format and paylo
 	test_cmp expect actual
 '
 
+for format in files reftable
+do
+	test_expect_success "default ref storage format with $format backend and payload" '
+		test_when_finished "rm -rf repo refdir" &&
+		BACKEND="$format://$(pwd)/refdir" &&
+		echo "$BACKEND" >expect &&
+		git init --ref-storage-format="$BACKEND" repo &&
+		test_path_is_dir refdir &&
+		case "$format" in
+		files)
+			test_path_is_file refdir/HEAD &&
+			test_path_is_dir refdir/refs
+			;;
+		reftable)
+			test_path_is_dir refdir/reftable &&
+			test_path_is_file refdir/reftable/tables.list
+			;;
+		*)
+			BUG "unknown ref storage format: $format"
+			;;
+		esac
+	'
+done
+
 test_expect_success 'cloning repository with alt ref directory' '
 	test_when_finished "rm -rf source repo refdir" &&
 	mkdir refdir &&
-- 
2.56.0.500.g5d06dc02cd.dirty
Karthik Nayak· Nov 6, 2026, 10:37 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 1/5] refs: create leading directories

Patrick Steinhardt <ps@pks.im> writes:
The patch looks good.
Show 10 quoted lines
> diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
> index ee2bb66b99c..85d1e876b82 100755
> --- a/t/t1423-ref-backend.sh
> +++ b/t/t1423-ref-backend.sh
> @@ -284,6 +284,30 @@ test_expect_success 'initializing repository with --ref-storage-format and paylo
>  	test_cmp expect actual
>  '
>
> +for format in files reftable
> +do
This would run the loop for both the files and reftable CI jobs.
Show 22 quoted lines
> +	test_expect_success "default ref storage format with $format backend and payload" '
> +		test_when_finished "rm -rf repo refdir" &&
> +		BACKEND="$format://$(pwd)/refdir" &&
> +		echo "$BACKEND" >expect &&
> +		git init --ref-storage-format="$BACKEND" repo &&
> +		test_path_is_dir refdir &&
> +		case "$format" in
> +		files)
> +			test_path_is_file refdir/HEAD &&
> +			test_path_is_dir refdir/refs
> +			;;
> +		reftable)
> +			test_path_is_dir refdir/reftable &&
> +			test_path_is_file refdir/reftable/tables.list
> +			;;
> +		*)
> +			BUG "unknown ref storage format: $format"
> +			;;
> +		esac
> +	'
> +done
> +
Should we instead do (untested):
test_expect_success "default ref storage format with $format backend
and payload" '
	test_when_finished "rm -rf repo refdir" &&
    format="$(test_detect_ref_format)" &&
	BACKEND="$format://$(pwd)/refdir" &&
	echo "$BACKEND" >expect &&
	git init --ref-storage-format="$BACKEND" repo &&
	test_path_is_dir refdir &&
	case "$format" in
	files)
		test_path_is_file refdir/HEAD &&
		test_path_is_dir refdir/refs
		;;
	reftable)
		test_path_is_dir refdir/reftable &&
		test_path_is_file refdir/reftable/tables.list
		;;
	*)
		BUG "unknown ref storage format: $format"
		;;
	esac
'

This eliminates the loop, while still testing both backends on their respective CI jobs.

Show 6 quoted lines
>  test_expect_success 'cloning repository with alt ref directory' '
>  	test_when_finished "rm -rf source repo refdir" &&
>  	mkdir refdir &&
>
> --
> 2.56.0.500.g5d06dc02cd.dirty
Patrick Steinhardt· Nov 6, 2026, 11:01 UTC · re: Karthik Nayak · lore

Re: [PATCH 1/5] refs: create leading directories

On Fri, Nov 06, 2026 at 02:37:23AM -0800, Karthik Nayak wrote:
Show 64 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
> > index ee2bb66b99c..85d1e876b82 100755
> > --- a/t/t1423-ref-backend.sh
> > +++ b/t/t1423-ref-backend.sh
> > @@ -284,6 +284,30 @@ test_expect_success 'initializing repository with --ref-storage-format and paylo
> >  	test_cmp expect actual
> >  '
> >
> > +for format in files reftable
> > +do
> 
> This would run the loop for both the files and reftable CI jobs.
> 
> > +	test_expect_success "default ref storage format with $format backend and payload" '
> > +		test_when_finished "rm -rf repo refdir" &&
> > +		BACKEND="$format://$(pwd)/refdir" &&
> > +		echo "$BACKEND" >expect &&
> > +		git init --ref-storage-format="$BACKEND" repo &&
> > +		test_path_is_dir refdir &&
> > +		case "$format" in
> > +		files)
> > +			test_path_is_file refdir/HEAD &&
> > +			test_path_is_dir refdir/refs
> > +			;;
> > +		reftable)
> > +			test_path_is_dir refdir/reftable &&
> > +			test_path_is_file refdir/reftable/tables.list
> > +			;;
> > +		*)
> > +			BUG "unknown ref storage format: $format"
> > +			;;
> > +		esac
> > +	'
> > +done
> > +
> 
> Should we instead do (untested):
> 
> test_expect_success "default ref storage format with $format backend
> and payload" '
> 	test_when_finished "rm -rf repo refdir" &&
>     format="$(test_detect_ref_format)" &&
> 	BACKEND="$format://$(pwd)/refdir" &&
> 	echo "$BACKEND" >expect &&
> 	git init --ref-storage-format="$BACKEND" repo &&
> 	test_path_is_dir refdir &&
> 	case "$format" in
> 	files)
> 		test_path_is_file refdir/HEAD &&
> 		test_path_is_dir refdir/refs
> 		;;
> 	reftable)
> 		test_path_is_dir refdir/reftable &&
> 		test_path_is_file refdir/reftable/tables.list
> 		;;
> 	*)
> 		BUG "unknown ref storage format: $format"
> 		;;
> 	esac
> '
> 
> This eliminates the loop, while still testing both backends on their
> respective CI jobs.

Well, in this case the system under test is explicitly the ref storage format itself and how it behaves together with the extension. The GIT_TEST_DEFAULT_REF_STORAGE format on the other hand is design to explicitly test that everything else works as-is with an alternate ref store.

So I think it makes sense to keep this explicit test.
Patrick
Patrick Steinhardt· Nov 5, 2026, 15:02 UTC · re: Patrick Steinhardt · lore

[PATCH 2/5] odb/files: create leading directories

With the same reasoning as in the preceding commit, create leading directories when creating the "files" object backend on disk.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb/source-files.c | 1 +
 t/t0001-init.sh    | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)
Show changes to 2 files +4 −3

odb/source-files.c, t/t0001-init.sh

diff --git a/odb/source-files.c b/odb/source-files.c
index 6b423499fe5..e7503548e38 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -97,6 +97,7 @@ static int odb_source_files_create_on_disk(struct odb_source *source,
 	struct strbuf line = STRBUF_INIT;
 	int ret;
 
+	safe_create_leading_directories_no_share(source->path);
 	safe_create_dir(source->odb->repo, source->path, 1);
 
 	strbuf_addf(&path, "%s/pack", source->path);
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 23317e0733a..735ab9d8475 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -944,13 +944,13 @@ test_expect_success 'init with --object-storage-format=files://<relative-path>'
 
 test_expect_success 'init with --object-storage-format=files://<absolute-path>' '
 	test_when_finished "rm -rf objformat objdir" &&
-	git init --object-storage-format=files://"$(pwd)/objdir" objformat &&
-	echo "files://$(pwd)/objdir" >expect &&
+	git init --object-storage-format=files://"$(pwd)/absolute/objdir" objformat &&
+	echo "files://$(pwd)/absolute/objdir" >expect &&
 	git -C objformat config get extensions.objectstorage >actual &&
 	test_cmp expect actual &&
 	test_commit -C objformat A &&
 	git -C objformat rev-parse --verify HEAD &&
-	test_path_is_dir objdir/pack
+	test_path_is_dir absolute/objdir/pack
 '
 
 test_expect_success 'GIT_OBJECT_DIRECTORY takes precedence over extensions.objectStorage payload' '
-- 
2.56.0.500.g5d06dc02cd.dirty
Patrick Steinhardt· Nov 5, 2026, 15:02 UTC · re: Patrick Steinhardt · lore

[PATCH 3/5] setup: allow speciyfing payloads with default storage formats

While the "git init --ref-storage-format=" switch allows the user to specify a payload that gets passed to the backend, this isn't possible when specifying a default via "init.defaultRefStorageFormat" or via `GIT_DEFAULT_REF_STORAGE_FORMAT`. The same restriction also applies to the object storage format, which needlessly restricts the flexibility and usefulness of these mechanism.

Extend the logic to allow specifying a payload.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 setup.c         | 24 +++++++++++++++++++-----
 t/t0001-init.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 5 deletions(-)
Show changes to 2 files +70 −5

setup.c, t/t0001-init.sh

diff --git a/setup.c b/setup.c
index 186815c7801..cedc031c54f 100644
--- a/setup.c
+++ b/setup.c
@@ -2797,7 +2797,9 @@ static void separate_git_dir(struct repository *repo,
 struct default_format_config {
 	int hash;
 	enum ref_storage_format ref_storage_format;
+	char *ref_storage_payload;
 	enum object_storage_format object_storage_format;
+	char *object_storage_payload;
 };
 
 static int read_default_format_config(const char *key, const char *value,
@@ -2823,9 +2825,12 @@ static int read_default_format_config(const char *key, const char *value,
 		ret = git_config_string(&str, key, value);
 		if (ret)
 			goto out;
-		cfg->ref_storage_format = ref_storage_format_by_name(str);
+
+		FREE_AND_NULL(cfg->ref_storage_payload);
+		cfg->ref_storage_format = ref_storage_format_by_uri(str, &cfg->ref_storage_payload);
 		if (cfg->ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			warning(_("unknown ref storage format '%s'"), str);
+
 		goto out;
 	}
 
@@ -2833,9 +2838,12 @@ static int read_default_format_config(const char *key, const char *value,
 		ret = git_config_string(&str, key, value);
 		if (ret)
 			goto out;
-		cfg->object_storage_format = object_storage_format_by_name(str);
+
+		FREE_AND_NULL(cfg->object_storage_payload);
+		cfg->object_storage_format = object_storage_format_by_uri(str, &cfg->object_storage_payload);
 		if (cfg->object_storage_format == OBJECT_STORAGE_FORMAT_UNKNOWN)
 			warning(_("unknown object storage format '%s'"), str);
+
 		goto out;
 	}
 
@@ -2947,17 +2955,19 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 		ref_storage_format = repo_fmt->ref_storage_format;
 		ref_storage_payload = xstrdup_or_null(repo_fmt->ref_storage_payload);
 	} else if ((env = getenv("GIT_DEFAULT_REF_STORAGE_FORMAT"))) {
-		ref_storage_format = ref_storage_format_by_name(env);
+		ref_storage_format = ref_storage_format_by_uri(env, &ref_storage_payload);
 		if (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			die(_("unknown ref storage format specified via %s: '%s'"),
 			    "GIT_DEFAULT_REF_STORAGE_FORMAT", env);
 	} else if ((env = getenv("GIT_DEFAULT_REF_FORMAT"))) {
-		ref_storage_format = ref_storage_format_by_name(env);
+		ref_storage_format = ref_storage_format_by_uri(env, &ref_storage_payload);
 		if (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			die(_("unknown ref storage format specified via %s: '%s'"),
 			    "GIT_DEFAULT_REF_FORMAT", env);
 	} else if (cfg.ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) {
 		ref_storage_format = cfg.ref_storage_format;
+		ref_storage_payload = cfg.ref_storage_payload;
+		cfg.ref_storage_payload = NULL;
 	} else {
 		ref_storage_format = REF_STORAGE_FORMAT_DEFAULT;
 	}
@@ -2995,12 +3005,14 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 		object_storage_format = repo_fmt->object_storage_format;
 		object_storage_payload = xstrdup_or_null(repo_fmt->object_storage_payload);
 	} else if ((env = getenv("GIT_DEFAULT_OBJECT_STORAGE_FORMAT"))) {
-		object_storage_format = object_storage_format_by_name(env);
+		object_storage_format = object_storage_format_by_uri(env, &object_storage_payload);
 		if (object_storage_format == OBJECT_STORAGE_FORMAT_UNKNOWN)
 			die(_("unknown object storage format specified via %s: '%s'"),
 			    "GIT_DEFAULT_OBJECT_STORAGE_FORMAT", env);
 	} else if (cfg.object_storage_format != OBJECT_STORAGE_FORMAT_UNKNOWN) {
 		object_storage_format = cfg.object_storage_format;
+		object_storage_payload = cfg.object_storage_payload;
+		cfg.object_storage_payload = NULL;
 	} else {
 		object_storage_format = OBJECT_STORAGE_FORMAT_FILES;
 	}
@@ -3024,10 +3036,12 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 			die(_("attempt to reinitialize repository with different object storage payload"));
 	}
 
+	free(cfg.ref_storage_payload);
 	free(repo_fmt->ref_storage_payload);
 	repo_fmt->ref_storage_format = ref_storage_format;
 	repo_fmt->ref_storage_payload = ref_storage_payload;
 
+	free(cfg.object_storage_payload);
 	free(repo_fmt->object_storage_payload);
 	repo_fmt->object_storage_format = object_storage_format;
 	repo_fmt->object_storage_payload = object_storage_payload;
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 735ab9d8475..09b7258bf10 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -745,6 +745,33 @@ do
 	'
 done
 
+test_expect_success 'initializing repository with init.defaultRefStorageFormat and payload' '
+	test_when_finished "rm -rf repo refdir" &&
+
+	BACKEND="files://$(pwd)/refdir" &&
+	test_config_global init.defaultRefStorageFormat "$BACKEND" &&
+	(
+		sane_unset GIT_DEFAULT_REF_STORAGE_FORMAT &&
+		git init repo
+	) &&
+	test_path_is_file refdir/HEAD &&
+	test_path_is_dir refdir/refs &&
+	git -C repo config get extensions.refstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'initializing repository with GIT_DEFAULT_REF_STORAGE_FORMAT and payload' '
+	test_when_finished "rm -rf repo refdir" &&
+	BACKEND="files://$(pwd)/refdir" &&
+	GIT_DEFAULT_REF_STORAGE_FORMAT="$BACKEND" git init repo &&
+	test_path_is_file refdir/HEAD &&
+	test_path_is_dir refdir/refs &&
+	git -C repo config get extensions.refstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
 test_expect_success "--ref-storage-format= overrides GIT_DEFAULT_REF_STORAGE_FOMAT" '
 	test_when_finished "rm -rf refformat" &&
 	GIT_DEFAULT_REF_STORAGE_FORMAT=files git init --ref-storage-format=reftable refformat &&
@@ -1064,6 +1091,30 @@ test_expect_success 'init with init.defaultObjectStorageFormat' '
 	test_cmp expect actual
 '
 
+test_expect_success 'init with init.defaultObjectStorageFormat and payload' '
+	test_when_finished "rm -rf repo objdir" &&
+	BACKEND="files://$(pwd)/objdir" &&
+	test_config_global init.defaultObjectStorageFormat "$BACKEND" &&
+	(
+		sane_unset GIT_DEFAULT_OBJECT_STORAGE_FORMAT &&
+		git init repo
+	) &&
+	test_path_is_dir objdir/pack &&
+	git -C repo config get extensions.objectstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'init with GIT_DEFAULT_OBJECT_STORAGE_FORMAT and payload' '
+	test_when_finished "rm -rf repo objdir" &&
+	BACKEND="files://$(pwd)/objdir" &&
+	GIT_DEFAULT_OBJECT_STORAGE_FORMAT="$BACKEND" git init repo &&
+	test_path_is_dir objdir/pack &&
+	git -C repo config get extensions.objectstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&
-- 
2.56.0.500.g5d06dc02cd.dirty
Patrick Steinhardt· Nov 5, 2026, 15:02 UTC · re: Patrick Steinhardt · lore

[PATCH 4/5] builtin/rev-parse: print payload for the ref storage format

While we print both the object storage format and its payload for the "--show-object-storage-format" switch, we only print the ref storage format with "--show-ref-storage-format". Adapt the latter to also show the payload.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/rev-parse.c  | 6 +++++-
 t/t1500-rev-parse.sh | 9 +++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)
Show changes to 2 files +14 −1

builtin/rev-parse.c, t/t1500-rev-parse.sh

diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index d2154eac8d2..fbc5ef87276 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -1136,7 +1136,11 @@ int cmd_rev_parse(int argc,
 				continue;
 			}
 			if (!strcmp(arg, "--show-ref-format") || !strcmp(arg, "--show-ref-storage-format")) {
-				puts(ref_storage_format_to_name(the_repository->ref_storage_format));
+				const char *format = ref_storage_format_to_name(the_repository->ref_storage_format);
+				if (the_repository->ref_storage_payload)
+					printf("%s://%s\n", format, the_repository->ref_storage_payload);
+				else
+					puts(format);
 				continue;
 			}
 			if (!strcmp(arg, "--show-object-storage-format")) {
diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh
index 30bf8a4d0a0..c4e6bc1dd89 100755
--- a/t/t1500-rev-parse.sh
+++ b/t/t1500-rev-parse.sh
@@ -209,6 +209,7 @@ test_expect_success 'rev-parse --show-object-format in repo' '
 
 
 test_expect_success RUST 'rev-parse --show-object-format in repo with compat mode' '
+	test_when_finished "rm -rf repo" &&
 	mkdir repo &&
 	(
 		sane_unset GIT_DEFAULT_HASH &&
@@ -247,6 +248,14 @@ test_expect_success 'rev-parse --show-ref-storage-format' '
 	test_cmp expect actual
 '
 
+test_expect_success 'rev-parse --show-ref-storage-format with payload' '
+	test_when_finished "rm -rf repo" &&
+	git init repo --ref-storage-format="files://$(pwd)/refdir" &&
+	git -C repo rev-parse --show-ref-storage-format >actual &&
+	echo "files://$(pwd)/refdir" >expect &&
+	test_cmp expect actual
+'
+
 test_expect_success 'rev-parse --show-ref-storage-format with invalid storage' '
 	test_when_finished "rm -rf repo" &&
 	git init repo &&
-- 
2.56.0.500.g5d06dc02cd.dirty
Karthik Nayak· Nov 6, 2026, 10:40 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 4/5] builtin/rev-parse: print payload for the ref storage format

Patrick Steinhardt <ps@pks.im> writes:
Show 5 quoted lines
> While we print both the object storage format and its payload for the
> "--show-object-storage-format" switch, we only print the ref storage
> format with "--show-ref-storage-format". Adapt the latter to also show
> the payload.
>
Nice, this makes sense and was clearly a miss. Thanks for fixing.
[snip]
Patrick Steinhardt· Nov 5, 2026, 15:02 UTC · re: Patrick Steinhardt · lore

[PATCH 5/5] t: allow testing with a non-default object storage foramt

It is already possible to test Git with a non-default ref storage format by specifying `GIT_TEST_DEFAULT_REF_STORAGE_FORMAT`. We don't provide the same mechanism yet for testing the object storage format, and will instead use the default ("files"). Even worse though, we may inherit whatever the user has configured in `GIT_DEFAULT_OBJECT_STORAGE_FORMAT`.

Introduce `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT` and have it default to the "files" backend.

Note that this environment variable isn't all that useful, as we do not have an alternative backend. But what it does allow is to figure out which tests will definitely break once we do have an alternative backend because they assume that objects exist in ".git/objects", as we can now move objects into a different location by specifying a payload.

Sure enough, running with "files://objdir" shows that around 1105 or our tests are failing. This is a surprisingly small number of failures, but we'll have to fix them if we ever want to introduce an alternative backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/test-lib.sh | 4 ++++
 1 file changed, 4 insertions(+)
Show changes to t/test-lib.sh +4 −0
diff --git a/t/test-lib.sh b/t/test-lib.sh
index a893963f86c..7fdd8272162 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -576,9 +576,13 @@ export EDITOR
 GIT_TEST_BUILTIN_HASH=$("$GIT_BINARY" version --build-options | sed -ne 's/^default-hash: //p')
 GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}"
 export GIT_DEFAULT_HASH
+
 GIT_DEFAULT_REF_STORAGE_FORMAT="${GIT_TEST_DEFAULT_REF_STORAGE_FORMAT:-files}"
 export GIT_DEFAULT_REF_STORAGE_FORMAT
 
+GIT_DEFAULT_OBJECT_STORAGE_FORMAT="${GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT:-files}"
+export GIT_DEFAULT_OBJECT_STORAGE_FORMAT
+
 # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
 GIT_TRACE_BARE=1
 export GIT_TRACE_BARE
-- 
2.56.0.500.g5d06dc02cd.dirty
Junio C Hamano· Nov 6, 2026, 18:59 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 5/5] t: allow testing with a non-default object storage foramt

Patrick Steinhardt <ps@pks.im> writes:
> Subject: Re: [PATCH 5/5] t: allow testing with a non-default object storage foramt
"foramt" -> "format".
Show 11 quoted lines
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index a893963f86c..7fdd8272162 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -576,9 +576,13 @@ export EDITOR
>  GIT_TEST_BUILTIN_HASH=$("$GIT_BINARY" version --build-options | sed -ne 's/^default-hash: //p')
>  GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}"
>  export GIT_DEFAULT_HASH
> +
>  GIT_DEFAULT_REF_STORAGE_FORMAT="${GIT_TEST_DEFAULT_REF_STORAGE_FORMAT:-files}"
>  export GIT_DEFAULT_REF_STORAGE_FORMAT
This unrelated change created unnecessary conflicts.
No need to resend.
Show 7 quoted lines
>  
> +GIT_DEFAULT_OBJECT_STORAGE_FORMAT="${GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT:-files}"
> +export GIT_DEFAULT_OBJECT_STORAGE_FORMAT
> +
>  # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
>  GIT_TRACE_BARE=1
>  export GIT_TRACE_BARE
Patrick Steinhardt· Nov 9, 2026, 07:31 UTC · re: Junio C Hamano · lore

Re: [PATCH 5/5] t: allow testing with a non-default object storage foramt

On Fri, Nov 06, 2026 at 10:59:00AM -0800, Junio C Hamano wrote:
Show 21 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Subject: Re: [PATCH 5/5] t: allow testing with a non-default object storage foramt
> 
> "foramt" -> "format".
> 
> > diff --git a/t/test-lib.sh b/t/test-lib.sh
> > index a893963f86c..7fdd8272162 100644
> > --- a/t/test-lib.sh
> > +++ b/t/test-lib.sh
> > @@ -576,9 +576,13 @@ export EDITOR
> >  GIT_TEST_BUILTIN_HASH=$("$GIT_BINARY" version --build-options | sed -ne 's/^default-hash: //p')
> >  GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}"
> >  export GIT_DEFAULT_HASH
> > +
> >  GIT_DEFAULT_REF_STORAGE_FORMAT="${GIT_TEST_DEFAULT_REF_STORAGE_FORMAT:-files}"
> >  export GIT_DEFAULT_REF_STORAGE_FORMAT
> 
> This unrelated change created unnecessary conflicts.
> 
> No need to resend.

Okay, I've adapted these locally so that I have them included in case I need to reroll. Thanks!

Patrick
Karthik Nayak· Nov 6, 2026, 10:41 UTC · re: Patrick Steinhardt · lore

Re: [PATCH 0/5] Improvements for storage format handling

Patrick Steinhardt <ps@pks.im> writes:
Show 28 quoted lines
> Hi,
>
> this patch series includes a couple of improvements for how we handle
> ref and object storage formats.
>
> Most importantly, this series allows us to run tests with a different
> object storage backend via `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT=`. As
> we don't have an alternative backend, I also introduce the ability to
> specify a default object (and ref) storage format that carries a payload
> so that you can run all tests with "files://objdir", for example. That
> surfaces a bunch of tests that assume objects exist in ".git/objects".
>
> Overall it's a surprisingly small number of test failures, only 1105 out
> of 35000 tests fail. Fixing those is not part of this patch series, but
> we'll start upstreaming fixes once this lands. Such tests will either
> have to be adapted to use plumbing commands, or they'll need to be
> tagged with a prerequisite. It's going to be a bunch of work, but it's
> basically preparing for the introduction of alternative backends
> already.
>
> The series is built on top of 0ffa3e35c7c (The 15th batch, 2026-11-04)
> with ps/odb-object-storage-extension at 87a55d790ca (builtin/submodule:
> propagate "--object-storage-format=", 2026-10-27) merged into it.
>
> Thanks!
>
> Patrick
>

This was a very easy read. I don't have much to add apart from a comment on 1/5. But you can also leave it as is! Thanks

Patrick Steinhardt· Nov 6, 2026, 11:02 UTC · re: Karthik Nayak · lore

Re: [PATCH 0/5] Improvements for storage format handling

On Fri, Nov 06, 2026 at 02:41:41AM -0800, Karthik Nayak wrote:
Show 33 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Hi,
> >
> > this patch series includes a couple of improvements for how we handle
> > ref and object storage formats.
> >
> > Most importantly, this series allows us to run tests with a different
> > object storage backend via `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT=`. As
> > we don't have an alternative backend, I also introduce the ability to
> > specify a default object (and ref) storage format that carries a payload
> > so that you can run all tests with "files://objdir", for example. That
> > surfaces a bunch of tests that assume objects exist in ".git/objects".
> >
> > Overall it's a surprisingly small number of test failures, only 1105 out
> > of 35000 tests fail. Fixing those is not part of this patch series, but
> > we'll start upstreaming fixes once this lands. Such tests will either
> > have to be adapted to use plumbing commands, or they'll need to be
> > tagged with a prerequisite. It's going to be a bunch of work, but it's
> > basically preparing for the introduction of alternative backends
> > already.
> >
> > The series is built on top of 0ffa3e35c7c (The 15th batch, 2026-11-04)
> > with ps/odb-object-storage-extension at 87a55d790ca (builtin/submodule:
> > propagate "--object-storage-format=", 2026-10-27) merged into it.
> >
> > Thanks!
> >
> > Patrick
> >
> 
> This was a very easy read. I don't have much to add apart from a comment
> on 1/5. But you can also leave it as is! Thanks

Thanks for your review! I'd prefer to leave 1/5 as-is as explained in my reply there.

Patrick

← back to recent threads