From: Patrick Steinhardt Date: Tue, 27 Oct 2026 06:59:24 GMT Subject: [PATCH v4 0/7] odb: introduce object storage extension Message-ID: <20261027-b4-pks-odb-extension-v4-0-5d28ca0ea1dc@pks.im> In-Reply-To: <20261019-b4-pks-odb-extension-v1-0-143a5963b77f@pks.im> Hi, this patch series introduces the object storage extension. The extension is of course similar in spirit and design to the ref storage extension: it can be configured via "extensions.objectStorage", has the same syntax with "[://]", and we have the same mechanisms to configure it ("--object-storage-format=", GIT_OBJECT_STORAGE_FORMAT, GIT_DEFAULT_OBJECT_STORAGE_FORMAT, "init.defaultObjetcStorageFormat"). For now, we only support the "files" object storage format, so not a lot of stuff really changes. That being said, its optional payload allows you to configure a different location for the ".git/objects" directory. So if you specify "extensions.objectStorage=files://objs", objects would be stored in ".git/objs" instead. The series is built on top of c51408889c (The 6th batch, 2026-10-16) with the following dependencies merged into it: - ps/odb-files-alternates at 0ba59c8453 (odb/source: drop `read_alternates` callback, 2026-10-08) merged into it. - ps/setup-enforce-repo-passed-to-create-repository-has-no-state (phew, what a mouthful) at 0ba59c8453 (odb/source: drop `read_alternates` callback, 2026-10-08). Note that these two series conflict with one another. The conflict can easily be resolved by just deleting the whole conflicting sections. Changes in v4: - Provide some justification for the payload of the "files" backend. - Remove stray added newline. - Some comment smithing. - Link to v3: https://patch.msgid.link/20261023-b4-pks-odb-extension-v3-0-b1bb30a006db@pks.im Changes in v3: - Split up object storage format URI into object storage format and payload. This matches how we do it for the ref storage format and thus brings us much closer to it, where we mostly use the exact same logic. For now though this restricts a couple cases, like for example submodule clones do not pass down payloads to the submodules anymore. This is intentional so that object and ref storage format have the same behaviour. - There's an en explicit `enum object_storage_format` now, matching how we handle this for the ref storage format. - Link to v2: https://patch.msgid.link/20261021-b4-pks-odb-extension-v2-0-8cf11b65eda8@pks.im Changes in v2: - Ensure that `repo->object_storage_format` is always populated. - Link to v1: https://patch.msgid.link/20261019-b4-pks-odb-extension-v1-0-143a5963b77f@pks.im Thanks! Patrick --- Patrick Steinhardt (7): loose: drop unused `repo_write_loose_object_map()` setup: introduce object storage extension setup: allow overriding object storage format via environment builtin: expose the object storage format builtin/init: introduce handling of object storage format builtin/clone: introduce "--object-storage-format=" builtin/submodule: propagate "--object-storage-format=" Documentation/config/extensions.adoc | 23 +++ Documentation/config/init.adoc | 5 + Documentation/git-clone.adoc | 11 ++ Documentation/git-init.adoc | 10 +- Documentation/git-repo.adoc | 6 + Documentation/git-rev-parse.adoc | 5 + Documentation/git-submodule.adoc | 10 +- Documentation/git.adoc | 10 ++ Documentation/object-storage-format.adoc | 6 + builtin/clone.c | 63 +++++++- builtin/init-db.c | 7 +- builtin/repo.c | 10 ++ builtin/rev-parse.c | 9 ++ builtin/submodule--helper.c | 40 +++++ environment.h | 1 + git-submodule.sh | 18 +++ loose.c | 46 ------ loose.h | 1 - odb.c | 46 +++++- odb.h | 13 ++ odb/source-files.c | 7 + odb/source-files.h | 1 + odb/source.c | 14 +- odb/source.h | 5 +- refs.c | 3 +- repository.c | 10 ++ repository.h | 18 +++ setup.c | 132 ++++++++++++++++- setup.h | 5 + t/meson.build | 1 + t/t0001-init.sh | 225 +++++++++++++++++++++++++++++ t/t1900-repo-info.sh | 3 + t/t5601-clone.sh | 68 +++++++++ t/t7427-submodule-object-storage-format.sh | 104 +++++++++++++ t/unit-tests/u-odb-inmemory.c | 1 + 35 files changed, 871 insertions(+), 66 deletions(-) Range-diff versus v3: 1: 906c3d3e3c3 = 1: f09fc3200f9 loose: drop unused `repo_write_loose_object_map()` 2: 3d3bcf0f825 ! 2: 655e1de3b9d setup: introduce object storage extension @@ Commit message allows you to configure a different location for the ".git/objects" directory. + Making the location of the "files" backend configurable is not he most + thrilling feature of this new extension. But it gives us a couple of + advantages: + + - It allows us to start exercising the payload logic in our tests so + that we know it's working properly for backends that will need it. + + - It allows us to easily detect tests that are broken with the new + extension because they for example assume that an object is a file + in ".git/objects/". Testing is not part of this series yet, but will + be done in subsequent patch series. + + - It aligns with how we handle the ref storage extension, where we can + also change its location to an arbitrary path. + + - It allows for some very esoteric and edge-case-y gimmicks. For + example, you can initialize a single repository "skeleton" and then + multiplex therein by setting both GIT_REF_STORAGE_FORMAT and + GIT_OBJECT_STORAGE_FORMAT to point to different databases. These + edge cases won't matter to many people out there, but we will use + this for example as a building block for migrations. + Eventually, this mechanism will be used to support alternative backends. Signed-off-by: Patrick Steinhardt @@ setup.c: static enum extension_result handle_extension(const char *var, return EXTENSION_OK; } else if (!strcmp(ext, "relativeworktrees")) { data->relative_worktrees = git_config_bool(var, value); -@@ setup.c: static enum extension_result handle_extension(const char *var, - data->submodule_path_cfg = git_config_bool(var, value); - return EXTENSION_OK; - } -+ - return EXTENSION_UNKNOWN; - } - @@ setup.c: void clear_repository_format(struct repository_format *format) free(format->work_tree); free(format->partial_clone); 3: 8c7fcbe8659 ! 3: 9b57baa86f4 setup: allow overriding object storage format via environment @@ setup.c: const char *setup_git_directory_gently(struct repository *repo, int *no } + /* -+ * Likewise, the GIT_OBJECT_STORAGE_FORMAT environment -+ * variable should override the repository config for ++ * The GIT_OBJECT_STORAGE_FORMAT env variable ++ * should override the repository config for + * 'extensions.objectStorage'. + */ + object_storage_uri = getenv(GIT_OBJECT_STORAGE_FORMAT_ENVIRONMENT); 4: e09beec6d1e = 4: 69f11546357 builtin: expose the object storage format 5: 7ff9419d250 = 5: 9a3d8abd624 builtin/init: introduce handling of object storage format 6: 03e119eadbb = 6: f37be19589a builtin/clone: introduce "--object-storage-format=" 7: 02d33ab036d = 7: aec7e0f12e8 builtin/submodule: propagate "--object-storage-format=" --- base-commit: 0d31382da9336683214372ba328fe9ecbc8bcb9a change-id: 20261006-b4-pks-odb-extension-36de693bb943