git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 0/7] odb: introduce object storage extension

From
Patrick Steinhardt <ps@pks.im>
Date
Oct 27, 2026, 06:59 UTC
Message-ID
<20261027-b4-pks-odb-extension-v4-0-5d28ca0ea1dc@pks.im>
In-Reply-To
<20261019-b4-pks-odb-extension-v1-0-143a5963b77f@pks.im>
Hi,

this patch series introduces the object storage extension. The extension is of course similar in spirit and design to the ref storage extension: it can be configured via "extensions.objectStorage", has the same syntax with "<schema>[://<payload>]", and we have the same mechanisms to configure it ("--object-storage-format=", GIT_OBJECT_STORAGE_FORMAT, GIT_DEFAULT_OBJECT_STORAGE_FORMAT, "init.defaultObjetcStorageFormat").

For now, we only support the "files" object storage format, so not a lot of stuff really changes. That being said, its optional payload allows you to configure a different location for the ".git/objects" directory. So if you specify "extensions.objectStorage=files://objs", objects would be stored in ".git/objs" instead.

The series is built on top of c51408889c (The 6th batch, 2026-10-16) with the following dependencies merged into it:

  - ps/odb-files-alternates at 0ba59c8453 (odb/source: drop
    `read_alternates` callback, 2026-10-08) merged into it.
  - ps/setup-enforce-repo-passed-to-create-repository-has-no-state
    (phew, what a mouthful) at 0ba59c8453 (odb/source: drop
    `read_alternates` callback, 2026-10-08).

Note that these two series conflict with one another. The conflict can easily be resolved by just deleting the whole conflicting sections.

Changes in v4:
  - Provide some justification for the payload of the "files" backend.
  - Remove stray added newline.
  - Some comment smithing.
  - Link to v3: https://patch.msgid.link/20261023-b4-pks-odb-extension-v3-0-b1bb30a006db@pks.im
Changes in v3:
  - Split up object storage format URI into object storage format and
    payload. This matches how we do it for the ref storage format and
    thus brings us much closer to it, where we mostly use the exact same
    logic. For now though this restricts a couple cases, like for
    example submodule clones do not pass down payloads to the submodules
    anymore. This is intentional so that object and ref storage format
    have the same behaviour.
  - There's an en explicit `enum object_storage_format` now, matching
    how we handle this for the ref storage format.
  - Link to v2: https://patch.msgid.link/20261021-b4-pks-odb-extension-v2-0-8cf11b65eda8@pks.im
Changes in v2:
  - Ensure that `repo->object_storage_format` is always populated.
  - Link to v1: https://patch.msgid.link/20261019-b4-pks-odb-extension-v1-0-143a5963b77f@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (7):
      loose: drop unused `repo_write_loose_object_map()`
      setup: introduce object storage extension
      setup: allow overriding object storage format via environment
      builtin: expose the object storage format
      builtin/init: introduce handling of object storage format
      builtin/clone: introduce "--object-storage-format="
      builtin/submodule: propagate "--object-storage-format="
 Documentation/config/extensions.adoc       |  23 +++
 Documentation/config/init.adoc             |   5 +
 Documentation/git-clone.adoc               |  11 ++
 Documentation/git-init.adoc                |  10 +-
 Documentation/git-repo.adoc                |   6 +
 Documentation/git-rev-parse.adoc           |   5 +
 Documentation/git-submodule.adoc           |  10 +-
 Documentation/git.adoc                     |  10 ++
 Documentation/object-storage-format.adoc   |   6 +
 builtin/clone.c                            |  63 +++++++-
 builtin/init-db.c                          |   7 +-
 builtin/repo.c                             |  10 ++
 builtin/rev-parse.c                        |   9 ++
 builtin/submodule--helper.c                |  40 +++++
 environment.h                              |   1 +
 git-submodule.sh                           |  18 +++
 loose.c                                    |  46 ------
 loose.h                                    |   1 -
 odb.c                                      |  46 +++++-
 odb.h                                      |  13 ++
 odb/source-files.c                         |   7 +
 odb/source-files.h                         |   1 +
 odb/source.c                               |  14 +-
 odb/source.h                               |   5 +-
 refs.c                                     |   3 +-
 repository.c                               |  10 ++
 repository.h                               |  18 +++
 setup.c                                    | 132 ++++++++++++++++-
 setup.h                                    |   5 +
 t/meson.build                              |   1 +
 t/t0001-init.sh                            | 225 +++++++++++++++++++++++++++++
 t/t1900-repo-info.sh                       |   3 +
 t/t5601-clone.sh                           |  68 +++++++++
 t/t7427-submodule-object-storage-format.sh | 104 +++++++++++++
 t/unit-tests/u-odb-inmemory.c              |   1 +
 35 files changed, 871 insertions(+), 66 deletions(-)
Range-diff versus v3:
1:  906c3d3e3c3 = 1:  f09fc3200f9 loose: drop unused `repo_write_loose_object_map()`
2:  3d3bcf0f825 ! 2:  655e1de3b9d setup: introduce object storage extension
    @@ Commit message
             allows you to configure a different location for the ".git/objects"
             directory.
     
    +    Making the location of the "files" backend configurable is not he most
    +    thrilling feature of this new extension. But it gives us a couple of
    +    advantages:
    +
    +      - It allows us to start exercising the payload logic in our tests so
    +        that we know it's working properly for backends that will need it.
    +
    +      - It allows us to easily detect tests that are broken with the new
    +        extension because they for example assume that an object is a file
    +        in ".git/objects/". Testing is not part of this series yet, but will
    +        be done in subsequent patch series.
    +
    +      - It aligns with how we handle the ref storage extension, where we can
    +        also change its location to an arbitrary path.
    +
    +      - It allows for some very esoteric and edge-case-y gimmicks. For
    +        example, you can initialize a single repository "skeleton" and then
    +        multiplex therein by setting both GIT_REF_STORAGE_FORMAT and
    +        GIT_OBJECT_STORAGE_FORMAT to point to different databases. These
    +        edge cases won't matter to many people out there, but we will use
    +        this for example as a building block for migrations.
    +
         Eventually, this mechanism will be used to support alternative backends.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
    @@ setup.c: static enum extension_result handle_extension(const char *var,
      		return EXTENSION_OK;
      	} else if (!strcmp(ext, "relativeworktrees")) {
      		data->relative_worktrees = git_config_bool(var, value);
    -@@ setup.c: static enum extension_result handle_extension(const char *var,
    - 		data->submodule_path_cfg = git_config_bool(var, value);
    - 		return EXTENSION_OK;
    - 	}
    -+
    - 	return EXTENSION_UNKNOWN;
    - }
    - 
     @@ setup.c: void clear_repository_format(struct repository_format *format)
      	free(format->work_tree);
      	free(format->partial_clone);
3:  8c7fcbe8659 ! 3:  9b57baa86f4 setup: allow overriding object storage format via environment
    @@ setup.c: const char *setup_git_directory_gently(struct repository *repo, int *no
      			}
      
     +			/*
    -+			 * Likewise, the GIT_OBJECT_STORAGE_FORMAT environment
    -+			 * variable should override the repository config for
    ++			 * The GIT_OBJECT_STORAGE_FORMAT env variable
    ++			 * should override the repository config for
     +			 * 'extensions.objectStorage'.
     +			 */
     +			object_storage_uri = getenv(GIT_OBJECT_STORAGE_FORMAT_ENVIRONMENT);
4:  e09beec6d1e = 4:  69f11546357 builtin: expose the object storage format
5:  7ff9419d250 = 5:  9a3d8abd624 builtin/init: introduce handling of object storage format
6:  03e119eadbb = 6:  f37be19589a builtin/clone: introduce "--object-storage-format="
7:  02d33ab036d = 7:  aec7e0f12e8 builtin/submodule: propagate "--object-storage-format="

--- base-commit: 0d31382da9336683214372ba328fe9ecbc8bcb9a change-id: 20261006-b4-pks-odb-extension-36de693bb943

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 50 of 59 in “odb: introduce object storage extension”
  1. 0/7 odb: introduce object storage extensionPatrick Steinhardt, Oct 19, 2026
  2. 1/7 loose: drop unused `repo_write_loose_object_map()`Patrick Steinhardt, Oct 19, 2026
  3. Karthik NayakOct 19, 2026
  4. Patrick SteinhardtOct 20, 2026
  5. Karthik NayakOct 20, 2026
  6. 2/7 setup: introduce object storage extensionPatrick Steinhardt, Oct 19, 2026
  7. Karthik NayakOct 19, 2026
  8. Junio C HamanoOct 20, 2026
  9. Karthik NayakOct 20, 2026
  10. Patrick SteinhardtOct 20, 2026
  11. Karthik NayakOct 20, 2026
  12. 3/7 setup: allow overriding object storage format via environmentPatrick Steinhardt, Oct 19, 2026
  13. Karthik NayakOct 19, 2026
  14. Patrick SteinhardtOct 20, 2026
  15. Karthik NayakOct 20, 2026
  16. 4/7 builtin: expose the object storage formatPatrick Steinhardt, Oct 19, 2026
  17. Karthik NayakOct 19, 2026
  18. Patrick SteinhardtOct 20, 2026
  19. Karthik NayakOct 20, 2026
  20. 5/7 builtin/init: introduce handling of object storage formatPatrick Steinhardt, Oct 19, 2026
  21. 6/7 builtin/clone: introduce "--object-storage-format="Patrick Steinhardt, Oct 19, 2026
  22. 7/7 builtin/submodule: propagate "--object-storage-format="Patrick Steinhardt, Oct 19, 2026
  23. 0/7 odb: introduce object storage extensionPatrick Steinhardt, Oct 21, 2026
  24. 1/7 loose: drop unused `repo_write_loose_object_map()`Patrick Steinhardt, Oct 21, 2026
  25. 2/7 setup: introduce object storage extensionPatrick Steinhardt, Oct 21, 2026
  26. 3/7 setup: allow overriding object storage format via environmentPatrick Steinhardt, Oct 21, 2026
  27. 4/7 builtin: expose the object storage formatPatrick Steinhardt, Oct 21, 2026
  28. 5/7 builtin/init: introduce handling of object storage formatPatrick Steinhardt, Oct 21, 2026
  29. Karthik NayakOct 21, 2026
  30. Patrick SteinhardtOct 23, 2026
  31. 6/7 builtin/clone: introduce "--object-storage-format="Patrick Steinhardt, Oct 21, 2026
  32. 7/7 builtin/submodule: propagate "--object-storage-format="Patrick Steinhardt, Oct 21, 2026
  33. Karthik NayakOct 21, 2026
  34. 0/7 odb: introduce object storage extensionPatrick Steinhardt, Oct 23, 2026
  35. 1/7 loose: drop unused `repo_write_loose_object_map()`Patrick Steinhardt, Oct 23, 2026
  36. 2/7 setup: introduce object storage extensionPatrick Steinhardt, Oct 23, 2026
  37. SZEDER GáborOct 24, 2026
  38. Patrick SteinhardtOct 26, 2026
  39. Karthik NayakOct 26, 2026
  40. Patrick SteinhardtOct 26, 2026
  41. 3/7 setup: allow overriding object storage format via environmentPatrick Steinhardt, Oct 23, 2026
  42. Karthik NayakOct 26, 2026
  43. Patrick SteinhardtOct 26, 2026
  44. 4/7 builtin: expose the object storage formatPatrick Steinhardt, Oct 23, 2026
  45. 5/7 builtin/init: introduce handling of object storage formatPatrick Steinhardt, Oct 23, 2026
  46. 6/7 builtin/clone: introduce "--object-storage-format="Patrick Steinhardt, Oct 23, 2026
  47. 7/7 builtin/submodule: propagate "--object-storage-format="Patrick Steinhardt, Oct 23, 2026
  48. Karthik NayakOct 26, 2026
  49. Patrick SteinhardtOct 26, 2026
  50. 0/7 odb: introduce object storage extensionPatrick Steinhardt, Oct 27, 2026
  51. 1/7 loose: drop unused `repo_write_loose_object_map()`Patrick Steinhardt, Oct 27, 2026
  52. 2/7 setup: introduce object storage extensionPatrick Steinhardt, Oct 27, 2026
  53. 3/7 setup: allow overriding object storage format via environmentPatrick Steinhardt, Oct 27, 2026
  54. 4/7 builtin: expose the object storage formatPatrick Steinhardt, Oct 27, 2026
  55. 5/7 builtin/init: introduce handling of object storage formatPatrick Steinhardt, Oct 27, 2026
  56. 6/7 builtin/clone: introduce "--object-storage-format="Patrick Steinhardt, Oct 27, 2026
  57. 7/7 builtin/submodule: propagate "--object-storage-format="Patrick Steinhardt, Oct 27, 2026
  58. Karthik NayakOct 28, 2026
  59. Patrick SteinhardtOct 28, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.