From: Patrick Steinhardt Date: Thu, 06 Aug 2026 07:50:58 GMT Subject: [PATCH v4 0/6] odb: make creation of object database pluggable Message-ID: <20260806-pks-odb-create-on-disk-v4-0-ba8b4fdd2e3c@pks.im> In-Reply-To: <20260724-pks-odb-create-on-disk-v1-0-3b3d265d979b@pks.im> Hi, when creating a new repository we create a couple of on-disk data structures for the object database. This includes the "objects/" directory hierarchy with "objects/info" and "objects/pack", which are specific to the backend. This patch series makes the creation of the on-disk data structures pluggable. While we continue to always create "objects/" regardless of the backend (it's required for a repository to be recognized as such), the other subdirectories are now created by the backend. This will allow other backends to plug in their own logic. The series starts with a small detour into the loose-object map. This detour is required so that we can defer initialization of the object database itself to a later point in time. The series is based on 9a0c4701dc (The 7th batch, 2026-07-22). Changes in v4: - Drop `APPLY_REPOSITOY_FORMAT_SKIP_ODB_CREATION` in favor of explicit calls to `odb_new()`. - Remove a useless call to `xstrdup()`. - Mark a string as translatable. - Link to v3: https://patch.msgid.link/20260805-pks-odb-create-on-disk-v3-0-c0ee3ac5141f@pks.im Changes in v3: - Move handling of GIT_OBJECT_DIRECTORY and GIT_ALTERNATE_OBJECT_DIRECTORIES into `odb_new()` itself. This deduplicates some of the logic and also preps us for a future where alternates are handled in the "files" backend itself. - Link to v2: https://patch.msgid.link/20260804-pks-odb-create-on-disk-v2-0-ddf8b59bd207@pks.im Changes in v2: - Add a testcase that demonstrates the bug fixed with alternate loose object maps. - Rename the "inmemory" bakcend to "in-memory". - Clarify some commit messages. - Link to v1: https://patch.msgid.link/20260724-pks-odb-create-on-disk-v1-0-3b3d265d979b@pks.im Thanks! Patrick --- Patrick Steinhardt (6): loose: load loose object map for the correct source setup: detangle loading of loose object maps setup: handle ODB-related environment variables in `odb_new()` setup: defer object database creation odb/source: introduce function to map source type to name odb: make creation of on-disk structures pluggable loose.c | 25 ++++++++++---------- loose.h | 1 + odb.c | 21 +++++++++-------- odb.h | 17 ++++++++++++-- odb/source-files.c | 19 +++++++++++++++ odb/source-files.h | 4 +++- odb/source-inmemory.h | 4 +++- odb/source-loose.c | 2 ++ odb/source-loose.h | 4 +++- odb/source-packed.h | 4 +++- odb/source.c | 19 +++++++++++++++ odb/source.h | 29 +++++++++++++++++++++++ repository.c | 3 +-- setup.c | 54 +++++++++++++++++++++---------------------- t/t1016-compatObjectFormat.sh | 18 +++++++++++++++ t/unit-tests/u-odb-inmemory.c | 2 +- 16 files changed, 169 insertions(+), 57 deletions(-) Range-diff versus v3: 1: e1a585a3f7 = 1: 6dd8d575c6 loose: load loose object map for the correct source 2: 1f1200f7ba = 2: 1e7adada64 setup: detangle loading of loose object maps 3: af02e520a2 ! 3: 2265f38695 setup: handle ODB-related environment variables in `odb_new()` @@ odb.c: int odb_write_object_stream(struct object_database *odb, + o->sources = odb_source_new(o, primary_source, true); o->sources_tail = &o->sources->next; - o->alternate_db = xstrdup_or_null(secondary_sources); +- o->alternate_db = xstrdup_or_null(secondary_sources); ++ o->alternate_db = secondary_sources; o->inmemory_objects = &odb_source_inmemory_new(o)->base; - free(to_free); - -+ free(secondary_sources); + free(primary_source); return o; } 4: 2c794be101 ! 4: 5274ee6bab setup: defer object database creation @@ Commit message object database, but it's not usable yet until we eventually call `create_object_directory()`. - Defer the object database creation so that we handle both steps in the - same function. + Lift the call to `odb_new()` out of `apply_repository_format()` so that + callers have more wiggle room with when exactly they call it, and adapt + them accordingly. The only exception is `init_db()`, where we now defer + creating the object database until we call `create_object_database()`. + + With this change, initializing and creating the object database on disk + is now neatly encapsulated in a single function, which will make it + easier for a subsequent commit to move creation of the on-disk data + structures into the `struct odb_source` backends. Signed-off-by: Patrick Steinhardt + ## repository.c ## +@@ repository.c: int repo_init(struct repository *repo, + warning("%s", err.buf); + goto error; + } ++ repo->objects = odb_new(repo, 0); + + if (worktree) + repo_set_worktree(repo, worktree); + ## setup.c ## @@ setup.c: int apply_repository_format(struct repository *repo, enum apply_repository_format_flags flags, @@ setup.c: int apply_repository_format(struct repository *repo, - if (flags & APPLY_REPOSITORY_FORMAT_HONOR_ENV) - odb_new_flags |= ODB_NEW_HONOR_ENV; - repo->objects = odb_new(repo, odb_new_flags); -+ if (!(flags & APPLY_REPOSITORY_FORMAT_SKIP_ODB_CREATION)) { -+ enum odb_new_flags odb_new_flags = 0; -+ if (flags & APPLY_REPOSITORY_FORMAT_HONOR_ENV) -+ odb_new_flags |= ODB_NEW_HONOR_ENV; -+ repo->objects = odb_new(repo, odb_new_flags); -+ } - +- return 0; } + +@@ setup.c: const char *enter_repo(struct repository *repo, const char *path, unsigned flags + read_and_verify_repository_format(&fmt, ".", NULL); + if (apply_repository_format(repo, &fmt, APPLY_REPOSITORY_FORMAT_HONOR_ENV, &err) < 0) + die("%s", err.buf); ++ repo->objects = odb_new(repo, ODB_NEW_HONOR_ENV); + startup_info->have_repository = 1; + + clear_repository_format(&fmt); +@@ setup.c: const char *setup_git_directory_gently(struct repository *repo, int *nongit_ok) + if (apply_repository_format(repo, &discovery.format, + APPLY_REPOSITORY_FORMAT_HONOR_ENV, &err) < 0) + die("%s", err.buf); ++ repo->objects = odb_new(repo, ODB_NEW_HONOR_ENV); + + clear_repository_format(&discovery.format); + strbuf_release(&err); @@ setup.c: static int create_default_files(struct repository *repo, return reinit; } @@ setup.c: int init_db(struct repository *repo, repository_format_configure(&repo_fmt, hash, ref_storage_format); - if (apply_repository_format(repo, &repo_fmt, APPLY_REPOSITORY_FORMAT_HONOR_ENV, &err) < 0) + if (apply_repository_format(repo, &repo_fmt, -+ APPLY_REPOSITORY_FORMAT_HONOR_ENV | -+ APPLY_REPOSITORY_FORMAT_SKIP_ODB_CREATION, &err) < 0) ++ APPLY_REPOSITORY_FORMAT_HONOR_ENV, &err) < 0) die("%s", err.buf); - startup_info->have_repository = 1; @@ setup.c: int init_db(struct repository *repo, if (repo_settings_get_shared_repository(repo)) { char buf[10]; - - ## setup.h ## -@@ setup.h: enum apply_repository_format_flags { - * relate to the object database. - */ - APPLY_REPOSITORY_FORMAT_HONOR_ENV = (1 << 0), -+ -+ /* -+ * Usually, the object database is created after the repository format -+ * was applied. This step is skipped if this flag is set, which leaves -+ * us with a partially-working repository. -+ * -+ * This is useful when initializing a new repository. -+ */ -+ APPLY_REPOSITORY_FORMAT_SKIP_ODB_CREATION = (1 << 1), - }; - - /* 5: 7397c760df = 5: b444a314a6 odb/source: introduce function to map source type to name 6: 7049e41a73 ! 6: acb48f1072 odb: make creation of on-disk structures pluggable @@ setup.c: static int create_default_files(struct repository *repo, - - strbuf_release(&path); + if (odb_source_create_on_disk(repo->objects->sources) < 0) -+ die("failed creating object database"); ++ die(_("failed creating object database")); } static void separate_git_dir(const char *git_dir, const char *git_link) --- base-commit: 9a0c4701dcd5725c4184599322b52933ff5005ca change-id: 20260710-pks-odb-create-on-disk-ae8757861c69