From: Patrick Steinhardt Date: Wed, 05 Aug 2026 09:28:50 GMT Subject: [PATCH v3 0/6] odb: make creation of object database pluggable Message-ID: <20260805-pks-odb-create-on-disk-v3-0-c0ee3ac5141f@pks.im> In-Reply-To: <20260724-pks-odb-create-on-disk-v1-0-3b3d265d979b@pks.im> Hi, when creating a new repository we create a couple of on-disk data structures for the object database. This includes the "objects/" directory hierarchy with "objects/info" and "objects/pack", which are specific to the backend. This patch series makes the creation of the on-disk data structures pluggable. While we continue to always create "objects/" regardless of the backend (it's required for a repository to be recognized as such), the other subdirectories are now created by the backend. This will allow other backends to plug in their own logic. The series starts with a small detour into the loose-object map. This detour is required so that we can defer initialization of the object database itself to a later point in time. The series is based on 9a0c4701dc (The 7th batch, 2026-07-22). Changes in v3: - Move handling of GIT_OBJECT_DIRECTORY and GIT_ALTERNATE_OBJECT_DIRECTORIES into `odb_new()` itself. This deduplicates some of the logic and also preps us for a future where alternates are handled in the "files" backend itself. - Link to v2: https://patch.msgid.link/20260804-pks-odb-create-on-disk-v2-0-ddf8b59bd207@pks.im Changes in v2: - Add a testcase that demonstrates the bug fixed with alternate loose object maps. - Rename the "inmemory" bakcend to "in-memory". - Clarify some commit messages. - Link to v1: https://patch.msgid.link/20260724-pks-odb-create-on-disk-v1-0-3b3d265d979b@pks.im Thanks! Patrick --- Patrick Steinhardt (6): loose: load loose object map for the correct source setup: detangle loading of loose object maps setup: handle ODB-related environment variables in `odb_new()` setup: defer object database creation odb/source: introduce function to map source type to name odb: make creation of on-disk structures pluggable loose.c | 25 +++++++++--------- loose.h | 1 + odb.c | 20 +++++++++------ odb.h | 17 ++++++++++-- odb/source-files.c | 19 ++++++++++++++ odb/source-files.h | 4 ++- odb/source-inmemory.h | 4 ++- odb/source-loose.c | 2 ++ odb/source-loose.h | 4 ++- odb/source-packed.h | 4 ++- odb/source.c | 19 ++++++++++++++ odb/source.h | 29 +++++++++++++++++++++ repository.c | 2 -- setup.c | 60 ++++++++++++++++++++++++------------------- setup.h | 9 +++++++ t/t1016-compatObjectFormat.sh | 18 +++++++++++++ t/unit-tests/u-odb-inmemory.c | 2 +- 17 files changed, 183 insertions(+), 56 deletions(-) Range-diff versus v2: 1: b0beb61a74 = 1: d384dd0635 loose: load loose object map for the correct source 2: 097bdcad14 = 2: 0ee1b3c032 setup: detangle loading of loose object maps -: ---------- > 3: f52992b9bd setup: handle ODB-related environment variables in `odb_new()` 3: 06645224ef ! 4: 4524fc5ec4 setup: defer object database creation @@ Commit message Signed-off-by: Patrick Steinhardt ## setup.c ## -@@ setup.c: enum discovery_result discover_git_directory_reason(struct strbuf *commondir, - return result; - } - -+static void get_object_directories(char **object_directory, -+ char **alternate_object_directories) -+{ -+ *object_directory = xstrdup_or_null(getenv(DB_ENVIRONMENT)); -+ *alternate_object_directories = xstrdup_or_null(getenv(ALTERNATE_DB_ENVIRONMENT)); -+} -+ - int apply_repository_format(struct repository *repo, - const struct repository_format *format, - enum apply_repository_format_flags flags, @@ setup.c: int apply_repository_format(struct repository *repo, - if (flags & APPLY_REPOSITORY_FORMAT_HONOR_ENV) { - const char *shallow_file; + enum apply_repository_format_flags flags, + struct strbuf *err) + { +- enum odb_new_flags odb_new_flags = 0; +- + if (verify_repository_format(format, err) < 0) + return -1; -- object_directory = xstrdup_or_null(getenv(DB_ENVIRONMENT)); -- alternate_object_directories = xstrdup_or_null(getenv(ALTERNATE_DB_ENVIRONMENT)); -+ get_object_directories(&object_directory, -+ &alternate_object_directories); -+ - shallow_file = getenv(GIT_SHALLOW_FILE_ENVIRONMENT); - if (shallow_file) - set_alternate_shallow_file(repo, shallow_file); @@ setup.c: int apply_repository_format(struct repository *repo, repo->repository_format_precious_objects = format->precious_objects; -- repo->objects = odb_new(repo, object_directory, -- alternate_object_directories); -+ if (!(flags & APPLY_REPOSITORY_FORMAT_SKIP_ODB_CREATION)) -+ repo->objects = odb_new(repo, object_directory, -+ alternate_object_directories); +- if (flags & APPLY_REPOSITORY_FORMAT_HONOR_ENV) +- odb_new_flags |= ODB_NEW_HONOR_ENV; +- repo->objects = odb_new(repo, odb_new_flags); ++ if (!(flags & APPLY_REPOSITORY_FORMAT_SKIP_ODB_CREATION)) { ++ enum odb_new_flags odb_new_flags = 0; ++ if (flags & APPLY_REPOSITORY_FORMAT_HONOR_ENV) ++ odb_new_flags |= ODB_NEW_HONOR_ENV; ++ repo->objects = odb_new(repo, odb_new_flags); ++ } - free(alternate_object_directories); - free(object_directory); + return 0; + } @@ setup.c: static int create_default_files(struct repository *repo, return reinit; } @@ setup.c: static int create_default_files(struct repository *repo, -static void create_object_directory(struct repository *repo) +static void create_object_database(struct repository *repo) { -+ char *object_directory, *alternate_object_directories; struct strbuf path = STRBUF_INIT; size_t baselen; -+ get_object_directories(&object_directory, &alternate_object_directories); -+ repo->objects = odb_new(repo, object_directory, -+ alternate_object_directories); ++ repo->objects = odb_new(repo, ODB_NEW_HONOR_ENV); + strbuf_addstr(&path, repo_get_object_directory(repo)); baselen = path.len; -@@ setup.c: static void create_object_directory(struct repository *repo) - strbuf_addstr(&path, "/info"); - safe_create_dir(repo, path.buf, 1); - -+ free(alternate_object_directories); -+ free(object_directory); - strbuf_release(&path); - } - @@ setup.c: int init_db(struct repository *repo, */ read_and_verify_repository_format(&repo_fmt, repo_get_git_dir(repo), NULL); 4: 46ad0386bb = 5: c526fd526b odb/source: introduce function to map source type to name 5: 3063325cf9 ! 6: d752e48eba odb: make creation of on-disk structures pluggable @@ odb/source.h: static inline void odb_source_close(struct odb_source *source) ## setup.c ## @@ setup.c: static int create_default_files(struct repository *repo, + static void create_object_database(struct repository *repo) { - char *object_directory, *alternate_object_directories; - struct strbuf path = STRBUF_INIT; - size_t baselen; - - get_object_directories(&object_directory, &alternate_object_directories); -- repo->objects = odb_new(repo, object_directory, -- alternate_object_directories); - -- strbuf_addstr(&path, repo_get_object_directory(repo)); -- baselen = path.len; -- -- safe_create_dir(repo, path.buf, 1); + /* + * Create the "objects" directory in the common directory. This is done + * so that the repository can be discovered regardless of the backend @@ setup.c: static int create_default_files(struct repository *repo, + * then we skip this step, as the repository won't be discoverable + * anyway without the environment variable. + */ -+ if (!object_directory) { ++ if (!getenv(DB_ENVIRONMENT)) { + struct strbuf objects_dir = STRBUF_INIT; + repo_common_path_append(repo, &objects_dir, "objects"); + safe_create_dir(repo, objects_dir.buf, 1); + strbuf_release(&objects_dir); + } + repo->objects = odb_new(repo, ODB_NEW_HONOR_ENV); + +- strbuf_addstr(&path, repo_get_object_directory(repo)); +- baselen = path.len; +- +- safe_create_dir(repo, path.buf, 1); +- - strbuf_setlen(&path, baselen); - strbuf_addstr(&path, "/pack"); - safe_create_dir(repo, path.buf, 1); -+ repo->objects = odb_new(repo, object_directory, -+ alternate_object_directories); - +- - strbuf_setlen(&path, baselen); - strbuf_addstr(&path, "/info"); - safe_create_dir(repo, path.buf, 1); +- +- strbuf_release(&path); + if (odb_source_create_on_disk(repo->objects->sources) < 0) + die("failed creating object database"); - - free(alternate_object_directories); - free(object_directory); -- strbuf_release(&path); } static void separate_git_dir(const char *git_dir, const char *git_link) --- base-commit: 9a0c4701dcd5725c4184599322b52933ff5005ca change-id: 20260710-pks-odb-create-on-disk-ae8757861c69