Re: [PATCH 00/17] object-store: carve out the object database subsystem
- From
Junio C Hamano <gitster@pobox.com>
- Date
- May 7, 2025, 17:02 UTC
- Message-ID
- <xmqqa57oe4mj.fsf@gitster.g>
- In-Reply-To
- <5bea19fe-6616-4f01-a78d-9b7da94db899@gmail.com>
Derrick Stolee <stolee@gmail.com> writes:
Show 19 quoted lines
> Patches 1 and 2 involve renaming some core structures, and I had > some questions around these names (since we hope to be stuck with > the new names for a long time). I was thinking out loud on a per- > patch basis, but now want to collect my thoughts around these: > > * raw_object_store currently describes the abstraction that contains > all objects that can be accessed within the repository. This may > include multiple alternates. Patch 1 renames this to > 'object_database'. > > * object_directory currently describes a single directory that > has the same structure as $GIT_DIR/objects/ but may be an alternate > or a submodule object directory. Patch 2 renames this to > 'odb_backend'. > > My concerns around this are basically around not liking "backend" for > this purpose. When I think of a backend, I'm thinking about the > implementation details (like the refs backend being files or reftable) > and not multiple distinct locations that have their own objects.
Yup, odb_backend_files (aot odb_backend_redis) or something?
Show 8 quoted lines
> * 'struct object_directory' could be renamed to 'struct odb_shard' or > 'struct odb_slice' or similar. I may even recommend 'odb_partition' > though that does imply some disjointness that is not guaranteed (an > object can exist in multiple parts). > > * In the event that we create multiple implementations for storing > objects, then a 'struct odb_shard' could point to a backend to help > find the appropriate methods for interacting with its storage.
Hmph, I do not have strong opinions, but I consider it an implementation detail of one particular backend, namely, the filesystem based backend, that it can link together multiple object_directory instances and present them as if they form a single object database, just like all files within a single object_directory form an illusion of a single object database (aka key-value store) even though some objects are stored in individual loose object files while many others are packed in a single packfile.
I did not expect you would want to go to the world where a single "shard" consists of an object_directory backed by the filesystem and some other more database-y backend. It is an interesting idea, but we'd need to worry about many things we do not have to worry about right now. E.g. what do the precedence rules among different components within a single "shard" look like? How do we express "in this repository, local filesystem-backed piece is consulted first, and then check this piece backed by low-cost but high-latency storage backend"?
> I do mention that the rename of the object-store.[c|h] files may be > unnecessary, or perhaps could be delayed until this series is merged > and the collateral is calmed.
Right now, merge-fix needed against all other topics in flight look like this, in order to merge it to 'seen'.
$ git show --oneline -U0 merge-fix/ps/object-store c8afa0ab8e merge-fix/ps/object-store
diff --git a/fetch-object-info.c b/fetch-object-info.c index 1b2c6c4d27..6c2abb7805 100644 --- a/fetch-object-info.c +++ b/fetch-object-info.c @@ -7 +7 @@ -#include "object-store.h" +#include "odb.h" diff --git a/fetch-object-info.h b/fetch-object-info.h index 7910d7f153..d35284bd6b 100644 --- a/fetch-object-info.h +++ b/fetch-object-info.h @@ -6 +6 @@ -#include "object-store.h" +#include "odb.h" diff --git a/transport.h b/transport.h index 5a4f27451a..3ef3acf865 100644 --- a/transport.h +++ b/transport.h @@ -8 +8 @@ -#include "object-store.h" +#include "odb.h" IOW, not too horrible.