Re: [PATCH 00/17] object-store: carve out the object database subsystem

From: Patrick Steinhardt <ps@pks.im>
Date: May 9, 2025, 11:25 UTC
Message-ID: <aB3mNiiAGE2KHbGY@pks.im>
In-Reply-To: <xmqqa57oe4mj.fsf@gitster.g>

On Wed, May 07, 2025 at 10:02:12AM -0700, Junio C Hamano wrote:

Show 23 quoted lines

> Derrick Stolee <stolee@gmail.com> writes:
> 
> > Patches 1 and 2 involve renaming some core structures, and I had
> > some questions around these names (since we hope to be stuck with
> > the new names for a long time). I was thinking out loud on a per-
> > patch basis, but now want to collect my thoughts around these:
> >
> >  * raw_object_store currently describes the abstraction that contains
> >    all objects that can be accessed within the repository. This may
> >    include multiple alternates. Patch 1 renames this to
> >    'object_database'.
> >
> >  * object_directory currently describes a single directory that
> >    has the same structure as $GIT_DIR/objects/ but may be an alternate
> >    or a submodule object directory. Patch 2 renames this to
> >    'odb_backend'.
> >
> > My concerns around this are basically around not liking "backend" for
> > this purpose. When I think of a backend, I'm thinking about the
> > implementation details (like the refs backend being files or reftable)
> > and not multiple distinct locations that have their own objects.
> 
> Yup, odb_backend_files (aot odb_backend_redis) or something?

Yeah, that was my vision indeed. I think it works equally well though in case we name this `odb_alternate`. The benefit of the "alternate" terminology is that we already use it and it's almost a perfect fit, and it gives the reader a hint that we may have multiple alternates. On the other hand, `odb_backend` sounds as if there would only be a single backend for a `struct object_database`.

So Stolee caused me to reconsider and favor `odb_alternate`. But in the end I guess that both names would work alright.

Show 27 quoted lines

> >  * 'struct object_directory' could be renamed to 'struct odb_shard' or
> >    'struct odb_slice' or similar. I may even recommend 'odb_partition'
> >    though that does imply some disjointness that is not guaranteed (an
> >    object can exist in multiple parts).
> >
> >  * In the event that we create multiple implementations for storing
> >    objects, then a 'struct odb_shard' could point to a backend to help
> >    find the appropriate methods for interacting with its storage.
> 
> Hmph, I do not have strong opinions, but I consider it an
> implementation detail of one particular backend, namely, the
> filesystem based backend, that it can link together multiple
> object_directory instances and present them as if they form a single
> object database, just like all files within a single object_directory
> form an illusion of a single object database (aka key-value store) even
> though some objects are stored in individual loose object files while
> many others are packed in a single packfile.
> 
> I did not expect you would want to go to the world where a single
> "shard" consists of an object_directory backed by the filesystem and
> some other more database-y backend.  It is an interesting idea, but
> we'd need to worry about many things we do not have to worry about
> right now.  E.g. what do the precedence rules among different
> components within a single "shard" look like?  How do we express "in
> this repository, local filesystem-backed piece is consulted first,
> and then check this piece backed by low-cost but high-latency
> storage backend"?

Well, in fact I want to design this from the start so that you can mix and match different backends. I think it falls out naturally from the design if an alternate can be backed by anything, and it has a lot of very interesting features.

Furthermore, it would cause a bunch of problems if we _didn't_ allow for this, at least for hosting providers:

  - Migrations would now need to be atomic across fork networks where
    all forks need to be migrated at once so that we don't mix backends.

  - Migrations in general would be a pain if we had to do an atomic
    migration even for a single object directory. With mixed backends we
    can already make a partially-migrated backend available while the
    old backend is still in use.

  - High-latency storage backends may work well for binary files, but
    not for smallish text files.

This all of course still needs to be hashed out. I do want to send an RFC document to the mailing list soonish, probably in the first half of the Git 2.51 release cycle, so that we can discuss where to go.

Show 6 quoted lines

> > I do mention that the rename of the object-store.[c|h] files may be
> > unnecessary, or perhaps could be delayed until this series is merged
> > and the collateral is calmed.
> 
> Right now, merge-fix needed against all other topics in flight look
> like this, in order to merge it to 'seen'.

Okay. In that case I'll keep that patch for now.

Patrick