Re: reftable & jgit compatibility

From: Patrick Steinhardt <ps@pks.im>
Date: Apr 4, 2024, 07:20 UTC
Message-ID: <Zg5Up_kSvOmQODO3@tanuki>
In-Reply-To: <CAOw_e7Zzc2uLX0FtkJ3fB+wuNJt5piMmoYVes+ayApe8BEN3+g@mail.gmail.com>

On Thu, Apr 04, 2024 at 08:44:44AM +0200, Han-Wen Nienhuys wrote:

Show 23 quoted lines

> On Thu, Apr 4, 2024 at 8:23 AM Patrick Steinhardt <ps@pks.im> wrote:
> 
> > > I think the easiest way to make this happen is if CGit would ship a
> > > command to dump a raw reftable in a release soonish. Then JGit could
> > > use that command to cross-check that a JGit-written reftable can be
> > > read correctly by the CGit code.  By shipping just the dumper you
> > > avoid having to wait for proper reftable support to land in git.
> >
> > You do realize that "proper reftable support" has already landed, right?
> 
> I had not realized this, and that's great news!
> 
> > So you can just use Git to create a reftable-enabled repository, write
> > commits and then use JGit to access the whole repository instead of only
> > checking a single table.
> 
> For testing, it's probably easier if you can work in terms of
> individual tables (because that is where the complexity lies:
> different blocksizes, restart frequencies, with index, without index,
> with reflog, without reflog etc.), but one can create controlled
> individual tables by creating a whole repo and then compacting it.
> OTOH, this would necessitate exposing all writer options to the git
> CLI, which is maybe a bit much.

Potentially, yeah. But as you say, it's likely quite some complexity to expose this via the CLI directly. So for now, I'm going to focus on some basic interoperability tests in Git that act on the repository level. We can build on that and expand them as required when the need arises.

Different blocksizes is definitely a bit of a sore spot right now. I do plan to expose write options via Git config options in the future, e.g. something like "reftable.blockSize" or "reftable.restartCount". But for all I know the CGit reftable library doesn't yet play nice with block sizes other than 4k.

I didn't yet want to introduce configs which are specific to reftables in the first release of Git with the reftable backend, so I pushed this issue further down. I do plan to work on that in the next release cycle though.

Patrick