Re: [Summit topic] The state of getting a reftable backend working in git.git

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Oct 25, 2021, 22:09 UTC
Message-ID: <211026.86wnm021ih.gmgdl@evledraar.gmail.com>
In-Reply-To: <CAFQ2z_NBOC5sDSL6AjCe-5mPVhU1B_guJEsHwVT3=AK1aAt8UA@mail.gmail.com>

On Mon, Oct 25 2021, Han-Wen Nienhuys wrote:

Show 11 quoted lines

> On Thu, Oct 21, 2021 at 1:56 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>>
>> This session was led by Ævar Arnfjörð Bjarmason (on behalf of Han-Wen
>> Nienhuys, the driving force behind the reftable patches, who did not
>> attend the Summit). Supporting cast: Jonathan "jrnieder" Nieder, Johannes
>> "Dscho" Schindelin, Philip Oakley, Jeff "Peff" King, and Junio Hamano.
>>
>
> Thanks Ævar for doing this. I wanted to be there, but I took a much
> needed 2 week computer-less vacation .

No problem, as is perhaps clear from the notes I had to hand-wave some questions away since I didn't know about those things.

Show 30 quoted lines

>>..
>>      9.  Reftable has a set of files that go together. May want debugging tool
>>          to dump the content of a binary reftable file. But we can
>>          incrementally add those
>
>
> The patch series includes a test-tool for dumping both individual
> tables and a stack of tables. It's not super-polished, but it gets the
> job done.
>
> $ touch a ; ~/vc/git/git add a; ~/vc/git/git commit -mx
> ...
>
> $  ~/vc/git/bin-wrappers/test-tool  dump-reftable -t
> .git/reftable/0x000000000002-0x000000000002-327b23c6.ref
> ref{refs/heads/main(2) val 1 ab21c324503544acca84eb55f5ee7dce24b23e15}
> log{HEAD(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200
> 0000000000000000000000000000000000000000 =>
> ab21c324503544acca84eb55f5ee7dce24b23e15
>
> commit (initial): x
>
> }
> log{refs/heads/main(2) Han-Wen Nienhuys <hanwen@google.com> 1635188263 0200
> 0000000000000000000000000000000000000000 =>
> ab21c324503544acca84eb55f5ee7dce24b23e15
>
> commit (initial): x
>
> }

Neat.

From memory I think the more general concern Philip Oakley was also expressing (but maybe he'll chime in) could also be addressed by a tool that just un-reftable-ifies a repository.

I think such a thing would be useful, and I think we don't have that already. Isn't the files backend or reftable usage now an "init"-time setting.

It would be useful if for no other reason than to give user who are looking at a repository that's weird somehow the ability to quickly migrate 100% away from reftable, to see if it has any impact on whatever they're seeing.

I wanted to implement a "git unpack-refs" a while ago for "pack-refs", just to simulate some performance aspects of loose-refs without writing an ad-hoc "ref exploder" one-liner again.

A migration tool would surely be pretty much that, no? I.e. we'd just create a .git/refs.migrate or whatever, then hold a lock on reftable, and in-place move .git/refs{.migrate,} (along with top-level files like HEAD et al, presumably...).

Maybe there's more complexity I'm not considering than just the *.lock dance in .git/*, but if not such a tool could also convert freely between the two backends, so you could try refable out in an existing checkout.