Re: [PATCH 1/2] BreakingChanges: announce switch to "reftable" format

From: Patrick Steinhardt <ps@pks.im>
Date: Jul 3, 2025, 04:43 UTC
Message-ID: <aGYKdYGvCIGR_YF3@pks.im>
In-Reply-To: <aGWi1bZMlbKCgB74@fruit.crustytoothpaste.net>

On Wed, Jul 02, 2025 at 09:21:25PM +0000, brian m. carlson wrote:

Show 31 quoted lines

> On 2025-07-02 at 17:03:25, Junio C Hamano wrote:
> > Patrick Steinhardt <ps@pks.im> writes:
> > 
> > > diff --git a/Documentation/BreakingChanges.adoc b/Documentation/BreakingChanges.adoc
> > > index c6bd94986c5..c96b5319cdd 100644
> > > --- a/Documentation/BreakingChanges.adoc
> > > +++ b/Documentation/BreakingChanges.adoc
> > > @@ -118,6 +118,45 @@ Cf. <2f5de416-04ba-c23d-1e0b-83bb655829a7@zombino.com>,
> > >  <20170223155046.e7nxivfwqqoprsqj@LykOS.localdomain>,
> > >  <CA+EOSBncr=4a4d8n9xS4FNehyebpmX8JiUwCsXD47EQDE+DiUQ@mail.gmail.com>.
> > >  
> > > +* The default storage format for references in newly created repositories will
> > > +  be changed from "files" to "reftable". The "reftable" format provides
> > > +  multiple advantages over the "files" format:
> > > ++
> > > +  ** It is impossible to store two references that only differ in casing on
> > > ...
> > > +  ** Writing many references at once is slow with the "files" backend because
> > > +     every reference is created as a separate file. The "reftable" backend
> > > +     significantly outperforms the "files" backend by multiple orders of
> > > +     magnitude.
> > 
> > These list benefits of using "reftable".  Can we also add one point
> > that stresses why we want to make it the default?  Something like
> > "Having to do X once per user to make them opt-in is too cumbersome"
> > is probably good enough.
> 
> Maybe an additional line about "most people pick the default option and,
> given the information above, we think that users will have a better
> experience with reftable as the default" (especially, in my view, users
> on case-insensitive file systems).

Yup, makes sense. This is what I've queued:

    Users that get immediate benefit from the "reftable" backend could
    continue to opt-in to the "reftable" format manually by setting the
    "init.defaultRefFormat" config. But defaults matter, and we think
    that overall users will have a better experience with less
    platform-specific quirks with the new backend.

Show 15 quoted lines

> > > +A prerequisite for this change is that the ecosystem is ready to support the
> > > +"reftable" format. Most importantly, alternative implementations of Git like
> > > +JGit, libgit2 and Gitoxide need to support it.
> > 
> > ... in order for them to access the same repository.
> > 
> > How common is it to use a single repository from these multiple
> > implementations these days, I have to wonder?
> 
> Pretty common.  I know Rust's Cargo package manager uses libgit2 and I'm
> sure there are other development tools that do so.  At a previous
> employer, we had a linting tool that used libgit2 and we used
> command-line Git for normal operations.  I don't work with Java on a
> regular basis, but I expect that similar kinds of things happen there,
> especially in Java-based IDEs.

Yeah, I have hit issues with Cargo myself. JGit users should be mostly fine as it already supports reftables, but IIRC it only supported v0 of the format where there is no explicit hash function yet.

I'll try to engage with the respective communities and figure out a way forward to get reftable support landed. For libgit2 I might be able to have my team do it. For Gitoxide I plan to have a chat with Byron to figure something out. The missing support for explicit hash functions in JGit I've already mentioned to folks.

Show 28 quoted lines

> > > diff --git a/t/t0001-init.sh b/t/t0001-init.sh
> > > index f11a40811f2..e0f27484192 100755
> > > --- a/t/t0001-init.sh
> > > +++ b/t/t0001-init.sh
> > > @@ -658,6 +658,22 @@ test_expect_success 'init warns about invalid init.defaultRefFormat' '
> > >  	test_cmp expected actual
> > >  '
> > >  
> > > +test_expect_success 'default ref format' '
> > > +	test_when_finished "rm -rf refformat" &&
> > > +	(
> > > +		sane_unset GIT_DEFAULT_REF_FORMAT &&
> > > +		git init refformat
> > > +	) &&
> > > +	if test_have_prereq WITH_BREAKING_CHANGES
> > > +	then
> > > +		echo reftable >expect
> > > +	else
> > > +		echo files >expect
> > > +	fi &&
> > > +	git -C refformat rev-parse --show-ref-format >actual &&
> > > +	test_cmp expect actual
> > > +'
> 
> I might just make a recommendation here for a `default-ref-format` key
> (or some similar name) to `git version --build-options` as well.  That
> will get put in bug reports and troubleshooting output and will help
> people figure out what might be going wrong if there are any problems.

D'oh, obviously, given that I have recommended the same on your series :P

Patrick