Re: [RFC/WIP] Pluggable reference backends

From: Shawn Pearce <spearce@spearce.org>
Date: Mar 11, 2014, 02:39 UTC
Message-ID: <CAJo=hJt6zoJ=53JNUT6fLXM+5_4Af8enE67z3Ozv4DOz1jU1Eg@mail.gmail.com>
In-Reply-To: <531E2986.8050604@alum.mit.edu>

On Mon, Mar 10, 2014 at 2:07 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:

Show 25 quoted lines

> On 03/10/2014 04:52 PM, Jeff King wrote:
>> On Mon, Mar 10, 2014 at 07:30:45AM -0700, Shawn Pearce wrote:
>>
>>>> * Store references in a SQLite database, to get correct transaction
>>>>   handling.
>>>
>>> No to SQLLite in git-core. Using it from JGit requires building
>>> SQLLite and a JNI wrapper, which makes JGit significantly less
>>> portable. I know SQLLite is pretty amazing, but implementing
>>> compatibility with it from JGit will be a big nightmare for us.
>>
>> That seems like a poor reason not to implement a pluggable feature for
>> git-core. If we implement it, then a site using only git-core can take
>> advantage of it. Sites with JGit cannot, and would use a different
>> pluggable storage mechanism that's supported by both. But if we don't
>> implement, it hurts people using only git-core, and it does not help
>> sites using JGit at all.
>
> I think it's important to distinguish between two types of backend:
>
> * Exotic backends, optimized for servers, or embedded systems, or other
> controlled environments where the person deploying Git can decide about
> the whole technology stack.  Here I say let a thousand flowers bloom.
> If user A wants to try an Oracle backend and only uses JGit, there's no
> need for him to implement the equivalent backend for git-core or libgit2.

FWIW I have been running JGit derived servers using Google Bigtable for reference storage for years. So yes in this sort of environment let people do what they think is best for them.

Show 13 quoted lines

> * Mainstream backends, intended for use by end-users on their
> workstations and notebooks.  Such backends will be pretty worthless if
> they are not supported more or less universally, because one user will
> want to use the command line and Eclipse, another Visual Studio and
> TortoiseGit, a third will use GitHub for Mac plus a bunch of shell
> scripts written by his IT department.  A backend that is not supported
> by the big three Git implementations (git-core, libgit2, and JGit) will
> probably be rejected by users.  Realistically there will be at most a
> couple of mainstream backends--in fact probably usually a single
> established one and occasionally a single next-generation one waiting
> for people to migrate slowly to it.  For mainstream backends I think it
> is important for the implementations to plan and coordinate ahead of
> time to make sure everybody's concerns are addressed.

Yes, this was my real concern. Eclipse users using EGit expect EGit to be compatible with git-core at the filesystem level so they can do something in EGit then switch to a shell and bang out a command, or run a script provided by their project or co-worker. Build systems often integrate with Git to e.g. embed `git describe` output into the binary. In mainstream use cross compatibility of the tools within a single working directory is something that I think users have come to expect.

> It sounds to me like Shawn is saying "please don't make a SQLite-based
> backend the new default git-core backend" and Peff is saying "there is
> no reason that a Git hosting service shouldn't experiment with a
> SQLite-based backend".  I see no contradiction there [1].

Yes. :-)

Show 10 quoted lines

> Also, please remember that I'm not advocating a SQLite backend or any
> other at this time.  I'm only refactoring code to open the way for
> *future* flamefests :-)
>
> Michael
>
> [1] There might of course be a technical argument about whether a
> SQLite-based backend would be SO AWESOME for end-users that switching to
> it would be worth the extra inconvenience for the JGit folks.
> Personally I'm skeptical.

If it was really that amazing, yes, we would probably support it in JGit for those that need that amazing.

But I tend to think we can (usually) find a simpler format that would provide many of the same benefits with less of the drawbacks of locking the data up into SQLLite's file format. I'm with Peff, I kind of like the fact that most of the Git data is easy to inspect by hand, or with some simple tools written in Git's source tree. Starting with "go get this other SQLLite tool first then write this code" is a lot less fun.