From: René Scharfe <l.s.r@web.de>
Date: Sat, 06 Dec 2025 16:27:11 GMT
Subject: Re: Git for structured data
Message-ID: <be503e5e-43e7-41e3-afbe-265cd3da0636@web.de>
In-Reply-To: <aTMNdQ_NHTVPtwG8@air>

On 12/5/25 5:51 PM, Cedric Sodhi wrote:
> 
> Why can't we have structured, version controlled data?
> 
> In recent years I've repeatedly struck cases where exactly that was
> needed. For amounts of data which are comparable to what you
> typically version with git; only structured. Without workarounds,
> either structure (table schemas) or versioning (Git) had to be
> sacrificed. Which is disappointing, in my opinion, seen how this
> only hinges on the type of source Git would have to read the data
> from.
> 
> I'd like to ask your opinion, on what you think is the most
> promising approach to unify structure and version control with Git.
> Currently, I can think of two, kind of complementary options:
> 
> A) Map structured data into a filesystem, possibly through FUSE,
> then version control that with Git. Pros: Can mix non-structured
> data and structured data. Cons: Expect terrible performance
> 
> B) Abstract Git's data backend to allow Git to read directly from
> databases Pros: Perhaps reasonable performance Cons: Additional
> changes to Git would be needed to allow mixing data.
> 
> What would you recommend?
Did you consider Data Version Control (https://dvc.org/) or Dolt
(https://github.com/dolthub/dolt)?  Not a recommendation, since I
haven't used them myself, but they match your description and call
themselves "Git for data".

René