git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git for structured data

From
René Scharfe <l.s.r@web.de>
Date
Dec 6, 2025, 16:27 UTC
Message-ID
<be503e5e-43e7-41e3-afbe-265cd3da0636@web.de>
In-Reply-To
<aTMNdQ_NHTVPtwG8@air>
On 12/5/25 5:51 PM, Cedric Sodhi wrote:
Show 24 quoted lines
> 
> Why can't we have structured, version controlled data?
> 
> In recent years I've repeatedly struck cases where exactly that was
> needed. For amounts of data which are comparable to what you
> typically version with git; only structured. Without workarounds,
> either structure (table schemas) or versioning (Git) had to be
> sacrificed. Which is disappointing, in my opinion, seen how this
> only hinges on the type of source Git would have to read the data
> from.
> 
> I'd like to ask your opinion, on what you think is the most
> promising approach to unify structure and version control with Git.
> Currently, I can think of two, kind of complementary options:
> 
> A) Map structured data into a filesystem, possibly through FUSE,
> then version control that with Git. Pros: Can mix non-structured
> data and structured data. Cons: Expect terrible performance
> 
> B) Abstract Git's data backend to allow Git to read directly from
> databases Pros: Perhaps reasonable performance Cons: Additional
> changes to Git would be needed to allow mixing data.
> 
> What would you recommend?

Did you consider Data Version Control (https://dvc.org/) or Dolt (https://github.com/dolthub/dolt)? Not a recommendation, since I haven't used them myself, but they match your description and call themselves "Git for data".

René
Previous: Cedric SodhiNext: Cedric Sodhi
Message 2 of 6 in “Git for structured data”
  1. Cedric SodhiDec 5, 2025
  2. René ScharfeDec 6, 2025
  3. Cedric SodhiDec 6, 2025
  4. Christian CouderDec 6, 2025
  5. Simon RichterDec 7, 2025
  6. Cedric SodhiDec 7, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.