Re: How do you best store structured data in git repositories?

 On Thu, Dec 3, 2009 at 7:14 PM, David Aguilar <davvid@gmail.com> wrote:
 > JSON's not too bad for data structures and is known to
 > be friendly to XML expats.
 >
 > http://json.org/

I am currently working on two projects in this direction :

1. mediawiki on git, using mediawiki markup files. I apologise that I
have not made progress on that lately, because I have had inspiration
on my older project
2. the gcc rdf introspector, storage of the files in rdf. It is
working now with a mysql database, using the librdf mysql driver, and
running on a catalyst framework using jquery/jstree on the front end.

None of those formats are perfect, the sizing of the files is important. I am returning individual nodes in json on the cataylst server and that works to deliver the AST nodes from the compiler to the jstree front end. But these fetches to the front end should be longer and contain direct components of the fetched node. I think that a cluster of nodes should be pulled together to make a more optimal system.

here is just my two cents:

if you are using a distributed git data repository as your central repository, then think about a database page. Imagine that you would have pages of data being retrieved and compared.

Would it not make sense to split your pages something that would be swapped into memory directly, or with very little parsing, and then used?

So, in effect, you would design the sizing of the pages and the page contents around the usage model, since git is a low level storage system.

I dont know what would be available if some database manager system like mysql or postgres could be taught to store table pages in git.

just some ideas,

mike