{"thread":{"id":"38516","subject":"Git for collaboration on RDF data","startedAt":"2015-02-06T19:43:50Z","lastAt":"2016-03-11T16:34:04Z","messageCount":2,"participants":["Mark Watts","Jordan Lewis"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"255673","messageId":"CALX5DUhUO9fM4SZ6F-Qof6JgDaWCaDmw+8ba-eT7iwD7v_7H0g@mail.gmail.com","threadId":"38516","inReplyTo":null,"subject":"Git for collaboration on RDF data","fromName":"Mark Watts","fromEmail":"watts.mark2015@gmail.com","sentAt":"2015-02-06T19:43:50Z","receivedAt":"2015-02-06T19:43:50Z","isPatch":false,"sender":{"key":"watts.mark2015@gmail.com","avatar":null},"body":"I'm interested in a collaboration and change management solution for\ndata stored in pre-existing RDF data stores set behind SPARQL\nendpoints. I would like some input on my idea before I invest too much\ntime in reading about Git internals. My main question is whether\npeople more experienced in how Git works internally think that my\nproblem could be solved by using git itself or if I would be better\nserved by developing my own toolkit. These first four paragraphs are\nto summarize why I'm even thinking of this solution.\n\nI consider that externally managing the versioning of data and not\nincluding that information in the data store would greatly reduce the\nusefulness of tracking changes. For example, if multiple versions are\nexposed through the SPARQL endpoint, readers would be able to compare\nversions through querying with SPARQL rather than by referring back to\na serialized representation of the data in an adjacent repository.\nThis is most pertinent when the data store is accessed by non-human\nagents since I expect that modifying a query or two in such an agent\nis easier than adding a feature for reading from an adjacent\nrepository using a distinct set of protocols. Beyond that are the\ndangers of expecting a different set of data than you receive and how\nthat's difficult to know without cryptographic guarantees of version\ninformation.\n\nI like the idea of using Git since it has gained a wide acceptance and\ngeneral understanding, even among the people outside of the software\ndevelopment profession, who I expect will be generating most of the\ndata to be tracked. Then, when it comes to collaboration, I can see\nthat if, for example, I generate some preliminary data in my lab and I\nwant to share it in RDF, branching like in Git allows me to set off\nthis preliminary data, but make it available to peers while still\nrelating it to previously existing data.\n\nMy initial requirements for this solution are that commits and merges\nshouldn't slow in the time it takes to complete them in proportion to\nthe size of the database since I want to track stores that can grow to\nbe millions of statements and several gigabytes in size. Based on my\nexpectations of the size of data being managed, I also think that\npartial sharing of a repository would be useful, but I'm not certain\nthat this is a requirement.\n\nMy idea is to embed at least the object graph of Git in the managed\nRDF graph and to make it possible to clone the tracked portion of a\ngraph by using SPARQL queries. Blobs would correspond to named graphs\nin RDF and their hashes would be computed from a serialization of the\ngraph with \"canonicalized\" BNodes, and trees would be sets of triples\nlinking \"tree nodes\" to \"tree entry nodes\" to named graph identifiers\npaired with blob object ids. For actually manipulating the commit\ngraph, I expect either to write my own tools or to use FUSE to expose\nthe RDF graph as a file system that git can manipulate like it does\nfor normal source code repositories. I like the second option, first,\nbecause it means people can use readily available tools in a way\nanalogous to how they already use them, and, second, because it allows\nfor accessing features of Git to manipulate the RDF graph (for better\nor worse) in ways that I don't have to explicitly define. My present\nconcerns for this second option are that I don't know yet everything\nthat git does on a file system to simulate it and whether I would like\nthe RDF graph that solution generates. The advantage of the first\noption is that I know better what to expect if I go that route, and\nthe disadvantages are, essentially, the advantages of the second.\n\nAny comment or criticism is welcome.\n\n-- \nCheers,\n\nMark W.\n"},{"id":"280622","messageId":"loom.20160311T172300-746@post.gmane.org","threadId":"38516","inReplyTo":"CALX5DUhUO9fM4SZ6F-Qof6JgDaWCaDmw+8ba-eT7iwD7v_7H0g@mail.gmail.com","subject":"Re: Git for collaboration on RDF data","fromName":"Jordan Lewis","fromEmail":"lewis.1378@gmail.com","sentAt":"2016-03-11T16:34:04Z","receivedAt":"2016-03-11T16:34:04Z","isPatch":false,"sender":{"key":"lewis.1378@gmail.com","avatar":null},"body":"Hello Mark,\n\nI realize this is an old post, but I was looking into doing exactly what you are\n talking about. I did some research into JGIt to see if I could leverage any of\nthat code to create an RDF Git-like experience. I came to the conclusion that\nI would either need to extend a lot of that codebase or create my own. I was\nleaning towards creating my own solution.\n\nI was wondering what you ended up doing for your case? Did you find any\nexternal libraries to help accomplish this? Was it a completely custom\nsolution?\n\nAny pointers or advice would be greatly appreciated.\n\nCheers,\nJordan Lewis\n"}]}