git/list[1] front-page[2] threads[3] people[4] search[5] about
 

How do you best store structured data in git repositories?

From
SSSebastian Setzer <sebastianspublicaddress@googlemail.com>
Date
Dec 2, 2009, 21:08 UTC
Message-ID
<1259788097.3590.29.camel@nord26-amd64>

Hi, when you design a file format to store structured data, and you want to manage these files with git, how do you do this best?

I'd like to hear about best practices, experiences, links to discussions on this subject, ...

Here are some of my questions:

Do you store everything in a single file and configure git to use special diff- and merge-tools? Do you use XML for this purpose? Do you take care that the contents of your file is as stable as possible when it's saved or do you let your diff tools cope with issues like reordering, reassignment of identifiers (for example when identifiers are offsets in the file), ...?

Do you store one object/record per file (with filename=id, for example with GUID-s) and hope that git will not mess them up when it merges them?

Do you store records as directories, with very small files which contain single attributes (because records can be considered sets of key-value-pairs and the same applies to directories)? Do you configure git to do a scalar merge on non-text "attributes" (with special file extensions)?

When you don't store everything in a single, binary file: Do you use git hooks to update an index for efficient queries on your structured data? Do you update the whole index for every change? Or do you use git hashes to decide which segment of your index needs to be updated?

greetings, Sebastian

Next: Avery Pennarun
Message 1 of 7 in “How do you best store structured data in git repositories?”
  1. Sebastian SetzerDec 2, 2009
  2. Avery PennarunDec 2, 2009
  3. David AguilarDec 4, 2009
  4. Avery PennarunDec 4, 2009
  5. jamesmikedupont@googlemail.comDec 4, 2009
  6. Sebastian SetzerDec 7, 2009
  7. David AguilarDec 8, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.