git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git's database structure

From
Andreas Ericsson <ae@op5.se>
Date
Sep 4, 2007, 16:29 UTC
Message-ID
<46DD87FE.7020007@op5.se>
In-Reply-To
<9e4733910709040919u3d252b91s2785ed4d20086c88@mail.gmail.com>
Jon Smirl wrote:
Show 40 quoted lines
> On 9/4/07, Andreas Ericsson <ae@op5.se> wrote:
>> Jon Smirl wrote:
>>> Let's back up a little bit from "Caclulating tree node".  What are the
>>> elements of git's data structures?
>>>
>>> Right now we have an index structure (tree nodes) integrated in to a
>>> base table. Integrating indexing into the data is not normally done in
>>> a database. Doing a normalization analysis like this may expose flaws
>>> in the way the data is structured. Of course we may also decide to
>>> leave everything the way it is.
>>>
>>> What about the special status of a rename? In the current model we
>>> effectively have three tables.
>>>
>>> commit - a set of all SHAs in the commit, previous commit, comment, author, etc
>>> blob - a file, permissions, etc.
>>> file names - name, SHA
>> commit - SHA1 of its parent(s) and its root-tree, along with
>>          author info and a free-form field
>> blob - content addressable by *multiple trees*
>> file names - List of path-names inside a tree object.
>>
>>
>> To draw some sort of relationship model here, you'd have
>>
>> commit 1<->M roottree
>> tree M<->M tree
>> tree M<->M blob
> 
> By introducing tree nodes you have blended a specific indexing scheme
> into the data. There are many other ways the path names could be
> indexed hash tables, binary trees, etc.
> 
> This problem exists in files systems. Since the path names have been
> encoded into the directory structures there is no way to query
> something like "all files created yesterday" from a file system
> without building another mapping table or a brute force search. I keep
> using Google as an example, Google is indexing hierarchical URLs but
> they do not use a hierarchical index to do it.
> 

Pathnames are by far the most common search-/delimiting criteria for git though, so I fail to see why this is a problem for you.

> Databases keep the knowledge of how things are indexed out of the
> data. A data structure analysis of git should remove the blended index
> and start from the set theory.
> 

Why? This is the core of the problem, really. You haven't specified a single, real-life reason *why* it should be any other way than it already is. It sounds a bit to me as if you've been to a really inspiring seminar about "how database-like things *should* be done" and then decided to go berserk on your favourite database-like thing, which is git.

Code and benchmarks or bust. In the meantime, I'll settle for a recount of what problems you're having with the current layout, or what gains you're hoping to achieve with the new one. As it's the 3rd time I'm asking, this'll be the last.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
Previous: Jon SmirlNext: Jeff King
Message 6 of 39 in “Git's database structure”
  1. Jon SmirlSep 4, 2007
  2. Andreas EricssonSep 4, 2007
  3. Mike HommeySep 4, 2007
  4. Andreas EricssonSep 4, 2007
  5. Jon SmirlSep 4, 2007
  6. Andreas EricssonSep 4, 2007
  7. Jeff KingSep 4, 2007
  8. David TweedSep 4, 2007
  9. Junio C HamanoSep 4, 2007
  10. Jon SmirlSep 4, 2007
  11. Andreas EricssonSep 4, 2007
  12. Jon SmirlSep 4, 2007
  13. Andreas EricssonSep 4, 2007
  14. Junio C HamanoSep 4, 2007
  15. Jon SmirlSep 4, 2007
  16. Mike HommeySep 4, 2007
  17. Reece DunnSep 4, 2007
  18. Junio C HamanoSep 4, 2007
  19. Theodore TsoSep 4, 2007
  20. Jon SmirlSep 4, 2007
  21. Andreas EricssonSep 5, 2007
  22. Jon SmirlSep 5, 2007
  23. Andreas EricssonSep 5, 2007
  24. Jon SmirlSep 5, 2007
  25. Julian PhillipsSep 5, 2007
  26. Jon SmirlSep 5, 2007
  27. Julian PhillipsSep 5, 2007
  28. Kyle MoffettSep 6, 2007
  29. Mike HommeySep 5, 2007
  30. Andreas EricssonSep 6, 2007
  31. Junio C HamanoSep 6, 2007
  32. Wincent ColaiutaSep 6, 2007
  33. Johannes SchindelinSep 6, 2007
  34. Steven GrimmSep 6, 2007
  35. Martin LanghoffSep 7, 2007
  36. Andy ParkinsSep 5, 2007
  37. Julian PhillipsSep 4, 2007
  38. Jon SmirlSep 4, 2007
  39. Andreas EricssonSep 4, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.