git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git's database structure

From
Mike Hommey <mh@glandium.org>
Date
Sep 4, 2007, 18:04 UTC
Message-ID
<20070904180429.GA626@glandium.org>
In-Reply-To
<9e4733910709041044r71264346n341d178565dd0521@mail.gmail.com>
On Tue, Sep 04, 2007 at 01:44:47PM -0400, Jon Smirl <jonsmirl@gmail.com> wrote:
Show 34 quoted lines
> On 9/4/07, Junio C Hamano <gitster@pobox.com> wrote:
> > "Jon Smirl" <jonsmirl@gmail.com> writes:
> >
> > > Another way of looking at the problem,
> > >
> > > Let's build a full-text index for git. You put a string into the index
> > > and it returns the SHAs of all the file nodes that contain the string.
> > > How do I recover the path names of these SHAs?
> >
> > That question does not make much sense without specifying "which
> > commit's path you are talking about".
> >
> > If you want to encode such "contextual information" in addition
> > to "contents", you could do so, but you essentially need to
> > record commit + pathname + mode bits + contents as "blob" and
> > hash that to come up with a name.
> 
> I left the details out of the full-text example to make it more
> obvious that we can't recover the path names.
> 
> Doing this type of analysis may point out that even more fields are
> missing from the blob table such as commit id.
> 
> The current data store design is not very flexible. Databases solved
> the flexibility problem long ago. I'm just wondering if we should
> steal some good ideas out of the database world and apply them to git.
> Ten years from now we may have 100GB git databases and really wish we
> had more flexible ways of querying them.
> 
> The reason databases don't encode the fields into the index is that
> you can only have a single index on the table if you do that.
> Databases do sometimes duplicate the field in both the index and the
> table. Databases also have the property that indexes are just a cache
> and can be dropped at any time.

The big difference between a database and git is that a database is a general purpose tool. git has a much more restricted scope. As such, it doesn't need *that much* flexibility.

Mike
Previous: Jon SmirlNext: Reece Dunn
Message 16 of 39 in “Git's database structure”
  1. Jon SmirlSep 4, 2007
  2. Andreas EricssonSep 4, 2007
  3. Mike HommeySep 4, 2007
  4. Andreas EricssonSep 4, 2007
  5. Jon SmirlSep 4, 2007
  6. Andreas EricssonSep 4, 2007
  7. Jeff KingSep 4, 2007
  8. David TweedSep 4, 2007
  9. Junio C HamanoSep 4, 2007
  10. Jon SmirlSep 4, 2007
  11. Andreas EricssonSep 4, 2007
  12. Jon SmirlSep 4, 2007
  13. Andreas EricssonSep 4, 2007
  14. Junio C HamanoSep 4, 2007
  15. Jon SmirlSep 4, 2007
  16. Mike HommeySep 4, 2007
  17. Reece DunnSep 4, 2007
  18. Junio C HamanoSep 4, 2007
  19. Theodore TsoSep 4, 2007
  20. Jon SmirlSep 4, 2007
  21. Andreas EricssonSep 5, 2007
  22. Jon SmirlSep 5, 2007
  23. Andreas EricssonSep 5, 2007
  24. Jon SmirlSep 5, 2007
  25. Julian PhillipsSep 5, 2007
  26. Jon SmirlSep 5, 2007
  27. Julian PhillipsSep 5, 2007
  28. Kyle MoffettSep 6, 2007
  29. Mike HommeySep 5, 2007
  30. Andreas EricssonSep 6, 2007
  31. Junio C HamanoSep 6, 2007
  32. Wincent ColaiutaSep 6, 2007
  33. Johannes SchindelinSep 6, 2007
  34. Steven GrimmSep 6, 2007
  35. Martin LanghoffSep 7, 2007
  36. Andy ParkinsSep 5, 2007
  37. Julian PhillipsSep 4, 2007
  38. Jon SmirlSep 4, 2007
  39. Andreas EricssonSep 4, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.