git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git's database structure

From
Jon Smirl <jonsmirl@gmail.com>
Date
Sep 4, 2007, 17:30 UTC
Message-ID
<9e4733910709041030ye912369nd574a5f78d3f521b@mail.gmail.com>
In-Reply-To
<Pine.LNX.4.64.0709041816340.29009@reaper.quantumfyre.co.uk>
On 9/4/07, Julian Phillips <julian@quantumfyre.co.uk> wrote:
Show 37 quoted lines
> On Tue, 4 Sep 2007, Jon Smirl wrote:
>
> > Let's back up a little bit from "Caclulating tree node".  What are the
> > elements of git's data structures?
> >
> > Right now we have an index structure (tree nodes) integrated in to a
> > base table. Integrating indexing into the data is not normally done in
> > a database. Doing a normalization analysis like this may expose flaws
> > in the way the data is structured. Of course we may also decide to
> > leave everything the way it is.
> >
> > What about the special status of a rename? In the current model we
> > effectively have three tables.
> >
> > commit - a set of all SHAs in the commit, previous commit, comment, author, etc
> > blob - a file, permissions, etc.
> > file names - name, SHA
> >
> > The file name table is encoded as an index and it has been
> > intermingled with the commit table.
> >
> > Looking at this from a set theory angle brings up the question, do we
> > really have three tables and file names are an independent variable
> > from the blobs, or should file names be an attribute of the blob?
>
> There isn't a one-to-one mapping of file names to blobs.  The blob only
> describes the contents of the file.  In the extreme case you could have
> one blob for every single file in your tree.  For example:
>
> # git ls-tree -r HEAD
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    bar/foo
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    foo
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    foo2
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    foo3
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    foo4
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    foo5
> 100644 blob 05303ef858aeeb01ca40590dd6fe65928096ee6c    foo6

Both schemes support aliasing. In the flat scheme you would create a second blob which contains the file and the aliased path name. When the blob gets delta'd the second copy of the file will disappear.

I'm not proposing a change to data being stored in git, it is a proposal to consider the impacts of how this data has been normalized in the data store.

Show 22 quoted lines
> > How this gets structured in the db is an independent question about
> > how renames get detected on a commit. The current scheme for detecting
> > renames by comparing diffs is working fine. The question is, once we
> > detect a rename how should it be stored?
> >
> > Ignoring the performance impacts and looking at the problem from the
> > set theory view point, should:
> > the pathnames be in their own table with a row for each alias
> > the pathnames be stored as an attribute of the blob
> >
> > Both of these are the same information, we're just looking at how
> > things are normalized.
> >
> >
>
> --
> Julian
>
>   ---
> "You shouldn't make my toaster angry."
> -- Household security explained in "Johnny Quest"
>
-- 
Jon Smirl
jonsmirl@gmail.com
Previous: Julian PhillipsNext: Andreas Ericsson
Message 38 of 39 in “Git's database structure”
  1. Jon SmirlSep 4, 2007
  2. Andreas EricssonSep 4, 2007
  3. Mike HommeySep 4, 2007
  4. Andreas EricssonSep 4, 2007
  5. Jon SmirlSep 4, 2007
  6. Andreas EricssonSep 4, 2007
  7. Jeff KingSep 4, 2007
  8. David TweedSep 4, 2007
  9. Junio C HamanoSep 4, 2007
  10. Jon SmirlSep 4, 2007
  11. Andreas EricssonSep 4, 2007
  12. Jon SmirlSep 4, 2007
  13. Andreas EricssonSep 4, 2007
  14. Junio C HamanoSep 4, 2007
  15. Jon SmirlSep 4, 2007
  16. Mike HommeySep 4, 2007
  17. Reece DunnSep 4, 2007
  18. Junio C HamanoSep 4, 2007
  19. Theodore TsoSep 4, 2007
  20. Jon SmirlSep 4, 2007
  21. Andreas EricssonSep 5, 2007
  22. Jon SmirlSep 5, 2007
  23. Andreas EricssonSep 5, 2007
  24. Jon SmirlSep 5, 2007
  25. Julian PhillipsSep 5, 2007
  26. Jon SmirlSep 5, 2007
  27. Julian PhillipsSep 5, 2007
  28. Kyle MoffettSep 6, 2007
  29. Mike HommeySep 5, 2007
  30. Andreas EricssonSep 6, 2007
  31. Junio C HamanoSep 6, 2007
  32. Wincent ColaiutaSep 6, 2007
  33. Johannes SchindelinSep 6, 2007
  34. Steven GrimmSep 6, 2007
  35. Martin LanghoffSep 7, 2007
  36. Andy ParkinsSep 5, 2007
  37. Julian PhillipsSep 4, 2007
  38. Jon SmirlSep 4, 2007
  39. Andreas EricssonSep 4, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.