git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCHv5 00/14] git notes

From
Johan Herland <johan@herland.net>
Date
Sep 12, 2009, 15:50 UTC
Message-ID
<200909121750.00733.johan@herland.net>
In-Reply-To
<200909080512.34634.johan@herland.net>
On Tuesday 08 September 2009, Johan Herland wrote:
Show 22 quoted lines
> Algorithm / Notes tree   git log -n10 (x100)   git log --all
> ------------------------------------------------------------
> before / no-notes              4.78s              63.90s
> before / no-fanout            56.85s              65.69s
> 
> 16tree / no-notes              4.77s              64.18s
> 16tree / no-fanout            30.35s              65.39s
> 16tree / 2_38                  5.57s              65.42s
> 16tree / 2_2_36                5.19s              65.76s
> 
> flexible / no-notes            4.78s              63.91s
> flexible / no-fanout          30.34s              65.57s
> flexible / 2_38                5.57s              65.46s
> flexible / 2_2_36              5.18s              65.72s
> flexible / ym                  5.13s              65.66s
> flexible / ym_2_38             5.08s              65.63s
> flexible / ymd                 5.30s              65.45s
> flexible / ymd_2_38            5.29s              65.90s
> flexible / y_m                 5.11s              65.72s
> flexible / y_m_2_38            5.08s              65.67s
> flexible / y_m_d               5.06s              65.50s
> flexible / y_m_d_2_38          5.07s              65.79s

Ok, I have been pondering this back and forth, and I'm not sure what to think. It seems allowing (not mandating) date-based fanout gives a slight runtime advantage if used correctly, but I'm not sure the slight runtime improvement is worth the added code complexity and worse maintainability. I'm starting to lean against SHA1-based fanout being "good enough".

But when we look at the memory consumption, it's clear that SHA1-based fanout loses out (because you cannot throw away subtrees without fear that they will be needed again soon). Then again, memory consumption has not been the major focus of the git project, and 14 MB (for holding all ~157000 notes in the kernel repo example) is not excessive for an average desktop computer.

Shawn, do you have any additional defence for the date-based fanout? Are there untested reasonable scenarios that would show the benefits of date- based fanout? How does the plan for notes usage in your code-review thingy compare to my test scenario?

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Previous: Johan HerlandNext: Shawn O. Pearce
Message 31 of 58 in “[PATCHv5 00/14] git notes”
  1. Johan HerlandSep 8, 2009
  2. 01/14 Introduce commit notesJohan Herland, Sep 8, 2009
  3. 02/14 Add a script to edit/inspect notesJohan Herland, Sep 8, 2009
  4. 03/14 Speed up git notes lookupJohan Herland, Sep 8, 2009
  5. 04/14 Add an expensive test for git-notesJohan Herland, Sep 8, 2009
  6. 05/14 Teach "-m <msg>" and "-F <file>" to "git notes edit"Johan Herland, Sep 8, 2009
  7. 06/14 fast-import: Add support for importing commit notesJohan Herland, Sep 8, 2009
  8. 07/14 t3302-notes-index-expensive: Speed up create_repo()Johan Herland, Sep 8, 2009
  9. 08/14 Add flags to get_commit_notes() to control the format of the note stringJohan Herland, Sep 8, 2009
  10. 09/14 Add '%N'-format for pretty-printing commit notesJohan Herland, Sep 8, 2009
  11. 10/14 Teach notes code to free its internal data structures on request.Johan Herland, Sep 8, 2009
  12. 11/14 Teach the notes lookup code to parse notes trees with various fanout schemesJohan Herland, Sep 8, 2009
  13. 12/14 Selftests verifying semantics when loading notes trees with various fanoutsJohan Herland, Sep 8, 2009
  14. 13/14 Allow flexible organization of notes trees, using both commit date and SHA1Johan Herland, Sep 8, 2009
  15. 14/14 Add test cases for date-based fanoutsJohan Herland, Sep 8, 2009
  16. Johan HerlandSep 8, 2009
  17. Junio C HamanoSep 8, 2009
  18. Johan HerlandSep 8, 2009
  19. Johannes SchindelinSep 8, 2009
  20. Johan HerlandSep 8, 2009
  21. Johannes SchindelinSep 8, 2009
  22. Johan HerlandSep 8, 2009
  23. Stephen R. van den BergSep 10, 2009
  24. Johan HerlandSep 10, 2009
  25. Junio C HamanoSep 8, 2009
  26. Shawn O. PearceSep 8, 2009
  27. Sverre RabbelierSep 8, 2009
  28. Shawn O. PearceSep 8, 2009
  29. Sverre RabbelierSep 8, 2009
  30. Johan HerlandSep 8, 2009
  31. Johan HerlandSep 12, 2009
  32. Shawn O. PearceSep 12, 2009
  33. Johan HerlandSep 12, 2009
  34. Geert BoschSep 10, 2009
  35. Michael J GruberSep 10, 2009
  36. Geert BoschSep 10, 2009
  37. Junio C HamanoSep 12, 2009
  38. Johan HerlandSep 12, 2009
  39. 00/14 git notesJohan Herland, Sep 12, 2009
  40. 01/14 Introduce commit notesJohan Herland, Sep 12, 2009
  41. 02/14 Add a script to edit/inspect notesJohan Herland, Sep 12, 2009
  42. 03/14 Speed up git notes lookupJohan Herland, Sep 12, 2009
  43. 04/14 Add an expensive test for git-notesJohan Herland, Sep 12, 2009
  44. 05/14 Teach "-m <msg>" and "-F <file>" to "git notes edit"Johan Herland, Sep 12, 2009
  45. 06/14 fast-import: Add support for importing commit notesJohan Herland, Sep 12, 2009
  46. 07/14 t3302-notes-index-expensive: Speed up create_repo()Johan Herland, Sep 12, 2009
  47. 08/14 Add flags to get_commit_notes() to control the format of the note stringJohan Herland, Sep 12, 2009
  48. 09/14 Add '%N'-format for pretty-printing commit notesJohan Herland, Sep 12, 2009
  49. 10/14 Teach notes code to free its internal data structures on request.Johan Herland, Sep 12, 2009
  50. Junio C HamanoSep 12, 2009
  51. Johan HerlandSep 12, 2009
  52. 11/14 Teach the notes lookup code to parse notes trees with various fanout schemesJohan Herland, Sep 12, 2009
  53. 12/14 Selftests verifying semantics when loading notes trees with various fanoutsJohan Herland, Sep 12, 2009
  54. 13/14 Allow flexible organization of notes trees, using both commit date and SHA1Johan Herland, Sep 12, 2009
  55. Junio C HamanoSep 12, 2009
  56. Johan HerlandSep 12, 2009
  57. Junio C HamanoSep 12, 2009
  58. 14/14 Add test cases for various date-based fanoutsJohan Herland, Sep 12, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.