git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/4] Notes reloaded

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Dec 19, 2008, 23:34 UTC
Message-ID
<alpine.DEB.1.00.0812192347261.30769@pacific.mpi-cbg.de>
In-Reply-To
<20081216085108.GA3031@coredump.intra.peff.net>
Hi,
On Tue, 16 Dec 2008, Jeff King wrote:
>   Johannes Schindelin's notes proposal (which is more or less 
>   the current proposal, but I think the on-disk notes index was not 
>   well liked): 
>   http://thread.gmane.org/gmane.comp.version-control.git/52598

I redid the benchmark (this time with a bit beefier machine), just comparing no notes with David's/Peff's idea:

-- snip --
$ GIT_NOTES_TIMING_TESTS=1 sh t3302-notes-index-expensive.sh -i -v
Initialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/.git/
* expecting success: create_repo 10
Initialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/10/.git/
*   ok 1: setup 10
* expecting success: test_notes 10
*   ok 2: notes work
* expecting success: time_notes 100
no-notes
0.08user 0.10system 0:00.18elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+58926minor)pagefaults 0swaps
notes
0.14user 0.07system 0:00.54elapsed 38%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+60319minor)pagefaults 0swaps
*   ok 3: notes timing
* expecting success: create_repo 100
Initialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/100/.git/
*   ok 1: setup 100
* expecting success: test_notes 100
*   ok 2: notes work
* expecting success: time_notes 100
no-notes
0.23user 0.21system 0:00.45elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+68043minor)pagefaults 0swaps
notes
0.38user 0.21system 0:00.59elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+78829minor)pagefaults 0swaps
*   ok 3: notes timing
* expecting success: create_repo 1000
Initialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/1000/.git/
*   ok 1: setup 1000
* expecting success: test_notes 1000
*   ok 2: notes work
* expecting success: time_notes 100
no-notes
2.06user 0.95system 0:04.26elapsed 70%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+159115minor)pagefaults 0swaps
notes
2.83user 1.54system 0:04.38elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+267416minor)pagefaults 0swaps
*   ok 3: notes timing
* expecting success: create_repo 10000
Initialized empty Git repository in /home/gitte/git/t/trash directory.t3302-notes-index-expensive/10000/.git/
*   ok 1: setup 10000
* expecting success: test_notes 10000
*   ok 2: notes work
* expecting success: time_notes 100
no-notes
20.46user 7.63system 0:28.30elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1083378minor)pagefaults 0swaps
notes
28.78user 13.74system 0:42.85elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+2240296minor)pagefaults 0swaps
*   ok 3: notes timing
* passed all 0 test(s)
-- snap --

Keep in mind that the tests run "git log" 99 times, and show the accumulated time.

So it seems that an increase of roughly 40% in the user time, and roughly 70% in the system time is the price to have notes associated with every single commit.

Note that in that very same repository, a single "git show" goes from

0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+561minor)pagefaults 0swaps

to this:

0.03user 0.02system 0:00.04elapsed 113%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+2294minor)pagefaults 0swaps

(In another run, it only used 90%CPU)

That's not too shabby, given that Git needs to unpack double the number of objects in this test when using notes vs. no notes.

For comparison, the numbers back then were something like 10% in user time with a penalty of an extraordinary magnitude everytime the notes are updated: around 800%.

Note: all these numbers are worst-case numbers, i.e. every commit has one 
note.

To be frank, I do not completely understand why the numbers are that high. I would have understood an increase roughly 4 seconds for reading the quite large tree 99 times, and then the same ~0.20 seconds back then. Maybe I made a huge mistake when implementing the thing.

And BTW, my code does not yet handle the case when refs/notes/commits:$commit is a tree instead of a blob. That is left as an exercise to the reader.

Johannes Schindelin (4):
  Introduce commit notes
  Add a script to edit/inspect notes
  Speed up git notes lookup
  Add an expensive test for git-notes
 .gitignore                       |    1 +
 Documentation/config.txt         |   15 ++++
 Documentation/git-notes.txt      |   46 +++++++++++
 Makefile                         |    3 +
 cache.h                          |    3 +
 command-list.txt                 |    1 +
 commit.c                         |    1 +
 config.c                         |    5 +
 environment.c                    |    1 +
 git-notes.sh                     |   65 +++++++++++++++
 notes.c                          |  159 ++++++++++++++++++++++++++++++++++++++
 notes.h                          |    7 ++
 pretty.c                         |    5 +
 t/t3301-notes.sh                 |   65 +++++++++++++++
 t/t3302-notes-index-expensive.sh |   98 +++++++++++++++++++++++
 15 files changed, 475 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-notes.txt
 create mode 100755 git-notes.sh
 create mode 100644 notes.c
 create mode 100644 notes.h
 create mode 100755 t/t3301-notes.sh
 create mode 100755 t/t3302-notes-index-expensive.sh
Previous: Stephan BeyerNext: Johannes Schindelin
Message 14 of 41 in “Git Notes idea.”
  1. Govind SalinasDec 16, 2008
  2. Jeff KingDec 16, 2008
  3. Jeff KingDec 16, 2008
  4. Govind SalinasDec 16, 2008
  5. Johannes SchindelinDec 16, 2008
  6. Jeff KingDec 17, 2008
  7. Petr BaudisDec 17, 2008
  8. Jeff KingDec 17, 2008
  9. Govind SalinasDec 17, 2008
  10. Jeff KingDec 18, 2008
  11. rebasing commits that have notes, was Re: Git Notes idea.Johannes Schindelin, Dec 17, 2008
  12. Johan HerlandDec 17, 2008
  13. Stephan BeyerDec 17, 2008
  14. 0/4 Notes reloadedJohannes Schindelin, Dec 19, 2008
  15. 1/4 Introduce commit notesJohannes Schindelin, Dec 19, 2008
  16. Jeff KingDec 20, 2008
  17. Robin RosenbergDec 20, 2008
  18. Jeff KingDec 20, 2008
  19. Junio C HamanoDec 20, 2008
  20. Jeff KingDec 20, 2008
  21. Junio C HamanoDec 20, 2008
  22. 0/4 Notes, reloadedJohannes Schindelin, Dec 20, 2008
  23. 1/4 Introduce commit notesJohannes Schindelin, Dec 20, 2008
  24. 2/4 Add a script to edit/inspect notesJohannes Schindelin, Dec 20, 2008
  25. 3/4 Speed up git notes lookupJohannes Schindelin, Dec 20, 2008
  26. 4/4 Add an expensive test for git-notesJohannes Schindelin, Dec 20, 2008
  27. 2/4 Add a script to edit/inspect notesJohannes Schindelin, Dec 19, 2008
  28. 3/4 Speed up git notes lookupJohannes Schindelin, Dec 19, 2008
  29. 4/4 Add an expensive test for git-notesJohannes Schindelin, Dec 19, 2008
  30. Boyd Stephen Smith Jr.Dec 19, 2008
  31. Johannes SchindelinDec 20, 2008
  32. Jeff KingDec 17, 2008
  33. Johannes SchindelinDec 17, 2008
  34. Junio C HamanoDec 17, 2008
  35. Johannes SchindelinDec 18, 2008
  36. Govind SalinasDec 19, 2008
  37. Govind SalinasDec 19, 2008
  38. Govind SalinasDec 19, 2008
  39. Jeff KingDec 19, 2008
  40. Govind SalinasDec 19, 2008
  41. Jeff KingDec 20, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.