git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/4] Speed up git-notes show

From
MBMaarten Bosmans <mkbosmans@gmail.com>
Date
Feb 5, 2024, 20:49 UTC
Message-ID
<20240205204932.16653-1-maarten.bosmans@vortech.nl>
First time contributor here, trying my first git.git patch series.
BACKGROUND
  We have a script that runs a range of build tests for all new commits in the
  repository and adds a line to the commit note with the result from the test.
  Something along the lines of:
      occa-build-jit-gnu-cuda-develop: PASSED (<hostname>, 2024-01-01 00:00:00+01:00)
  Pretty useful to quickly check that all commits at least build, not only for
  master, but also in progress feature branches.  (a passing test suite is
  generally only required at the merge point)
    
PROBLEM
  The bash script loops over all remote refs and lists the commits newer than
  <N> days ago.  For each commit its note is read and grep'ed for an existing
  test name to see whether the build test needs to run again.  The `git note show`
  command that is in this loop nest only takes 14ms to execute, but as it is in
  a loop, those times add up.
ANALYSIS
  When asked to show a note for a specific commit, git looks up the blob hash
  for the note and executes `git show` with that hash.  That of course adds
  the child process overhead, but also causes the initialization of a lot of
  log related configuration, such as for decorations or the mailmap.  Simply
  outputting the blob directly in the main process reduces the run time by
  almost halve.

When looking through the git show implementation for useful stuff that command does that should also be done when showing a note, I could only find the `setup_pager()` call. All other git show functionality was related to showing commits or other non-blob objects.

The only thing I was not 100% sure of was the textconv_object stuff. From what I could deduce was that this is only ever used on blobs that represent files in a tree, not on blobs that represent note objects. So I did not include any textconv calls in the git notes code.

The first commit is the main one fixing performance. The other three are just eliminating some overhead I noticed when going through the git notes code.

Maarten Bosmans (4):
  notes: print note blob to stdout directly
  notes: use exisisting function stream_blob_to_fd
  notes: do not clean up right before calling die()
  notes: use strbuf_attach to take ownership of the object contents
 builtin/notes.c | 37 ++++++++++---------------------------
 1 file changed, 10 insertions(+), 27 deletions(-)
-- 
2.35.3
Next: Maarten Bosmans
Message 1 of 36 in “Speed up git-notes show”
  1. 0/4 Speed up git-notes showMaarten Bosmans, Feb 5, 2024
  2. 2/4 notes: use exisisting function stream_blob_to_fdMaarten Bosmans, Feb 5, 2024
  3. Eric SunshineFeb 5, 2024
  4. 1/4 notes: print note blob to stdout directlyMaarten Bosmans, Feb 5, 2024
  5. Junio C HamanoFeb 6, 2024
  6. Maarten BosmansFeb 6, 2024
  7. Junio C HamanoFeb 6, 2024
  8. Jeff KingFeb 13, 2024
  9. Junio C HamanoFeb 13, 2024
  10. Jeff KingFeb 15, 2024
  11. Junio C HamanoFeb 16, 2024
  12. Jeff KingFeb 17, 2024
  13. Junio C HamanoFeb 17, 2024
  14. Jeff KingFeb 17, 2024
  15. Maarten BosmansFeb 15, 2024
  16. Jeff KingFeb 15, 2024
  17. Maarten BosmansFeb 17, 2024
  18. Jeff KingFeb 20, 2024
  19. Maarten BosmansFeb 15, 2024
  20. Kristoffer HaugsbakkFeb 6, 2024
  21. 3/4 notes: do not clean up right before calling die()Maarten Bosmans, Feb 5, 2024
  22. 4/4 notes: use strbuf_attach to take ownership of the object contentsMaarten Bosmans, Feb 5, 2024
  23. Kristoffer HaugsbakkFeb 6, 2024
  24. Maarten BosmansFeb 6, 2024
  25. 0/5 Speed up git-notes showMaarten Bosmans, Feb 18, 2024
  26. 1/5 log: Move show_blob_object() to log.cMaarten Bosmans, Feb 18, 2024
  27. Junio C HamanoFeb 20, 2024
  28. Jeff KingFeb 20, 2024
  29. Junio C HamanoFeb 20, 2024
  30. Maarten BosmansFeb 20, 2024
  31. 3/5 notes: use existing function stream_blob_to_fdMaarten Bosmans, Feb 18, 2024
  32. 2/5 notes: avoid launching a child process to show a note blobMaarten Bosmans, Feb 18, 2024
  33. 5/5 notes: use strbuf_attach to take ownership of the object contentsMaarten Bosmans, Feb 18, 2024
  34. Jeff KingFeb 20, 2024
  35. Maarten BosmansFeb 20, 2024
  36. 4/5 notes: do not clean up right before calling die()Maarten Bosmans, Feb 18, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.