git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/9] Trace2 stopwatch timers and global counters

From
Derrick Stolee <stolee@gmail.com>
Date
Dec 21, 2021, 14:51 UTC
Message-ID
<92923ca0-fbf9-e763-5735-214f3ad0cc3a@gmail.com>
In-Reply-To
<pull.1099.git.1640012469.gitgitgadget@gmail.com>
On 12/20/2021 10:01 AM, Jeff Hostetler via GitGitGadget wrote:
> Extend Trace2 to provide multiple "stopwatch timers" and "global counters".
Show 24 quoted lines
>  3. Rationale
> 
> Timers and counters are an alternative to the existing "region" and "data"
> events. The latter are intended to trace the major flow (or phases) of the
> program and possibly capture the amount of work performed within a loop, for
> example. The former are offered as a way to measure activity that is not
> localized, such as the time spent in zlib or lstat, which may be called from
> many different parts of the program.
> 
> There are currently several places in the Git code where we want to measure
> such activity -- changed-path Bloom filter stats, topo-walk commit counts,
> and tree-walk counts and max-depths. A conversation in [1] suggested that we
> should investigate a more general mechanism to collect stats so that each
> instance doesn't need to recreate their own atexit handling mechanism.
> 
> This is an attempt to address that and let us easily explore other areas in
> the future.
> 
> This patch series does not attempt to refactor those three instances to use
> the new timers and counters. That should be a separate effort -- in part
> because we may want to retool them rather than just translate them. For
> example, rather than just translating the existing four Bloom filter counts
> (in revision.c) into Trace2 counters, we may instead want to have a "happy
> path timer" and a "sad path timer" if that would provide more insight.

I'm excited for these API features. It might be nice to have an RFC- quality series demonstrating how these examples could work with the new API. Makes sense to delay in case there were recommended changes to the API from review in this v1.

I also like your attention to thread contexts. I think these timers would be very interesting to use in parallel checkout. CC'ing Matheus for his thoughts on where he would want timer summaries for that feature. I would probably want the per-thread summary to know if we are blocked on one really long thread while the others finish quickly. Within that: what are the things causing us to be slow? Is it zlib? Is it lstat()?

Thanks, -Stolee

Previous: Jeff HostetlerNext: Matheus Tavares
Message 37 of 55 in “Trace2 stopwatch timers and global counters”
  1. 0/9 Trace2 stopwatch timers and global countersJeff Hostetler via GitGitGadget, Dec 20, 2021
  2. 1/9 trace2: use size_t alloc,nr_open_regions in tr2tls_thread_ctxJeff Hostetler via GitGitGadget, Dec 20, 2021
  3. 2/9 trace2: convert tr2tls_thread_ctx.thread_name from strbuf to char*Jeff Hostetler via GitGitGadget, Dec 20, 2021
  4. Ævar Arnfjörð BjarmasonDec 20, 2021
  5. Jeff HostetlerDec 20, 2021
  6. Ævar Arnfjörð BjarmasonDec 20, 2021
  7. Jeff HostetlerDec 22, 2021
  8. Junio C HamanoDec 21, 2021
  9. Junio C HamanoDec 21, 2021
  10. Jeff HostetlerDec 22, 2021
  11. Junio C HamanoDec 22, 2021
  12. 3/9 trace2: defer free of TLS CTX until program exit.Jeff Hostetler via GitGitGadget, Dec 20, 2021
  13. Junio C HamanoDec 21, 2021
  14. Jeff HostetlerDec 22, 2021
  15. Junio C HamanoDec 22, 2021
  16. Jeff HostetlerDec 22, 2021
  17. Johannes SixtDec 23, 2021
  18. Junio C HamanoDec 23, 2021
  19. Jeff HostetlerDec 27, 2021
  20. 4/9 trace2: add thread-name override to event targetJeff Hostetler via GitGitGadget, Dec 20, 2021
  21. 5/9 trace2: add thread-name override to perf targetJeff Hostetler via GitGitGadget, Dec 20, 2021
  22. 6/9 trace2: add timer events to perf and event target formatsJeff Hostetler via GitGitGadget, Dec 20, 2021
  23. Ævar Arnfjörð BjarmasonDec 20, 2021
  24. Jeff HostetlerDec 20, 2021
  25. Derrick StoleeDec 21, 2021
  26. 8/9 trace2: add counter events to perf and event target formatsJeff Hostetler via GitGitGadget, Dec 20, 2021
  27. Ævar Arnfjörð BjarmasonDec 20, 2021
  28. Jeff HostetlerDec 22, 2021
  29. 7/9 trace2: add stopwatch timersJeff Hostetler via GitGitGadget, Dec 20, 2021
  30. Ævar Arnfjörð BjarmasonDec 20, 2021
  31. Jeff HostetlerDec 22, 2021
  32. Derrick StoleeDec 21, 2021
  33. Jeff HostetlerDec 22, 2021
  34. 9/9 trace2: add global countersJeff Hostetler via GitGitGadget, Dec 20, 2021
  35. Ævar Arnfjörð BjarmasonDec 20, 2021
  36. Jeff HostetlerDec 22, 2021
  37. Derrick StoleeDec 21, 2021
  38. Matheus TavaresDec 21, 2021
  39. 0/9 Trace2 stopwatch timers and global countersJeff Hostetler via GitGitGadget, Dec 28, 2021
  40. 1/9 trace2: use size_t alloc,nr_open_regions in tr2tls_thread_ctxJeff Hostetler via GitGitGadget, Dec 28, 2021
  41. Ævar Arnfjörð BjarmasonDec 29, 2021
  42. 2/9 trace2: convert tr2tls_thread_ctx.thread_name from strbuf to flex arrayJeff Hostetler via GitGitGadget, Dec 28, 2021
  43. Ævar Arnfjörð BjarmasonDec 29, 2021
  44. Jeff HostetlerDec 29, 2021
  45. 3/9 trace2: defer free of thread local storage until program exit.Jeff Hostetler via GitGitGadget, Dec 28, 2021
  46. 4/9 trace2: add thread-name override to event targetJeff Hostetler via GitGitGadget, Dec 28, 2021
  47. 7/9 trace2: add stopwatch timersJeff Hostetler via GitGitGadget, Dec 28, 2021
  48. 5/9 trace2: add thread-name override to perf targetJeff Hostetler via GitGitGadget, Dec 28, 2021
  49. Ævar Arnfjörð BjarmasonDec 29, 2021
  50. Jeff HostetlerDec 29, 2021
  51. 6/9 trace2: add timer events to perf and event target formatsJeff Hostetler via GitGitGadget, Dec 28, 2021
  52. 8/9 trace2: add counter events to perf and event target formatsJeff Hostetler via GitGitGadget, Dec 28, 2021
  53. 9/9 trace2: add global countersJeff Hostetler via GitGitGadget, Dec 28, 2021
  54. Ævar Arnfjörð BjarmasonDec 29, 2021
  55. Jeff HostetlerDec 30, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.