git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 16/15] Add tracing to measure where most of the time is spent

From
Duy Nguyen <pclouds@gmail.com>
Date
Apr 20, 2016, 12:36 UTC
Message-ID
<CACsJy8A_M7pjVFea3oK82dz5h7SHJ=-eCEwUmv6St6q1YOuwEw@mail.gmail.com>
In-Reply-To
<alpine.DEB.2.20.1604201427440.2826@virtualbox>

On Wed, Apr 20, 2016 at 7:28 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 30 quoted lines
> Hi Duy,
>
> On Wed, 20 Apr 2016, Nguyễn Thái Ngọc Duy wrote:
>
>> All the known heavy code blocks are measured (except object database
>> access). This should help identify if an optimization is effective or
>> not. An unoptimized git-status would give something like below (92% of
>> time is accounted). To sum up the effort of making git scale better:
>>
>>  - read cache line is being addressed by index-helper
>>  - preload/refresh index by watchman
>>  - read packed refs by lmdb backend
>>  - diff-index by rebuilding cache-tree
>>  - read directory by untracked cache and watchman
>>  - write index by split index
>>  - name hash potientially by index-helper
>>
>> read-cache.c:2075         performance: 0.004058570 s: read cache .../index
>> preload-index.c:104       performance: 0.004419864 s: preload index
>> read-cache.c:1265         performance: 0.000185224 s: refresh index
>> refs/files-backend.c:1100 performance: 0.001935788 s: read packed refs
>> diff-lib.c:240            performance: 0.000144132 s: diff-files
>> diff-lib.c:506            performance: 0.013592000 s: diff-index
>> name-hash.c:128           performance: 0.000614177 s: initialize name hash
>> dir.c:2030                performance: 0.015814103 s: read directory
>> read-cache.c:2565         performance: 0.004052343 s: write index, changed mask = 2
>> trace.c:420               performance: 0.048365509 s: git command: './git' 'status'
>
> Thank you for doing this! It will be *highly* valuable to get the
> performance on Windows where I want it to be, too.

Just to be clear, these are unoptimized numbers, no untracked cache, no split index, no index helper, and on a tiny repository that is git.git. It's mostly to show the percentage of time spent in each phase. We probably can do similar measurement and record it in the watchman patch (can't wait for lmdb to be here :).

-- 
Duy
Previous: Johannes Schindelin
Message 35 of 35 in “index-helper/watchman”
  1. 00/15 index-helper/watchmanDavid Turner, Apr 19, 2016
  2. 01/15 read-cache.c: fix constness of verify_hdr()David Turner, Apr 19, 2016
  3. 02/15 read-cache: allow to keep mmap'd memory after readingDavid Turner, Apr 19, 2016
  4. Johannes SchindelinApr 20, 2016
  5. David TurnerApr 20, 2016
  6. Duy NguyenApr 20, 2016
  7. David TurnerApr 20, 2016
  8. 03/15 index-helper: new daemon for caching index and related stuffDavid Turner, Apr 19, 2016
  9. Johannes SchindelinApr 20, 2016
  10. Duy NguyenApr 20, 2016
  11. David TurnerApr 20, 2016
  12. 04/15 index-helper: add --strictDavid Turner, Apr 19, 2016
  13. 05/15 daemonize(): set a flag before exiting the main processDavid Turner, Apr 19, 2016
  14. 06/15 index-helper: add --detachDavid Turner, Apr 19, 2016
  15. Duy NguyenApr 19, 2016
  16. David TurnerApr 20, 2016
  17. Duy NguyenApr 20, 2016
  18. David TurnerApr 25, 2016
  19. 07/15 read-cache: add watchman 'WAMA' extensionDavid Turner, Apr 19, 2016
  20. 08/15 Add watchman support to reduce index refresh costDavid Turner, Apr 19, 2016
  21. 09/15 index-helper: use watchman to avoid refreshing index with lstat()David Turner, Apr 19, 2016
  22. Duy NguyenApr 20, 2016
  23. David TurnerApr 20, 2016
  24. Duy NguyenApr 20, 2016
  25. 10/15 update-index: enable/disable watchman supportDavid Turner, Apr 19, 2016
  26. Duy NguyenApr 19, 2016
  27. David TurnerApr 20, 2016
  28. 11/15 unpack-trees: preserve index extensionsDavid Turner, Apr 19, 2016
  29. 12/15 index-helper: kill modeDavid Turner, Apr 19, 2016
  30. 13/15 index-helper: don't run if already runningDavid Turner, Apr 19, 2016
  31. 14/15 index-helper: autorun modeDavid Turner, Apr 19, 2016
  32. 15/15 index-helper: optionally automatically runDavid Turner, Apr 19, 2016
  33. 16/15 Add tracing to measure where most of the time is spentNguyễn Thái Ngọc Duy, Apr 20, 2016
  34. Johannes SchindelinApr 20, 2016
  35. Duy NguyenApr 20, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.