git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Some rough edges of core.fsmonitor

From
Duy Nguyen <pclouds@gmail.com>
Date
Jan 27, 2018, 01:36 UTC
Message-ID
<CACsJy8BpO0s6facg+zcKC9icijpefkipM326n6xOArjn=ZW6+w@mail.gmail.com>
In-Reply-To
<87efmcw3fa.fsf@evledraar.gmail.com>

On Sat, Jan 27, 2018 at 7:28 AM, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

> 3) A lot of time spend reading the index (or something..)

I'm resending a patch from my old index-helper series. It should measure all big time consuming blocks in git. Maybe we should get it merged...

Show 7 quoted lines
> While the hook itself takes ~20ms (and watchman itself 1/4 of that)
> status as a whole takes much longer. gprof reveals:
>
>     Each sample counts as 0.01 seconds.
>       %   cumulative   self              self     total
>      time   seconds   seconds    calls  ms/call  ms/call  name
>      15.38      0.02     0.02   221690     0.00     0.00  memihash
This sounds like name-hash to me.
Show 14 quoted lines
>      15.38      0.04     0.02   221689     0.00     0.00  create_from_disk
>       7.69      0.05     0.01  2216897     0.00     0.00  git_bswap32
>       7.69      0.06     0.01   222661     0.00     0.00  ce_path_match
>       7.69      0.07     0.01   221769     0.00     0.00  hashmap_add
>       7.69      0.08     0.01    39941     0.00     0.00  prep_exclude
>       7.69      0.09     0.01    39940     0.00     0.00  strbuf_addch
>       7.69      0.10     0.01        1    10.00    10.00  read_one
>       7.69      0.11     0.01        1    10.00    10.00  refresh_index
>       7.69      0.12     0.01        1    10.00    10.00  tweak_fsmonitor
>       7.69      0.13     0.01                             preload_thread
>
> The index is 24M in this case, I guess it's unpacking it, but I wonder
> if this couldn't be much faster if we saved away the result of the last
> "status" in something that's quick to access, and then if nothing

No we could do better, we could cache parsed index content so everybody benefits. I demonstrated it with my "index v254" patch a while back:

https://public-inbox.org/git/1399980019-8706-1-git-send-email-pclouds@gmail.com/

With the patch I'm sending soon, we can see how much time reading an index take out of that ~140-150ms (and we probably can cut down index read time to like 10-20% when cached).

> changed we just report that, and no need to re-write the index (or just
> write the "it was clean at this time" part).
Hmm.. does an index write increase that much time?
-- 
Duy
Previous: Ævar Arnfjörð BjarmasonNext: Nguyễn Thái Ngọc Duy
Message 2 of 30 in “Some rough edges of core.fsmonitor”
  1. Ævar Arnfjörð BjarmasonJan 27, 2018
  2. Duy NguyenJan 27, 2018
  3. trace: measure where the time is spent in the index-heavy operationsNguyễn Thái Ngọc Duy, Jan 27, 2018
  4. Thomas GummererJan 27, 2018
  5. trace: measure where the time is spent in the index-heavy operationsNguyễn Thái Ngọc Duy, Jan 27, 2018
  6. Ævar Arnfjörð BjarmasonJan 27, 2018
  7. Duy NguyenJan 27, 2018
  8. Duy NguyenJan 27, 2018
  9. Ævar Arnfjörð BjarmasonJan 27, 2018
  10. Ben PeartJan 30, 2018
  11. Duy NguyenJan 29, 2018
  12. Ben PeartJan 29, 2018
  13. Duy NguyenFeb 1, 2018
  14. Johannes SchindelinJan 28, 2018
  15. Ævar Arnfjörð BjarmasonJan 28, 2018
  16. Ben PeartJan 30, 2018
  17. Duy NguyenJan 31, 2018
  18. dir.c: ignore paths containing .git when invalidating untracked cacheNguyễn Thái Ngọc Duy, Feb 4, 2018
  19. Ben PeartFeb 5, 2018
  20. Duy NguyenFeb 6, 2018
  21. dir.c: ignore paths containing .git when invalidating untracked cacheNguyễn Thái Ngọc Duy, Feb 7, 2018
  22. dir.c: ignore paths containing .git when invalidating untracked cacheNguyễn Thái Ngọc Duy, Feb 7, 2018
  23. Ben PeartFeb 7, 2018
  24. Duy NguyenFeb 13, 2018
  25. Junio C HamanoFeb 13, 2018
  26. Duy NguyenFeb 14, 2018
  27. Junio C HamanoFeb 14, 2018
  28. Ben PeartJan 30, 2018
  29. Ævar Arnfjörð BjarmasonJan 30, 2018
  30. Ben PeartJan 31, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.