Re: Windows performance / threading file access

From: Karsten Blees <karsten.blees@gmail.com>
Date: Oct 17, 2013, 16:50 UTC
Message-ID: <52601562.2090301@gmail.com>
In-Reply-To: <3bb056f6-5f8b-486e-8e5e-9bf541bd0d0b@googlegroups.com>

Am 16.10.2013 00:22, schrieb pro-logic:

Show 9 quoted lines

> I also get fairly slow performance out of the checkout / reset 
> operations on windows.
> 
> This discussion got me trying to work out what's taking so long on 
> windows. To help I used killcache [1] to flush the HDD cache and
> Very Sleepy [2] to profile the code. I couldn't use the 
> GIT_TRACE_PERFORMANCE [3] patch as that seems to only work on script 
> commands, and in my case I just get a result of "335 seconds git 
> reset --hard head" from the log.

The trace_performance functions require manual instrumentation of the code sections you want to measure, e.g. like this [1]. Output looks like this for a full WebKit checkout (Win7 x64, Core i7 860, WD VelociRaptor 300, NTFS, no virus scanner, no luafv, no defragger):

trace: at entry.c:128, time: 135.786 s: write_entry::create trace: at entry.c:129, time: 101.6 s: write_entry::stream trace: at entry.c:130, time: 0 s: write_entry::read trace: at entry.c:131, time: 0 s: write_entry::convert trace: at entry.c:132, time: 0 s: write_entry::write trace: at entry.c:133, time: 4.71825 s: write_entry::close trace: at compat/mingw.c:2150, time: 5.68786 s: mingw_lstat (called 661660 times) trace: at compat/mingw.c:2151, time: 259.219 s: command: c:\git\msysgit\git\git-checkout.exe -f HEAD

> After running killcache I ran very sleepy connected to git, and 
> according to the profile: 95.5% of the time is spent in do_lstat 
> (mingw.c) / NtQueryFullAttributeFile (ntdll)

Very Sleepy confirmed my numbers from above: lstat was always much smaller than create/stream/read/write. Could you post details about your test setup? Are you still using WebKit for your tests?

> For fun, not knowing if I would break anything or not (it probably 
> does), I wrapped the entire unpack_trees method in the fscache [4] 
> and the total git reset --hard head time fell from 335 seconds to 28 
> seconds, a 11x improvement.

Hmmm...this doesn't work for me at all. Fscache isn't updated during checkout, so lstat-checks whether creating a file or directory succeeded will fail.

$ git config core.fscache true $ time git checkout -f HEAD Unlink of file 'Examples' failed. Should I try again? (y/n) n warning: unable to unlink Examples: Permission denied fatal: cannot create directory at 'Examples': Permission denied

Karsten

[1] https://github.com/kblees/git/commit/b8eca278

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.