Re: Why is git submodule slow under windows

From: Jens Lehmann <jens.lehmann@web.de>
Date: Jul 18, 2014, 18:01 UTC
Message-ID: <53C96110.7020801@web.de>
In-Reply-To: <20140718130241.GD14720@paksenarrion.iveqy.com>

Am 18.07.2014 15:02, schrieb Fredrik Gustafsson:

Show 30 quoted lines

> On Fri, Jul 18, 2014 at 02:08:36PM +0200, Thomas Braun wrote:
>> Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1):
>>> Hello,
>>>
>>> We split a monolithic repository into ~50 submodules. The stored data
>>> has the same size. In the 1:1 comparison to the monolithic
>>> repository, the submodule handling is very slow. Under Linux
>>> everything remains fast, but windows is slow.
>>>
>>> So, why is git getting slow when it has to deal with a lot of
>>> submodules? I read something about the lack of the underlying cygwin
>>> to handle NTFS in a efficient way. Is this the root cause, or are
>>> there other causes also?
>>>
>>
>> Hi,
>>
>> I assume you are using the latetst git from https://msysgit.github.io on
>> windows.
>>
>> I would guess that submodules on windows are slow because
>> git-submodules.sh is a shell script, and bash on windows is not really
>> that fast.
> 
> My guess is that because the shell script uses fork() heavily and fork()
> is an expensive operation on Windows, that alone causes the slowddown.
> 
> I did a quick test a while back when I rewrote part of git-submodule.sh
> in lua and runned it on my repo with ~45 submodules. The speedup was
> significant and should be even bigger on windows.

Without having looked at your Lua rewrite I suspect my recursive submodule checkout series could speed that up even more, as it only needs to fork a "git status" followed by a "git checkout" or "git read-tree" for each submodule. Now that my submodule test harness and Heiko's submodule config lookup API did hit next, this is the thing I'm currently working on.