{"thread":{"id":"37473","subject":"git submodule, multi-threading the update process for large number of registered submodules","startedAt":"2014-09-02T07:53:29Z","lastAt":"2014-09-02T19:30:10Z","messageCount":2,"participants":["Jose Paredes","Jens Lehmann"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"248710","messageId":"CAK9Skq_-cidRHrbVNgJqoV3nYCxRNDO6vTifOJRUFKaWEZyzkw@mail.gmail.com","threadId":"37473","inReplyTo":null,"subject":"git submodule, multi-threading the update process for large number of registered submodules","fromName":"Jose Paredes","fromEmail":"jose.paredes.rios@gmail.com","sentAt":"2014-09-02T07:53:29Z","receivedAt":"2014-09-02T07:53:29Z","isPatch":false,"sender":{"key":"jose.paredes.rios@gmail.com","avatar":null},"body":"Dear Git community\n\nIs there any way to run \"git submodule update\" in multiple threads?\n\nThe use case:\n-------------------\n\"git submodule update\" seems to be inefficient when running\nsequentially on a large .gitmodules file.\nAssuming a git forest with over 7K gits it takes hours to complete the\nupdate (running on Windows+Cygwin)\n\n\nIf not supported, this feature could be a good candidate for \"git\nsubmodule\" enhancement.\nWhat is your opinion or advice?\n\nThanks and regards\n\nJosé\n"},{"id":"248734","messageId":"54061AC2.9020705@web.de","threadId":"37473","inReplyTo":"CAK9Skq_-cidRHrbVNgJqoV3nYCxRNDO6vTifOJRUFKaWEZyzkw@mail.gmail.com","subject":"Re: git submodule, multi-threading the update process for large number of registered submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2014-09-02T19:30:10Z","receivedAt":"2014-09-02T19:30:10Z","isPatch":false,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Hi José,\n\nAm 02.09.2014 um 09:53 schrieb Jose Paredes:\n> Is there any way to run \"git submodule update\" in multiple threads?\n>\n> The use case:\n> -------------------\n> \"git submodule update\" seems to be inefficient when running\n> sequentially on a large .gitmodules file.\n> Assuming a git forest with over 7K gits it takes hours to complete the\n> update (running on Windows+Cygwin)\n\nWow, over 7000 submodules under Windows? I can imagine that a\nsubmodule update takes some time under these circumstances.\n\n> If not supported, this feature could be a good candidate for \"git\n> submodule\" enhancement.\n> What is your opinion or advice?\n\nHmm, are we really sure parallel execution would be faster? I can\nimagine that \"git fetch\" could profit a lot from fetching more than\none submodule at a time (assuming you are not fetching from the same\nupstream server), but I'm not so sure about submodule update. And\nthings might also be rather different e.g. on Windows and Linux.\n\nCould you do some benchmarks comparing \"git submodule update\" with a\nscript that does something like:\n\n   (cd <submodule>; git checkout <commit recorded in superproject>)&\n\nfor each changed submodule to see if we could gain anything from\nexecuting the update in parallel? (You should be able to extract\nwhich submodules must be checked out to what commit by doing e.g.\n\"git diff --raw <start> <target> | grep ^:160000\" and processing\nthe output a bit further).\n"}]}