{"thread":{"id":"31116","subject":"[PATCH] Enable parallelism in git submodule update.","startedAt":"2012-07-27T18:37:34Z","lastAt":"2012-11-03T19:07:02Z","messageCount":12,"participants":["Stefan Zager","Junio C Hamano","Heiko Voigt","Jens Lehmann"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"195955","messageId":"20120727185925.793121C0FDC@stefro.sfo.corp.google.com","threadId":"31116","inReplyTo":null,"subject":"[PATCH] Enable parallelism in git submodule update.","fromName":"Stefan Zager","fromEmail":"szager@google.com","sentAt":"2012-07-27T18:37:34Z","receivedAt":"2012-07-27T18:37:34Z","isPatch":true,"sender":{"key":"szager@google.com","avatar":null},"body":"The --jobs parameter may be used to set the degree of per-submodule\nparallel execution.\n\nSigned-off-by: Stefan Zager \n---\n Documentation/git-submodule.txt | 8 +++++++-\n git-submodule.sh | 23 ++++++++++++++++++++++-\n 2 files changed, 29 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex fbbbcb2..34f81fb 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -14,7 +14,8 @@ SYNOPSIS\n 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...]\n 'git submodule' [--quiet] init [--] [...]\n 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]\n-\t [--reference ] [--merge] [--recursive] [--] [...]\n+\t [--reference ] [--merge] [--recursive]\n+\t [-j|--jobs [jobs]] [--] [...]\n 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ]\n \t [commit] [--] [...]\n 'git submodule' [--quiet] foreach [--recursive] \n@@ -147,6 +148,11 @@ If the submodule is not yet initialized, and you just want to use the\n setting as stored in .gitmodules, you can automatically initialize the\n submodule with the `--init` option.\n +\n+By default, each submodule is treated serially. You may specify a degree of\n+parallel execution with the --jobs flag. If a parameter is provided, it is\n+the maximum number of jobs to run in parallel; without a parameter, all jobs are\n+run in parallel.\n++\n If `--recursive` is specified, this command will recurse into the\n registered submodules, and update any nested submodules within.\n \ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex dba4d39..761420a 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -8,7 +8,7 @@ dashless=$(basename \"$0\" | sed -e 's/-/ /')\n USAGE=\"[--quiet] add [-b branch] [-f|--force] [--reference ] [--] []\n or: $dashless [--quiet] status [--cached] [--recursive] [--] [...]\n or: $dashless [--quiet] init [--] [...]\n- or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...]\n+ or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [-j|--jobs [jobs]] [--] [...]\n or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...]\n or: $dashless [--quiet] foreach [--recursive] \n or: $dashless [--quiet] sync [--] [...]\"\n@@ -473,6 +473,7 @@ cmd_update()\n {\n \t# parse $args after \"submodule ... update\".\n \torig_flags=\n+\tjobs=\"1\"\n \twhile test $# -ne 0\n \tdo\n \t\tcase \"$1\" in\n@@ -491,6 +492,20 @@ cmd_update()\n \t\t-r|--rebase)\n \t\t\tupdate=\"rebase\"\n \t\t\t;;\n+\t\t-j|--jobs)\n+\t\t\tcase \"$2\" in\n+\t\t\t''|-*)\n+\t\t\t\tjobs=\"0\"\n+\t\t\t\t;;\n+\t\t\t*)\n+\t\t\t\tjobs=\"$2\"\n+\t\t\t\tshift\n+\t\t\t\t;;\n+\t\t\tesac\n+\t\t\t# Don't preserve this arg.\n+\t\t\tshift\n+\t\t\tcontinue\n+\t\t\t;;\n \t\t--reference)\n \t\t\tcase \"$2\" in '') usage ;; esac\n \t\t\treference=\"--reference=$2\"\n@@ -529,6 +544,12 @@ cmd_update()\n \t\tcmd_init \"--\" \"$@\" || return\n \tfi\n \n+\tif test \"$jobs\" != \"1\"\n+\tthen\n+\t\tmodule_list \"$@\" | awk '{print $4}' | xargs -L 1 -P \"$jobs\" git submodule update $orig_args\n+\t\treturn\n+\tfi\n+\n \tcloned_modules=\n \tmodule_list \"$@\" | {\n \terr=\n-- \n1.7.11.rc2\n"},{"id":"195967","messageId":"7vwr1ozxz5.fsf@alter.siamese.dyndns.org","threadId":"31116","inReplyTo":"20120727185925.793121C0FDC@stefro.sfo.corp.google.com","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-27T21:38:06Z","receivedAt":"2012-07-27T21:38:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Zager writes:\n\n> +\t\tmodule_list \"$@\" | awk '{print $4}' | xargs -L 1 -P \"$jobs\" git submodule update $orig_args\n\nCapital-P option to xargs is not even in POSIX, no?\n"},{"id":"195982","messageId":"7vk3xoyeex.fsf@alter.siamese.dyndns.org","threadId":"31116","inReplyTo":"CAHOQ7J_jYAe7r1q6Cg9OJb8f+79UfS=JfRk9NrS4R4a+oLM8LA@mail.gmail.com","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-27T23:25:58Z","receivedAt":"2012-07-27T23:25:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Zager writes:\n\n> On Fri, Jul 27, 2012 at 2:38 PM, Junio C Hamano wrote:\n>\n>> Stefan Zager writes:\n>>\n>> > + module_list \"$@\" | awk '{print $4}' | xargs -L 1 -P\n>> \"$jobs\" git submodule update $orig_args\n>>\n>> Capital-P option to xargs is not even in POSIX, no?\n>\n> I wasn't aware of that, but you appear to be correct. Don't know if you\n> have a policy about that, but anecdotally, -P is supported on my linux,\n> mac, and win/msys systems.\n\nAbout \"policy\", we use POSIX as a rough yardstick to warn us that we\nmight be breaking people on minority platforms. We do _not_ say \"It\nis in POSIX, so it is safe to use it\", but we say \"It is not even in\nPOSIX, so we need to think twice.\" We do not usually say \"Linux,\nMac and Windows are the only things that matter, and they all\nsupport it.\"\n\nOf course, any set of rules have exceptions ;-) There are a few\nthings to which we say \"Even though it is not in POSIX, everybody\nwho matters supports it, and without taking advantage of it, what we\nwant to achieve will become too cumbersome to express\".\n\nIn the core parts of the system, we try to be very conservative. In\nthe fringe where nobody cares about, we tend to be looser.\n"},{"id":"196015","messageId":"20120728102209.GA13370@book.hvoigt.net","threadId":"31116","inReplyTo":"20120727185925.793121C0FDC@stefro.sfo.corp.google.com","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-07-28T10:22:11Z","receivedAt":"2012-07-28T10:22:11Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi Stefan,\n\nneat patch. See below for a few notes.\n\nOn Fri, Jul 27, 2012 at 11:37:34AM -0700, Stefan Zager wrote:\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index dba4d39..761420a 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -491,6 +492,20 @@ cmd_update()\n> \t\t-r|--rebase)\n> \t\t\tupdate=\"rebase\"\n> \t\t\t;;\n> +\t\t-j|--jobs)\n> +\t\t\tcase \"$2\" in\n> +\t\t\t''|-*)\n> +\t\t\t\tjobs=\"0\"\n> +\t\t\t\t;;\n> +\t\t\t*)\n> +\t\t\t\tjobs=\"$2\"\n> +\t\t\t\tshift\n> +\t\t\t\t;;\n> +\t\t\tesac\n> +\t\t\t# Don't preserve this arg.\n> +\t\t\tshift\n> +\t\t\tcontinue\n> +\t\t\t;;\n> \t\t--reference)\n> \t\t\tcase \"$2\" in '') usage ;; esac\n> \t\t\treference=\"--reference=$2\"\n> @@ -529,6 +544,12 @@ cmd_update()\n> \t\tcmd_init \"--\" \"$@\" || return\n> \tfi\n> \n> +\tif test \"$jobs\" != \"1\"\n> +\tthen\n> +\t\tmodule_list \"$@\" | awk '{print $4}' | xargs -L 1 -P \"$jobs\" git submodule update $orig_args\n\nI do not see orig_args set anywhere in submodule.sh. It seems the\nexisting usage of it in cmd_status() is a leftover from commit\n98dbe63 when this variable got renamed to orig_flags.\n\nI will follow up with a patch to that location.\n\nAnother problem here is the passing of arguments. Have a look at\na7eff1a8 to see how this was solved for other locations.\n\nThe next thing I noticed is that the parallelism is not recursive. You\ndrop the option and only execute the first depth in parallel. How about\nusing the amount of modules defined by arguments left in $@ as an\nindicator whether you need to fork parallel execution or not. If there\nis exactly one you do the update if there are more you do the parallel\nthing. That way you can just keep passing the --jobs flag to the\nsubprocesses.\n\nThe next question to solve is UI: Since the output lines of the parallel\nupdate jobs will be mixed we need some way to distinguish them. Imagine\none of the update fails somewhere how do we find out which it was?\n\nTwo possible solutions come to my mind:\n\n 1. Prefix each line with a job number. This way you can distinguish\n which process outputted what and still have immediate feedback.\n\n 2. Cache the output (to stderr and stdout) of each job and output it\n once one job is done. I imagine this needs some infrastructure which\n we need to implement. We already have some ideas how to collect such\n output in C here[1].\n\nI would prefer solution 2 since the output of 1 will be hard to read but\nI guess we could start with 1 and then move over to 2 later on.\n\nCheers Heiko\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/197747\n"},{"id":"196016","messageId":"20120728105159.GB13370@book.hvoigt.net","threadId":"31116","inReplyTo":"7vk3xoyeex.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-07-28T10:52:01Z","receivedAt":"2012-07-28T10:52:01Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"Hi,\n\nOn Fri, Jul 27, 2012 at 04:25:58PM -0700, Junio C Hamano wrote:\n> Stefan Zager writes:\n> \n> > On Fri, Jul 27, 2012 at 2:38 PM, Junio C Hamano wrote:\n> >\n> >> Stefan Zager writes:\n> >>\n> >> > + module_list \"$@\" | awk '{print $4}' | xargs -L 1 -P\n> >> \"$jobs\" git submodule update $orig_args\n> >>\n> >> Capital-P option to xargs is not even in POSIX, no?\n> >\n> > I wasn't aware of that, but you appear to be correct. Don't know if you\n> > have a policy about that, but anecdotally, -P is supported on my linux,\n> > mac, and win/msys systems.\n> \n> About \"policy\", we use POSIX as a rough yardstick to warn us that we\n> might be breaking people on minority platforms. We do _not_ say \"It\n> is in POSIX, so it is safe to use it\", but we say \"It is not even in\n> POSIX, so we need to think twice.\" We do not usually say \"Linux,\n> Mac and Windows are the only things that matter, and they all\n> support it.\"\n> \n> Of course, any set of rules have exceptions ;-) There are a few\n> things to which we say \"Even though it is not in POSIX, everybody\n> who matters supports it, and without taking advantage of it, what we\n> want to achieve will become too cumbersome to express\".\n\nI was about to write that since this is limited to a given --jobs\noptions the majority platforms should be enough as a start and others\ncould add a parallelism mechanism later. Its only a matter of efficiency\nand not features.\n\nBut if you look at my other post to this thread I described that we need\nsome UI output extension so the user can still make sense of it.\nIn short: The user should be able distinguish which job said what.\n\nI was already thinking about how an output caching could be implemented in\ncore git. How about exposing it as a git command like this?\n\n\tgit run [-j] ...\n\nIt works like the xargs call above except that it caches each jobs\noutput to stderr and stdout until its done and then replays the output\nto stderr/out in the correct order.\n\nWe could design the code so that it can be reused later on to do the\ncaching in parallel fetch/push/... .\n\nWhat do you think? If we decide to go this route I would have a look\ninto whipping something up.\n\nCheers Heiko\n"},{"id":"196018","messageId":"20120728121956.GA36429@book.hvoigt.net","threadId":"31116","inReplyTo":"20120728102209.GA13370@book.hvoigt.net","subject":"[PATCH] cleanup argument passing in submodule status command","fromName":"Heiko Voigt","fromEmail":"hvoigt@hvoigt.net","sentAt":"2012-07-28T12:19:56Z","receivedAt":"2012-07-28T12:19:56Z","isPatch":true,"sender":{"key":"hvoigt@hvoigt.net","avatar":"https://avatars.githubusercontent.com/u/184958?v=4"},"body":"In commit 98dbe63 the variable $orig_args was renamed to $orig_flags.\nOne location in cmd_status() was missed.\n\nNote: This is a code cleanup and does not fix any bugs. As a side effect\nthe variables containing the parsed flags to \"git submodule status\" are\npassed down recursively. So everything was already behaving as expected.\n\nSigned-off-by: Heiko Voigt \n---\n git-submodule.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex dba4d39..3a3f0a4 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -961,7 +961,7 @@ cmd_status()\n \t\t\t\tprefix=\"$displaypath/\"\n \t\t\t\tclear_local_git_env\n \t\t\t\tcd \"$sm_path\" &&\n-\t\t\t\teval cmd_status \"$orig_args\"\n+\t\t\t\teval cmd_status \"$orig_flags\"\n \t\t\t) ||\n \t\t\tdie \"$(eval_gettext \"Failed to recurse into submodule path '\\$sm_path'\")\"\n \t\tfi\n-- \n1.7.12.rc0.23.g3c7cae0\n"},{"id":"196053","messageId":"7vtxwrw0g0.fsf@alter.siamese.dyndns.org","threadId":"31116","inReplyTo":"20120728121956.GA36429@book.hvoigt.net","subject":"Re: [PATCH] cleanup argument passing in submodule status command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-29T06:22:55Z","receivedAt":"2012-07-29T06:22:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Heiko Voigt writes:\n\n> Note: This is a code cleanup and does not fix any bugs. As a side effect\n> the variables containing the parsed flags to \"git submodule status\" are\n> passed down recursively. So everything was already behaving as expected.\n\nIf that is the case, shouldn't we stop passing anything down, if we\nwant it to be a \"clean-up only, no behaviour changes\" patch? While\nat it, we may want to kill that code to accumulate the original\noptions in orig_flags because we haven't been using the variable.\n\nWe _know_ $orig_args has been empty, i.e. the code has been working\nfine with only cmd_status there. Nobody has tried what happens when\nwe pass the original arguments to cmd_status on that line. The\npatch changes the behaviour of the code; it makes the command line\nparsing \"while\" loop to run again, and if the code that accumulates\noriginal options in orig_flags have been buggy, now that bug will be\nexposed.\n\n\n\n\n> Signed-off-by: Heiko Voigt \n> ---\n> git-submodule.sh | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index dba4d39..3a3f0a4 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -961,7 +961,7 @@ cmd_status()\n> \t\t\t\tprefix=\"$displaypath/\"\n> \t\t\t\tclear_local_git_env\n> \t\t\t\tcd \"$sm_path\" &&\n> -\t\t\t\teval cmd_status \"$orig_args\"\n> +\t\t\t\teval cmd_status \"$orig_flags\"\n> \t\t\t) ||\n> \t\t\tdie \"$(eval_gettext \"Failed to recurse into submodule path '\\$sm_path'\")\"\n> \t\tfi\n"},{"id":"196066","messageId":"501556CF.1000605@web.de","threadId":"31116","inReplyTo":"7vtxwrw0g0.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] cleanup argument passing in submodule status command","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-07-29T15:29:19Z","receivedAt":"2012-07-29T15:29:19Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 29.07.2012 08:22, schrieb Junio C Hamano:\n> Heiko Voigt writes:\n> \n>> Note: This is a code cleanup and does not fix any bugs. As a side effect\n>> the variables containing the parsed flags to \"git submodule status\" are\n>> passed down recursively. So everything was already behaving as expected.\n> \n> If that is the case, shouldn't we stop passing anything down, if we\n> want it to be a \"clean-up only, no behaviour changes\" patch? While\n> at it, we may want to kill that code to accumulate the original\n> options in orig_flags because we haven't been using the variable.\n> \n> We _know_ $orig_args has been empty, i.e. the code has been working\n> fine with only cmd_status there. Nobody has tried what happens when\n> we pass the original arguments to cmd_status on that line.\n\nI tried today. Before this change no arguments got passed down and\nafterwards they are (but just the arguments, no submodule paths\nwere passed on in either case; which is what Kevin fixed in the\ncommit Heiko referenced). Three arguments are allowed for \"git\nsubmodule status\":\n\n--recursive:\nIt doesn't matter if we pass that on or not because $recursive is\nreused when \"eval cmd_status\" is executed.\n\n--quiet:\nSame as recursive, GIT_QUIET is set the first time and then reused\nin the recursion.\n\n--cached:\nThis was dropped when recursing into submodules but isn't anymore\nwith Heiko's change, so we do have a change in behavior here.\n\n> The\n> patch changes the behaviour of the code; it makes the command line\n> parsing \"while\" loop to run again, and if the code that accumulates\n> original options in orig_flags have been buggy, now that bug will be\n> exposed.\n\nHmm, when --cached is used together with --recursive, I would expect\nit to show the commit stored in the index for the deeper submodules\ntoo (and not magically switch to show their HEAD again after the\nfirst level of submodules). To me this looks like a bug which Kevin\naccidentally introduced and nobody noticed and/or reported until now.\n\nSo I'd vote for making this a bugfix patch for \"git submodule status\n--cached --recursive\" (and would love to see a test for it ;-).\n\n>> Signed-off-by: Heiko Voigt \n>> ---\n>> git-submodule.sh | 2 +-\n>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>\n>> diff --git a/git-submodule.sh b/git-submodule.sh\n>> index dba4d39..3a3f0a4 100755\n>> --- a/git-submodule.sh\n>> +++ b/git-submodule.sh\n>> @@ -961,7 +961,7 @@ cmd_status()\n>> \t\t\t\tprefix=\"$displaypath/\"\n>> \t\t\t\tclear_local_git_env\n>> \t\t\t\tcd \"$sm_path\" &&\n>> -\t\t\t\teval cmd_status \"$orig_args\"\n>> +\t\t\t\teval cmd_status \"$orig_flags\"\n>> \t\t\t) ||\n>> \t\t\tdie \"$(eval_gettext \"Failed to recurse into submodule path '\\$sm_path'\")\"\n>> \t\tfi\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n> \n"},{"id":"196067","messageId":"501558AD.6010402@web.de","threadId":"31116","inReplyTo":"20120727185925.793121C0FDC@stefro.sfo.corp.google.com","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-07-29T15:37:17Z","receivedAt":"2012-07-29T15:37:17Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 27.07.2012 20:37, schrieb Stefan Zager:\n> The --jobs parameter may be used to set the degree of per-submodule\n> parallel execution.\n\nI think this is a sound idea, but it would be good to see some\nactual measurements. What are the performance numbers with and\nwithout this change? Which cases do benefit and are there some\nwhich run slower when run in parallel?\n"},{"id":"196084","messageId":"7vk3xmut63.fsf@alter.siamese.dyndns.org","threadId":"31116","inReplyTo":"501556CF.1000605@web.de","subject":"Re: [PATCH] cleanup argument passing in submodule status command","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-29T21:57:40Z","receivedAt":"2012-07-29T21:57:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann writes:\n\n> I tried today. Before this change no arguments got passed down and\n> afterwards they are (but just the arguments, no submodule paths\n> were passed on in either case; which is what Kevin fixed in the\n> commit Heiko referenced). Three arguments are allowed for \"git\n> submodule status\":\n>\n> --recursive:\n> It doesn't matter if we pass that on or not because $recursive is\n> reused when \"eval cmd_status\" is executed.\n>\n> --quiet:\n> Same as recursive, GIT_QUIET is set the first time and then reused\n> in the recursion.\n>\n> --cached:\n> This was dropped when recursing into submodules but isn't anymore\n> with Heiko's change, so we do have a change in behavior here.\n> ...\n> Hmm, when --cached is used together with --recursive, I would expect\n> it to show the commit stored in the index for the deeper submodules\n> too (and not magically switch to show their HEAD again after the\n> first level of submodules). To me this looks like a bug which Kevin\n> accidentally introduced and nobody noticed and/or reported until now.\n>\n> So I'd vote for making this a bugfix patch for \"git submodule status\n> --cached --recursive\" (and would love to see a test for it ;-).\n\nYeah, I am not opposed to a \"fix\". I just wanted it to be labelled\nas such, and analysed correctly.\n\nAnd with test ;-)\n\nThanks.\n"},{"id":"196085","messageId":"7vfw8aut35.fsf@alter.siamese.dyndns.org","threadId":"31116","inReplyTo":"20120728105159.GB13370@book.hvoigt.net","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-29T21:59:26Z","receivedAt":"2012-07-29T21:59:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Heiko Voigt writes:\n\n> On Fri, Jul 27, 2012 at 04:25:58PM -0700, Junio C Hamano wrote:\n> ...\n>> Of course, any set of rules have exceptions ;-) There are a few\n>> things to which we say \"Even though it is not in POSIX, everybody\n>> who matters supports it, and without taking advantage of it, what we\n>> want to achieve will become too cumbersome to express\".\n>\n> I was about to write that since this is limited to a given --jobs\n> options the majority platforms should be enough as a start and others\n> could add a parallelism mechanism later. Its only a matter of efficiency\n> and not features.\n\nAs long as \"git submodule --jobs 9\" on a platform without GNU\nenhanced xargs does not error out and gracefully degrade to non\nparallel execution, I do not have any problem with it. As posted,\nthe patch has not yet achieved that doneness yet.\n"},{"id":"202469","messageId":"50956B56.1010603@web.de","threadId":"31116","inReplyTo":"501558AD.6010402@web.de","subject":"Re: [PATCH] Enable parallelism in git submodule update.","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2012-11-03T19:07:02Z","receivedAt":"2012-11-03T19:07:02Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 29.07.2012 17:37, schrieb Jens Lehmann:\n> Am 27.07.2012 20:37, schrieb Stefan Zager:\n>> The --jobs parameter may be used to set the degree of per-submodule\n>> parallel execution.\n> \n> I think this is a sound idea, but it would be good to see some\n> actual measurements. What are the performance numbers with and\n> without this change? Which cases do benefit and are there some\n> which run slower when run in parallel?\n\nping?\n"}]}