{"thread":{"id":"40570","subject":"[PATCH 02/12] git submodule update: Announce uninitialized modules on stderr","startedAt":"2015-10-16T01:52:01Z","lastAt":"2015-10-20T21:21:30Z","messageCount":21,"participants":["Stefan Beller","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":12},"messages":[{"id":"271784","messageId":"1444960333-16003-1-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":null,"subject":"[RFC PATCHv1 00/12] git submodule update in C with parallel cloning","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:01Z","receivedAt":"2015-10-16T01:52:01Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"So eventually we want to have projects with lots of submodules such as Android\n(which would have O(1000) submodules).\n\nThe very first thing a user does is cloning a project, so we want to impress\nwith speed there as well. git clone however is lazy and just calls\n`git submodule update --init --recursive`. So we need to make that fast.\n\nThis series rewrites parts of git submodule update in C and in the second-last\npatch it separates cloning and doing the other actions(checkout/rebase/merge etc)\nby doing the cloning first and then the rest.\n\nThe last patch (which is broken in the first version of the series), then \nproceeds to put the cloning of the submodules into the get_next_task callback\nof the parallel process API.\n\nThat said, the first few patches introduce some churn in the behavior and tests\nof Git, so maybe put your eyes there?\n\nThanks for any advice,\nStefan\n\nStefan Beller (12):\n  git submodule update: Announce skipping submodules on stderr\n  git submodule update: Announce uninitialized modules on stderr\n  git submodule update: Move branch calculation to where it's needed\n  git submodule update: Announce outcome of submodule operation to\n    stderr\n  git submodule update: Use its own list implementation.\n  git submodule update: Handle unmerged submodules in C\n  submodule config: keep update strategy around\n  git submodule update: check for \"none\" in C\n  git submodule update: Check url in C\n  git submodule update: Clone projects from within C\n  submodule--helper: Do not emit submodules to process directly.\n  WIP/broken Clone all outstanding submodules in parallel\n\n builtin/submodule--helper.c | 221 ++++++++++++++++++++++++++++++++++++++++++++\n git-submodule.sh            |  38 +++-----\n submodule-config.c          |  11 +++\n submodule-config.h          |   1 +\n t/t7400-submodule-basic.sh  |  12 +--\n t/t7406-submodule-update.sh |  12 +--\n 6 files changed, 256 insertions(+), 39 deletions(-)\n\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271785","messageId":"1444960333-16003-2-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 01/12] git submodule update: Announce skipping submodules on stderr","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:02Z","receivedAt":"2015-10-16T01:52:02Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n git-submodule.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 8b0eb9a..578ec48 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -684,7 +684,7 @@ cmd_update()\n \n \t\tif test \"$update_module\" = \"none\"\n \t\tthen\n-\t\t\techo \"Skipping submodule '$displaypath'\"\n+\t\t\techo >&2 \"Skipping submodule '$displaypath'\"\n \t\t\tcontinue\n \t\tfi\n \n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271781","messageId":"1444960333-16003-3-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 02/12] git submodule update: Announce uninitialized modules on stderr","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:03Z","receivedAt":"2015-10-16T01:52:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n git-submodule.sh           |  2 +-\n t/t7400-submodule-basic.sh | 12 ++++++------\n 2 files changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 578ec48..eea27f8 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -693,7 +693,7 @@ cmd_update()\n \t\t\t# Only mention uninitialized submodules when its\n \t\t\t# path have been specified\n \t\t\ttest \"$#\" != \"0\" &&\n-\t\t\tsay \"$(eval_gettext \"Submodule path '\\$displaypath' not initialized\n+\t\t\tsay >&2 \"$(eval_gettext \"Submodule path '\\$displaypath' not initialized\n Maybe you want to use 'update --init'?\")\"\n \t\t\tcontinue\n \t\tfi\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 540771c..32a89b8 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -462,9 +462,9 @@ test_expect_success 'update --init' '\n \tgit config --remove-section submodule.example &&\n \ttest_must_fail git config submodule.example.url &&\n \n-\tgit submodule update init > update.out &&\n-\tcat update.out &&\n-\ttest_i18ngrep \"not initialized\" update.out &&\n+\tgit submodule update init 2> update.err &&\n+\tcat update.err &&\n+\ttest_i18ngrep \"not initialized\" update.err &&\n \ttest_must_fail git rev-parse --resolve-git-dir init/.git &&\n \n \tgit submodule update --init init &&\n@@ -480,9 +480,9 @@ test_expect_success 'update --init from subdirectory' '\n \tmkdir -p sub &&\n \t(\n \t\tcd sub &&\n-\t\tgit submodule update ../init >update.out &&\n-\t\tcat update.out &&\n-\t\ttest_i18ngrep \"not initialized\" update.out &&\n+\t\tgit submodule update ../init 2>update.err &&\n+\t\tcat update.err &&\n+\t\ttest_i18ngrep \"not initialized\" update.err &&\n \t\ttest_must_fail git rev-parse --resolve-git-dir ../init/.git &&\n \n \t\tgit submodule update --init ../init\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271783","messageId":"1444960333-16003-4-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 03/12] git submodule update: Move branch calculation to where it's needed","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:04Z","receivedAt":"2015-10-16T01:52:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The branch variable is used only once so calculate it only when needed.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n git-submodule.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex eea27f8..56a0524 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -668,7 +668,6 @@ cmd_update()\n \t\tfi\n \t\tname=$(git submodule--helper name \"$sm_path\") || exit\n \t\turl=$(git config submodule.\"$name\".url)\n-\t\tbranch=$(get_submodule_config \"$name\" branch master)\n \t\tif ! test -z \"$update\"\n \t\tthen\n \t\t\tupdate_module=$update\n@@ -718,6 +717,7 @@ Maybe you want to use 'update --init'?\")\"\n \t\t\t\tdie \"$(eval_gettext \"Unable to fetch in submodule path '\\$sm_path'\")\"\n \t\t\tfi\n \t\t\tremote_name=$(clear_local_git_env; cd \"$sm_path\" && get_default_remote)\n+\t\t\tbranch=$(get_submodule_config \"$name\" branch master)\n \t\t\tsha1=$(clear_local_git_env; cd \"$sm_path\" &&\n \t\t\t\tgit rev-parse --verify \"${remote_name}/${branch}\") ||\n \t\t\tdie \"$(eval_gettext \"Unable to find current ${remote_name}/${branch} revision in submodule path '\\$sm_path'\")\"\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271792","messageId":"1444960333-16003-5-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 04/12] git submodule update: Announce outcome of submodule operation to stderr","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:05Z","receivedAt":"2015-10-16T01:52:05Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n git-submodule.sh            |  2 +-\n t/t7406-submodule-update.sh | 12 ++++++------\n 2 files changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 56a0524..bb8b2c7 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -780,7 +780,7 @@ Maybe you want to use 'update --init'?\")\"\n \n \t\t\tif (clear_local_git_env; cd \"$sm_path\" && $command \"$sha1\")\n \t\t\tthen\n-\t\t\t\tsay \"$say_msg\"\n+\t\t\t\tsay >&2 \"$say_msg\"\n \t\t\telif test -n \"$must_die_on_failure\"\n \t\t\tthen\n \t\t\t\tdie_with_status 2 \"$die_msg\"\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex dda3929..f65b81c 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -111,8 +111,8 @@ test_expect_success 'submodule update does not fetch already present commits' '\n \t(cd super &&\n \t  git submodule update > ../actual 2> ../actual.err\n \t) &&\n-\ttest_i18ncmp expected actual &&\n-\t! test -s actual.err\n+\ttest_i18ncmp expected actual.err &&\n+\t! test -s actual\n '\n \n test_expect_success 'submodule update should fail due to local changes' '\n@@ -702,8 +702,8 @@ test_expect_success 'submodule update places git-dir in superprojects git-dir re\n \trm -rf super_update_r2 &&\n \tgit clone super_update_r super_update_r2 &&\n \t(cd super_update_r2 &&\n-\t git submodule update --init --recursive >actual &&\n-\t test_i18ngrep \"Submodule path .submodule/subsubmodule.: checked out\" actual &&\n+\t git submodule update --init --recursive 2>actual.err &&\n+\t test_i18ngrep \"Submodule path .submodule/subsubmodule.: checked out\" actual.err &&\n \t (cd submodule/subsubmodule &&\n \t  git log > ../../expected\n \t ) &&\n@@ -770,8 +770,8 @@ test_expect_success 'submodule update --recursive drops module name before recur\n \t (cd deeper/submodule/subsubmodule &&\n \t  git checkout HEAD^\n \t ) &&\n-\t git submodule update --recursive deeper/submodule >actual &&\n-\t test_i18ngrep \"Submodule path .deeper/submodule/subsubmodule.: checked out\" actual\n+\t git submodule update --recursive deeper/submodule 2>actual.err &&\n+\t test_i18ngrep \"Submodule path .deeper/submodule/subsubmodule.: checked out\" actual.err\n \t)\n '\n test_done\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271791","messageId":"1444960333-16003-6-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 05/12] git submodule update: Use its own list implementation.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:06Z","receivedAt":"2015-10-16T01:52:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Discussions turned out that we cannot parallelize the whole loop below\n`git submodule--helper list` in `git submodule update`, because some\nchanges should be done only one at a time, such as messing up a submodule\nand leave it up to the user to cleanup the conflicted rebase or merge.\n\nThe submodules which are need to be cloned however do not expect to create\nproblems which require attention by the user one at a time, so we want to\nparallelize that first.\n\nTo do so we will start with a literal copy of `git submodule--helper list`\nand port over features gradually.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 40 ++++++++++++++++++++++++++++++++++++++++\n git-submodule.sh            |  2 +-\n 2 files changed, 41 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex f4c3eff..47dc9cb 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -255,6 +255,45 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \treturn 0;\n }\n \n+static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n+{\n+\tint i;\n+\tstruct pathspec pathspec;\n+\tstruct module_list list = MODULE_LIST_INIT;\n+\n+\tstruct option module_list_options[] = {\n+\t\tOPT_STRING(0, \"prefix\", &prefix,\n+\t\t\t   N_(\"path\"),\n+\t\t\t   N_(\"alternative anchor for relative paths\")),\n+\t\tOPT_END()\n+\t};\n+\n+\tconst char *const git_submodule_helper_usage[] = {\n+\t\tN_(\"git submodule--helper list [--prefix=<path>] [<path>...]\"),\n+\t\tNULL\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, module_list_options,\n+\t\t\t     git_submodule_helper_usage, 0);\n+\n+\tif (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0) {\n+\t\tprintf(\"#unmatched\\n\");\n+\t\treturn 1;\n+\t}\n+\n+\tfor (i = 0; i < list.nr; i++) {\n+\t\tconst struct cache_entry *ce = list.entries[i];\n+\n+\t\tif (ce_stage(ce))\n+\t\t\tprintf(\"%06o %s U\\t\", ce->ce_mode, sha1_to_hex(null_sha1));\n+\t\telse\n+\t\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n+\n+\t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n+\t}\n+\treturn 0;\n+}\n+\n struct cmd_struct {\n \tconst char *cmd;\n \tint (*fn)(int, const char **, const char *);\n@@ -264,6 +303,7 @@ static struct cmd_struct commands[] = {\n \t{\"list\", module_list},\n \t{\"name\", module_name},\n \t{\"clone\", module_clone},\n+\t{\"list-or-clone\", module_list_or_clone}\n };\n \n int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex bb8b2c7..d2d80e2 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -656,7 +656,7 @@ cmd_update()\n \tfi\n \n \tcloned_modules=\n-\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" | {\n+\tgit submodule--helper list-or-clone --prefix \"$wt_prefix\" \"$@\" | {\n \terr=\n \twhile read mode sha1 stage sm_path\n \tdo\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271790","messageId":"1444960333-16003-7-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 06/12] git submodule update: Handle unmerged submodules in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:07Z","receivedAt":"2015-10-16T01:52:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 15 +++++++++++----\n git-submodule.sh            |  6 +-----\n 2 files changed, 12 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 47dc9cb..f81f37a 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -284,11 +284,18 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \tfor (i = 0; i < list.nr; i++) {\n \t\tconst struct cache_entry *ce = list.entries[i];\n \n-\t\tif (ce_stage(ce))\n-\t\t\tprintf(\"%06o %s U\\t\", ce->ce_mode, sha1_to_hex(null_sha1));\n-\t\telse\n-\t\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n+\t\tchar *env_prefix = getenv(\"prefix\");\n+\t\tif (ce_stage(ce)) {\n+\t\t\tif (env_prefix)\n+\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s/%s\",\n+\t\t\t\t\tenv_prefix, ce->name);\n+\t\t\telse\n+\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s\",\n+\t\t\t\t\tce->name);\n+\t\t\tcontinue;\n+\t\t}\n \n+\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n \t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n \t}\n \treturn 0;\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex d2d80e2..0754ecd 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -661,11 +661,7 @@ cmd_update()\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n-\t\tif test \"$stage\" = U\n-\t\tthen\n-\t\t\techo >&2 \"Skipping unmerged submodule $prefix$sm_path\"\n-\t\t\tcontinue\n-\t\tfi\n+\n \t\tname=$(git submodule--helper name \"$sm_path\") || exit\n \t\turl=$(git config submodule.\"$name\".url)\n \t\tif ! test -z \"$update\"\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271793","messageId":"1444960333-16003-8-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 07/12] submodule config: keep update strategy around","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:08Z","receivedAt":"2015-10-16T01:52:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"We need the submodule update strategies in a later patch.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n submodule-config.c | 11 +++++++++++\n submodule-config.h |  1 +\n 2 files changed, 12 insertions(+)\n\ndiff --git a/submodule-config.c b/submodule-config.c\nindex 393de53..175bcbb 100644\n--- a/submodule-config.c\n+++ b/submodule-config.c\n@@ -194,6 +194,7 @@ static struct submodule *lookup_or_create_by_name(struct submodule_cache *cache,\n \n \tsubmodule->path = NULL;\n \tsubmodule->url = NULL;\n+\tsubmodule->update = NULL;\n \tsubmodule->fetch_recurse = RECURSE_SUBMODULES_NONE;\n \tsubmodule->ignore = NULL;\n \n@@ -326,6 +327,16 @@ static int parse_config(const char *var, const char *value, void *data)\n \t\tfree((void *) submodule->url);\n \t\tstrbuf_addstr(&url, value);\n \t\tsubmodule->url = strbuf_detach(&url, NULL);\n+\t} else if (!strcmp(item.buf, \"update\")) {\n+\t\tif (!value)\n+\t\t\tret = config_error_nonbool(var);\n+\t\telse if (!me->overwrite && submodule->update != NULL)\n+\t\t\twarn_multiple_config(me->commit_sha1, submodule->name,\n+\t\t\t\t\t     \"update\");\n+\t\telse {\n+\t\t\tfree((void *)submodule->update);\n+\t\t\tsubmodule->update = xstrdup(value);\n+\t\t}\n \t}\n \n release_return:\ndiff --git a/submodule-config.h b/submodule-config.h\nindex 9061e4e..f9e2a29 100644\n--- a/submodule-config.h\n+++ b/submodule-config.h\n@@ -14,6 +14,7 @@ struct submodule {\n \tconst char *url;\n \tint fetch_recurse;\n \tconst char *ignore;\n+\tconst char *update;\n \t/* the sha1 blob id of the responsible .gitmodules file */\n \tunsigned char gitmodules_sha1[20];\n };\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271789","messageId":"1444960333-16003-9-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 08/12] git submodule update: check for \"none\" in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:09Z","receivedAt":"2015-10-16T01:52:09Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 38 ++++++++++++++++++++++++++++++++++++--\n git-submodule.sh            |  8 +-------\n 2 files changed, 37 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex f81f37a..73954ac 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -255,9 +255,15 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \treturn 0;\n }\n \n+static int git_submodule_config(const char *var, const char *value, void *cb)\n+{\n+\treturn parse_submodule_config_option(var, value);\n+}\n+\n static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n {\n \tint i;\n+\tchar *update = NULL;\n \tstruct pathspec pathspec;\n \tstruct module_list list = MODULE_LIST_INIT;\n \n@@ -265,6 +271,9 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\tOPT_STRING(0, \"prefix\", &prefix,\n \t\t\t   N_(\"path\"),\n \t\t\t   N_(\"alternative anchor for relative paths\")),\n+\t\tOPT_STRING(0, \"update\", &update,\n+\t\t\t   N_(\"string\"),\n+\t\t\t   N_(\"update command for submodules\")),\n \t\tOPT_END()\n \t};\n \n@@ -281,20 +290,45 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\treturn 1;\n \t}\n \n+\tgitmodules_config();\n+\t/* Overlay the parsed .gitmodules file with .git/config */\n+\tgit_config(git_submodule_config, NULL);\n+\n \tfor (i = 0; i < list.nr; i++) {\n+\t\tconst struct submodule *sub = NULL;\n+\t\tconst char *displaypath = NULL;\n \t\tconst struct cache_entry *ce = list.entries[i];\n+\t\tstruct strbuf sb = STRBUF_INIT;\n+\t\tconst char *update_module = NULL;\n \n \t\tchar *env_prefix = getenv(\"prefix\");\n \t\tif (ce_stage(ce)) {\n \t\t\tif (env_prefix)\n-\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s/%s\",\n+\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s/%s\\n\",\n \t\t\t\t\tenv_prefix, ce->name);\n \t\t\telse\n-\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s\",\n+\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s\\n\",\n \t\t\t\t\tce->name);\n \t\t\tcontinue;\n \t\t}\n \n+\t\tsub = submodule_from_path(null_sha1, ce->name);\n+\t\tif (env_prefix)\n+\t\t\tdisplaypath = relative_path(env_prefix, ce->name, &sb);\n+\t\telse\n+\t\t\tdisplaypath = ce->name;\n+\n+\t\tif (update)\n+\t\t\tupdate_module = update;\n+\t\tif (!update_module)\n+\t\t\tupdate_module = sub->update;\n+\t\tif (!update_module)\n+\t\t\tupdate_module = \"checkout\";\n+\t\tif (!strcmp(update_module, \"none\")) {\n+\t\t\tfprintf(stderr, \"Skipping submodule '%s'\\n\", displaypath);\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n \t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n \t}\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 0754ecd..227fed6 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -656,7 +656,7 @@ cmd_update()\n \tfi\n \n \tcloned_modules=\n-\tgit submodule--helper list-or-clone --prefix \"$wt_prefix\" \"$@\" | {\n+\tgit submodule--helper list-or-clone --prefix \"$wt_prefix\" ${update:+--update \"$update\"} \"$@\" | {\n \terr=\n \twhile read mode sha1 stage sm_path\n \tdo\n@@ -677,12 +677,6 @@ cmd_update()\n \n \t\tdisplaypath=$(relative_path \"$prefix$sm_path\")\n \n-\t\tif test \"$update_module\" = \"none\"\n-\t\tthen\n-\t\t\techo >&2 \"Skipping submodule '$displaypath'\"\n-\t\t\tcontinue\n-\t\tfi\n-\n \t\tif test -z \"$url\"\n \t\tthen\n \t\t\t# Only mention uninitialized submodules when its\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271787","messageId":"1444960333-16003-10-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 09/12] git submodule update: Check url in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:10Z","receivedAt":"2015-10-16T01:52:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 21 +++++++++++++++++++++\n git-submodule.sh            | 10 ----------\n 2 files changed, 21 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 73954ac..7a2fd4e 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -300,6 +300,7 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\tconst struct cache_entry *ce = list.entries[i];\n \t\tstruct strbuf sb = STRBUF_INIT;\n \t\tconst char *update_module = NULL;\n+\t\tconst char *url = NULL;\n \n \t\tchar *env_prefix = getenv(\"prefix\");\n \t\tif (ce_stage(ce)) {\n@@ -329,6 +330,26 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\t\tcontinue;\n \t\t}\n \n+\t\t/*\n+\t\t * Looking up the url in .git/config.\n+\t\t * We cannot fall back to .gitmodules as we only want to process\n+\t\t * configured submodules. This renders the submodule lookup API\n+\t\t * useless, as it cannot lookup without fallback.\n+\t\t */\n+\t\tstrbuf_reset(&sb);\n+\t\tstrbuf_addf(&sb, \"submodule.%s.url\", sub->name);\n+\t\tgit_config_get_string_const(sb.buf, &url);\n+\t\tif (!url) {\n+\t\t\t/*\n+\t\t\t * Only mention uninitialized submodules when its\n+\t\t\t * path have been specified\n+\t\t\t */\n+\t\t\tif (pathspec.nr)\n+\t\t\t\tfprintf(stderr, _(\"Submodule path '%s' not initialized\\n\"\n+\t\t\t\t\t\"Maybe you want to use 'update --init'?\"), displaypath);\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n \t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n \t}\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 227fed6..80f41b2 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -677,16 +677,6 @@ cmd_update()\n \n \t\tdisplaypath=$(relative_path \"$prefix$sm_path\")\n \n-\t\tif test -z \"$url\"\n-\t\tthen\n-\t\t\t# Only mention uninitialized submodules when its\n-\t\t\t# path have been specified\n-\t\t\ttest \"$#\" != \"0\" &&\n-\t\t\tsay >&2 \"$(eval_gettext \"Submodule path '\\$displaypath' not initialized\n-Maybe you want to use 'update --init'?\")\"\n-\t\t\tcontinue\n-\t\tfi\n-\n \t\tif ! test -d \"$sm_path\"/.git && ! test -f \"$sm_path\"/.git\n \t\tthen\n \t\t\tgit submodule--helper clone ${GIT_QUIET:+--quiet} --prefix \"$prefix\" --path \"$sm_path\" --name \"$name\" --url \"$url\" \"$reference\" \"$depth\" || exit\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271786","messageId":"1444960333-16003-11-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 10/12] git submodule update: Clone projects from within C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:11Z","receivedAt":"2015-10-16T01:52:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 56 ++++++++++++++++++++++++++++++++++++++++++++-\n git-submodule.sh            | 12 ++++++----\n 2 files changed, 63 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 7a2fd4e..d1684cf 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -260,9 +260,40 @@ static int git_submodule_config(const char *var, const char *value, void *cb)\n \treturn parse_submodule_config_option(var, value);\n }\n \n+static void fill_clone_command(struct child_process *cp, int quiet,\n+\t\t\t       const char *prefix, const char *path,\n+\t\t\t       const char *name, const char *url,\n+\t\t\t       const char *reference, const char *depth)\n+{\n+\tcp->git_cmd = 1;\n+\targv_array_push(&cp->args, \"submodule--helper\");\n+\targv_array_push(&cp->args, \"clone\");\n+\tif (quiet)\n+\t\targv_array_push(&cp->args, \"--quiet\");\n+\n+\tif (prefix) {\n+\t\targv_array_push(&cp->args, \"--prefix\");\n+\t\targv_array_push(&cp->args, prefix);\n+\t}\n+\targv_array_push(&cp->args, \"--path\");\n+\targv_array_push(&cp->args, path);\n+\n+\targv_array_push(&cp->args, \"--name\");\n+\targv_array_push(&cp->args, name);\n+\n+\targv_array_push(&cp->args, \"--url\");\n+\targv_array_push(&cp->args, url);\n+\tif (reference)\n+\t\targv_array_push(&cp->args, reference);\n+\tif (depth)\n+\t\targv_array_push(&cp->args, depth);\n+}\n+\n static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n {\n \tint i;\n+\tint quiet;\n+\tchar *reference = NULL, *depth = NULL;\n \tchar *update = NULL;\n \tstruct pathspec pathspec;\n \tstruct module_list list = MODULE_LIST_INIT;\n@@ -274,6 +305,13 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\tOPT_STRING(0, \"update\", &update,\n \t\t\t   N_(\"string\"),\n \t\t\t   N_(\"update command for submodules\")),\n+\t\tOPT_STRING(0, \"reference\", &reference, \"<repository>\",\n+\t\t\tN_(\"Use the local reference repository \"\n+\t\t\t   \"instead of a full clone\")),\n+\t\tOPT_STRING(0, \"depth\", &depth, \"<depth>\",\n+\t\t\tN_(\"Create a shallow clone truncated to the \"\n+\t\t\t   \"specified number of revisions\")),\n+\t\tOPT__QUIET(&quiet, N_(\"do't print cloning progress\")),\n \t\tOPT_END()\n \t};\n \n@@ -301,6 +339,7 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\tstruct strbuf sb = STRBUF_INIT;\n \t\tconst char *update_module = NULL;\n \t\tconst char *url = NULL;\n+\t\tint just_cloned = 0;\n \n \t\tchar *env_prefix = getenv(\"prefix\");\n \t\tif (ce_stage(ce)) {\n@@ -350,7 +389,22 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\t\tcontinue;\n \t\t}\n \n-\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n+\t\tstrbuf_reset(&sb);\n+\t\tstrbuf_addf(&sb, \"%s/.git\", ce->name);\n+\t\tjust_cloned = !file_exists(sb.buf);\n+\n+\t\tif (just_cloned) {\n+\t\t\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\t\t\tfill_clone_command(&cp, quiet, prefix, ce->name,\n+\t\t\t\t\t   sub->name, url, reference, depth);\n+\n+\t\t\tif (run_command(&cp)) {\n+\t\t\t\tprintf(\"#unmatched\\n\");\n+\t\t\t\treturn 1;\n+\t\t\t}\n+\t\t}\n+\n+\t\tprintf(\"%06o %s %d %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce), just_cloned);\n \t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n \t}\n \treturn 0;\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 80f41b2..28f1757 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -656,9 +656,14 @@ cmd_update()\n \tfi\n \n \tcloned_modules=\n-\tgit submodule--helper list-or-clone --prefix \"$wt_prefix\" ${update:+--update \"$update\"} \"$@\" | {\n+\tgit submodule--helper list-or-clone ${GIT_QUIET:+--quiet} \\\n+\t\t--prefix \"$wt_prefix\" \\\n+\t\t${update:+--update \"$update\"} \\\n+\t\t${reference:+--reference \"$reference\"} \\\n+\t\t${depth:+--depth \"$depth\"} \\\n+\t\t\"$@\" | {\n \terr=\n-\twhile read mode sha1 stage sm_path\n+\twhile read mode sha1 stage just_cloned sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n \n@@ -677,9 +682,8 @@ cmd_update()\n \n \t\tdisplaypath=$(relative_path \"$prefix$sm_path\")\n \n-\t\tif ! test -d \"$sm_path\"/.git && ! test -f \"$sm_path\"/.git\n+\t\tif test \"$just_cloned\" = 1\n \t\tthen\n-\t\t\tgit submodule--helper clone ${GIT_QUIET:+--quiet} --prefix \"$prefix\" --path \"$sm_path\" --name \"$name\" --url \"$url\" \"$reference\" \"$depth\" || exit\n \t\t\tcloned_modules=\"$cloned_modules;$name\"\n \t\t\tsubsha1=\n \t\telse\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271782","messageId":"1444960333-16003-12-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 11/12] submodule--helper: Do not emit submodules to process directly.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:12Z","receivedAt":"2015-10-16T01:52:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This will allow us to refactor the loop to use the parallel process\nAPI.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 12 ++++++++++--\n 1 file changed, 10 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex d1684cf..fa8c008 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -297,6 +297,8 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \tchar *update = NULL;\n \tstruct pathspec pathspec;\n \tstruct module_list list = MODULE_LIST_INIT;\n+\tstruct string_list projectlines = STRING_LIST_INIT_DUP;\n+\tstruct string_list_item *item;\n \n \tstruct option module_list_options[] = {\n \t\tOPT_STRING(0, \"prefix\", &prefix,\n@@ -403,9 +405,15 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\t\t\treturn 1;\n \t\t\t}\n \t\t}\n+\t\tstrbuf_reset(&sb);\n+\t\tstrbuf_addf(&sb, \"%06o %s %d %d\\t%s\\n\", ce->ce_mode,\n+\t\t\t\tsha1_to_hex(ce->sha1), ce_stage(ce),\n+\t\t\t\tjust_cloned, ce->name);\n+\t\tstring_list_append(&projectlines, sb.buf);\n+\t}\n \n-\t\tprintf(\"%06o %s %d %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce), just_cloned);\n-\t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n+\tfor_each_string_list_item(item, &projectlines) {\n+\t\tutf8_fprintf(stdout, \"%s\", item->string);\n \t}\n \treturn 0;\n }\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271788","messageId":"1444960333-16003-13-git-send-email-sbeller@google.com","threadId":"40570","inReplyTo":"1444960333-16003-1-git-send-email-sbeller@google.com","subject":"[PATCH 12/12] WIP/broken Clone all outstanding submodules in parallel","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T01:52:13Z","receivedAt":"2015-10-16T01:52:13Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Our tests scream at this patch, it's just to show what I plan to do.\nEssentially moving the content of the loop into the get_next_task\ncallback from the run_processes_parallel.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 181 +++++++++++++++++++++++++++++---------------\n 1 file changed, 119 insertions(+), 62 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex fa8c008..c66aa53 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -289,65 +289,40 @@ static void fill_clone_command(struct child_process *cp, int quiet,\n \t\targv_array_push(&cp->args, depth);\n }\n \n-static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n-{\n-\tint i;\n-\tint quiet;\n-\tchar *reference = NULL, *depth = NULL;\n-\tchar *update = NULL;\n+struct submodule_list_or_clone {\n \tstruct pathspec pathspec;\n-\tstruct module_list list = MODULE_LIST_INIT;\n-\tstruct string_list projectlines = STRING_LIST_INIT_DUP;\n-\tstruct string_list_item *item;\n-\n-\tstruct option module_list_options[] = {\n-\t\tOPT_STRING(0, \"prefix\", &prefix,\n-\t\t\t   N_(\"path\"),\n-\t\t\t   N_(\"alternative anchor for relative paths\")),\n-\t\tOPT_STRING(0, \"update\", &update,\n-\t\t\t   N_(\"string\"),\n-\t\t\t   N_(\"update command for submodules\")),\n-\t\tOPT_STRING(0, \"reference\", &reference, \"<repository>\",\n-\t\t\tN_(\"Use the local reference repository \"\n-\t\t\t   \"instead of a full clone\")),\n-\t\tOPT_STRING(0, \"depth\", &depth, \"<depth>\",\n-\t\t\tN_(\"Create a shallow clone truncated to the \"\n-\t\t\t   \"specified number of revisions\")),\n-\t\tOPT__QUIET(&quiet, N_(\"do't print cloning progress\")),\n-\t\tOPT_END()\n-\t};\n-\n-\tconst char *const git_submodule_helper_usage[] = {\n-\t\tN_(\"git submodule--helper list [--prefix=<path>] [<path>...]\"),\n-\t\tNULL\n-\t};\n-\n-\targc = parse_options(argc, argv, prefix, module_list_options,\n-\t\t\t     git_submodule_helper_usage, 0);\n-\n-\tif (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0) {\n-\t\tprintf(\"#unmatched\\n\");\n-\t\treturn 1;\n-\t}\n+\tstruct module_list list;\n+\tstruct string_list projectlines;\n+\tint count;\n+\tint quiet;\n+\tchar *reference;\n+\tchar *depth;\n+\tchar *update;\n+\tchar *env_prefix;\n+\tconst char *prefix;\n+\tint print_unmatched;\n+};\n \n-\tgitmodules_config();\n-\t/* Overlay the parsed .gitmodules file with .git/config */\n-\tgit_config(git_submodule_config, NULL);\n+static int get_next_task(void **pp_task_cb,\n+\t\t\t struct child_process *cp,\n+\t\t\t struct strbuf *err,\n+\t\t\t void *pp_cb)\n+{\n+\tstruct submodule_list_or_clone *pp = pp_cb;\n \n-\tfor (i = 0; i < list.nr; i++) {\n+\tfor (; pp->count < pp->list.nr; pp->count++) {\n \t\tconst struct submodule *sub = NULL;\n \t\tconst char *displaypath = NULL;\n-\t\tconst struct cache_entry *ce = list.entries[i];\n+\t\tconst struct cache_entry *ce = pp->list.entries[pp->count];\n \t\tstruct strbuf sb = STRBUF_INIT;\n \t\tconst char *update_module = NULL;\n \t\tconst char *url = NULL;\n \t\tint just_cloned = 0;\n \n-\t\tchar *env_prefix = getenv(\"prefix\");\n \t\tif (ce_stage(ce)) {\n-\t\t\tif (env_prefix)\n+\t\t\tif (pp->env_prefix)\n \t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s/%s\\n\",\n-\t\t\t\t\tenv_prefix, ce->name);\n+\t\t\t\t\tpp->env_prefix, ce->name);\n \t\t\telse\n \t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s\\n\",\n \t\t\t\t\tce->name);\n@@ -355,13 +330,13 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\t}\n \n \t\tsub = submodule_from_path(null_sha1, ce->name);\n-\t\tif (env_prefix)\n-\t\t\tdisplaypath = relative_path(env_prefix, ce->name, &sb);\n+\t\tif (pp->env_prefix)\n+\t\t\tdisplaypath = relative_path(pp->env_prefix, ce->name, &sb);\n \t\telse\n \t\t\tdisplaypath = ce->name;\n \n-\t\tif (update)\n-\t\t\tupdate_module = update;\n+\t\tif (pp->update)\n+\t\t\tupdate_module = pp->update;\n \t\tif (!update_module)\n \t\t\tupdate_module = sub->update;\n \t\tif (!update_module)\n@@ -385,7 +360,7 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\t\t * Only mention uninitialized submodules when its\n \t\t\t * path have been specified\n \t\t\t */\n-\t\t\tif (pathspec.nr)\n+\t\t\tif (pp->pathspec.nr)\n \t\t\t\tfprintf(stderr, _(\"Submodule path '%s' not initialized\\n\"\n \t\t\t\t\t\"Maybe you want to use 'update --init'?\"), displaypath);\n \t\t\tcontinue;\n@@ -396,23 +371,105 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n \t\tjust_cloned = !file_exists(sb.buf);\n \n \t\tif (just_cloned) {\n-\t\t\tstruct child_process cp = CHILD_PROCESS_INIT;\n-\t\t\tfill_clone_command(&cp, quiet, prefix, ce->name,\n-\t\t\t\t\t   sub->name, url, reference, depth);\n-\n-\t\t\tif (run_command(&cp)) {\n-\t\t\t\tprintf(\"#unmatched\\n\");\n-\t\t\t\treturn 1;\n-\t\t\t}\n+\t\t\tfill_clone_command(cp, pp->quiet, pp->prefix, ce->name,\n+\t\t\t\t\t   sub->name, url, pp->reference, pp->depth);\n+\t\t\treturn 1;\n \t\t}\n \t\tstrbuf_reset(&sb);\n \t\tstrbuf_addf(&sb, \"%06o %s %d %d\\t%s\\n\", ce->ce_mode,\n \t\t\t\tsha1_to_hex(ce->sha1), ce_stage(ce),\n \t\t\t\tjust_cloned, ce->name);\n-\t\tstring_list_append(&projectlines, sb.buf);\n+\t\tstring_list_append(&pp->projectlines, sb.buf);\n+\t}\n+\treturn 0;\n+}\n+\n+static int start_failure(struct child_process *cp,\n+\t\t\t struct strbuf *err,\n+\t\t\t void *pp_cb,\n+\t\t\t void *pp_task_cb)\n+{\n+\tstruct submodule_list_or_clone *pp = pp_cb;\n+\n+\tpp->print_unmatched = 1;\n+\n+\treturn 1;\n+}\n+\n+static int task_finished(int result,\n+\t\t\t struct child_process *cp,\n+\t\t\t struct strbuf *err,\n+\t\t\t void *pp_cb,\n+\t\t\t void *pp_task_cb)\n+{\n+\tstruct submodule_list_or_clone *pp = pp_cb;\n+\n+\tif (!result)\n+\t\treturn 0;\n+\telse {\n+\t\tpp->print_unmatched = 1;\n+\t\treturn 1;\n+\t}\n+}\n+\n+static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n+{\n+\tstruct submodule_list_or_clone pp;\n+\tstruct string_list_item *item;\n+\n+\tstruct option module_list_options[] = {\n+\t\tOPT_STRING(0, \"prefix\", &pp.prefix,\n+\t\t\t   N_(\"path\"),\n+\t\t\t   N_(\"alternative anchor for relative paths\")),\n+\t\tOPT_STRING(0, \"update\", &pp.update,\n+\t\t\t   N_(\"string\"),\n+\t\t\t   N_(\"update command for submodules\")),\n+\t\tOPT_STRING(0, \"reference\", &pp.reference, \"<repository>\",\n+\t\t\tN_(\"Use the local reference repository \"\n+\t\t\t   \"instead of a full clone\")),\n+\t\tOPT_STRING(0, \"depth\", &pp.depth, \"<depth>\",\n+\t\t\tN_(\"Create a shallow clone truncated to the \"\n+\t\t\t   \"specified number of revisions\")),\n+\t\tOPT__QUIET(&pp.quiet, N_(\"do't print cloning progress\")),\n+\t\tOPT_END()\n+\t};\n+\n+\tconst char *const git_submodule_helper_usage[] = {\n+\t\tN_(\"git submodule--helper list [--prefix=<path>] [<path>...]\"),\n+\t\tNULL\n+\t};\n+\n+\tpp.prefix = NULL;\n+\tpp.list.entries = NULL;\n+\tpp.list.alloc = 0;\n+\tpp.list.nr = 0;\n+\tstring_list_init(&pp.projectlines, 1);\n+\tpp.count = 0;\n+\tpp.reference = NULL;\n+\tpp.depth = NULL;\n+\tpp.update = NULL;\n+\tpp.env_prefix = getenv(\"prefix\");\n+\tpp.print_unmatched = 0;\n+\n+\targc = parse_options(argc, argv, prefix, module_list_options,\n+\t\t\t     git_submodule_helper_usage, 0);\n+\n+\tif (module_list_compute(argc, argv, pp.prefix, &pp.pathspec, &pp.list) < 0) {\n+\t\tprintf(\"#unmatched\\n\");\n+\t\treturn 1;\n+\t}\n+\n+\tgitmodules_config();\n+\t/* Overlay the parsed .gitmodules file with .git/config */\n+\tgit_config(git_submodule_config, NULL);\n+\n+\trun_processes_parallel(1, get_next_task, start_failure, task_finished, &pp);\n+\tif (pp.print_unmatched) {\n+\t\tprintf(\"#unmatched\\n\");\n+\t\treturn 1;\n \t}\n \n-\tfor_each_string_list_item(item, &projectlines) {\n+\tfor_each_string_list_item(item, &pp.projectlines) {\n \t\tutf8_fprintf(stdout, \"%s\", item->string);\n \t}\n \treturn 0;\n-- \n2.5.0.277.gfdc362b.dirty\n"},{"id":"271833","messageId":"xmqqbnbyeei6.fsf@gitster.mtv.corp.google.com","threadId":"40570","inReplyTo":"1444960333-16003-2-git-send-email-sbeller@google.com","subject":"Re: [PATCH 01/12] git submodule update: Announce skipping submodules on stderr","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-16T20:37:37Z","receivedAt":"2015-10-16T20:37:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  git-submodule.sh | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 8b0eb9a..578ec48 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -684,7 +684,7 @@ cmd_update()\n>  \n>  \t\tif test \"$update_module\" = \"none\"\n>  \t\tthen\n> -\t\t\techo \"Skipping submodule '$displaypath'\"\n> +\t\t\techo >&2 \"Skipping submodule '$displaypath'\"\n>  \t\t\tcontinue\n>  \t\tfi\n\nMakes sense, but see 02/12.\n"},{"id":"271834","messageId":"CAGZ79kZH1sdboAzFCjAUKtg0sJcPZGjzVrHPa_r8dG4bK4uT-w@mail.gmail.com","threadId":"40570","inReplyTo":"xmqqbnbyeei6.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 01/12] git submodule update: Announce skipping submodules on stderr","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T20:47:13Z","receivedAt":"2015-10-16T20:47:13Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Oct 16, 2015 at 1:37 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>>  git-submodule.sh | 2 +-\n>>  1 file changed, 1 insertion(+), 1 deletion(-)\n>>\n>> diff --git a/git-submodule.sh b/git-submodule.sh\n>> index 8b0eb9a..578ec48 100755\n>> --- a/git-submodule.sh\n>> +++ b/git-submodule.sh\n>> @@ -684,7 +684,7 @@ cmd_update()\n>>\n>>               if test \"$update_module\" = \"none\"\n>>               then\n>> -                     echo \"Skipping submodule '$displaypath'\"\n>> +                     echo >&2 \"Skipping submodule '$displaypath'\"\n>>                       continue\n>>               fi\n>\n> Makes sense, but see 02/12.\n\nThe patch (I can't see a reply there) ?\n\nI split them on purpose. This one uses echo as opposed to say and has\nno tests to fail.\n\nSo this patch documents, that there are no breaking tests. I can just change it\n2/12 tells another story: We codified the behavior in tests and rely on it, so\nwe need to carefully decide if that's a breaking change.\n\n\n\n>\n"},{"id":"271835","messageId":"xmqq7fmmedq2.fsf@gitster.mtv.corp.google.com","threadId":"40570","inReplyTo":"1444960333-16003-3-git-send-email-sbeller@google.com","subject":"Re: [PATCH 02/12] git submodule update: Announce uninitialized modules on stderr","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-16T20:54:29Z","receivedAt":"2015-10-16T20:54:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  git-submodule.sh           |  2 +-\n>  t/t7400-submodule-basic.sh | 12 ++++++------\n>  2 files changed, 7 insertions(+), 7 deletions(-)\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index 578ec48..eea27f8 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -693,7 +693,7 @@ cmd_update()\n>  \t\t\t# Only mention uninitialized submodules when its\n>  \t\t\t# path have been specified\n>  \t\t\ttest \"$#\" != \"0\" &&\n> -\t\t\tsay \"$(eval_gettext \"Submodule path '\\$displaypath' not initialized\n> +\t\t\tsay >&2 \"$(eval_gettext \"Submodule path '\\$displaypath' not initialized\n>  Maybe you want to use 'update --init'?\")\"\n>  \t\t\tcontinue\n>  \t\tfi\n\nThere are quite a few other calls to \"say\" in this script, and you\nare changing only this one to emit it to the standard error output.\n\nMy quick eyeballing of the script tells me that most of them, other\nthan the ones that are used in cmd_status to report the information\nthat the user asked to be shown on the standard output, are of \"Now\nI am doing this\" kind fo output, which I feel are the same category\nas this one that shouldn't be on the standard output.\n\nAnother thing (which relates to the one in 01/12) is that not all\noutput from this command comes from \"say\".\n\nPerhaps the first thing to do before doing 01/12 is to sift these\nmessages into types and have them consistently use helpers designed\nfor different purposes, e.g.\n\n - a progress, like this one, the one in 01/12, and many other uses\n   of \"say\"; which may want to become e.g. \"say_progress\".\n\n - an error or a warning, like \"Could not remove working tree\", \"not\n   initialized, maybe you want to do 'init' first?\"; which may want\n   to become something else e.g. \"say_warning\".\n\n - the real output from the program, e.g. output from cmd_status,\n   would use yet another, e.g. \"printf '%s\\n'\".\n\ninstead of converting each message that you happened to have noticed.\n\nNote that \"say\" is squelched under GIT_QUIET (i.e. --quiet).  The\nformer two helpers we would want to make quiet (or for errors we may\nnot---I don't know offhand).  I do not think of any valid reason why\nwe want to squelch the output from cmd_status under --quiet; it is\nnot like the the while loop on the downstream of \"list |\" pipe tells\nsome status via its exit code.\n"},{"id":"271836","messageId":"xmqq37xaedpf.fsf@gitster.mtv.corp.google.com","threadId":"40570","inReplyTo":"1444960333-16003-4-git-send-email-sbeller@google.com","subject":"Re: [PATCH 03/12] git submodule update: Move branch calculation to where it's needed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-16T20:54:52Z","receivedAt":"2015-10-16T20:54:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> The branch variable is used only once so calculate it only when needed.\n>\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n\nMakes sense.\n\n>  git-submodule.sh | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index eea27f8..56a0524 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -668,7 +668,6 @@ cmd_update()\n>  \t\tfi\n>  \t\tname=$(git submodule--helper name \"$sm_path\") || exit\n>  \t\turl=$(git config submodule.\"$name\".url)\n> -\t\tbranch=$(get_submodule_config \"$name\" branch master)\n>  \t\tif ! test -z \"$update\"\n>  \t\tthen\n>  \t\t\tupdate_module=$update\n> @@ -718,6 +717,7 @@ Maybe you want to use 'update --init'?\")\"\n>  \t\t\t\tdie \"$(eval_gettext \"Unable to fetch in submodule path '\\$sm_path'\")\"\n>  \t\t\tfi\n>  \t\t\tremote_name=$(clear_local_git_env; cd \"$sm_path\" && get_default_remote)\n> +\t\t\tbranch=$(get_submodule_config \"$name\" branch master)\n>  \t\t\tsha1=$(clear_local_git_env; cd \"$sm_path\" &&\n>  \t\t\t\tgit rev-parse --verify \"${remote_name}/${branch}\") ||\n>  \t\t\tdie \"$(eval_gettext \"Unable to find current ${remote_name}/${branch} revision in submodule path '\\$sm_path'\")\"\n"},{"id":"271837","messageId":"xmqqy4f2cysl.fsf@gitster.mtv.corp.google.com","threadId":"40570","inReplyTo":"1444960333-16003-6-git-send-email-sbeller@google.com","subject":"Re: [PATCH 05/12] git submodule update: Use its own list implementation.","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-16T21:02:18Z","receivedAt":"2015-10-16T21:02:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Discussions turned out that we cannot parallelize the whole loop below\n> `git submodule--helper list` in `git submodule update`, because some\n> changes should be done only one at a time, such as messing up a submodule\n> and leave it up to the user to cleanup the conflicted rebase or merge.\n>\n> The submodules which are need to be cloned however do not expect to create\n> problems which require attention by the user one at a time, so we want to\n> parallelize that first.\n>\n> To do so we will start with a literal copy of `git submodule--helper list`\n> and port over features gradually.\n\nI am not sure what you mean by this.\n\nSurely, the current implementation of \"update\" does the fetching and\nupdating as a single unit of task and iterate over these tasks, and\nwe would rather want to instead have one iteration of submodules to\ndo the fetching part (without doing other things that can fail and\nhave to get attention of the end user), followed by another\niteration that does the \"other things\", in order to get closer to\nthe end goal of doing the fetch in parallel and then doing the\nremainder one-module-at-a-time sequencially.\n\nI would imagine that the logical first step towards the end goal, if\nI understood you correctly, would be to split that single large loop\nthat does a fetch and other things for a single module in each\niteration into two, one that iterates and fetches all, followed by a\nnew one that does the checkout/merge.\n\nWhat I do not understand is why that requires a different kind of\nenumerator (unless this is a kind of premature optimization, knowing\nthat the set of modules iterated by these two loops are slightly\ndifferent or something).\n\n>  int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index bb8b2c7..d2d80e2 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -656,7 +656,7 @@ cmd_update()\n>  \tfi\n>  \n>  \tcloned_modules=\n> -\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" | {\n> +\tgit submodule--helper list-or-clone --prefix \"$wt_prefix\" \"$@\" | {\n>  \terr=\n>  \twhile read mode sha1 stage sm_path\n>  \tdo\n"},{"id":"271838","messageId":"CAGZ79kY+_MH5Y9RF7ZdY1j7WQFpM+97_GROwuMFO6034SCvf6Q@mail.gmail.com","threadId":"40570","inReplyTo":"xmqqy4f2cysl.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 05/12] git submodule update: Use its own list implementation.","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-16T21:08:22Z","receivedAt":"2015-10-16T21:08:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Oct 16, 2015 at 2:02 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> Discussions turned out that we cannot parallelize the whole loop below\n>> `git submodule--helper list` in `git submodule update`, because some\n>> changes should be done only one at a time, such as messing up a submodule\n>> and leave it up to the user to cleanup the conflicted rebase or merge.\n>>\n>> The submodules which are need to be cloned however do not expect to create\n>> problems which require attention by the user one at a time, so we want to\n>> parallelize that first.\n>>\n>> To do so we will start with a literal copy of `git submodule--helper list`\n>> and port over features gradually.\n>\n> I am not sure what you mean by this.\n>\n> Surely, the current implementation of \"update\" does the fetching and\n> updating as a single unit of task and iterate over these tasks, and\n> we would rather want to instead have one iteration of submodules to\n> do the fetching part (without doing other things that can fail and\n> have to get attention of the end user), followed by another\n> iteration that does the \"other things\", in order to get closer to\n> the end goal of doing the fetch in parallel and then doing the\n> remainder one-module-at-a-time sequencially.\n\nI differentiated a bit more and moved the clone parts only.\nFetching should also be no problem. I initially assumed that to be a\nproblem too.\n\n>\n> I would imagine that the logical first step towards the end goal, if\n> I understood you correctly, would be to split that single large loop\n> that does a fetch and other things for a single module in each\n> iteration into two, one that iterates and fetches all, followed by a\n> new one that does the checkout/merge.\n\nThat was also one of the patch series I wrote (not sent to list)\n1) split up into 2 phases\n2) rewrite first phase in C\n3) parallelize the first phase.\n\nThis series merges 1 and 2, so you don't have to review\nthe same functionality two times.\n\n>\n> What I do not understand is why that requires a different kind of\n> enumerator (unless this is a kind of premature optimization, knowing\n> that the set of modules iterated by these two loops are slightly\n> different or something).\n\nIt is just moving all code before the clone step into the C part, so\nwe can call the clone in C.\n\n>\n>>  int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\n>> diff --git a/git-submodule.sh b/git-submodule.sh\n>> index bb8b2c7..d2d80e2 100755\n>> --- a/git-submodule.sh\n>> +++ b/git-submodule.sh\n>> @@ -656,7 +656,7 @@ cmd_update()\n>>       fi\n>>\n>>       cloned_modules=\n>> -     git submodule--helper list --prefix \"$wt_prefix\" \"$@\" | {\n>> +     git submodule--helper list-or-clone --prefix \"$wt_prefix\" \"$@\" | {\n>>       err=\n>>       while read mode sha1 stage sm_path\n>>       do\n"},{"id":"271995","messageId":"xmqqlhax6y9s.fsf@gitster.mtv.corp.google.com","threadId":"40570","inReplyTo":"1444960333-16003-7-git-send-email-sbeller@google.com","subject":"Re: [PATCH 06/12] git submodule update: Handle unmerged submodules in C","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-10-20T21:11:27Z","receivedAt":"2015-10-20T21:11:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n> Signed-off-by: Stefan Beller <sbeller@google.com>\n> ---\n>  builtin/submodule--helper.c | 15 +++++++++++----\n>  git-submodule.sh            |  6 +-----\n>  2 files changed, 12 insertions(+), 9 deletions(-)\n>\n> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n> index 47dc9cb..f81f37a 100644\n> --- a/builtin/submodule--helper.c\n> +++ b/builtin/submodule--helper.c\n> @@ -284,11 +284,18 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n>  \tfor (i = 0; i < list.nr; i++) {\n>  \t\tconst struct cache_entry *ce = list.entries[i];\n>  \n> -\t\tif (ce_stage(ce))\n> -\t\t\tprintf(\"%06o %s U\\t\", ce->ce_mode, sha1_to_hex(null_sha1));\n> -\t\telse\n> -\t\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n> +\t\tchar *env_prefix = getenv(\"prefix\");\n\nThis somehow makes me feel dirty.  Do we really export such an\nenvironment variable that is named overly generically to communicate\nwith our own helpers?\n\nI can see why you need to be able to prefix leading paths (i.e. you\nwould need to prefix path to the enclosing submodule to a path to\nobtain the \"global view\" from the very top-level superproject while\nrecursing into nested submodules), but still...\n\n> +\t\tif (ce_stage(ce)) {\n> +\t\t\tif (env_prefix)\n> +\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s/%s\",\n> +\t\t\t\t\tenv_prefix, ce->name);\n> +\t\t\telse\n> +\t\t\t\tfprintf(stderr, \"Skipping unmerged submodule %s\",\n> +\t\t\t\t\tce->name);\n> +\t\t\tcontinue;\n> +\t\t}\n>  \n> +\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n>  \t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n>  \t}\n>  \treturn 0;\n> diff --git a/git-submodule.sh b/git-submodule.sh\n> index d2d80e2..0754ecd 100755\n> --- a/git-submodule.sh\n> +++ b/git-submodule.sh\n> @@ -661,11 +661,7 @@ cmd_update()\n>  \twhile read mode sha1 stage sm_path\n>  \tdo\n>  \t\tdie_if_unmatched \"$mode\"\n> -\t\tif test \"$stage\" = U\n> -\t\tthen\n> -\t\t\techo >&2 \"Skipping unmerged submodule $prefix$sm_path\"\n> -\t\t\tcontinue\n> -\t\tfi\n> +\n>  \t\tname=$(git submodule--helper name \"$sm_path\") || exit\n>  \t\turl=$(git config submodule.\"$name\".url)\n>  \t\tif ! test -z \"$update\"\n"},{"id":"271996","messageId":"CAGZ79kZKZhE_LyM5yvjrGXcxQXmN0F_2Jnag2PsAXAboO6qdzw@mail.gmail.com","threadId":"40570","inReplyTo":"xmqqlhax6y9s.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 06/12] git submodule update: Handle unmerged submodules in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-10-20T21:21:30Z","receivedAt":"2015-10-20T21:21:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Oct 20, 2015 at 2:11 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>> Signed-off-by: Stefan Beller <sbeller@google.com>\n>> ---\n>>  builtin/submodule--helper.c | 15 +++++++++++----\n>>  git-submodule.sh            |  6 +-----\n>>  2 files changed, 12 insertions(+), 9 deletions(-)\n>>\n>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\n>> index 47dc9cb..f81f37a 100644\n>> --- a/builtin/submodule--helper.c\n>> +++ b/builtin/submodule--helper.c\n>> @@ -284,11 +284,18 @@ static int module_list_or_clone(int argc, const char **argv, const char *prefix)\n>>       for (i = 0; i < list.nr; i++) {\n>>               const struct cache_entry *ce = list.entries[i];\n>>\n>> -             if (ce_stage(ce))\n>> -                     printf(\"%06o %s U\\t\", ce->ce_mode, sha1_to_hex(null_sha1));\n>> -             else\n>> -                     printf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n>> +             char *env_prefix = getenv(\"prefix\");\n>\n\n[Just checked the date, it's the old series. I am about to send out a new\nseries which collapses some patches in here, is on top of the fixes series and\noff course fixes this issue ;) ]\n\n> This somehow makes me feel dirty.  Do we really export such an\n> environment variable that is named overly generically to communicate\n> with our own helpers?\n\nI agree that this is bad. It was the fastest way.\nI should have taken the slower road. I think I'll replace this with\nanother argument.\n\n>\n> I can see why you need to be able to prefix leading paths (i.e. you\n> would need to prefix path to the enclosing submodule to a path to\n> obtain the \"global view\" from the very top-level superproject while\n> recursing into nested submodules), but still...\n>\n>> +             if (ce_stage(ce)) {\n>> +                     if (env_prefix)\n>> +                             fprintf(stderr, \"Skipping unmerged submodule %s/%s\",\n>> +                                     env_prefix, ce->name);\n>> +                     else\n>> +                             fprintf(stderr, \"Skipping unmerged submodule %s\",\n>> +                                     ce->name);\n>> +                     continue;\n>> +             }\n>>\n>> +             printf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n>>               utf8_fprintf(stdout, \"%s\\n\", ce->name);\n>>       }\n>>       return 0;\n>> diff --git a/git-submodule.sh b/git-submodule.sh\n>> index d2d80e2..0754ecd 100755\n>> --- a/git-submodule.sh\n>> +++ b/git-submodule.sh\n>> @@ -661,11 +661,7 @@ cmd_update()\n>>       while read mode sha1 stage sm_path\n>>       do\n>>               die_if_unmatched \"$mode\"\n>> -             if test \"$stage\" = U\n>> -             then\n>> -                     echo >&2 \"Skipping unmerged submodule $prefix$sm_path\"\n>> -                     continue\n>> -             fi\n>> +\n>>               name=$(git submodule--helper name \"$sm_path\") || exit\n>>               url=$(git config submodule.\"$name\".url)\n>>               if ! test -z \"$update\"\n"}]}