{"thread":{"id":"25705","subject":"[PATCH v3 0/3] Teach fetch and pull to recursively fetch submodules","startedAt":"2010-11-10T23:53:07Z","lastAt":"2010-12-10T18:03:42Z","messageCount":24,"participants":["Jens Lehmann","Jonathan Nieder","Junio C Hamano"],"isPatch":true,"patchVersion":3,"patchTotal":3},"messages":[{"id":"155632","messageId":"4CDB3063.5010801@web.de","threadId":"25705","inReplyTo":null,"subject":"[PATCH v3 0/3] Teach fetch and pull to recursively fetch submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-10T23:53:07Z","receivedAt":"2010-11-10T23:53:07Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"So here is the third iteration of the recursive fetch/pull series.\n\nChanges since v2:\n\n* The default now is to not recurse into submodules (also see \"Still\n to do\")\n\n* The command line option has been renamed to \"--recurse-submodules\" as\n proposed at the GitTogether\n\n* The config options are named \"submodule..fetchRecurseSubmodules\"\n and \"fetch.recurseSubmodules\" now\n\n* Switched the order of the two patches adding the config options\n\n\nQuestion:\n\n* Should the \"--submodule-prefix\" option - which is only used internally\n now - be a hidden option to \"git fetch\"?\n\n\nStill to do:\n\n* Add a mode to only fetch those submodules where new recorded commits\n are fetched in the superproject (and maybe later even fetch only those\n commits in the submodule which are referenced in the superprojects\n fetch). This could be a sane default - especially for projects having\n lots of submodules - and could be enabled by using then to be added\n \"--recurse-submodules=changed\" and \"fetch[.]recurseSubmodules=changed\"\n options where the configuration uses other defaults (I'm not really\n convinced yet 'changed' is a very good name but couldn't come up with\n a better one yet. So suggestions for alternatives are very welcome :-)\n\n* Boost performance by concurrently fetching submodules (after my first\n experiments this must be configurable, e.g. how many fetch commands\n to run at the same time, as some git servers barf on too many fetches\n run at the same time)\n\n\nBut nonetheless I think this patch series is ok for inclusion as it does\nnot change default behavior and gives people the opportunity to play with\nrecursive fetch/pull by enabling one of the introduced config options.\n\n\nJens Lehmann (3):\n fetch/pull: Add the --recurse-submodules option\n Add the 'fetch.recurseSubmodules' config setting\n Submodules: Add the \"fetchRecurseSubmodules\" config option\n\n Documentation/config.txt | 12 +++\n Documentation/fetch-options.txt | 13 +++\n Documentation/gitmodules.txt | 8 ++\n builtin/fetch.c | 64 ++++++++++---\n git-pull.sh | 10 ++-\n submodule.c | 98 +++++++++++++++++++-\n submodule.h | 5 +\n t/t5526-fetch-submodules.sh | 195 +++++++++++++++++++++++++++++++++++++++\n 8 files changed, 388 insertions(+), 17 deletions(-)\n create mode 100755 t/t5526-fetch-submodules.sh\n\n-- \n1.7.3.2.337.g9376c\n"},{"id":"155633","messageId":"4CDB30AB.8020107@web.de","threadId":"25705","inReplyTo":"4CDB3063.5010801@web.de","subject":"[PATCH v3 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-10T23:54:19Z","receivedAt":"2010-11-10T23:54:19Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Until now you had to call \"git submodule update\" (without -N|--no-fetch\noption) or something like \"git submodule foreach git fetch\" to fetch\nnew commits in populated submodules from their remote.\n\nThis could lead to \"(commits not present)\" messages in the output of\n\"git diff --submodule\" (which is used by \"git gui\" and \"gitk\") after\nfetching or pulling new commits in the superproject and is an obstacle for\nimplementing recursive checkout of submodules. Also \"git submodule\nupdate\" cannot fetch changes when disconnected, so it was very easy to\nforget to fetch the submodule changes before disconnecting only to\ndiscover later that they are needed.\n\nThis patch adds the \"--recurse-submodules\" option to recursively fetch\neach populated submodule from the url configured in the .git/config of the\nsubmodule at the end of each \"git fetch\" or during \"git pull\" in the\nsuperproject. The submodule paths are taken from the index.\n\nThe option \"--submodule-prefix\" is added to \"git fetch\" to be able to\nprint out the full paths of nested submodules.\n\nSigned-off-by: Jens Lehmann \n---\n Documentation/fetch-options.txt | 12 ++++\n builtin/fetch.c | 53 ++++++++++++++-----\n git-pull.sh | 7 ++-\n submodule.c | 66 +++++++++++++++++++++++-\n submodule.h | 3 +\n t/t5526-fetch-submodules.sh | 109 +++++++++++++++++++++++++++++++++++++++\n 6 files changed, 233 insertions(+), 17 deletions(-)\n create mode 100755 t/t5526-fetch-submodules.sh\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 5ce1e72..bd36466 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -66,6 +66,18 @@ ifndef::git-pull[]\n \tlinkgit:git-config[1].\n endif::git-pull[]\n\n+--recurse-submodules::\n+\tUse this option to fetch new commits of all populated submodules too.\n+\n+ifndef::git-pull[]\n+--submodule-prefix::\n+\tWhen recursing into nested submodules it is not sufficient to just\n+\tprint out the submodule path, all directories already traversed from\n+\tthe superproject have to be printed too. This option is used to provide\n+\tthat information, its value is prepended to the local submodule path\n+\tbefore it is printed out.\n+endif::git-pull[]\n+\n -u::\n --update-head-ok::\n \tBy default 'git fetch' refuses to update the head which\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex d35f000..658bb4c 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -12,6 +12,7 @@\n #include \"parse-options.h\"\n #include \"sigchain.h\"\n #include \"transport.h\"\n+#include \"submodule.h\"\n\n static const char * const builtin_fetch_usage[] = {\n \t\"git fetch [] [ [...]]\",\n@@ -28,12 +29,13 @@ enum {\n };\n\n static int all, append, dry_run, force, keep, multiple, prune, update_head_ok, verbosity;\n-static int progress;\n+static int progress, recurse_submodules;\n static int tags = TAGS_DEFAULT;\n static const char *depth;\n static const char *upload_pack;\n static struct strbuf default_rla = STRBUF_INIT;\n static struct transport *transport;\n+static const char *submodule_prefix = \"\";\n\n static struct option builtin_fetch_options[] = {\n \tOPT__VERBOSITY(&verbosity),\n@@ -53,6 +55,8 @@ static struct option builtin_fetch_options[] = {\n \t\t \"do not fetch all tags (--no-tags)\", TAGS_UNSET),\n \tOPT_BOOLEAN('p', \"prune\", &prune,\n \t\t \"prune tracking branches no longer on remote\"),\n+\tOPT_BOOLEAN(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t \"control recursive fetching of submodules\"),\n \tOPT_BOOLEAN(0, \"dry-run\", &dry_run,\n \t\t \"dry run\"),\n \tOPT_BOOLEAN('k', \"keep\", &keep, \"keep downloaded pack\"),\n@@ -61,6 +65,8 @@ static struct option builtin_fetch_options[] = {\n \tOPT_BOOLEAN(0, \"progress\", &progress, \"force progress reporting\"),\n \tOPT_STRING(0, \"depth\", &depth, \"DEPTH\",\n \t\t \"deepen history of shallow clone\"),\n+\tOPT_STRING(0, \"submodule-prefix\", &submodule_prefix, \"DIR\",\n+\t\t \"prepend this to submodule path output\"),\n \tOPT_END()\n };\n\n@@ -784,28 +790,36 @@ static int add_remote_or_group(const char *name, struct string_list *list)\n \treturn 1;\n }\n\n-static int fetch_multiple(struct string_list *list)\n+static void add_options_to_argv(int *argc, const char **argv)\n {\n-\tint i, result = 0;\n-\tconst char *argv[11] = { \"fetch\", \"--append\" };\n-\tint argc = 2;\n-\n \tif (dry_run)\n-\t\targv[argc++] = \"--dry-run\";\n+\t\targv[(*argc)++] = \"--dry-run\";\n \tif (prune)\n-\t\targv[argc++] = \"--prune\";\n+\t\targv[(*argc)++] = \"--prune\";\n \tif (update_head_ok)\n-\t\targv[argc++] = \"--update-head-ok\";\n+\t\targv[(*argc)++] = \"--update-head-ok\";\n \tif (force)\n-\t\targv[argc++] = \"--force\";\n+\t\targv[(*argc)++] = \"--force\";\n \tif (keep)\n-\t\targv[argc++] = \"--keep\";\n+\t\targv[(*argc)++] = \"--keep\";\n+\tif (recurse_submodules)\n+\t\targv[(*argc)++] = \"--recurse-submodules\";\n \tif (verbosity >= 2)\n-\t\targv[argc++] = \"-v\";\n+\t\targv[(*argc)++] = \"-v\";\n \tif (verbosity >= 1)\n-\t\targv[argc++] = \"-v\";\n+\t\targv[(*argc)++] = \"-v\";\n \telse if (verbosity < 0)\n-\t\targv[argc++] = \"-q\";\n+\t\targv[(*argc)++] = \"-q\";\n+\n+}\n+\n+static int fetch_multiple(struct string_list *list)\n+{\n+\tint i, result = 0;\n+\tconst char *argv[12] = { \"fetch\", \"--append\" };\n+\tint argc = 2;\n+\n+\tadd_options_to_argv(&argc, argv);\n\n \tif (!append && !dry_run) {\n \t\tint errcode = truncate_fetch_head();\n@@ -926,6 +940,17 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \t\t}\n \t}\n\n+\tif (!result && recurse_submodules) {\n+\t\tconst char *options[10];\n+\t\tint num_options = 0;\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tadd_options_to_argv(&num_options, options);\n+\t\tresult = fetch_populated_submodules(num_options, options,\n+\t\t\t\t\t\t submodule_prefix,\n+\t\t\t\t\t\t verbosity < 0);\n+\t}\n+\n \t/* All names were strdup()ed or strndup()ed */\n \tlist.strdup_strings = 1;\n \tstring_list_clear(&list, 0);\ndiff --git a/git-pull.sh b/git-pull.sh\nindex 8eb74d4..5804c62 100755\n--- a/git-pull.sh\n+++ b/git-pull.sh\n@@ -38,7 +38,7 @@ test -z \"$(git ls-files -u)\" || die_conflict\n test -f \"$GIT_DIR/MERGE_HEAD\" && die_merge\n\n strategy_args= diffstat= no_commit= squash= no_ff= ff_only=\n-log_arg= verbosity= progress=\n+log_arg= verbosity= progress= recurse_submodules=\n merge_args=\n curr_branch=$(git symbolic-ref -q HEAD)\n curr_branch_short=\"${curr_branch#refs/heads/}\"\n@@ -105,6 +105,9 @@ do\n \t--no-r|--no-re|--no-reb|--no-reba|--no-rebas|--no-rebase)\n \t\trebase=false\n \t\t;;\n+\t--recurse-submodules)\n+\t\trecurse_submodules=--recurse-submodules\n+\t\t;;\n \t--d|--dr|--dry|--dry-|--dry-r|--dry-ru|--dry-run)\n \t\tdry_run=--dry-run\n \t\t;;\n@@ -220,7 +223,7 @@ test true = \"$rebase\" && {\n \tdone\n }\n orig_head=$(git rev-parse -q --verify HEAD)\n-git fetch $verbosity $progress $dry_run --update-head-ok \"$@\" || exit 1\n+git fetch $verbosity $progress $dry_run $recurse_submodules --update-head-ok \"$@\" || exit 1\n test -z \"$dry_run\" || exit 0\n\n curr_head=$(git rev-parse -q --verify HEAD)\ndiff --git a/submodule.c b/submodule.c\nindex 91a4758..4d9b774 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -63,7 +63,7 @@ void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n \t}\n }\n\n-static int submodule_config(const char *var, const char *value, void *cb)\n+int submodule_config(const char *var, const char *value, void *cb)\n {\n \tif (!prefixcmp(var, \"submodule.\"))\n \t\treturn parse_submodule_config_option(var, value);\n@@ -229,6 +229,70 @@ void show_submodule_summary(FILE *f, const char *path,\n \tstrbuf_release(&sb);\n }\n\n+int fetch_populated_submodules(int num_options, const char **options,\n+\t\t\t const char *prefix, int quiet)\n+{\n+\tint i, result = 0, argc = 0;\n+\tstruct child_process cp;\n+\tconst char **argv;\n+\tstruct string_list_item *name_for_path;\n+\tconst char *work_tree = get_git_work_tree();\n+\tif (!work_tree)\n+\t\treturn 0;\n+\n+\tif (!the_index.initialized)\n+\t\tif (read_cache() < 0)\n+\t\t\tdie(\"index file corrupt\");\n+\n+\targv = xcalloc(num_options + 5, sizeof(const char *));\n+\targv[argc++] = \"fetch\";\n+\tfor (i = 0; i < num_options; i++)\n+\t\targv[argc++] = options[i];\n+\targv[argc++] = \"--submodule-prefix\";\n+\n+\tmemset(&cp, 0, sizeof(cp));\n+\tcp.argv = argv;\n+\tcp.env = local_repo_env;\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\n+\tfor (i = 0; i < active_nr; i++) {\n+\t\tstruct strbuf submodule_path = STRBUF_INIT;\n+\t\tstruct strbuf submodule_git_dir = STRBUF_INIT;\n+\t\tstruct strbuf submodule_prefix = STRBUF_INIT;\n+\t\tstruct cache_entry *ce = active_cache[i];\n+\t\tconst char *git_dir, *name;\n+\n+\t\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\t\tcontinue;\n+\n+\t\tname = ce->name;\n+\t\tname_for_path = unsorted_string_list_lookup(&config_name_for_path, ce->name);\n+\t\tif (name_for_path)\n+\t\t\tname = name_for_path->util;\n+\n+\t\tstrbuf_addf(&submodule_path, \"%s/%s\", work_tree, ce->name);\n+\t\tstrbuf_addf(&submodule_git_dir, \"%s/.git\", submodule_path.buf);\n+\t\tstrbuf_addf(&submodule_prefix, \"%s%s/\", prefix, ce->name);\n+\t\tgit_dir = read_gitfile_gently(submodule_git_dir.buf);\n+\t\tif (!git_dir)\n+\t\t\tgit_dir = submodule_git_dir.buf;\n+\t\tif (is_directory(git_dir)) {\n+\t\t\tif (!quiet)\n+\t\t\t\tprintf(\"Fetching submodule %s%s\\n\", prefix, ce->name);\n+\t\t\tcp.dir = submodule_path.buf;\n+\t\t\targv[argc] = submodule_prefix.buf;\n+\t\t\tif (run_command(&cp))\n+\t\t\t\tresult = 1;\n+\t\t}\n+\t\tstrbuf_release(&submodule_path);\n+\t\tstrbuf_release(&submodule_git_dir);\n+\t\tstrbuf_release(&submodule_prefix);\n+\t}\n+\tfree(argv);\n+\treturn result;\n+}\n+\n unsigned is_submodule_modified(const char *path, int ignore_untracked)\n {\n \tssize_t len;\ndiff --git a/submodule.h b/submodule.h\nindex 386f410..b39d7a1 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -5,6 +5,7 @@ struct diff_options;\n\n void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n \t\tconst char *path);\n+int submodule_config(const char *var, const char *value, void *cb);\n void gitmodules_config();\n int parse_submodule_config_option(const char *var, const char *value);\n void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);\n@@ -12,6 +13,8 @@ void show_submodule_summary(FILE *f, const char *path,\n \t\tunsigned char one[20], unsigned char two[20],\n \t\tunsigned dirty_submodule,\n \t\tconst char *del, const char *add, const char *reset);\n+int fetch_populated_submodules(int num_options, const char **options,\n+\t\t\t const char *prefix, int quiet);\n unsigned is_submodule_modified(const char *path, int ignore_untracked);\n int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],\n \t\t const unsigned char a[20], const unsigned char b[20]);\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nnew file mode 100755\nindex 0000000..107317a\n--- /dev/null\n+++ b/t/t5526-fetch-submodules.sh\n@@ -0,0 +1,109 @@\n+#!/bin/sh\n+# Copyright (c) 2010, Jens Lehmann\n+\n+test_description='Recursive \"git fetch\" for submodules'\n+\n+. ./test-lib.sh\n+\n+pwd=$(pwd)\n+\n+add_upstream_commit() {\n+\t(\n+\t\tcd submodule &&\n+\t\thead1=$(git rev-parse --short HEAD) &&\n+\t\techo new >> subfile &&\n+\t\ttest_tick &&\n+\t\tgit add subfile &&\n+\t\tgit commit -m new subfile &&\n+\t\thead2=$(git rev-parse --short HEAD) &&\n+\t\techo \"From $pwd/submodule\" > ../expect.err &&\n+\t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n+\t)\n+\t(\n+\t\tcd deepsubmodule &&\n+\t\thead1=$(git rev-parse --short HEAD) &&\n+\t\techo new >> deepsubfile &&\n+\t\ttest_tick &&\n+\t\tgit add deepsubfile &&\n+\t\tgit commit -m new deepsubfile &&\n+\t\thead2=$(git rev-parse --short HEAD) &&\n+\t\techo \"From $pwd/deepsubmodule\" >> ../expect.err &&\n+\t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n+\t)\n+}\n+\n+test_expect_success setup '\n+\tmkdir deepsubmodule &&\n+\t(\n+\t\tcd deepsubmodule &&\n+\t\tgit init &&\n+\t\techo deepsubcontent > deepsubfile &&\n+\t\tgit add deepsubfile &&\n+\t\tgit commit -m new deepsubfile\n+\t) &&\n+\tmkdir submodule &&\n+\t(\n+\t\tcd submodule &&\n+\t\tgit init &&\n+\t\techo subcontent > subfile &&\n+\t\tgit add subfile &&\n+\t\tgit submodule add \"$pwd/deepsubmodule\" deepsubmodule &&\n+\t\tgit commit -a -m new\n+\t) &&\n+\tgit submodule add \"$pwd/submodule\" submodule &&\n+\tgit commit -am initial &&\n+\tgit clone . downstream &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit submodule update --init --recursive\n+\t) &&\n+\techo \"Fetching submodule submodule\" > expect.out &&\n+\techo \"Fetching submodule submodule/deepsubmodule\" >> expect.out\n+'\n+\n+test_expect_success \"fetch --recurse-submodules recurses into submodules\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_expect_success \"fetch alone only fetches superproject\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"--quiet propagates to submodules\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules --quiet >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"--dry-run propagates to submodules\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules --dry-run >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_done\n-- \n1.7.3.2.337.g9376c\n"},{"id":"155635","messageId":"4CDB30D6.5040302@web.de","threadId":"25705","inReplyTo":"4CDB3063.5010801@web.de","subject":"[PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-10T23:55:02Z","receivedAt":"2010-11-10T23:55:02Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"This new boolean option can be used to override the default for \"git\nfetch\" and \"git pull\", which is to not recurse into populated submodules\nand fetch all new commits there too.\n\nSigned-off-by: Jens Lehmann \n---\n Documentation/config.txt | 5 +++++\n Documentation/fetch-options.txt | 5 +++--\n builtin/fetch.c | 21 ++++++++++++++++-----\n git-pull.sh | 3 +++\n submodule.c | 18 +++++++++++++++++-\n submodule.h | 4 +++-\n t/t5526-fetch-submodules.sh | 36 ++++++++++++++++++++++++++++++++++++\n 7 files changed, 83 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 538ebb5..453c1eb 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -879,6 +879,11 @@ diff.wordRegex::\n \tsequences that match the regular expression are \"words\", all other\n \tcharacters are *ignorable* whitespace.\n\n+fetch.recurseSubmodules::\n+\tA boolean value which changes the behavior for fetch and pull, the\n+\tdefault is to not recursively fetch populated sumodules unless\n+\tconfigured otherwise.\n+\n fetch.unpackLimit::\n \tIf the number of objects fetched over the git native\n \ttransfer is below this\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex bd36466..6aea5f6 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -66,8 +66,9 @@ ifndef::git-pull[]\n \tlinkgit:git-config[1].\n endif::git-pull[]\n\n---recurse-submodules::\n-\tUse this option to fetch new commits of all populated submodules too.\n+--[no-]recurse-submodules::\n+\tThis option controls if new commits of all populated submodules should\n+\tbe fetched too (see linkgit:git-config[1]).\n\n ifndef::git-pull[]\n --submodule-prefix::\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 658bb4c..eb2296d 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -28,8 +28,14 @@ enum {\n \tTAGS_SET = 2\n };\n\n+enum {\n+\tRECURSE_SUBMODULES_OFF = 0,\n+\tRECURSE_SUBMODULES_DEFAULT = 1,\n+\tRECURSE_SUBMODULES_ON = 2\n+};\n+\n static int all, append, dry_run, force, keep, multiple, prune, update_head_ok, verbosity;\n-static int progress, recurse_submodules;\n+static int progress, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int tags = TAGS_DEFAULT;\n static const char *depth;\n static const char *upload_pack;\n@@ -55,8 +61,9 @@ static struct option builtin_fetch_options[] = {\n \t\t \"do not fetch all tags (--no-tags)\", TAGS_UNSET),\n \tOPT_BOOLEAN('p', \"prune\", &prune,\n \t\t \"prune tracking branches no longer on remote\"),\n-\tOPT_BOOLEAN(0, \"recurse-submodules\", &recurse_submodules,\n-\t\t \"control recursive fetching of submodules\"),\n+\tOPT_SET_INT(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t \"control recursive fetching of submodules\",\n+\t\t RECURSE_SUBMODULES_ON),\n \tOPT_BOOLEAN(0, \"dry-run\", &dry_run,\n \t\t \"dry run\"),\n \tOPT_BOOLEAN('k', \"keep\", &keep, \"keep downloaded pack\"),\n@@ -802,7 +809,7 @@ static void add_options_to_argv(int *argc, const char **argv)\n \t\targv[(*argc)++] = \"--force\";\n \tif (keep)\n \t\targv[(*argc)++] = \"--keep\";\n-\tif (recurse_submodules)\n+\tif (recurse_submodules == RECURSE_SUBMODULES_ON)\n \t\targv[(*argc)++] = \"--recurse-submodules\";\n \tif (verbosity >= 2)\n \t\targv[(*argc)++] = \"-v\";\n@@ -940,14 +947,18 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \t\t}\n \t}\n\n-\tif (!result && recurse_submodules) {\n+\tif (!result && (recurse_submodules != RECURSE_SUBMODULES_OFF)) {\n \t\tconst char *options[10];\n \t\tint num_options = 0;\n+\t\t/* Set recursion as default when we already are recursing */\n+\t\tif (submodule_prefix[0])\n+\t\t\tset_config_fetch_recurse_submodules(1);\n \t\tgitmodules_config();\n \t\tgit_config(submodule_config, NULL);\n \t\tadd_options_to_argv(&num_options, options);\n \t\tresult = fetch_populated_submodules(num_options, options,\n \t\t\t\t\t\t submodule_prefix,\n+\t\t\t\t\t\t recurse_submodules == RECURSE_SUBMODULES_ON,\n \t\t\t\t\t\t verbosity < 0);\n \t}\n\ndiff --git a/git-pull.sh b/git-pull.sh\nindex 5804c62..a5ab195 100755\n--- a/git-pull.sh\n+++ b/git-pull.sh\n@@ -108,6 +108,9 @@ do\n \t--recurse-submodules)\n \t\trecurse_submodules=--recurse-submodules\n \t\t;;\n+\t--no-recurse-submodules)\n+\t\trecurse_submodules=--no-recurse-submodules\n+\t\t;;\n \t--d|--dr|--dry|--dry-|--dry-r|--dry-ru|--dry-run)\n \t\tdry_run=--dry-run\n \t\t;;\ndiff --git a/submodule.c b/submodule.c\nindex 4d9b774..01d75f5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -11,6 +11,7 @@\n\n struct string_list config_name_for_path;\n struct string_list config_ignore_for_name;\n+static int config_fetch_recurse_submodules;\n\n static int add_submodule_odb(const char *path)\n {\n@@ -67,6 +68,10 @@ int submodule_config(const char *var, const char *value, void *cb)\n {\n \tif (!prefixcmp(var, \"submodule.\"))\n \t\treturn parse_submodule_config_option(var, value);\n+\telse if (!strcmp(var, \"fetch.recursesubmodules\")) {\n+\t\tconfig_fetch_recurse_submodules = git_config_bool(var, value);\n+\t\treturn 0;\n+\t}\n \treturn 0;\n }\n\n@@ -229,8 +234,14 @@ void show_submodule_summary(FILE *f, const char *path,\n \tstrbuf_release(&sb);\n }\n\n+void set_config_fetch_recurse_submodules(int value)\n+{\n+\tconfig_fetch_recurse_submodules = value;\n+}\n+\n int fetch_populated_submodules(int num_options, const char **options,\n-\t\t\t const char *prefix, int quiet)\n+\t\t\t const char *prefix, int ignore_config,\n+\t\t\t int quiet)\n {\n \tint i, result = 0, argc = 0;\n \tstruct child_process cp;\n@@ -271,6 +282,11 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\tif (name_for_path)\n \t\t\tname = name_for_path->util;\n\n+\t\tif (!ignore_config) {\n+\t\t\tif (!config_fetch_recurse_submodules)\n+\t\t\t\tcontinue;\n+\t\t}\n+\n \t\tstrbuf_addf(&submodule_path, \"%s/%s\", work_tree, ce->name);\n \t\tstrbuf_addf(&submodule_git_dir, \"%s/.git\", submodule_path.buf);\n \t\tstrbuf_addf(&submodule_prefix, \"%s%s/\", prefix, ce->name);\ndiff --git a/submodule.h b/submodule.h\nindex b39d7a1..4729023 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -13,8 +13,10 @@ void show_submodule_summary(FILE *f, const char *path,\n \t\tunsigned char one[20], unsigned char two[20],\n \t\tunsigned dirty_submodule,\n \t\tconst char *del, const char *add, const char *reset);\n+void set_config_fetch_recurse_submodules(int value);\n int fetch_populated_submodules(int num_options, const char **options,\n-\t\t\t const char *prefix, int quiet);\n+\t\t\t const char *prefix, int ignore_config,\n+\t\t\t int quiet);\n unsigned is_submodule_modified(const char *path, int ignore_untracked);\n int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],\n \t\t const unsigned char a[20], const unsigned char b[20]);\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 107317a..69ed3e2 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -106,4 +106,40 @@ test_expect_success \"--dry-run propagates to submodules\" '\n \ttest_cmp expect.err actual.err\n '\n\n+test_expect_success \"recurseSubmodules=true propagates into submodules\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config fetch.recurseSubmodules true\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_expect_success \"--recurse-submodules overrides config in submodule\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\t(\n+\t\t\tcd submodule &&\n+\t\t\tgit config fetch.recurseSubmodules false\n+\t\t) &&\n+\t\tgit fetch --recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_expect_success \"--no-recurse-submodules overrides config setting\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config fetch.recurseSubmodules true\n+\t\tgit fetch --no-recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n test_done\n-- \n1.7.3.2.337.g9376c\n"},{"id":"155634","messageId":"4CDB30FD.80003@web.de","threadId":"25705","inReplyTo":"4CDB3063.5010801@web.de","subject":"[PATCH v3 3/3] Submodules: Add the \"fetchRecurseSubmodules\" config option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-10T23:55:41Z","receivedAt":"2010-11-10T23:55:41Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"The new boolean \"fetchRecurseSubmodules\" config option controls the\nbehavior for \"git fetch\" and \"git pull\". It specifies if these commands\nshould recurse into submodules and fetch new commits there too and can be\nset separately for each submodule.\n\nIn the .gitmodules file \"submodule..fetchRecurseSubmodules\" entries\nare read before looking for them in .git/config. Thus settings found in\n.git/config will override those from .gitmodules, thereby allowing the\nuser to ignore settings given by the remote side while also letting\nupstream set reasonable defaults for those users who don't have special\nneeds.\n\nThis configuration can be overridden by the command line option\n\"--[no-]recurse-submodules\" of \"git fetch\" and \"git pull\".\n\nSigned-off-by: Jens Lehmann \n---\n Documentation/config.txt | 7 +++++\n Documentation/fetch-options.txt | 2 +-\n Documentation/gitmodules.txt | 8 ++++++\n submodule.c | 20 ++++++++++++++-\n t/t5526-fetch-submodules.sh | 50 +++++++++++++++++++++++++++++++++++++++\n 5 files changed, 84 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 453c1eb..cad9658 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1796,6 +1796,13 @@ submodule..update::\n \tURL and other values found in the `.gitmodules` file. See\n \tlinkgit:git-submodule[1] and linkgit:gitmodules[5] for details.\n\n+submodule..fetchRecurseSubmodules::\n+\tThis option can be used to enable/disable recursive fetching of this\n+\tsubmodule. It can be overriden by using the --[no-]recurse-submodules\n+\tcommand line option to \"git fetch\" and \"git pull\".\n+\tThis setting will override that from in the linkgit:gitmodules[5]\n+\tfile.\n+\n submodule..ignore::\n \tDefines under what circumstances \"git status\" and the diff family show\n \ta submodule as modified. When set to \"all\", it will never be considered\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 6aea5f6..dea7a29 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -68,7 +68,7 @@ endif::git-pull[]\n\n --[no-]recurse-submodules::\n \tThis option controls if new commits of all populated submodules should\n-\tbe fetched too (see linkgit:git-config[1]).\n+\tbe fetched too (see linkgit:git-config[1] and linkgit:gitmodules[5]).\n\n ifndef::git-pull[]\n --submodule-prefix::\ndiff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt\nindex bcffd95..6c93202 100644\n--- a/Documentation/gitmodules.txt\n+++ b/Documentation/gitmodules.txt\n@@ -44,6 +44,14 @@ submodule..update::\n \tThis config option is overridden if 'git submodule update' is given\n \tthe '--merge' or '--rebase' options.\n\n+submodule..fetchRecurseSubmodules::\n+\tThis option can be used to enable/disable recursive fetching of this\n+\tsubmodule. If this option is also present in the submodules entry in\n+\t.git/config of the superproject, the setting there will override the\n+\tone found in .gitmodules.\n+\tBoth settings can be overriden on the command line by using the\n+\t\"--[no-]recurse-submodules\" option to \"git fetch\" and \"git pull\"..\n+\n submodule..ignore::\n \tDefines under what circumstances \"git status\" and the diff family show\n \ta submodule as modified. When set to \"all\", it will never be considered\ndiff --git a/submodule.c b/submodule.c\nindex 01d75f5..4e62900 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -10,6 +10,7 @@\n #include \"string-list.h\"\n\n struct string_list config_name_for_path;\n+struct string_list config_fetch_recurse_submodules_for_name;\n struct string_list config_ignore_for_name;\n static int config_fetch_recurse_submodules;\n\n@@ -105,6 +106,14 @@ int parse_submodule_config_option(const char *var, const char *value)\n \t\t\tconfig = string_list_append(&config_name_for_path, xstrdup(value));\n \t\tconfig->util = strbuf_detach(&submodname, NULL);\n \t\tstrbuf_release(&submodname);\n+\t} else if ((len > 23) && !strcmp(var + len - 23, \".fetchrecursesubmodules\")) {\n+\t\tstrbuf_add(&submodname, var, len - 23);\n+\t\tconfig = unsorted_string_list_lookup(&config_fetch_recurse_submodules_for_name, submodname.buf);\n+\t\tif (!config)\n+\t\t\tconfig = string_list_append(&config_fetch_recurse_submodules_for_name,\n+\t\t\t\t\t\t strbuf_detach(&submodname, NULL));\n+\t\tconfig->util = git_config_bool(var, value) ? (void *)1 : NULL;\n+\t\tstrbuf_release(&submodname);\n \t} else if ((len > 7) && !strcmp(var + len - 7, \".ignore\")) {\n \t\tif (strcmp(value, \"untracked\") && strcmp(value, \"dirty\") &&\n \t\t strcmp(value, \"all\") && strcmp(value, \"none\")) {\n@@ -283,8 +292,15 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\tname = name_for_path->util;\n\n \t\tif (!ignore_config) {\n-\t\t\tif (!config_fetch_recurse_submodules)\n-\t\t\t\tcontinue;\n+\t\t\tstruct string_list_item *fetch_recurse_submodules_option;\n+\t\t\tfetch_recurse_submodules_option = unsorted_string_list_lookup(&config_fetch_recurse_submodules_for_name, name);\n+\t\t\tif (fetch_recurse_submodules_option) {\n+\t\t\t\tif (!fetch_recurse_submodules_option->util)\n+\t\t\t\t\tcontinue;\n+\t\t\t} else {\n+\t\t\t\tif (!config_fetch_recurse_submodules)\n+\t\t\t\t\tcontinue;\n+\t\t\t}\n \t\t}\n\n \t\tstrbuf_addf(&submodule_path, \"%s/%s\", work_tree, ce->name);\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 69ed3e2..c070faf 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -81,6 +81,56 @@ test_expect_success \"fetch alone only fetches superproject\" '\n \t! test -s actual.err\n '\n\n+test_expect_success \"fetch --no-recurse-submodules only fetches superproject\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --no-recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"using fetchRecurseSubmodules=true in .gitmodules recurses into submodules\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config -f .gitmodules submodule.submodule.fetchRecurseSubmodules true &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_expect_success \"--no-recurse-submodules overrides .gitmodules config\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --no-recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"using fetchRecurseSubmodules=false in .git/config overrides setting in .gitmodules\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config submodule.submodule.fetchRecurseSubmodules false &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"--recurse-submodules overrides fetchRecurseSubmodules setting from .git/config\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules >../actual.out 2>../actual.err &&\n+\t\tgit config -f --unset .gitmodules submodule.submodule.fetchRecurseSubmodules true &&\n+\t\tgit config --unset submodule.submodule.fetchRecurseSubmodules\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n test_expect_success \"--quiet propagates to submodules\" '\n \t(\n \t\tcd downstream &&\n-- \n1.7.3.2.337.g9376c\n"},{"id":"155636","messageId":"20101111000216.GA14189@burratino","threadId":"25705","inReplyTo":"4CDB30D6.5040302@web.de","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-11T00:02:16Z","receivedAt":"2010-11-11T00:02:16Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n\n> --- a/t/t5526-fetch-submodules.sh\n> +++ b/t/t5526-fetch-submodules.sh\n> @@ -106,4 +106,40 @@ test_expect_success \"--dry-run propagates to submodules\" '\n> \ttest_cmp expect.err actual.err\n> '\n> \n> +test_expect_success \"recurseSubmodules=true propagates into submodules\" '\n> +\tadd_upstream_commit &&\n> +\t(\n> +\t\tcd downstream &&\n> +\t\tgit config fetch.recurseSubmodules true\n> +\t\tgit fetch >../actual.out 2>../actual.err\n> +\t) &&\n> +\ttest_cmp expect.out actual.out &&\n> +\ttest_cmp expect.err actual.err\n> +'\n\nThis configuration item is read from .gitmodules, too, right? Would\nit be easy (or desirable) to make it not read from there? Either way,\nit would be nice to have a test so the behavior doesn't change without\nanyone noticing.\n"},{"id":"155637","messageId":"20101111000538.GB14189@burratino","threadId":"25705","inReplyTo":"4CDB3063.5010801@web.de","subject":"Re: [PATCH v3 0/3] Teach fetch and pull to recursively fetch submodules","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-11T00:05:38Z","receivedAt":"2010-11-11T00:05:38Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n\n> * Should the \"--submodule-prefix\" option - which is only used internally\n> now - be a hidden option to \"git fetch\"?\n\nYes. (Any option that is useless outside scripts should be, imho.)\n\n> But nonetheless I think this patch series is ok for inclusion as it does\n> not change default behavior and gives people the opportunity to play with\n> recursive fetch/pull by enabling one of the introduced config options.\n\nExcept for the .gitmodules detail I mentioned, it looks good to me.\nLooking forward to trying it out.\n\nThanks,\nJonathan\n"},{"id":"155653","messageId":"4CDBA5FD.20802@web.de","threadId":"25705","inReplyTo":"20101111000216.GA14189@burratino","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-11T08:14:53Z","receivedAt":"2010-11-11T08:14:53Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 11.11.2010 01:02, schrieb Jonathan Nieder:\n> Jens Lehmann wrote:\n> \n>> --- a/t/t5526-fetch-submodules.sh\n>> +++ b/t/t5526-fetch-submodules.sh\n>> @@ -106,4 +106,40 @@ test_expect_success \"--dry-run propagates to submodules\" '\n>> \ttest_cmp expect.err actual.err\n>> '\n>>\n>> +test_expect_success \"recurseSubmodules=true propagates into submodules\" '\n>> +\tadd_upstream_commit &&\n>> +\t(\n>> +\t\tcd downstream &&\n>> +\t\tgit config fetch.recurseSubmodules true\n>> +\t\tgit fetch >../actual.out 2>../actual.err\n>> +\t) &&\n>> +\ttest_cmp expect.out actual.out &&\n>> +\ttest_cmp expect.err actual.err\n>> +'\n> \n> This configuration item is read from .gitmodules, too, right? Would\n> it be easy (or desirable) to make it not read from there? Either way,\n> it would be nice to have a test so the behavior doesn't change without\n> anyone noticing.\n\n\"fetch.recurseSubmodules\" is only read from .git/config. The one read\nfirst from .gitmodules and then from .git/config is the per-submodule\nsetting \"submodule..fetchRecurseSubmodules\" added in 3/3. But\nmaybe I should add a test that \"fetch.recurseSubmodules\" also works\nwhen set in the global config ...\n"},{"id":"155654","messageId":"4CDBA6DD.4040701@web.de","threadId":"25705","inReplyTo":"20101111000538.GB14189@burratino","subject":"Re: [PATCH v3 0/3] Teach fetch and pull to recursively fetch submodules","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-11T08:18:37Z","receivedAt":"2010-11-11T08:18:37Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 11.11.2010 01:05, schrieb Jonathan Nieder:\n> Jens Lehmann wrote:\n> \n>> * Should the \"--submodule-prefix\" option - which is only used internally\n>> now - be a hidden option to \"git fetch\"?\n> \n> Yes. (Any option that is useless outside scripts should be, imho.)\n\nOk, I will change that.\n\n>> But nonetheless I think this patch series is ok for inclusion as it does\n>> not change default behavior and gives people the opportunity to play with\n>> recursive fetch/pull by enabling one of the introduced config options.\n> \n> Except for the .gitmodules detail I mentioned, it looks good to me.\n> Looking forward to trying it out.\n\nThanks!\n"},{"id":"155655","messageId":"20101111082748.GA15525@burratino","threadId":"25705","inReplyTo":"4CDBA5FD.20802@web.de","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-11T08:27:48Z","receivedAt":"2010-11-11T08:27:48Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n> Am 11.11.2010 01:02, schrieb Jonathan Nieder:\n\n>> This configuration item is read from .gitmodules, too, right? Would\n>> it be easy (or desirable) to make it not read from there? Either way,\n>> it would be nice to have a test so the behavior doesn't change without\n>> anyone noticing.\n>\n> \"fetch.recurseSubmodules\" is only read from .git/config. The one read\n> first from .gitmodules and then from .git/config is the per-submodule\n> setting \"submodule..fetchRecurseSubmodules\" added in 3/3.\n\nSorry for the nonsense. Would it be easy (or desirable) to make\n_that_ one not be read from .gitmodules?\n\n> But\n> maybe I should add a test that \"fetch.recurseSubmodules\" also works\n> when set in the global config ...\n\nYes, I agree that such a test would be good (though less important\nthan a test for the behavior with no configuration set at all, say).\nA test to demonstrate that configuration aside from submodule.* is not\nread from .gitmodules would also be good imho, but I don't think it's\na blocker for anything. :)\n"},{"id":"155725","messageId":"7v1v6rhfut.fsf@alter.siamese.dyndns.org","threadId":"25705","inReplyTo":"20101111082748.GA15525@burratino","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-11-11T18:31:22Z","receivedAt":"2010-11-11T18:31:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n>> \"fetch.recurseSubmodules\" is only read from .git/config. The one read\n>> first from .gitmodules and then from .git/config is the per-submodule\n>> setting \"submodule..fetchRecurseSubmodules\" added in 3/3.\n>\n> Sorry for the nonsense. Would it be easy (or desirable) to make\n> _that_ one not be read from .gitmodules?\n\nI think the motivation behind having a way to read it from .gitmodules is\nso that project can suggest the default for convenience (e.g. \"almost\neverybody who interacts with this project wants these submodules checked\nout and kept updated\").\n\nTraditionally the suggestions kept in .gitmodules were propagated to the\nconfig when the submodule was initialized, and at runtime we read only\nfrom the config from then on without reading from .gitmodules, so that\nonce the user decides to follow what the project suggests (or customize\nthat away), the preference would stick to the repository.\n\nThat arrangement does not cater well to people who want to follow along\nwhatever the project's suggestion of the day, so we might want to change\nthings so that we if we find it in the config, we stop there and use what\nwe found, otherwise we use what is in the in-tree gitmodules; I suspect we\nmight require some changes to \"submodule init\" not to copy certain things\nto the config for that to work, though...\n"},{"id":"155728","messageId":"20101111190053.GH16972@burratino","threadId":"25705","inReplyTo":"7v1v6rhfut.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-11T19:00:53Z","receivedAt":"2010-11-11T19:00:53Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n\n> I think the motivation behind having a way to read it from .gitmodules is\n> so that project can suggest the default for convenience (e.g. \"almost\n> everybody who interacts with this project wants these submodules checked\n> out and kept updated\").\n\nYes, that makes some sense to me. Except wouldn't it be a single\nconfiguration item? \"These submodules should be checked out in all\nbut unusual situations, so check them out automatically and keep them\nupdated.\"\n\nMaybe a person setting this to false actually means \"This submodule\nhas its url set to a repository that is updated very frequently, and\nmost updates are not relevant to the superproject.\" Unfortunately, I\nthink the result would be a poor user experience: when an update comes\nthat _is_ important to the superproject, what happens?\n\n $ git fetch\n ... go on plane ...\n $ git merge @{u} && git submodule update --no-fetch --recursive\n [...]\n fatal: reference is not a tree: f1c596a3895643d0969a15b8e945bf0c0072e470\n\nHmm. I think in that scenario a better solution would be to point the\nsubmodule url point to a project-specific clone that is updated less\nfrequently.\n\nWhat am I missing?\n\nJonathan\n"},{"id":"155775","messageId":"4CDD27B6.3070907@web.de","threadId":"25705","inReplyTo":"7v1v6rhfut.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-12T11:40:38Z","receivedAt":"2010-11-12T11:40:38Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 11.11.2010 19:31, schrieb Junio C Hamano:\n> Jonathan Nieder writes:\n> \n>>> \"fetch.recurseSubmodules\" is only read from .git/config. The one read\n>>> first from .gitmodules and then from .git/config is the per-submodule\n>>> setting \"submodule..fetchRecurseSubmodules\" added in 3/3.\n>>\n>> Sorry for the nonsense. Would it be easy (or desirable) to make\n>> _that_ one not be read from .gitmodules?\n> \n> I think the motivation behind having a way to read it from .gitmodules is\n> so that project can suggest the default for convenience (e.g. \"almost\n> everybody who interacts with this project wants these submodules checked\n> out and kept updated\").\n\nYes, and to achieve that it should /not/ be necessary to run a \"git\nsubmodule sync\" manually afterwards to activate those changes.\n\nJonathan, I think when we allow upstream to configure which submodules\nare to be cloned and checked out by default (which seems an option almost\neveryone likes to have), doesn't it make sense to let upstream set a\ndefault which submodules should be fetched too (so that new commits there\ncan be checked out recursively later)? I would rather not like to do that\nimplicitly just because a submodule is configured for recursive checkout\n...\n\n\n> Traditionally the suggestions kept in .gitmodules were propagated to the\n> config when the submodule was initialized, and at runtime we read only\n> from the config from then on without reading from .gitmodules, so that\n> once the user decides to follow what the project suggests (or customize\n> that away), the preference would stick to the repository.\n> \n> That arrangement does not cater well to people who want to follow along\n> whatever the project's suggestion of the day, so we might want to change\n> things so that we if we find it in the config, we stop there and use what\n> we found, otherwise we use what is in the in-tree gitmodules; I suspect we\n> might require some changes to \"submodule init\" not to copy certain things\n> to the config for that to work, though...\n\nRight. But this would enable you to have branches where different sets\nof submodules are fetched, checked out and/or considered to make the\nsuperproject dirty depending on the topic the branch is for.\n"},{"id":"155776","messageId":"4CDD2AF9.6040403@web.de","threadId":"25705","inReplyTo":"20101111190053.GH16972@burratino","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-12T11:54:33Z","receivedAt":"2010-11-12T11:54:33Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 11.11.2010 20:00, schrieb Jonathan Nieder:\n> Junio C Hamano wrote:\n> \n>> I think the motivation behind having a way to read it from .gitmodules is\n>> so that project can suggest the default for convenience (e.g. \"almost\n>> everybody who interacts with this project wants these submodules checked\n>> out and kept updated\").\n> \n> Yes, that makes some sense to me. Except wouldn't it be a single\n> configuration item? \"These submodules should be checked out in all\n> but unusual situations, so check them out automatically and keep them\n> updated.\"\n\nHmm, but we have at least three modes of how to update them:\n\n1) Never fetch the submodule (to get new commits the user has to run\n \"git fetch --recurse-submodules\" by hand)\n\n2) Fetch the submodule each time you fetch the superproject (Which is\n really handy when you do development in the submodule too but can\n be really inconvenient when you don't)\n\n3) Update submodules only when new recorded commits are fetched in\n the superproject (This mode is not added with the current patch\n series but will be in one of the next)\n\nSo you would need a config option for that anyway, no? And that is why\nI'd rather like to have a separate fetch option to control that behavior\ninstead of an implicit \"if-it's-to-be-checked-out-fetch-it-too\" approach.\n\n\n> Maybe a person setting this to false actually means \"This submodule\n> has its url set to a repository that is updated very frequently, and\n> most updates are not relevant to the superproject.\" Unfortunately, I\n> think the result would be a poor user experience: when an update comes\n> that _is_ important to the superproject, what happens?\n> \n> $ git fetch\n> ... go on plane ...\n> $ git merge @{u} && git submodule update --no-fetch --recursive\n> [...]\n> fatal: reference is not a tree: f1c596a3895643d0969a15b8e945bf0c0072e470\n> \n> Hmm. I think in that scenario a better solution would be to point the\n> submodule url point to a project-specific clone that is updated less\n> frequently.\n> \n> What am I missing?\n\nThat situation should be handled by method 3) above which was proposed\nfor such a use case.\n"},{"id":"155778","messageId":"4CDD391C.2000905@web.de","threadId":"25705","inReplyTo":"20101111000538.GB14189@burratino","subject":"[PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-12T12:54:52Z","receivedAt":"2010-11-12T12:54:52Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Until now you had to call \"git submodule update\" (without -N|--no-fetch\noption) or something like \"git submodule foreach git fetch\" to fetch\nnew commits in populated submodules from their remote.\n\nThis could lead to \"(commits not present)\" messages in the output of\n\"git diff --submodule\" (which is used by \"git gui\" and \"gitk\") after\nfetching or pulling new commits in the superproject and is an obstacle for\nimplementing recursive checkout of submodules. Also \"git submodule\nupdate\" cannot fetch changes when disconnected, so it was very easy to\nforget to fetch the submodule changes before disconnecting only to\ndiscover later that they are needed.\n\nThis patch adds the \"--recurse-submodules\" option to recursively fetch\neach populated submodule from the url configured in the .git/config of the\nsubmodule at the end of each \"git fetch\" or during \"git pull\" in the\nsuperproject. The submodule paths are taken from the index.\n\nThe hidden option \"--submodule-prefix\" is added to \"git fetch\" to be able\nto print out the full paths of nested submodules.\n\nSigned-off-by: Jens Lehmann \n---\n\nAm 11.11.2010 01:05, schrieb Jonathan Nieder:\n> Jens Lehmann wrote:\n> \n>> * Should the \"--submodule-prefix\" option - which is only used internally\n>> now - be a hidden option to \"git fetch\"?\n> \n> Yes. (Any option that is useless outside scripts should be, imho.)\n\nOk, here is the updated version of patch 1/3 making \"--submodule-prefix\" a\nhidden option which is not documented in Documentation/fetch-options.txt .\n\n\n Documentation/fetch-options.txt | 3 +\n builtin/fetch.c | 53 ++++++++++++++-----\n git-pull.sh | 7 ++-\n submodule.c | 66 +++++++++++++++++++++++-\n submodule.h | 3 +\n t/t5526-fetch-submodules.sh | 109 +++++++++++++++++++++++++++++++++++++++\n 6 files changed, 224 insertions(+), 17 deletions(-)\n create mode 100755 t/t5526-fetch-submodules.sh\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 5ce1e72..85b7f84 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -66,6 +66,9 @@ ifndef::git-pull[]\n \tlinkgit:git-config[1].\n endif::git-pull[]\n\n+--recurse-submodules::\n+\tUse this option to fetch new commits of all populated submodules too.\n+\n -u::\n --update-head-ok::\n \tBy default 'git fetch' refuses to update the head which\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex d35f000..db3fba3 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -12,6 +12,7 @@\n #include \"parse-options.h\"\n #include \"sigchain.h\"\n #include \"transport.h\"\n+#include \"submodule.h\"\n\n static const char * const builtin_fetch_usage[] = {\n \t\"git fetch [] [ [...]]\",\n@@ -28,12 +29,13 @@ enum {\n };\n\n static int all, append, dry_run, force, keep, multiple, prune, update_head_ok, verbosity;\n-static int progress;\n+static int progress, recurse_submodules;\n static int tags = TAGS_DEFAULT;\n static const char *depth;\n static const char *upload_pack;\n static struct strbuf default_rla = STRBUF_INIT;\n static struct transport *transport;\n+static const char *submodule_prefix = \"\";\n\n static struct option builtin_fetch_options[] = {\n \tOPT__VERBOSITY(&verbosity),\n@@ -53,6 +55,8 @@ static struct option builtin_fetch_options[] = {\n \t\t \"do not fetch all tags (--no-tags)\", TAGS_UNSET),\n \tOPT_BOOLEAN('p', \"prune\", &prune,\n \t\t \"prune tracking branches no longer on remote\"),\n+\tOPT_BOOLEAN(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t \"control recursive fetching of submodules\"),\n \tOPT_BOOLEAN(0, \"dry-run\", &dry_run,\n \t\t \"dry run\"),\n \tOPT_BOOLEAN('k', \"keep\", &keep, \"keep downloaded pack\"),\n@@ -61,6 +65,8 @@ static struct option builtin_fetch_options[] = {\n \tOPT_BOOLEAN(0, \"progress\", &progress, \"force progress reporting\"),\n \tOPT_STRING(0, \"depth\", &depth, \"DEPTH\",\n \t\t \"deepen history of shallow clone\"),\n+\t{ OPTION_STRING, 0, \"submodule-prefix\", &submodule_prefix, \"DIR\",\n+\t\t \"prepend this to submodule path output\", PARSE_OPT_HIDDEN },\n \tOPT_END()\n };\n\n@@ -784,28 +790,36 @@ static int add_remote_or_group(const char *name, struct string_list *list)\n \treturn 1;\n }\n\n-static int fetch_multiple(struct string_list *list)\n+static void add_options_to_argv(int *argc, const char **argv)\n {\n-\tint i, result = 0;\n-\tconst char *argv[11] = { \"fetch\", \"--append\" };\n-\tint argc = 2;\n-\n \tif (dry_run)\n-\t\targv[argc++] = \"--dry-run\";\n+\t\targv[(*argc)++] = \"--dry-run\";\n \tif (prune)\n-\t\targv[argc++] = \"--prune\";\n+\t\targv[(*argc)++] = \"--prune\";\n \tif (update_head_ok)\n-\t\targv[argc++] = \"--update-head-ok\";\n+\t\targv[(*argc)++] = \"--update-head-ok\";\n \tif (force)\n-\t\targv[argc++] = \"--force\";\n+\t\targv[(*argc)++] = \"--force\";\n \tif (keep)\n-\t\targv[argc++] = \"--keep\";\n+\t\targv[(*argc)++] = \"--keep\";\n+\tif (recurse_submodules)\n+\t\targv[(*argc)++] = \"--recurse-submodules\";\n \tif (verbosity >= 2)\n-\t\targv[argc++] = \"-v\";\n+\t\targv[(*argc)++] = \"-v\";\n \tif (verbosity >= 1)\n-\t\targv[argc++] = \"-v\";\n+\t\targv[(*argc)++] = \"-v\";\n \telse if (verbosity < 0)\n-\t\targv[argc++] = \"-q\";\n+\t\targv[(*argc)++] = \"-q\";\n+\n+}\n+\n+static int fetch_multiple(struct string_list *list)\n+{\n+\tint i, result = 0;\n+\tconst char *argv[12] = { \"fetch\", \"--append\" };\n+\tint argc = 2;\n+\n+\tadd_options_to_argv(&argc, argv);\n\n \tif (!append && !dry_run) {\n \t\tint errcode = truncate_fetch_head();\n@@ -926,6 +940,17 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \t\t}\n \t}\n\n+\tif (!result && recurse_submodules) {\n+\t\tconst char *options[10];\n+\t\tint num_options = 0;\n+\t\tgitmodules_config();\n+\t\tgit_config(submodule_config, NULL);\n+\t\tadd_options_to_argv(&num_options, options);\n+\t\tresult = fetch_populated_submodules(num_options, options,\n+\t\t\t\t\t\t submodule_prefix,\n+\t\t\t\t\t\t verbosity < 0);\n+\t}\n+\n \t/* All names were strdup()ed or strndup()ed */\n \tlist.strdup_strings = 1;\n \tstring_list_clear(&list, 0);\ndiff --git a/git-pull.sh b/git-pull.sh\nindex 8eb74d4..5804c62 100755\n--- a/git-pull.sh\n+++ b/git-pull.sh\n@@ -38,7 +38,7 @@ test -z \"$(git ls-files -u)\" || die_conflict\n test -f \"$GIT_DIR/MERGE_HEAD\" && die_merge\n\n strategy_args= diffstat= no_commit= squash= no_ff= ff_only=\n-log_arg= verbosity= progress=\n+log_arg= verbosity= progress= recurse_submodules=\n merge_args=\n curr_branch=$(git symbolic-ref -q HEAD)\n curr_branch_short=\"${curr_branch#refs/heads/}\"\n@@ -105,6 +105,9 @@ do\n \t--no-r|--no-re|--no-reb|--no-reba|--no-rebas|--no-rebase)\n \t\trebase=false\n \t\t;;\n+\t--recurse-submodules)\n+\t\trecurse_submodules=--recurse-submodules\n+\t\t;;\n \t--d|--dr|--dry|--dry-|--dry-r|--dry-ru|--dry-run)\n \t\tdry_run=--dry-run\n \t\t;;\n@@ -220,7 +223,7 @@ test true = \"$rebase\" && {\n \tdone\n }\n orig_head=$(git rev-parse -q --verify HEAD)\n-git fetch $verbosity $progress $dry_run --update-head-ok \"$@\" || exit 1\n+git fetch $verbosity $progress $dry_run $recurse_submodules --update-head-ok \"$@\" || exit 1\n test -z \"$dry_run\" || exit 0\n\n curr_head=$(git rev-parse -q --verify HEAD)\ndiff --git a/submodule.c b/submodule.c\nindex 91a4758..4d9b774 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -63,7 +63,7 @@ void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n \t}\n }\n\n-static int submodule_config(const char *var, const char *value, void *cb)\n+int submodule_config(const char *var, const char *value, void *cb)\n {\n \tif (!prefixcmp(var, \"submodule.\"))\n \t\treturn parse_submodule_config_option(var, value);\n@@ -229,6 +229,70 @@ void show_submodule_summary(FILE *f, const char *path,\n \tstrbuf_release(&sb);\n }\n\n+int fetch_populated_submodules(int num_options, const char **options,\n+\t\t\t const char *prefix, int quiet)\n+{\n+\tint i, result = 0, argc = 0;\n+\tstruct child_process cp;\n+\tconst char **argv;\n+\tstruct string_list_item *name_for_path;\n+\tconst char *work_tree = get_git_work_tree();\n+\tif (!work_tree)\n+\t\treturn 0;\n+\n+\tif (!the_index.initialized)\n+\t\tif (read_cache() < 0)\n+\t\t\tdie(\"index file corrupt\");\n+\n+\targv = xcalloc(num_options + 5, sizeof(const char *));\n+\targv[argc++] = \"fetch\";\n+\tfor (i = 0; i < num_options; i++)\n+\t\targv[argc++] = options[i];\n+\targv[argc++] = \"--submodule-prefix\";\n+\n+\tmemset(&cp, 0, sizeof(cp));\n+\tcp.argv = argv;\n+\tcp.env = local_repo_env;\n+\tcp.git_cmd = 1;\n+\tcp.no_stdin = 1;\n+\n+\tfor (i = 0; i < active_nr; i++) {\n+\t\tstruct strbuf submodule_path = STRBUF_INIT;\n+\t\tstruct strbuf submodule_git_dir = STRBUF_INIT;\n+\t\tstruct strbuf submodule_prefix = STRBUF_INIT;\n+\t\tstruct cache_entry *ce = active_cache[i];\n+\t\tconst char *git_dir, *name;\n+\n+\t\tif (!S_ISGITLINK(ce->ce_mode))\n+\t\t\tcontinue;\n+\n+\t\tname = ce->name;\n+\t\tname_for_path = unsorted_string_list_lookup(&config_name_for_path, ce->name);\n+\t\tif (name_for_path)\n+\t\t\tname = name_for_path->util;\n+\n+\t\tstrbuf_addf(&submodule_path, \"%s/%s\", work_tree, ce->name);\n+\t\tstrbuf_addf(&submodule_git_dir, \"%s/.git\", submodule_path.buf);\n+\t\tstrbuf_addf(&submodule_prefix, \"%s%s/\", prefix, ce->name);\n+\t\tgit_dir = read_gitfile_gently(submodule_git_dir.buf);\n+\t\tif (!git_dir)\n+\t\t\tgit_dir = submodule_git_dir.buf;\n+\t\tif (is_directory(git_dir)) {\n+\t\t\tif (!quiet)\n+\t\t\t\tprintf(\"Fetching submodule %s%s\\n\", prefix, ce->name);\n+\t\t\tcp.dir = submodule_path.buf;\n+\t\t\targv[argc] = submodule_prefix.buf;\n+\t\t\tif (run_command(&cp))\n+\t\t\t\tresult = 1;\n+\t\t}\n+\t\tstrbuf_release(&submodule_path);\n+\t\tstrbuf_release(&submodule_git_dir);\n+\t\tstrbuf_release(&submodule_prefix);\n+\t}\n+\tfree(argv);\n+\treturn result;\n+}\n+\n unsigned is_submodule_modified(const char *path, int ignore_untracked)\n {\n \tssize_t len;\ndiff --git a/submodule.h b/submodule.h\nindex 386f410..b39d7a1 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -5,6 +5,7 @@ struct diff_options;\n\n void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n \t\tconst char *path);\n+int submodule_config(const char *var, const char *value, void *cb);\n void gitmodules_config();\n int parse_submodule_config_option(const char *var, const char *value);\n void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);\n@@ -12,6 +13,8 @@ void show_submodule_summary(FILE *f, const char *path,\n \t\tunsigned char one[20], unsigned char two[20],\n \t\tunsigned dirty_submodule,\n \t\tconst char *del, const char *add, const char *reset);\n+int fetch_populated_submodules(int num_options, const char **options,\n+\t\t\t const char *prefix, int quiet);\n unsigned is_submodule_modified(const char *path, int ignore_untracked);\n int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],\n \t\t const unsigned char a[20], const unsigned char b[20]);\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nnew file mode 100755\nindex 0000000..107317a\n--- /dev/null\n+++ b/t/t5526-fetch-submodules.sh\n@@ -0,0 +1,109 @@\n+#!/bin/sh\n+# Copyright (c) 2010, Jens Lehmann\n+\n+test_description='Recursive \"git fetch\" for submodules'\n+\n+. ./test-lib.sh\n+\n+pwd=$(pwd)\n+\n+add_upstream_commit() {\n+\t(\n+\t\tcd submodule &&\n+\t\thead1=$(git rev-parse --short HEAD) &&\n+\t\techo new >> subfile &&\n+\t\ttest_tick &&\n+\t\tgit add subfile &&\n+\t\tgit commit -m new subfile &&\n+\t\thead2=$(git rev-parse --short HEAD) &&\n+\t\techo \"From $pwd/submodule\" > ../expect.err &&\n+\t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n+\t)\n+\t(\n+\t\tcd deepsubmodule &&\n+\t\thead1=$(git rev-parse --short HEAD) &&\n+\t\techo new >> deepsubfile &&\n+\t\ttest_tick &&\n+\t\tgit add deepsubfile &&\n+\t\tgit commit -m new deepsubfile &&\n+\t\thead2=$(git rev-parse --short HEAD) &&\n+\t\techo \"From $pwd/deepsubmodule\" >> ../expect.err &&\n+\t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n+\t)\n+}\n+\n+test_expect_success setup '\n+\tmkdir deepsubmodule &&\n+\t(\n+\t\tcd deepsubmodule &&\n+\t\tgit init &&\n+\t\techo deepsubcontent > deepsubfile &&\n+\t\tgit add deepsubfile &&\n+\t\tgit commit -m new deepsubfile\n+\t) &&\n+\tmkdir submodule &&\n+\t(\n+\t\tcd submodule &&\n+\t\tgit init &&\n+\t\techo subcontent > subfile &&\n+\t\tgit add subfile &&\n+\t\tgit submodule add \"$pwd/deepsubmodule\" deepsubmodule &&\n+\t\tgit commit -a -m new\n+\t) &&\n+\tgit submodule add \"$pwd/submodule\" submodule &&\n+\tgit commit -am initial &&\n+\tgit clone . downstream &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit submodule update --init --recursive\n+\t) &&\n+\techo \"Fetching submodule submodule\" > expect.out &&\n+\techo \"Fetching submodule submodule/deepsubmodule\" >> expect.out\n+'\n+\n+test_expect_success \"fetch --recurse-submodules recurses into submodules\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_expect_success \"fetch alone only fetches superproject\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"--quiet propagates to submodules\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules --quiet >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"--dry-run propagates to submodules\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules --dry-run >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_done\n-- \n1.7.3.2.335.g2e3006\n"},{"id":"155784","messageId":"20101112155210.GA15141@burratino","threadId":"25705","inReplyTo":"4CDD2AF9.6040403@web.de","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-12T15:52:10Z","receivedAt":"2010-11-12T15:52:10Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n> Am 11.11.2010 20:00, schrieb Jonathan Nieder:\n\n>> Yes, that makes some sense to me. Except wouldn't it be a single\n>> configuration item? \"These submodules should be checked out in all\n>> but unusual situations, so check them out automatically and keep them\n>> updated.\"\n>\n> Hmm, but we have at least three modes of how to update them:\n>\n> 1) Never fetch the submodule (to get new commits the user has to run\n> \"git fetch --recurse-submodules\" by hand)\n>\n> 2) Fetch the submodule each time you fetch the superproject (Which is\n> really handy when you do development in the submodule too but can\n> be really inconvenient when you don't)\n>\n> 3) Update submodules only when new recorded commits are fetched in\n> the superproject (This mode is not added with the current patch\n> series but will be in one of the next)\n>\n> So you would need a config option for that anyway, no? And that is why\n> I'd rather like to have a separate fetch option to control that behavior\n> instead of an implicit \"if-it's-to-be-checked-out-fetch-it-too\" approach.\n\nI still think I am missing something.\n\nTraditionally, git has been a _content_ tracker. The configuration that\ngets transmitted (.gitignore, .gitattributes, .gitmodules) would only\nrepresent basic information needed for that content to remain usable and\nsensible.\n\nIn the case of .gitmodules, it seems that two concepts are being\nconflated:\n\n A. Configuration based on the user's preferences. Absolutely, a person\n deserves to be able to easily choose between (1), (2), and (3) as\n described above.\n\n B. Metadata about the content that should be shared. For example, \"this\n submodule would be checked out in all but unusual circumstances\" is a\n useful thing to be able to declare.\n\nProbably I am missing something big, but fetchsubmodules as currently\ndefined seem like something in category A and not in category B.\nPartially because if we ever implement option (3), that is what almost\n_every_ casual consumer will want. So why should they be stuck with these\nconfiguration files specifying (1) and (2) when they check out old\nrevisions?\n\nAll that said, I do not think it is unreasonable in some situations for a\nproject to want to share configuration of type (A) between members of a\nproject; for example, lots of projects share hooks and that's great. I\njust don't think git should set it up automatically --- better to require\nan explicit user action like \"sh useprojectconfiguration.sh\" (an action\nmore explicit than \"please initialize and checkout all relevant\nsubmodules\") for such cases.\n\nHoping that is clearer,\nJonathan\n"},{"id":"155794","messageId":"4CDD9A02.6000507@web.de","threadId":"25705","inReplyTo":"20101112155210.GA15141@burratino","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-12T19:48:18Z","receivedAt":"2010-11-12T19:48:18Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 12.11.2010 16:52, schrieb Jonathan Nieder:\n> Jens Lehmann wrote:\n>> Am 11.11.2010 20:00, schrieb Jonathan Nieder:\n> \n>>> Yes, that makes some sense to me. Except wouldn't it be a single\n>>> configuration item? \"These submodules should be checked out in all\n>>> but unusual situations, so check them out automatically and keep them\n>>> updated.\"\n>>\n>> Hmm, but we have at least three modes of how to update them:\n>>\n>> 1) Never fetch the submodule (to get new commits the user has to run\n>> \"git fetch --recurse-submodules\" by hand)\n>>\n>> 2) Fetch the submodule each time you fetch the superproject (Which is\n>> really handy when you do development in the submodule too but can\n>> be really inconvenient when you don't)\n>>\n>> 3) Update submodules only when new recorded commits are fetched in\n>> the superproject (This mode is not added with the current patch\n>> series but will be in one of the next)\n>>\n>> So you would need a config option for that anyway, no? And that is why\n>> I'd rather like to have a separate fetch option to control that behavior\n>> instead of an implicit \"if-it's-to-be-checked-out-fetch-it-too\" approach.\n> \n> I still think I am missing something.\n\nOk, I'll try harder to explain my view ;-)\n\n> Traditionally, git has been a _content_ tracker. The configuration that\n> gets transmitted (.gitignore, .gitattributes, .gitmodules) would only\n> represent basic information needed for that content to remain usable and\n> sensible.\n\nAck.\n\n> In the case of .gitmodules, it seems that two concepts are being\n> conflated:\n> \n> A. Configuration based on the user's preferences. Absolutely, a person\n> deserves to be able to easily choose between (1), (2), and (3) as\n> described above.\n> \n> B. Metadata about the content that should be shared. For example, \"this\n> submodule would be checked out in all but unusual circumstances\" is a\n> useful thing to be able to declare.\n\nNo, I think these concepts aren't conflated at all:\n\n- Category A is handled by .git/config\n\n- Category B is handled by the .gitmodules file\n\nCategory A should not be handled by a file in the work tree while\ncategory B stuff belongs there to be able to share it. That is why git\nsearches all new options in both files with the values from .git/config\noverriding those from .gitmodules: to provide useful defaults and let\nthe user choose otherwise if he wants to.\n\n> Probably I am missing something big, but fetchsubmodules as currently\n> defined seem like something in category A and not in category B.\n> Partially because if we ever implement option (3), that is what almost\n> _every_ casual consumer will want. So why should they be stuck with these\n> configuration files specifying (1) and (2) when they check out old\n> revisions?\n\nWith submodules there never is a one-size-fits-all solution :-)\n(And we are currently working on option (3), so it will exist rather\nsooner than later)\n\nThere are people putting lots of large files in submodules for better\nperformance and they almost always never want to fetch (or even stat)\nthem, so (1) is for them and it's cool that their upstream can configure\nthat, avoiding to have every developer to repeat their \"obvious\" choice\nafter each clone again (and that might only be needed for some submodules,\nso a repo-wide config doesn't necessarily help them).\n\nAnd when you are on a superproject branch actively developing inside a\nsubmodule, you may want to increase fetch-activity to fetch all new\ncommits in the submodule even if they aren't referenced in the\nsuperproject (yet), as that might be just what your fellow developers\nare about to do. And the person setting up that branch could do that\nonce for all users so they don't have to repeat it in every clone. And\nwhen switching away from that branch all those developers cannot forget\nto reconfigure to fetch-on-demand, so not having that in .git/config is\na plus here too.\n\nAnd right, \"almost _every_ casual consumer\" should start with (3), and\nthat is why it will be the default. So most people will get along\nwithout having ever to configure fetch behavior in .gitmodules because\ngit does The Right Thing for them. But in my opinion we should allow\nthat for those other users ...\n\n> All that said, I do not think it is unreasonable in some situations for a\n> project to want to share configuration of type (A) between members of a\n> project; for example, lots of projects share hooks and that's great. I\n> just don't think git should set it up automatically --- better to require\n> an explicit user action like \"sh useprojectconfiguration.sh\" (an action\n> more explicit than \"please initialize and checkout all relevant\n> submodules\") for such cases.\n\nYou have no other choice for hooks because of security concerns. But I\ncan't see any downsides in leaving upstream *the choice* to configure\ndefault submodule behavior. Lots of people - including me - want that for\nclone and checkout. And to be consistent the same configuration abilities\nshould be available for fetch too, and if only to follow the principle of\nleast surprise.\n\n\nThanks\nJens\n"},{"id":"155795","messageId":"20101112195453.GA25248@burratino","threadId":"25705","inReplyTo":"4CDD391C.2000905@web.de","subject":"Re: [PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-12T19:54:53Z","receivedAt":"2010-11-12T19:54:53Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n\n> Ok, here is the updated version of patch 1/3 making \"--submodule-prefix\" a\n> hidden option which is not documented in Documentation/fetch-options.txt .\n\nOh, I missed that you were proposing leaving it undocumented. On one hand,\nit is nice when debugging if all options are documented; but on the other\nhand, there is precedent in --rebasing for hiding implementation details\nlike this. Unlike -h output, the manual is not so space-constrainted, so...\n\n> --- /dev/null\n> +++ b/t/t5526-fetch-submodules.sh\n> @@ -0,0 +1,109 @@\n> +#!/bin/sh\n> +# Copyright (c) 2010, Jens Lehmann\n> +\n> +test_description='Recursive \"git fetch\" for submodules'\n> +\n> +. ./test-lib.sh\n> +\n> +pwd=$(pwd)\n> +\n> +add_upstream_commit() {\n> +\t(\n> +\t\tcd submodule &&\n> +\t\thead1=$(git rev-parse --short HEAD) &&\n> +\t\techo new >> subfile &&\n> +\t\ttest_tick &&\n> +\t\tgit add subfile &&\n> +\t\tgit commit -m new subfile &&\n> +\t\thead2=$(git rev-parse --short HEAD) &&\n> +\t\techo \"From $pwd/submodule\" > ../expect.err &&\n> +\t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n> +\t)\n> +\t(\n\nMissing &&?\n\nHope that helps. These are just tiny nitpicks; I am happy with the patch\nwith or without them.\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex 594f7ce..a9cb8ab 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -67,6 +67,13 @@ endif::git-pull[]\n --recurse-submodules::\n \tUse this option to fetch new commits of all populated submodules too.\n \n+ifndef::git-pull[]\n+--submodule-prefix=::\n+\tPrepend to paths printed in informative messages\n+\tsuch as \"Fetching submodule foo\". This option is used\n+\tinternally when recursing over submodules.\n+endif::git-pull[]\n+\n -u::\n --update-head-ok::\n \tBy default 'git fetch' refuses to update the head which\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 107317a..9c1d3a0 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -18,7 +18,7 @@ add_upstream_commit() {\n \t\thead2=$(git rev-parse --short HEAD) &&\n \t\techo \"From $pwd/submodule\" > ../expect.err &&\n \t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n-\t)\n+\t) &&\n \t(\n \t\tcd deepsubmodule &&\n \t\thead1=$(git rev-parse --short HEAD) &&\n"},{"id":"155796","messageId":"20101112201640.GB25248@burratino","threadId":"25705","inReplyTo":"4CDD9A02.6000507@web.de","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-11-12T20:16:40Z","receivedAt":"2010-11-12T20:16:40Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jens Lehmann wrote:\n\n> No, I think these concepts aren't conflated at all:\n> \n> - Category A is handled by .git/config\n> \n> - Category B is handled by the .gitmodules file\n\nI meant that the two files currently support the same submodule.*\noptions.\n\n> There are people putting lots of large files in submodules for better\n> performance and they almost always never want to fetch (or even stat)\n> them, so (1) is for them and it's cool that their upstream can configure\n> that, avoiding to have every developer to repeat their \"obvious\" choice\n> after each clone again (and that might only be needed for some submodules,\n> so a repo-wide config doesn't necessarily help them).\n\nWouldn't (3) work for these people, too?\n\nI think we are getting closer to an explanation. I can look into\nadding documentation for this on top when finished.\n\n> And when you are on a superproject branch actively developing inside a\n> submodule, you may want to increase fetch-activity to fetch all new\n> commits in the submodule even if they aren't referenced in the\n> superproject (yet), as that might be just what your fellow developers\n> are about to do. And the person setting up that branch could do that\n> once for all users so they don't have to repeat it in every clone.\n\nThis one seems less reasonable to me. It seems like a way to\nremotely help developers get a nice setup, rather than a declaration\nabout the content.\n\nLet me take an unrelated example to illustrate what I mean. Some\nprojects might want all their developers to use branch.autosetuprebase,\nto avoid confusion since the update hook is going to reject mergy\nhistory anyway. That seems like a perfectly reasonable desire to me,\nand I'd encourage them to add a script that sets everything up\nfollowing the policies of their project.\n\nNow git could even learn to read a .gitconfig file including settings\nlike that one that do not have a security impact. It would make lots\nof people happy, and individuals could override settings they really\ndislike in ~/.gitconfig. Should we do it?\n\nI think no, for reasons of intuitiveness and predictability.\n\nOn the other hand, scenarios like (1) might mean we have to support\nsuch things despite that downside.\n\n> And\n> when switching away from that branch all those developers cannot forget\n> to reconfigure to fetch-on-demand, so not having that in .git/config is\n> a plus here too.\n\nYes, the \"read .gitmodules first and then .git/config\" is a very nice\nenhancement --- thanks!\n\n> You have no other choice for hooks because of security concerns. But I\n> can't see any downsides in leaving upstream *the choice* to configure\n> default submodule behavior. Lots of people - including me - want that for\n> clone and checkout.\n\nThere is one setting that it is obvious to me for upstream should be\nable to set:\n\n\t\"these submodules are a necessary part of the project;\n\t always (at clone time, fetch time, checkout, etc) make\n\t sure they are available\"\n\nI could easily be convinced about others, but there ought to be a use\ncase to outweigh the \"subtle behavior changing behind my back\" syndrome.\n\nAnd again: thanks for doing all this work. It's inspiring. (Next step\nrecursive push?)\nJonathan\n"},{"id":"155797","messageId":"4CDDA1F5.4090504@web.de","threadId":"25705","inReplyTo":"20101112195453.GA25248@burratino","subject":"Re: [PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-12T20:22:13Z","receivedAt":"2010-11-12T20:22:13Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 12.11.2010 20:54, schrieb Jonathan Nieder:\n> Jens Lehmann wrote:\n> \n>> Ok, here is the updated version of patch 1/3 making \"--submodule-prefix\" a\n>> hidden option which is not documented in Documentation/fetch-options.txt .\n> \n> Oh, I missed that you were proposing leaving it undocumented. On one hand,\n> it is nice when debugging if all options are documented; but on the other\n> hand, there is precedent in --rebasing for hiding implementation details\n> like this. Unlike -h output, the manual is not so space-constrainted, so...\n\nWhoops, my question was about hiding the option and so I have been a bit\novereager to remove the documentation too ...\n\n>> --- /dev/null\n>> +++ b/t/t5526-fetch-submodules.sh\n>> @@ -0,0 +1,109 @@\n>> +#!/bin/sh\n>> +# Copyright (c) 2010, Jens Lehmann\n>> +\n>> +test_description='Recursive \"git fetch\" for submodules'\n>> +\n>> +. ./test-lib.sh\n>> +\n>> +pwd=$(pwd)\n>> +\n>> +add_upstream_commit() {\n>> +\t(\n>> +\t\tcd submodule &&\n>> +\t\thead1=$(git rev-parse --short HEAD) &&\n>> +\t\techo new >> subfile &&\n>> +\t\ttest_tick &&\n>> +\t\tgit add subfile &&\n>> +\t\tgit commit -m new subfile &&\n>> +\t\thead2=$(git rev-parse --short HEAD) &&\n>> +\t\techo \"From $pwd/submodule\" > ../expect.err &&\n>> +\t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n>> +\t)\n>> +\t(\n> \n> Missing &&?\n\nGood eyes!\n\n> Hope that helps. These are just tiny nitpicks; I am happy with the patch\n> with or without them.\n\nThanks!\n\nJunio, do you want me to send an updated version of this patch or are you ok\nwith squashing this in?\n\n\n> diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\n> index 594f7ce..a9cb8ab 100644\n> --- a/Documentation/fetch-options.txt\n> +++ b/Documentation/fetch-options.txt\n> @@ -67,6 +67,13 @@ endif::git-pull[]\n> --recurse-submodules::\n> \tUse this option to fetch new commits of all populated submodules too.\n> \n> +ifndef::git-pull[]\n> +--submodule-prefix=::\n> +\tPrepend to paths printed in informative messages\n> +\tsuch as \"Fetching submodule foo\". This option is used\n> +\tinternally when recursing over submodules.\n> +endif::git-pull[]\n> +\n> -u::\n> --update-head-ok::\n> \tBy default 'git fetch' refuses to update the head which\n> diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\n> index 107317a..9c1d3a0 100755\n> --- a/t/t5526-fetch-submodules.sh\n> +++ b/t/t5526-fetch-submodules.sh\n> @@ -18,7 +18,7 @@ add_upstream_commit() {\n> \t\thead2=$(git rev-parse --short HEAD) &&\n> \t\techo \"From $pwd/submodule\" > ../expect.err &&\n> \t\techo \" $head1..$head2 master -> origin/master\" >> ../expect.err\n> -\t)\n> +\t) &&\n> \t(\n> \t\tcd deepsubmodule &&\n> \t\thead1=$(git rev-parse --short HEAD) &&\n> \n"},{"id":"155801","messageId":"4CDDB87C.2030803@web.de","threadId":"25705","inReplyTo":"20101112201640.GB25248@burratino","subject":"Re: [PATCH v3 2/3] Add the 'fetch.recurseSubmodules' config setting","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-11-12T21:58:20Z","receivedAt":"2010-11-12T21:58:20Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 12.11.2010 21:16, schrieb Jonathan Nieder:\n> Jens Lehmann wrote:\n>> There are people putting lots of large files in submodules for better\n>> performance and they almost always never want to fetch (or even stat)\n>> them, so (1) is for them and it's cool that their upstream can configure\n>> that, avoiding to have every developer to repeat their \"obvious\" choice\n>> after each clone again (and that might only be needed for some submodules,\n>> so a repo-wide config doesn't necessarily help them).\n> \n> Wouldn't (3) work for these people, too?\n\nAs I understand there are people who use submodules as something rather\nunrelated to the superproject line of development and thus recursive on\ndemand fetch (and recursive checkout) would not be what they want (and\ncould take considerable time they don't want to spend).\n\n> I think we are getting closer to an explanation. I can look into\n> adding documentation for this on top when finished.\n\nThat would be great! I had the same thing in mind so I would be happy\nto help you with that if you want.\n\n>> And when you are on a superproject branch actively developing inside a\n>> submodule, you may want to increase fetch-activity to fetch all new\n>> commits in the submodule even if they aren't referenced in the\n>> superproject (yet), as that might be just what your fellow developers\n>> are about to do. And the person setting up that branch could do that\n>> once for all users so they don't have to repeat it in every clone.\n> \n> This one seems less reasonable to me. It seems like a way to\n> remotely help developers get a nice setup, rather than a declaration\n> about the content.\n\nYeah, I know this example is not as convincing as (1). But it was the\nbest I could come up with trying to illustrate the \"treat-submodules-\nas-if-everything-were-in-one-tree\" model which some people use.\n(And we use this model at my dayjob, but we would be fine setting the\nglobal config option once on every developers computer. But hey, adding\nthat to the .gitmodules file would get rid of that too ;-)\n\n> Let me take an unrelated example to illustrate what I mean. Some\n> projects might want all their developers to use branch.autosetuprebase,\n> to avoid confusion since the update hook is going to reject mergy\n> history anyway. That seems like a perfectly reasonable desire to me,\n> and I'd encourage them to add a script that sets everything up\n> following the policies of their project.\n> \n> Now git could even learn to read a .gitconfig file including settings\n> like that one that do not have a security impact. It would make lots\n> of people happy, and individuals could override settings they really\n> dislike in ~/.gitconfig. Should we do it?\n\nHmm, I never thought of that. But you are right, this is the same\nprinciple I'm using for .gitmodules.\n\n> I think no, for reasons of intuitiveness and predictability.\n> \n> On the other hand, scenarios like (1) might mean we have to support\n> such things despite that downside.\n\nYup.\n\n>> And\n>> when switching away from that branch all those developers cannot forget\n>> to reconfigure to fetch-on-demand, so not having that in .git/config is\n>> a plus here too.\n> \n> Yes, the \"read .gitmodules first and then .git/config\" is a very nice\n> enhancement --- thanks!\n\nYou are welcome!\n\n>> You have no other choice for hooks because of security concerns. But I\n>> can't see any downsides in leaving upstream *the choice* to configure\n>> default submodule behavior. Lots of people - including me - want that for\n>> clone and checkout.\n> \n> There is one setting that it is obvious to me for upstream should be\n> able to set:\n> \n> \t\"these submodules are a necessary part of the project;\n> \t always (at clone time, fetch time, checkout, etc) make\n> \t sure they are available\"\n> \n> I could easily be convinced about others, but there ought to be a use\n> case to outweigh the \"subtle behavior changing behind my back\" syndrome.\n\nI don't have a problem with dropping 3/3 from this series if use case (1)\ndoesn't convince people. I added it to be consistent with the behavior of\nthe 'ignore' flag I added earlier and because I it is needed to support\nmore than one use case for submodules in the same superproject. But we\ncould wait if someone complains and add it if that happens ... I dunno.\n\n> And again: thanks for doing all this work. It's inspiring. (Next step\n> recursive push?)\n\nThanks a lot! (And nope, it's recursive checkout - thats why I started\nrecursive fetch, so that I do have something to check out ;-).\n\nJens\n"},{"id":"157729","messageId":"7vei9qfxzy.fsf@alter.siamese.dyndns.org","threadId":"25705","inReplyTo":"4CDD391C.2000905@web.de","subject":"Re: [PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-12-09T21:16:17Z","receivedAt":"2010-12-09T21:16:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann writes:\n\n> Until now you had to call \"git submodule update\" (without -N|--no-fetch\n> option) or something like \"git submodule foreach git fetch\" to fetch\n> new commits in populated submodules from their remote.\n> ...\n> diff --git a/builtin/fetch.c b/builtin/fetch.c\n> index d35f000..db3fba3 100644\n> --- a/builtin/fetch.c\n> +++ b/builtin/fetch.c\n> ...\n> @@ -784,28 +790,36 @@ static int add_remote_or_group(const char *name, struct string_list *list)\n> \treturn 1;\n> }\n>\n> -static int fetch_multiple(struct string_list *list)\n> +static void add_options_to_argv(int *argc, const char **argv)\n> {\n> -\tint i, result = 0;\n> -\tconst char *argv[11] = { \"fetch\", \"--append\" };\n> -...\n> +static int fetch_multiple(struct string_list *list)\n> +{\n> +\tint i, result = 0;\n> +\tconst char *argv[12] = { \"fetch\", \"--append\" };\n\nThis used to be 11; are we adding something new? Ahh, possibly\n\"--recurse_submodules\".\n\n> diff --git a/submodule.c b/submodule.c\n> index 91a4758..4d9b774 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -63,7 +63,7 @@ void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n> \t}\n> }\n>\n> -static int submodule_config(const char *var, const char *value, void *cb)\n> +int submodule_config(const char *var, const char *value, void *cb)\n> {\n> \tif (!prefixcmp(var, \"submodule.\"))\n> \t\treturn parse_submodule_config_option(var, value);\n> @@ -229,6 +229,70 @@ void show_submodule_summary(FILE *f, const char *path,\n> \tstrbuf_release(&sb);\n> }\n>\n> +int fetch_populated_submodules(int num_options, const char **options,\n> +\t\t\t const char *prefix, int quiet)\n> +{\n> +\tint i, result = 0, argc = 0;\n> +\tstruct child_process cp;\n> +\tconst char **argv;\n> +\tstruct string_list_item *name_for_path;\n> +\tconst char *work_tree = get_git_work_tree();\n> +\tif (!work_tree)\n> +\t\treturn 0;\n> +\n> +\tif (!the_index.initialized)\n> +\t\tif (read_cache() < 0)\n> +\t\t\tdie(\"index file corrupt\");\n> +\n> +\targv = xcalloc(num_options + 5, sizeof(const char *));\n\nWhere is this '5' coming from? \"fetch\" \"--submodule-prefix\", the prefix,\nand the terminating NULL? What did I miss?\n"},{"id":"157761","messageId":"4D016146.9010907@web.de","threadId":"25705","inReplyTo":"7vei9qfxzy.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-12-09T23:07:50Z","receivedAt":"2010-12-09T23:07:50Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 09.12.2010 22:16, schrieb Junio C Hamano:\n> Jens Lehmann writes:\n> \n>> Until now you had to call \"git submodule update\" (without -N|--no-fetch\n>> option) or something like \"git submodule foreach git fetch\" to fetch\n>> new commits in populated submodules from their remote.\n>> ...\n>> diff --git a/builtin/fetch.c b/builtin/fetch.c\n>> index d35f000..db3fba3 100644\n>> --- a/builtin/fetch.c\n>> +++ b/builtin/fetch.c\n>> ...\n>> @@ -784,28 +790,36 @@ static int add_remote_or_group(const char *name, struct string_list *list)\n>> \treturn 1;\n>> }\n>>\n>> -static int fetch_multiple(struct string_list *list)\n>> +static void add_options_to_argv(int *argc, const char **argv)\n>> {\n>> -\tint i, result = 0;\n>> -\tconst char *argv[11] = { \"fetch\", \"--append\" };\n>> -...\n>> +static int fetch_multiple(struct string_list *list)\n>> +{\n>> +\tint i, result = 0;\n>> +\tconst char *argv[12] = { \"fetch\", \"--append\" };\n> \n> This used to be 11; are we adding something new? Ahh, possibly\n> \"--recurse_submodules\".\n\nYup, that's why! (see a few lines down between \"--keep\" and \"-v\").\n\n>> diff --git a/submodule.c b/submodule.c\n>> index 91a4758..4d9b774 100644\n>> --- a/submodule.c\n>> +++ b/submodule.c\n>> @@ -63,7 +63,7 @@ void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n>> \t}\n>> }\n>>\n>> -static int submodule_config(const char *var, const char *value, void *cb)\n>> +int submodule_config(const char *var, const char *value, void *cb)\n>> {\n>> \tif (!prefixcmp(var, \"submodule.\"))\n>> \t\treturn parse_submodule_config_option(var, value);\n>> @@ -229,6 +229,70 @@ void show_submodule_summary(FILE *f, const char *path,\n>> \tstrbuf_release(&sb);\n>> }\n>>\n>> +int fetch_populated_submodules(int num_options, const char **options,\n>> +\t\t\t const char *prefix, int quiet)\n>> +{\n>> +\tint i, result = 0, argc = 0;\n>> +\tstruct child_process cp;\n>> +\tconst char **argv;\n>> +\tstruct string_list_item *name_for_path;\n>> +\tconst char *work_tree = get_git_work_tree();\n>> +\tif (!work_tree)\n>> +\t\treturn 0;\n>> +\n>> +\tif (!the_index.initialized)\n>> +\t\tif (read_cache() < 0)\n>> +\t\t\tdie(\"index file corrupt\");\n>> +\n>> +\targv = xcalloc(num_options + 5, sizeof(const char *));\n> \n> Where is this '5' coming from? \"fetch\" \"--submodule-prefix\", the prefix,\n> and the terminating NULL? What did I miss?\n\nNo, you didn't miss anything but I have been off by one ... '4' is\nsufficient here.\n"},{"id":"157828","messageId":"7vlj3xedru.fsf@alter.siamese.dyndns.org","threadId":"25705","inReplyTo":"4D016146.9010907@web.de","subject":"Re: [PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-12-10T17:30:45Z","receivedAt":"2010-12-10T17:30:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jens Lehmann writes:\n\n> Am 09.12.2010 22:16, schrieb Junio C Hamano:\n> ...\n>>> +int fetch_populated_submodules(int num_options, const char **options,\n>>> +\t\t\t const char *prefix, int quiet)\n>>> +{\n>>> +\tint i, result = 0, argc = 0;\n>>> +\tstruct child_process cp;\n>>> +\tconst char **argv;\n>>> +\tstruct string_list_item *name_for_path;\n>>> +\tconst char *work_tree = get_git_work_tree();\n>>> +\tif (!work_tree)\n>>> +\t\treturn 0;\n>>> +\n>>> +\tif (!the_index.initialized)\n>>> +\t\tif (read_cache() < 0)\n>>> +\t\t\tdie(\"index file corrupt\");\n>>> +\n>>> +\targv = xcalloc(num_options + 5, sizeof(const char *));\n>> \n>> Where is this '5' coming from? \"fetch\" \"--submodule-prefix\", the prefix,\n>> and the terminating NULL? What did I miss?\n>\n> No, you didn't miss anything but I have been off by one ... '4' is\n> sufficient here.\n\nOk, thanks for double checking.\n\n-- >8 --\nSubject: [PATCH] fetch_populated_submodules(): document dynamic allocation size\n\n... while fixing a miscounting.\n\nSigned-off-by: Junio C Hamano \n---\n submodule.c | 3 ++-\n 1 files changed, 2 insertions(+), 1 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 4e62900..6f1c107 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -264,7 +264,8 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\tif (read_cache() < 0)\n \t\t\tdie(\"index file corrupt\");\n \n-\targv = xcalloc(num_options + 5, sizeof(const char *));\n+\t/* 4: \"fetch\" (options) \"--submodule-prefix\" prefix NULL */\n+\targv = xcalloc(num_options + 4, sizeof(const char *));\n \targv[argc++] = \"fetch\";\n \tfor (i = 0; i < num_options; i++)\n \t\targv[argc++] = options[i];\n"},{"id":"157829","messageId":"4D026B7E.3000008@web.de","threadId":"25705","inReplyTo":"7vlj3xedru.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v4 1/3] fetch/pull: Add the --recurse-submodules option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2010-12-10T18:03:42Z","receivedAt":"2010-12-10T18:03:42Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 10.12.2010 18:30, schrieb Junio C Hamano:\n> Jens Lehmann writes:\n>> No, you didn't miss anything but I have been off by one ... '4' is\n>> sufficient here.\n> \n> Ok, thanks for double checking.\n> \n> -- >8 --\n> Subject: [PATCH] fetch_populated_submodules(): document dynamic allocation size\n\nThanks and ack.\n\n(This was a leftover from the time the \"--recurse-submodules\" option had been\nadded in this function before I moved that into add_options_to_argv())\n"}]}