{"thread":{"id":"26635","subject":"[PATCH v2 0/7] Teach fetch/pull the on-demand mode and make it the default","startedAt":"2011-03-01T22:59:28Z","lastAt":"2011-03-02T23:35:13Z","messageCount":16,"participants":["Jens Lehmann","Marc Branchaud"],"isPatch":true,"patchVersion":2,"patchTotal":7},"messages":[{"id":"162601","messageId":"4D6D7A50.5090802@web.de","threadId":"26635","inReplyTo":null,"subject":"[PATCH v2 0/7] Teach fetch/pull the on-demand mode and make it the default","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T22:59:28Z","receivedAt":"2011-03-01T22:59:28Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"So here is version 2 of the on-demand recursive fetch for submodules.\n\nChanges since the last version:\n\n*) Add a name parameter to parse_fetch_recurse_submodules_arg()\n*) Enhance description of the \"--submodule-default\" fetch option\n*) Free the changed_submodule_paths string_list at the end of\n   fetch_populated_submodules() to aid libification and to make\n   valgrind happy\n*) Add a comment for the while loop in check_for_new_submodule_commits()\n*) Use \"--all\" instead of \"--branches\" and \"--remotes\" for rev-list to\n   include tags too\n*) Compute the argc value properly instead of hard coding it in\n   check_for_new_submodule_commits()\n*) Change size_t casts to intptr_t where appropriate\n*) Use \"rev-list --not --all\" when checking if a submodule commit exists\n   instead of a simple lookup/rev-parse to make sure they are not only\n   present but are reachable from a ref\n*) Describe that fetch and pull only work on populated submodules for\n   now in the BUGS section of their man pages (done in a separate commit\n   because this doesn't belong to a special commit in this series but\n   also to the functionality that has already been merged)\n\nThanks to Jonathan an Junio for the feedback that lead to these changes.\n\nIssues that were raised but not addressed yet:\n\n*) Should this option be called \"on-demand\" or \"changed\" or maybe even\n   \"updated\"?\n   I have no strong feelings about this, me just thinks the name should\n   tell somehow that only submodules where new commits are recorded in\n   the superproject which aren't already present locally will be fetched.\n   Opinions?\n*) Check the remote tracking refs in the submodule in tests instead of\n   the output of the fetch command\n\n\nJens Lehmann (7):\n  fetch/pull: recurse into submodules when necessary\n  fetch/pull: Add the 'on-demand' value to the --recurse-submodules\n    option\n  config: teach the fetch.recurseSubmodules option the 'on-demand'\n    value\n  Submodules: Add 'on-demand' value for the 'fetchRecurseSubmodule'\n    option\n  fetch/pull: Don't recurse into a submodule when commits are already\n    present\n  submodule update: Don't fetch when the submodule commit is already\n    present\n  fetch/pull: Describe --recurse-submodule restrictions in the BUGS\n    section\n\n Documentation/config.txt        |   12 ++-\n Documentation/fetch-options.txt |   27 ++++-\n Documentation/git-fetch.txt     |    9 ++\n Documentation/git-pull.txt      |   11 ++-\n Documentation/gitmodules.txt    |    4 +-\n builtin/fetch.c                 |   47 ++++++--\n git-pull.sh                     |    3 +\n git-submodule.sh                |    5 +-\n submodule.c                     |  152 +++++++++++++++++++++--\n submodule.h                     |   11 ++-\n t/t5526-fetch-submodules.sh     |  255 +++++++++++++++++++++++++++++++++++++++\n t/t7403-submodule-sync.sh       |    2 +-\n t/t7406-submodule-update.sh     |   20 +++\n 13 files changed, 521 insertions(+), 37 deletions(-)\n\n-- \n1.7.4.1.291.g6060\n"},{"id":"162602","messageId":"4D6D7AB2.4010509@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v2 1/7] fetch/pull: recurse into submodules when necessary","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:01:06Z","receivedAt":"2011-03-01T23:01:06Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"To be able to access all commits of populated submodules referenced by the\nsuperproject it is sufficient to only then let \"git fetch\" recurse into a\nsubmodule when the new commits fetched in the superproject record new\ncommits for it. Having these commits present is extremely useful when\nusing the \"--submodule\" option to \"git diff\" (which is what \"git gui\" and\n\"gitk\" do since 1.6.6), as all submodule commits needed for creating a\ndescriptive output can be accessed. Also merging submodule commits (added\nin 1.7.3) depends on the submodule commits in question being present to\nwork. Last but not least this enables disconnected operation when using\nsubmodules, as all commits necessary for a successful \"git submodule\nupdate -N\" will have been fetched automatically. So we choose this mode as\nthe default for fetch and pull.\n\nBefore a new or changed ref from upstream is updated in update_local_ref()\n\"git rev-list <new-sha1> --not --branches --remotes\" is used to determine\nall newly fetched commits. These are then walked and diffed against their\nparent(s) to see if a submodule has been changed. If that is the case, its\npath is stored to be fetched after the superproject fetch is completed.\n\nUsing the \"--recurse-submodules\" or the \"--no-recurse-submodules\" option\ndisables the examination of the fetched refs because the result will be\nignored anyway.\n\nThere is currently no infrastructure for storing deleted and new\nsubmodules in the .git directory of the superproject. Thats why fetch and\npull for now only fetch submodules that are already checked out and are\nnot renamed.\n\nIn t7403 the \"--no-recurse-submodules\" argument had to be added to \"git\npull\" to avoid failure because of the moved upstream submodule repo.\n\nThanks-to: Jonathan Nieder <jrnieder@gmail.com>\nThanks-to: Heiko Voigt <hvoigt@hvoigt.net>\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n Documentation/fetch-options.txt |   11 ++++\n builtin/fetch.c                 |   25 ++++++---\n submodule.c                     |  106 +++++++++++++++++++++++++++++++++++---\n submodule.h                     |    9 +++\n t/t5526-fetch-submodules.sh     |  109 +++++++++++++++++++++++++++++++++++++++\n t/t7403-submodule-sync.sh       |    2 +-\n 6 files changed, 245 insertions(+), 17 deletions(-)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex f37276e..ae22f75 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -73,6 +73,17 @@ ifndef::git-pull[]\n \tPrepend <path> to paths printed in informative messages\n \tsuch as \"Fetching submodule foo\".  This option is used\n \tinternally when recursing over submodules.\n+\n+--submodule-default=[yes|on-demand]::\n+\tThis option is used internally to set the submodule recursion default\n+\tto either a boolean configuration value representing \"true\" (for\n+\tunconditonal recursion) or to \"on-demand\" (when only those submodules\n+\tshould be fetched of which new commits have been fetched in its\n+\tsuperproject).\n+\tThis option will be overridden by the 'fetch.recurseSubmodules' and\n+\t'submodule.<name>.fetchRecurseSubmodules' settings in\n+\tlinkgit:gitmodules[5] and linkgit:git-config[1] while all of them\n+\tare ignored when the \"--[no-]recurse-submodules\" option is given.\n endif::git-pull[]\n\n -u::\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 7efecfe..55d6b1a 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -28,12 +28,6 @@ enum {\n \tTAGS_SET = 2\n };\n\n-enum {\n-\tRECURSE_SUBMODULES_OFF = 0,\n-\tRECURSE_SUBMODULES_DEFAULT = 1,\n-\tRECURSE_SUBMODULES_ON = 2\n-};\n-\n static int all, append, dry_run, force, keep, multiple, prune, update_head_ok, verbosity;\n static int progress, recurse_submodules = RECURSE_SUBMODULES_DEFAULT;\n static int tags = TAGS_DEFAULT;\n@@ -42,6 +36,7 @@ static const char *upload_pack;\n static struct strbuf default_rla = STRBUF_INIT;\n static struct transport *transport;\n static const char *submodule_prefix = \"\";\n+static const char *submodule_default;\n\n static struct option builtin_fetch_options[] = {\n \tOPT__VERBOSITY(&verbosity),\n@@ -73,6 +68,8 @@ static struct option builtin_fetch_options[] = {\n \t\t   \"deepen history of shallow clone\"),\n \t{ OPTION_STRING, 0, \"submodule-prefix\", &submodule_prefix, \"dir\",\n \t\t   \"prepend this to submodule path output\", PARSE_OPT_HIDDEN },\n+\t{ OPTION_STRING, 0, \"submodule-default\", &submodule_default, NULL,\n+\t\t   \"default mode for recursion\", PARSE_OPT_HIDDEN },\n \tOPT_END()\n };\n\n@@ -284,6 +281,9 @@ static int update_local_ref(struct ref *ref,\n \t\telse {\n \t\t\tmsg = \"storing head\";\n \t\t\twhat = \"[new branch]\";\n+\t\t\tif ((recurse_submodules != RECURSE_SUBMODULES_OFF) &&\n+\t\t\t    (recurse_submodules != RECURSE_SUBMODULES_ON))\n+\t\t\t\tcheck_for_new_submodule_commits(ref->new_sha1);\n \t\t}\n\n \t\tr = s_update_ref(msg, ref, 0);\n@@ -299,6 +299,9 @@ static int update_local_ref(struct ref *ref,\n \t\tstrcpy(quickref, find_unique_abbrev(current->object.sha1, DEFAULT_ABBREV));\n \t\tstrcat(quickref, \"..\");\n \t\tstrcat(quickref, find_unique_abbrev(ref->new_sha1, DEFAULT_ABBREV));\n+\t\tif ((recurse_submodules != RECURSE_SUBMODULES_OFF) &&\n+\t\t    (recurse_submodules != RECURSE_SUBMODULES_ON))\n+\t\t\tcheck_for_new_submodule_commits(ref->new_sha1);\n \t\tr = s_update_ref(\"fast-forward\", ref, 1);\n \t\tsprintf(display, \"%c %-*s %-*s -> %s%s\", r ? '!' : ' ',\n \t\t\tTRANSPORT_SUMMARY_WIDTH, quickref, REFCOL_WIDTH, remote,\n@@ -310,6 +313,9 @@ static int update_local_ref(struct ref *ref,\n \t\tstrcpy(quickref, find_unique_abbrev(current->object.sha1, DEFAULT_ABBREV));\n \t\tstrcat(quickref, \"...\");\n \t\tstrcat(quickref, find_unique_abbrev(ref->new_sha1, DEFAULT_ABBREV));\n+\t\tif ((recurse_submodules != RECURSE_SUBMODULES_OFF) &&\n+\t\t    (recurse_submodules != RECURSE_SUBMODULES_ON))\n+\t\t\tcheck_for_new_submodule_commits(ref->new_sha1);\n \t\tr = s_update_ref(\"forced-update\", ref, 1);\n \t\tsprintf(display, \"%c %-*s %-*s -> %s  (%s)\", r ? '!' : '+',\n \t\t\tTRANSPORT_SUMMARY_WIDTH, quickref, REFCOL_WIDTH, remote,\n@@ -949,9 +955,10 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \tif (!result && (recurse_submodules != RECURSE_SUBMODULES_OFF)) {\n \t\tconst char *options[10];\n \t\tint num_options = 0;\n-\t\t/* Set recursion as default when we already are recursing */\n-\t\tif (submodule_prefix[0])\n-\t\t\tset_config_fetch_recurse_submodules(1);\n+\t\tif (submodule_default) {\n+\t\t\tint arg = parse_fetch_recurse_submodules_arg(\"--submodule-default\", submodule_default);\n+\t\t\tset_config_fetch_recurse_submodules(arg);\n+\t\t}\n \t\tgitmodules_config();\n \t\tgit_config(submodule_config, NULL);\n \t\tadd_options_to_argv(&num_options, options);\ndiff --git a/submodule.c b/submodule.c\nindex 6f1c107..8f991ce 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -12,7 +12,8 @@\n struct string_list config_name_for_path;\n struct string_list config_fetch_recurse_submodules_for_name;\n struct string_list config_ignore_for_name;\n-static int config_fetch_recurse_submodules;\n+static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;\n+struct string_list changed_submodule_paths;\n\n static int add_submodule_odb(const char *path)\n {\n@@ -152,6 +153,20 @@ void handle_ignore_submodules_arg(struct diff_options *diffopt,\n \t\tdie(\"bad --ignore-submodules argument: %s\", arg);\n }\n\n+int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)\n+{\n+\tswitch (git_config_maybe_bool(opt, arg)) {\n+\tcase 1:\n+\t\treturn RECURSE_SUBMODULES_ON;\n+\tcase 0:\n+\t\treturn RECURSE_SUBMODULES_OFF;\n+\tdefault:\n+\t\tif (!strcmp(arg, \"on-demand\"))\n+\t\t\treturn RECURSE_SUBMODULES_ON_DEMAND;\n+\t\tdie(\"bad %s argument: %s\", opt, arg);\n+\t}\n+}\n+\n void show_submodule_summary(FILE *f, const char *path,\n \t\tunsigned char one[20], unsigned char two[20],\n \t\tunsigned dirty_submodule,\n@@ -248,27 +263,95 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n\n+static void submodule_collect_changed_cb(struct diff_queue_struct *q,\n+\t\t\t\t\t struct diff_options *options,\n+\t\t\t\t\t void *data)\n+{\n+\tint i;\n+\tfor (i = 0; i < q->nr; i++) {\n+\t\tstruct diff_filepair *p = q->queue[i];\n+\t\tif (!S_ISGITLINK(p->two->mode))\n+\t\t\tcontinue;\n+\n+\t\tif (S_ISGITLINK(p->one->mode)) {\n+\t\t\t/* NEEDSWORK: We should honor the name configured in\n+\t\t\t * the .gitmodules file of the commit we are examining\n+\t\t\t * here to be able to correctly follow submodules\n+\t\t\t * being moved around. */\n+\t\t\tstruct string_list_item *path;\n+\t\t\tpath = unsorted_string_list_lookup(&changed_submodule_paths, p->two->path);\n+\t\t\tif (!path)\n+\t\t\t\tstring_list_append(&changed_submodule_paths, xstrdup(p->two->path));\n+\t\t} else {\n+\t\t\t/* Submodule is new or was moved here */\n+\t\t\t/* NEEDSWORK: When the .git directories of submodules\n+\t\t\t * live inside the superprojects .git directory some\n+\t\t\t * day we should fetch new submodules directly into\n+\t\t\t * that location too when config or options request\n+\t\t\t * that so they can be checked out from there. */\n+\t\t\tcontinue;\n+\t\t}\n+\t}\n+}\n+\n+void check_for_new_submodule_commits(unsigned char new_sha1[20])\n+{\n+\tstruct rev_info rev;\n+\tstruct commit *commit;\n+\tconst char *argv[] = {NULL, NULL, \"--not\", \"--all\", NULL};\n+\tint argc = ARRAY_SIZE(argv) - 1;\n+\n+\tinit_revisions(&rev, NULL);\n+\targv[1] = xstrdup(sha1_to_hex(new_sha1));\n+\tsetup_revisions(argc, argv, &rev, NULL);\n+\tif (prepare_revision_walk(&rev))\n+\t\tdie(\"revision walk setup failed\");\n+\n+\t/*\n+\t * Collect all submodules (whether checked out or not) for which new\n+\t * commits have been recorded upstream in \"changed_submodule_paths\".\n+\t */\n+\twhile ((commit = get_revision(&rev))) {\n+\t\tstruct commit_list *parent = commit->parents;\n+\t\twhile (parent) {\n+\t\t\tstruct diff_options diff_opts;\n+\t\t\tdiff_setup(&diff_opts);\n+\t\t\tdiff_opts.output_format |= DIFF_FORMAT_CALLBACK;\n+\t\t\tdiff_opts.format_callback = submodule_collect_changed_cb;\n+\t\t\tif (diff_setup_done(&diff_opts) < 0)\n+\t\t\t\tdie(\"diff_setup_done failed\");\n+\t\t\tdiff_tree_sha1(parent->item->object.sha1, commit->object.sha1, \"\", &diff_opts);\n+\t\t\tdiffcore_std(&diff_opts);\n+\t\t\tdiff_flush(&diff_opts);\n+\t\t\tparent = parent->next;\n+\t\t}\n+\t}\n+\tfree((char *)argv[1]);\n+}\n+\n int fetch_populated_submodules(int num_options, const char **options,\n \t\t\t       const char *prefix, int ignore_config,\n \t\t\t       int quiet)\n {\n-\tint i, result = 0, argc = 0;\n+\tint i, result = 0, argc = 0, default_argc;\n \tstruct child_process cp;\n \tconst char **argv;\n \tstruct string_list_item *name_for_path;\n \tconst char *work_tree = get_git_work_tree();\n \tif (!work_tree)\n-\t\treturn 0;\n+\t\tgoto out;\n\n \tif (!the_index.initialized)\n \t\tif (read_cache() < 0)\n \t\t\tdie(\"index file corrupt\");\n\n-\t/* 4: \"fetch\" (options) \"--submodule-prefix\" prefix NULL */\n-\targv = xcalloc(num_options + 4, sizeof(const char *));\n+\t/* 6: \"fetch\" (options) --submodule-default default \"--submodule-prefix\" prefix NULL */\n+\targv = xcalloc(num_options + 6, sizeof(const char *));\n \targv[argc++] = \"fetch\";\n \tfor (i = 0; i < num_options; i++)\n \t\targv[argc++] = options[i];\n+\targv[argc++] = \"--submodule-default\";\n+\tdefault_argc = argc++;\n \targv[argc++] = \"--submodule-prefix\";\n\n \tmemset(&cp, 0, sizeof(cp));\n@@ -282,7 +365,7 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\tstruct strbuf submodule_git_dir = STRBUF_INIT;\n \t\tstruct strbuf submodule_prefix = STRBUF_INIT;\n \t\tstruct cache_entry *ce = active_cache[i];\n-\t\tconst char *git_dir, *name;\n+\t\tconst char *git_dir, *name, *default_argv;\n\n \t\tif (!S_ISGITLINK(ce->ce_mode))\n \t\t\tcontinue;\n@@ -292,6 +375,7 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\tif (name_for_path)\n \t\t\tname = name_for_path->util;\n\n+\t\tdefault_argv = \"yes\";\n \t\tif (!ignore_config) {\n \t\t\tstruct string_list_item *fetch_recurse_submodules_option;\n \t\t\tfetch_recurse_submodules_option = unsorted_string_list_lookup(&config_fetch_recurse_submodules_for_name, name);\n@@ -299,8 +383,13 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\t\tif (!fetch_recurse_submodules_option->util)\n \t\t\t\t\tcontinue;\n \t\t\t} else {\n-\t\t\t\tif (!config_fetch_recurse_submodules)\n+\t\t\t\tif (config_fetch_recurse_submodules == RECURSE_SUBMODULES_OFF)\n \t\t\t\t\tcontinue;\n+\t\t\t\tif (config_fetch_recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND) {\n+\t\t\t\t\tif (!unsorted_string_list_lookup(&changed_submodule_paths, ce->name))\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\tdefault_argv = \"on-demand\";\n+\t\t\t\t}\n \t\t\t}\n \t\t}\n\n@@ -314,6 +403,7 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\tif (!quiet)\n \t\t\t\tprintf(\"Fetching submodule %s%s\\n\", prefix, ce->name);\n \t\t\tcp.dir = submodule_path.buf;\n+\t\t\targv[default_argc] = default_argv;\n \t\t\targv[argc] = submodule_prefix.buf;\n \t\t\tif (run_command(&cp))\n \t\t\t\tresult = 1;\n@@ -323,6 +413,8 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\tstrbuf_release(&submodule_prefix);\n \t}\n \tfree(argv);\n+out:\n+\tstring_list_clear(&changed_submodule_paths, 1);\n \treturn result;\n }\n\ndiff --git a/submodule.h b/submodule.h\nindex 4729023..3434a8e 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -3,17 +3,26 @@\n\n struct diff_options;\n\n+enum {\n+\tRECURSE_SUBMODULES_ON_DEMAND = -1,\n+\tRECURSE_SUBMODULES_OFF = 0,\n+\tRECURSE_SUBMODULES_DEFAULT = 1,\n+\tRECURSE_SUBMODULES_ON = 2\n+};\n+\n void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,\n \t\tconst char *path);\n int submodule_config(const char *var, const char *value, void *cb);\n void gitmodules_config();\n int parse_submodule_config_option(const char *var, const char *value);\n void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);\n+int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);\n void show_submodule_summary(FILE *f, const char *path,\n \t\tunsigned char one[20], unsigned char two[20],\n \t\tunsigned dirty_submodule,\n \t\tconst char *del, const char *add, const char *reset);\n void set_config_fetch_recurse_submodules(int value);\n+void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n int fetch_populated_submodules(int num_options, const char **options,\n \t\t\t       const char *prefix, int ignore_config,\n \t\t\t       int quiet);\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex a5f4585..6d92f7a 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -192,4 +192,113 @@ test_expect_success \"--no-recurse-submodules overrides config setting\" '\n \t! test -s actual.err\n '\n\n+test_expect_success \"Recursion doesn't happen when no new commits are fetched in the superproject\" '\n+\t(\n+\t\tcd downstream &&\n+\t\t(\n+\t\t\tcd submodule &&\n+\t\t\tgit config --unset fetch.recurseSubmodules\n+\t\t) &&\n+\t\tgit config --unset fetch.recurseSubmodules\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"Recursion stops when no new submodule commits are fetched\" '\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"new submodule\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"Fetching submodule submodule\" > expect.out.sub &&\n+\techo \"From $pwd/.\" > expect.err.sub &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.sub\n+\thead -2 expect.err >> expect.err.sub &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.err.sub actual.err &&\n+\ttest_cmp expect.out.sub actual.out\n+'\n+\n+test_expect_success \"Recursion doesn't happen when new superproject commits don't change any submodules\" '\n+\tadd_upstream_commit &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\techo a > file &&\n+\tgit add file &&\n+\tgit commit -m \"new file\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err.file &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.file &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\ttest_cmp expect.err.file actual.err\n+'\n+\n+test_expect_success \"Recursion picks up config in submodule\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules &&\n+\t\t(\n+\t\t\tcd submodule &&\n+\t\t\tgit config fetch.recurseSubmodules true\n+\t\t)\n+\t) &&\n+\tadd_upstream_commit &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"new submodule\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err.sub &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.sub &&\n+\tcat expect.err >> expect.err.sub &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err &&\n+\t\t(\n+\t\t\tcd submodule &&\n+\t\t\tgit config --unset fetch.recurseSubmodules\n+\t\t)\n+\t) &&\n+\ttest_cmp expect.err.sub actual.err &&\n+\ttest_cmp expect.out actual.out\n+'\n+\n+test_expect_success \"Recursion picks up all submodules when necessary\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd submodule &&\n+\t\t(\n+\t\t\tcd deepsubmodule &&\n+\t\t\tgit fetch &&\n+\t\t\tgit checkout -q FETCH_HEAD\n+\t\t) &&\n+\t\thead1=$(git rev-parse --short HEAD^) &&\n+\t\tgit add deepsubmodule &&\n+\t\tgit commit -m \"new deepsubmodule\"\n+\t\thead2=$(git rev-parse --short HEAD) &&\n+\t\techo \"From $pwd/submodule\" > ../expect.err.sub &&\n+\t\techo \"   $head1..$head2  master     -> origin/master\" >> ../expect.err.sub\n+\t) &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"new submodule\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err.2 &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.2 &&\n+\tcat expect.err.sub >> expect.err.2 &&\n+\ttail -2 expect.err >> expect.err.2 &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\ttest_cmp expect.err.2 actual.err &&\n+\ttest_cmp expect.out actual.out\n+'\n+\n test_done\ndiff --git a/t/t7403-submodule-sync.sh b/t/t7403-submodule-sync.sh\nindex e5b1953..d600583 100755\n--- a/t/t7403-submodule-sync.sh\n+++ b/t/t7403-submodule-sync.sh\n@@ -52,7 +52,7 @@ test_expect_success 'change submodule url' '\n\n test_expect_success '\"git submodule sync\" should update submodule URLs' '\n \t(cd super-clone &&\n-\t git pull &&\n+\t git pull --no-recurse-submodules &&\n \t git submodule sync\n \t) &&\n \ttest -d \"$(git config -f super-clone/submodule/.git/config \\\n-- \n1.7.4.1.291.g6060\n"},{"id":"162603","messageId":"4D6D7ADA.6090302@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v2 2/7] fetch/pull: Add the 'on-demand' value to the --recurse-submodules option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:01:46Z","receivedAt":"2011-03-01T23:01:46Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Until now the --recurse-submodules option could only be used to either\nfetch all populated submodules recursively or to disable recursion\ncompletely. As fetch and pull now by default just fetch those submodules\nfor which new commits have been fetched in the superproject, a command\nline option to enforce that behavior is needed to be able to override\nconfiguration settings.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n Documentation/fetch-options.txt |   16 +++++++--\n Documentation/git-pull.txt      |    2 +-\n builtin/fetch.c                 |   22 ++++++++++--\n git-pull.sh                     |    3 ++\n submodule.c                     |    8 +++-\n submodule.h                     |    2 +-\n t/t5526-fetch-submodules.sh     |   71 +++++++++++++++++++++++++++++++++++++++\n 7 files changed, 114 insertions(+), 10 deletions(-)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex ae22f75..f28c0b1 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -65,9 +65,19 @@ ifndef::git-pull[]\n \tspecified with the remote.<name>.tagopt setting. See\n \tlinkgit:git-config[1].\n\n---[no-]recurse-submodules::\n-\tThis option controls if new commits of all populated submodules should\n-\tbe fetched too (see linkgit:git-config[1] and linkgit:gitmodules[5]).\n+--recurse-submodules[=yes|on-demand|no]::\n+\tThis option controls if and under what conditions new commits of all\n+\tpopulated submodules should be fetched too. It can be used as a\n+\tboolean option to completely disable recursion when set to 'no' or to\n+\tunconditionally recurse into all populated submodules when set to\n+\t'yes', which is the default when this option is used without any\n+\tvalue. If 'on-demand' is used, it will only recurse into those\n+\tsubmodules where new commits have been fetched in the superproject\n+\t(also see linkgit:git-config[1] and linkgit:gitmodules[5]).\n+\n+--no-recurse-submodules::\n+\tDisable recursive fetching of submodules (this has the same effect as\n+\tusing the '--recurse-submodules=no' option).\n\n --submodule-prefix=<path>::\n \tPrepend <path> to paths printed in informative messages\ndiff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt\nindex b33e6be..c45efb3 100644\n--- a/Documentation/git-pull.txt\n+++ b/Documentation/git-pull.txt\n@@ -84,7 +84,7 @@ must be given before the options meant for 'git fetch'.\n --verbose::\n \tPass --verbose to git-fetch and git-merge.\n\n---[no-]recurse-submodules::\n+--[no-]recurse-submodules[=yes|on-demand|no]::\n \tThis option controls if new commits of all populated submodules should\n \tbe fetched too (see linkgit:git-config[1] and linkgit:gitmodules[5]).\n \tThat might be necessary to get the data needed for merging submodule\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 55d6b1a..7d24e47 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -38,6 +38,20 @@ static struct transport *transport;\n static const char *submodule_prefix = \"\";\n static const char *submodule_default;\n\n+static int option_parse_recurse_submodules(const struct option *opt,\n+\t\t\t\t   const char *arg, int unset)\n+{\n+\tif (unset) {\n+\t\trecurse_submodules = RECURSE_SUBMODULES_OFF;\n+\t} else {\n+\t\tif (arg)\n+\t\t\trecurse_submodules = parse_fetch_recurse_submodules_arg(opt->long_name, arg);\n+\t\telse\n+\t\t\trecurse_submodules = RECURSE_SUBMODULES_ON;\n+\t}\n+\treturn 0;\n+}\n+\n static struct option builtin_fetch_options[] = {\n \tOPT__VERBOSITY(&verbosity),\n \tOPT_BOOLEAN(0, \"all\", &all,\n@@ -55,9 +69,9 @@ static struct option builtin_fetch_options[] = {\n \t\t    \"do not fetch all tags (--no-tags)\", TAGS_UNSET),\n \tOPT_BOOLEAN('p', \"prune\", &prune,\n \t\t    \"prune remote-tracking branches no longer on remote\"),\n-\tOPT_SET_INT(0, \"recurse-submodules\", &recurse_submodules,\n+\t{ OPTION_CALLBACK, 0, \"recurse-submodules\", NULL, \"on-demand\",\n \t\t    \"control recursive fetching of submodules\",\n-\t\t    RECURSE_SUBMODULES_ON),\n+\t\t    PARSE_OPT_OPTARG, option_parse_recurse_submodules },\n \tOPT_BOOLEAN(0, \"dry-run\", &dry_run,\n \t\t    \"dry run\"),\n \tOPT_BOOLEAN('k', \"keep\", &keep, \"keep downloaded pack\"),\n@@ -816,6 +830,8 @@ static void add_options_to_argv(int *argc, const char **argv)\n \t\targv[(*argc)++] = \"--keep\";\n \tif (recurse_submodules == RECURSE_SUBMODULES_ON)\n \t\targv[(*argc)++] = \"--recurse-submodules\";\n+\telse if (recurse_submodules == RECURSE_SUBMODULES_ON_DEMAND)\n+\t\targv[(*argc)++] = \"--recurse-submodules=on-demand\";\n \tif (verbosity >= 2)\n \t\targv[(*argc)++] = \"-v\";\n \tif (verbosity >= 1)\n@@ -964,7 +980,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \t\tadd_options_to_argv(&num_options, options);\n \t\tresult = fetch_populated_submodules(num_options, options,\n \t\t\t\t\t\t    submodule_prefix,\n-\t\t\t\t\t\t    recurse_submodules == RECURSE_SUBMODULES_ON,\n+\t\t\t\t\t\t    recurse_submodules,\n \t\t\t\t\t\t    verbosity < 0);\n \t}\n\ndiff --git a/git-pull.sh b/git-pull.sh\nindex 86517e9..7a97d8e 100755\n--- a/git-pull.sh\n+++ b/git-pull.sh\n@@ -108,6 +108,9 @@ do\n \t--recurse-submodules)\n \t\trecurse_submodules=--recurse-submodules\n \t\t;;\n+\t--recurse-submodules=*)\n+\t\trecurse_submodules=\"$1\"\n+\t\t;;\n \t--no-recurse-submodules)\n \t\trecurse_submodules=--no-recurse-submodules\n \t\t;;\ndiff --git a/submodule.c b/submodule.c\nindex 8f991ce..86a99af 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -330,7 +330,7 @@ void check_for_new_submodule_commits(unsigned char new_sha1[20])\n }\n\n int fetch_populated_submodules(int num_options, const char **options,\n-\t\t\t       const char *prefix, int ignore_config,\n+\t\t\t       const char *prefix, int command_line_option,\n \t\t\t       int quiet)\n {\n \tint i, result = 0, argc = 0, default_argc;\n@@ -376,7 +376,7 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\tname = name_for_path->util;\n\n \t\tdefault_argv = \"yes\";\n-\t\tif (!ignore_config) {\n+\t\tif (command_line_option == RECURSE_SUBMODULES_DEFAULT) {\n \t\t\tstruct string_list_item *fetch_recurse_submodules_option;\n \t\t\tfetch_recurse_submodules_option = unsorted_string_list_lookup(&config_fetch_recurse_submodules_for_name, name);\n \t\t\tif (fetch_recurse_submodules_option) {\n@@ -391,6 +391,10 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\t\t\tdefault_argv = \"on-demand\";\n \t\t\t\t}\n \t\t\t}\n+\t\t} else if (command_line_option == RECURSE_SUBMODULES_ON_DEMAND) {\n+\t\t\tif (!unsorted_string_list_lookup(&changed_submodule_paths, ce->name))\n+\t\t\t\tcontinue;\n+\t\t\tdefault_argv = \"on-demand\";\n \t\t}\n\n \t\tstrbuf_addf(&submodule_path, \"%s/%s\", work_tree, ce->name);\ndiff --git a/submodule.h b/submodule.h\nindex 3434a8e..5350b0d 100644\n--- a/submodule.h\n+++ b/submodule.h\n@@ -24,7 +24,7 @@ void show_submodule_summary(FILE *f, const char *path,\n void set_config_fetch_recurse_submodules(int value);\n void check_for_new_submodule_commits(unsigned char new_sha1[20]);\n int fetch_populated_submodules(int num_options, const char **options,\n-\t\t\t       const char *prefix, int ignore_config,\n+\t\t\t       const char *prefix, int command_line_option,\n \t\t\t       int quiet);\n unsigned is_submodule_modified(const char *path, int ignore_untracked);\n int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 6d92f7a..4cd723c 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -301,4 +301,75 @@ test_expect_success \"Recursion picks up all submodules when necessary\" '\n \ttest_cmp expect.out actual.out\n '\n\n+test_expect_success \"'--recurse-submodules=on-demand' doesn't recurse when no new commits are fetched in the superproject (and ignores config)\" '\n+\tadd_upstream_commit &&\n+\t(\n+\t\tcd submodule &&\n+\t\t(\n+\t\t\tcd deepsubmodule &&\n+\t\t\tgit fetch &&\n+\t\t\tgit checkout -q FETCH_HEAD\n+\t\t) &&\n+\t\thead1=$(git rev-parse --short HEAD^) &&\n+\t\tgit add deepsubmodule &&\n+\t\tgit commit -m \"new deepsubmodule\"\n+\t\thead2=$(git rev-parse --short HEAD) &&\n+\t\techo \"From $pwd/submodule\" > ../expect.err.sub &&\n+\t\techo \"   $head1..$head2  master     -> origin/master\" >> ../expect.err.sub\n+\t) &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config fetch.recurseSubmodules true &&\n+\t\tgit fetch --recurse-submodules=on-demand >../actual.out 2>../actual.err &&\n+\t\tgit config --unset fetch.recurseSubmodules\n+\t) &&\n+\t! test -s actual.out &&\n+\t! test -s actual.err\n+'\n+\n+test_expect_success \"'--recurse-submodules=on-demand' recurses as deep as necessary (and ignores config)\" '\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"new submodule\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\ttail -2 expect.err > expect.err.deepsub &&\n+\techo \"From $pwd/.\" > expect.err &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err\n+\tcat expect.err.sub >> expect.err &&\n+\tcat expect.err.deepsub >> expect.err &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config fetch.recurseSubmodules false &&\n+\t\t(\n+\t\t\tcd submodule &&\n+\t\t\tgit config -f .gitmodules submodule.deepsubmodule.fetchRecursive false\n+\t\t) &&\n+\t\tgit fetch --recurse-submodules=on-demand >../actual.out 2>../actual.err &&\n+\t\tgit config --unset fetch.recurseSubmodules\n+\t\t(\n+\t\t\tcd submodule &&\n+\t\t\tgit config --unset -f .gitmodules submodule.deepsubmodule.fetchRecursive\n+\t\t)\n+\t) &&\n+\ttest_cmp expect.out actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n+test_expect_success \"'--recurse-submodules=on-demand' stops when no new submodule commits are found in the superproject (and ignores config)\" '\n+\tadd_upstream_commit &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\techo a >> file &&\n+\tgit add file &&\n+\tgit commit -m \"new file\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err.file &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.file &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules=on-demand >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\ttest_cmp expect.err.file actual.err\n+'\n+\n test_done\n-- \n1.7.4.1.291.g6060\n"},{"id":"162604","messageId":"4D6D7B00.1020807@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v3 3/7] config: teach the fetch.recurseSubmodules option the 'on-demand' value","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:02:24Z","receivedAt":"2011-03-01T23:02:24Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"To enable the user to change the default behavior of \"git fetch\" and \"git\npull\" regarding submodule recursion add the new \"on-demand\" value which\nhas just been added to the \"--recurse-submodules\" command line option.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n Documentation/config.txt    |   10 +++++++---\n submodule.c                 |    2 +-\n t/t5526-fetch-submodules.sh |   28 ++++++++++++++++++++++++++++\n 3 files changed, 36 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 96ce272..7b06fe5 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -900,9 +900,13 @@ diff.wordRegex::\n \tcharacters are *ignorable* whitespace.\n\n fetch.recurseSubmodules::\n-\tA boolean value which changes the behavior for fetch and pull, the\n-\tdefault is to not recursively fetch populated submodules unless\n-\tconfigured otherwise.\n+\tThis option can be either set to a boolean value or to 'on-demand'.\n+\tSetting it to a boolean changes the behavior of fetch and pull to\n+\tunconditionally recurse into submodules when set to true or to not\n+\trecurse at all when set to false. When set to 'on-demand' (the default\n+\tvalue), it tells fetch and pull to recurse only into those submodules\n+\twhere new commits are recorded in the commmits fetched for the\n+\tsuperproject.\n\n fetch.unpackLimit::\n \tIf the number of objects fetched over the git native\ndiff --git a/submodule.c b/submodule.c\nindex 86a99af..a1753f0 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -71,7 +71,7 @@ int submodule_config(const char *var, const char *value, void *cb)\n \tif (!prefixcmp(var, \"submodule.\"))\n \t\treturn parse_submodule_config_option(var, value);\n \telse if (!strcmp(var, \"fetch.recursesubmodules\")) {\n-\t\tconfig_fetch_recurse_submodules = git_config_bool(var, value);\n+\t\tconfig_fetch_recurse_submodules = parse_fetch_recurse_submodules_arg(var, value);\n \t\treturn 0;\n \t}\n \treturn 0;\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 4cd723c..e6d873a 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -372,4 +372,32 @@ test_expect_success \"'--recurse-submodules=on-demand' stops when no new submodul\n \ttest_cmp expect.err.file actual.err\n '\n\n+test_expect_success \"'fetch.recurseSubmodules=on-demand' overrides global config\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules\n+\t) &&\n+\tadd_upstream_commit &&\n+\tgit config --global fetch.recurseSubmodules false &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"new submodule\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err.2 &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.2\n+\thead -2 expect.err >> expect.err.2 &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config fetch.recurseSubmodules on-demand &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\tgit config --global --unset fetch.recurseSubmodules &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config --unset fetch.recurseSubmodules\n+\t) &&\n+\ttest_cmp expect.out.sub actual.out &&\n+\ttest_cmp expect.err.2 actual.err\n+'\n+\n test_done\n-- \n1.7.4.1.291.g6060\n"},{"id":"162605","messageId":"4D6D7B2D.4060408@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v2 4/7] Submodules: Add 'on-demand' value for the 'fetchRecurseSubmodule' option","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:03:09Z","receivedAt":"2011-03-01T23:03:09Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Now the behavior of fetch and pull can be configured to the recently added\n'on-demand' mode separately for each submodule too.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n Documentation/config.txt     |    2 +-\n Documentation/gitmodules.txt |    4 ++--\n submodule.c                  |    9 +++++++--\n t/t5526-fetch-submodules.sh  |   28 ++++++++++++++++++++++++++++\n 4 files changed, 38 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 7b06fe5..d47a95d 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1824,7 +1824,7 @@ submodule.<name>.update::\n \tlinkgit:git-submodule[1] and linkgit:gitmodules[5] for details.\n\n submodule.<name>.fetchRecurseSubmodules::\n-\tThis option can be used to enable/disable recursive fetching of this\n+\tThis option can be used to control recursive fetching of this\n \tsubmodule. It can be overridden by using the --[no-]recurse-submodules\n \tcommand line option to \"git fetch\" and \"git pull\".\n \tThis setting will override that from in the linkgit:gitmodules[5]\ndiff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt\nindex 6897794..25daee2 100644\n--- a/Documentation/gitmodules.txt\n+++ b/Documentation/gitmodules.txt\n@@ -45,12 +45,12 @@ submodule.<name>.update::\n \tthe '--merge' or '--rebase' options.\n\n submodule.<name>.fetchRecurseSubmodules::\n-\tThis option can be used to enable/disable recursive fetching of this\n+\tThis option can be used to control recursive fetching of this\n \tsubmodule. If this option is also present in the submodules entry in\n \t.git/config of the superproject, the setting there will override the\n \tone found in .gitmodules.\n \tBoth settings can be overridden on the command line by using the\n-\t\"--[no-]recurse-submodules\" option to \"git fetch\" and \"git pull\"..\n+\t\"--[no-]recurse-submodules\" option to \"git fetch\" and \"git pull\".\n\n submodule.<name>.ignore::\n \tDefines under what circumstances \"git status\" and the diff family show\ndiff --git a/submodule.c b/submodule.c\nindex a1753f0..3be6654 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -113,7 +113,7 @@ int parse_submodule_config_option(const char *var, const char *value)\n \t\tif (!config)\n \t\t\tconfig = string_list_append(&config_fetch_recurse_submodules_for_name,\n \t\t\t\t\t\t    strbuf_detach(&submodname, NULL));\n-\t\tconfig->util = git_config_bool(var, value) ? (void *)1 : NULL;\n+\t\tconfig->util = (void *)(intptr_t)parse_fetch_recurse_submodules_arg(var, value);\n \t\tstrbuf_release(&submodname);\n \t} else if ((len > 7) && !strcmp(var + len - 7, \".ignore\")) {\n \t\tif (strcmp(value, \"untracked\") && strcmp(value, \"dirty\") &&\n@@ -380,8 +380,13 @@ int fetch_populated_submodules(int num_options, const char **options,\n \t\t\tstruct string_list_item *fetch_recurse_submodules_option;\n \t\t\tfetch_recurse_submodules_option = unsorted_string_list_lookup(&config_fetch_recurse_submodules_for_name, name);\n \t\t\tif (fetch_recurse_submodules_option) {\n-\t\t\t\tif (!fetch_recurse_submodules_option->util)\n+\t\t\t\tif ((intptr_t)fetch_recurse_submodules_option->util == RECURSE_SUBMODULES_OFF)\n \t\t\t\t\tcontinue;\n+\t\t\t\tif ((intptr_t)fetch_recurse_submodules_option->util == RECURSE_SUBMODULES_ON_DEMAND) {\n+\t\t\t\t\tif (!unsorted_string_list_lookup(&changed_submodule_paths, ce->name))\n+\t\t\t\t\t\tcontinue;\n+\t\t\t\t\tdefault_argv = \"on-demand\";\n+\t\t\t\t}\n \t\t\t} else {\n \t\t\t\tif (config_fetch_recurse_submodules == RECURSE_SUBMODULES_OFF)\n \t\t\t\t\tcontinue;\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex e6d873a..09701aa 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -400,4 +400,32 @@ test_expect_success \"'fetch.recurseSubmodules=on-demand' overrides global config\n \ttest_cmp expect.err.2 actual.err\n '\n\n+test_expect_success \"'submodule.<sub>.fetchRecurseSubmodules=on-demand' overrides fetch.recurseSubmodules\" '\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch --recurse-submodules\n+\t) &&\n+\tadd_upstream_commit &&\n+\tgit config fetch.recurseSubmodules false &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"new submodule\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err.2 &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err.2\n+\thead -2 expect.err >> expect.err.2 &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config submodule.submodule.fetchRecurseSubmodules on-demand &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\tgit config --unset fetch.recurseSubmodules &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit config --unset submodule.submodule.fetchRecurseSubmodules\n+\t) &&\n+\ttest_cmp expect.out.sub actual.out &&\n+\ttest_cmp expect.err.2 actual.err\n+'\n+\n test_done\n-- \n1.7.4.1.291.g6060\n"},{"id":"162606","messageId":"4D6D7B49.3030005@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v2 5/7] fetch/pull: Don't recurse into a submodule when commits are already present","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:03:37Z","receivedAt":"2011-03-01T23:03:37Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"When looking for submodules where new commits have been recorded in the\nsuperproject ignore those cases where the submodules commits are already\npresent locally. This can happen e.g. when the submodule has been rewound\nto an earlier state. Then there is no need to fetch the submodule again\nas the commit recorded in the newly fetched superproject commit has\nalready been fetched earlier into the submodule.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n submodule.c                 |   29 ++++++++++++++++++++++++++++-\n t/t5526-fetch-submodules.sh |   19 +++++++++++++++++++\n 2 files changed, 47 insertions(+), 1 deletions(-)\n\ndiff --git a/submodule.c b/submodule.c\nindex 3be6654..11e0ef5 100644\n--- a/submodule.c\n+++ b/submodule.c\n@@ -263,6 +263,33 @@ void set_config_fetch_recurse_submodules(int value)\n \tconfig_fetch_recurse_submodules = value;\n }\n\n+static int is_submodule_commit_present(const char *path, unsigned char sha1[20])\n+{\n+\tint is_present = 0;\n+\tif (!add_submodule_odb(path) && lookup_commit_reference(sha1)) {\n+\t\t/* Even if the submodule is checked out and the commit is\n+\t\t * present, make sure it is reachable from a ref. */\n+\t\tstruct child_process cp;\n+\t\tconst char *argv[] = {\"rev-list\", \"-n\", \"1\", NULL, \"--not\", \"--all\", NULL};\n+\t\tstruct strbuf buf = STRBUF_INIT;\n+\n+\t\targv[3] = sha1_to_hex(sha1);\n+\t\tmemset(&cp, 0, sizeof(cp));\n+\t\tcp.argv = argv;\n+\t\tcp.env = local_repo_env;\n+\t\tcp.git_cmd = 1;\n+\t\tcp.no_stdin = 1;\n+\t\tcp.out = -1;\n+\t\tcp.dir = path;\n+\t\tif (!run_command(&cp) && !strbuf_read(&buf, cp.out, 1024))\n+\t\t\tis_present = 1;\n+\n+\t\tclose(cp.out);\n+\t\tstrbuf_release(&buf);\n+\t}\n+\treturn is_present;\n+}\n+\n static void submodule_collect_changed_cb(struct diff_queue_struct *q,\n \t\t\t\t\t struct diff_options *options,\n \t\t\t\t\t void *data)\n@@ -280,7 +307,7 @@ static void submodule_collect_changed_cb(struct diff_queue_struct *q,\n \t\t\t * being moved around. */\n \t\t\tstruct string_list_item *path;\n \t\t\tpath = unsorted_string_list_lookup(&changed_submodule_paths, p->two->path);\n-\t\t\tif (!path)\n+\t\t\tif (!path && !is_submodule_commit_present(p->two->path, p->two->sha1))\n \t\t\t\tstring_list_append(&changed_submodule_paths, xstrdup(p->two->path));\n \t\t} else {\n \t\t\t/* Submodule is new or was moved here */\ndiff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\nindex 09701aa..3decfae 100755\n--- a/t/t5526-fetch-submodules.sh\n+++ b/t/t5526-fetch-submodules.sh\n@@ -428,4 +428,23 @@ test_expect_success \"'submodule.<sub>.fetchRecurseSubmodules=on-demand' override\n \ttest_cmp expect.err.2 actual.err\n '\n\n+test_expect_success \"don't fetch submodule when newly recorded commits are already present\" '\n+\t(\n+\t\tcd submodule &&\n+\t\tgit checkout -q HEAD^^\n+\t) &&\n+\thead1=$(git rev-parse --short HEAD) &&\n+\tgit add submodule &&\n+\tgit commit -m \"submodule rewound\" &&\n+\thead2=$(git rev-parse --short HEAD) &&\n+\techo \"From $pwd/.\" > expect.err &&\n+\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err &&\n+\t(\n+\t\tcd downstream &&\n+\t\tgit fetch >../actual.out 2>../actual.err\n+\t) &&\n+\t! test -s actual.out &&\n+\ttest_cmp expect.err actual.err\n+'\n+\n test_done\n-- \n1.7.4.1.291.g6060\n"},{"id":"162607","messageId":"4D6D7B75.9040300@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v2 6/7] submodule update: Don't fetch when the submodule commit is already present","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:04:21Z","receivedAt":"2011-03-01T23:04:21Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"If the commit to be checked out on \"git submodule update\" has already been\nfetched in the submodule there is no need to run \"git fetch\" again. Since\n\"git fetch\" recently learned recursion (and the new on-demand mode to\nfetch commits recorded in the superproject is enabled by default) this\nwill happen pretty often, thereby making the unconditional fetch during\n\"git submodule update\" unnecessary.\n\nIf the commit is not present in the submodule (e.g. the user disabled the\nfetch on-demand mode) the fetch will be run as before.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n git-submodule.sh            |    5 ++++-\n t/t7406-submodule-update.sh |   20 ++++++++++++++++++++\n 2 files changed, 24 insertions(+), 1 deletions(-)\n\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 3a13397..c16e275 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -466,8 +466,11 @@ cmd_update()\n\n \t\t\tif test -z \"$nofetch\"\n \t\t\tthen\n+\t\t\t\t# Run fetch only if $sha1 isn't present or it\n+\t\t\t\t# is not reachable from a ref.\n \t\t\t\t(clear_local_git_env; cd \"$path\" &&\n-\t\t\t\t\tgit-fetch) ||\n+\t\t\t\t\t((rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&\n+\t\t\t\t\t test -z \"$rev\") || git-fetch)) ||\n \t\t\t\tdie \"Unable to fetch in submodule path '$path'\"\n \t\t\tfi\n\ndiff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh\nindex fa9d23a..bf7c788 100755\n--- a/t/t7406-submodule-update.sh\n+++ b/t/t7406-submodule-update.sh\n@@ -74,6 +74,26 @@ test_expect_success 'submodule update detaching the HEAD ' '\n \t)\n '\n\n+apos=\"'\";\n+test_expect_success 'submodule update does not fetch already present commits' '\n+\t(cd submodule &&\n+\t  echo line3 >> file &&\n+\t  git add file &&\n+\t  test_tick &&\n+\t  git commit -m \"upstream line3\"\n+\t) &&\n+\t(cd super/submodule &&\n+\t  head=$(git rev-parse --verify HEAD) &&\n+\t  echo \"Submodule path ${apos}submodule$apos: checked out $apos$head$apos\" > ../../expected &&\n+\t  git reset --hard HEAD~1\n+\t) &&\n+\t(cd super &&\n+\t  git submodule update > ../actual 2> ../actual.err\n+\t) &&\n+\ttest_cmp expected actual &&\n+\t! test -s actual.err\n+'\n+\n test_expect_success 'submodule update --rebase staying on master' '\n \t(cd super/submodule &&\n \t  git checkout master\n-- \n1.7.4.1.291.g6060\n"},{"id":"162608","messageId":"4D6D7B8D.3020409@web.de","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"[PATCH v2 7/7] fetch/pull: Describe --recurse-submodule restrictions in the BUGS section","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-01T23:04:45Z","receivedAt":"2011-03-01T23:04:45Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Using the --recurse-submodules option with fetch and pull might not always\nfetch all the submodule commits the user expects, as this will only work\nwhen the submodule is already checked out. Document that and warn that\nthis is expected to change in the future.\n\nSigned-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n---\n Documentation/git-fetch.txt |    9 +++++++++\n Documentation/git-pull.txt  |    9 +++++++++\n 2 files changed, 18 insertions(+), 0 deletions(-)\n\ndiff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt\nindex c76e313..f907ef5 100644\n--- a/Documentation/git-fetch.txt\n+++ b/Documentation/git-fetch.txt\n@@ -76,6 +76,15 @@ The `pu` branch will be updated even if it is does not fast-forward,\n because it is prefixed with a plus sign; `tmp` will not be.\n\n\n+BUGS\n+----\n+Using --recurse-submodules can only fetch new commits in alrady checked\n+out submodules right now. When e.g. upstream added a new submodule in the\n+just fetched commits of the superproject the submodule itself can not be\n+fetched, making it impossible to check out that submodule later whithout\n+having to do a fetch again. This is expected to be fixed in a future git\n+version.\n+\n SEE ALSO\n --------\n linkgit:git-pull[1]\ndiff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt\nindex c45efb3..1596d2b 100644\n--- a/Documentation/git-pull.txt\n+++ b/Documentation/git-pull.txt\n@@ -220,6 +220,15 @@ If you tried a pull which resulted in a complex conflicts and\n would want to start over, you can recover with 'git reset'.\n\n\n+BUGS\n+----\n+Using --recurse-submodules can only fetch new commits in alrady checked\n+out submodules right now. When e.g. upstream added a new submodule in the\n+just fetched commits of the superproject the submodule itself can not be\n+fetched, making it impossible to check out that submodule later whithout\n+having to do a fetch again. This is expected to be fixed in a future git\n+version.\n+\n SEE ALSO\n --------\n linkgit:git-fetch[1], linkgit:git-merge[1], linkgit:git-config[1]\n-- \n1.7.4.1.291.g6060\n"},{"id":"162641","messageId":"4D6E6583.8090202@xiplink.com","threadId":"26635","inReplyTo":"4D6D7AB2.4010509@web.de","subject":"Re: [PATCH v2 1/7] fetch/pull: recurse into submodules when necessary","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-03-02T15:42:59Z","receivedAt":"2011-03-02T15:42:59Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-03-01 06:01 PM, Jens Lehmann wrote:\n> To be able to access all commits of populated submodules referenced by the\n> superproject it is sufficient to only then let \"git fetch\" recurse into a\n> submodule when the new commits fetched in the superproject record new\n> commits for it. Having these commits present is extremely useful when\n> using the \"--submodule\" option to \"git diff\" (which is what \"git gui\" and\n> \"gitk\" do since 1.6.6), as all submodule commits needed for creating a\n> descriptive output can be accessed. Also merging submodule commits (added\n> in 1.7.3) depends on the submodule commits in question being present to\n> work. Last but not least this enables disconnected operation when using\n> submodules, as all commits necessary for a successful \"git submodule\n> update -N\" will have been fetched automatically. So we choose this mode as\n> the default for fetch and pull.\n> \n> Before a new or changed ref from upstream is updated in update_local_ref()\n> \"git rev-list <new-sha1> --not --branches --remotes\" is used to determine\n> all newly fetched commits. These are then walked and diffed against their\n> parent(s) to see if a submodule has been changed. If that is the case, its\n> path is stored to be fetched after the superproject fetch is completed.\n> \n> Using the \"--recurse-submodules\" or the \"--no-recurse-submodules\" option\n> disables the examination of the fetched refs because the result will be\n> ignored anyway.\n> \n> There is currently no infrastructure for storing deleted and new\n> submodules in the .git directory of the superproject. Thats why fetch and\n> pull for now only fetch submodules that are already checked out and are\n> not renamed.\n> \n> In t7403 the \"--no-recurse-submodules\" argument had to be added to \"git\n> pull\" to avoid failure because of the moved upstream submodule repo.\n> \n> Thanks-to: Jonathan Nieder <jrnieder@gmail.com>\n> Thanks-to: Heiko Voigt <hvoigt@hvoigt.net>\n> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n> ---\n>  Documentation/fetch-options.txt |   11 ++++\n>  builtin/fetch.c                 |   25 ++++++---\n>  submodule.c                     |  106 +++++++++++++++++++++++++++++++++++---\n>  submodule.h                     |    9 +++\n>  t/t5526-fetch-submodules.sh     |  109 +++++++++++++++++++++++++++++++++++++++\n>  t/t7403-submodule-sync.sh       |    2 +-\n>  6 files changed, 245 insertions(+), 17 deletions(-)\n> \n> diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\n> index f37276e..ae22f75 100644\n> --- a/Documentation/fetch-options.txt\n> +++ b/Documentation/fetch-options.txt\n> @@ -73,6 +73,17 @@ ifndef::git-pull[]\n>  \tPrepend <path> to paths printed in informative messages\n>  \tsuch as \"Fetching submodule foo\".  This option is used\n>  \tinternally when recursing over submodules.\n> +\n> +--submodule-default=[yes|on-demand]::\n> +\tThis option is used internally to set the submodule recursion default\n> +\tto either a boolean configuration value representing \"true\" (for\n> +\tunconditonal recursion) or to \"on-demand\" (when only those submodules\n> +\tshould be fetched of which new commits have been fetched in its\n> +\tsuperproject).\n> +\tThis option will be overridden by the 'fetch.recurseSubmodules' and\n> +\t'submodule.<name>.fetchRecurseSubmodules' settings in\n> +\tlinkgit:gitmodules[5] and linkgit:git-config[1] while all of them\n> +\tare ignored when the \"--[no-]recurse-submodules\" option is given.\n>  endif::git-pull[]\n\nRolling out the bike...\n\nI think the option name and it's explanation need to be a little clearer:\n\n--recurse-submodules-default=[yes|on-demand]::\n\tThis option is used internally to temporarily provide a\n\tnon-negative default value for the --recurse-submodules\n\toption.  All other methods of configuring fetch's submodule\n\trecursion (such as settings in linkgit:gitmodules[5] and\n\tlinkgit:git-config[1]) override this option, as does\n\tspecifying --[no-]recurse-submodules directly.\n\n\n\t\tM.\n"},{"id":"162643","messageId":"4D6E699D.3040000@xiplink.com","threadId":"26635","inReplyTo":"4D6D7ADA.6090302@web.de","subject":"Re: [PATCH v2 2/7] fetch/pull: Add the 'on-demand' value to the --recurse-submodules option","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-03-02T16:00:29Z","receivedAt":"2011-03-02T16:00:29Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-03-01 06:01 PM, Jens Lehmann wrote:\n> Until now the --recurse-submodules option could only be used to either\n> fetch all populated submodules recursively or to disable recursion\n> completely. As fetch and pull now by default just fetch those submodules\n> for which new commits have been fetched in the superproject, a command\n> line option to enforce that behavior is needed to be able to override\n> configuration settings.\n\nStrictly speaking, this patch should add the 'on-demand' value to the first\npatch's --submodule-default option.  Perhaps I'm being a bit too uptight\nthough...\n\n> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n> ---\n>  Documentation/fetch-options.txt |   16 +++++++--\n>  Documentation/git-pull.txt      |    2 +-\n>  builtin/fetch.c                 |   22 ++++++++++--\n>  git-pull.sh                     |    3 ++\n>  submodule.c                     |    8 +++-\n>  submodule.h                     |    2 +-\n>  t/t5526-fetch-submodules.sh     |   71 +++++++++++++++++++++++++++++++++++++++\n>  7 files changed, 114 insertions(+), 10 deletions(-)\n> \n> diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\n> index ae22f75..f28c0b1 100644\n> --- a/Documentation/fetch-options.txt\n> +++ b/Documentation/fetch-options.txt\n> @@ -65,9 +65,19 @@ ifndef::git-pull[]\n>  \tspecified with the remote.<name>.tagopt setting. See\n>  \tlinkgit:git-config[1].\n> \n> ---[no-]recurse-submodules::\n> -\tThis option controls if new commits of all populated submodules should\n> -\tbe fetched too (see linkgit:git-config[1] and linkgit:gitmodules[5]).\n> +--recurse-submodules[=yes|on-demand|no]::\n> +\tThis option controls if and under what conditions new commits of all\n\nNit: Remove \"all\" from the above line.\n\n> +\tpopulated submodules should be fetched too. It can be used as a\n> +\tboolean option to completely disable recursion when set to 'no' or to\n> +\tunconditionally recurse into all populated submodules when set to\n> +\t'yes', which is the default when this option is used without any\n> +\tvalue. If 'on-demand' is used, it will only recurse into those\n> +\tsubmodules where new commits have been fetched in the superproject\n> +\t(also see linkgit:git-config[1] and linkgit:gitmodules[5]).\n\nNit: Replace that last sentence with\n\n\tUse 'on-demand' to only recurse into a submodule when the\n\tsuperproject retrieves a commit that updates the submodule's\n\treference.\n\nAlso, this has to change again with patch 5 (Don't recurse into a submodule\nwhen commits are already present).\n\n\t\tM.\n"},{"id":"162644","messageId":"4D6E6A03.2090902@xiplink.com","threadId":"26635","inReplyTo":"4D6D7B00.1020807@web.de","subject":"Re: [PATCH v3 3/7] config: teach the fetch.recurseSubmodules option the 'on-demand' value","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-03-02T16:02:11Z","receivedAt":"2011-03-02T16:02:11Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-03-01 06:02 PM, Jens Lehmann wrote:\n> To enable the user to change the default behavior of \"git fetch\" and \"git\n> pull\" regarding submodule recursion add the new \"on-demand\" value which\n> has just been added to the \"--recurse-submodules\" command line option.\n> \n> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n> ---\n>  Documentation/config.txt    |   10 +++++++---\n>  submodule.c                 |    2 +-\n>  t/t5526-fetch-submodules.sh |   28 ++++++++++++++++++++++++++++\n>  3 files changed, 36 insertions(+), 4 deletions(-)\n> \n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 96ce272..7b06fe5 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -900,9 +900,13 @@ diff.wordRegex::\n>  \tcharacters are *ignorable* whitespace.\n> \n>  fetch.recurseSubmodules::\n> -\tA boolean value which changes the behavior for fetch and pull, the\n> -\tdefault is to not recursively fetch populated submodules unless\n> -\tconfigured otherwise.\n> +\tThis option can be either set to a boolean value or to 'on-demand'.\n> +\tSetting it to a boolean changes the behavior of fetch and pull to\n> +\tunconditionally recurse into submodules when set to true or to not\n> +\trecurse at all when set to false. When set to 'on-demand' (the default\n> +\tvalue), it tells fetch and pull to recurse only into those submodules\n> +\twhere new commits are recorded in the commmits fetched for the\n> +\tsuperproject.\n\nNit: Replace the last sentence with\n\n\tWhen set to 'on-demand' (the default value), fetch and pull will\n\tonly recurse into a populated submodule when its superproject\n\tretrieves a commit that updates the submodule's reference.\n\n\t\tM.\n"},{"id":"162645","messageId":"4D6E6A32.7030908@xiplink.com","threadId":"26635","inReplyTo":"4D6E699D.3040000@xiplink.com","subject":"Re: [PATCH v2 2/7] fetch/pull: Add the 'on-demand' value to the --recurse-submodules option","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-03-02T16:02:58Z","receivedAt":"2011-03-02T16:02:58Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-03-02 11:00 AM, Marc Branchaud wrote:\n> On 11-03-01 06:01 PM, Jens Lehmann wrote:\n>> Until now the --recurse-submodules option could only be used to either\n>> fetch all populated submodules recursively or to disable recursion\n>> completely. As fetch and pull now by default just fetch those submodules\n>> for which new commits have been fetched in the superproject, a command\n>> line option to enforce that behavior is needed to be able to override\n>> configuration settings.\n> \n> Strictly speaking, this patch should add the 'on-demand' value to the first\n> patch's --submodule-default option.  Perhaps I'm being a bit too uptight\n> though...\n> \n>> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n>> ---\n>>  Documentation/fetch-options.txt |   16 +++++++--\n>>  Documentation/git-pull.txt      |    2 +-\n>>  builtin/fetch.c                 |   22 ++++++++++--\n>>  git-pull.sh                     |    3 ++\n>>  submodule.c                     |    8 +++-\n>>  submodule.h                     |    2 +-\n>>  t/t5526-fetch-submodules.sh     |   71 +++++++++++++++++++++++++++++++++++++++\n>>  7 files changed, 114 insertions(+), 10 deletions(-)\n>>\n>> diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\n>> index ae22f75..f28c0b1 100644\n>> --- a/Documentation/fetch-options.txt\n>> +++ b/Documentation/fetch-options.txt\n>> @@ -65,9 +65,19 @@ ifndef::git-pull[]\n>>  \tspecified with the remote.<name>.tagopt setting. See\n>>  \tlinkgit:git-config[1].\n>>\n>> ---[no-]recurse-submodules::\n>> -\tThis option controls if new commits of all populated submodules should\n>> -\tbe fetched too (see linkgit:git-config[1] and linkgit:gitmodules[5]).\n>> +--recurse-submodules[=yes|on-demand|no]::\n>> +\tThis option controls if and under what conditions new commits of all\n> \n> Nit: Remove \"all\" from the above line.\n> \n>> +\tpopulated submodules should be fetched too. It can be used as a\n>> +\tboolean option to completely disable recursion when set to 'no' or to\n>> +\tunconditionally recurse into all populated submodules when set to\n>> +\t'yes', which is the default when this option is used without any\n>> +\tvalue. If 'on-demand' is used, it will only recurse into those\n>> +\tsubmodules where new commits have been fetched in the superproject\n>> +\t(also see linkgit:git-config[1] and linkgit:gitmodules[5]).\n> \n> Nit: Replace that last sentence with\n> \n> \tUse 'on-demand' to only recurse into a submodule when the\n> \tsuperproject retrieves a commit that updates the submodule's\n> \treference.\n\nDoh!  That should read:\n\n\tUse 'on-demand' to only recurse into a populated submodule when the\n\tsuperproject retrieves a commit that updates the submodule's\n\treference.\n\n\t\tM.\n"},{"id":"162646","messageId":"4D6E6B71.2030800@xiplink.com","threadId":"26635","inReplyTo":"4D6D7B49.3030005@web.de","subject":"Re: [PATCH v2 5/7] fetch/pull: Don't recurse into a submodule when commits are already present","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-03-02T16:08:17Z","receivedAt":"2011-03-02T16:08:17Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"As I said earlier, this patch should also tweak the documentation in\nfetch-options.txt and config.txt.\n\nIn my suggested wording I used the phrase\n\n\t... only recurse into a populated submodule when its superproject\n\tretrieves a commit that updates the submodule's reference.\n\nWith this patch, that should now be\n\n\t... only recurse into a populated submodule when its superproject\n\tretrieves a commit that updates the submodule's reference to a\n\tcommit that isn't already in the local submodule clone.\n\n\t\tM.\n\n\nOn 11-03-01 06:03 PM, Jens Lehmann wrote:\n> When looking for submodules where new commits have been recorded in the\n> superproject ignore those cases where the submodules commits are already\n> present locally. This can happen e.g. when the submodule has been rewound\n> to an earlier state. Then there is no need to fetch the submodule again\n> as the commit recorded in the newly fetched superproject commit has\n> already been fetched earlier into the submodule.\n> \n> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>\n> ---\n>  submodule.c                 |   29 ++++++++++++++++++++++++++++-\n>  t/t5526-fetch-submodules.sh |   19 +++++++++++++++++++\n>  2 files changed, 47 insertions(+), 1 deletions(-)\n> \n> diff --git a/submodule.c b/submodule.c\n> index 3be6654..11e0ef5 100644\n> --- a/submodule.c\n> +++ b/submodule.c\n> @@ -263,6 +263,33 @@ void set_config_fetch_recurse_submodules(int value)\n>  \tconfig_fetch_recurse_submodules = value;\n>  }\n> \n> +static int is_submodule_commit_present(const char *path, unsigned char sha1[20])\n> +{\n> +\tint is_present = 0;\n> +\tif (!add_submodule_odb(path) && lookup_commit_reference(sha1)) {\n> +\t\t/* Even if the submodule is checked out and the commit is\n> +\t\t * present, make sure it is reachable from a ref. */\n> +\t\tstruct child_process cp;\n> +\t\tconst char *argv[] = {\"rev-list\", \"-n\", \"1\", NULL, \"--not\", \"--all\", NULL};\n> +\t\tstruct strbuf buf = STRBUF_INIT;\n> +\n> +\t\targv[3] = sha1_to_hex(sha1);\n> +\t\tmemset(&cp, 0, sizeof(cp));\n> +\t\tcp.argv = argv;\n> +\t\tcp.env = local_repo_env;\n> +\t\tcp.git_cmd = 1;\n> +\t\tcp.no_stdin = 1;\n> +\t\tcp.out = -1;\n> +\t\tcp.dir = path;\n> +\t\tif (!run_command(&cp) && !strbuf_read(&buf, cp.out, 1024))\n> +\t\t\tis_present = 1;\n> +\n> +\t\tclose(cp.out);\n> +\t\tstrbuf_release(&buf);\n> +\t}\n> +\treturn is_present;\n> +}\n> +\n>  static void submodule_collect_changed_cb(struct diff_queue_struct *q,\n>  \t\t\t\t\t struct diff_options *options,\n>  \t\t\t\t\t void *data)\n> @@ -280,7 +307,7 @@ static void submodule_collect_changed_cb(struct diff_queue_struct *q,\n>  \t\t\t * being moved around. */\n>  \t\t\tstruct string_list_item *path;\n>  \t\t\tpath = unsorted_string_list_lookup(&changed_submodule_paths, p->two->path);\n> -\t\t\tif (!path)\n> +\t\t\tif (!path && !is_submodule_commit_present(p->two->path, p->two->sha1))\n>  \t\t\t\tstring_list_append(&changed_submodule_paths, xstrdup(p->two->path));\n>  \t\t} else {\n>  \t\t\t/* Submodule is new or was moved here */\n> diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh\n> index 09701aa..3decfae 100755\n> --- a/t/t5526-fetch-submodules.sh\n> +++ b/t/t5526-fetch-submodules.sh\n> @@ -428,4 +428,23 @@ test_expect_success \"'submodule.<sub>.fetchRecurseSubmodules=on-demand' override\n>  \ttest_cmp expect.err.2 actual.err\n>  '\n> \n> +test_expect_success \"don't fetch submodule when newly recorded commits are already present\" '\n> +\t(\n> +\t\tcd submodule &&\n> +\t\tgit checkout -q HEAD^^\n> +\t) &&\n> +\thead1=$(git rev-parse --short HEAD) &&\n> +\tgit add submodule &&\n> +\tgit commit -m \"submodule rewound\" &&\n> +\thead2=$(git rev-parse --short HEAD) &&\n> +\techo \"From $pwd/.\" > expect.err &&\n> +\techo \"   $head1..$head2  master     -> origin/master\" >> expect.err &&\n> +\t(\n> +\t\tcd downstream &&\n> +\t\tgit fetch >../actual.out 2>../actual.err\n> +\t) &&\n> +\t! test -s actual.out &&\n> +\ttest_cmp expect.err actual.err\n> +'\n> +\n>  test_done\n"},{"id":"162647","messageId":"4D6E6BB9.9000904@xiplink.com","threadId":"26635","inReplyTo":"4D6D7A50.5090802@web.de","subject":"Re: [PATCH v2 0/7] Teach fetch/pull the on-demand mode and make it the default","fromName":"Marc Branchaud","fromEmail":"marcnarc@xiplink.com","sentAt":"2011-03-02T16:09:29Z","receivedAt":"2011-03-02T16:09:29Z","isPatch":true,"sender":{"key":"marcnarc@xiplink.com","avatar":"https://avatars.githubusercontent.com/u/14980203?v=4"},"body":"On 11-03-01 05:59 PM, Jens Lehmann wrote:\n> So here is version 2 of the on-demand recursive fetch for submodules.\n> \n> Changes since the last version:\n> \n> *) Add a name parameter to parse_fetch_recurse_submodules_arg()\n> *) Enhance description of the \"--submodule-default\" fetch option\n> *) Free the changed_submodule_paths string_list at the end of\n>    fetch_populated_submodules() to aid libification and to make\n>    valgrind happy\n> *) Add a comment for the while loop in check_for_new_submodule_commits()\n> *) Use \"--all\" instead of \"--branches\" and \"--remotes\" for rev-list to\n>    include tags too\n> *) Compute the argc value properly instead of hard coding it in\n>    check_for_new_submodule_commits()\n> *) Change size_t casts to intptr_t where appropriate\n> *) Use \"rev-list --not --all\" when checking if a submodule commit exists\n>    instead of a simple lookup/rev-parse to make sure they are not only\n>    present but are reachable from a ref\n> *) Describe that fetch and pull only work on populated submodules for\n>    now in the BUGS section of their man pages (done in a separate commit\n>    because this doesn't belong to a special commit in this series but\n>    also to the functionality that has already been merged)\n> \n> Thanks to Jonathan an Junio for the feedback that lead to these changes.\n> \n> Issues that were raised but not addressed yet:\n> \n> *) Should this option be called \"on-demand\" or \"changed\" or maybe even\n>    \"updated\"?\n>    I have no strong feelings about this, me just thinks the name should\n>    tell somehow that only submodules where new commits are recorded in\n>    the superproject which aren't already present locally will be fetched.\n>    Opinions?\n\nFirst, thanks again for this work!\n\nI have a slight preference for \"updated\" or \"updated-only\", but no really\nstrong feelings either way.\n\n\t\tM.\n"},{"id":"162668","messageId":"4D6EAE3F.9070202@web.de","threadId":"26635","inReplyTo":"4D6D7B8D.3020409@web.de","subject":"Re: [PATCH v2 7/7] fetch/pull: Describe --recurse-submodule restrictions in the BUGS section","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-02T20:53:19Z","receivedAt":"2011-03-02T20:53:19Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 02.03.2011 00:04, schrieb Jens Lehmann:\n> Using the --recurse-submodules option with fetch and pull might not always\n> fetch all the submodule commits the user expects, as this will only work\n> when the submodule is already checked out. Document that and warn that\n> this is expected to change in the future.\n\nThis patch needs the following diff to get rid of some typos. Will be\nfixed in v3.\n\n--------8<--------\ndiff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt\nindex f907ef5..8669227 100644\n--- a/Documentation/git-fetch.txt\n+++ b/Documentation/git-fetch.txt\n@@ -78,10 +78,10 @@ because it is prefixed with a plus sign; `tmp` will not be.\n\n BUGS\n ----\n-Using --recurse-submodules can only fetch new commits in alrady checked\n+Using --recurse-submodules can only fetch new commits in already checked\n out submodules right now. When e.g. upstream added a new submodule in the\n just fetched commits of the superproject the submodule itself can not be\n-fetched, making it impossible to check out that submodule later whithout\n+fetched, making it impossible to check out that submodule later without\n having to do a fetch again. This is expected to be fixed in a future git\n version.\n\ndiff --git a/Documentation/git-pull.txt b/Documentation/git-pull.txt\nindex 1596d2b..1aad8bf 100644\n--- a/Documentation/git-pull.txt\n+++ b/Documentation/git-pull.txt\n@@ -222,10 +222,10 @@ would want to start over, you can recover with 'git reset'\n\n BUGS\n ----\n-Using --recurse-submodules can only fetch new commits in alrady checked\n+Using --recurse-submodules can only fetch new commits in already checked\n out submodules right now. When e.g. upstream added a new submodule in the\n just fetched commits of the superproject the submodule itself can not be\n-fetched, making it impossible to check out that submodule later whithout\n+fetched, making it impossible to check out that submodule later without\n having to do a fetch again. This is expected to be fixed in a future git\n version.\n"},{"id":"162691","messageId":"4D6ED431.6090300@web.de","threadId":"26635","inReplyTo":"4D6E6BB9.9000904@xiplink.com","subject":"Re: [PATCH v2 0/7] Teach fetch/pull the on-demand mode and make it the default","fromName":"Jens Lehmann","fromEmail":"jens.lehmann@web.de","sentAt":"2011-03-02T23:35:13Z","receivedAt":"2011-03-02T23:35:13Z","isPatch":true,"sender":{"key":"jens.lehmann@web.de","avatar":"https://avatars.githubusercontent.com/u/135220?v=4"},"body":"Am 02.03.2011 17:09, schrieb Marc Branchaud:\n> On 11-03-01 05:59 PM, Jens Lehmann wrote:\n>> *) Should this option be called \"on-demand\" or \"changed\" or maybe even\n>>    \"updated\"?\n>>    I have no strong feelings about this, me just thinks the name should\n>>    tell somehow that only submodules where new commits are recorded in\n>>    the superproject which aren't already present locally will be fetched.\n>>    Opinions?\n> \n> First, thanks again for this work!\n\nThanks for your feedback on this series, I'll include that in v3.\n\n> I have a slight preference for \"updated\" or \"updated-only\", but no really\n> strong feelings either way.\n\nI'm looking forward to hear what others think ...\n"}]}