{"thread":{"id":"37562","subject":"[PATCH/RFC 1/2] submodule: add ability to shallowly clone any branch in a repo as a submodule","startedAt":"2014-09-14T10:38:36Z","lastAt":"2014-09-14T23:41:43Z","messageCount":3,"participants":["Cole Minnaar","Eric Sunshine"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"249395","messageId":"ede7b63c6028591281a7eefea5e9cd45cccd0a93.1410691049.git.cole.minnaar@gmail.com","threadId":"37562","inReplyTo":null,"subject":"[PATCH/RFC 1/2] submodule: add ability to shallowly clone any branch in a repo as a submodule","fromName":"Cole Minnaar","fromEmail":"cole.minnaar@gmail.com","sentAt":"2014-09-14T10:38:36Z","receivedAt":"2014-09-14T10:38:36Z","isPatch":true,"sender":{"key":"cole.minnaar@gmail.com","avatar":null},"body":"Currently when specifying the `--depth` option to the 'submodule add'\ncommand, it can only create a shallow submodule clone of the currently\nactive branch from the cloned repository. If a branch is specified using\nthe `--branch` option, and the `--depth` option is also specified, the\n'submodule add' command will result in an error as the branch will not\nbe present in the cloned repository. If a repository is shallowly cloned\nas a submodule, there is no way to specify that the shallowly cloned\nsubmodule should setup remote-tracking branches.\n\nAdded the ability to shallowly clone any branch as a submodule, not just\nthe current active branch in the cloned repository.\nAdded support to the 'submodule add' and 'submodule update' commands to\nhandle `--no-single-branch` option, which is in turn passed to the clone\ncommand in order to setup remote-tracking branches in the shallowly\ncloned submodule.\n\nSigned-off-by: Cole Minnaar <cole.minnaar@gmail.com>\n---\n Documentation/git-submodule.txt |  8 ++++++--\n git-submodule.sh                | 24 ++++++++++++++++++++----\n t/t7400-submodule-basic.sh      | 33 ++++++++++++++++++++++++++++++++-\n 3 files changed, 58 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt\nindex 8e6af65..5b913ab 100644\n--- a/Documentation/git-submodule.txt\n+++ b/Documentation/git-submodule.txt\n@@ -10,13 +10,14 @@ SYNOPSIS\n --------\n [verse]\n 'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]\n-\t      [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]\n+\t      [--reference <repository>] [--depth <depth>] [--no-single-branch]\n+\t      [--] <repository> [<path>]\n 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]\n 'git submodule' [--quiet] init [--] [<path>...]\n 'git submodule' [--quiet] deinit [-f|--force] [--] <path>...\n 'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]\n \t      [-f|--force] [--rebase|--merge] [--reference <repository>]\n-\t      [--depth <depth>] [--recursive] [--] [<path>...]\n+\t      [--depth <depth>] [--recursive] [--no-single-branch] [--] [<path>...]\n 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]\n \t      [commit] [--] [<path>...]\n 'git submodule' [--quiet] foreach [--recursive] <command>\n@@ -354,6 +355,9 @@ for linkgit:git-clone[1]'s `--reference` and `--shared` options carefully.\n \tclone with a history truncated to the specified number of revisions.\n \tSee linkgit:git-clone[1]\n \n+--no-single-branch::\n+\tThis option is valid for add and update commands. Fetch histories near the tips\n+\tof all branches and create remote-tracking branches in the submodule.\n \n <path>...::\n \tPaths to submodule(s). When specified this will restrict the command\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 9245abf..9c0c858 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -5,11 +5,11 @@\n # Copyright (c) 2007 Lars Hjemli\n \n dashless=$(basename \"$0\" | sed -e 's/-/ /')\n-USAGE=\"[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]\n+USAGE=\"[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--depth <depth>] [--no-single-branch] [--] <repository> [<path>]\n    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]\n    or: $dashless [--quiet] init [--] [<path>...]\n    or: $dashless [--quiet] deinit [-f|--force] [--] <path>...\n-   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--reference <repository>] [--recursive] [--] [<path>...]\n+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--reference <repository>] [--recursive] [--depth <depth>] [--no-single-branch] [--] [<path>...]\n    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]\n    or: $dashless [--quiet] foreach [--recursive] <command>\n    or: $dashless [--quiet] sync [--recursive] [--] [<path>...]\"\n@@ -259,6 +259,13 @@ module_clone()\n \turl=$3\n \treference=\"$4\"\n \tdepth=\"$5\"\n+\tclone_branch=\n+\tif test -n \"$6\"\n+\tthen\n+\t\tclone_branch=\"--branch=$6\"\n+\tfi\n+\n+\tno_single_branch=\"$7\"\n \tquiet=\n \tif test -n \"$GIT_QUIET\"\n \tthen\n@@ -282,6 +289,7 @@ module_clone()\n \t\t(\n \t\t\tclear_local_git_env\n \t\t\tgit clone $quiet ${depth:+\"$depth\"} -n ${reference:+\"$reference\"} \\\n+\t\t\t\t${clone_branch:+\"$clone_branch\"} ${no_single_branch:+\"$no_single_branch\"} \\\n \t\t\t\t--separate-git-dir \"$gitdir\" \"$url\" \"$sm_path\"\n \t\t) ||\n \t\tdie \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$sm_path' failed\")\"\n@@ -328,6 +336,7 @@ cmd_add()\n {\n \t# parse $args after \"submodule ... add\".\n \treference_path=\n+\tno_single_branch=\n \twhile test $# -ne 0\n \tdo\n \t\tcase \"$1\" in\n@@ -363,6 +372,9 @@ cmd_add()\n \t\t--depth=*)\n \t\t\tdepth=$1\n \t\t\t;;\n+\t\t--no-single-branch)\n+\t\t\tno_single_branch=\"--no-single-branch\"\n+\t\t\t;;\n \t\t--)\n \t\t\tshift\n \t\t\tbreak\n@@ -472,7 +484,7 @@ Use -f if you really want to add it.\" >&2\n \t\t\t\techo \"$(eval_gettext \"Reactivating local git directory for submodule '\\$sm_name'.\")\"\n \t\t\tfi\n \t\tfi\n-\t\tmodule_clone \"$sm_path\" \"$sm_name\" \"$realrepo\" \"$reference\" \"$depth\" || exit\n+\t\tmodule_clone \"$sm_path\" \"$sm_name\" \"$realrepo\" \"$reference\" \"$depth\" \"$branch\" \"$no_single_branch\" || exit\n \t\t(\n \t\t\tclear_local_git_env\n \t\t\tcd \"$sm_path\" &&\n@@ -724,6 +736,7 @@ cmd_deinit()\n cmd_update()\n {\n \t# parse $args after \"submodule ... update\".\n+\tno_single_branch=\n \twhile test $# -ne 0\n \tdo\n \t\tcase \"$1\" in\n@@ -770,6 +783,9 @@ cmd_update()\n \t\t--depth=*)\n \t\t\tdepth=$1\n \t\t\t;;\n+\t\t--no-single-branch)\n+\t\t\tno_single_branch=\"--no-single-branch\"\n+\t\t\t;;\n \t\t--)\n \t\t\tshift\n \t\t\tbreak\n@@ -834,7 +850,7 @@ Maybe you want to use 'update --init'?\")\"\n \n \t\tif ! test -d \"$sm_path\"/.git && ! test -f \"$sm_path\"/.git\n \t\tthen\n-\t\t\tmodule_clone \"$sm_path\" \"$name\" \"$url\" \"$reference\" \"$depth\" || exit\n+\t\t\tmodule_clone \"$sm_path\" \"$name\" \"$url\" \"$reference\" \"$depth\" \"$branch\" \"$no_single_branch\" || exit\n \t\t\tcloned_modules=\"$cloned_modules;$name\"\n \t\t\tsubsha1=\n \t\telse\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 7c88245..81c05ae 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -136,7 +136,6 @@ test_expect_success 'submodule add --branch' '\n \techo \"refs/heads/initial\" >expect-head &&\n \tcat <<-\\EOF >expect-heads &&\n \trefs/heads/initial\n-\trefs/heads/master\n \tEOF\n \t>empty &&\n \n@@ -982,5 +981,37 @@ test_expect_success 'submodule add clone shallow submodule' '\n \t)\n '\n \n+test_expect_success 'submodule add --branch --depth' '\n+\t(\n+\t\tcd addtest2 &&\n+\t\tgit submodule add -b initial --depth 1 -- file://\"$submodurl\" submod-branch-depth &&\n+\t\ttest \"initial\" = \"$(git config -f .gitmodules submodule.submod-branch-depth.branch)\" &&\n+\t\t(\n+\t\t\tcd submod-branch-depth &&\n+\t\t\ttest 1 = $(git log --oneline | wc -l)\n+\t\t)\n+\t)\n+'\n+\n+cat >remote <<\\EOF\n+  origin/HEAD -> origin/second\n+  origin/initial\n+  origin/master\n+  origin/second\n+EOF\n+\n+test_expect_success 'submodule add --branch --depth --no-single-branch' '\n+\t(\n+\t\tcd addtest2 &&\n+\t\tgit submodule add -b initial --depth 1 --no-single-branch -- file://\"$submodurl\" submod-branch-depth-all &&\n+\t\ttest \"initial\" = \"$(git config -f .gitmodules submodule.submod-branch-depth-all.branch)\" &&\n+\t\t(\n+\t\t\tcd submod-branch-depth-all &&\n+\t\t\ttest 1 = $(git log --oneline | wc -l)\n+\t\t\tgit branch -r >../../remote-out\n+\t\t)\n+\t) &&\n+\ttest_cmp remote remote-out\n+'\n \n test_done\n-- \n2.1.0.238.gce1d3a9.dirty\n"},{"id":"249396","messageId":"0a7689bea41b2a4714380bb7b351e92f31a64e7b.1410691049.git.cole.minnaar@gmail.com","threadId":"37562","inReplyTo":"ede7b63c6028591281a7eefea5e9cd45cccd0a93.1410691049.git.cole.minnaar@gmail.com","subject":"[PATCH/RFC 2/2] submodule: modify clone command to recursively shallow clone submodules","fromName":"Cole Minnaar","fromEmail":"cole.minnaar@gmail.com","sentAt":"2014-09-14T10:38:37Z","receivedAt":"2014-09-14T10:38:37Z","isPatch":true,"sender":{"key":"cole.minnaar@gmail.com","avatar":null},"body":"When cloning a repository that contains submodules and specifying the\n`--depth` option to the 'git clone' command, the top level repository will be\ncloned with the specified depth, but all submodules within the\nrepository will be cloned in their entirety.\n\nModified 'git clone' to pass the `--depth` option, if specified, to any\nsubmodule clone commands.\nModified 'git clone' to pass the `--no-single-branch`, if specified, to any\nsubmodule clone commands.\n\nSigned-off-by: Cole Minnaar <cole.minnaar@gmail.com>\n---\n Documentation/git-clone.txt |  5 ++++-\n builtin/clone.c             | 15 +++++++++------\n 2 files changed, 13 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex 0363d00..f2fd8c8 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -178,7 +178,8 @@ objects from the source repository into a pack in the cloned repository.\n \n --depth <depth>::\n \tCreate a 'shallow' clone with a history truncated to the\n-\tspecified number of revisions.\n+\tspecified number of revisions. If `--recursive` was also specified,\n+\tthe depth value will be passed to all submodules within when cloning.\n \n --[no-]single-branch::\n \tClone only the history leading to the tip of a single branch,\n@@ -192,6 +193,8 @@ objects from the source repository into a pack in the cloned repository.\n \tinitial cloning.  If the HEAD at the remote did not point at any\n \tbranch when `--single-branch` clone was made, no remote-tracking\n \tbranch is created.\n+\tIf `--recursive` was also specified, this option will be passed\n+\tto all submodules when cloning.\n \n --recursive::\n --recurse-submodules::\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex dd4092b..c906d8e 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -48,6 +48,7 @@ static int option_verbosity;\n static int option_progress = -1;\n static struct string_list option_config;\n static struct string_list option_reference;\n+statis struct argv_array argv_submodule_cmd = ARGV_ARRAY_INIT;\n \n static int opt_parse_reference(const struct option *opt, const char *arg, int unset)\n {\n@@ -100,10 +101,6 @@ static struct option builtin_clone_options[] = {\n \tOPT_END()\n };\n \n-static const char *argv_submodule[] = {\n-\t\"submodule\", \"update\", \"--init\", \"--recursive\", NULL\n-};\n-\n static char *get_repo_path(const char *repo, int *is_bundle)\n {\n \tstatic char *suffix[] = { \"/.git\", \"\", \".git/.git\", \".git\" };\n@@ -663,8 +660,14 @@ static int checkout(void)\n \terr |= run_hook_le(NULL, \"post-checkout\", sha1_to_hex(null_sha1),\n \t\t\t   sha1_to_hex(sha1), \"1\", NULL);\n \n-\tif (!err && option_recursive)\n-\t\terr = run_command_v_opt(argv_submodule, RUN_GIT_CMD);\n+\tif (!err && option_recursive) {\n+\t\targv_array_pushl(&argv_submodule_cmd, \"submodule\", \"update\", \"--init\", \"--recursive\", NULL);\n+\t\tif (option_depth)\n+\t\t\targv_array_pushf(&argv_submodule_cmd, \"--depth=%d\", atoi(option_depth));\n+\t\tif (!option_single_branch)\n+\t\t\targv_array_pushl(&argv_submodule_cmd, \"--no-single-branch\", NULL);\n+\t\terr = run_command_v_opt(argv_submodule_cmd.argv, RUN_GIT_CMD);\n+\t}\n \n \treturn err;\n }\n-- \n2.1.0.238.gce1d3a9.dirty\n"},{"id":"249413","messageId":"CAPig+cSdJGzbwWGQTTO1JJpHSk9gfZcdCqFmG4DeSDewDbgbXQ@mail.gmail.com","threadId":"37562","inReplyTo":"ede7b63c6028591281a7eefea5e9cd45cccd0a93.1410691049.git.cole.minnaar@gmail.com","subject":"Re: [PATCH/RFC 1/2] submodule: add ability to shallowly clone any branch in a repo as a submodule","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2014-09-14T23:41:43Z","receivedAt":"2014-09-14T23:41:43Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sun, Sep 14, 2014 at 6:38 AM, Cole Minnaar <cole.minnaar@gmail.com> wrote:\n> Currently when specifying the `--depth` option to the 'submodule add'\n> command, it can only create a shallow submodule clone of the currently\n> active branch from the cloned repository. If a branch is specified using\n> the `--branch` option, and the `--depth` option is also specified, the\n> 'submodule add' command will result in an error as the branch will not\n> be present in the cloned repository. If a repository is shallowly cloned\n> as a submodule, there is no way to specify that the shallowly cloned\n> submodule should setup remote-tracking branches.\n>\n> Added the ability to shallowly clone any branch as a submodule, not just\n> the current active branch in the cloned repository.\n> Added support to the 'submodule add' and 'submodule update' commands to\n> handle `--no-single-branch` option, which is in turn passed to the clone\n> command in order to setup remote-tracking branches in the shallowly\n> cloned submodule.\n>\n> Signed-off-by: Cole Minnaar <cole.minnaar@gmail.com>\n> ---\n> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\n> index 7c88245..81c05ae 100755\n> --- a/t/t7400-submodule-basic.sh\n> +++ b/t/t7400-submodule-basic.sh\n> @@ -136,7 +136,6 @@ test_expect_success 'submodule add --branch' '\n>         echo \"refs/heads/initial\" >expect-head &&\n>         cat <<-\\EOF >expect-heads &&\n>         refs/heads/initial\n> -       refs/heads/master\n>         EOF\n>         >empty &&\n>\n> @@ -982,5 +981,37 @@ test_expect_success 'submodule add clone shallow submodule' '\n>         )\n>  '\n>\n> +test_expect_success 'submodule add --branch --depth' '\n> +       (\n> +               cd addtest2 &&\n> +               git submodule add -b initial --depth 1 -- file://\"$submodurl\" submod-branch-depth &&\n> +               test \"initial\" = \"$(git config -f .gitmodules submodule.submod-branch-depth.branch)\" &&\n> +               (\n> +                       cd submod-branch-depth &&\n> +                       test 1 = $(git log --oneline | wc -l)\n> +               )\n> +       )\n> +'\n> +\n> +cat >remote <<\\EOF\n> +  origin/HEAD -> origin/second\n> +  origin/initial\n> +  origin/master\n> +  origin/second\n> +EOF\n> +\n> +test_expect_success 'submodule add --branch --depth --no-single-branch' '\n> +       (\n> +               cd addtest2 &&\n> +               git submodule add -b initial --depth 1 --no-single-branch -- file://\"$submodurl\" submod-branch-depth-all &&\n> +               test \"initial\" = \"$(git config -f .gitmodules submodule.submod-branch-depth-all.branch)\" &&\n> +               (\n> +                       cd submod-branch-depth-all &&\n> +                       test 1 = $(git log --oneline | wc -l)\n\nBroken &&-chain.\n\n> +                       git branch -r >../../remote-out\n> +               )\n> +       ) &&\n> +       test_cmp remote remote-out\n> +'\n>\n>  test_done\n> --\n> 2.1.0.238.gce1d3a9.dirty\n"}]}