{"thread":{"id":"42144","subject":"[PATCHv2] clone: add `--shallow-submodules` flag","startedAt":"2016-04-26T01:12:27Z","lastAt":"2016-04-26T17:51:18Z","messageCount":3,"participants":["Stefan Beller","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"284505","messageId":"1461633147-23131-1-git-send-email-sbeller@google.com","threadId":"42144","inReplyTo":null,"subject":"[PATCHv2] clone: add `--shallow-submodules` flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-26T01:12:27Z","receivedAt":"2016-04-26T01:12:27Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When creating a shallow clone of a repository with submodules, the depth\nargument does not influence the submodules, i.e. the submodules are done\nas non-shallow clones. It is unclear what the best default is for the\ndepth of submodules of a shallow clone, so we need to have the possibility\nto do all kinds of combinations:\n\n* shallow super project with shallow submodules\n  e.g. build bots starting always from scratch. They want to transmit\n  the least amount of network data as well as using the least amount\n  of space on their hard drive.\n* shallow super project with unshallow submodules\n  e.g. The superproject is just there to track a collection of repositories\n  and it is not important to have the relationship between the repositories\n  intact. However the history of the individual submodules matter.\n* unshallow super project with shallow submodules\n  e.g. The superproject is the actual project and the submodule is a\n  library which is rarely touched.\n\nThe new switch to select submodules to be shallow or unshallow supports\nall of these three cases.\n\nIt is easy to transition from the first to the second case by just\nunshallowing the submodules (`git submodule foreach git fetch\n--unshallow`), but it is not possible to transition from the second to the\nfirst case (as we would have already transmitted the non shallow over\nthe network). That is why we want to make the first case the default in\ncase of a shallow super project. This leads to the inconvenience in the\nsecond case with the shallow super project and unshallow submodules,\nas you need to pass `--no-shallow-submodules`.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\n This is origin/sb/clone-shallow-passthru, with $pwd instead of $p\n in the tests.\n \n I did not rebase this on 85705cfb (Merge branch 'ss/clone-depth-single-doc',\n 2016-01-20) or later, but worked on it with the base unchanged.\n\n Documentation/git-clone.txt | 13 +++++--\n builtin/clone.c             |  7 ++++\n t/t5614-clone-submodules.sh | 85 +++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 102 insertions(+), 3 deletions(-)\n create mode 100755 t/t5614-clone-submodules.sh\n\ndiff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt\nindex 6db7b6d..e1a21b7 100644\n--- a/Documentation/git-clone.txt\n+++ b/Documentation/git-clone.txt\n@@ -14,8 +14,8 @@ SYNOPSIS\n \t  [-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]\n \t  [--dissociate] [--separate-git-dir <git dir>]\n \t  [--depth <depth>] [--[no-]single-branch]\n-\t  [--recursive | --recurse-submodules] [--jobs <n>] [--] <repository>\n-\t  [<directory>]\n+\t  [--recursive | --recurse-submodules] [--[no-]shallow-submodules]\n+\t  [--jobs <n>] [--] <repository> [<directory>]\n \n DESCRIPTION\n -----------\n@@ -190,7 +190,11 @@ objects from the source repository into a pack in the cloned repository.\n \n --depth <depth>::\n \tCreate a 'shallow' clone with a history truncated to the\n-\tspecified number of revisions.\n+\tspecified number of revisions. Implies `--single-branch` unless\n+\t`--no-single-branch` is given to fetch the histories near the\n+\ttips of all branches. This implies `--shallow-submodules`. If\n+\tyou want to have a shallow superproject clone, but full submodules,\n+\talso pass `--no-shallow-submodules`.\n \n --[no-]single-branch::\n \tClone only the history leading to the tip of a single branch,\n@@ -214,6 +218,9 @@ objects from the source repository into a pack in the cloned repository.\n \trepository does not have a worktree/checkout (i.e. if any of\n \t`--no-checkout`/`-n`, `--bare`, or `--mirror` is given)\n \n+--[no-]shallow-submodules::\n+\tAll submodules which are cloned will be shallow with a depth of 1.\n+\n --separate-git-dir=<git dir>::\n \tInstead of placing the cloned repository where it is supposed\n \tto be, place the cloned repository at the specified directory,\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex b004fb4..ecdf308 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -40,6 +40,7 @@ static const char * const builtin_clone_usage[] = {\n \n static int option_no_checkout, option_bare, option_mirror, option_single_branch = -1;\n static int option_local = -1, option_no_hardlinks, option_shared, option_recursive;\n+static int option_shallow_submodules = -1;\n static char *option_template, *option_depth;\n static char *option_origin = NULL;\n static char *option_branch = NULL;\n@@ -91,6 +92,8 @@ static struct option builtin_clone_options[] = {\n \t\t    N_(\"create a shallow clone of that depth\")),\n \tOPT_BOOL(0, \"single-branch\", &option_single_branch,\n \t\t    N_(\"clone only one branch, HEAD or --branch\")),\n+\tOPT_BOOL(0, \"shallow-submodules\", &option_shallow_submodules,\n+\t\t    N_(\"any cloned submodules will be shallow\")),\n \tOPT_STRING(0, \"separate-git-dir\", &real_git_dir, N_(\"gitdir\"),\n \t\t   N_(\"separate git dir from working tree\")),\n \tOPT_STRING_LIST('c', \"config\", &option_config, N_(\"key=value\"),\n@@ -727,6 +730,10 @@ static int checkout(void)\n \t\tstruct argv_array args = ARGV_ARRAY_INIT;\n \t\targv_array_pushl(&args, \"submodule\", \"update\", \"--init\", \"--recursive\", NULL);\n \n+\t\tif (option_shallow_submodules == 1\n+\t\t    || (option_shallow_submodules == -1 && option_depth))\n+\t\t\targv_array_push(&args, \"--depth=1\");\n+\n \t\tif (max_jobs != -1)\n \t\t\targv_array_pushf(&args, \"--jobs=%d\", max_jobs);\n \ndiff --git a/t/t5614-clone-submodules.sh b/t/t5614-clone-submodules.sh\nnew file mode 100755\nindex 0000000..62044c5\n--- /dev/null\n+++ b/t/t5614-clone-submodules.sh\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+\n+test_description='Test shallow cloning of repos with submodules'\n+\n+. ./test-lib.sh\n+\n+pwd=$(pwd)\n+\n+test_expect_success 'setup' '\n+\tgit checkout -b master &&\n+\ttest_commit commit1 &&\n+\ttest_commit commit2 &&\n+\tmkdir sub &&\n+\t(\n+\t\tcd sub &&\n+\t\tgit init &&\n+\t\ttest_commit subcommit1 &&\n+\t\ttest_commit subcommit2 &&\n+\t\ttest_commit subcommit3\n+\t) &&\n+\tgit submodule add \"file://$pwd/sub\" sub &&\n+\tgit commit -m \"add submodule\"\n+'\n+\n+test_expect_success 'nonshallow clone implies nonshallow submodule' '\n+\ttest_when_finished \"rm -rf super_clone\" &&\n+\tgit clone --recurse-submodules \"file://$pwd/.\" super_clone &&\n+\t(\n+\t\tcd super_clone &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 3 lines\n+\t) &&\n+\t(\n+\t\tcd super_clone/sub &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 3 lines\n+\t)\n+'\n+\n+test_expect_success 'shallow clone implies shallow submodule' '\n+\ttest_when_finished \"rm -rf super_clone\" &&\n+\tgit clone --recurse-submodules --depth 2 \"file://$pwd/.\" super_clone &&\n+\t(\n+\t\tcd super_clone &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 2 lines\n+\t) &&\n+\t(\n+\t\tcd super_clone/sub &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 1 lines\n+\t)\n+'\n+\n+test_expect_success 'shallow clone with non shallow submodule' '\n+\ttest_when_finished \"rm -rf super_clone\" &&\n+\tgit clone --recurse-submodules --depth 2 --no-shallow-submodules \"file://$pwd/.\" super_clone &&\n+\t(\n+\t\tcd super_clone &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 2 lines\n+\t) &&\n+\t(\n+\t\tcd super_clone/sub &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 3 lines\n+\t)\n+'\n+\n+test_expect_success 'non shallow clone with shallow submodule' '\n+\ttest_when_finished \"rm -rf super_clone\" &&\n+\tgit clone --recurse-submodules --no-local --shallow-submodules \"file://$pwd/.\" super_clone &&\n+\t(\n+\t\tcd super_clone &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 3 lines\n+\t) &&\n+\t(\n+\t\tcd super_clone/sub &&\n+\t\tgit log --oneline >lines &&\n+\t\ttest_line_count = 1 lines\n+\t)\n+'\n+\n+test_done\n-- \n2.8.0.44.g26aa6d4\n"},{"id":"284550","messageId":"xmqqtwiogt4y.fsf@gitster.mtv.corp.google.com","threadId":"42144","inReplyTo":"1461633147-23131-1-git-send-email-sbeller@google.com","subject":"Re: [PATCHv2] clone: add `--shallow-submodules` flag","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-04-26T17:46:37Z","receivedAt":"2016-04-26T17:46:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller <sbeller@google.com> writes:\n\n>  I did not rebase this on 85705cfb (Merge branch 'ss/clone-depth-single-doc',\n>  2016-01-20) or later, but worked on it with the base unchanged.\n\nThanks, will replace.\n\n> diff --git a/t/t5614-clone-submodules.sh b/t/t5614-clone-submodules.sh\n> new file mode 100755\n> index 0000000..62044c5\n> --- /dev/null\n> +++ b/t/t5614-clone-submodules.sh\n> @@ -0,0 +1,85 @@\n> +#!/bin/sh\n> +\n> +test_description='Test shallow cloning of repos with submodules'\n> +\n> +. ./test-lib.sh\n> +\n> +pwd=$(pwd)\n> +\n> +test_expect_success 'setup' '\n> +\tgit checkout -b master &&\n> +\ttest_commit commit1 &&\n> +\ttest_commit commit2 &&\n> +\tmkdir sub &&\n> +\t(\n> +\t\tcd sub &&\n> +\t\tgit init &&\n> +\t\ttest_commit subcommit1 &&\n> +\t\ttest_commit subcommit2 &&\n> +\t\ttest_commit subcommit3\n> +\t) &&\n> +\tgit submodule add \"file://$pwd/sub\" sub &&\n> +\tgit commit -m \"add submodule\"\n> +'\n> +\n> +test_expect_success 'nonshallow clone implies nonshallow submodule' '\n> +\ttest_when_finished \"rm -rf super_clone\" &&\n> +\tgit clone --recurse-submodules \"file://$pwd/.\" super_clone &&\n\nAll of these \"$path/.\" made me wonder one thing.  I know these URLs\nthat ends with slash-dot ought to work, but shouldn't they work\nwithout them, too?  The \"consistency\" in this test that ends\nanything that would have ended with \"$pwd\" with \"$pwd/.\" somewhat\nbothered me.\n"},{"id":"284552","messageId":"CAGZ79kan6bBH236nBvTUdV2KiaB9hW6KOCz_rWqk513CGN30mg@mail.gmail.com","threadId":"42144","inReplyTo":"xmqqtwiogt4y.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCHv2] clone: add `--shallow-submodules` flag","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-04-26T17:51:18Z","receivedAt":"2016-04-26T17:51:18Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Apr 26, 2016 at 10:46 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Stefan Beller <sbeller@google.com> writes:\n>\n>>  I did not rebase this on 85705cfb (Merge branch 'ss/clone-depth-single-doc',\n>>  2016-01-20) or later, but worked on it with the base unchanged.\n>\n> Thanks, will replace.\n>\n>> diff --git a/t/t5614-clone-submodules.sh b/t/t5614-clone-submodules.sh\n>> new file mode 100755\n>> index 0000000..62044c5\n>> --- /dev/null\n>> +++ b/t/t5614-clone-submodules.sh\n>> @@ -0,0 +1,85 @@\n>> +#!/bin/sh\n>> +\n>> +test_description='Test shallow cloning of repos with submodules'\n>> +\n>> +. ./test-lib.sh\n>> +\n>> +pwd=$(pwd)\n>> +\n>> +test_expect_success 'setup' '\n>> +     git checkout -b master &&\n>> +     test_commit commit1 &&\n>> +     test_commit commit2 &&\n>> +     mkdir sub &&\n>> +     (\n>> +             cd sub &&\n>> +             git init &&\n>> +             test_commit subcommit1 &&\n>> +             test_commit subcommit2 &&\n>> +             test_commit subcommit3\n>> +     ) &&\n>> +     git submodule add \"file://$pwd/sub\" sub &&\n>> +     git commit -m \"add submodule\"\n>> +'\n>> +\n>> +test_expect_success 'nonshallow clone implies nonshallow submodule' '\n>> +     test_when_finished \"rm -rf super_clone\" &&\n>> +     git clone --recurse-submodules \"file://$pwd/.\" super_clone &&\n>\n> All of these \"$path/.\" made me wonder one thing.  I know these URLs\n> that ends with slash-dot ought to work, but shouldn't they work\n> without them, too?  The \"consistency\" in this test that ends\n> anything that would have ended with \"$pwd\" with \"$pwd/.\" somewhat\n> bothered me.\n>\n\nAnother case of me not thinking it through. There used to be just '.'\nin the former\nseries with the --no-local option. And to make it a file url I just\nprefixed that dot\nwithout thinking if the dot is still needed.\n\nIn case another reroll is needed, I'll fix that up, too.\n\nThanks,\nStefan\n"}]}