{"thread":{"id":"29243","subject":"[RFC PATCH] Allow cloning branches selectively","startedAt":"2011-12-23T20:13:19Z","lastAt":"2011-12-25T20:37:40Z","messageCount":7,"participants":["Carlos Martín Nieto","Junio C Hamano","Nguyen Thai Ngoc Duy","Jakub Narebski"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"181649","messageId":"1324671199-7074-1-git-send-email-cmn@elego.de","threadId":"29243","inReplyTo":null,"subject":"[RFC PATCH] Allow cloning branches selectively","fromName":"Carlos Martín Nieto","fromEmail":"cmn@elego.de","sentAt":"2011-12-23T20:13:19Z","receivedAt":"2011-12-23T20:13:19Z","isPatch":true,"sender":{"key":"cmn@elego.de","avatar":"https://avatars.githubusercontent.com/u/335443?v=4"},"body":"Sometimes it's useful to clone only a subset of branches from a remote\nwe're cloning. Teach clone the --fetch option to select which branches\nshould get fetched.\n\nEach --fetch sets up a fetch refspec for that branch. Previously this\nwas only possible by initializing a repo and manually setting up the\nconfig.\n---\n\nThis is still a WIP, as clone will still always fetch and checkout the\nremote's HEAD, which leaves you with a detached HEAD if you didn't\nwant that branch, which is clearly a bug.\n\nOtherwise it works as expected. A better name for this feature would\nalso be nice, as in git you don't clone branches but repos. Maybe\nsomething like \"selectively fetch branches on clone\"? If there isn't\nan outcry against this I'll add --fetch to the manpage as well.\n\n   cmn\n\n builtin/clone.c          |   82 ++++++++++++++++++++++++++++++++++------------\n t/t5702-clone-options.sh |   22 +++++++++++--\n 2 files changed, 80 insertions(+), 24 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 86db954..f14ca2a 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -48,6 +48,17 @@ static int option_verbosity;\n static int option_progress;\n static struct string_list option_config;\n static struct string_list option_reference;\n+static struct string_list option_fetch;\n+static const char **refspecs;\n+static int refspecs_nr;\n+static int refspecs_alloc;\n+\n+static void add_refspec(const char *ref)\n+{\n+\trefspecs_nr++;\n+\tALLOC_GROW(refspecs, refspecs_nr, refspecs_alloc);\n+\trefspecs[refspecs_nr-1] = ref;\n+}\n \n static int opt_parse_reference(const struct option *opt, const char *arg, int unset)\n {\n@@ -88,6 +99,8 @@ static struct option builtin_clone_options[] = {\n \t\t   \"use <name> instead of 'origin' to track upstream\"),\n \tOPT_STRING('b', \"branch\", &option_branch, \"branch\",\n \t\t   \"checkout <branch> instead of the remote's HEAD\"),\n+\tOPT_STRING_LIST(0, \"fetch\", &option_fetch, \"refspec\",\n+\t\t   \"fetch <refspec> instead of all the branches\"),\n \tOPT_STRING('u', \"upload-pack\", &option_upload_pack, \"path\",\n \t\t   \"path to git-upload-pack on the remote\"),\n \tOPT_STRING(0, \"depth\", &option_depth, \"depth\",\n@@ -421,13 +434,16 @@ static void remove_junk_on_signal(int signo)\n }\n \n static struct ref *wanted_peer_refs(const struct ref *refs,\n-\t\tstruct refspec *refspec)\n+\t\tstruct refspec *refspec, int refspec_nr)\n {\n \tstruct ref *head = copy_ref(find_ref_by_name(refs, \"HEAD\"));\n \tstruct ref *local_refs = head;\n \tstruct ref **tail = head ? &head->next : &local_refs;\n+\tint i;\n+\n+\tfor (i = 0; i < refspec_nr; i++)\n+\t\tget_fetch_map(refs, &refspec[i], &tail, 0);\n \n-\tget_fetch_map(refs, refspec, &tail, 0);\n \tif (!option_mirror)\n \t\tget_fetch_map(refs, tag_refspec, &tail, 0);\n \n@@ -482,7 +498,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tint err = 0;\n \n \tstruct refspec *refspec;\n-\tconst char *fetch_pattern;\n \n \tjunk_pid = getpid();\n \n@@ -508,6 +523,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \t\toption_no_checkout = 1;\n \t}\n \n+\tif (option_mirror && refspecs)\n+\t\tdie(_(\"--mirror and --fetch options are incompatible\"));\n+\n \tif (!option_origin)\n \t\toption_origin = \"origin\";\n \n@@ -594,30 +612,55 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tgit_config(git_default_config, NULL);\n \n \tif (option_bare) {\n-\t\tif (option_mirror)\n+\t\tgit_config_set(\"core.bare\", \"true\");\n+\t\tif (option_mirror) {\n \t\t\tsrc_ref_prefix = \"refs/\";\n-\t\tstrbuf_addstr(&branch_top, src_ref_prefix);\n+\t\t\tstrbuf_addf(&key, \"remote.%s.mirror\", option_origin);\n+\t\t\tgit_config_set(key.buf, \"true\");\n+\t\t\tstrbuf_reset(&key);\n+\t\t}\n \n-\t\tgit_config_set(\"core.bare\", \"true\");\n+\t\tstrbuf_addstr(&branch_top, src_ref_prefix);\n \t} else {\n \t\tstrbuf_addf(&branch_top, \"refs/remotes/%s/\", option_origin);\n \t}\n \n-\tstrbuf_addf(&value, \"+%s*:%s*\", src_ref_prefix, branch_top.buf);\n+\tstrbuf_reset(&key);\n+\tstrbuf_addf(&key, \"remote.%s.fetch\", option_origin);\n+\n+\t/* If the user didn't override it, use the default values */\n+\tif (option_fetch.nr == 0) {\n+\t\tstrbuf_reset(&value);\n+\t\tstrbuf_addf(&value, \"+%s*:%s*\", src_ref_prefix,\n+\t\t\t    branch_top.buf);\n \n-\tif (option_mirror || !option_bare) {\n-\t\t/* Configure the remote */\n-\t\tstrbuf_addf(&key, \"remote.%s.fetch\", option_origin);\n \t\tgit_config_set_multivar(key.buf, value.buf, \"^$\", 0);\n-\t\tstrbuf_reset(&key);\n+\t\tadd_refspec(strbuf_detach(&value, NULL));\n+\t} else {\n+\t\tint i;\n+\t\tfor (i = 0; i < option_fetch.nr; i++) {\n+\t\t\tconst char *ref = option_fetch.items[i].string;\n+\t\t\tif (!valid_fetch_refspec(ref))\n+\t\t\t\tdie(_(\"Not a valid fetch refspec: %s\"), ref);\n+\n+\t\t\t/* If we only got a branch name, make it a proper refspec */\n+\t\t\tif (!strchr(ref, ':')) {\n+\t\t\t\tstrbuf_reset(&value);\n+\t\t\t\tstrbuf_addf(&value, \"refs/heads/%s:%s%s\",\n+\t\t\t\t\t    ref, branch_top.buf, ref);\n+\t\t\t\tref = value.buf;\n+\t\t\t}\n \n-\t\tif (option_mirror) {\n-\t\t\tstrbuf_addf(&key, \"remote.%s.mirror\", option_origin);\n-\t\t\tgit_config_set(key.buf, \"true\");\n-\t\t\tstrbuf_reset(&key);\n+\t\t\tprintf(\"Adding refpsec %s\\n\", ref);\n+\t\t\tgit_config_set_multivar(key.buf, value.buf, \"^$\", 0);\n+\t\t\tadd_refspec(strbuf_detach(&value, NULL));\n \t\t}\n+\n+\t\tstrbuf_reset(&value);\n+\t\tstrbuf_reset(&key);\n \t}\n \n+\tstrbuf_reset(&key);\n \tstrbuf_addf(&key, \"remote.%s.url\", option_origin);\n \tgit_config_set(key.buf, repo);\n \tstrbuf_reset(&key);\n@@ -625,14 +668,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \tif (option_reference.nr)\n \t\tsetup_reference();\n \n-\tfetch_pattern = value.buf;\n-\trefspec = parse_fetch_refspec(1, &fetch_pattern);\n-\n-\tstrbuf_reset(&value);\n+\trefspec = parse_fetch_refspec(refspecs_nr, refspecs);\n \n \tif (is_local) {\n \t\trefs = clone_local(path, git_dir);\n-\t\tmapped_refs = wanted_peer_refs(refs, refspec);\n+\t\tmapped_refs = wanted_peer_refs(refs, refspec, refspecs_nr);\n \t} else {\n \t\tstruct remote *remote = remote_get(option_origin);\n \t\ttransport = transport_get(remote, remote->url[0]);\n@@ -654,7 +694,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \n \t\trefs = transport_get_remote_refs(transport);\n \t\tif (refs) {\n-\t\t\tmapped_refs = wanted_peer_refs(refs, refspec);\n+\t\t\tmapped_refs = wanted_peer_refs(refs, refspec, refspecs_nr);\n \t\t\ttransport_fetch_refs(transport, mapped_refs);\n \t\t}\n \t}\ndiff --git a/t/t5702-clone-options.sh b/t/t5702-clone-options.sh\nindex 02cb024..53d914f 100755\n--- a/t/t5702-clone-options.sh\n+++ b/t/t5702-clone-options.sh\n@@ -8,15 +8,17 @@ test_expect_success 'setup' '\n \tmkdir parent &&\n \t(cd parent && git init &&\n \t echo one >file && git add file &&\n-\t git commit -m one)\n+\t git commit -m one &&\n+\t git branch other)\n \n '\n \n test_expect_success 'clone -o' '\n \n \tgit clone -o foo parent clone-o &&\n-\t(cd clone-o && git rev-parse --verify refs/remotes/foo/master)\n-\n+\t(cd clone-o &&\n+\t git rev-parse --verify refs/remotes/foo/master &&\n+\t git rev-parse --verify refs/remotes/foo/other)\n '\n \n test_expect_success 'redirected clone' '\n@@ -33,4 +35,18 @@ test_expect_success 'redirected clone -v' '\n \n '\n \n+test_expect_success 'select one branch to fetch' '\n+\tgit clone --progress --fetch=master \"file://$(pwd)/parent\" clone-select-one &&\n+\t(cd clone-sel &&\n+\t git rev-parse --verify refs/remotes/origin/master &&\n+\t test_must_fail git rev-parse --verify refs/remotes/origin/other)\n+'\n+\n+test_expect_success 'select several branches to fetch' '\n+\tgit clone --progress --fetch=master --fetch=other \"file://$(pwd)/parent\" clone-select-many &&\n+\t(cd clone-sel2 &&\n+\t git rev-parse --verify refs/remotes/origin/master &&\n+\t git rev-parse --verify refs/remotes/origin/other)\n+'\n+\n test_done\n-- \n1.7.8.352.g876a6f\n"},{"id":"181652","messageId":"7vmxajaswj.fsf@alter.siamese.dyndns.org","threadId":"29243","inReplyTo":"1324671199-7074-1-git-send-email-cmn@elego.de","subject":"Re: [RFC PATCH] Allow cloning branches selectively","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-12-23T21:18:36Z","receivedAt":"2011-12-23T21:18:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Carlos Martín Nieto <cmn@elego.de> writes:\n\n> Sometimes it's useful to clone only a subset of branches from a remote\n> we're cloning. Teach clone the --fetch option to select which branches\n> should get fetched.\n\nThis is just a knee-jerk reaction without reading the patch text (which I\nwon't be doing over the holiday weekend anyway), but is the workflow of\nthe primarly intended audience to clone \"a subset of branches\" or \"a\nsingle branch\"?\n\nI have a slight suspicion that this started out as \"I often want to create\na clone to _track_ a single branch, but because I am mucking with the code\nrelated to cloning anyway, I might as well allow more than one to be\nfetched, even though I do not have any need for that, somebody might find\nit useful\". And that is why it is important to answer the first question.\n\nIf the primary motivation is for a single branch, I suspect supporting\nonly a single branch and advertising the feature as \"tracking only one\nbranch\" might make it much easier to understand to the end users.\n\nUpon \"git clone --track cn/single-clone $there x.git\", you would do\nsomething like:\n\n  it=cn/single-clone &&\n  git init x.git &&\n  cd x.git &&\n\n  # configure \"git fetch origin\" to only get branch $it\n  git config remote.origin.url \"$there\" &&\n  git config remote.origin.fetch refs/heads/$it:refs/remotes/origin/$it &&\n\n  # declare that the primary branch at origin is $it as far as we are concerned\n  git symbolic-ref -m clone refs/remotes/origin/HEAD refs/remotes/origin/$it &&\n\n  # Git aware prompt reminds us that this repository is to track branch $it\n  git symbolic-ref -m clone HEAD refs/heads/$it &&\n\n  # And Go!\n  git fetch origin &&\n  git reset --hard remotes/origin/$it &&\n  git config branch.$it.remote origin &&\n  git config branch.$it.merge $it\n\nOf course you _could_ support more than one pretty easily, but the point\nis that it is unclear how you explain to the end user what the feature\ndoes and what it is for in easily understoodd terms, once you start doing\nso. It will no longer be \"this new clone is to track that branch\", but\nsomething else, and I do not know what that something else is.\n\nAnd depending on what that \"something else\" is, which branch should be\nchecked out and what refs/remotes/origin/HEAD should name as the primary\nbranch of the remote would be different.\n\nThanks.\n"},{"id":"181659","messageId":"CACsJy8ANZvkY+na-tM95prHEfXD+N2OT8+3NLeccycGL1BmbCg@mail.gmail.com","threadId":"29243","inReplyTo":"1324671199-7074-1-git-send-email-cmn@elego.de","subject":"Re: [RFC PATCH] Allow cloning branches selectively","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2011-12-24T04:31:51Z","receivedAt":"2011-12-24T04:31:51Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Dec 24, 2011 at 3:13 AM, Carlos Martín Nieto <cmn@elego.de> wrote:\n> Sometimes it's useful to clone only a subset of branches from a remote\n> we're cloning. Teach clone the --fetch option to select which branches\n> should get fetched.\n\nWhat about tags? Are all tags fetched or only ones part of the\nselected branches?\n-- \nDuy\n"},{"id":"181689","messageId":"m3hb0ofwem.fsf@localhost.localdomain","threadId":"29243","inReplyTo":"1324671199-7074-1-git-send-email-cmn@elego.de","subject":"Re: [RFC PATCH] Allow cloning branches selectively","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2011-12-25T16:28:39Z","receivedAt":"2011-12-25T16:28:39Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Carlos Martín Nieto <cmn@elego.de> writes:\n\n> Sometimes it's useful to clone only a subset of branches from a remote\n> we're cloning. Teach clone the --fetch option to select which branches\n> should get fetched.\n> \n> Each --fetch sets up a fetch refspec for that branch. Previously this\n> was only possible by initializing a repo and manually setting up the\n> config.\n\nI haven't read the code, but I guess it should be possible to share\ncode with \"git remote add\", which allows to select which branches to\ntrack, and which branch is to be 'main' on remote.\n\n  git remote add [-t <branch>] [-m <master>] [-f] ...\n\n-- \nJakub Narebski\n"},{"id":"181691","messageId":"20111225174322.GB6361@beez.lab.cmartin.tk","threadId":"29243","inReplyTo":"m3hb0ofwem.fsf@localhost.localdomain","subject":"Re: [RFC PATCH] Allow cloning branches selectively","fromName":"Carlos Martín Nieto","fromEmail":"cmn@elego.de","sentAt":"2011-12-25T17:43:22Z","receivedAt":"2011-12-25T17:43:22Z","isPatch":true,"sender":{"key":"cmn@elego.de","avatar":"https://avatars.githubusercontent.com/u/335443?v=4"},"body":"On Sun, Dec 25, 2011 at 08:28:39AM -0800, Jakub Narebski wrote:\n> Carlos Martín Nieto <cmn@elego.de> writes:\n> \n> > Sometimes it's useful to clone only a subset of branches from a remote\n> > we're cloning. Teach clone the --fetch option to select which branches\n> > should get fetched.\n> > \n> > Each --fetch sets up a fetch refspec for that branch. Previously this\n> > was only possible by initializing a repo and manually setting up the\n> > config.\n> \n> I haven't read the code, but I guess it should be possible to share\n> code with \"git remote add\", which allows to select which branches to\n> track, and which branch is to be 'main' on remote.\n> \n>   git remote add [-t <branch>] [-m <master>] [-f] ...\n\nAh, very nice. I didn't know about those options. Hopefully I can use\nit. Thanks for pointing it out.\n\n   cmn\n"},{"id":"181694","messageId":"20111225190053.GC6361@beez.lab.cmartin.tk","threadId":"29243","inReplyTo":"CACsJy8ANZvkY+na-tM95prHEfXD+N2OT8+3NLeccycGL1BmbCg@mail.gmail.com","subject":"Re: [RFC PATCH] Allow cloning branches selectively","fromName":"Carlos Martín Nieto","fromEmail":"cmn@elego.de","sentAt":"2011-12-25T19:00:53Z","receivedAt":"2011-12-25T19:00:53Z","isPatch":true,"sender":{"key":"cmn@elego.de","avatar":"https://avatars.githubusercontent.com/u/335443?v=4"},"body":"On Sat, Dec 24, 2011 at 11:31:51AM +0700, Nguyen Thai Ngoc Duy wrote:\n> On Sat, Dec 24, 2011 at 3:13 AM, Carlos Martín Nieto <cmn@elego.de> wrote:\n> > Sometimes it's useful to clone only a subset of branches from a remote\n> > we're cloning. Teach clone the --fetch option to select which branches\n> > should get fetched.\n> \n> What about tags? Are all tags fetched or only ones part of the\n> selected branches?\n\nI haven't really touched that part of the code, so I think it'll fetch\nevery tag, as that's what clone does by default. Certainly something\nthat should get fixed.\n\n   cmn\n"},{"id":"181696","messageId":"20111225203740.GD6361@beez.lab.cmartin.tk","threadId":"29243","inReplyTo":"7vmxajaswj.fsf@alter.siamese.dyndns.org","subject":"Re: [RFC PATCH] Allow cloning branches selectively","fromName":"Carlos Martín Nieto","fromEmail":"cmn@elego.de","sentAt":"2011-12-25T20:37:40Z","receivedAt":"2011-12-25T20:37:40Z","isPatch":true,"sender":{"key":"cmn@elego.de","avatar":"https://avatars.githubusercontent.com/u/335443?v=4"},"body":"On Fri, Dec 23, 2011 at 01:18:36PM -0800, Junio C Hamano wrote:\n> Carlos Martín Nieto <cmn@elego.de> writes:\n> \n> > Sometimes it's useful to clone only a subset of branches from a remote\n> > we're cloning. Teach clone the --fetch option to select which branches\n> > should get fetched.\n> \n> This is just a knee-jerk reaction without reading the patch text (which I\n> won't be doing over the holiday weekend anyway), but is the workflow of\n> the primarly intended audience to clone \"a subset of branches\" or \"a\n> single branch\"?\n> \n> I have a slight suspicion that this started out as \"I often want to create\n> a clone to _track_ a single branch, but because I am mucking with the code\n> related to cloning anyway, I might as well allow more than one to be\n> fetched, even though I do not have any need for that, somebody might find\n> it useful\". And that is why it is important to answer the first question.\n\nThe main usefulness is indeed to track single branches. Limiting it to\none looked to me to be an arbitrary limitation and I don't like\nthose. Tracking between two and all branches is probably quite a niche\nhowever, so I'll go with allowing one -t/--track option and if enough\npeople want to extend it, we'll see what we do then.\n\n> \n> If the primary motivation is for a single branch, I suspect supporting\n> only a single branch and advertising the feature as \"tracking only one\n> branch\" might make it much easier to understand to the end users.\n> \n> Upon \"git clone --track cn/single-clone $there x.git\", you would do\n> something like:\n> \n>   it=cn/single-clone &&\n>   git init x.git &&\n>   cd x.git &&\n> \n>   # configure \"git fetch origin\" to only get branch $it\n>   git config remote.origin.url \"$there\" &&\n>   git config remote.origin.fetch refs/heads/$it:refs/remotes/origin/$it \n> \n>   # declare that the primary branch at origin is $it as far as we are concerned\n>   git symbolic-ref -m clone refs/remotes/origin/HEAD refs/remotes/origin/$it &&\n\nAs Jakub pointed out, remote already has this option, so you can\nreplace these calls with\n\n    git remote add origin $there -t $it -m $it\n\n> \n>   # Git aware prompt reminds us that this repository is to track branch $it\n>   git symbolic-ref -m clone HEAD refs/heads/$it &&\n> \n>   # And Go!\n>   git fetch origin &&\n>   git reset --hard remotes/origin/$it &&\n>   git config branch.$it.remote origin &&\n>   git config branch.$it.merge $it\n\nA lazier man (or one who doesn't work with internals every day) might do\n\n    git checkout -t origin/$it\n\nwhich the documentation claims would do the same thing. So it boils\ndown to doing three commands and it feels like it'd be much easier to\njust write a small script than to modify the clone code. Putting it in\nC will hopefully make the code a bit cleaner, if I use the code that\nalready exists in remote.\n\n> \n> Of course you _could_ support more than one pretty easily, but the point\n> is that it is unclear how you explain to the end user what the feature\n> does and what it is for in easily understoodd terms, once you start doing\n> so. It will no longer be \"this new clone is to track that branch\", but\n> something else, and I do not know what that something else is.\n\nLooking at the explanation for the -t (--track) option in git-remote,\nit's certainly not very friendly unless you already know exactly what\na -t option would do if you were to implement it.\n\n   cmn\n"}]}